In addition to all the other reasons mentioned here, another idea: managers don't know how to use programmer talent very efficiently. Of all the things programmers are told to do, 90+% of them are useless or worse than useless, in that they simply make long-term maintenance worse without adding value.
If you cannot use a resource efficiently at all, wasting most of it, then it is not surprising that you end up with a shortage. When faced with this situation, management can:
1) admit that they are the problem, and get much better at allocating programmer effort wisely, a task that will be very difficult even once it's well understood (which it currently isn't). Or...
2) hire more programmers, paying whatever is required to get more
Very true, all the managers and their managers.. and theirs, who's technical skills have since atrophied, live in these hierarchies and its a great place for good and bad programmers (mostly bad programmers), to hide, get paid, be lazy, nice pension and not talk to anyone much, project gets binned, move onto another one. The large non-tech companies usually have this issue.
True that, but even managers who were good programmers sometimes don't have a great ability to figure out which features, either user-facing or architectural, are a good idea and which are not really going to impact the bottom line. I saw it at tech companies as well. In at least some cases it was based on a need to impress VC's more than to get things done.
If you cannot use a resource efficiently at all, wasting most of it, then it is not surprising that you end up with a shortage. When faced with this situation, management can: 1) admit that they are the problem, and get much better at allocating programmer effort wisely, a task that will be very difficult even once it's well understood (which it currently isn't). Or... 2) hire more programmers, paying whatever is required to get more