where have all the programmers gone?

by Carla Schroder

In my inestimably valuable opinion, we need programmers (and related disciplines like QA, advanced math, and design) more than almost anything else. We're in the very infancy of the computer age, and already we are intensively computerized, from toasters and refrigerators to inventory tags to vehicles to big ole factories.

16 Comments

Adam Turoff
2006-06-22 09:57:55
Carla,


I understand your concern and your point of view, but your conclusion is absolutely wrong.


Let us say that over 30 years, the number of cars on the road is going to increase from 100 million to 200 million. If every car needs a major repair about once per year, then over 10 years, the number of auto mechanics needs to double. Where are they going to come from, if we aren't churning out enough auto mechanics domestically to support current demand, let alone 2x demand?


The question is specious, because it focuses on the symptom, not the problem. The number of cars on the road has increased significantly in the last 30 years. I don't have any hard numbers off hand, but there may be nearly 2x as many cars in use today than there were 30 years ago, and probably fewer auto mechanics. Yet there is no crisis. Why? Because the 'one major auto repair per car per year' was a symptom, not the root problem. The root problem was poor quality in automotive engineering, which is where the auto industry spent a significant amount of time and effort, and why the relative dearth of auto mechanics isn't a call to arms.


Software is in the same state. 30 years ago, we didn't really know what problems needed to be solved. Today, we're much better at it. Some of the high paying jobs 10 years ago in software simply don't exist anymore because we have better systems; what used to require a room full of programmers and designers and man years of effort can now be accomplished by a lone programmer in a basement over a weekend. Don't believe me? Compare how much time and effort it took to build a PR website from scratch in 1996 compared to how long it takes to install MovableType/WordPress and tweak a few templates today.


Don't get me wrong -- we will always need talented and skilled people. But we need fewer of them to handle the mundane work of the world.

Ainsley
2006-06-22 12:12:24
Adam, I don't quite see a valid comparison here. Sort of, yes, when you talk about increased reliability decreasing the need for mechanics. Someday we'll see a similar trend in computing, but we're not there yet. Unlike the tech industry, the automotive industry is fairly static. Advances are small and incremental. The tech industry, in contrast, is changing and expanding in many different directions.


I think salaries are artificially depressed. The skill levels required have not decreased, they're higher than ever. At least to turn out good code- a lot of bad code gets shipped because of penny-pinching management, who never want to hire enough help, and who hire the cheapest they can get.

Rod
2006-06-23 21:38:41
I'd agree with Ainsley's last paragraph. I have been programming for thirty-five years - since high-school, when I taught myself BASIC. I've programmed on teletypes, paper tape, punched cards, CRT terminals, In-Circuit Emulators, single-board computers, PCs, Macs, and some machines I've designed and wire-wrapped myself. I've designed microcontrollers, computers, graphics controllers, audio cards, touch sensor controllers, and other devices. I've written in assembly for 650x, 680x0, x86, as well as BASIC, C, C++, and FORTH. I've written my own assemblers, disassemblers, and operating systems and created products that received patents. My name is on documents that are (were?) international standards that I helped write.
But I find myself unemployed because my skillset can be "replicated" overseas for a fraction of what I would draw in the U.S. Employers don't want or can't afford to hire quality people (not just programmers) because they increase the cost of the end product, which means that it costs a few cents more at Wal-Mart. If one employer increases his cost to hire quality, he loses out in sales - virtually no one in the U.S. wants to pay for a better product.
One of my mottos of life is "You don't always get what you pay for, but you always pay for what you get." There's a huge difference between a "cheap" product and an "inexpensive" one. Too many "cheap" manufacturers cut corners (including hiring "cheap" engineers) and end up with defective, short-lived, or dangerous products. But virtually no one seems to care.
Too bad. There's some very good workers who would like to get to work, but they're in competition with poor (and I mean that in several ways) workers who are cheaper. In the meantime, some of us are losing our skillsets from lack of use, or changing careers. Working in a cardshop isn't high-tech or high-pay, but it's paying the bills and I don't have to fight 2-hour traffic.


As for the comment about increased reliability of computing, you forget that true cutting edge is always fraught with reliability problems. As it becomes easier to do (whether it's processor speed or cutting-edge software) we pile up more "bleeding-edge" onto it. As an example, computer graphics that would have taken weeks to render years ago are now done on the fly, and the resolution and quality are considered "commonplace" or "mediocre".
But it takes highly skilled, talented, articulate, and visionary people to tackle these problems and to reduce them down to managable items. Again, we're just not willing to pay for that, either in our education system or in our workplace.

Rick
2006-06-24 01:31:13
I'm with Rod on this one. I've been programming for 35 years as well, have my name on a number of recent highly regarded patents and have helped design and implement more than one award winning product. I'm also now unemployed. I left the company that I used to work for after they slashed my pension (illegally, it turns out) and transferred the money to their executive pension fund. They also rewrote their retiree medical program to insure that I would never qualify for it. It was a dramatic demonstration of their priorities. The HR department in this same company now likes to brag to their stockholders that they turn over their programming staff quickly ("less than five years") to keep costs down.


It turns out that they're not alone in their view of "senior" technical staff. That's why they're such strong growth in outsourcing to India and other locations. It's not that there aren't qualified people in this country available to do the work. That's industry propaganda. It's that the industry isn't willing to pay the salary and benefits that US workers expect and that they need to recoup the cost of the university education that you scorn.


Yes - this could be an interesting new 'career' for you. Just be sure you go into it with the expectation that this one won't last long either.

geekbeard
2006-06-26 19:29:52
The negative comments here are interesting and mostly true, and they all have one thing in common- as long as you're at the mercy of an employer, your options are limited. Don't shackle yourself to someone whose only interest is using you to further their own interests. Freelancing is the way out. It requires a different skill set, and not everyone can handle the "insecurity" of freelancing. But the job market isn't secure at all. You're much better off being your own boss.
naum
2006-06-26 23:04:24
where have all the programmers gone?


off to pursue alternative careers, after being kicked around enough. or answered another calling, opting for stints like freelance web development work or tackling the risky venture of company startup.


sure, there are those that answer their calling, no matter the state of career prospects in a chosen industry (i.e., see teaching...). but our best and brightest are now deterred away from careers in programming as they witness the fate of their elders -- unstable employment and diminished earning prospects.


yes, there has been a recent uptick in job advertisements, but outside of specialty realms where a niche expertise (say, in a particular vendor software package or mission critical software/process control) can pay huge dividents, rates and salaries are lower now than I was earning 12+ years ago.


the race to the bottom by "american" companies for lower cost foreign programmers has had a crippling effect upon our future ranks. it is no wonder why computer science and engineering enrollments are declining. meanwhile, for aspiring foreign programmers, it is their ticket to another realm. savvy american students reset their career preference, instead, to fields in sales and marketing, where rewards are greater, and reflect the precarious state of position permanence.


i've been programming computing machines since the 1980s and have written code for just about everything from burroughs and ibm mainframes, to unix and linux, and now server side web development platforms. i've witnessed firsthand corporate america discard home grown talent in lieu of cheaper foreign programmers, both overseas and imported via H1B or other NIV program.

Dave
2006-06-27 20:07:02
I'm another "old-timer" (programming since '68) that agrees with Ainsley, Rod, Rick, and Naum. I can meaningfully add only a small amount more.

Adam, you're wrong; I've been hearing "in the future, computers will be programming themselves and there will be no more programmers" since the early '70s. Programmers are just working on different things: what was once pushing the envelope is now mundane -- but those people are now working on other new things. And not all programming is sexy: some folks still need to know Cobol. (But that's outsourced!). As far as "30 years ago, we didn't really know what problems needed to be solved", let me just say that the people back then were just as clueful, hip, bright-eyed, and intelligent as now. (Nah, we were smarter. Heh heh :-))

Geekbeard, I'm doing freelance; been doing it since 1989 when I was laid off because I was stupid enough to tell the company "the sky is falling" -- and being right.

(Begin "rant"). I don't recommend freelancing, it's very much harder than being employed. You're always in "sales mode", trying to keep busy. It's hard trying to convince people you're a unique individual worth a bit more -- people want to pay only commodity pricing, no matter what part of the market you're in. It's like looking for a new job every week (because that's what you're doing!). You skimp on training and insurance because it takes time and money, neither of which you can afford. You don't bother to put comments on websites because you can't take the time to edit your reply so you sound like you have a moderately positive IQ.

If you freelance and can find long-term engagements, that's wonderful! But remember every other laid-off tech guy is eyeing the same golden opportunities you are.

The market has changed -- work you can get is hands-on stuff that can't be done overseas.
So... why not get a job? Try being over 45 years old and looking for a tech job! They look for the date when you got your degree, and it's amazing how much silence there is. And this is before you talk about money. (End "rant").

One final observation is that old-timers will remember the last time there was a craze to outsource. Datamation et al were full of the wonders of outsourcing. A couple years later, I distinctly remember the first article I saw from some CFO/CIO who re-hired local people and discovered to his amazement that it was cheaper than outsourcing because he had far less project management overhead and team communication issues than when outsourced. It was a front page article because the concept of re-hiring people back was so new.

Simon Hibbs
2006-06-28 02:28:02
Offshoring is the big bogey-man of the IT sector these days, but I just don't see it. Yes some jobs are being offshored, but compared to to the main driving factors behind the ebb and flow of IT jobs it's a minor trend. The fact is that the IT industry was massively over-manned as a result of the Dotcom and Y2K bubble. A reduction in headcounts was inevitable and offshoring has only accounted for a fraction of this. Yes times are tough for some, but those darned furriners aren't the main culprits. In fact the ability of some firms to cut costs by outsourcing has saved the companies, and therefore the jobs of everyone else employed by them, reducing the impact of the downturn.


As it happens I'll be out of a job at the end of the year, our department is being moved abroad - not to India but to Sweden of all places. Still, I'll be getting a nice payoff (good old UK employment law!). Time to brush up on my marketable skills and get stuck into the jobs market!

swampyankee
2006-06-28 06:31:45
I think there are several issues here:


One is that many professions require a certain level of programming skill. CFD, structural analysis, numerical relativity, and n-body programs, are largely written by engineers and physicists, not programmers. These people don't count as "programmers."


The second is that programming, as a college course of study, is more difficult than, say, business. Why work twice as hard as a business major when a technical degree gets you a marginally greater starting salary, but poorer perqs (no free golf weekends in Aruba for the programming staff..), lower long-term earnings, and much greater propensity for companies to shed you faster than a mangy dog sheds fur?


The third is that US companies seem to view money spent on managers--especially senior managers--as magically different than money spent on programmers or engineers. Money spent on PHB's is an investment; money spent on the people who actually design or make the product is an expense.

Caitlyn Martin
2006-06-28 11:19:00
I started out as a programmer professionally in 1980, and moved into networking where there were better opportunities in a burgeoning field around 1986. Since then I've been a systems administrator, a consultant, and a security analyst. I've mostly stayed employed throughout my career and maintained and even increased my income by reinventing myself and keeping my skills sharp. It's hard work but it can be done.


Carla, I have nieces and a nephew in the public education system. The education they are getting in suburban Atlanta is as fine as what was available to me in the 1960s and '70s and they are all doing very well and are years ahead of grade level and taking advanced classes. In well funded school districts I do not believe the educational system has deteriorated and in poor inner city and rural districts it was never good to start with. The system of funding education in the United States is and always has been woefully uneven. In a nation where socialism is a dirty word and most Americans don't even know what it means and equate it with the old Soviet system the ideal of equality of opportunity is unlikely to ever be realized. Bottom line: if you are middle class, upper middle class, or wealthy your children will likely receive an excellent education, particularly if you, as a parent, are engaged and aware of what is going on in the school system and what your opportunities are. If you are poor, well... in some cities there may be opportunities to take advantage but the odds are stacked heavily against poor children regardless of their ability.


Your characterization of American universities is equally flawed. Overall there are still resources for those who have both talent and money to receive an excellent education. The system is far from the laughably poor, football-oriented picture you paint, which I find wildly inaccurate. Rather it is woefully uneven. What you described exists, as does excellence, as does everything in between. The university I went to is still Division 3 because they refuse to spend money on sports scholarships and athletics is looked upon as a joke there. The emphasis is strictly academic.


Dave is exactly right when he talks about agism in high tech. My clients' managers are all way younger than I am. They don't hire older people for the most part except as consultants. They don't want to pay much so young kids fresh out of school are what they bring in. They also fear that an older, wiser head may know more than they do or else will want to do things their own way and won't take direction. It's sad. Some of the most talented people out there are being discarded.


Simon, you get it half right. The dot com bubble and Y2K did cause some necessary shedding of IT jobs, but many companies not only cut fat, but also cut meat and bone, at least in the U.S. where quarterly profits are king. This short term mentality has lead to unemployed or underemployed IT people and deteriorating corporate technology infrastructure. What remains are poorly administered systems, overworked and underappreciated IT staff who take the blame when they don't work miracles, and general dissatisfaction all the way around. I've walked into places where house was cleaned after a major security incident that only happened because the company didn't take information security seriously and ignored their now fired IT people's warnings. The new people are left with undocumented systems and code and have a ghastly time fixing failures. The previous regime gets the blame but when you read their reports you realize that they told management what was needed and were denied the resources to resolve the issues. So... someone like me gets called in short term to help set things right and stays until funding gets cut with the job half done, setting up the new regime for failure again. I am invariably asked to look for cost savings as well. It's pretty pitiful and I've seen it over and over. At least as the expert consultant I get paid well, albeit for a limited time.


Simon, you're in the U.K. In the U.S. outsourcing isn't a bogeyman. It's real. The three countries that benefit most are India, Ireland, and Israel. India gets the help desk, accounting, and low level development work. Ireland and Israel split the R&D, engineering, and high level development work. Hardware manufacturing is what goes to low cost Asian markets.


Most of my family lives in Israel. High tech is booming there. Guess where I'm moving to next year? Israel, as a nation, places very high value on education and technical skills. The U.S., sadly, does not and the American political and religious right wing are downright anti-intellectual. That, I think, is what Carla is picking up on and reacting to.

Adam Turoff
2006-06-28 13:01:47
Ainsley, I think you're missing the point. That analogy is meant to be just that -- an analogy. Computing is getting simpler, because yesterday's cutting edge is today's standard operating procedure. Databases used to be pie in the sky theory. Now, they're well understood and used everywhere. The computing industry is dynamic, but some parts are much more dynamic than others; many parts are quite static. Your points about salary are important, but don't change the fact that there is little value in hiring someone to fix yesterday's (or last decade's) problems, and a lot of value in finding someone to do cutting edge work; unfortunately, very little work today is cutting edge.


Dave, I never mentioned the "computers will program themselves" fallacy. Also, in calling me wrong, you've only further proven my point: "what was once pushing the envelope is now mundane". Sure, many people follow the bleeding edge, but I've worked at many a company that need people to work on mundane systems that are both critical to a line of business and highly profitable. There's no reason to pay increasing salaries iff nearly anyone can get the work done (and you don't have/need/care about institutional memory). So it's natural for that kind of work to be commoditized, or the staff will find ways to get more done with less labor (slowly moving closer to the bleeding edge), keeping themselves relevant while decreasing the staff size (but usually the other way around).


Caitlyn, that was a very good summary of a bunch of points that mostly never get mentioned. Rick makes a few excellent points himself about the high cost of labor in the US. Another point that often gets lost in the noise is that the US does not have a lock on the best programming talent on the planet. Outsourcing often does mean sending lots of jobs to India (a lot of "commodity" work, but also an increasing amount of high paying cutting edge stuff), but also sending jobs to Ireland, Israel, Australia and Canada. These other nations have excellent technical training, low labor costs, and virtually no language barrier (at least compared to, say, China or Russia). Even if someone was to waive a magic wand and quintuple the number of Comp Sci grads in the US, these deeper problems wouldn't magically go away.

Simon Hibbs
2006-06-29 03:38:58
Caitlyn, my current company (well, current but two takeovers ago) dabbled with outsourced development 6 years ago and screwed it up, so now all development is done here in London and in Canada. It's an engineering application, so lots of domain knowledge required.


I don't see outsourcing to places like Israel and Ireland as being in the same league. Many multinationals have development centres in Europe and America. We're competing on an equal footing,a nd there's a constant flow of personnel back and forth as well, which compensates for the flow of jobs. Having been on business trips to Israel, I'd live there too. What worries people about India and China is that they compete based on 'unfairly' low wages, which threatens the whole cost base and viability of the Euro-American IT economies. Even the most pessimistic take on outsourcing to Israel is that you just need to get cleverer, but the most pessimistic take on outsourcing to India is that it's just flat out impossible for us to compete at all. I don't buy it, but that's how I understand the debate.

Caitlyn Martin
2006-07-01 15:13:16
Simon, perhaps because you are in the U.K. and your perspective is different you see things differently. Whatever the reason you have completely missed my point.


The long term unemployed senior programmers who posted above don't care if their jobs were lost to Israel or India, Ireland or Malaysia. They're gone and they're not coming back. There is no one who will hire them for what they used to make even if their skills are up to date. If they are over 40 years old it makes things many times harder due to agism in the U.S. IT market. Those are facts of life for many talented and experienced U.S. IT professionals, and not just programmers. Outsourcing is a very real issue with life changing consequences for them.


Can they reinvent themselves and start again? Over age 40 it's tough. Can they hang their shingle and latch on as consultants or hired guns? Sure, but it's not easy and not everybody has the connections or marketing skills to do it.


My own career changes have worked out well. Some of that is talent and some is luck. I didn't chose my direction. As a 46 year old woman in IT my choices are much more limited that they once were.

Simon Hibbs
2006-07-03 03:52:16
>The long term unemployed senior programmers who posted above
>don't care if their jobs were lost to Israel or India, Ireland
>or Malaysia. They're gone and they're not coming back.


It depends how you read the numbers. Sure the US department of labour reckons that programming jobs are showing weak growth (they cite outsourcing as one of many factors) and high competition, but they reckon that "Computer software engineers are projected to be one of the fastest-growing occupations from 2004 to 2014."


Software Engineering is further up the food chain that programming and in fact they categorise many of the people we call programmers as Software Engineers. In 2004 there were 455,000 programming jobs, but 800,000 software engineering jobs and they earn more too.


This gives a picture of overall growth in jobs with programming skills, with an overall trend for more jobs further up the value ladder.

naum
2006-07-05 14:27:34

Offshoring is the big bogey-man of the IT sector these days, but I just don't see it. Yes some jobs are being offshored, but compared to to the main driving factors behind the ebb and flow of IT jobs it's a minor trend. The fact is that the IT industry was massively over-manned as a result of the Dotcom and Y2K bubble. A reduction in headcounts was inevitable and offshoring has only accounted for a fraction of this. Yes times are tough for some, but those darned furriners aren't the main culprits. In fact the ability of some firms to cut costs by outsourcing has saved the companies, and therefore the jobs of everyone else employed by them, reducing the impact of the downturn.


Within 10 minutes driving distance of my home (I reside in Glendale, AZ, USA), I can easily tally thousands of jobs at IT centers staffed by a combination of imported NIV programmers/programmers at offshore locales mainly in India that used to be stocked by Americans.


Used to be, that companies, in times of "labor shortages", would actively seek within and offer an arduous program for those in other departments of the business to retrain for an IT career, leveraging their in-depth business knowledge with a desire for a better career path and to capitalize on their aptitude for computing discipline, if present. Now, "American" multinational corporations simply have entered into a "race to the bottom".


Is it a universal truth? No, absolutely not as there are are logistical positions to support such schemes and numerous cracks where specialized knowledge will still command a hefty price.


But what's being missed here, is the wholesale degradation of programming as a career field to pursue. Even before the recent bouts of offshoring and outsourcing, it's a career that's been historically marked with age discrimination and burnout.

seth
2006-07-12 05:42:54
You can have my job. I can make twice as much in sales for half the effort. The number of ways to make significant income on the internet via ebay and direct sales globally has made it possible for humble little programmers like myself to get a real life minus the hassle of corporate nonsense.


The corporations are reaping their karma and I choose not to be part of it.