What does programmer productivity look like?

I am unable to judge the details of its contents, but this article intrigued me.  The key question is why pay across highly-talented and lesser-talented programmers isn't more unequal.  (That's a question I'd like economists to study more generally, given the disparities in productivity across individuals within a firm.)  Here is an excerpt:

Software output cannot be measured as easily as dollars or bricks. The best programmers do not write 10x as many lines of code and they certainly do not work 10x longer hours.

Programmers are most effective when they avoid writing code. They may realize the problem they’re being asked to solve doesn’t need to be solved, that the client doesn’t actually want what they’re asking for. They may know where to find reusable or re-editable code that solves their problem. They may cheat. But just when they are being their most productive, nobody says “Wow! You were just 100x more productive than if you’d done this the hard way. You deserve a raise.” At best they say “Good idea!” and go on.  It may take a while to realize that someone routinely comes up with such time-saving insights. Or to put it negatively, it may take a long time to realize that others are programming with sound and fury but producing nothing.

For the pointer I thank Hamilton Ulmer.

Comments

Right.

I believe that this is the case across the board, from gardeners to economists to attorneys.

One would think that the better programmer paid by the hour or even with a salary, who could do an 8 hour problem in 1 hour would likely spend 1 hour on the problem and 7 hours surfing the internet (or something else) as, if he completed the work in 1 hour, he would just get more work, but not more pay.

Things are actually worse than he implies. There are substantial numbers of working programmers with negative productivity, destroying value for their companies every day. They create designs that must be expensively scrapped or rewritten, engender defects that suck up customer support dollars, or in the worst case create products that are essentially unusable and unsaleable. A lot of programmers, even some with over a decade experience, have never successfully shipped code into production. That means that their entire compensation for their careers has been purely wasted dollars. Substantial amounts are spent in software recruiting simply to avoid hiring negative-productivity developers, but it's an uphill battle, and even the best companies slip up at it. Once hired, it can take months to figure out whether a programmer is net-negative productivity, convince management of it, and transition them to a position where they can exhibit positive productivity (or, failing that, negative productivity on somebody else's dime). With information this imperfect and expensive to acquire, it's hardly surprising that compensation doesn't match productivity more closely.

the discrepancy is pretty wide, but there are some tasks that don't require the best programmers.

as in most situations the people who contribute the most to productivity are the ones who can figure out how to avoid unnecessary work and allocate and explain work to others. this requires a lot of knowledge of detail in at least three domains (business domain, technology domain, and the 'soft skills' domain) and it's hard to find people who can do all three.

It reminds me of all those critical jobs in firms where the person's occupation is to say "no" to lots of people who want to muck up a clean operation. Google has such a person which is why google.com continues to look so clean. Or the standard in the gaming industry to avoid "feature creep;" games are best when they do one thing very, very well. It's a hard job to appreciate because it's built around stopping production instead of encouraging it (or in some cases, taking stuff out). But it's so important.

What Kingm said, and others following.

My consulting specialty is strategic IT planning in the public sector. This scale of productivity differential is especially difficult to reconcile with civil service pay scales and work rules.

My experience is that it's pretty obvious after a few months who's mediocre and who's the superstar. The problem is communicating this to management.

There are essentially three problems that make it hard to set developer compensation based on their contribution:

1. Its extremely hard to measure the quality of development decisions. Properly understood, the job of developers is to understand and solve problems, and the understanding part in fact takes most of the time. So by definition team leaders cannot follow in detail all the decisions their developers are making, let alone communicate that information to the managers who actually set compensation. Its therefore impossible to assess, except in a very vague and imprecise way, how good those decisions are.

2. Its at best difficult to determine the contribution of particular developers. This will make some of laugh, but software development is an intensely social process. Much more so than almost any other white collar job. Its closest perhaps to restaurant cooking, or team sports. It requires constant, clear, intensive communication and precise coordination between developers and between the development team and their "customers" to make it work. Its extremely hard to even remember who had a particular idea once its become part of the accepted lore within the team, and even then credit may be unclear - was it the guy who had the idea, the guy who wrote the code, or the guy who tested it and spotted the vital flaw that deserves the credit?

3. Its impossible to measure the impact of development decisions on the company's bottom line. The main contribution of development to sales is in not screwing up, and its necessarily difficult to figure out what's not wrong with the prodict that could be wrong with it. No-one is going to get a bonus for the application not crashing, or the file dialog not coming up in the wrong directory all the time. Its a strangely rarely remarked on part of the silicon valley compensation model that sales and support staff get bonuses but little or no equity, whereas developers get equity but usually no bonus. But it makes sense, since developers really determine the quality of a software company's main capital assets.

These criticisms apply to poorly-run software shops, yes. And yes, those are probably in the majority. But within an organization of competent people, it's not hard to tell who's more productive, even when "producing" means not-writing-code, and reward them for it. The trick is the management must also be competent, which means any software operation run by non-technical folks is doomed to either fail or limp along.

Also if you want to see what actual technical people think about this, check out the Slashdot comments on the same article:
http://developers.slashdot.org/story/09/12/23/1820214/Why-Coder-Pay-Isnt-Proportional-To-Productivity

Isn't this true of almost everyone who doesn't have a "bricklayer" kind of job?

In "hardware" companies the amount of software work always seems large compared to the amount of electronics work. The electrical engineers often run the projects (otherwise what else could they do with their time) and many would have trouble justifying compensation disparity when they have trouble fully appreciating the productivity disparity that is so apparent to programmers.

Projects or assignments are usually unique enough to introduce significant uncertainty into relative productivity assessment in the short run.

Judging the power of a problem solver by their programming skills? A smart high school kid can come up to speed on a language in two or three weeks. Hiring programmers is still an oddity and we don't have it quite right.

Thanks for the link.

I agree with the comments about negative productivity. The worst programmers create work. As David Parnas said here, "One bad programmer can easily create two new jobs a year."

If you don't know that programming isn't about writing code, then you shouldn't be a programmer. Unfortunately, almost every college course related to computers make it all about programming.

And I'm not surprised that an economist thinks its all about lines of code. Economists think making cars, toasters, etc., is all about numbers produced and minimizing the number of low paid workers. After all a $100 crap product is the same as a $100 high quality reliable product when the economists computes the GDP. That the American company is bankrupt in a few years while the Asian firm dominates the market doesn't show up in GDP.

The cost of bad code has been hidden by advances in semiconductors, where the metric that matters most of all is defect rate. The firms that survived were the ones who placed all the emphasis on quality, halting output if it was too low, and ultimately ended up with high volume and low cost. Programming over that same period has been marked by quests for the holy grail of quality without effort in a seemingly endless stream of new programming languages and paradigm fads.

The post yesterday about the Afghan dependencies is a simple rendering of many real world software systems - hundreds of mostly anonymous programmers have contributed to the system, but if the code of just a few of them isn't right, the entire system is as dysfunctional as Afghanistan.

But who am I to complain; I got paid very well for my role in test engineering: figuring out which part of the system was causing the system crash, data corruption, or any of a hundred other problems. Then came getting the guy who wrote the code to figure out the problem and fix it. For those devoted to cranking out code, they seemed to view me as the problem: how dare I tell them their code was wrong.

Now think about the compensation metric for test engineers. Should they be paid by the number of days they delay shipment to fix showstoppers? If that is the metric, then the programmer cranking out hundreds of lines of low quality code is ideal.

There's also a processes and systems problem. If you know you have a broad spectrum of programmers of varying ability, and any one of them has the ability to screw your code base up, one very common approach is to require your developers to do a *lot* of work to demonstrate they won't screw stuff up. What this ends up doing is forcing the best programmers to take a massive (and unnecessary) productivity hit in order to protect the code base from the mediocre to poor programmers. If in the normal course of affair an average programmer takes 2 days to do a feature, and 3 days to jump through hoops (and I've seen it that bad) and an above average programmer can reliably do 10 features in 2 days, but still takes 3 days a piece to jump through hoops, then you go from an average productivity multiplier of 1:10 (10 features in 2 days vs 10 feature in 20 days) to 32:50 (10 features in 32 days vs 10 features in 50 days). So even though your best developer is (oversimplifying here) 10 times as productive at writing code... you've hobbled him to less than 2 times as productive in so that you can safely let the lesser programmer work on your code base.

Perhaps an even more interesting question for social scientists: What do the highly productive programmers do to communicate their value to management?

One problem is that productivity isn't single dimensional. Do you want the code guru who can solve the problem 10x faster than anyone else. Or do you want the code guru who can write code that lesser mortals will be able to understand and maintain 10 years after he's left the company.

As well, very few projects are blank paper designs where the programmer can exercise his skill to maximize his productivity. In fact, there are tons if institutions that do their best to ensure that they can't. Far more important than superstar-dom is uniformity of code allowing programmer fungibility so the project doesn't hiccup when any individual member goes on vacation.

I do remember one friend who worked at a company that (he claimed) let go any programmer that was so good that the department was beginning to depend upon him. To be fair, they also fired the incompetent.

I think my value in life is that I'm good at pointing out what shouldn't be done and don't do it. I'm not sure how this can be measured. Maybe I'm just a lazy ass. Or maybe I know something. I'm not sure.

The best programmers get rich from stock options in start-ups.

How do I know they are the best? Because they got rich from options in start-ups.

Here is another question. If your management can't parse worker quality, how can they tell you how to improve? If you are a mid-tier but motivated worker, how does that make you feel?

I haven't actually seen good coaching and training on a systemic scale, I think it's kind of a myth, but a lot of people seem to think it exists and should be sought.

"Seems like there could be computer programs that rate code on some parameters."

This is been attempted, but such metrics are trivial to game (really, this is right smack in the skill set of even a mediocre developer). Any more complex static analysis of code quality can probably (if I remember my theory courses from graduate school) be proven to be impossible (equivalent to the halting problem).

"There could be 3rd party auditors that go around and check code for elegance or whatever you want to call it."

"Code Inspection" is a process that is in effect having auditors review code. It can work, if done correctly. Typically the inspectors (auditors) themselves are developers, because there is no one else qualified to understand the code. If you have someone qualified to inspect the code, and they aren't a developer, then you are wasting money when you could be having them develop code. However, I've seen one mediocre developer inspect another mediocre developer's code, and the results weren't pretty.

"Sometimes just having unreasonable workloads will cause the marginal workers to opt out voluntarily."

Truly unreasonable workloads will cause the good programmers to opt out voluntarily as well. Given a hard deadline with a strong disincentive to miss it (like getting laid off), developers, mediocre and otherwise, will simply submit whatever code they have whether it works or not, gaming the system as necessary to pass all the gates. In fact, this is the right thing to do: management has decreed via incentives that schedules are more important than quality. Employees of any ilk really only know what their employer finds important via the incentives that are in place; the latest PR ploy to "improve quality" is meaningless if the incentives don't match.

I recommend most strongly Robert D. Austin, MEASURING AND MANAGING PERFORMANCE IN ORGANIZATIONS, Dorset House, 1996. At the time he wrote it he was an executive with Ford Motor Co. in Europe. Now he's at the Harvard Business School, I believe. His Ph.D. dissertation is worth a look too, available from University Microfilm. He frames a model based on agency theory that shows how difficult it is to evaluate all aspects of an employee's performance in an information-based job (like software development).

http://www.amazon.com/Measuring-Managing-Performance-Organizations-Robert/dp/0932633366/ref=sr_1_1?ie=UTF8&s=books

This book changed my whole way of thinking about incentives and processes in software development.

Another thing is that programming attracts a lot of people who like computers and gadgets but who just aren't that great at math or logic in general. A great programmer has a lot of training but he also has innate logical ability, which is difficult to measure. A poor programmer may have the same amount of training, but if he's not good at logic puzzles then he should probably get another job. Unfortunately there probably aren't enough signals (e.g. low pay) to push him to do so.

When I have done development work (either design or actual code), I have always compared my work against the company's bottom line. Did what I have just implemented impact positively the bottom line. If so how much? For example, on one job - I designed the data in such a way, that a CPU intenstive process could work the same database - completelely in parallel - with no data contentions, this allowed the system to run in nearly half the time (We split the work over two CPUs). This saved the company from paying IBM for more CPU capabilities on their machine.

Regardless of the work, I always asked the question, how does this effort help the organization? When I know the answer - I proceed, otherwise - I look for more productive uses of the time.

Now, I consider myself only a fair developer. It is not my passion. How should I be compensated? With the amount agreed upon between me and my employer. Everything else, including how much other people are getting paid is irrelevant.

This is also true to systems administrators. There is a significant difference between the 'good' engineers and the 'bad' engineers - but the productivity output doesn't look much different and the salary differences are relatively small.

As a manager of an IT department I have thought about this quite a bit. What I have concluded is that there is a substantial amount of non-wage compensation, especially in reputation.

The 'good' engineers (and developers) typically get:
1. More interesting projects.
2. Their pick of projects.
3. Wide latitude in deciding how their projects are designed and implemented.
4. Are highly sought out for opinions outside their core job role.
5. Quite a bit of discretion in working on unofficial projects (skunk-works).

"IT often uses certifications"

Be careful not to confuse information technology with software development. I've done both, but they're completely different jobs and completely different skill sets. There are those that are skilled at both, just as there are people that can write fiction and do advertising copy, but they're not the same thing, even though to the outside observer they both involve putting words on a page.

I was a little surprised to find that there are hiring managers looking for software developers that use certifications as a negative signal.

The best programmers get rich from stock options in start-ups.

How do I know they are the best? Because they got rich from options in start-ups.

Proof: Bill Gates.

He delivered the absolutely best code, with lots of misfeatures, bugs, illogic, but superb marketing to ensure people bought the cheap crap and then had to keep coming back year after year in the hopes of getting a new version that doesn't blue screen when you are just thinking of saving your last two hours of work.

The losers were those who carefully designed and implemented the systems to operate non-stop as promised for as long as the power remained on. This requires limiting features to just the essential ones, employing tedious design and code reviews, and extensive testing, as well as follow up support and maintenance.

You don't find any billionaire programmers who ever released their code without the exclusions: "This software is not designed or valid in any safety critical applications, specifically nuclear power plants, medical devices, aircraft control systems, or any other application where death or destruction may occur."

Now imagine you are considering undergoing some experimental medical procedure to possibly save your life from a fatal brain tumor, and to help reassure you, the surgeon introduces you to the programmer who developed the robotic control software with "Joe is our code monkey - he cranks out at least a 100 lines of code an hour, twenty hours a day - he is just a real wiz on the programming. He just coded up the special 3000 lines of code for your case in the last day, and by the time of your surgery this afternoon, he'll have completed the last thousand lines he thinks are needed to cure you. We were really lucky to get him to leave Microsoft to work for us. He is a real wizard and we trust him with your life."

I am the lowest of the low. A support programmer for a business application for a Customer Service department.

This is somewhat exaggerated in bits and I have only every worked for one company but

Almost everyone else can get the job done. They are over educated and over talented for the job. Good people. You need to deliver something to Customer Service that gets the job done and doesn't always and completely blow up.

Beyond that the little bit of extra quality from person A to person B just doesn't matter. They can all do the job. Who is best at tic tac toe?

I can see how that might-should be different when your software is your companies product!

But I would think most programmers are business application programmers or maybe, gasp, government progammers.

In my world it would be insane to pay someone 10xs what someone else is making.

My own experience was mostly one-off real-time embedded systems stuff, often for some sort of testing application. The real-time aspects of the situation meant that you couldn't just brute-force the problem: success usually depended on an "Aha!" moment of insight into algorithms and data structures that made a solution feasible. There were programmers that seemed to never have those "Aha!" moments, but who did quite well in other types of programming positions.

Of course, that kind of programming is/was the last place where software was an art, rather than engineering.

Increased productivity correlates positively with a decrease in wages or with maintaining wages at the same level. There is nothing special in the work of programmers that makes this so. It is a global trend in capitalist production relations.

1. Having worked as a programmer and manager for many years, I "believe" the 10x ratio, but honestly I can't produce much quantitative evidence for it.
2. A lot of the best programmers like their work and would do it for free, which depresses wages. You see the same effect in music and art.
3. I know specialists who are among the handful of people in the world who can do certain kinds of essential programming. Their rates as independent contractors aren't consistent with the scarcity and apparent value of their skills, so I share TC's interest in having economists study this kind of disparity.

Usually, it is quite obvious after six months who is great, and who is bad. But then, software is a collective endeavor: you need the composition of multiple pieces to give the final product. After six months, you are still willing to keep the worse guys, but maybe not the worst, in order to avoid training a new guy who could be great.

Of course, this does not mean the pay has to be the same. But usually, pay inequalities in a programming team result in the sloppy becoming sloppier, and the product quality itself then hangs on those bottlenecks. In some abstract sense, the "superstardom" of a few programmers is only worth as much as it is shared to the remaining team. Maybe a network effect?

I think the top programmers are adequately rewarded for their productivity - they get stock options and opportunity to create a breakthrough product.
As for developers who are fixing bugs in that awful in-house systems, they are just at the bottom of the food chain and can't get management recognition for their achievement.

In my experience programmers often run aground because they either did not understand the requirements, did not have the technical knowledge to come up with a efficient solution or did not make use of existing software and re-invented the wheel. Management is constantly trying to measure programmer productivity and get a handle on what their employees are adding to the company usually unsuccessfully

Comments for this post are closed