Matt Yglesias suggests the notion is implausible, but I am surprised to read those words. Keep in mind, we have had a recovery in output, but not in employment. That means a smaller number of laborers are working, but we are producing as much as before. As a simple first cut, how should we measure the marginal product of those now laid-off workers? I would start with the number zero. If a restored level of output wouldn't count as evidence for the zero marginal product hypothesis, what would? If I ran a business, fired ten people, and output didn't go down, might I start by asking whether those people produced anything useful?
It is true that the ceteris are not paribus, But the observed changes if anything favor the hypothesis of zero marginal product. There has been no major technological breakthrough in the meantime. If anything, there has been bad monetary policy and a dose of regulatory uncertainty. And yet again we can produce just as much without those workers. Think of "labor hoarding" yet without…the hoarding.
You might cite oligopoly models and argue that the workers can produce something, but firms won't hire them because they don't want to expand output, due to lack of demand. That doesn't seem to explain that output has recovered and that profits are high. And since there is plenty of corporate cash, it is hard to claim that liquidity constraints are preventing the reemployment of those workers.
There is another striking fact about the recession, namely that unemployment is quite low for highly educated workers but about sixteen percent for the less educated workers with no high school degree. (When it comes to income groups, the lowest decile has an unemployment rate of over thirty percent, while it is three percent for the highest decile; I'm not sure of the time horizon for that income measure.) This is consistent with the zero marginal product hypothesis, and yet few analysts ask whether their preferred explanation for unemployment addresses this pattern.
Garett Jones suggests that many unemployed workers are potentially productive, but that businesses do not, at this moment, want to invest in future productive improvements. The workers only appear to have zero marginal product, because their marginal product lies in future returns not current returns. I see this hypothesis as part of the picture, although I am not sure it explains why current unemployment is so much higher among the unskilled. Is unskilled labor the fundamental capability-builder for the future? I'm not so sure.
It's also interesting to look at the composition of the long-term unemployed (not the same as the composition of all the unemployed, of course). Older workers with a college education are quite stuck, conditional on their being unemployed. And in this group, more education predicts a longer spell of unemployment. Is this ongoing "recalculation," optimal search theory, or is the roulette wheel simply coming up zero each time it is spun for these workers? Maybe a bit of each. If you want, call some of it age discrimination and relabel the idea "perceived zero marginal product."
In general, which hypotheses predict lots more short-term unemployment among the less educated, but among the long-term unemployed, a disproportionately high degree of older, more educated people? This stylized fact seems to point toward search and recalculation ideas, with some zero marginal products tossed in. Do aggregate demand theories yield that same data-matching prediction? I don't see it, at least not without being paired with a theory of concomitant real shocks.
Nothing in the zero marginal product hypothesis requires that these marginal products be zero forever. As the entire economy expands more rapidly (when will that happen?), the value of even a low quality worker can quickly become much higher. If you are opening up a new building, suddenly you really need that extra janitor and he is indeed more productive at the new margin.
Some people identify the zero marginal product hypothesis with the "hopeless dregs of the earth" description, but the two are not necessarily the same. Complementarity, combined with some fixed initial factors, can yield zero or near-zero marginal products of labor. (You'll see the phrase "excess capacity" used in this context, though that matches the oligopoly hypothesis more closely.) The "dregs of the earth" view is pessimistic, but the complementarity version of the zero marginal product idea can be quite optimistic, predicting a very rapid recovery in the labor market, once the interactions turn positive.
The "dregs" and the "complementarities" views also have different policy recommendations. The dregs view implies either hopelessness or a lot of fundamental retraining or ongoing assistance, while the complementarity view leads one to ask how we might mobilize positive complementarities (rather than leaving orphaned factors of production) more quickly. Perhaps there are some fixed factors, such as managerial oversight, and entrepreneurs do not want to strain those fixed factors too hard. How can we make such fixed factors more replicable or more flexible?
Addendum: Arnold Kling comments.