Two things are absent in this debate, however.
First, much evidence shows that what determines technological innovations isn’t some sort of “exogenous innovation capacity,” but incentives…
Schmookler illustrated these ideas vividly with the example of the horseshoe. He documented that there was a very high rate of innovation leading to improvements in the horseshoe throughout the late 19th and early 20th centuries because the increased demand for transport meant increased demand for better and cheaper horseshoes. It didn’t look like there was any sort of limit to the improvements or any evidence of an “exogenous innovation capacity” in this ancient technology, which had been around since 2nd century BC. Then suddenly, innovations came to an end, but this had nothing to do with running out of low hanging fruit. Instead, as Schmookler put it (p. 93), it was because the incentives to innovate in this technology disappeared because “the steam traction engine and, later, internal combustion engine began to displace the horse… “
Their full post is here.
I have changed my mind on this issue quite a bit over the last four to five years. Yes incentives matter, but outside of extreme environments are changes in incentives explaining the changes in what we observe? I now think it is of critical importance where a sector or economy is on “the innovation curve.” It was easier to innovate in game theory in the 1980s than it is today, even though the salaries of top economists have risen significantly. The pharmaceuticals market is larger than ever before, and yet the pipeline is largely dry. We are simply at a point where further breakthroughs are hard (and it is not obvious that FDA innovation taxes are getting worse over this period of change.) Weren’t so many inventors of the 19th century largely “yahoos,” with no fancy degrees, relatively low pay, little or no peer review, not at the peak of the Flynn effect, and so on, and yet they were on a fruitful part of the innovation curve and performed wonders.
I think in terms of general purpose technologies and platform-like breakthroughs. Once you get them, innovation runs wild, otherwise it is tough sledding, with incentives still accounting for some of the variation within a particular place on the innovation curve.