Where does economics fail the worst at quantification?

by on June 11, 2016 at 1:17 am in Data Source, Economics | Permalink

Pete asked that question, more specifically:

Economics tries to put a price on most everything, even things that are nearly impossible to quantify such as happiness. Where in your opinion does the economic approach currently fail the worst at measurement and quantification?

I believe economics does not measure “culture” very well, including the all-important “corporate culture.”  It is hard to think of a good way of encoding companies to reflect how things actually work inside the firm, all the harder for a region or nation.  In this context you could define culture as consisting of expectations about the reciprocal behavior of others, across a variety of contexts.  We do OK however if it is just a univariate measure of trust, see Knack and Keefer.

It is hard to measure rates of price inflation over longer periods of time.  Since 1900, has the American economy had massive inflation or massive deflation?  If you are spending fifty cents, it is massive inflation, as today that sum hardly buys you anything, but earlier you could have received a nice white shirt.  If you are spending 100k a year, there has been radical deflation.

Those to me are two of the worst failures of quantification in economics.  What else?

Here is Robin Hanson on “the data we need.”

1 prognostication June 11, 2016 at 1:29 am

My observation is that any attempt to place price values on things that are never bought and sold in any market tends to go pretty badly. I used to work in a firm that did a lot of regulatory environmental economics, and the prevailing cost-benefit framework of the federal government means that there is a big show made of trying to value both use and non-use benefits of environmental goods and services. Well-established theory makes it clear that revealed preference methods don’t capture all of the values people place on these things, so those measures are a lower bound at best, and yet when you see the results that WTP/WTA studies find, it immediately becomes clear that people have NO IDEA how to assign a dollar value to these things.

2 Rich Berger June 11, 2016 at 9:37 am

Bingo. And yet they regulate away, assuming that more regulations must be better.

3 anon June 11, 2016 at 9:47 am

Right. Because we do not know how to price non-carcinogenic tap water, it is unimportant. Hard to imagine the socialist impulse sometimes.

4 Rich Berger June 11, 2016 at 2:32 pm

Apparently we do know how to price it. You assume that companies want to poison their customers unless government stops them. That is false.

5 anon June 11, 2016 at 3:45 pm

You can’t double back, Rich. You were the one that said we “regulate away, assuming that more regulations must be better.”

Nothing to do with carcinogens there.

6 NPW June 11, 2016 at 4:30 pm

Always assuming that more regulations must be better =! more carcinogens in water must be better

7 mulp June 11, 2016 at 5:48 pm

You are arguing that Blankenship did not want to kill the miners working for him, just that he placed zero dollar value on preventing killing them, and that justifies conspiring to violate safety rules adopted with industry experience to protect workers from being killed.

The question is whether the lives of the workers had a monetary value higher than the cost of compliance with rules that are often less comprehensive than a rule maker protecting their child or spouse due to profit motive pressure from industry. Blankenship clearly believed lives were worth less, or even worthless, relative to the opportunity cost of reduced production to comply with safety regulations.

Without society, would Blankenship have resumed mining operations within two weeks by focusing only on retrieving equipment, repairing transport, and ignoring ventilation and dust suppression as unimportant relative to the opportunity cost of lost output?

From your standpoint, the mine being shutdown effectively forever because of the unjustified burden of complying 100% with safety rules, must be totally unjustified. Human life is worthless in comparison?

8 Rich Berger June 12, 2016 at 9:34 am

No. The price of non-carcinogenic water is what the water company charges. In my case, it’s just the cost to run my well.

9 Troll me June 12, 2016 at 8:38 pm

Obviously companies don’t WANT to poison customers. But they would like to under-portray the risks (or present themselves as having a certain quality but then cutting corners) so they can provide a lower quality output without the customer knowing. So we have minimum regulations and the cost is obviously included in the price.

10 prognostication June 11, 2016 at 11:56 am

Yeah, I should point out that this is not the argument I was making, although I’m sympathetic to it. There are things we definitely should be doing where we can’t quantify sufficient benefits, and there are things we probably shouldn’t be doing but do anyway by fudging the benefits. Both circumstances exist. I think the problem might be with the framework.

11 Joan June 11, 2016 at 1:31 am

In 1900 the median wage was about $500 so 100k would be enough to hire 200 people to wait on you. Many people today would prefer that to all the labor saving things we have today. The biggest short fall would be the available entertainment, more books for readers, movies etc.

12 Nigel June 11, 2016 at 4:32 am

Indeed, but ten times that wouldn’t have purchased you a single does of antibiotics, for example.

13 daguix June 11, 2016 at 4:35 am

Not so sure. There has been a large rise in income inequality during the last 35 years. But the number of helps has not increased.

14 Careless June 11, 2016 at 11:50 am

Many people today would prefer that to all the labor saving things we have today

Really? Of the thousands of people in this country who could easily afford to hire a hundred people to wait on them, how many do so? Because AFAICT, it’s zero

15 planxty June 11, 2016 at 1:17 pm

@Careless: The thousands of people in this country who could easily afford to hire a hundred people “to wait on them,” do so by being owners of assets.

Anyway, what parameters do we use to frame a period of time during which measures of inflation and deflation make sense? How about looking at water supplies, the places people live that require expensive delivery systems for water, and measuring the price of water over long periods of time?

Would love to read more on this insane topic.

16 Corvus June 11, 2016 at 1:48 pm

Your self-naming tempts me to taunt that naming is more accurate than your opinion stated here. I would retort that of the “thousands . . . who could easily afford to hire” – that 100% do hire people to wait on them. In one capacity or another. While hiring a butler or a maid is no longer stylish, having your lawn mowed for you, changing your auto’s oil, hiring someone to pre-cook your meals, or to be a nanny to your children, or to babysit your pets, the list just goes on. From fast food, to your dry cleaners, there are thousands of ways we hire people to “wait on us” that did not exist in 1890. And hire them we do. You want conspicuous consumption? How about personal trainers? How about “appearance” consultants – who advise on everything from how you dress to your spoken accent. Give me some time, I am sure I can come up with some more – bodyguards, secretaries, security consultants . . .

Therefore, your estimation that 0% hire those hundreds? Is careless indeed.

17 Ryukendo June 11, 2016 at 2:36 am

Status.

18 Steve Y. June 11, 2016 at 3:13 am

This point is related to corporate culture: A fundamental approach in financial analysis is to value the parts of a business and compare the sum to the valuation of the entire enterprise, which is easy to do if the business is a public company. Depending on how well the pieces fit together, enormous variations can occur in the whole-company market cap. But how does one quantify the value of a different leader or a different system/organizational structure?

Let’s illustrate this question in less abstract terms: in the 2013-2014 NBA season the Golden State Warriors’ core lineup consisted of Stephen Curry, Klay Thompson, Andrew Bogut, Harrison Barnes, Andre Iguodala, David Lee, and Draymond Green. The Warriors finished the season with 51 wins and were knocked out in the first round of the playoffs by the LA Clippers. In the 2014-2015 season the same lineup produced 67 wins and won the championship under new coach Steve Kerr. In 2015-2016 , minus David Lee, they set a regular-season record of 73 wins; experts are saying the Warriors are revolutionizing basketball with their style of play. Improvement in individual players’ skills cannot explain the dramatic improvement in the team’s performance.

How does one quantify the difference between Mark Jackson’s coaching (which was pretty good because the Warriors had very rarely made the playoffs) and Steve Kerr’s? How does one measure the value of different ways to organize the same components? Answer the question comprehensively and the world will beat a path to your door.

19 Greg June 11, 2016 at 12:11 pm

I agree with your general point, but it’s even harder to quantify than you lay out. The players did get much better individually year over year, especially because many of them were relatively young. The team as a whole probably also became a better whole, apart from the coaching change. There’s even a little bit of a natural experiment at play, where Kerr was out for a while and the team put together a great record under Luke Walton. Singling out Kerr as being responsible for the team’s improvement is an over-simplification.

20 Claude Emer June 11, 2016 at 5:00 pm

And Iguodala, the NBA finals MVP in 2014-2015 was not on the the 2013-2014. Neither was Shaun Livingston

21 Anatoly Karlin June 11, 2016 at 3:41 am

My impression is that attempts to quantify GDP per capita for preindustrial (Malthusian) economies remains a very hazy and muddled affair.

Also of course quantifying technology is very hard.

22 Todd K June 11, 2016 at 7:12 am

Good one.

Gregory Clark has said that Agnus Maddison’s GDP per capita “estimates for the years before 1820 are fictions. They are based not on empirical evidence, but on unsubstantiated and demonstrably implausible theories of the nature of life in preindustrial societies. Even some of the post 1820 estimates have an equivalently dubious provenance.”

23 Barry June 11, 2016 at 7:54 am

Clark’s posit, as you cite, withholds substantiated basis. Thus, mutual typing propagates moot discourse. Please refrain and delete your account.

24 akarlin June 11, 2016 at 8:24 am

I think the problem is that for Malthusian societies a distinction really really has to be made between:

(a) Food availability per capita (i.e. standard measure of calculating historical GDPs – how much grain average carpenter’s salary would buy);

(b) The amount of “stuff” per capita that Malthusian civilizations produced such as housing, jewelry, etc.

What’s somewhat paradoxical and I suspect causing a lot of confusion is that these two things tend to go against each other, i.e. during times of plenty when there is lots of unused carrying capacity food availability is high but non-agricultural GDP per capita is low because these periods tend to be highly rural; while societies that reach the Malthusian limits eject tons of peasants into the societies (where many become workers) and even agriculture itself needs increasing numbers of artificial inputs to feed the teeming masses all, with the result that “non food” GDP goes way up.

My cursory impression is that a lot of people tend to ignore one or the other. E.g., Greg Cochran himself in 10,000 Year Explosion doesn’t really go into (b), while someone like Ian Morris largely ignores (a).

25 prognostication June 11, 2016 at 10:40 pm

Yeah, I’ve run into this with long historical time series… if you go back even to, say, the 1870s in the U.S., a given amount of money means such a wildly different set of things that translating it into present-day value quickly becomes silly. We do it, because reviewers insist upon it, but the numbers don’t make much sense if you think about them.

26 Troll me June 12, 2016 at 9:49 pm

A Malthusian minimum per capita output among peasant classes could mean a lot of different things.

Among other things, there’s also the fact that not all land had been developed, which suggests that something was clearly off in that consideration. Also, it is doubtful that very many people at all literally lived at the Malthusian minimum with, say, zero education, zero luxuries of any sort, etc. The counterexample that comes to mind is the situation of the Russian peasants pre-revolution.

And were any sort of “peasant” aware that calculations were being made to avoid such outcomes (a revolution against a system which extracted anything more than the minimum required to prevent revolution), most likely nothing short of “fair” would be enough, with a laissez-faire property rights protection assumption where one gets to keep at least one’s own production, serving as a rock bottom basis for what might be deemed tolerable. Surely, there would be those who would like to extract all the positive externalities, but it is unlikely that the broader population would see this as tolerable unless it were done in a manner which tended to funnel those resources back into further capacity of the economy rather than mere extraction of “rents” (illegitimate in such a case) by those who happened to position themselves to do so.

27 Rishabh June 11, 2016 at 5:25 am

ABC

28 Ray Lopez June 11, 2016 at 5:41 am

Economics does not measure anything that well. For example, Total Factor Productivity is a residual. Another is GDP/GNP, see Cambridge Capital Controversy. Yet another is innovation: it’s assumed to be exogenous. Another is the Cobb-Douglas production function, which is just assumed to exist to make the math easy. Another is partial vs general equilibrium, which are just assumed. In fact, pretty much most of economics is hocus-pocus, not grounded in anything empirical.

29 Rich Berger June 11, 2016 at 8:36 am

It pains me to write this, but I have to agree with Ruy. I don’t think economics measures anything very well.

30 Noah Carl June 11, 2016 at 6:27 am

Re Knack and Keefer, trust may be confounded by intelligence:

http://www.sciencedirect.com/science/article/pii/S0160289614001275

31 Gulic June 11, 2016 at 6:29 pm

Also, I seem to remember that Knack and Keefer used distance to the equator as an instrumental variable. That paper was universally derided by all the profs in my department as being empirically very weak. You probably need to look to experimental economics literature for decent trust measures.

32 Jon June 11, 2016 at 6:57 am

Economics is often misapplied in value judgments in a form of circular reasoning. A common form is of this is “In a free market there is scarcity, but not shortages because everyone gets what they are willing to pay.” That is small comfort if your willingness to pay is limited by your resources and the item you are “unwilling” to pay for is a lifesaving medicine.

One assumes that a measure of what one is willing to pay for something is a proxy for its value; then shows that people get what people receive when they pay for something then is fair value. Therefore anything that causes people to get something different is sub optimal.

33 rayward June 11, 2016 at 7:52 am

Your point is captured by Hanson’s post (“the data we need”), although I suspect he didn’t intend to make that point. “Value judgments in everything” should be Cowen’s calling card (as opposed to “markets (including prediction markets) in everything”), markets being this blog’s value judgment. Hanson suggests collecting a data set for government regulatory policies, certain that he is that the data will confirm his value judgment that government regulatory policies (almost always) produce sub-optimal results. Sub-optimal compared to what, one might ask. Cass Sunstein might look at the data and decide that government regulatory policies can be improved (that’s his value judgment), while others (Hanson, Cowen, et al.) might look at the data and decide that government regulatory policies can’t be improved because, by their nature, they produce sub-optimal results (that’s their value judgment) – and, besides, government regulatory policies are just signaling anyway. Which leads to the other line of inquiry Hanson suggests: a data set for particular human behaviors, including what promotes cooperation that produces optimal (or near optimal) results. This is tricky, since markets are based on competition, which is the opposite of cooperation; indeed, cooperation between competitors undermines markets and is discouraged (by, among other things, criminal sanctions). I suppose government policies (regulations) are intended to find the space between markets and cooperation that produces optimal (or near optimal) results. Of course, such government policies satisfy almost nobody, not the competitors (each believing in his own virtue), not those who believe markets alone (including prediction markets) produce optimal results, and not the man (or woman) on the street who doesn’t think about such things and wouldn’t know the difference if he (or she) did.

34 Bill June 11, 2016 at 7:36 am

Maybe the question should be where does economics fail the worst in quantification WHERE IT MATTERS.

My recommendation for that one is self-production: doing things where you had the alternative to work but you chose work that was not recorded because it was not taxed.

You’re probably thinking criminal activity, but I’m thinking fixing up and remodeling a house with your own effort and then selling it because of capital gains exclusion.

People were investing their time in a house, not because they wanted it, but because they thought you wanted it. You could measure the amount of Home Depot sales during the housing run up as a proxy, and this might have told you how heated the housing market…and peoples expectations…were, but no one tracked the amount of time individuals were putting into their house based on the belief that you would later want the change and they would make the money.

35 Bill June 11, 2016 at 7:38 am

My next door neighbor would show up on the employment statistics as unemployed, when in fact he was self-producing and fixing up his house for sale.

36 Barry June 11, 2016 at 7:42 am

Corporate culture and sport team coaching(warriors/”dubs”) ought to be observed, realized by synthesizing respectful base components and corresponding variables with explicit results identifying unique outcomes vis-a-vis exogenous factors. Our aggregate data shall suggest beer-drinking and further discourse.

37 anon June 11, 2016 at 7:49 am

Economics is a funny field. It effectively describes aspects of life, but constantly reaches for more, to be a universal theory of life.

I don’t think that can ever work. Econ is a highly cognitive overlay on what are often very non cognitive choices.

Why is there obesity?

38 Barry June 11, 2016 at 7:59 am

Propensity to indulge exceeds a nature to restrain. Biologically speaking, gut expansion occurs when a physiological system adapts to increased chicken wing consumption.

39 anon June 11, 2016 at 8:23 am

Possibly, though there are quite a few more psychological, metabolic and lifestyle theories about this.

The thing is though, econ is often brought in as a factor or even a solution in a problem that is not very well understood.

Should you have to pay a friend if you don’t go to the gym? This has been proposed.

40 anon June 11, 2016 at 8:26 am

But getting back to “why is there”, I think it is a problem for the economic view that so many people are such bad managers of their own health. Same for other substance abuse.

People are bad at these things, but good at lifecycle consumption?

41 Troll me June 12, 2016 at 10:42 pm

Taking optimization too far is not good for quality of life. It kind of gets back to one of those question of “what you measure gets optimized”.

I really hope we don’t bother with any sort of optimization of physical or neurological bases of happiness. Because then we just become a bunch of pleasure-bots to be optimally motivated for something else, no?

Just don’t even go there, imo.

42 Scott Gustafson June 11, 2016 at 8:40 am

“Where does economics fail the worst in quantification WHERE IT MATTERS.”

Uncertainty. Economic behavior changes significantly as uncertainty changes. We have little if any way of quantifying the uncertainty households and businesses feel.

Regulation. Measuring the expected effect of a regulation is difficult at best. Measuring it’s effect after it has been imposed is almost as difficult.

43 mulp June 12, 2016 at 2:05 pm

Wasn’t uncertainty far higher in the 50s and 60s than today?

Wasn’t regulation inn the areas economists talk about the most highest in the 50s and 60s: banks and Wall Street restriction limited options for 99% of the people, blacks and browns could not buy most real estate, blacks, browns, white trash couldn’t vote

Do you see certainty when you do regular duck and cover drills and build bomb shelters and worry about keeping out neighbors who didn’t build them?

(I owned a house in the early 80s which had been owned by an employee of a defense contractor in the 50s and 60s, and it had a fallout shelter far superior to any of the plans I read as a kid in the 50s and 60s and thought about building because having one would be cool.)

In the 50s and 60s, uncertainty seemed to power the economy, power the nation.

44 Ironman June 11, 2016 at 9:58 am

If you want a topic ripped from recent headlines, how about quantifying the social cost of carbon? I’ve seen estimates range from a high of $220 per ton, to somewhere between $11 and $57 per ton, to possibly negative?!?

45 anon June 11, 2016 at 10:09 am

Agreed.

Is there any economic theory on when the price should have been set? Some kind of equilibrium theory? Are we allowed to be wrong, for decades?

46 mulp June 12, 2016 at 2:25 pm

What will be the price of unburned carbon in 2100, 2200, …

How about 2576 when the US is the similar 800 years Britain is beyond the Magna Carta?

Can the plunder of fossil fuels continue to grow for the next 1000 years, a mere 50% of Western history, 20% of world history and civilization?

What magic beyond the simple investment of labor will replace fossil fuels?

Is there a shortage of labor today that makes replacing fossil fuels with substitutes a serious drag on the economy?

Is there any way that a rising carbon tax, say an increasing tax per ton of fossil carbon based on successive numbers in the fibinoci series, $1 $1 $2 $3 $5 $8 $13 $21 $34 $55 $89 $144 $233 $377, would not increase gdp growth?

How can something that costs more kill jobs and growth when millions of workers getting paid is the reason it costs more?

If a magic technology were discovered that produced electric power for $1000 equal to what the Tesla Model S requires that lasts for 100 years in a 100kb box, would that create more jobs than the fossil fuel industry?

47 Ray Lopez June 11, 2016 at 10:26 am

The social cost of carbon depends on whether you think AGW is real or not, and whether it is beneficial or not. Some claim a warmer earth is actually beneficial, and even when not one study calculated the USA, out of the entire world, would benefit from warmer temperatures.

48 anon June 11, 2016 at 10:31 am

I don’t think your questions are compatible with rational expectations theory.

We are supposed to know, have decided, reached a new equilibrium.

49 mulp June 12, 2016 at 3:30 pm

So, you consider eliminating all acidic oxides, heavy metals, from the flue games of burning fossil fuels to be extremely cheap, and not something that makes burning fossil fuels uncompetitive with substitutes that do not burn fossil fuels?

The “war on coal” by Obama and EPA that is in law has nothing to do with carbon emissions. Not one case has been brought about an actual EPA rule that has the effect of law had dealt with carbon emissions. (The CPP lawsuits are to prevent rule making.) The war on coal is purely the EPA implementing the 1970 Clean Air Act modified by the 1990 amendments to allow for pricing NOx, SOx, mercury, lead, particulates instead of only absolute limits.

By the way, why has global job creation rates gone down after global oil prices have fallen by 60-80%. Isn’t cutting labor costs to cut prices the key to creating jobs faster?

Surely job cuts to deliver cheap energy is not a harmful social cost because people should not be required to work but should get free stuff! Right? In the ideal world, gasoline would flow from the ground like a spring. Then all fossil fuel jobs would be eliminated down the level of delivering water, and the power grid would be eliminated because you just buy a gasoline generator if you need electricity. The social costs of eliminating jobs plundering and delivering fossil fuels would be zero, and the sound of generators would be like music by Yanni.

50 Troll me June 12, 2016 at 11:08 pm

The cost of a warmer planet in the 100 years timeframe is almost certainly quite negative. But in the billions of years timeframe, it would be worth debating just how hot is too hot, considering that the livable period on earth could be extended by some millions of years by having a warmer planet.

In the meantime, my gut feeling is that warming is likely to be too fast. Moreover, having a stock of hydrocarbon fuels for other uses could prove useful in many other ways. It’s really not obvious, but transforming all hydrocarbons into CO2 as fast as possible seems pretty dumb to me.

51 Jer June 11, 2016 at 1:03 pm

Economics fails to measure/predict/quantify/model the results/benefits of having a hyper-rationale and hyper-quantitive approach to life’s diverse and emergent behaviors.

Witness the ‘economization’ of a beloved fairy tale trope:

http://www.smbc-comics.com/index.php?id=4136

(contributed for levity)

52 Merijn Knibbe June 11, 2016 at 3:43 pm

Head and shoulders the worst fail is utility in its individual as well as in its intertemporal macro variant. Despite all efforts, there is no general agreed upon procedure how to measure individual or social utility. There are no manuals how to do it. There are no succesfull attempts.There isn’t even a viable definition of utility.

53 Troll me June 12, 2016 at 11:14 pm

Not really true. A lot of what is learned about modelling utility has to do with comparing utility sets across a very large diversity of mathematical forms, any of which can be understood to be inferred from some sorts of assumptions about underlying preferences.

Ranking, in short. I’m not sure it would be desirable to go any further than that.

Generally speaking, willingness to pay (ignoring inequality) across a distribution of people is assumed to hold information about this.

54 Michael Tinkler June 11, 2016 at 9:27 pm

Agatha Christie once said (and don’t sue me – I read it in a BOOK, not online!): “I never imagined I would ever be so poor as not to be able to have a servant, or so rich as to have an automobile.”

55 Todd Kreider June 12, 2016 at 9:41 pm

Great quote!

(If true… If not, my lawyer will be contacting your lawyer)

56 Todd Ramsey June 12, 2016 at 11:52 am

A couple fails, although I’m not sure they fall squarely into this category.

First, the economic press and blogosphere refer to the real interest rate as equal to today’s observed interest rate minus observed past inflation. The real interest rate is actually equal to today’s observed interest rate minus future expected inflation. Of course, expected inflation is harder to quantify than prior observed inflation, so the misnomer continues forward.

Second, the idea that no individual can achieve above-average returns over lengthy periods of time, as stated in the efficient market hypothesis, comes because of an inability to quantify the emotion of herd mentality. Knowledge available to the market is easily observable and “quantifiable”; the emotion of the herd is neither easily observable nor easily quantifiable, and is therefore ignored or presumed to not exist by efficient market theorists. They therefore claim that consistent above-market returns are “luck” and will regress to the mean over a sufficiently lengthy time horizon.

57 Willitts June 12, 2016 at 12:19 pm

Public goods?

58 zincirli vinçler June 15, 2016 at 5:02 am

They therefore claim that consistent above-market returns are “luck” and will regress to the mean over a sufficiently lengthy time horizon. hidrolik asansör

59 Anon June 15, 2016 at 5:11 am

.50$ buys you a lot more transistors over the last 100 years.

Comments on this entry are closed.

Previous post:

Next post: