Opportunity Cost
The answer is b, $10. Your next best alternative to the Clapton concert is attending the Dylan concert which has a benefit of $50 and a cost of $40 or a net benefit of $10. The net benefit is what you give up by attending the Clapton concert.
Why Most Published Research Findings are False
Writing in PLoS Medicine, John Ioannidis says:
There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims. However, this should not be surprising. It can be proven that most claimed research findings are false.
Ioannidis presents a Bayesian analysis of the problem which most people will find utterly confusing. Here’s the idea in a diagram.
Suppose there are 1000 possible hypotheses to be tested. There are an infinite number of false hypotheses about the world and only a finite number of true hypotheses so we should expect that most hypotheses are false. Let us assume that of every 1000 hypotheses 200 are true and 800 false.
It is inevitable in a statistical study that some false hypotheses are accepted as true. In fact, standard statistical practice guarantees that at least 5% of false hypotheses are accepted as true. Thus, out of the 800 false hypotheses 40 will be accepted as “true,” i.e. statistically significant.
It is also inevitable in a statistical study that we will fail to accept some true hypotheses (Yes, I do know that a proper statistician would say “fail to reject the null when the null is in fact false,” but that is ugly). It’s hard to say what the probability is of not finding evidence for a true hypothesis because it depends on a variety of factors such as the sample size but let’s say that of every 200 true hypotheses we will correctly identify 120 or 60%. Putting this together we find that of every 160 (120+40) hypotheses for which there is statistically significant evidence only 120 will in fact be true or a rate of 75% true.
(By the way, the multiplying factors in the diagram are for those who wish to compare with Ioannidis’s notation.)
Ioannidis says most published research findings are false. This is plausible in his field of medicine where it is easy to imagine that there are more than 800 false hypotheses out of 1000. In medicine, there is hardly any theory to exclude a hypothesis from being tested. Want to avoid colon cancer? Let’s see if an apple a day keeps the doctor away. No? What about a serving of bananas? Let’s try vitamin C and don’t forget red wine. Studies in medicine also have notoriously small sample sizes. Lots of studies that make the NYTimes involve less than 50 people – that reduces the probability that you will accept a true hypothesis and raises the probability that the typical study is false.
So economics does ok on the main factors in the diagram but there are other effects which also reduce the probability the typical result is true and economics has no advantages on these – see the extension.
Sadly, things get really bad when lots of researchers are chasing the same set of hypotheses. Indeed, the larger the number of researchers the more likely the average result is to be false! The easiest way to see this is to note that when we have lots of researchers every true hypothesis will be found to be true but eventually so will every false hypothesis. Thus, as the number of researchers increases, the probability that a given result is true goes to the probability in the population, in my example 200/1000 or 20 percent.
A meta analysis will go some way to fixing the last problem so the point is not that knowledge declines with the number of researchers but
rather that with lots of researchers every crackpot theory will have at least one scientific study that it can cite in it’s support.
The meta analysis approach, however, will work well only if the results that are published reflect the results that are discovered. But editors and referees (and authors too) like results which reject the null – i.e. they want to see a theory that is supported not a paper that says we tried this and this and found nothing (which seems like an admission of failure).
Brad DeLong and Kevin Lang wrote a classic paper suggesting that one of the few times that journals will accept a paper that fails
to reject the null is when the evidence against the null is strong (and thus failing to reject the null is considered surprising and
important). DeLong and Lang show that this can result in a paradox. Taken on its own, a paper which fails to reject the null provides evidence in favor of the null, i.e. against the alternative hypothesis and so should increase the probability that a rational person thinks the null is true. But when a rational person takes into account the selection effect, the fact that the only time papers which fail to reject the null are published is when the evidence against the null is strong, the publication of a paper failing to reject the null can cause him to increase his belief in the alternative theory!
What can be done about these problems? (Some cribbed straight from Ioannidis and some my own suggestions.)
1) In evaluating any study try to take into account the amount of background noise. That is, remember that the more hypotheses which are tested and the less selection which goes into choosing hypotheses the more likely it is that you are looking at noise.
2) Bigger samples are better. (But note that even big samples won’t help to solve the problems of observational studies which is a whole other problem).
3) Small effects are to be distrusted.
4) Multiple sources and types of evidence are desirable.
5) Evaluate literatures not individual papers.
6) Trust empirical papers which test other people’s theories more than empirical papers which test the author’s theory.
7) As an editor or referee, don’t reject papers that fail to reject the null.
UHaul Pricing
I’m not convinced by his example but Andrew Roth (a former student of mine) has a found a nice source of data that could be used to discuss demand and supply, the difficulties of identifying price discrimination, and why it’s efficient to have "women enter free" nights at clubs.
Some high income earners are leaving
California because of its punitive tax rates. Could low- and
middle-income workers be leaving as well? One crude measure is to
examine the one-way rental rates for U-Haul vans. Using U-Haul’s website, I queried a one-way rental for a 10-foot van for October 1st, 2005.
One-Way Trip
Price Los Angeles to Las Vegas
$454.00 Las Vegas to Los Angeles
$119.00
Hattip to E. Frank Stephenson at Division of Labor.
No Pain Relief for Tort Sufferers
James Hamilton takes a look at one of the key studies on Vioxx and heart attacks. He is not greatly impressed.
I took a look at one of the studies on which the decision was
justified, written by Dr. David Graham and co-authors and published in Lancet
in February. This study looked at 8,143 Kaiser Permanente patients who
had suffered a heart attack and had also at some point taken a
nonsteroidal anti-inflammatory drug (NSAID), of which Vioxx (rofecoxib)
is one. Of these patients, 68 were taking rofecoxib while 4,658 were
receiving no medication at the time of their heart attack, a ratio of
(68/4658) = 1.46%. For comparison, the study looked at 31,496 other
patients who had also at some point taken an NSAID, matched for
characteristics like age and gender with the first group, but who
didn’t have a heart attack. The ratio of rofecoxib users to those with
no current medication was slightly lower (1.05%) in this second group,
which one might summarize as a (1.46/1.05) = 1.39-fold increase risk of
heart attack from taking rofecoxib compared to no NSAID. Is that
statistically significant, in other words, can you rule out that you’d
see a difference of that size just by chance? Yes, the study claimed,
but just barely.On the other hand, this was not a controlled experiment, in which
you give the rofecoxib randomly to some patients and not others in
order to see what happens. Rather, something about either these
patients or their doctors led some of them to be using rofecoxib and
others not. Dr. Graham and co-authors looked at a variety of indicators
that suggested that the rofecoxib patients already had slightly
elevated risk factors for coronary heart disease. Once they controlled
for these with a logistic regression, their study found an elevated
risk factor of heart attack for rofecoxib takers of 1.34, which was not
statistically significantly distinguishable from 1.0.The strongest evidence from this study was a claimed dose-effect
relation. Of these 68 rofecoxib-using heart-attack patients, 10 of them
were taking doses above 25 mg per day. Only 8 patients in the much
larger control group were taking so high a dose, implying an elevated
risk factor of 5 to 1 for high-dose patients. Again observable risk
factors could explain some of this, with the conditional logistic
regression analysis bringing the implied drug-induced risk down to 3 to
1. According to the study, this elevated risk factor was still
statistically significant, even though the inference is based on the
experience of just 10 patients.The obvious question here is whether in fact the authors were able
to observe all the relevant risk factors. The study openly acknowledged
that it did not, missing such important information as smoking and
family history of myocardial infarction.…[E]ven if
there actually is an elevated risk of the magnitude the studies suggest
but can’t prove, the question is whether I might want to accept a 1 in 4,000 risk of dying from a heart attack in order to get the only medication timt makes my pain bearable and a mobile life livable. And if I say no to the Vioxx, I may end up taking something that is less effective for my pain but has risks of its own.…. How did we arrive at a
system in which 12 random Texans are assigned responsibility for
evaluating the scientific merits of statistical evidence of this type,
weighing the costs and benefits, and potentially sending a productive blue-chip American company into bankruptcy protection?
See also my op-ed Bringing the Consumer Revolution to the FDA.
Not since Mark Twain
Jon Stewart destroyed Christopher Hitchens on Friday’s Daily Show. Hitchens tried to take the line that people against the war in Iraq were capitulationist, blame America-firsters. Here’s part of the transcript:
Stewart: The people who say we shouldn’t fight in Iraq
aren’t saying it’s our fault. That is the conflation that is the
most disturbing to me.
Hitch: Don’t you hear people saying that we made them nasty. . .
Stewart: I hear people saying a lot of stupid
[bleep]. . . But there is reasonable dissent in this country
about the way this war has been conducted, that has nothing to do with
people believing that we should cut and run from the terrorists, or that we
should show weakness in the face of terrorism, or that we believe that
we have in some way brought this upon ourselves. They believe that this war is being conducted without transparency, without credibility, and without competence…
Hitch: I’m sorry, sunshine. I just watched you ridicule the president for saying he wouldn’t give a timetable…
Stewart: No, you misunderstood why. . . .What I ridiculed the president [about] was [that] he refuses to answer questions from
adults as though we were adults and falls back upon platitudes and
phrases and talking points; that does a disservice to the goals that he
himself shares with the very people he himself needs to convince.
Hitchens knows he has been beat and can hardly wait to escape at the close (watch the video here).
The Virtues of Judicial Independence
I worry when conservatives rail against out-of-control judges. Who else but an independent judge can smack-down out-of-control bureaucrats and politicians like this?
Econometric Society 2005 World Congress
Mahalanobis covers some of the latest papers from the Econometric Society meetings. Here is a selection, more summaries at the previous link and much more here.
Economists
find money is more precious than time: They say that time is more precious
than money but economists have shown that this isn’t the case. Researchers
presented a paper that shows people are more willing to share the fruits of
their labour rather than their money.Legalising
abortion increases a woman’s economic power: The legalisation of abortion
and innovations in birth control have increased a wife’s clout in the home.
Results show that all women are better off, including those who don’t use birth
control, but only if it’s available to single women as well.Model
predicts how to get ahead in the research stakes: The conundrum over whether
to share interim research results or play your cards close to your chest has
been answered.Younger
siblings fare worse in educational attainment: Children born later in
families don’t perform as well as their older siblings.
No Prisoner’s Dilemma on the Western Front
Robert Axelrod’s story of how cooperation developed between British and German soldiers in the trench warfare of World War I is so elegant few people have questioned it. Yet in a single sentence, Andrew Gelman says the emperor has no clothes and looky, looky, he’s right!
The crux of Axelrod’s story is that the soldiers were trapped in a prisoner’s dilemma: individual incentives were to shoot the enemy while the socially optimal outcome was cooperation. Axelrod then introduces his famous ideas of tit for tat etc. etc. to explain how cooperation could evolve even under these most hostile of conditions.
But Gelman asks why should we think that shooting the enemy was in a soldier’s best interest? Indeed,
…it seems more reasonable to
suppose that, as a soldier in the trenches, you would do better to avoid firing: shooting
your weapon exposes yourself as a possible target, and the enemy soldiers might
very well shoot back at where your shot came from.
I believe that on this point Gelman is totally correct [insert dope slap here]. But, as he continues, "If you have no
short-term motivation to fire, then cooperation is completely natural and
requires no special explanation."
Axelrod’s story and the large literature following it sometimes suggest that cooperation is always the thing to be explained. Cooperation is what happens when the natural order is overcome. Gelman reminds us that sometimes cooperation is the norm, it’s conflict that needs to be explained. In this case, we need to explain why the soldiers fought.
Comments are open.
Modern Germs
Guns, Germs, and Steel emphasized the role that germs played in the clash of civilizations of the early modern period, say up to about 1700. I was surprised to learn from John M. Barry’s excellent book The Great Influenza that germs continued to have a disproprtionate influence on the civilizations well into the twentieth century and perhaps even today.
The great influenza of 1918 probably killed 100 million people, about five percent of the entire world’s population. An even higher percentage of young people died and most shockingly all of this occured in about 12 weeks. Death was not evenly distributed:
The Western world suffered the least, not because its medicine was so advanced but because urbanization had exposed its population to influenza viruses so immune systems were not naked to it. In the United States, roughly 0.65 percent of the total population died, with roughly double that percentage of young adults killed. Of developed countries, Italy suffered the worst, losing approximately 1 percent of its total population….
The virus simply ravaged the less developed world. In Mexico the most conservative estimate of the death toll was 2.3 percent of the entire population, and other reasonable estimates put the death toll over 4 percent. That means between 5 and 9 percent of all young adults died.
In even more remote areas the death toll was much higher. One doctor visiting Inuit in Alaska found everyone dead in 3 villages and 7 other villages with a death toll of 85%. We don’t know how many people died in India and China but the rates were certainly higher than in the more urban United States.
By the way, an enterprising researcher should be able to make use of the 1918 influenza in Mexico (and elsewhere as well) as a shock to population that likely had important reverbations throughout the economy and society for decades.
Addendum: Bill Johnson at UVA points me to, Is the 1918 Influenza Pandemic Over? (NBER), a very recent paper by Douglas Almond. From the abstract:
In the 1960-1980 Decennial U.S. Census data, cohorts in utero during the height of the Pandemic typically display reduced educational attainment, increased rates of physical disability, lower income, lower socioeconomic status, as well as accelerated adult mortality compared with other birth cohorts. In addition, persons born in states with more severe exposure to the Pandemic experienced worse outcomes than those born in states with less severe Pandemic exposures. These results demonstrate that investments aimed at improving fetal health can have substantial long-term effects on subsequent health and economic outcomes.
Now consider, if the effects in the United States are large then in Mexico, not to mention India and China (where data will be much harder to gather), the effects could have been devastating on a macro scale.
Bubbles and the real price of housing
Robert Shiller has put together the first, long, true index of home prices. By true I mean that as much as possible it looks at repeated sales of the same or very similar houses over time. Conventional indices confuse changes in size and quality with changes in the price of housing per se.
What the index shows is that real house prices have remained stable over the past 100 years. The contrary impression is driven by inflation and as noted above, changes in what is being measured. Stability, however, is what we should expect. The United States remains a relatively unpopulated country. When house prices in current population centers increase, suburbs and smaller cities expand. People move to less populated areas and in so doing alleviate the press on house prices. In the long run, the supply of housing is very elastic.
The glaring exception to stability is the last 6 or 7 years when house prices have skyrocketed far beyond where they have ever been before. Can you hear the pop coming?
Graphic from the NYTimes (click to open in new window).
Not putting their money where their mouths are
Inspired by Robin Hanson’s work on betting markets, James Annan, a climate scientist, has been trying to get skeptics of global warming to put up or shut up, mostly with no success on either front. A number of prominent skeptics refused to bet (perhaps having learnt from Paul Ehrlich’s embarassment) or offered to bet only at very high odds in their favor (i.e. implicitly admitting that they thought the probability of global warming was high). The failure to bet is telling and a nice reminder that even markets with no trades can tell you things of importance!
Finally, however, Annan has found some takers. From Nature (subs. required):
James Annan, who is based at the Japan Agency for Marine-Earth Science and Technology in Yokohama, has agreed a US$10,000 bet with Galina Mashnich and Vladimir Bashkirtsev, two solar physicists who argue that global temperatures are driven by changes in the Sun’s activity and will fall over the next decade. The bet, which both sides say they are willing to formalize in a legal document, came after other climate sceptics refused to wager money…
Both sides have agreed to compare the average global surface temperature between 1998 and 2003 with that between 2012 and 2017, as defined by the records of the US National Climatic Data Center. If the temperature drops, Annan will pay Mashnich and Bashkirtsev $10,000 in 2018, with the same sum going the other way if the temperature rises.
I hope that a TradeScience market like TradeSports can be established to make such bets more routine and even more informative.
Waste and the Value of Time
Concerning yesterday’s post, Beware Free Apples, a number of people wrote to me along the following lines, "some people have a low value of time, these people will be the ones who will be attracted to the giveaway so the time spent waiting in line is not as wasteful as you suggest." Surprisingly, this plausible analysis is not correct or at least seriously incomplete.
To see why suppose that the giveaway were held in a poor country. Would the waste be any less? No. Everyone in line would have a low value of time but for precisely this reason the waiting time would increase and the total waste would not change.
So long as there are more people with a low value of time than there are iBooks the waste will be complete. What does make a difference is diversity. If there are a few low value people and lots of high value people then the low value people can earn a rent. A direct analogy is to gold mines. If there are a lot of low cost gold mines then the price of gold is low and none earn a rent. If there are just a few low cost gold mines and many high cost gold mines then the price of gold will be high; the marginal mine will just break even and the low cost mines will earn a rent. To earn a rent there must be a scarcity – scarce land, scarce mines, or scarce low time-value people.
Suppose that we have a continuum of high to low-value types. We can say immediately that "The total price for the marginal consumer will tend to rise so that it equals the marginal value of the good." In other words, the marginal consumer will do only slightly better than if he were to buy the good at the market price (if he were to do much better then by continuity there is another consumer willing to outbid him by waiting in line a bit longer.) Thus on the margin dissipation is complete. What about the infra-marginal consumers?
The infra-marginal consumers will earn a rent but given some plausible assumptions about the distribution of types it’s surprising how little difference this makes to total dissipation. I did some very basic calculations in Mathematica assuming that the value of time is Normally distributed with mean $15 and sd $4. Under these assumptions the "first" person in line has a value of time of only $.84 per hour. Nevertheless, the total rent dissipation is 80 percent of what it would be if everyone had a value of time of $11.63, the value of time of the marginal consumer. Some brief experiments suggest that this sort of result if quite robust. Specifics will depend on the exact distribution assumed. Here is a pdf
and here is the Mathematica Notebook if anyone wants to generalize.
Beware Free Apples
You can get rid of the market but you can never get rid of competition. Goods not allocated by market prices have to be allocated somehow and so long as goods are scarce there will be competition to obtain them, if not by outbidding competing buyers with money then by outbidding them in time spent waiting in line, doing political favors or some other method.
What happened in Henrico county is the same type of thing that happens when there is a price control. The diagram below explains.
At the controlled price the quantity demanded exceeds the quantity supplied so buyers compete to obtain the good by, for example, arriving early and standing in line (or stomping on their competitors!). Waiting in line is costly so the total price rises above the money price by the time price. The total price for the marginal consumer will tend to rise so that it equals the marginal value of the good – only when the total price is equal to the marginal value (at the controlled quantity) is there no excess demand.
It’s very important to notice that that the shop owner gets your money but does not get your time. Thus, money expenditures are a transfer but time expenditures are a waste. Money expenditures = controlled price times*controlled quantity. Time expenditures = time price*controlled quantity so the shaded area indicates the waste.
It’s also important to notice that the total price is higher than the market price! A price control, therefore, doesn’t even necessarily reduce prices!
Evil Kelo
The Kelo v. New London eminent domain case started five years ago when New London condemned a number of buildings, including the lifelong home of one 87 year old resident. The residents took the case to court and, of course, lost. Now get this. The city is claiming that since the original seizure was legal the residents have been living on city property for five years and thus owe back rent.
Namesake Susette Kelo, who owns a single-family house with her husband,
learned she would owe in the ballpark of 57 grand. "I’d leave here
broke," says Kelo. "I wouldn’t have a home or any money to get one. I
could probably get a large-size refrigerator box and live under the
bridge."
Thanks to David Theroux for the pointer.
The Dragon Illusion
If you liked the chessboard you will also enjoy the Dragon Illusion. A video at the link shows you what the illusion looks like but it’s more fun to make a dragon for yourself using the PDF (provided at the link) and some scissors and tape. You will be amazed!
Thanks to Mitch Berkson for the link.