Do researchers suffer from winner’s curse?

OH NY GOD…READ THIS!!!!!!!!!!!!!!!!!

From The Economist:

In economic theory the winner’s curse refers to the idea that
someone who places the winning bid in an auction may have paid too
much…The same thing may be happening in scientific publishing, according
to a new analysis. With so many scientific papers chasing so few pages
in the most prestigious journals, the winners could be the ones most
likely to oversell themselves–to trumpet dramatic or important results
that later turn out to be false. This would produce a distorted picture
of scientific knowledge, with less dramatic (but more accurate) results
either relegated to obscure journals or left unpublished.

My colleague Omar Al-Ubaydli was part of this work.  Fortunately, it is mentioned on the very cover of the magazine.  Nonetheless I believe it is true and it is most true for "hot" fields.  The original paper is here.

Comments

This is a paradox in teaching medical research writing. We tell students to be accurate and cautious in their interpretation of data. But we then have a culture where those who "over-interpret" get the best press. Given that a few top papers can influence tenure and hiring decisions, I am concerned that incentives may be mis-aligned.

Comments for this post are closed

Papers that attack the foundation of the logic(or lack thereof) of the public giving a private monoply to the federal reserve bank also tends not to get hyped in the media too much.

Economist who attack the bailout also tend not to do well in companies that are owned in part by GE, Ruppert Murdoch or Warren Buffet among others.

Comments for this post are closed

Markets in everything?
http://www.jnrbm.com/

I propose the B.S. prize. Get a million bucks for calling B.S. on someone published in Science or Nature. $500k for JAMA and $2mil for a Nobel winner.

Comments for this post are closed

Can researchers on winner's curse suffer from winner's curse as well?

Comments for this post are closed

I thought the winner's curse says a person who wins an auction paid the most, therefore, they feel like they lost because they paid more than the next highest bidder, IE they got cheated out of a small amount of money.

Then logically, the winner's curse applied to papers would be that published papers present more information than was necessary to get published, IE they could have published 2 papers instead of the one.

Therefore, I refuse to read the paper because I disagree with the conclusion from the snippet Tyler posted... Why does the extra information have to be incorrect? Doesn't the Journal check the validity of the information before it is published? (And I'm not talking about the economist.)

Comments for this post are closed

http://www.boston.com/news/world/europe/articles/2008/10/10/betting_makes_nobel_literature_jury_suspect_leak/ <-- Check it out. Markets in Le Clezio.

Comments for this post are closed

Who Says Capitalism Is Dying?
Some good news for the weekend:

http://www.nytimes.com/2008/10/11/world/asia/11china.html?hp

"BEIJING — Chinese leaders are expected to allow peasants to buy or sell land-use rights for the first time, a step that could draw hundreds of millions of farmers more firmly into the city-centered market economy.
The new policy, which is being discussed this weekend by Communist Party leaders and could be announced within days, would be the biggest economic reform in many years and would mark another significant departure from the system of collective ownership and state control that China built after the 1949 revolution."

Read on.

Who says capitalism is dying?

Comments for this post are closed

@Andrew at Oct 10, 2008 5:47:34 PM

>I propose the B.S. prize. Get a million bucks for calling B.S. on someone published in Science or Nature. $500k for JAMA and $2mil for a Nobel winner.

We now know who controls the $700 billion and what he plans to do with it ... :)

Comments for this post are closed

One way to get around this is to be a Bayesian and adopt informed priors on issues. That way, instead of looking for a "p value of such and such" you start demanding extraordinary evidence for extraordinary claims.

Comments for this post are closed

"We now know who controls the $700 billion and what he plans to do with it ... :)"

Don't I wish. It would be used for just about the opposite of its current destiny. On that note, it seems like there ought to be some kind of do-over on the election now that the government is going to have vastly different powers. I mean, why elect a socialist or a warrior when you've already enacted socialism and there's no way in hell you'll have any money to fight wars.

Brain,

"Then logically, the winner's curse applied to papers would be that published papers present more information than was necessary to get published,"

Yes, and then the point is that the extra information is NOT TRUE, or at least sketchy, and if everyone agreed to not stretch, then the playing field would be even.

It's kind of like doping in sports where doping really helps.

So, I suppose the direct analogy to winner's curse as explained would be that auction winner's beat the next highest bidder by an amount paid in counterfeit money.

Comments for this post are closed

To clarify, in this case it is not the "winners" in the sense of the researchers who are cursed or suffer. It is knowledge or academia or the journals that do from "buying" faulty research.

I think it was Emily Ostrom, not Oster, whom Tyler praised, and Ostrom is certainly admirable, not a case of a winner's curse in regards to her research.

Econ Journal Watch may be the nascent economics journal of negative results.

Comments for this post are closed

Barkley - there are 7 past MR posts about Emily Oster, and zero about Elinor Ostrom (she has been mentioned in the comments). I tried to copy and paste the 7 posts that mention Oster but the TypePad spam filter thinks it's spam.

Comments for this post are closed

I propose an alternative (and more accurate) title for the article on the winner's curse. The title should be: "Why this Article is a Good Example of Why Current Publication Practices May Distort Sicence".

Comments for this post are closed

I'm trying to figure out who the NY GOD!!!!! is. Could you mean Brett Favre? Dave Letterman? Martin Scorcese? Robert De Niro?

But why would they want to read it?

With that thought in mind, perhaps you are referring to Taleb? Or Easterly? Or Mundell?

Oh wait, you are imitating Higgins from Magnum PI!!! How silly of me.

Comments for this post are closed

Brainwashed and Andrew are confusing Winner's Curse with Buyer's Remorse.

Buyer's Remorse is indeed the phenomenon where a buyer realises ex post they could have got the goods for only a slight margin more than the second bidder - the seller has extracted a rent. But it doesn't mean they can't make a profit from the purchase.

Winner's Curse arises where the value of the good is uncertain and different bidders have different ex ante estimates of its value. The winner will be the most optimistic valuer, who by definition will be the one most likely to overestimate the true value. The publishing bias discussed here is that of Winner's Curse, not Buyer's Remorse.

Comments for this post are closed

There's a nice paper on this topic from 1992 by DeLong and Lang.

http://www.j-bradford-delong.net/movable_type/archives/000620.html

This also provides a nice example of another huge problem in academia: nobody has any idea what's going on outside each person's specialty, so there's massive duplication of effort.

Comments for this post are closed

all the things have two sides

Comments for this post are closed

Is it realistic?

Comments for this post are closed

More are waiting a very nice article thank you for the love

Comments for this post are closed

Great tips, I would like to join your blog anyway,

Comments for this post are closed

Comments for this post are closed