Piketty update

by on May 23, 2014 at 2:16 pm in Books, Data Source, Economics, History | Permalink

…according to a Financial Times investigation, the rock-star French economist appears to have got his sums wrong.

The data underpinning Professor Piketty’s 577-page tome, which has dominated best-seller lists in recent weeks, contain a series of errors that skew his findings. The FT found mistakes and unexplained entries in his spreadsheets, similar to those which last year undermined the work on public debt and growth of Carmen Reinhart and Kenneth Rogoff.

The central theme of Prof Piketty’s work is that wealth inequalities are heading back up to levels last seen before the first world war. The investigation undercuts this claim, indicating there is little evidence in Prof Piketty’s original sources to bear out the thesis that an increasing share of total wealth is held by the richest few.

Prof Piketty, 43, provides detailed sourcing for his estimates of wealth inequality in Europe and the US over the past 200 years. In his spreadsheets, however, there are transcription errors from the original sources and incorrect formulas. It also appears that some of the data are cherry-picked or constructed without an original source.

For example, once the FT cleaned up and simplified the data, the European numbers do not show any tendency towards rising wealth inequality after 1970. An independent specialist in measuring inequality shared the FT’s concerns.

The full FT story is here.

Addendum: Here is the in-depth discussion.  Here is Piketty’s response.

C May 23, 2014 at 2:20 pm

Why don’t they publish the errors that they found in their own investigation?

C May 23, 2014 at 2:21 pm
C May 23, 2014 at 2:27 pm

And Picketty’s response in full, here:

http://blogs.ft.com/money-supply/2014/05/23/piketty-response-to-ft-data-concerns/

Interesting editorializing by the FT.

tt May 23, 2014 at 2:49 pm

anyway to view that without ‘registering’

Michael May 23, 2014 at 2:55 pm

It’s really sad that many not-very-bright people will be fooled by Giles’s misleading and tangential nitpicking, which (as Piketty clearly points out) does not actually change the thrust of Piketty’s thesis.

It’s a sad attempt by a Murdoch-owned rag to undercut a very good thesis that undermines Murdoch’s political view. Note that the FT’s treatment of the R-R scandal was much, much more charitable:

“But the corrections do not affect Mr Rogoff and Ms Reinhart’s most up-to-date work, which still shows a slowdown in growth when debt hits 90 per cent of gross domestic product.” http://www.ft.com/cms/s/0/433778c4-b7e8-11e2-9f1a-00144feabdc0.html#ixzz32ZBOhLyn

I’m sure it will fool a lot of Cowen’s fans, and readers of other Koch-supported blogs, but fortunately the clear trend of growing inequality is making it harder and harder for this kind of distracting propaganda to really make a meaningful impact on the hearts and minds of the majority.

C May 23, 2014 at 3:02 pm

Hint: The reason they don’t post specific criticisms in the main article is because those criticisms seem to be a little weak. By simply reporting that criticisms were made, they can create more of a splash.

QWERTY May 23, 2014 at 3:10 pm

It is those evil rightwing people again. They are all over the place these days. We cant trust a word from them, and yet we cant do anything about them being so powerfull.

Koch-brothers, Murdoch and ………….

Urso May 23, 2014 at 3:19 pm

Yes, amazing how those Koch brothers have kept Piketty completely out of the news for the past two months. Why, you can’t open a newspaper or turn on the TV without not hearing about Piketty.

Michael Fisk May 23, 2014 at 3:44 pm

Well, Piketty’s main counterargument is that there are data sets that reinforce his original findings that have been “extended” and “improved” by Emmanuel Saez – a frequent collaborator of Piketty’s (so frequent, in fact, that it’s hard to distinguish the professional careers of the two).

That’s the equivalent of a burglar being on trial, testifying in his own defense that his accomplice has an ironclad alibi for him.

Andrew May 23, 2014 at 3:50 pm

FT is owned by Pearson, not News Corp.

Z May 23, 2014 at 5:04 pm

Look out! There’s a Koch brother under your bed!

andrew' May 23, 2014 at 7:45 pm

Anyone else here think we get fooled by anything?

Eric Falkenstein May 23, 2014 at 7:49 pm

MR is Koch funded!? Too simple. It’s the Freemasons (get it, George *Mason*).

Albert May 23, 2014 at 7:57 pm

With a name like Koch, I thought for sure the Teutonic Knights were bankrolling them….

Mark Thorson May 23, 2014 at 9:16 pm

That’s just a cover story. They’re actually reptilian-human hybrid shapeshifters. Go to Alex Jones’ website if you want to know the truth.

Rahul May 23, 2014 at 11:07 pm

“Look out! There’s a Koch brother under your bed!”

Koch Roach?

Bill Reeves May 25, 2014 at 11:32 am

Don’t you understand that the moment you use the fright word “Koch” that you destroy the credibility of your argument just as surely as if you said “fairies at the bottom of my garden did it all”?

marris May 23, 2014 at 5:16 pm

Wait, what? Picketty’s response does not seem to actually address any of the problems identified in the numbers. For example, is there actually a transcription error or an adjustment? Why is arithmetic mean the right choice rather than weighted average?

It’s funny that Picketty wants a wealth tax, even at very low rates, in order to get better data. Somehow, I don’t see low rates being a stable equilibrium.

JJ May 23, 2014 at 5:42 pm

Piketty had that statement prepared, has to be.

dearieme May 23, 2014 at 2:35 pm

“The problems seem most acute for Britain”: what do you expect, the bloke is French.

“The US is tricky as the source data is even sketchier than that for the three European countries included in the study.” That just proves that America is best because exceptionalism.

dearieme May 23, 2014 at 2:23 pm

“Prof Piketty, 43, provides detailed sourcing for his estimates of wealth inequality in Europe and the US over the past 200 years.” Well done, that man!

“In his spreadsheets, however, there are transcription errors from the original sources and incorrect formulas.” To err is human.

“It also appears that some of the data are cherry-picked or constructed without an original source.” He will find a new career in Climate Science.

cheesetrader May 23, 2014 at 5:08 pm

That’s delightfully harsh – two birds, one insinuation stone

Larry Siegel May 23, 2014 at 6:16 pm

That was actually very funny. No idea if it’s an accurately thrown stone.

chuck martel May 23, 2014 at 2:27 pm

Oh. Never mind.

cfh May 23, 2014 at 2:42 pm

M. Piketty need not worry; in economic data, volume trumps accuracy.

JWatts May 23, 2014 at 2:43 pm

“similar to those which last year undermined the work on public debt and growth of Carmen Reinhart and Kenneth Rogoff.”

I expect a lot of people on both sides of the debate will be stymied in their response of the Piketty data issue due to their previous statements on the Reinhart/Rogoff kerfuffle.

JWatts May 23, 2014 at 3:44 pm

After reading some of the comments here, I’m going to apologize for my prior post and amend it to the following:

I expect a lot of thoughtful people on both sides of the debate will be stymied in their response of the Piketty data issue due to their previous statements on the Reinhart/Rogoff kerfuffle.

derek May 23, 2014 at 6:26 pm

I think that Piketty is right that capitalism and free markets engender inequality. No matter how his spelling and addition skills.
I think that national debt is ok up to a point, and can become a problem. No matter what Reinhard and Rogoff had to say, and their spreadsheet skills.
I think that under certain circumstances deficits are ok, I also think that not spending more than can be taxed is a good thing.

I also think there are genetic differences between populations that show up in certain characteristics. There are genetic diseases that are specific to populations, so why not others. I also think that there are cultural differences; Germans are different than the French who are different from the Greeks, etc.

I also think there are substantial differences between men and women, and they are not for the most part substitutable.

The variations, the amounts, the importance of all of the above can be debated.

There is one thing that I am absolutely certain of. Any policy that intends to correct for or adjust any of the above will make a total hash of it, and will either kill, maim or socially dismember those who they are intending to assist. The 20th century is my proof.

The First Guy That Called Himself Michael May 24, 2014 at 1:11 am

Fundamentally, wasn’t Reinhart/Rogoff a single mistake that missed a couple rows (and didn’t change the answer much), plus a defensible argument about weighting of various countries? A cursory reading of the detailed FT article seems to show quite a few errors.

A number of Pikettys errors seem systematic– especially consistently using the wrong years when the correct year is available in the original data. The adding of random numbers to years seems too obvious, I’m sure he has some reasoning behind most of those, it’ll be interesting to hear detailed defenses come out over the coming days.

T. Shaw May 23, 2014 at 2:44 pm

Pinketty is simply this year’s purveyor of pretenses for legal plunder (higher taxes and regulations) and decreased liberty.

dearieme May 23, 2014 at 2:44 pm

“Professor Paul Krugman of Princeton University, said it was safe to say the book “will be the most important economics book of the year – and maybe of the decade”.
Professor Joseph Stiglitz of Columbia University said Prof Piketty’s “fundamental contribution” was the provision of data on the distribution of wealth.
In Britain, Ed Miliband, Labour leader, told the Evening Standard: “I’m in the early stages of the book. In a way, he is symptomatic of what people are actually feeling”.”

Fascinating: the economists talk tripe while a ruddy politician gets to the nub of the matter – it’s about feeling.

Thor May 23, 2014 at 3:51 pm

Brilliant catch. That’s comedy gold.

Piketty: He feels your pain.

And in the case of Miliband, HE feels your pain before he knows what kind of pain it is that he should be feeling.

So Much For Subtlety May 23, 2014 at 7:12 pm

It is not fair to say the economists are talking tripe. Krugman says the book agrees with him and everything he has been saying. Stiglitz clearly thinks the conclusion is crap – saying a book has nice data is like saying a girl has a nice personality.

While Miliband recognizes that everyone is talking about it at smart dinner parties in Islington these days so he hopes to ride the wave. He sees a Hampstead Mob and hopes to be its head.

Paul Zrimsek May 23, 2014 at 8:05 pm

Despite the kiss of death from Paul “Enron is more important than 9/11″ Krugman, I think it quite likely that Piketty’s will be the most important economics book of the year. Not that that’s the highest bar anyone’s ever set.

Just Another MR Blogger May 23, 2014 at 2:49 pm

Maybe he should’ve learned how to use Excel from Reinhart and Rogoff.

Cliff May 23, 2014 at 11:30 pm

You mean, maybe he did?

Please May 23, 2014 at 2:54 pm

Non-paywalled links?

improbable May 23, 2014 at 5:22 pm

Sadly lmgtfy.com doesn’t work… but google “Data problems with Capital in the 21st Century” and
“Piketty response to FT data concerns”

Rostovin Bake-off May 23, 2014 at 5:33 pm

QZ has a lot of the details on their story on it. They also include graphs showing the data corrected from various sources:

http://qz.com/213081/are-there-major-mistakes-in-the-bombshell-economics-book-of-the-year/

The First Guy That Called Himself Michael May 24, 2014 at 1:16 am

If you register, you can read it for free. FT allows something like 5 free articles per month.

DK May 24, 2014 at 10:59 am
Michael Foody May 23, 2014 at 2:57 pm

The tone of condemnation seems way out of line given the scope of the problems. Piketty also frequently makes pains to describe the considerable uncertainty he must grapple with. I have no issues or tools for evaluating the criticisms themselves but the editorial tone is really strange and more than a little troubling given only the damning summary is open to the public and the details and rebuttal are paywalled.

Just Another MR Blogger 2 May 23, 2014 at 3:05 pm

“I have no issues or tools for evaluating the criticisms themselves but the editorial tone is really strange and more than a little troubling given only the damning summary is open to the public and the details and rebuttal are paywalled” Exactly. Public criticism, paywalled details. Which is not to say that they are wrong, just that they are sketchy.

Also, could MR turn off the “- See more at: http://marginalrevolution.com/marginalrevolution/2014/05/piketty-update.html#comments” thing that gets added to every cut and paste from the site. It’s extremely annoying.

dan1111 May 24, 2014 at 3:48 am

Also, could MR turn off the “- See more at: http://marginalrevolution.com/marginalrevolution/2014/05/piketty-update.html#comments” thing that gets added to every cut and paste from the site. It’s extremely annoying. – See more at: http://marginalrevolution.com/marginalrevolution/2014/05/piketty-update.html#comments

I second that!

Ray Lopez May 24, 2014 at 2:37 am

The NY Times today (after this post by TC) has an economist who opines along the same lines: that the errors pointed out by FT are rather small and also subject to interpretation, except the ones that are obvious, like not taking a weighted average by country population in Europe but just summing large and small countries, which, BTW, Rogoff et al in their book “This Time Is Different” did the same thing for financial panics, and explicitly said that (paraphrasing) ‘it’s OK to take a numeric average rather than weighed average by population, since both large and small countries responses to financial panics are the same, and there should *not* be a correction by population, since small populations are as representative as large populations’.

In short, the errors in Piketty are not that glaring, contrary to FT’s oped. Further, the graphs as shown by Quartz newssite (see the link in these comments) show clearly that inequality is real, but has been falling in most countries, seemingly contrary somewhat to Piketty’s thesis, except for the fact that Piketty is saying we will *return* to these “bad old days of inequality” since the world is past the golden era of growth after major wars that reduced inequality temporarily.

QWERTY May 23, 2014 at 3:08 pm

” the editorial tone is really strange and more than a little troubling given only the damning summary is open to the public and the details and rebuttal are paywalled”

OMG – we have to pay to read a newspaper. I thought journalist were working for free.
Clearly thats very “troubling” and it becomes fair to say that it is not “open to the public”

Hmm, can someone tell me, were I can go a get a copy of Piketty’s book for free?
Otherwise, in all fairness, I think we should call the book more than a little troubling, since the public has no access to it.

Michael Foody May 23, 2014 at 3:19 pm

Have you read the details? The criticisms seem pretty niggling as far as consequences go, but the ungated article treats them as decisive. This is probably just the ordinary sleaze of “Find out if your water is killing you after the commercial break” before seeing an actual story of mild consequence. However on the web the ungated version will reach a wider audience so the effect is a lot like swiftboating.

I apologize for taking you seriously if it wasn’t merited.

Troll? May 23, 2014 at 3:22 pm

“I apologize for taking you seriously if it wasn’t merited,” haha. On MR, you can never tell.

C May 23, 2014 at 3:25 pm

Don’t bother, he’s not interested in the argument, the criticism, or the rebuttal. He also doesn’t know about public libraries, which is not exactly shocking.

Steve Sailer May 23, 2014 at 3:29 pm

Remember when Steven Levitt’s most famous theory was shown to be the result of his coding errors?

“Freakonomics’ Abortion Research Is Faulted by a Pair of Economists”

http://online.wsj.com/news/articles/SB113314261192407815

Jeff May 23, 2014 at 3:31 pm

False Alarm. Not so subtle Piketty-trolling is MR pastime. It’s almost 5:00 guys, hang in there.

http://blogs.ft.com/money-supply/2014/05/23/piketty-response-to-ft-data-concerns/

yang May 23, 2014 at 3:33 pm

It’s just data-fudging turtles all the way down.

rayward May 23, 2014 at 3:46 pm

As I often say, inequality is a good thing except when it’s a bad thing; and when it’s a bad thing, it’s a very bad thing. No, the mob will not be descending upon the Hamptons, pitchforks in hand. The bad thing is more basic, more alarming, and more widely felt. I appreciate the academic interest in Piketty’s research, but I’m more interested in the practical consequences of excessive inequality, which are observable even by academics who would prefer not to see.

Don Kenner May 23, 2014 at 3:47 pm

Thank the gods for income inequality. I’d hate to live anywhere where it wasn’t pervasive.

Unless there’s a finite amount of wealth in the world. Then it might be bad.

But no one’s stupid enough to believe that, right?

This just in: Piketty now makes a lot more money than most slimy, left-wing, fabricators.

Zach May 23, 2014 at 3:57 pm

Is it just me, or is Piketty’s response non-responsive?

The correct response to “your data handling is sloppy and affects your conclusions” is “my data handling is just fine and my conclusions are sound.” Piketty accepts that the FT numbers are correct.

Citing the Berkeley research is a non-sequitur. Agreeing with a result which is wrong because of fat-fingered data entry is not real agreement. The Berkeley paper should be compared to the FT graph, not the graph in Piketty’s book.

Arguing that he didn’t include offshore wealth is also a non-sequitur. Arguing that you could measure things differently to get the result you want is the reverse of the correct procedure, which is to find the best measurement and apply it without fear or favoritism. The procedure in the book is the one he decided upon, and he shouldn’t abandon it lightly.

marris May 23, 2014 at 5:02 pm

+1

dwb May 23, 2014 at 4:01 pm

“I would be very surprised if any of the substantive conclusion about the long-run evolution of wealth distributions was much affected by these improvements…”

Because that’s the conclusion I wanted in the first place!

Thor May 23, 2014 at 4:04 pm

My worries about Piketty started when I saw the title of his earlier book of essays (which is: Vive la gauche américaine!). Yes, complete with exclamation mark.

It’s not that he’d necessarily fabricate data but I suspected that his solution would be to advocate class war. Or the upshot of his popularity would be a spur to another culture war. But then, I worry about inequality for pessimistic, Hobbesian reasons.

GiT May 23, 2014 at 9:35 pm

Your worry is that a Frenchman praising the *American* left is going to “advocate class war?” You probably ought to check the relative extremity of the left parties in France as opposed to the United States.

Thor May 24, 2014 at 1:03 am

No. I am too afraid of what I might see.

Pierre May 24, 2014 at 3:50 am

Thor, when a French Socialist says “vive la gauche americaine”, it means he wants more markets and less socialism.

TMC May 24, 2014 at 8:37 am

This makes more sense, Pierre. To us, the gauche american is the socialist.

Zach May 23, 2014 at 4:05 pm

It seems to me that there’s a good argument for moving away from spreadsheets for serious data analysis here. From the extended discussion, it seems that Piketty was promiscously mixing the data he was taking from his sources with the adjustments he needed to make to create the desired graph. A much better procedure is to have one file which contains only the original data and a separate file which contains the properly commented operations you perform upon that data.

Bill May 23, 2014 at 4:11 pm

Today we should be able to download and hash out data. Am all for criticism and a right to rebut.

Bill May 23, 2014 at 4:13 pm

By the way, here is an ungated response by Piketty:
http://blogs.ft.com/money-supply/2014/05/23/piketty-response-to-ft-data-concerns/

Bill May 23, 2014 at 4:17 pm

here is an excerpt:
“For instance, my US series have already been extended and improved by an important new research paper by Emmanuel Saez (Berkeley) and Gabriel Zucman (LSE). This work was done after my book was written, so unfortunately I could not use it for my book. Saez and Zucman use much more systematic data than I used in my book, especially for the recent period. Also their series are constructed using a completely different data source and methodology (namely, the capitalisation method using capital income flows and income statements by asset class). The main results are available here: http://gabriel-zucman.eu/files/SaezZucman2014Slides.pdf.

As you can see by yourself, their results confirm and reinforce my own findings: the rise in top wealth shares in the US in recent decades has been even larger than what I show in my book.

In the attached graph, I compare their series with the approximate series that I provide in the book. As you can see by yourself, the general historical profiles are very similar. This is exactly what I expect as we collect more data in other countries as well: we will certainly improve upon my series and adjustments (some of which can certainly be discussed), but I don’t think this will have much of an impact on the general findings.

(see also this paper pp. 91-92 of pdf: http://gabriel-zucman.eu/files/PikettyZucman2014HID.pdf)

Finally, let me say that my estimates on wealth concentration do not fully take into account offshore wealth, and are likely to err on the low side. I am certainly not trying to make the picture look darker than it it.”

Zach May 23, 2014 at 4:40 pm

The Berkeley paper agrees with Piketty’s conclusions. But the graph in the book is not the correct output of Piketty’s procedures, because of data entry problems.

Piketty needs to stand by his procedures more firmly than he does here. If the procedure he advocated really does produce the FT graph, he needs to defend his result, not just swap in the Berkeley paper because it’s more convenient for the point he would really like to make.

Nick May 23, 2014 at 4:21 pm

A spreadsheet is a poor tool for both storing and analyzing data and in this day and age one should question the competence of any researcher who uses one for their final published analysis. It is like a contractor who wants to build you a house using only chalk and a t-square. Surely it is useful for some preliminary drafting but once it is time to start building, I expect them to get out the power tools. If the guy isn’t skilled enough to use the proper tools for the job then he should take up another profession. Errors like this and R&R are an embarrassment to the field.

XVO May 23, 2014 at 4:32 pm

relational database ftw?

Steve Sailer May 23, 2014 at 4:35 pm

Spreadsheets definitely have their weaknesses, but it’s worth noting that Steven Levitt massively screwed up his abortion-crime research using statistical coding software rather than a spreadsheet.

Art Deco May 23, 2014 at 5:53 pm

I’m confused. What was he using? I think SAS would have been the preferred application for regression analysis then.

Steve Sailer May 23, 2014 at 6:30 pm

I don’t what brand Levitt was using, but he made two coding errors:

http://www.economist.com/node/5246700

You can see the results of his mistakes in our 1999 debate in Slate where I pointed out that Levitt’s claims don’t make sense at the national level, and he responded, well, my program is at the state level, so there.

Of course, it turned out six years later that Levitt had screwed up his state level program, which is why he couldn’t reconcile them to the national trends.

Steve Sailer May 23, 2014 at 6:38 pm

Here’s the 1999 Slate debate between Levitt and me:

http://isteve.blogspot.com/search/label/Does%20Abortion%20Prevent%20Crime%3F

scott cunningham May 23, 2014 at 7:41 pm

Levitt was using Stata. I think his error was that he reported a table claiming that he had run state-by-year interacted fixed effects and he in fact had not.

But, fwiw, Piketty and Levitt have something else in common that is worth screaming at the top of your lungs — they both provided their files to anyone who asked for them. Levitt’s actually downright unusual among economists for always posting his do files and data (or usually when he can). That’s how people found the errors.

When you have something to hide, you don’t share the files and the do files — not in economics, anyway, where no one is obligated or expected to do so.

Larry Siegel May 24, 2014 at 4:10 am

Interesting that you brought up this old controversy since I’d been thinking about it. I don’t need data, or accurate econometric techniques, to observe that some children who were aborted would have grown up to commit crimes, so *those* crimes were prevented by abortion. (This is different from saying that crime, overall, was lowered by abortion; I can’t come to that conclusion from logic alone.) Now, I realize there are many other moving parts but what is wrong with my statement?

Art Deco May 24, 2014 at 8:36 am

But, fwiw, Piketty and Levitt have something else in common that is worth screaming at the top of your lungs — they both provided their files to anyone who asked for them. Levitt’s actually downright unusual among economists for always posting his do files and data (or usually when he can). That’s how people found the errors.

Fair enough. That having been said, it seems to me (have done a wee bit of this sort of work int he past) that his supposed error was indicative of abnormal sloppiness. Given that he was repeatedly challenged in various fora and given the eugenical nature of the hypothesis he was testing would induce most of us to anticipate challenges, you’d have thought he’d be extra careful. Then again, maybe he thought the hypothesis was a spoonful of sugar; academic subcultures are odd.

David R. Henderson May 24, 2014 at 10:22 am

@Steve Sailer,
Interesting debate. One of the points Levitt made in his second article was that if he’s right, murder rates should continue to decline. I have no dog in this hunt: I thought John Lott’s criticisms of Levitt were good but I’m open to being told otherwise. But how did it work out? Was Levitt right that murder rates continued to decline? I haven’t followed the data closely.

Steve Sailer May 24, 2014 at 12:53 pm

The empirical problem is that the cohort born in the half-decade after legalization abortion had teen homicide rates several times higher than the cohort born in the half decade before legalization.

The main reason was the Crack Wars, of course. More subtly, legal abortion may have reduced the number of middle class and working class children born, while largely leaving untouched the number of underclass children being born. That would be a good topic for scholarly research, but there doesn’t seem to be much interest in it.

Dan Lavatan May 23, 2014 at 5:01 pm

It is possible to falsify, cherry-pick, or mis-enter data using any tool and customized or hand-coded tools might make it easier to conceal manipulations. Using a spreadsheet which is widely available makes it easier to check and re-use the work. Used properly, spreadsheets will return accurate results.

Curt F. May 23, 2014 at 5:32 pm

Even as a big Excel fan, I have to say I completely disagree. The most popular spreadsheet software (Microsoft) is not free and cannot be easily (accurately) translated to free formats. Checking formulas in Excel means you have to check every cell. In every sheet.

In contrast, relational database tools like MySQL are free, as are statistical or computing tools like Python or R. And checking formulas in those databases means you only have to read the code once.

improbable May 23, 2014 at 6:10 pm

I’m especially horrified by the practice of including adjustments as hard-coded numbers in a formula that is only visible when you put the cursor there! And adding 26% mysteriously, in a table that you would otherwise assume was input data, not calculation!

I’m not an excel fan, but this doesn’t seem like a tool problem. At best it reeks of just not actually giving a damn about treating the data honestly.

Also the idea of casually re-using the data from 1980 because you didn’t have any for 1970. Or adjusting 1935 (or 1938!) to be 1930 because you’re too lazy to just plot the points where they actually should be. Did he sleep through the “advanced plotting in excel” course?

prior_approval May 24, 2014 at 4:57 am

‘At best it reeks of just not actually giving a damn about treating the data honestly.’

If only that data was available publicly, so that others could check it.

scott cunningham May 23, 2014 at 7:47 pm

I agree. Also, the problem really in my mind isn’t Excel vs. R vs. STATA, but I do think there is a problem. If you read Shapiro and Gentzkow’s practioner’s guide for economists doing empirical work (here), you sense that they are trying to nudge themselves, their students and anyone they can towards a work flow that can minimize the error rate in our empirical work. Software doesn’t itself minimize the error rate. Economists could learn a lot from professional programmers and computer scientists who have developed practices that carry over from language to language designed explicitly to minimize the errors in our work.

But, that said, I actually see Piketty as doing some of that already because he made the files available on his website. That’s one things economists should try to do as much as possible.

Zach May 23, 2014 at 5:52 pm

The idea of segregating the data from the operations on the data isn’t to prevent fraud, it’s to make it easier to verify that the result you get is the result you wanted. From the in-depth discussion:

The first example relates to French inequality between 1810 and 1960. The original source reports data relative to the distribution of wealth among the dead. In order to obtain the distribution of wealth across the living, Prof Piketty augments the share of the top 10 per cent of the dead by 1 per cent and the wealth share of the top 1 per cent by 5 per cent (this is shown in the screen grab below). An adjustment of this sort is standard practice in this type of calculations to correct for the fact that those who die are not representative of the living population.

Prof. Piketty does not explain why the adjustment is usually constant. But in one year, 1910, it is not constant and the adjustment scale rises to 2 per cent and 8 per cent respectively. There is no explanation.

In an ideal world, you would not have to refer to a technical appendix or a lab notebook, because the motivation for making this adjustment would be explained in a comment, right next to the code that makes the adjustment. That also makes it easier to write the technical appendix. When I write code, I will frequenty include journal references in the comments to explain a particularly tricky step.

The format of spreadsheets pushes you toward using inexpressive variable names and omitting comments. Math being math, it’s possible to do things correctly using any tool. But that shouldn’t blind you to obvious failure modes of popular tools.

Steve Sailer May 23, 2014 at 6:33 pm

Right. I built a lot of business tools in Lotus 1-2-3 and Excel between 1984 and 1993. They worked great for me but they were hard to pass on to others to maintain.

Steve Sailer May 23, 2014 at 6:34 pm

One unkind theory for Nate Silver’s frequent career changes is that his Excel spreadsheets tend to become unmanageable. The guys who took over for him after he left Baseball Prospectus wound up rebuilding his main model from the ground up rather than try to keep his Rube Goldberg spreadsheet going.

scott cunningham May 23, 2014 at 7:51 pm

You’re describing nearly every social scientist, though, with that. That’s why all these books and articles about work flow in programming start off with that very anecdote. Heck, when I was dissertating, my practices were so bad that if ever I had to go back and redo some part of the analysis, it was like walking into a hoarder’s house. Hundred files, everything in one directory. A lot of social scientists have idiosyncratic work flows that look just like their desks — either incredibly neat and organized or total mayhem.

improbable May 23, 2014 at 6:35 pm

Right, this is a terrible way to work even if your only concern is to avoid fooling yourself and sloppy mistakes.

It’s also telling. What’s the chance that he checked how strongly his final graphs depended on these various fudge factors like “augments the share of the top 10 per cent of the dead by 1 per cent”?

jdm May 23, 2014 at 10:23 pm

If you had told me that Piketty wrote his papers with quill and ink pen and performed his calculations with an abacus, I could scarcely be more shocked than to learn that he (or any other serious researcher) relies on a spreadsheet to do their analysis. It’s positively medieval.

One of the many nice tools available in the R environment is a program call Sweave. Sweave allows you to embed in your Latex file the R commands that you use to read, clean, and analyze your data. The R code which produces every table and every plot is embedded in the Sweave/Latex document. Given the data and the Sweave/Latex file, anyone can reproduce the paper. You can go through the R code line by line, running each line yourself and examining the output, and make whatever modifications and checks you like. Other than allowing others to see exactly how you obtained your results, and allowing them to easily do their own analysis, one other nice thing about this way of writing papers is that you can produce the paper and results yourself “on demand” as it were. If you come back to a subject after some lapse of time you don’t have to waste time figuring out how you got some results – all the code to do the analysis is embedded in the paper itself.

dan1111 May 24, 2014 at 4:29 am

This is not shocking to anyone familiar with non-technical research fields, unfortunately. Primitive tools, poor methods, and the failure to do even the most rudimentary testing abound.

Noumenon72 May 23, 2014 at 5:34 pm

Specifically, when somebody criticizes his formula = 2 + J11 * (F$12/F$11), how is he or anyone else supposed to know where that 2 came from? A programming language would give you variable names: IRS_OVERREPORTING_PERCENTAGE_1970 + wealth_share_1percent * average_wealth_share.

Eric Rasmusen May 23, 2014 at 8:08 pm

These are useful comments. Someone should write on this for J. of Economic Perspectives. Scott Cunningham, are you listening? My first reaction was that Excel was good, because it’s very simple and everyone has it. I’m mainly a theoretical economist, but it may well be that empiricists don’t know about these things, and would be grateful to hear about them.

Scott Cunningham May 23, 2014 at 8:33 pm

I’m listening. I have been reading the Shapiro and Gentzkow practitioner guide that Tyler or Alex posted a few months back and it seems to me written in the same vein as these comments. It’s longer than a JEP, but it’s written with the same expressed purpose of what I think you’re suggesting.

faculty.chicagobooth.edu/jesse.shapiro/research/CodeAndData.pdf

Noumenon72 May 24, 2014 at 12:09 am

That may be the nicest reply I’ve gotten on this site, ever.

Excel kind of runs the world now, in banks and business too, but I bet when this new generation where everyone knows how to actually program grows up, they will look at Excel like it was doing math by dropping stones in cups.

Steve Sailer May 24, 2014 at 1:34 am

One question is whether Excel is simply a bad implementation of the spreadsheet concept. I wrote a firm’s sales forecasting system in Lotus 123 3.0 — the 3-D version — in 1991, then wrote a competitor’s sales forecasting system in Excel in 1993. It took me 3 times as long to do it in Excel as in Lotus. Like most Office products, Excel doesn’t seem to have improved much since then, either.

Larry Siegel May 26, 2014 at 2:54 pm

You can use variable names in Excel. Most people don’t bother, but skilled Excel users do bother.

Nathan W May 24, 2014 at 12:24 pm

There are some educators who feel that the fact of access and familiarity with common spreadsheet software make a good argument to use Excel or Lotus notes instead of SPSS, STATA, Matlab, etc. Unless you’re doing multi-period macro analysis across many sectors and with millions of data points, or diverse simulations to search for optimal solutions/points, then why not use the simple-to-use tool?

Perhaps economics educators in France should prioritize programming skills? I think they would rather spend their time focusing on fundamentals of the field, and use spreadsheets instead of fancy programming. But I’m sure every department has a few trouble makers who are pushing for more course offerings which offer relevant programming skills.

Jan May 23, 2014 at 4:33 pm

Piketty must fully own up to errors. So far, a compelling case has not been made that this changes core findings. If FT continues to imply grave, fundamental errors but does not show how exactly, it raises questions.

Nathan W May 24, 2014 at 12:17 pm

He forgot to exclude real estate from calculations of wealth.

ummm May 23, 2014 at 5:23 pm

if Pickets really had a solid augment it would have been published in a journal, with the whole r>g thing. But he wrote a book instead and got paid while everyone else had to do the refereeing because he knew it would have never made it in a journal anyway. pretty clever

Maciste May 23, 2014 at 5:26 pm

Can’t wait for Krugman’s blog on this.
Here is the assignment once he addresses it: Print a copy of his blog post on Reinhart and Rogoff. Compare and Contrast!

Mark May 23, 2014 at 6:43 pm

Based on past practice, it will mainly consist of (1) repeating that Reinhart & Rogoff made errors too; (2) they were defended for ideological reasons because people liked their conclusions; (3) now the same people who defended R&R attack Piketty for spreadsheet errors; (4) they’re so hypocritical, their attacks are not even worth considering on the merits. Followed by 100 comments with variations on “Koch”, “Murdoch” and “plutocrats”.

Tevyeh May 23, 2014 at 11:07 pm

He’ll doubtless spout something about the triviality of the errors, and about right-wing “desperation” or “flailing.”

Nathan W May 24, 2014 at 12:16 pm

Well, when you want to exclude one of the main indicators of wealth (the size and value of one’s house) from analysis of wealth, then “desperate” or “flailing” do not seem so off the mark.

However, it is good to know that the data issues will be drawn attention to so we can be assured of more careful analysis in the likely second edition.

ummm May 23, 2014 at 5:33 pm

I wonder what happens in cases of academic fraud. This is pretty serious if he intentionally omitted information that contradicts his entire thesis

Art Deco May 23, 2014 at 5:51 pm

Michael Bellesiles was given the choice of resignation or dismissal. What was interesting in that case was its demonstration of the systemic failure of peer review in American history. Bellesiles was smoked out by a software engineer who wrote histories avocationally (in one discrete area where he might just be close to the expert) and a law professor doing sociological research. It was amplified by a pair of journalists. Bellesiles had been given the Bancroft prize and the software engineer in question (Clayton Cramer) was given the brush off by the editor of the Journal of the Early Republic and other periodicals; the legal scholar/sociologist (James Lindgren) was able to get his work published in law journals. (His work was dismissed by the likes of Garry Wills because law journals are not peer reviewed).

Larry Siegel May 23, 2014 at 6:15 pm

My reaction is about the same as it was to Reinhart and Rogoff’s error. They didn’t have to get all the facts exactly right to raise an important concern (in their case, about sovereign debt). Whatever you think of inequality, and I probably care about it less than anyone I know, there is evidence that it’s risen and you don’t have to prove it in every country or every time period to think it is a legitimate issue.

If anyone is following my posts, please see my Piketty post at http://www.apviewpoint.com/component/kunena/?func=view&catid=93&id=1218&Itemid=0 (may be gated). It’s called “We need more ‘inequality,’ not less” and argues that if inequality is declining globally, we’ll perceive it as an increase in inequality in the First World.

Mark May 23, 2014 at 6:46 pm

Your link is gated

Larry Siegel May 24, 2014 at 4:15 am

Sorry, here’s the text of the post.

Increased inequality in the rich world is a natural and completely predictable consequence of greater equality in the world overall. (I could call this argument “We Need More ‘Inequality,’ Not Less.”) The argument is that the rich world will automatically become more unequal as the developing world gets ahead, and you don’t want that to stop (unless you only care about people in the United States). Imagine a world consisting of two countries, let’s call them Luxembourg, average GDP $100,000, and Lesotho, average GDP $1000. Everyone in Luxembourg has the same income and so does everyone in Lesotho. Both have perfect equality. Now let’s say they start to trade with each other. The best accountant in Lesotho gets some customers in Luxembourg and earns $10,000. This brings down the wage for accountants in Luxembourg to $50,000. This process goes on for a while, until both countries have a lot of inequality, but there are also a lot fewer poor people in our two-country world and Luxembourg is still moderately rich.

Hans Rosling points out that the world reached maximum inequality around 1948, when the U.S. was at maximum equality. This is not a coincidence. We had just blown up most of our competitors, and Third World had not yet started to emerge as a global economic force. China and India had lower standards of living than they did in medieval times while much of the developed world was doing fine. See Rosling’s video at http://www.youtube.com/watch?v=jbkSRLYSojo.

There’s another area where it sounds like Piketty is way off base. He argues that inequality is increasing because r, the rate of return on capital, is greater than g, the growth rate of GDP (a proxy for wage growth). Of course r>g. It was when inequality was decreasing; and it still is, now that inequality is increasing. The dividend discount model says r = d/V + g. So the difference between r and g is the current cash flow yield on the stock of capital assets. I wonder if Piketty studied any finance.

Finally, I care about the alleviation of poverty, not about inequality per se. If inequality is good for the great mass of people, I’m for it; if it’s bad, I’m against it. As to which is more likely to be the case, anyone who thinks that the poor are poor because there are too many rich people, not too few, needs to go back to economics class.

Nathan W May 24, 2014 at 12:13 pm

Comparing inequality between and within countries are very different because of data aggregation.

Extreme inequality in a neighbourhood may lead to greater risk of theft or a sense that economic outcomes are unfair because they do not accurately reflect effort. Globally, it’s more likely to lead to a demand to emigrate to access opportunities.

rapsacllion May 23, 2014 at 6:21 pm

Let’s be real. He mentions like 20 grad students in his acknowledgments; he’s probably never seen half these files. If the tide turns against him I expect he’ll throw a few RAs under the bus.

MD2 May 23, 2014 at 6:26 pm

Bingo. These guys (adding Reinhart, Rogoff, etc.) don’t do data entry or enter formulas into Excel. Some hung over grad student messed it up. What we do to prevent that, I don’t know.

Dismalist May 23, 2014 at 7:01 pm

That would be the start of going down a long road: The grad student screwed up, the nurse screwed up, the copilot screwed up… .

Stalin probably got it right: Trust is good, but control is better!

JWatts May 23, 2014 at 7:28 pm

Some employees of the “Determinations Unit” of the Cincinnati office of the IRS screwed up.

Eric Rasmusen May 23, 2014 at 8:11 pm

Screw-ups are efficient. Most empirical studies get no attention, so it’s not worth tripling the time you spend by being ultra-careful. Instead, do a good-faith effort, and if anybody cares, they’ll check it and find the mistakes. Anybody who doesn’t make mistakes is misusing his time.

TMC May 24, 2014 at 9:00 am

Trouble is, Eric, that in some branches of science, all the screw-ups go in the same direction.

Nathan W May 24, 2014 at 12:10 pm

Open data. Hung over grad students will always make mistakes.

Larry Siegel May 24, 2014 at 10:23 pm

We try to formulate ideas that are robust to making a few unavoidable data errors.

Zach May 23, 2014 at 6:40 pm

Yet another instance of the humanities’ bizarre insistence that the group leader is the only author of a paper.

andrew' May 23, 2014 at 7:23 pm

Credit default options!

Something something peer review something something.

Devil's Dictionary May 23, 2014 at 7:17 pm

It has been known for some time that Piketty deliberately distorted the data. His French colleagues wrote about it a couple of years ago.
http://www.devilsdictionaries.com/blog/piketty-cooked-his-book

Bill May 23, 2014 at 8:26 pm

Piketty seems to be making a defense based on Saez’s more recent work, which seems to be undisputed by the Financial Tkimes, yet.

However, I am interested in another take: for those who argued that education and automation accounted for the increase in disparity using Piketty data, how will they now argue, that, when there is less disparity, those attributes don’t account for as much.

No man changes more than another than he does himself at a different time.

Bill May 23, 2014 at 8:32 pm
Bill May 23, 2014 at 9:29 pm
David Sollars May 23, 2014 at 9:52 pm

Decades ago, when I was doing empirical work, I used spreadsheets to get the data together. Hand punched more data into Lotus 123 than I really care to remember. Probably made a few mistakes along the way. But then I would run the summary stats, and if something seemed out of wack, went back and checked, and checked. Outliers were important, because either you made a mistake or they told a story. Back then, it was not widely expected that you would make your data publicly available– heck , the WWW didn’t yet exist, but somehow you knew deep down you never wanted to make a mistake that would come back to haunt you. So one would spend lot of time with data, and was careful. Then the data was moved into SAS, TSP, or LIMDEP. Then I ran ran the regressions, created the charts, and looked at the residuals to see if there was something askew. Weren’t all economists taught this?

What is truly surprising in the current story, or in the other data error stories, is that the author–Pinketty in this case, seems not to have the close familiarity with data that should happen. Instead, his data seemed to buttress his priors, and he was careless. I won’t win the Nobel Prize in Economics. I didn’t earn my doctorate from a top ten program, and I don’t work at one. But geez, professionalism is required in the economics profession, isn’t it?

Nathan W May 24, 2014 at 12:09 pm

Do you have any idea how much data went into that book?

I’m sure if he had a team of research assistants to comb over the data and processes to pick out errors there would not have been these problems. I do not doubt there will be resources to ensure this for the next edition.

Jay May 23, 2014 at 11:11 pm

Michael Mann takes all the raw temperature data and manipulates it, I mean uses tricks, to transform the data into a hockey stick. Piketty has learned well.

Nathan W May 24, 2014 at 12:07 pm

He’s not the one who wants to assume that housing wealth should be defined as not-wealth for inequality analysis purposes.

Nick May 23, 2014 at 11:59 pm

It seems the economics books and papers that get the most popular press are often shown to have some glaring errors.

I bet if you scrutinize the vast majority of economic works you’ll find similar faults.

Andrew' May 24, 2014 at 5:27 am

I think I’ll set aside my objective criticism of academia to make opportunistic attacks on the lefty darling.

So I won’t speculate there might be even more which is why progress generally happens through corroboration not single works.

Nathan W May 24, 2014 at 12:05 pm

I think it’s more like “books and papers which arrive at conclusion that people don’t like get enormous scrutiny, and then errors are drawn out and fixed”.

triclops May 24, 2014 at 1:41 pm

Very good point

Nathan W May 24, 2014 at 12:04 pm

One day, I will be able to save enough money for a down payment for a house.

If wealthy people don’t bid housing prices up another 200%.

Total BS to exclude housing prices then say that wealth has not changed during the period of massive housing price increases. Trillions and trillions of dollars worth of assets are locked up in housing, in a manner which effectively locks the poorest 20-30% (?) from ever having a hope of entering the housing market.

triclops May 24, 2014 at 5:04 pm

Your description of the cause of housing prices increases shows very little understanding. Housing is suitable in most of the country. Where it isn’t, it’s because of restrictions on new housing and regulations, not the evil rich bidding up prices.

Nathan W May 25, 2014 at 5:48 pm

Sorry, you’re right poor people bid up housing prices via rent controls.

Axa May 24, 2014 at 12:59 pm

the FT critique was half-assed. For a strong one, read the one from Sala i Martin http://salaimartin.com/randomthoughts/item/720-piketty-y-capital-en-el-siglo-xxi.html

Comments on this entry are closed.

Previous post:

Next post: