*Everybody Lies*

by on April 11, 2017 at 2:54 am in Books, Data Source, Economics, Web/Tech | Permalink

That is the new and fascinating book by Seth Stephens-Davidowitz, with the subtitle Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are.  Here is one of many interesting bits:

Urban areas tend to be well supplied with models of success.  To see the value of being near successful practitioners of a craft when young, compare New York City, Boston, and Los Angeles.  Among the three, new York City produces notable journalists at the highest rate; Boston produces notable scientists at the highest rate; and Los Angeles produces notable actors at the highest rate.  Remember, we are not talking about people who moved there.  And this holds true even after subtracting people with notable parents in that field.

Many of the results in the book are taken from Google data and Google searches.  I was a little chuffed to read this part:

A child born in New York City is 80 percent more likely to make it into Wikipedia than a kid born in Bergen County.

[Actually I was born in Hudson County, but grew up in Bergen.]  And this:

Of the trillions of Google searches during that time [2004-2011], what do you think turned out to be most tightly connected to unemployment?  You might imagine “unemployment office” — or something similar…The highest during the period I searched — and these terms do shift — was “Slutload.”  That’s right, the most frequent search was for a pornographic site.

Here is previous MR coverage of Seth Stephens-Davidowitz.

1 Steve Sailer April 11, 2017 at 3:38 am

The Coens, for example, don’t even much like Los Angeles and they come up with their top young acting finds out of the Valley: Hailee Steinfeldt and now Alden Ehrenreich.

2 Steven Sailer April 11, 2017 at 5:07 am

Alden Ehrenreich, who was so delightful as the cowboy movie star in “Hail Caesar,” has landed what seems like the top role in the movie business: taking over Han Solo from Harrison Ford in what’s likely to be multiple Star Wars movies.

How’d he get discovered? When he was about 13 he made a funny video to play at a bat mitzvah in Pacific Palisades. Another guest was Steven Spielberg.

3 msgkings April 11, 2017 at 1:30 pm

Didn’t the original Han Solo (Ford) get discovered putting in a doorframe in some building where George Lucas worked?

4 Cy Nical April 11, 2017 at 2:15 pm

“Success is 80% just hanging around.” – Woody Allen

5 Jan April 11, 2017 at 6:05 am

And don’t forget immigrant Oscar Isaac (originally Isaac Hernandez Estrada, from Guatemala), who was nominated for a Golden Globe for his breakout performance in the Coen Bros’ Inside Llewyn Davis.

6 prior_test2 April 11, 2017 at 4:05 am

‘A child born in New York City is 80 percent more likely to make it into Wikipedia than a kid born in Bergen County.’

Assuming that one has the correct ratio between the two different population to make a meaningful comparison – after all, it is possible that Bergen County, properly weighted, has more children likely to make it into wikipedia. Providing another opportunity for someone else to do the checking of whatever is pasted here.

7 JWatts April 11, 2017 at 10:36 am

” Providing another opportunity for someone else to do the checking of whatever is pasted here.”

Wow, the irony is ripe.

8 So Much For Subtlety April 11, 2017 at 5:06 am

That’s right, the most frequent search was for a pornographic site.

It makes sense. What makes being born in Boston more likely to result in a scientific career? It must be whom you are sleeping with. If you are more likely to end up in Wikipedia if you’re born in New York it is because you’re more likely to be banging nasty with someone who will help you rise to a position where you could be in Wikipedia.

And conversely if you are demotivated and not going anywhere, you’re probably at home whacking off to porn. Cause and effect may be hard to determine in this case.

9 Lurker April 11, 2017 at 5:56 pm

Why can’t it be both? Gotta do something with all that recently minted Leisure Time…

10 Steven Sailer April 11, 2017 at 5:14 am

One interesting thing is how long one location can stay on top in its field. Palo Alto, CA was where one important version of the vacuum tube was invented over 100 years ago by Lee de Forrest.

Lexington, MA currently has the highest test score public school district in the country. It was also home to a large fraction of America’s top writers about 200 years ago.

I think universities, which tend to be remarkably stable, play a big role in these enduring patterns: e.g., Lexington is a bedroom community for professors employed in Cambridge, like Noam Chomsky and Edward O. Wilson.

11 Thiago Ribeiro April 11, 2017 at 6:42 am

“Lexington is a bedroom community for professors employed in Cambridge, like Noam Chomsky and Edward O. Wilson.”
Lexington, Massachusetts is not even the most important Lexington. The first information I got from Google after typing Lexington was that that Lexington, Kentucky is the place to go if one wants a horse. And it is a shame Chomsky and Wilson never teemed up to explain why Universal Grammar is universal.

12 anonymous regular April 11, 2017 at 9:57 am

I live in Lexington, MA. My recent neighbors include an engineering professor married to a consultant, a chemistry professor married to a CEO/engineer, and a specialist MD married to a different specialist MD. And people claim the test scores are because of “good schools.” But I don’t think it’s because of the town population 200 years ago, except perhaps as historical path dependence. There’s a clustering effect at work: when smart people move to Boston they want to live where all the other smart people are. Lexington also has a good commuting location for universities and technology. And it’s a nice New England town.

I doubt any of these people would qualify as “notable,” but their kids certainly have a head start on achieving notable status.

13 Sean April 11, 2017 at 10:02 am

Lexington is full of A-holes. The nice people live in Concord.

14 anonymous regular April 11, 2017 at 10:13 am

I’m not sure I’d argue. But to Steve’s point, there’s a significant difference in the school outcomes. 🙂

Can we at least agree that Newton is full of lawyers and people who work in insurance?

15 Thor April 11, 2017 at 10:50 am

“There’s a clustering effect at work: when smart people move to Boston they want to live where all the other smart people are. ”

Thinking about the above, it occurs to me that there’s something wrong with this explanation of clustering. I know it is anecdotal but consider yourself and your friends. (This will assume MR readers are smart; I think we can safely assume this.) Did you choose your cluster of friends because they were smart? My wife and I are professionals. We are university educated (I teach history in a college). But we chose our friends because we like the activities they engage in: cooking, walking, reading, drinking wine, fitness. Our friends are smart but not overwhelmingly so, and we have declined to become close friends with the cognitive elite in our area (4 mathematicians) because they are nerdy and friendless for that reason. YMMV but my intuition is that smart people cluster because of values more than smarts, even though smarts is part of what is valued.

16 anonymous regular April 11, 2017 at 11:33 am

When I wrote that, I was thinking that they mostly cluster because of the good schools and a widespread incorrect assumption about the direction of causality. So people move to Lexington because they think the schools make kids smart, which smart people value disproportionately. There are other nice things about clustering that have this circular property as well. For example, the commuting is good partially because the right set of employers has clustered around Lexington.

17 Cy Nical April 11, 2017 at 2:42 pm

200 years ago people in Lexington had guns and they weren’t afraid to use them.

I know the area well.

18 anonymous regular April 11, 2017 at 3:55 pm

Maura Healey would not have been on their side.

19 Cy Nical April 11, 2017 at 2:39 pm

True. Anecdotal reinforcement: Our university town has the highest rated public charter high school in the US. Though admission is by lottery, for the small number of un-allocated openings, siblings of current or past students and children of board members and employees gain automatic entry. The board has historically consisted of university professors and some (many) of the teachers are post docs. No need for phony union badge CA teaching credential there – a phd in Eng Lit can teach english I guess. Many of the students are science fair stars. A local science prof and his slaves, I mean his grad students, run the science far. It’s a nice, tight little publicly funded school, a stone’s throw from the university.

The cognitive elite are effective at capturing educational opportunities for their offspring – the new royalty.

Not surprising, there is much wailing and gnashing of teeth among the townies and jealous public school officials about a lack of diversity snd the loss of high performing students to the chummy charter.

The new rich get richer.

20 Thiago Ribeiro April 11, 2017 at 3:39 pm

The new rich are university professors?

21 Cy Nical April 11, 2017 at 5:26 pm


Ok, you got me. How about the following mods?

The cognitive elite get eliter.

The cognitive elite get cognitiver.

In all seriousness, they live a very nice life in a nice small city in very nice houses. It used to be considered an upper-middle class when we had a middle class. Their kids have enormous educational opportunities compared to the children of the townies that live just outside of the university school district. So, yes, their kids are more likely to have a nice life.

22 Thiago Ribeiro April 11, 2017 at 5:38 pm

Nice lives, the kind of life I myself would appreciate – I myself live in a kind of college town, but I doubt the Trumps and Buffetts of the world envy the professors. Well, anyway I am happy to know at least distinguished professors can afford to lead nice lives in America

23 Paul A Sand April 11, 2017 at 5:36 am

You have a title for your autobiography: “Against All Odds”.

24 chuck martel April 11, 2017 at 6:09 am

The pseudo-knowledge or non-knowledge or mis-knowledge derived from this kind of “big data” doesn’t have any utility. Considering it is a waste of synapse activity that could be used for seeking out a good ethnic restaurant or evaluating current trends in public landscape architecture..

25 rayward April 11, 2017 at 6:52 am

David Brooks’s column today considers the difference between public intellectuals and thought leaders, and he laments the dearth of intellectuals today. https://www.nytimes.com/2017/04/11/opinion/this-age-of-wonkery.html?ref=opinion&_r=0 Fame today is more the product of celebrity than accomplishment. Thus, we have celebrity doctors, celebrity writers, and, yes, celebrity economists. Of course, celebrity is the result of promotion, self-promotion and co-promotion: one’s status as a celebrity is a function of like status of one’s colleagues and friends. It’s not a coincidence that “successful” people would reside in the same area. As Brooks points out, today’s thought leaders are more likely seen in an uplifting TED talk, while intellectuals are more likely seen giving a pessimistic prognosis of our future. Optimism sells, pessimism doesn’t. Is Cowen a public intellectual or a thought leader? His new book is both pessimistic (complacency and the Great Reset) and optimistic (better days ahead after the Great Reset). And he has the good sense not to describe the unpleasant details of the Great Reset. Is a life without celebrity a life worth living? Does the absence of a Wikipedia page denote failure? In the media age, is it possible to have a fulfilling life, a successful life, without celebrity? Modesty does not become one who aspires to be a celebrity.

26 rayward April 11, 2017 at 7:08 am
27 Todd K April 11, 2017 at 10:15 am

“Optimism sells, pessimism doesn’t.”

Can you think of an optimistic book in the past few years that sold well? As for pessimism: “The Coming Apart of America”, “The Great Stagnation”, “Hot, Flat and Crowded”, “Capital in the Twenty First Century”, “End This Depression Now!”, “The Rise and Fall of American Growth” and of course my personal favorite, “We Are All Doomed.”

Optimism: “The Singularity is Near” but that goes back to 2005 and for some reason not everyone interprets having his or her brain getting nano implants in 2028 and then being uploaded into The Cloud in 2041 as optimism.

28 A Definite Beta Guy April 11, 2017 at 10:21 am

“The World is Flat” sold better than “Hot, Flat, and Crowded,” no?

29 Thor April 11, 2017 at 10:54 am

Pessimism outsells optimism. Matt Ridley, the author of the Rational Optimist, is one of the only non-pessimists to opine on Big Picture matters.

30 Steve April 11, 2017 at 9:50 am

David Brooks, unlike other public commentators, scratched his way up to be paid millions for fact-averse bloviation.

31 Cy Nical April 11, 2017 at 3:40 pm

David Brooks wrote a book about character while publicly admitting he didn’t have any.

Character alone conferred status in the past. Not so anymore.

32 chuck martel April 12, 2017 at 5:38 pm

During their journalistic careers H.L. Mencken, Will Rogers and Westbrook Pegler, people that could actually string coherent thoughts together on a regular basis, were world famous. Today only a few people even remember their names. We should look forward to the day, soon to come, when a second-rater like Brooks fades into obscurity.

33 Axa April 11, 2017 at 7:03 am

What do search engines queries represent? First, only 40-50% of web traffic is human traffic. The rest are bots, good or bad ones. Thus welcome to the world of search engine optimization (SEO) where bots are used to “generate google organic traffic”. This means, simulating a human is doing a search in google and then visiting the site. One example: http://bestmacros.com/gtraffic-bot/

Porn sites have free videos as bait to get clicks on ads about casinos, dating, viagra and shady get rick in a week schemes. Their objective is that visitors click on those ads and some idiot with credit card buys something. It makes sense for porn site owners to use bots to increase their site relevance on search engines. If the bot searches for “slutload” a million times, search query data is useless.

The most interesting question here is how much of Google’s “organic traffic” is human traffic?

34 kevin April 11, 2017 at 7:44 am

How would a bot’s searches be correlated with unemployment?

35 Thiago Ribeiro April 11, 2017 at 8:21 am

His job may have been outsourced to Chinese robots, who work harder, earn more money and have fewer benefits.

36 Cy Nical April 11, 2017 at 2:47 pm

Chinese robots are stealing the jobs of our robots.

37 Thiago Ribeiro April 11, 2017 at 5:17 pm

Rhe only way out is making them pay for a Wall – and build it, too.

38 Cy Nical April 11, 2017 at 5:29 pm

The wall will be made in China and shipped to the US.

We should test for lead.

39 Thiago Ribeiro April 11, 2017 at 5:39 pm

Unless you intend to eat it, you are safe. Thr ones whomaressuposed to get close to it are illegal immigrants.

40 dearieme April 11, 2017 at 7:18 am

Is “big data” coming to mean a method for generating endless correlations that can be passed off as causal relationships?

41 Cy Nical April 11, 2017 at 2:52 pm

Yeah, you can make a lot of dough trying to predict the future by looking in the rear mirror. Let’s all get big data credentials and get in on the deal.

Ssshhhhh… don’t tell anybody about chaos, contingency, or path dependence or you will ruin everything!


42 Ray Lopez April 11, 2017 at 7:18 am

Yawn, non-story. Correlation is not causation. Big Data is a bust of backwards looking data mining. And, as prior_test2 says upstream, assuming they did the math right, all the ‘big cities that specialize in XYZ produce people good at doing XYZ’ shows is what Krugman wrote about years ago: the spillover effects (even among natives) of specialization in a big city. Big deal.

Bonus trivia: by this logic, the Washington DC area specializes in B.S., posturing, and hot air: and have a competitive advantage over other cities in this regard. A perfect city for a Donald Trump to succeed in.

43 Cy Nical April 11, 2017 at 2:53 pm

We should change the name to Washington, BS.

We the people say Washington, FU!

44 Andrew M April 11, 2017 at 8:50 am

The 80% of NYC-born figure, and most of his other insights, can also be found his 2014 article in the NYT, which you wrote about at the time. There, you’ve saved $18.29 (and a couple of hours).

The secret is that it’s all about who you invite to dinner parties at your home. If you work in NYC but live in Bergen County, your dinner parties will be full of your wife’s local friends; or the parents of your kids’ friends. The conversation will be safe topics like family, school, and local issues. By contrast if you both live and work in NYC, you can invite your colleagues to regular dinner parties. Your conversations will be passionate angry talk about work; and the importance of work will rub off on your kids.

Of course if you live in Lexington, Kentucky, your passionate talk might be about tractors and monster trucks, not about work. Follow your passions.

45 Pshrnk April 11, 2017 at 12:50 pm

“Of course if you live in Lexington, Kentucky, your passionate talk might be about tractors and monster trucks, not about work. Follow your passions.”

No, it will be about UK basketball, horses and Toyotas! Do you think its just a bunch of hicks?

46 Thiago Ribeiro April 11, 2017 at 5:41 pm

I am told theynare the capital of horses, so yes, pretty much yes.

47 hello April 11, 2017 at 9:20 am

I don’t know… Here’s a question: SHOULD I be impressed? Is the reaction SUPPOSED TO BE “Whoa! That’s amazing!” Looking back at the previous posts about SSD on MR, I feel pretty much the same thing: little to nothing.

Also, the conclusions could be legit, but has anyone challenged this guy’s methodology? Super easy to find correlations and differences among populations. People do this all the time but it’s garbage because the quality of the analysis/methodology is poor.

48 GoneWithTheWind April 11, 2017 at 10:19 am

A lot of people don’t want to end up in Wikipedia.

49 Mc April 11, 2017 at 11:03 am

we all will though . . . eventually, because we is all so special

50 Bill April 11, 2017 at 11:45 am

Is the past the future or is the past the past?

NYC entries are of people who have accomplished (or been notorious) based on their past activities in association with others in their local network. In other words, residence in NYC made it easier to establish links or ties with others who historically had been accomplished, placed on in proximity to critics who propagated reviews, placed on in proximity to national news, network, tv, radio and newspaper outlets.

But, with the internet, will these network effects continue? Do you need to be proximate physically to a hub, or can you be famous, like me, sitting in your office typing on a computer?

51 Cy Nical April 11, 2017 at 5:33 pm

Budding jazz musicians go to NYC.

Are famous jazz musicians more likely to be born in NYC?

52 Bill April 11, 2017 at 8:22 pm

Good question. When recording studio headquarters were in NYC or LA or Memphis, it made sense to be close to the record label execs. But, does it now?

53 Michael April 11, 2017 at 11:55 am

Why “Everybody Lies”?

54 Cy Nical April 11, 2017 at 5:44 pm
55 Lurker April 11, 2017 at 5:46 pm

Maybe that’s Tyler’s way of calling bullshit on this ‘new and fascinating book’.

56 Mark Thorson April 11, 2017 at 12:08 pm

Is “make it into Wikipedia” some kind of index of achievement, like having 100,000 followers on Twitter? We need a more rational metric, like +100 for being in WIkipedia and -1000 for also being in the Encyclopedia of American Loons.

Comments on this entry are closed.

Previous post:

Next post: