The costs of measuring value too precisely (model this)

by on April 2, 2014 at 2:24 pm in Current Affairs, Web/Tech | Permalink

…the editors at The Verge have a policy that seems a little bit odd and anachronistic: They don’t let writers see how much traffic their stories generate. Ever.

As the American Journalism Review reported, in a piece called “No Analytics for You: Why The Verge Declines To Share Detailed Metrics With Reporters,” the editors at The Verge simply don’t want their writers thinking about traffic.

What’s more, The Verge is not alone in this practice. Re/code, a tech site run by Kara Swisher and Walt Mossberg, the longtime Wall Street Journal tech columnist, also won’t share traffic stats with writers. MIT Technology Review holds numbers back too.

“We used to show the writers and editors traffic, and told them to grow it; but it had the wrong effect. So we stopped,“ says Jason Pontin, CEO, editor in chief and publisher of MIT Technology Review. ”The unintended consequence of showing them traffic, and encouraging them to work to grow total audience, is that they became traffic whores. Whereas I really wanted them to focus on insight, storytelling, and scoops: quality.”

That phrase – “traffic whore” – tells you everything you need to know about why some journalists have an aversion to chasing traffic. They fear it creates an incentive to do the wrong things.

Of course these policies hold only at some margins I believe…nor are they used at Gawker.

The full article, by Dan Lyons, is here.

Alexei Sadeski April 2, 2014 at 2:27 pm

Already been done: http://www.theonion.com/video/new-live-poll-allows-pundits-to-pander-to-viewers,14350/

Also, when the The Verge become filled with boring commie articles?

Stephen April 2, 2014 at 2:54 pm

Maybe when Vox partnered with Ezra Klein?

Alexei Sadeski April 2, 2014 at 9:40 pm

Ah, I’d no idea Verge was related to Vox. It all makes sense now.

Errorr April 3, 2014 at 10:22 pm

Vox media is the parent company for Verge, SB Nation, Curbed, Racked, etc… They chose their name Vox for the new site.

Das April 3, 2014 at 5:42 am

Wasn’t The Verge always targeted at SWPLs?

Oh. Let me rephrase that: “10 top reasons why only white people read The Verge – inside: boobs”.

charlie April 2, 2014 at 2:51 pm

I view this post as a test to see whether we can identify Alex when he pretends to post as Tyler.

F. Lynx Pardinus April 2, 2014 at 2:51 pm

Here’s a fun post by Deadspin about Bleacher Report from 2012: http://deadspin.com/5948776/this-is-how-your-bleacher-report-sausage-gets-made

jtf April 2, 2014 at 2:58 pm

Content pushers hate this one weird trick!

Drifting April 3, 2014 at 11:56 am

You better watch that viral video before it gets banned, you know.

Ted Craig April 2, 2014 at 2:58 pm

I wonder how much of this has to do with concerns high-traffic writers will demand higher pay.

Contemplationist April 2, 2014 at 3:17 pm

‘Traffic whores’ <- what an apt description for the Gawker empire (I exempt Lifehacker and io9 from this)

GiT April 2, 2014 at 4:50 pm

Lifehacker is an exemplar of insipid traffic whoring.

Anon. April 2, 2014 at 5:31 pm

lifehacker and io9 are traffic whores par excellence, what are you talking about?

Alexei Sadeski April 2, 2014 at 7:11 pm

Isn’t it obvious? Anything one personally enjoys must by definition not be traffic whoring.

Contemplationist April 2, 2014 at 11:48 pm

I rarely read them, but I often find *useful* stuff in Lifehacker for personal projects. IO9 rarely shows up on radar unlike Jezebel, or Gawker.com or Valleywag.
Gizmodo was decent a few years ago, same with Jalopnik.

RJB April 2, 2014 at 3:41 pm

Hiding traffic numbers from authors seems like a very reasonable response to measure management: taking actions to improve measures of performance, rather than the true underlying performance those measures purport to capture. Measure management is especially problematic when you are interested in true performance that is hard to quantify and/or is revealed only with delay, like quality, impact on high-value readers, insight, accuracy, etc. (all mentioned in the article).

From a measure-management perspective, the problem with “traffic whores” isn’t that they are overly motivated to succeed, but that they are narrowly focused on a pretty awful measure of the types of performance their employers define as success.

Now, apply a similar analysis to high-stakes testing in schools, publicizing patient ratings of doctors, and you can see why attempts at market-based, libertarian and meritocratic policies often run into trouble: everyone starts chasing the measures of performance, rather than performance. If you think competitive markets will solve everything, click my handle link and read the review papers on traditional and behavioral finance. But markets are definitely more resilient to this than organizations are, where measure management is a persistent and nearly universal challenge.

david April 2, 2014 at 4:17 pm

Economists call it Goodhart’s law.

RJB April 2, 2014 at 5:51 pm

In education, it often goes by the name Campbell’s Law . My only complaint with Campbell’s definition is that it seems to assume that the measure is reasonable in the first place, and only corrupted by gaming. If only!

ummm April 2, 2014 at 6:41 pm

But what if measures of performance are highly correlated with ability? The left has criticized standardized testing for years, but it’s still the most objective way of measuring ability because grade inflation and differences of curriculum has rendered high school GPA’s nearly useless. A meritocracy benefits the most competent, instead of who has the most connections or the most privileged background. Online news sources want clicks because that’s how the pay their staff through the ad revenue generated from the pageviews. In a free market, individuals and corporations have to compete for attention.

GiT April 2, 2014 at 7:33 pm

Except SAT doesn’t predict college performance any better than GPA.

http://www.nacacnet.org/media-center/PressRoom/2014/Pages/BillHiss.aspx

uffs April 2, 2014 at 10:40 pm

If only human society were more straightforward (at least some fraction of) ideologues could be right all the time.

Turkey Vulture April 3, 2014 at 1:41 am

That study doesn’t appear to control for major, for institution, or really for anything. If so, it is worse than worthless.

Zach April 2, 2014 at 4:30 pm

It makes sense as a strategy to build the site’s brand rather than the writers’. At the moment, the predominant strategies for driving traffic to individual stories are 1) manipulative headlines 2) schmaltz 3) political flamebait. As an editor trying to build a site’s reputation, you would ideally like your writers to do none of these things.

ummm April 2, 2014 at 4:43 pm

S&P 500 makes a historic high. everyone getting richer than ever. everything is amazing and no one is happy.

Noumenon72 April 2, 2014 at 5:46 pm

That’s not relevant at all. Go away. Also, learn to adjust for inflation.

anon April 2, 2014 at 6:03 pm

“If that doesn’t convince you, just listen to all those Keynesians who are proudly calling this a form of useful economic stimulus, akin to pyramid-building, or an invasion from outer space…oh wait…”

So this *wasn’t* a trolling comment to drive traffic?

Christopher Finocchio April 2, 2014 at 6:36 pm

Editors see quality defined far differently from popularity and think optimizing for popularity in short term hurts the brand and lowers own statuses.

Steve Sailer April 2, 2014 at 6:58 pm

Something that’s become apparent over the last couple of years is that clickbait experts have noticed that an effective way to generate visits and comments is with racist and sexist articles denouncing white males. There is both a market for racial and sexual animosity, but it’s practically impossible to get in trouble for expressing anti-white and/or anti-male attitudes. In turn, this kind of low-brow anti white male hate speech generates lots of traffic and huge numbers of comments from white men pointing out the stupidity of the article.

Millian April 2, 2014 at 7:26 pm

Your bunch hounded Sherrod out of office while demonising black women as welfare thieves.

GiT April 2, 2014 at 7:36 pm

Steve Sailer, of course, is an expert on markets in racist and sexist articles, racial and sexual animosity, low brow hate, and stupidity.

carlospln April 2, 2014 at 8:45 pm

Post of the Day!

Contemplationist April 2, 2014 at 11:51 pm

Progressives are of course experts in deflection and snark covering up their vapidity, ignorance and malice.

GiT April 4, 2014 at 2:21 pm

Nice bit of projection.

prior_approval April 3, 2014 at 12:19 am

Vox Day disagrees with you, and has the site traffic numbers to prove it (according to him).

But then, Vox Day does not actually like playing the victim card, unlike some.

CMOT April 2, 2014 at 7:15 pm

Writers can only become clickwhores if they are paid for clicks. You’ll get more of whatever you incent. In the absense of some sort of system to determine/rate quality, known both to management and talent, this is simply an way to keep talent from gauging (and asking for) it’s true worth.

It’s a dark pool for talent …

So Much for Subtlety April 2, 2014 at 7:24 pm

You have never gone out with an attention whore? Writers don’t need to be paid to want the clicks. They are perfectly able to desire all that attention all on their own.

CMOT April 2, 2014 at 7:35 pm

That’s a fine argument for not counting clicks.

But its not an argument for not measuring their output in any way. The editors are doing this, that’s what they are paid to do. The owners do this too, that’s how they stay in business. So the only folks who have no idea how well the writers are doing what they are paid to do are the ones who might ask to be paid more money for excelling at it.

Millian April 2, 2014 at 7:27 pm

It makes most sense for publications on the margin of popular and specialist interest that wish to remain in the latter category.

KLO April 2, 2014 at 9:15 pm

I was thinking something along the same lines. One approach is to focus on a narrow area, hire good writers, and cultivate a small, dedicated audience. With this model, people land on your front page frequently and read several articles per visit. The overall traffic of the site is the relevant metric rather than traffic for each individual article.

The other model is to allow basically anyone to write for you on any topic and pay each writer based on page views. Most visitors land on your site via individual articles themselves and do not linger on your site after finishing that article. Under this model, it makes sense to pay writers for traffic.

Dismalist April 2, 2014 at 9:25 pm

Great comments, on average! RJB’s is best.

Had the technology been available then, I would have been content to be a “click”, well, call-girl! We all have our standards.

Donald Pretari April 2, 2014 at 9:37 pm

”The unintended consequence of showing them traffic, and encouraging them to work to grow total audience, is that they became traffic whores. Whereas I really wanted them to focus on insight, storytelling, and scoops: quality.”

In other words, Quality Whores.

Jon April 2, 2014 at 10:36 pm

Reminds me of the new bus driver who had a route on the outskirts of town. One day he brought in a huge amount of cash and his boss exclaimed—”I can’t believe it, that route never generated even 1/3 this much. How did you achieve this” to which the driver responded “business was slow so I took the bus to the Main Street Route.”

Samuel April 3, 2014 at 1:02 am

Easy to model by the median buyer. The bigger the audience, the more shallow your median buyer’s appeals will be, intellectually, creatively, etc. It probably drops off quite quickly.

What they are trying to do is keep only the part of the audience distribution that they want, but to get their attention more often. Say they have already captured the attention of a particular 5% of potential readers, mostly due to hype from the debut, Now they need to deepen that 5%’s attention without increasing the total audience size.

There is a powerful corner to solution to generating clicks to a website. They’re a class of websites that look and act like buzzfeed. It takes work to pull out of its gravitational pull. Slate.com, Salon.com are examples of sites that failed to

Insurance Dojo April 3, 2014 at 11:47 am

Ryan Holiday (marketing/PR/social media consultant and former head of marketing for American Apparel) covers this issue perfectly in his book Trust Me, I’m Lying: Confessions of a Media Manipulator. He writes about how easy it is to get an article placed on the Gawker network and many of the other widely read blogs as long as the article brings in traffic. One of the more benign examples: use a fake email address and leak a ‘controversial advertisement’ that your company is planning to run and get free press. Anything that polarizes people and makes them feel strongly for or against the article’s theme works incredibly well at generating traffic since then they feel compelled to share the story and comment on it. More shares and more comments == more advertising revenue. If a story is hot enough on the internet, the mainstream media will pick it up and before you know it, you’ve achieved national cable news exposure.

Internet media is a throwback to the days of yellow journalism (in contrast to the later era of subscription based journalism) where newspapers had to be sold on the street each day instead of having a base of subscribers who would pay regularly. As a consequence, the stories had to be shocking and sensationalist in order to drive sales. Truth and objectivity were secondary to entertainment, scandal and hysteria. One of the reasons I enjoy reading Marginal Revolution so much is that it runs contrary to that trend and doesn’t pander to infantile emotionalism over statistical anomalies, yet it stays entertaining with off-beat posts about cool topics.

Comments on this entry are closed.

Previous post:

Next post: