*Spin* magazine moves from a five star system to ten

The old rating system granted up to five stars but now the maximum number of stars is ten.  This signals that they wish to start exaggerating the quality of the product.  When there are only five stars you know that they are laying their reputation on the line when they grant five stars to a new CD.  (Michelin of course won't give a restaurant more than three stars.  They don't calculate out to the fourth decimal place along a scale of one thousand.)  If the music isn't good you can decide to stop trusting them.  But say they give a new release eight, nine, or who knows maybe eight and a half stars?  What exactly are they trying to say?  Yes they are putting their reputation on the line when they give ten stars, but this will happen so infrequently that it will be harder to judge their overall trustworthiness.

Evaluation systems with fewer and grosser distinctions are often more credible because they are easier to monitor.


could it be they're trying to catch up with pitchfork? how do you read their rating system?

Interesting. How many different ratings to the credit agencies give out?

According to Wikipedia, Moody's divides securities into: Aaa, Aa1, Aa2, Aa3 A1, A2, A3, Baa1, Baa2, Baa3, Ba1, Ba2, Ba3, B1, B2, B3, Caa1, Caa2, Caa3, Ca, C...

Spin... forgot about that one

If you find it easier to understand what they're saying when it's expressed in five stars rather than ten, divide by two.

The truly brave reviewer uses only one non-divisible star. I challenge you, Tyler, to start doing restaurant/book reviews this way.

What this analysis neglects is that the "reputation-staking" force of a high rating is as much a factor of its percentile as its absolute rating.

So if Pitchfork gives out 4 9+'s a year, I will pay close attention unless/until I stop trusting them.

More precisely, the "degree of reputation-staking" of a high rating depends on the percentage of albums that rate "significantly" lower. "Significantly" meaning that e.g. 8.7 and 8.5 are considered as having the same rating.

Now, if readers don't in fact interpret rating systems in a percentile way, then maybe Tyler is right that a finer-granularity rating system is an indication that the publication is trying to make a smaller splash with its ratings.

Interesting post.


This kind of thing all started when they made that amplifier that went to 11. Never should have done that.

In BKS Iyengar's Light on Yoga he goes up to sixty stars and feels the need to include half stars.

Don't we all just want someone to say "you will like this."

Maybe that's why we put up with friends.

It does appear to be a nod to Pitchfork. Pitchfork's problem, though, is that they almost never award newly released records a 10/10. The last was in 2005, for Scott Pollard's "Relaxation of the Asshole." Before that it was 2002, for "Yankee Hotel Foxtrot" by Wilco.

I'd also like to hear about the extension of this argument to grading systems. Do plus/minus systems add information about student performance, or subtract it?

I almost exclusively read film reviews through the filter of Metacritic, which converts all the written reviews to a 100 point scale. I find the small scales typical of newspapers uninformative.

I wrote more here.

A ten-star system is silly.

A distinction between 8 and 9 on something as subjective as an opinion about a movie or restaurant is meaningless to me. Who buys this CD instead of that one because of that difference, or worse, the difference between an 88 and an 85?

A coarser scale conveys useful information as to what broad category something fits into - it's a stinker, etc., without presuming to measure quality with unattainable precision.

I find the small scales typical of newspapers uninformative.

A finer grained system is only relevant if the rating criteria support that level of differentiation. In all probability it doesn't. I expect that if a movie critic had his memory wiped and re-reviewed a movie, the rating could easily fluctuate up or down a star in a 5 star system depending on the phase of the moon.

For a physics problem, you don't say the answer is 5.346 +/- 2

The drive to measure things to 3 digits of accuracy is simply a delusion for those who desperately want there to be accurate metrics for things as subjective as movies, books, or wine.

However, given there's a pretty big audience of people who want to believe it, why should a magazine disabuse them and lose sales?

"This all started when they made that amplifier that went to 11. Never should have done that."

Alright, but most blokes are playing at 10, and if they need it a bit louder, where can they go? Nowhere.

Comments for this post are closed