The NBA study of referee bias

As a response to Justin Wolfers and Joseph Price, the NBA financed a study supposedly showing there is no racial bias in refereeing.  Here is a WSJ analysis of that study.  Here is part of what they found:

Columbia University statistician Andrew Gelman, who has blogged
about the Wolfers-Price study and participated in a conference call
with Segal and me, said, “What the statistics tell you is that there’s
a pattern in the data that’s not explainable by chance.” University of
California-Irvine statistician Hal Stern told me the NBA’s study “can’t
be said to disprove the Price-Wolfers analysis.”

the NBA’s study didn’t include players who weren’t called for any
fouls, making Segal’s results “suspect,” according to Mr. Gelman. Mr.
Fluhr responded, “I’m not sure if you’re looking at non-calls, it would
affect the data.” He added that Segal had the data necessary to
incorporate such players, but didn’t consider the data relevant,
instead only focusing on foul calls. Messrs. Wolfers and Price included
all players who appeared in the games they examined.

The NBA does promise to examine non-calls and redo some of the results.  I do not think we have yet gotten to the bottom of this, but my "haven’t read anything but the initial study" intuition (and Steve Levitt’s comments; see also Voxbaby) is that the result of bias will hold up.

Thanks to Chris Masse for the pointer.

Addendum: Wolfers claims the NBA study supports his results.


Quite apart from sample selection considerations etc,I am not entirely sure Wolfers and Price capture the effect they intent to.

Consider these three facts:

First of all, interpreting the rules of the game is to some extent subjective, a truth every sports fan is familiar with. How much of a push constitutes a foul? How aggressive must a player get before having to be disciplined? How important is intention to foul?

Secondly, black and white players have different playing styles. This is not only due to physical characteristics and body types but also due to the different environments where black and white players learn to play basketball.

Finally, this also goes for the referees and their perception of what constitutes foul play.

Combining the above, it's very likely black referees grew up learning to interpret the rules of the game amongst a majority of black players, and so came to tolerate some 'black' playing behaviour that a white referee would not view so favourably - and vice versa. If this is indeed the case, the results in the Wolfers and Price paper could be driven by this fairly innocuous, colour-blind factor - the authors' research design does not allow us to distinguish between these two hypotheses.

You can


In the Daily Pennsylvanian – which they describe as the independent student newspaper at the University of Pennsylvania – was a story by Emily Babay on the Price-Wolfers study. One might expect this story, entitled “Study indicates racial bias in NBA refereeing†, to simply re-hash the story that appeared in The New York Times. Babay, though, offered a bit more. Nestled inside Babay’s article are the following two quotes from Justin Wolfers:

Wolfers, who has read the NBA report, said the league misinterpreted its own study. “Their data shows they actually agree with me,† he said. “Their statistics show there’s evidence of own-race bias.†

I don't see why people seem to like the Wolfers-Price study so much. Here are my problems with it
(which others elsewhere have brought up):

1) "Calls against whites/blacks" isn't defined properly. If you call a foul against a black player, that
counts as an "anti-black" call, but what if his foul was against a black player? It helped one black and
hurt another. That should count as neutral for purposes to gauging bias, but the study recorded it as

2) Often times, the best thing a ref can do is ignore a bona fide foul, which would not show up in the data.
If a ref lets blacks get away with fouls as much as he calls them, that would be bad ref-ing but it wouldn't
be anti-black, yet would still show up as anti-black in the method.

Those together create a big enough margin of error to eclipse the ~5% deviation they found.

To do a real study, you have to define "what is a foul". You'd then have to gather reams of clips of called
AND un-called fouls (the second is harder). Then have an evenly-mixed race panel view them. Then have each
give their judgment. Then have them discuss and see if they change their opinion. Have a good representation
of white-on-white, black-on-black, white-on-black, and black-on-white fouls, and see if their is any
variation. THEN compare to the call the refs made in the actual game.

But then ... that would be real work, wouldn't it?

Given that centers foul out much more often than guards, no, I don't see wh ywe should especially worry.

I wonder if the fact that many of the white players are Europeans may have something to do with it? The European game is somewhat different than the American game, and may involve less fouls.


I am not sure that the paper would have attracted so much attention if it weren't for the moral implications of blatant racial discrimination. If 'discrimination' was seen to be due to an issue other than colour per se, the implications would be far less important. A similar paper demonstrating east coast white refs called more fouls on west coast white players relative to everyone else would never solicit the same reaction from the NBA, the NY times and the wider public.

The most interesting thing about this study is how the NYT chose to spin the results. You might expect them to frame it as implying that refs from the group representing the vastly dominant majority of NBA players were discriminating against the minority of players who only get 17% of the playing time.

But you can see why the Times discarded their normal majority discriminates against minority assumption here. because the majority of players are black (getting 83% of the playing time), and the Times' nurtures a deep hunger to find evidence of The White Man Keeping the Black Man Down (or Black Woman the case may be, as shown by the NYT's enormous enthusiasm for framing the Duke lacrosse team in 2006).

One fascinating finding of this vast study is that the team that plays more black players, as measured in minutes, wins 48.6% of the time. This suggests that there isn't much bias in the NBA, but what there is is in favor of blacks. But that's of very little interest to the NYT!

CNN on the Wolfers NBA paper:

This doesnt need to get into a racial battle. The NBA has the players to do the job. It is not because they are black or white or chinese or whatever. They want the players that will put the points on the score board. Yes, I do agree that some referees are biased toward different teams but not toward the race of the players. They dont have time to discriminate when refereeing. They have to think quickly to make the right call on the situation. I agree in some aspects of the NBA being like street ball because it is a contact sport. What do you think is going to happen in a basketball game? Your going to get hurt, your going to not like calls here and there, but you still play the game. I think its not right to blame the referee calls on race instead of what is really going on.

Please come to shaiya gold, we will give you a great surprise.

Comments for this post are closed