How meritocratic is meritocracy?

There is some new research by Castilla and Bernard:

In this article, we develop and empirically test the theoretical argument that when an organizational culture promotes meritocracy (compared with when it does not), managers in that organization may ironically show greater bias in favor of men over equally performing women in translating employee performance evaluations into rewards and other key career outcomes; we call this the “paradox of meritocracy.” To assess this effect, we conducted three experiments with a total of 445 participants with managerial experience who were asked to make bonus, promotion, and termination recommendations for several employee profiles. We manipulated both the gender of the employees being evaluated and whether the company’s core values emphasized meritocracy in evaluations and compensation. The main finding is consistent across the three studies: when an organization is explicitly presented as meritocratic, individuals in managerial positions favor a male employee over an equally qualified female employee by awarding him a larger monetary reward. This finding demonstrates that the pursuit of meritocracy at the workplace may be more difficult than it first appears and that there may be unrecognized risks behind certain organizational efforts used to reward merit. We discuss possible underlying mechanisms leading to the paradox of meritocracy effect as well as the scope conditions under which we expect the effect to occur.

The link is here, and for the pointer I thank Samarth Bhaskar.


I don't deny the finding but it's hard for me to internalise this without some idea of what is the mechanism... Is the alternative to meritocracy less subjective criteria like seniority?

XX good, XY bad... unless one identifies as transgenic.

With the backdrop of World War II, Diana frequently pontificated on the need to preserve the morally supreme ideals of democracy from the threat of Nazis, even in private to her sister. When doing so, she made no attempt to reconcile her democratic sentiments with their Amazonian system of hereditary monarchy, to which Diana herself was heiress apparent. Likewise, in each of the first two 1-hour episodes, Diana asserted that the Nazis had no regard for their women, ignoring the vastly lower value which her own people placed upon men.

I guess the argument is that people are really, really bad at judging merit, so much so that when they try they end up doing a worse job than chance.

Theory 1) intuitively, they judge the men to be more qualified. Thus under the meritocratic condition, they assumed the men are indeed more qualified.

Theory 2) meritocracy is percieved stands for multiple traits. The intuition was that those traits are more common in the men judged.

Check the actual study's numbers. In 2 out of 3 experiments, the results were actually more meritocratic when the judge was primed with meritocracy. That's because the non-meritocratic condition led to outcomes where women were overpaid relative to men. The meritocratic condition led to men being overpaid, but less so than women in the non-meritocratic condition. It can't be said that meritocratic culture leads to less meritocracy, unless you somehow judge that underpaid men are somehow less transgressive than underpaid women.


"The results of studies 1 and 2 supported our prediction that women would receive smaller average bonuses than men in the meritocratic condition. One unpredicted finding in both studies, however, was that women received greater average bonuses in the non-meritocratic condition."

I was going to say, I wonder how this jibes with the recent finding of extreme pro-female bias in science

How did they control for the meritocratic probability of pregnancy and the extended absence associated with it likely to occur in the future?


Doesn't sound reasonable to me to penalize women for the possibility of them becoming pregnant in the future.
I understand that maternity leave can be understood as a salary benefit but some women might choose not to "cash in" on it.
What about the career driven women who have no intentions of having kids?.
AFTER they had a baby, though, that's a different story.
Sort of like a student loan, women pay later through smaller wages.

This "new research" was published in 2010.

Having followed the replication crisis discussion in psychology, I have become extremely skeptical about all research findings that have not been independently replicated multiple times and then meta-analyzed using tests for publication bias. As far as I can see, Castilla and Bernard's findings have not been reproduced by others, so there's no reason to think that they are robust. But if the effect were shown to be real, I would still wonder if a study of students assessing imaginary employees in imaginary companies has any ecological validity.

+1 Well said.

You just can't cheat biology: men never get pregnant. With regards to pregnancy, which is a risk from the employer's point of view, women are probably not underpaid in most professions (a hypothesis).

'which is a risk from the employer’s point of view'

A fascinating thesis - so, male employees who just quit are not a risk like pregnant women?

'male employees who just quit are not a risk like pregnant women?'

Are women not also a risk to quit? Pregnancy is an additional risk which men don't suffer.

Well, let's see - some employees will just quit, which is a risk to the employer (though my interest in sympathizing with an 'employer' is pretty much close to zero - it isn't as if employers sympathize with employees much, otherwise this discussion would be close to moot).

A subset of the people that will 'quit' are women that get pregnant (do note, however, in essentially all industrialized countries outside of the U.S., a woman has a right to return to her job after becoming a mother, thus minimizing that risk to a plannable time span on the part of an employer - assuming, of course, that the person does not join the group of people who quit). Is that subset of women who quit really that large when we are talking about such significant private American employers as Walmart, Yum! brands, McDonald's, IBM, and UPS (the top 5)?

Because the one company in that list which actually employs a large number of skilled employees is IBM - which is notable for its American maternity benefits (though those American benefits wouldn't match the German legal minimum - and we all know what a socialist hellhole Germany is).

Women are far more likely to quit (or take extended leave from) their jobs than men, due to pregnancy and caring for children. This is blatantly obvious.

See one discussion of that here for example:

What sort of response one thinks is appropriate to this is a matter of genuine debate, of course. But we should all admit reality at the outset, not deny it for ideological reasons.

my interest in sympathizing with an ‘employer’ is pretty much close to zero


(do note, however, in essentially all industrialized countries outside of the U.S., a woman has a right to return to her job

Yes, the condition of labor markets in Western Europe is such a wonderful example for us to follow.

And they are so gloriously fecund in Europe as well.

"Women are far more likely to quit (or take extended leave from) their jobs than men, due to pregnancy and caring for children. This is blatantly obvious."

No, that's only true of white women. The one thing I remember from my Labor Economics classes was that the most reliable sector of the labor market is black married women. If you ask me to cite my source, I can't, this was a long time ago and my texts are buried in the attic and I'm not getting them out. And also, I will just respond by asking you to cite yours. "This is blatantly obvious" is NOT proof, in the real world.

Married white women are the least stable element in the labor market. At least 'twas so when I went to the ILR school.

White women constitute the majority of the female labor force in the US, so you're technically in agreement with the parent comment.

"White women constitute the majority of the female labor force in the US, so you’re technically in agreement with the parent comment."

No, I'm not. And you are an idiot.

I enjoyed this second paragraph immensely.

Numbers? It would be great to compare "pregnancy risk" to "average man risk". Average man risk should account accidental death, incarceration, temporal and permanent disability.

Ps. what about women over 45-50? They become pregnant as much as men.

But it's not just pregnancy risk, it's lower career dedication risk:

The women provided 33.0% fewer services per year than the men in family and general practice (p < 0.001), 25.0% fewer services in general internal medicine (p < 0.01), 22.1% fewer services in pediatrics (p < 0.05) and 22.3% fewer services in psychiatry (p < 0.001). Total billings by the women in these fields were also significantly less than those of their male colleagues.

And note, that childcare doesn't seem to be the issue. Not only are these well-paid women who could easily afford all the child-care help they desired, but:
Contrary to conventional wisdom, shorter work weeks for women physicians are not the result of child care responsibilities. Nor would higher earnings encourage women physicians to work longer hours. Instead, we found significant work reductions among married women physicians (but not men), implying subordination of careers by women where combined family incomes are high.

Oh, and just to pre-empt the the inevitable comment claiming that that in Germany all the women are ambitious (and all the children above average):

In a report published on its website on Tuesday, the German Federal Institute for Population Research (BiB) said that a markedly lower percentage of German women between the ages of 25 and 59 had full-time jobs compared to their peers in many other European Union member states. The study said only 41 percent of female Germans in that age bracket worked full-time, while the EU average currently stood at 48 percent.

And, of course, 48% itself must be far below the male full-time rate (even in the EU).

Not the right sample. When you hire people you are not selecting among all the women between 25-59, you are selecting from the women who actually care to apply for a job. That's a subset of the whole set. You can not evaluate individual performance by the whole population performance.

The proper statistic is tenure, look at 2014 numbers. In table 1 for Jan 2014 median years of tenure with current employer for men and women are really close.

"The proper statistic is tenure, look at 2014 numbers."

No, what these numbers show (both in the U.S. and Germany) is not that women are leaving the labor force entirely (though many do), but even those in the labor force are much more likely than their male counterparts to work part-time. And this is not -- as the female physicians demonstrate -- because they cannot afford to pay for child-care, cleaning-services, etc, but because they prefer part-time work when it's financially possible. And more power to them, if that's what they want, but a meritocratic employer has good reasons to prefer employees who are nominally at 40 hours but who often work 50 vs those who will, in a few years, likely want to cut back to 30 hours and coast.

Depending on the type of incarceration, this can actually lead to better performance, believe it or not. My dad runs a small business with a dozen or so rather low-skilled employees. Many of them over the years have been in and out of jail, mostly for repeated DUI offenses. He's actually quite happy when one of them gets sent to jail again, because odds are he can get that person out on work release during the day, and whatever the length of sentence, he knows that particular employee will not call in sick unless absolutely necessary, won't show up late or hungover, won't ask to leave early, etc.

My coworkers (male and female) are at essentially zero risk of being incarcerated.

The pregnancy risk does apply earlier in the career, not later. A woman who has already had kids and stuck with her career is pretty close to a man who has already had kids and stuck to his career.

This is totally unsubstantiated and probably wrong, but I can't help feeling like there's a lot of low-hanging meritocratic "fruit" to be picked in white-collar corporate America simply by improving interview processes and the logic that governs when to release a given employee.

I say that because, in the context of tech, I've worked with some objectively terrible co-workers who have nevertheless kept their jobs for shockingly long periods of time and who, after leaving a given job (voluntarily or no) are usually able to interview well enough to get another one.

So, at the very least, there are a significant number of companies out there with "broken" interview processes that fail to categorize these individuals as not worth hiring.

In order for this to indicate lack of meritocracy, though, there would need to exist other individuals who could do these jobs better who aren't able to get them. Possibly because they never considered the field in the first place (and so never acquired the necessary education/skills) and are consequently now employed in some other (less productive) capacity.

Maybe there are no such individuals. But these coworkers were just so bad that it's hard to imagine there aren't folks out there (maybe waiting tables or cutting grass) who couldn't do a better job if they had the requisite education and training.

The tech interview is completely broken. A bunch of people who sit and think "interviews should be run this way" with no more evidence than "I (think I) would do well at this kind of interview, and I'm a good employee, and we need more of me."

Alluding to Cowen's second law, there is really a whole big pile of research into this. The tldr is: if you are American, the best interview method is a work sample test. If not American, combine a work sample test with an IQ test.

I tend to think interviewing sucks because coming up with a "good" interview process (questions, maybe a project, etc.) actually takes some effort, and because the people tasked with designing the process don't see the advantage to methods that take more time and forethought to set up.

I did interview with one company, though, that sat me down w/ a computer in a room by myself for 3 hours and had me work on a toy project. Then they had a ~1 hour code review of my work in front of a panel of their developers. It was so, so much more useful than having someone code up Fibonacci or binary search in pseudocode on a whiteboard.

IQ tests reveal little to nothing about the sorts of traits that cause job failure in skilled jobs. The most common being lack of skills, which is why a skills test is pretty necessary to weed out people with inflated resumes. Beyond that, "bad" employees are not usually bad because they are stupid, but because they are abrasive, undependable, dishonest, lazy, etc. That's a problem with character that no intelligence test can catch.

Generally I agree. But for specific jobs I'm not as willing to discount the importance of IQ. In my particular field, software development, and in relation to the specific individuals I mentioned, I think it really was an aptitude deficit and not a skills deficit. I wish it weren't so, but there's a difference between a +1 sigma individual and a +3 sigma individual.

btw, at least one of the coworkers I spoke of was a generally pleasant person and reasonably hard worker. She was just clueless.

IQ tests reveal little to nothing about the sorts of traits that cause job failure in skilled jobs.

[Chuckles]. You're due to be accused of 'not noticing things' or 'IQ denialism' in the next issue of HBD Review.

Re: So, at the very least, there are a significant number of companies out there with “broken” interview processes that fail to categorize these individuals as not worth hiring.

Not really. HR reps and hiring managers do not have the powers of the Oracle of Delphi or of an Kwisatz Haderach. The best they can do is judge a candidate based on a resume, an interview (maybe more than one), references and maybe a skills-based test. And every hiring manager who's been at it for a while will have horror stories to tell about someone who did very well by all those criteria, and was still a disastrous hire. As with everything else in life, there are no sure things.

Agreed. There are no sure things. But I just can't believe the majority of interviews I've participated in (both as interviewer and interviewee) are the best possible means of evaluating a candidate.

Depends very much on the interviewers. Some people are natural empaths and will gain a very good sense of the person they are talking to after ten minutes. Most people aren't however. And of tech is famous for introverts and such people tend to hate doing interviews so they read a bunch of rote questions, grade according to whether they get expected rote answers, and never really engage at all with the interview subject.

Some interviewers / companies don't even do that. Where I'm now, the interview is almost entirely a "culture fit" / "are you a jerk" evaluation. We do "okay" at weeding out people who are jerks. However, we do a terrible job of evaluating candidates' actual technical skills. Consequently we end up with developers who are easy going and pleasant to work with but don't produce quality output.

One place I used to work, there was precisely one wretched hire during my years there: the department head. Red flags have for years been waved in front of her superordinates and the personnel department, part of the staff went union, and there has been a hemorrhaging of salaried staff for over seven years which would have been quite irregular under her predecessor (while her wage staff avoid her in the hallways). Net effect on the thinking of the succession of her supervisors (she's had three) and of the personnel director (there have been two) = nil.

Our disaster interviewed much better than the competition. It's just that no one with the authority to review her contract in the intervening years cares to pay attention to the information percolating up about her actual performance.

are usually able to interview well enough to get another one.

I know the type. I think they predominate in trades which do not have robust operational measures of competence. I would not have expected it in software...

My clients often do business with a large hospital company, most of the lawyers (inside and outside) for which are women. I thought it curious, and was told that the company prefers women lawyers because they are perceived as being more loyal. Meritocracy is overrated, if for no other reason than it's subjective - meritocracy is in the eyes of the beholder. I once was a partner in a law firm that, like most, was a meritocracy, dividing profits based on the partners' productivity (based on a formula), but the compensation committee (actually, the senior partner) had discretion to take into account subjective factors (like loyalty, I suppose). One year I had been particularly productive and, according to the formula, I would have been paid X. However, the senior partner exercised his discretion and lowered the amount because I hadn't spent much time training the young lawyers. The following year I took on two young lawyers and spent lots of time training them by having them work on matters for my clients. I was as productive as in the prior year but my profitability declined because much of the compensation paid to the two young lawyers was charged to me. The senior partner commended me for training the young lawyers, but said he didn't want to deviate from the objective formula because it might open it up to questions of favoritism. I moved on, but my point remains that meritocracy can be highly subjective. As for the loyalty of women lawyers, I suppose it too depends on the meaning of loyalty.

It's pretty rare to actually want the very best candidates and be willing to pay the premium for them.

In professional sports, where there is only one winner, the person who can hit the ball 34% of the time versus 33% of the time could be worth a million dollars more a year.

But even in head-to-head legal confrontations, being marginally smarter than the other guy doesn't necessarily translate into winning. Often the facts and the laws are what they are, and the trick is to be competent enough to avoid the unforced errors.

"when an organization is explicitly presented as meritocratic"

There is any organization that does not claim to be meritocratic?


A classic example of begging the question.

This study is not a thing.

Social scientists manipulatng variables and coding results until they get a publishable finding is not research.

There's a reason why double blinds are used. This is the exact opposite, it's double biased. It's not even good for a focus group type result.

It's just log rolling that produces career advancing derp, not knowledge.

Feudalism was meritocratic.

It seems to me that there are real, good reasons for some employers to favor male employees over female ones with identical performance records -- the main one being the greater likelihood that a female employee will quit, or take long leaves of absence, for family-related reasons. These reasons will of course be stronger in places where the employer is required by law to subsidize those choices by their employees (whether or not men are also entitled to the subsidy if they want it).

Control for this problem and I think you'll find that meritocracy really is meritocratic -- and that there is no gender pay gap.

I apologize for clumsy reuse of the word "reasons". Make that "for family-related purposes" and it reads better.

With inadequate and unrefined graphics, your site can be seriously ruined.

Comments for this post are closed