Motivation and IQ, incentives matter

by on April 27, 2011 at 7:29 am in Education, Science, Uncategorized | Permalink

There is an excellent new paper by Angela Lee Duckworth, et.al., and here is the punchline:

…material incentives in random-assignment studies increased IQ scores by an average of 0.64 SD, suggesting that test motivation can deviate substantially from maximal under low-stakes research conditions.  The effect of incentives was moderated by IQ score: Incentives increased IQ scores by 0.96 SD among individuals with below-average IQs at baseline and by only 0.26 SD among individuals with above-average IQs at baseline.

Here is one popular summary of the results.  I interpret the finding to suggest some mix of a) conscientiousness is more important than we think (when we think we are measuring the importance of IQ), and b) there are some smart people, smarter than we often think they are, and they pick and choose their spots.

For the pointer I thank Michelle Dawson.

JL April 27, 2011 at 7:48 am

“Low-stakes” is probably the operative word here. I haven’t read this paper, but Duckworth has previously published a really incompetent or, rather, dishonest anti-IQ paper called Self-Discipline Outdoes IQ in Predicting Academic Performance of Adolescents (no it doesn’t except in Duckworth’s range-restricted sample).

Andrew' April 27, 2011 at 8:59 am

In elementary school I found out that doing well just landed you harder work. I have found the same thing in grad school.

Todd April 27, 2011 at 9:29 am

The implicit premise of both the paper and this post seems to be: IQ tests are silly, and something that a lot of people do not take seriously. It’s tough to find a test that adults take wherein a large percentage of the test takers have little or no incentive to do well because most people understand that the “results” are hogwash. But I guess IQ tests fit the bill. Hey, let’s incentivize and then study the relative results on personality tests, mall surveys, marketing questionnaires and the like. I bet, across the board, people do better on “tests” they don’t take seriously when they start to take them more seriously. Quite an insight.

TGGP April 27, 2011 at 9:35 am

Sounds a bit like Karl Smith’s theory:
“I am sticking a flag in the sand and declaring my hypothesis that the genetic component of intelligence is preference. That is, smart people are people who like intellectually stimulating experiences.”
His theory helps explain why heritability rises over time.

lemmy caution April 27, 2011 at 1:03 pm

The increase in heritability over time could just be a matter of variations in development. Think about height. If people have growth spurts at different times then heritability of height will rise over time as people approach their maximum adult height.

TGGP April 28, 2011 at 1:03 am

Does the heritability of height increase with age? I don’t know.

Ken Rhodes April 27, 2011 at 10:11 am

“Incentives increased IQ scores by 0.96 SD among individuals with below-average IQs at baseline and by only 0.26 SD among individuals with above-average IQs at baseline.”

What I read there is that smarter folks already tried harder, which then leads to the conclusion that smarter folks always care(d) more about seeming smarter, whether there was some external incentive.

Seems right to me. The smarter of my friends try harder on things that have NO external incentive, like the crossword puzzle in the newspaper.

John Mansfield April 27, 2011 at 11:18 am

This reminds me of the case of students at a high school in Torrance, California. As I recall it, the juniors at a high-performing school lost the school spirit and decided to throw a state standardized exam that had no consequences for them personally.

Greg Ransom April 27, 2011 at 12:34 pm

These results always seemed common sense to me.

It’s telling that “data” trumphs common sense among academics, until someone comes up with a way to beat academics over the head with data vindicating the obvious.

This happens more than economists and human science people wish to admit.

I’m sure you could put together data on the phenomena.

Todd April 27, 2011 at 2:36 pm

Just give economists a test they don’t care about and then incentive it. The data is both tautological and ironic.

conchis May 3, 2011 at 11:37 am

For any piece of data analysis, there will exist a non-empty set of people for whom the results confirm ‘common sense’, and another non-empty set of people for whom the results disconfirm ‘common sense’.

The reason data trumps ‘common sense’ is that without it there’s often little reason to think you’re in the first group rather than the second.

ad*m April 27, 2011 at 1:19 pm

How many angles on a pin.

Isn’t the point whether the ‘incentivized IQ test’ better predicts such things as income, success, health etc than the ‘non-incentivized IQ test’ currently does. Few care how you measure the number, just as long as the number is reproducible and relevant.

It may not be what Duckham hoped for, but I bet you that individual variability in the novel ‘incentivized IQ test score’ is just as much heritable, genetic, etc. as the standard old IQ score. Maybe even more so…

ad*m April 27, 2011 at 1:19 pm

sorry, ‘How many angels on a pin’. iOS.

stephen April 27, 2011 at 1:29 pm

I don’t have access to the paper, but I read the article (…among other things):

“By constructing a series of computer models of the data, the team found that higher motivation accounted for a significant amount of the differences in IQ scores and also in how well IQ predicted later success in life. “

So, there was a meta study of about N= 2000, then another study of about N = 500, then all this was used to make a series of models. Out comes the “Hedges’s g” values.

Small samples, multiple models used….I am not going long on the robustness of this one study.

Aside from that, the idea that motivation and talent are required for success is obvious. Has anyone ever claimed otherwise?

The Anti-Gnostic April 27, 2011 at 1:56 pm

The idea is that IQ

stephen April 27, 2011 at 1:39 pm

BTW, I wonder how well the resultant model (I am assuming they settled on one) performs at predicting future data, now that it has been fit. I doubt this will happen though.

foo April 28, 2011 at 8:52 am

The vast majority of papers on IQ that make it to publication outside of specialist venues are those that take a contemptuous attitude towards the topic and refuse to engage the totality of evidence.

In the meantime Paul Thompson at UCLA and BGI are respectively moving forward the IQ/neuroscience and IQ/genomics fields respectively.

Keep pretending that doesn’t exist.

Jason Malloy April 28, 2011 at 7:27 pm

“The Bell Curve by Richard Herrnstein and Charles Murray, which suggested that the lower average IQ scores of some ethnic groups, such as African-Americans and Hispanics, were due in large part to genetic differences between them and Caucasian groups.”

My, that’s some top reporting from the news arm of Science right there. The Bell Curve did not state that lower Hispanic IQ was due to genetics or anything else.

I wonder if the leading science journal in America will have as much class and integrity as the New Yorker when it apologized for Malcolm Gladwell’s embarrassing claim that TBC advocated quarantining low IQ people on reservations.

gFactor May 7, 2011 at 1:20 pm

I interpret the finding to suggest some mix of a) conscientiousness is more important than we think (when we think we are measuring the importance of IQ).

This statement has implications for the SAT and ACT, which are effectively tests of general intelligence (g) as they correlate strongly with g and IQ (~.80).

Coyle (Intelligence, 2008; PAID, 2011) and colleagues separated g variance (related to IQ) and non-g variance (related to personality traits such as conscientiousness) in the SAT and ACT. Consistent with the view that “conscientiousness is more important than we think (when we think we are measuring the importance of IQ)”, they found:

(a) The g component of the SAT and ACT predicted college GPA. This finding is consistent with a century’s worth of evidence documenting the predictive validity of g and IQ.

(b) The non-g component of the SAT and ACT also predicted GPA. This finding is inconsistent with (or in tension with) the assumption that cognitive tests derive their predictive validity mostly or exclusively through g. It also suggests that the SAT and ACT measure something besides g that contributes to their predictive validity (e.g., conscientiousness).

FYI: Sailer and Caplan reference this post:

http://econlog.econlib.org/archives/2011/05/what_is_iq.html
http://isteve.blogspot.com/2011/05/bryan-caplan-writes-years-ago-i-told.html

Comments on this entry are closed.

Previous post:

Next post: