Wealth and PISA scores: why doesn’t money help U.S. performance more?

The data was provided to The WorldPost by Pablo Zoido, an analyst at the Organisation for Economic Co-operation and Development, the group behind PISA. It shows that students’ wealth does not necessarily make them more competitive on an international scale. In the United States, for example, the poorest kids scored around a 433 out of 700 on the math portion of PISA, while the wealthiest ones netted about a 547. The lower score comes in just below the OECD average for the bottom decile (436), but the higher score also comes in below the OECD average for the top decile (554).

“At the top of the distribution, our performance is surprisingly bad given our top decile is among the wealthiest in the world,” said Morgan Polikoff, a professor at the University of Southern California’s School of Education who reviewed the data.

The data also makes for some jarring comparisons: Canada’s fourth decile performs as well as Chile’s top socioeconomic tier. Taiwan’s bottom sliver performs about as well as Montenegro’s wealthiest 10 percent. Vietnam’s bottom 10 percent slightly eclipses Peru’s top 10 percent. And the poorest kids in Poland perform about on par with Americans in the fourth decile.

The article is by Diehm and Resmovitz, pictures at the link, hat tip goes to @ModeledBehavior.  One possible implication is that the so-so U.S. performance on these tests is about culture in a way that just doesn’t reduce to poverty for the lower deciles.

Comments

This looks like solid evidence that the ability to perform well on tests of mathematics is vastly overrated as a path to wealth.

......once you reach wealth you may no longer value the tools needed on the path to wealth.

Math not being one of them anyway

Or is it solid evidence that wealth is vastly overrated as a path to performing well on tests of mathematics?

I think both are true since the variables do not seem well correlated.

It does seem to call into question the assumption that high scores on such mathematics tests are especially desirable for economic performance. Computation does seem like the human cognitive skill most easily replicated by computers.

Lifting a heavy object over and over is also something that can better be done by machines. But humans do that as a means of excelling at other tasks.

It would be very interesting to see what proportion of economically very successful people can either bench press 200 lbs or calculate the volume of an ellipsoid.

Am I allowed to look up the equation?

+1
Maybe we should consider the wealthiest countries as the best effectively educated. I do not think that PISA measures anything important. Most effective education occurs outside of schools, like on a job!

Is it really possible that 90% of Shanghai-China scores above 90% of US students?

And what is this nonsense of China using only a single (not very representative) city for it's comparision.

The table doesn't really say that. It depends on the within-decile variance.

Is that a coding error or is the fifth economic decile of Lichtenstein indeed outperforming all the richer deciles above it? If not an artifact, it might be interesting to know why. Offspring of diplomats?

Could just be a statistical artifact. Lichtenstein only has like 35k people. 10% of the US population are minors, so assume the same there, and they only have 3.5k kids. The PISA test is only given to 15 year olds, so that's like 200 kids. A decile is then only going to be like 20 people, which is a pretty small sample size. One smart, low-income kid could throw the numbers off.

The underlying premise is that if you have wealth, you have the resources---but do you have motivation to work hard.

Have you ever seen kids of wealthy parents who graduate from college with soft degrees in theatre arts, museum management, and women's studies.

Last year there was a NYT article about kids who were graduating from college. One of the interviewees was a young women who lamented the difficulty of her friends finding a job, saying how grateful she was that she could work with her dad (as a real estate developer) preparing the sales literature using her art degree.

The underlying premise of what? The article? More likely it's IQ that matters and not wealth and people in the U.S. are not smarter than elsewhere although they are wealthier. (Actually they probably are a little smarter and hence every U.S. demographic outscores natives in their country of origin)

IQ does not equate with equal opportunity.

Cliif, perhaps you might read the quote of Prof Kolikoff above if you can't find the premise.

?? You mean the quote in the post? I read it... the top 10% wealthiest Americans don't do as well as the top 10% in other less wealthy countries. So what? Why does that mean the wealthy in the U.S. are not motivated? Sounds like a complete fabrication.

Obviously wealth matters, since there is a correlation between wealth and performance within every country. However, the inter-country differences matter even more.

Does it matter if you control for IQ??

Obviously wealth matters, since there is a correlation between wealth and performance within every country.

Care to think about that some more?

Yep, you are right. I messed up on that one. I do think it suggests that wealth matters. But it is not definitive.

This is a related read from Talking Points Memo:

http://talkingpointsmemo.com/cafe/book-club-poverty-doesn-t-explain-poor-american-school-performance

Race/ethnicity dominates income or socio-economic status is predicting test scores.

America traditionally is a business economy. You need basic math, but there's diminishing returns. You get rich by delivering what your customers need, which includes lots of skills that are not math. Rich nerds are a phenomena of the past 20 years.

East Asia's exam system filters the elite much more strongly than in the West and certainly the U.S. Fewer low IQ people make it. Which I thought was why America was always considered the land of opportunity, because there were these rigid systems designed to keep people from wealth or power.

Rigid systems in other countries.

+10
The story of the average Joe who became incredibly weathy through a mixture of hard work and luck is almost unheard of here in France.
Here poor people who make it are always extremely smart and found a way through exams and academical success despite their poor cultural background.

And yet in Germany, it is considered to be quite common.

It most certainly applies to the five founders of the world's most successful business software company, for example, as an extreme example of 4 well known more or less self-made billionaires (the fifth left the three letter company quit due to health problems before the company grew to dominate the ERP market).

Is the alignment of genetic talent & wealth equally well-matched in all nations. The best outcomes possibly arise when the smartest kid is given the best instruction and tools. Motivation which probably anti-correlates to wealth also must be compensated for. Seems a complex model.

Seriously? Motivation probably anti-correlates to wealth? I doubt it.

I could be wrong. Are there studies on this? I can see arguments both ways or perhaps it is uncorrelated. I'd love to know.

Look at the numerous royal and noble families in oil-producing nations. Their scions number in tens of thousands, and seem to live in a world of casinos and golden palaces.

Not a single of them made it as an outstanding scientist or engineer, as far as I know, even though there statistically must be numerous talents in such a big group of people.

On the other hand, impoverished Lebanon and its diaspora seems to produce very original thinkers and enterpreneurs.

And how many outstanding non-royal Saudi scientists can you name?

Culture is the key here, but it's hard to tease out from race, class and national origin. When you put data like this in the context of Chua and some of Sailer's work on comparative education, it makes perfect sense. Some nations have lower classes with a pro-education culture. Some nations don't. In multi-cultural nations like the US, we have a lot of under-performing cultures, but they mostly do better here than they do anywhere else. Asians in America outperform most asian countries, hispanics outperform Latin America, so on and so forth (selection bias notwithstanding).

The easy and stupid answer is that we don't throw enough cash at certain groups for education. The harder, more complicated view is that you have to change culture to change the outcomes, and there isn't a government program for that. Once you understand why Japanese-Americans over-perform, and Hmongs under-perform, you are on your way to seeing the solution. I am skeptical it is anything amenable to collective action. The last time we tried to change a culture to improve education was the missionary schools for indian children. We don't have the will or amorality for that these days, and the results weren't all that great anyway.

I think that the very "throw the cash on the problem" idea is wrong. It is a very American way of thinking, but there does not seem to be good empirical evidence that cash can do wonders.

In personal life, people seem to understand the limitations more. For example, no amount of cash would make me a good dancer, and no amount of cash would make one of my acquaintances a good singer; but it went the other way easily for both of us respectively, not much cash needed.

That may be so but culture can be compensated by other means. I've seen cousins from the same immigrant family, one ending up in a good school & other in a crappy one. They grew up in very similar cultures & levels of parental education but ended up with grossly different educational outcomes.

That's why they invented the concept of the "average."

The top 10% may not be fine-grained enough to pick up on the U.S. wealth "advantage" at the high end due by virtue of its greater inequality. I wonder, for instance, if the trend of U.S. students being slightly below the OECD average (after adjusting for income) would still hold if only the top 1% were considered? (Maybe it would.)

The 90th percentile of U.S. households makes about $145,000 a year. Exactly how much wealth do you think is needed before it becomes an advantage?

Yes, but the top 10% in other wealthy countries also have high earnings. Suppose the ratio of the average earnings of a "top 10%" U.S. household vs. the earnings of a "top 10%" Swedish household is 1.5. I would expect that ratio to be higher if you were comparing the top 1% of U.S. households to the top 1% of Swedish households. If wealth still correlates with PISA scores even at the margins then a comparison of the top 1% should be more favorable for the U.S. than a comparison of the top 10%.

My Straussie-sense is tingling!

Personal experience of schools with three primary school children:

Canada - little work and low expectations
Singapore - lots of work and high expectations

Perhaps Poland is similar to high scoring Asian countries in that newly won liberty motivates a generation to succeed, while the US and others drift into complacency and eventual senescence.

Science and education need not be poor in unfree countries. In the red block, social sciences were politically castrated, but hard sciences were remarkably fine, not least because the regimes wanted to keep up with the development of weapons. Many Czech scientists even improvised their own laboratory equipment from toys etc., and the necessary initiative might even improve the process (the contact lens was developed by Prof. Wichterle using a toy set).

What was lacking was the production sector, which often could not adapt the discoveries into practice. But the science itself was quite well off.

America is like a factory in that it produces a lot of average educated kids

Filling in for Steve Sailer, let me ask the obvious. How do PISA scores correlate with wealth once you control for racial differences in scores on IQ tests?

Probably depends on what racial groupings you use.

By looking at deciles, this innuendo be revealed as the whole cloth that it is. US low performance in both rich and poor deciles cannot be due to "unfavourable ethnicity X", unless it's equally represented among both rich and poor.

Your statement is false on its face.

And what are you referring to as "innuendo"? It is fact that every U.S. demographic does better than natives of that demographic in their home country.

For the richest US decile does one need to control race? How many Black / Hispanics make it to our top 10% richest?

How does one apply Sailer's usual rhetoric ("stupid blacks & Hispanics spoil our scores") to explain why the richest 10% of US is still ~30 points behind Germany, Poland, Switzerland etc.

@Rahul (Why do I have to the one to say this) if your bottom 20% are mostly non-white and your top 10% are mostly white then you might have someone down to the 15th percentile white in the top 10% for the USA.

I thought we couldn't tell anything about quality of education from standardized tests.

Accepting as a predicate that there is no biological determinism and that different cultures assign different values to critical wealth creating variables such as work ethic, education attainment, etc., in a multicultural nation, you will have outcomes varying by cultural traditions. It is a given that there is a high correlation between cultural traditions and ethnic origins and this allows a different way to view the PISA results.

This is an international version of the Texas-Wisconsin paradox. Every ethnic group in Texas scores higher on NAEP than their counterparts in Wisconsin but overall Texas has a lower score than Wisconsin. The reason is that the gaps between cultural/ethnic groups are large and Texas has a different and much greater diversity than does Wisconsin. The conclusion is that the Texan education systems are producing better results for all their different students than does Wisconsin, even though the overall average is lower.

If you track US PISA results by continent of origin (as an extremely rough proxy for culture), the US has a higher score for every region on both maths and literacy. White Americans score higher than every country in Europe (except for Finns in reading); Asian Americans score higher than every country in Asia (save Shanghai which is almost certainly a sample selection issue), and American Hispanics score higher than every Latin American country. There are no African countries in the PISA results but African Americans score higher than both Brazil and Trinidad & Tobago (two countries with a very high African diaspora population). The American averages by continent of origin are 25-55 points higher than the respective continental averages.

So different cultures are better or worse adapted to creating wealth and better or worse oriented towards education attainment. The US has a population mix that creates enormous wealth overall and education systems which outperform their cultural/regional origin counterparts everywhere else. The only reason for the middling results overall is that the US has such a large mix of cultural traditions.

"This is an international version of the Texas-Wisconsin paradox."

That's an interesting idea and several posters have assumed it's true, but has it been verified with respect to the PISA scores? Have the results been broken down with respect to different cultures and scores across the board.

I have done it for literacy scores and confirmed. Haven't done the maths scores but have read research elsewhere indicating that the results were broadly the same.

Do you have a link pointing to your results?

I haven't published my analysis in detail. Here is one from a Swedish economist that looks at 2009 PISA results overall (literacy, science and maths) with much the same results. http://super-economy.blogspot.com/2010/12/amazing-truth-about-pisa-scores-usa.html

And, of course, there is Steve Sailer's pictogram:

http://isteve.blogspot.com/2013/12/graph-of-2012-pisa-scores-for-65_4.html

After looking at the scores, it appears that reality has an Asian bias, at least with respect to PISA math scores.

Do these tests count in determining a student's overall grade?

If not, many students will not be motivated to give 100% on the test - and that will vary between cultures.

Throwing cash at the problem? That's not on the article. It only talks about deciles of income, not about the % of parents income or government budget used in math education.

Quoting Sala i Martin, the US can benefit from the best minds out of a 7 billion world population. If the brightest students in math are from Shanghai, the surge of China graduate students in the US is an example of success.

One way to fall from the top tier to the bottom is to continue to say "data is," making educated folks barf.

Personally, I think everyone should refuse to go along with the scientific game of pedantically insisting on "these data".

If you are considering something as a single entity, then it is correct to refer to it in the singular, even if it happens to be made up of many individual items--e.g. "this team". That is almost always what people are thinking of when they use the word data. The fact that it comes from a Latin plural is irrelevant. Do you also say "My agenda are full today?"

'Taiwan’s bottom sliver performs about as well as Montenegro’s wealthiest 10 percent'

Considering that Montenegro has a population of 625,000, and was a not exactly disinterested observer of the Balkan wars of the 1990s, the idea of comparing Montenegro to Taiwan (population of metro Taipei alone - 6,950,646) is amusing.

These are income deciles, not score deciles. I'd like to see score deciles. To me this just looks like we're seeing high income Americans are less likely to be 'academic/engineering' types than in Japan/Korea/Germany/Finland, which then gets passed on to kids.

The title of this post is wrong: money helps American math performance quite a bit. The 119-point difference between the first and tenth deciles is large. Note also, that difference is almost exactly the same as the difference between the OECD averages for the first and tenth deciles. Money makes a hell of a lot of difference in American math performance.

The question of the post ought to be: why does the US lag the OECD in math development? I think most commenters are over-complicating the answer: Americans are more interested in wealth than knowledge. Foreigners and immigrants have been commenting on it for 300 years; it's not a new tendency or anything.

The real surprise for me, and the real pickle for the race "realists", is how poorly lily-white Chile, Uruguay, and Argentina fare.

"The real surprise for me, and the real pickle for the race “realists”, is how poorly lily-white Chile, Uruguay, and Argentina fare. "

Was that meant as irony? Because, Chile, Uruguay and Argentina are not even close to lily-white. I'm assuming by lily-white you mean northern European ancestry with fair skin.

Comparing school spending? Let's first strip out things in school budgets that most other countries don't have, starting with inter-school sports programs, for which a district must pony up for coaches and athletic directors, equipment, uniforms, stadiums, lighting, and liability insurance, and often very expensive training facilities for teams that typically include around 1% of a school's student body. Then, let's look at school busing, which in most of the western world is integrated into the public transportation system. Then come school lunch and activity programs, college and career counseling, psychologists and nurses, etc.. I'm not saying that these are bad things to have in a school, but they do make comparison of school budgets internationally more difficult.

Should legal fees be included in that list, too?

How do they get the parental income figures to form deciles from? The students probably don't know and the parents (at least in the US) are probably very reluctant to say. If they're using something like IRS reported local income figures by district from the US and assuming every child in a district goes to the local high school, that's not very accurate. Schools report the fraction of students getting free or reduced-price lunches but that's only a guide to the bottom decile or two. I hope someone has chased the links enough to know!

My own standardized testing is in the past (as is that of my children), but I think I would have remembered being asked for an income figure and I know I would have done my best not to give one.

"Vietnam’s bottom 10 percent slightly eclipses Peru’s top 10 percent."

The PISA appears to be a pretty good test relative to the immense challenges involved, but we shouldn't get carried away with over-interpreting it. For example, Vietnam scored well, but one way they did it was by not testing 44% of the target population. Likewise, Mexico lost 37% of the 15-year-olds they were supposed to test, Peru missed 28%. The U.S. 11% and high-scoring Finland 4%.

So, the First World to Third World gaps reported are probably underestimates of the reality.

http://isteve.blogspot.com/2013/12/pisa-which-countries-not-to-trust.html

The methodological problems of a semi-global test like PISA are immensely daunting, as you'll realize by stopping and thinking about what it would take to pull it off. The reason the PISA isn't a trainwreck is because of the movie business saying, "Don't worry, we'll fix it in post-production." They use Item-Response Theory to throw out what turn out to be poorly translated questions, adjust for unrepresentative samples, and otherwise pull a lot of psychometric rabbits out of hats in the year between giving the test and publishing the data. My vague impression is that they are using their Rasch Model adjusting of the data mostly responsibly, but still ...

So, I'd be hesitant to put too much weight on precise details of the results. The big picture result of PISA is that we see the same overall racial pattern around the world as in the U.S.: East Asians on top, then Europeans, etc. PISA has huge samples to back that up. But taking on faith smaller comparisons, e.g., the top decile in income of Vietnam versus Peru, are asking an awful lot of a black box test.

For broad coverage of the latest PISA results, see
http://isteve.blogspot.com/search/label/PISA

In other words, trust PISA to the extent it validates your pet theory but don't trust those parts where the data threatens to upset your theory.

Perhaps it's self-inflicted? I just now recieved what might be a relevant email:

The Common Core Standards for California has gone through revisions since originally passed in November 2010. One area with a major shift is with the progression of Math courses. Currently, a number of students skip 6th grade Mathematics and begin middle school in Pre-algebra. With the change in Math progression, skipping Math courses is no longer an option...In this new configuration, most students will progress through Algebra 1, Geometry, Algebra 2 and Trig/Math Analysis through their four years in high school.

So, California schools will not assist you to excel at math, you have to (1) pay for and arrange after-school math to complete AP calculus and (2) force your child to sit through 5 hours a week of shit he mastered 2 years ago.

Wow, absurd. Algebra 2/Trig was my first year high school math class.

Wait, what? So then there's no way to take Calculus in high school? That's idiotic! How does that improve the American school system?

That is pretty idiotic! What are these guys thinking!

I think this is actually worse than it sounds, if that's possible. Generally Trig/Math Analysis (also sometimes called "Algebra 3" or "Precalculus") is actually a rehash of what is (or ought to be) in a decent Algebra 2 class. So either they're slowing down their curriculum so Algebra 2 is being taught in two years, or they're repeating it for seniors.

However, I would recommend bypassing the after-school AP calculus class in favor of buying a copy of Mathematica and encouraging your child to learn how to use it (if you don't know yourself, you can learn it together). Much more fun, and much much more useful. Buy a calculus textbook and work a good sample of the problems using Mathematica.

Comments for this post are closed