Don’t Take a Test on a Hot Polluted Day

Taking a test on a hot and polluted day can result in a measurably lower score which, if the test is for something like a university entrance exam, can have permanent consequences. I find both of these results hard to believe which doesn’t necessarily mean that they shouldn’t be believed.

Heat Stress and Human Capital Production by Jisung Park

How does temperature affect the human capital production process? Evidence from 4.6 million New York City high school exit exams suggests that heat stress on exam days reduces test scores and educational attainment by economically significant magnitudes, and that cumulative heat exposure during the school-year prior may affect the rate of learning. Taking an exam on a 90°F day relative to a 72°F day leads to a 0.19 standard deviation reduction in exam performance, equivalent to a quarter of the Black-White achievement gap, and a 12.3% higher likelihood of failing an exam. Teachers clearly try to offset the impacts of exam day heat stress by selectively boosting grades just below passing thresholds, while existing air conditioning seems to have a limited protective effect.  These findings may have implications for estimating the social cost of carbon, for designing education policy, and for understanding of climate in explaining income gaps across individuals and nations.

The Long-Run Economic Consequences of High-Stakes Examinations: Evidence from Transitory Variation in Pollution by Avraham Ebenstein, Victor Lavy and Sefi Roth.

Cognitive performance during high-stakes exams can be affected by random disturbances that, even if transitory, may have permanent consequences. We evaluate this hypothesis among Israeli students who took a series of matriculation exams between 2000 and 2002. Exploiting variation across the same student taking multiple exams, we find that transitory PM2.5 exposure is associated with a significant decline in student performance. We then examine these students in 2010 and find that PM2.5 exposure during exams is negatively associated with postsecondary educational attainment and earnings. The results highlight how reliance on noisy signals of student quality can lead to allocative inefficiency.

Comments

One question for any psychometricians out there: are the claimed magnitudes of these effects consistent with (smaller than) the known test-retest reliability of these exams? Such environmental factors as room heat should be folded into the random error of the tests. Or are these proxies for SES/race through variables such as heat islands/trees?

Sciencey!

Yes, and one that reinforces the gut. I want to do more when it is 72 than when it is 90 (like yesterday), and so this is easy to believe.

As always, replicate. But southern schools should prefer winter testing, just to be on the safe side.

These results are consistent with Miller and Vela (http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2370257) and Bharadwaj, et al (http://web.williams.edu/Economics/wp/BharadwajGibsonGraffZivinNeilson_GrayMatters.pdf) both of whom use Chilean data.

Pollution is bad for you, it turns out.

The good news is you can offset the effect with the right power poses just prior to testing, but only if you aren't suffering from ego depletion at the time.

Not if a Himmicane is coming.

If the tests are graded on a curve, this might actually be an advantage for the prepared!

I suppose lack of pollution is why students in Shanghai do well on PISA. :-;

People in Singapore was dumb until 100% of them moved into air conditioned spaces.

This is totally compatible with the history of Singapore, though.

Gardels: Anything else besides multicultural tolerance that enabled Singapore's success?

Lee: Air conditioning. Air conditioning was a most important invention for us, perhaps one of the signal inventions of history. It changed the nature of civilization by making development possible in the tropics.

Without air conditioning you can work only in the cool early-morning hours or at dusk. The first thing I did upon becoming prime minister was to install air conditioners in buildings where the civil service worked. This was key to public efficiency.

Air conditioning did wonders for the U.S. South as well. Even just electric fans and refrigerators were essential to populating South Florida. When Miami was incorporated in 1896 it only had about 1800 residents.

Yep.

I remember hearing a while back that Houston was the most air-conditioned (artificially cooled) city on Earth. Seemed right then and really I would think it's still true now. What else could it be, Singapore maybe? Hong Kong? Dubai?

Just think how much better they would perform in the absence of such pollution!

Are New York City classrooms not air-conditioned?

> while existing air conditioning seems to have a limited protective effect

Either the aircon isn't installed properly, or the problem is caused by bright sunlight, not heat. I can see how direct sunlight would affect the results of pupils sitting by the window.

Not necessarily. Having a hot day could cause them to be more likely to be dehydrated (for example. It could also be due to other things like worse sleep if their homes aren't air conditioned, etc.) prior to testing time, in which case air conditioning would prevent discomfort while taking the test but wouldn't necessarily prevent students from going into the test in a disadvantaged state.

It could also be due to other things like worse sleep if their homes aren’t air conditioned

Can confirm that heat-related lack of sleep can fuck up your day.

td;wr

(means "too dumb, won't replicate")

Also, don't drink a large coffee with nothing else in your stomach the morning of the test. I would wager I lost about 3 points on my LSAT to a desperate need to take a dump.

When are the PISA tests taken?

The best argument for fighting AGW, it makes you stupid.

If you believe you can fight AGW, you are already well past stupid.

To paraphrase Paul E. Meehl "the whole tradition of testing substantive theories [of complex subtle causal claims with multiple and often unknown and unmeasured variables] by null hypothesis refutation is a mistake."

The PM2.5 literature is an excellent example. Actual exposures aren't measured but instead proxies are used (e.g. distance from busy highway) and everyone pretends that there can't possibly be any reason other than the proxy variable tested that might explain why people who live next to railroad tracks or a freeway tend as a group to do worse on tests than those who live in quiet, tree lined neighborhoods. It's all B.S. Alex. All of it.

It doesn't seem like your criticisms of the PM2.5 literature in general apply to this particular paper. You can read it by googling "The Long-Run Economic Consequences of High-Stakes Examinations: Evidence from Transitory Variation in Pollution" the first result is an ungated draft.

The necessary conclusion is that we need to supply air conditioning to inner city schools. Will the teacher's union support AC at a small cost to their salaries? Of course they will, they care about minorities.

Because the only way to find money to air condition the schools where our children are for 40 hours/week is to take it from those huge teacher paychecks. Oh wait, it's not our children, it's those filthy vermin children.

If a tax cut = a loss to the government, a dollar extra spent on AC instead of salaries = a loss to Teachers. Just following the logic of the left

Did they account for time of year/Summer Learning Loss?

I'm not at all familiar with the school calendar in New York City; I'm assuming there is a summer break, and, therefore, assuming that most of the 90 degree days that occur during the school year are in August and September. If this is the case, then I wouldn't be surprised if Summer Learning Loss is a major factor in lowering test scores.

Another factor could be the relationship between temperature and study time. I don't have any studies to back this up, but anecdotally, my experience with students at Cornell is that there isn't a whole lot to do outdoors from November to March, so lots of students end up studying indoors with nothing better to do. (Though, your average Cornell student isn't your average NYC high school student.)

I really can't think of a test in America which has life-determining effects. Even with SATs, LSATs, etc., children are allowed to take them multiple times, so a single performance is not crucial. Add to that the fact that the difference between going to Yale and Brown (which is about what .19 standard deviations on the SAT would equate to) is actually not life-altering. (I mean, in terms of quality of education or career prospects--obviously anyone who doesn't get to sing a fight song written by Cole Porter leads an empty life forever.)

At Yale we have our very own Matthew Carter font for our logo. This helps our students graduate with 0.19 sigma more pretentiousness than Brown's, who have to make do with Minion.

Wisconsin has a very good "fight song." John Philip Sousa said so. I knew one or two Yale grads in the military - they had done the ROTC thing at UConn, I think, and I was their squadron leader at intel school (that is, the 4 month course where intelligence officers get introduced to the assessment of enemy intentions and capabilities). They were not pretentious but they were, in fact, amusingly disingenuous. I would not have recommended them for any important assignment, to tell the truth. I met a Harvard-grad officer once - he was a complete disaster - a pretentious self-centered guy (at least when he was in his 20s, he is certainly a different person now). Connecticut is a beautiful state - the Long Island Sound is nicer and more poetic on its southern (Suffolk county) shores, but Connecticut is a beautiful state.

Seems plausible.

Dispersing heat from CPUs is a major challenge to computer chip designers.

I remember being stuck on a camping trip with 95 degree temperature and 95% humidity. If I laid perfectly still on my cot and didn't try to think about anything, I could be comfortable. But as soon as I started simply thinking about anything in a sequential manner, I started to overheat.

It could be that there is an ideal latitude at which the cost of keeping the brain warm is balanced by the cost of keeping the brain cool at lowest overall cost. In 1911, Yale Professor of Geography Ellsworth Huntington conducted a study of climate's effect on human achievement. He concluded that the ideal climate was roughly that of New Haven, Connecticut. In a recent article, Malcolm Gladwell had great fun with that: here we are, 100 years later, and we can see what a biased moron Huntington was! Proving how much things have changed in 100 years, Malcolm's article appeared in that glossy, ad-packed magazine, The Lagoser.

Alex, just wondering why you find this hard to believe? The heat stress result seems highly plausible to me, though the pollution result less so.

Comments for this post are closed