Teacher expectations matter

From Nicholas W. Papageorge, Seth Gershenson, and Kyung Min Kang:

We develop and estimate a joint model of the education and teacher-expectation production functions that identifies both the distribution of biases in teacher expectations and the impact of those biases on student outcomes via self-fulfilling prophecies. Our approach leverages a unique feature of a nationally representative dataset: two teachers provided their educational expectations for each student. Identification of causal effects exploits teacher disagreements about the same student, an idea we formalize using lessons from the measurement error literature. We provide novel, arguably causal evidence that teacher expectations affect students’ educational attainment: Estimates suggest an elasticity of college completion with respect to teachers’ expectations of about 0.12. On average, teachers are overly optimistic about students’ ability to complete a four-year college degree. However, the degree of over-optimism of white teachers is significantly larger for white students than for black students. This highlights a nuance that is frequently overlooked in discussions of biased beliefs: less biased (i.e., more accurate) beliefs can be counterproductive if there are positive returns to optimism or if there are socio-demographic gaps in the degree of teachers’ optimism; we find evidence of both.

This is the most important thing I will have to tell you today.  Here is commentary from Vera.


Hopefully Greg Cochran comments :)

If you only compare whites and blacks who finish their homework and study about the same each day, they get about the same grades. The achievement gap among whites and blacks disappear if you compare those who put in the same amount of effort and work.

So why do they need affirmative action?

Ask Sandra Day O'Connor.

Do you have a reference for this?

Of course, this blog post and the one immediately preceding are two sides of the same coin. Teachers' expectations are the product of where the students came from not where they are going. As for economists, they are the soothsayers of our time, and are about as accurate with their soothsaying as soothsayers of the past. In the immediately preceding blog post, economists attempt to add natural science data to their soothsaying, whether to increase accuracy or give the illusion of accuracy (confirmation bias?) is unclear, it's soothsaying nevertheless. Teachers' expectations, economists' expectations, they derive whence we came not where we are going: judging human intelligence is judging humans. Will judging artificial intelligence be subject to the same human expectations (biases), based on, for example, the creator of it? Will expectations for artificial intelligence created by Google be the same as expectations for artificial intelligence created by "somebody sitting on their bed that weighs 400 pounds"? Will self-driving cars be required to include a label identifying the creator of its artificial intelligence? Who created the artificial intelligence (i.e., the stabilization system) for the Boeing 737-MAX?

Have you heard of Zhores Medvedev? I didn't think so. I hadn't, until I read his obituary in today's NYT. Read the obituary and consider whether we need a journal of controversial ideas. https://www.nytimes.com/2018/11/16/obituaries/zhores-medvedev-dead.html

What distinguishes Medvedev from other Soviet dissidents is that he was a scientist and his objection was to Soviet pseudo-science (in particular pseudoscience known as Lysenkoism), which he did largely via underground literature (known as samizdat). Of course, pseudo-science has exploded in popularity here in America as of late. While some may complain of a lack of scientific discoveries, there's no lack of discoveries in pseudo-science. Medvedev was committed to an insane asylum for his attempts to expose Soviet pseudo-science. The current administration is chock-full of quacks promoting pseudo-science. That's the current relevance of Medvedev and why his obituary struck a chord.

" less biased (i.e., more accurate) beliefs can be counterproductive if there are positive returns to optimism " << this is also especially true about entrepreneurs, who have to have a bit more optimistic self-confidence about "their" new business than the statistics indicate.

There's also a placebo-voodoo issue on belief about healing. I'm sure there are cases where the statistics say survival is medium low, in the 20-30% of cases, but where this survival is biased based on doctor words: a) doctor says most recover, if they get a lot of care, which looks like this case (actual rate 30%), with the patient not preparing much for death; b) doctor says many recover, about 20%, so the patient can improve the odds with good care but should be realistic; the patient does prepare for death (actual rate 20%).
Should the doctor be not-so-accurately optimistic to maximize the survival rate?

There are many cases of optimism leading to the optimistic outcome.
Famous Star Wars quote (Han): "Never tell me the odds".

But are we sure that teachers' overly optimistic assessments are a good thing? After all, it sounds as though, but for inflated teacher expectations, fewer marginal students might end up going to college and ending up saddled with loan debt but no degree.

I would guess that what you might find is that in more detail teachers are less over optimistic about white students' ability, but more unrealistically overly optimistic about their circumstances ("privilege", "every advantage") and willingness to choose easy subjects and be soft pedalled by teachers.

I don't have access to the full paper yet, but from the abstract and some of their earlier research it appears that the data on expectations is collected in the 10th grade. Are we to believe that these expectations of the 10th grade teachers are causally impacting college graduation of these students? I suppose we are to take the 10th grade teacher expectations as a proxy for expectations of other teachers these students subsequently encounter - but we don't have data on their racial/ethnic/gender mix. And, speaking of data, I'd like the authors to release the data they used. They appear to be milking this data for a dozen papers - building a career on a data set that they don't publicly provide, but doing research with real public policy implications.

Sorry, but I'm getting tired of the academic publication mill which says "trust me because I have the right pedigrees, no you can't see the data, and I want to get as many publications from this data as possible." Don't misunderstand me - these authors may have done a very good job, but I would like to see them rise above the tiresome academic games. Also, when the dataset goes from 16,000 observations to 6,000 observations, I'd like to see a bit more about why (they do provide a table showing why observations were not usable, but the issue of selection bias looms large when there is this much of a difference in sample sizes).

My father was a teacher who worked in inner city schools. My mom was on staff there for a while. I am here to tell you that they, and their friends, really had "teacher spirit." (That is a little bit of a joke on samurai spirit.)

In my whole life I never heard a dinner table conversation about a kid which even incorporated race with any idea of destiny. Sure there were problem kids, sure there was wry recognition that they had younger brothers who would be future problems. They were certainly staff conflicts and very political community relations.

But kids were kids. If anything, my shock was growing out of that teacher spirit bubble and learning how much discrimination there was in the wider world.

And yeah, including the structural kind.

This is one of the oldest and most replicated findings in psychology as well as educational psychology, originally formulated as the 'halo effect' by Thorndike in his paper, "A Source of Error in Psychological Ratings," Jour App Psych, 1920, pps. 25-29. Thorndike's paper was based on a WWI analysis of performance ratings of soldiers by their superiors.

One additional point is worth making here. Lumping 'blacks' into a single bucket is incorrect since there is a strong interaction with gender: black women are much more likely to be successful than black men and, as a consequence, are less vulnerable to white racism than are black men. If there's a reason for this it may be that whites are afraid of black men but not of black women.

If explored, this finding would almost certainly hold for the ed psych literature as well.

It would be interesting to see how this played out for education success other than college, e.g. skilled trades, skilled military MOSs, etc.

The empirical strategy in this paper appears to be based upon a very strong and, in my opinion, dubious assumption. Namely that the pair of teachers has exactly the same information about each particular student. The possibility that one of the teachers has a better or even different knowledge about students strengths or weaknesses is ruled out by the model.

This is discussed in section 3.4 and spelled out in equations in section 4.

The model "passes" a set of specification tests, but that is very unconvincing. Just because you show that you can't REJECT that a variable is exogenous at p<.05, you can't conclude that it is in fact exogenous. Hypothesis testing doesn't work like that. For example, table 8 row 2 shows that the coefficient on the specification test has the "expected" sign and is very close to passing a 1-tailed test. They take this as evidence that the true coefficient is zero!

Teacher N of 2. Awfully damned small N.

Comments for this post are closed