Does Professor Quality Matter?

by on June 10, 2010 at 12:26 pm in Education | Permalink

That's the title of the new lead article (gated) in the Journal of Political Economy, by Scott E. Carrell and James E. West, and here is the answer:

In primary and secondary education, measures of teacher quality are often based on contemporaneous student performance on standard-ized achievement tests. In the postsecondary environment, scores on student evaluations of professors are typically used to measure teaching quality. We possess unique data that allow us to measure relative student performance in mandatory follow-on classes. We compare metrics that capture these three different notions of instructional quality and present evidence that professors who excel at promoting contemporaneous student achievement teach in ways that improve their student evaluations but harm the follow-on achievement of their students in more advanced classes.

I found this to be an impressive piece of research.  Here is one summary sentence:

The overall pattern of the results shows that students of less experienced and less qualified professors perform significantly better in the contemporaneous course being taught.  In contrast, the students of more experienced and more highly qualified introductory professors perform significantly better in the follow-on courses.

Here is an ungated version, it may or may not be exactly the same.

Andrew June 10, 2010 at 12:36 pm

So, follow-on professors should “buy” students from previous professors.

Brian Moore June 10, 2010 at 12:59 pm

A friend who is on a tenure track professorship indicates that they are highly incentivized to focus on producing positive student evaluations (which are factors in their tenure review) at the expense of quality education (which is not a factor in their tenure review), a state that would seem to contribute to the findings of this study.

Amy June 10, 2010 at 1:29 pm

I may be misinterpreting the snippet Mr Cowen shares with us, but if “students of less experienced and less qualified professors perform significantly better in the contemporaneous course” …according to standardized test scores, whereas “students of more experienced and more highly qualified introductory professors perform significantly better in the follow-on courses”, there may be an anecdotal explanation from my own observation of my friends’ and parents’ career tragectories in education. To me this says: inexperienced professors teach to the test, perhaps because that is a clear objective standard of success for a newbie to target. Experienced professors teach to the material, perhaps based on an assessment of important vs irrelevant information earned from judging the performance of a larger sample of students for a longer time. I would guess the longer a professor teaches, the broader their perspective of what is meaningful to the subject area, which would make their instruction more valuable for subsequent advanced material. Standardized tests probably aren’t written to take the long view.

Mesa June 10, 2010 at 1:47 pm

Junior faculty may be incentived to inflate grades to promote class size and better reviews.

liberalarts June 10, 2010 at 2:15 pm

@Amy – I largely agree with your assessment.
@E. Bar. – I am missing the connection between your comment and the post?

bill June 10, 2010 at 3:05 pm

“Does Professor Quality Matter?”

Never heard of him, so I guess not.

E. Barandiaran June 10, 2010 at 3:32 pm

Liberalarts, just quality or –to be exact– poor quality. What can affect more the future of my American grandchildren — poor professors (something on which my son and his wife have some control) or poor presidents (something on which they have no control)?

Jake June 10, 2010 at 4:19 pm

The overall pattern of the results shows that students of less experienced and less qualified professors perform significantly better in the contemporaneous course being taught. In contrast, the students of more experienced and more highly qualified introductory professors perform significantly better in the follow-on courses.

I wonder if this is because the older teachers have better protection from retribution than younger ones do. In high schools, union rules kick in after one to three years, preventing you from being fired and thus lowering the cost of student dissatisfaction. In universities, tenure can help you become more far-sighted and more resistant to doing things that are popular but less likely to improve skills over the long term.

Bruno June 10, 2010 at 4:59 pm

What a great dataset.

Baphomet June 10, 2010 at 5:39 pm

While this study seems very interesting, it is yet another example of the alarming recent tendency for leading economics journals to publish papers that have absolutely nothing to do with economics.

DP Roberts June 11, 2010 at 11:10 am

I’ve been both a teacher and a Dean in higher ed and not all of this rings true.

Indeed, I’ve seen teachers try to buy good evaluations with good grades, but in my experience the students call the teaching as they see it.

I’ve seen good teachers who made learning fun with games, but failed to prepare students well for difficult follow-on coursework.

But I have never seen a demanding teacher who produced students better able to perform follow-on work get low evaluations. Again, students are smart enough to know the difference between academic rigor and bad teaching. The sole exception may be the professor who piles on work without regard for the other four classes the student is taking.

More “experienced” and “qualified” professors? What the hell does that mean? Those terms are usually labels for good researchers and, in my extensive experience, good research and good teaching are nearly mutually exclusive. The skill sets are different and the rewards to research are so much higher that every additional minute spent in preparation for teaching carries a high opportunity cost.

Academia needs to recognize the benefits of specialization and reward the good teachers better. Most research is crap anyway. I have never known good teaching to be a function of “experience”. It’s a function of EFFORT and CARE. Neither of these traits is rewarded by current professor incentives.

Department heads or committees need to take an active role in ensuring that prerequisites teach what follow-on courses require. If the pre-req teacher can’t get through the material with due diligence, the the expectations of the follow on professors need to be dialled back.

Graduate student June 11, 2010 at 3:09 pm

Bad evaluations don’t give people tenure. Though research always dominates teaching evaluations, the fact of the matter is that younger professors have incentives to keep their students happier while they teach class. which means they give their students better grades. Hence improve evaluations. Like much of economic research this says something we already knew.

A tenured professor (a more experienced professor) has he autonomy to fail his students without worrying about teaching evaluations. In economics which is a field that is a lot like the physics of social science, and requires mathematics at the advanced levels, its not surprising more experineced tenured professors in aggregate are able to teach students better. They have the autonomy to actually expect students to do lagrangians on their tests without them kicking screaming and crying about it.

Candadai Tirumalai June 12, 2010 at 9:54 am

One aspect of teaching in America is that the professor prepares the
examination questions and then grades the answers. This applies to
papers as well. Thus it is possible for a student to do well
with one professor but not with another. The best students do well
in a variety of courses, despite differences in subject and the professor’s
approach to it.
In English universities final exams were (I use the past tense because things
may have changed) marked by several professors and the student’s class (First,
Second, Third) was determined by their assessment.

anonymous June 12, 2010 at 11:38 am

While this study seems very interesting, it is yet another example of the alarming recent tendency for leading economics journals to publish papers that have absolutely nothing to do with economics.

“Absolutely nothing to do with economics”? That’s a stretch. This is a study that helps us better understand the human capital production function. That’s as economics as apple pie.

Comments on this entry are closed.

Previous post:

Next post: