ASA Against Student Evaluations

Yesterday in Active Learning Works But Students Don’t Like It I pointed out that student evaluations do not correlate well with teacher effectiveness and may discourage teachers from using more effective but less student-preferred methods of teaching. Coincidentally the American Sociological Association issued a statement yesterday discouraging student evaluations for tenure and promotion decisions.

SETs are weakly related to other measures of teaching effectiveness and student learning (Boring, Ottoboni, and Stark 2016; Uttl, White, and Gonzalez 2017); they are used in statistically problematic ways (e.g., categorical measuresare treated as interval, response rates are ignored, small differences are given undue weight, and distributionsare not reported) (Boysen 2015; Stark and Freishtat 2014); and they can be influenced by course characteristics like time of day, subject, class size, and whether the course is required, all of which are unrelated to teaching effectiveness. In addition, in both observational studies and experiments, SETs have been found to be biased against women and people of color (for recent reviews of the literature, see Basow and Martin 2012 and Spooren, Brockx, and Mortelmans 2015).

Student evaluations mostly evaluate entertainment value but, as my colleague Bryan Caplan notes, given how boring most classes are, entertainment value is worth something! Thus, the ASA notes that student evaluations can be useful as a form of feedback.

Questions on SETs should focus on student experiences, and the instruments should be framed as an opportunity for student feedback, rather than an opportunity for formal ratings of teaching effectiveness.

Student evaluations are on their way out in Canada, as a tool for tenure and promotion. I expect the same here, at least on paper. The ASA is big on “holistic” measures of teacher evaluation as a replacement which strikes me as weaselly. I’d prefer more objective measures such as value added scores.