SWORD for Peer Grading

Jennifer Imazeki who blogs at Economics for Teachers has an interesting post about SWORD, a peer review system for grading writing. Basically students write an assignment and submit it to SWORD which randomly assigns 5 other students to grade the assignment and offer helpful comments; the gradees, in turn, grade the quality of the comments.  Here’s Imazeki on some neat aspects of the system:

One of the coolest things about the SWoRD system is how it calculates grades. Students receive a grade both for reviewing and for writing. The reviewing grades are based half on ‘consistency’, which takes into account things like if a student just gives all high scores or all low scores, or scores that are really different from the other reviewers of the same papers, and half on the back evaluation ‘helpfulness’ scores. The writing grades are based on the numeric rubric scores from the reviewers but adjusted for the consistency of the reviewers so, for example, if a paper has four reviewers who give high scores and one reviewer who gives low scores, the low scores from that one reviewer will be given less weight. The instructor can also adjust how much weight is given to the reviewing and the writing scores for each assignment.

The big virtue of SWORD is that it makes written assignments feasible even in a large class.  I will be interested to read how Imazeki’s pilot experiment works out.