Implementing a System for 'Tournament' Peer Grading

Ricky Chu


Supervised by Richard Booth; Moderated by Shancang Li

Peer grading is an increasingly popular way to grade students' work, especially in MOOCs (Massive Open Online Courses). The idea is that each student grades the work of a subset of the other students, and from this information a final overall ranking of the students is calculated. Intuitively we want the opinion of students that are ranked more highly to carry more weight than those students ranked lower down, since they, presumably, are better qualified to recognise good quality work. What makes this an interesting problem is that a student's ranking depends on the rankings of those students that grade their work highly, which in turn depends on the ranking of those students that grade their work highly, and so on.

In this project we explore a special kind of 'tournament' peer grading in which the grading task of each student is to compare several pairs of submitted works A1 vs B1, A2 vs B2,..., An vs Bn and to say, for each pair, which work is better. The project has both an implementation part and a theoretical part. The implementation should be a web-based system for tournament peer grading. The theoretical part involves coming up with a good method for calculating the overall ranking based on this kind of input and with considering its theoretical properties (i.e., what are the 'good' properties that make it an attractive method.)

Initial Plan (03/02/2023) [Zip Archive]

Final Report (10/05/2023) [Zip Archive]

Publication Form