Calibrating Peer Review in Project-Based CS Courses with Lightweight Reviewer Training
Sabbir Hosen Mamun, Beatriz M. Cardoso, Frederik A. Jensen
Koli Calling International Conference on Computing Education Research, pp. 77-88, Koli, Finland · ACM
Abstract
Peer review can scale feedback in project-based courses, but students need support to distinguish actionable critique from surface-level comments. We compare three reviewer-training formats: annotated examples, contrastive comment ranking, and short synchronous calibration. Contrastive ranking produced the largest improvement in review specificity, while synchronous calibration best reduced severity mismatch between student and instructor judgements.
Cite this work
@inproceedings{Mamun2025Calibrating,
title = {Calibrating Peer Review in Project-Based CS Courses with Lightweight Reviewer Training},
author = {Sabbir Hosen Mamun and Beatriz M. Cardoso and Frederik A. Jensen},
year = {2025},
booktitle = {Koli Calling International Conference on Computing Education Research},
pages = {77-88},
address = {Koli, Finland},
publisher = {ACM},
abstract = {Peer review can scale feedback in project-based courses, but students need support to distinguish actionable critique from surface-level comments. We compare three reviewer-training formats: annotated examples, contrastive comment ranking, and short synchronous calibration. Contrastive ranking produced the largest improvement in review specificity, while synchronous calibration best reduced severity mismatch between student and instructor judgements.},
}S. H. Mamun, B. M. Cardoso, F. A. Jensen, "Calibrating Peer Review in Project-Based CS Courses with Lightweight Reviewer Training," in Koli Calling International Conference on Computing Education Research, 2025, pp. 77-88.
Sabbir Hosen Mamun, Beatriz M. Cardoso, and Frederik A. Jensen. 2025. Calibrating Peer Review in Project-Based CS Courses with Lightweight Reviewer Training. In Koli Calling International Conference on Computing Education Research (Koli, Finland). ACM, 77-88.
Mamun, S. H., Cardoso, B. M., & Jensen, F. A. (2025). Calibrating Peer Review in Project-Based CS Courses with Lightweight Reviewer Training. In Koli Calling International Conference on Computing Education Research (pp. 77-88). ACM.