Peer Review in Canvas
Tim Jackson (School of Engineering), discusses using Canvas to enable summative peer review of lab reports written by second year students studying electrical machines.
(With thanks to Ian Wells for his assistance on this project)
There were three reasons for doing peer review. Within the literature, there are many studies stating that students gain a better understanding of how to use marking schemes to improve their work through peer review. We expect that students will view feedback from their peers as having a different relevance compared with that from the instructor; they will gain a sense of the position of their learning with respect to the group. Finally, although a lab report is written individually, students do collaborate within the lab and we wanted to extend that social learning.
Peer review is said to work best when students are involved in devising the marking scheme. Now, I used that tactic before, but found it quite difficult to come up with a rigorous scheme based on quality benchmarks. The marking scheme became very much oriented to that particular assignment, which made me think that students might not take much from the exercise forward to their next assignment.
This time, I didn’t get the students to devise the scheme. Instead I adapted a published mark scheme for lab reports written around generic quality benchmarks like “expert”, “proficient”, “apprentice” and “novice” that could clearly be translated to other assessments students might do later. We did have several tutorial sessions exploring what such the benchmarks mean in practice, with examples of each quality description. I created an assignment in Canvas for students to submit their lab report. They were instructed to not put their name on the cover sheet of the report. The optional assignment settings within Canvas were used to randomly assign the peer reviews, to set the number or reviews allocated to each student, and to make the reviews anonymous. A date was set for reviewing one week after the original submission. On that date students received a notification to do their peer review. The review task appeared to students as an assignment in Speedgrader within Canvas, with an anonymous report for them to view. I kept the quantitative marking ratings really simple, 1-4 from novice to expert, and asked students to type the number in a comments box. They didn’t have to write comments, we said it would be helpful if they did. I also set up a self-assessment assignment for every student, to be scored in the same way, and made my own assessment which I double weighted.
The staff in HEFI were very helpful in working out how to make this work, we worked together using the Sandbox feature in Canvas to test it all out.
So what happened? Students were keen, as well as a little worried so there were lots of questions to answer. The vast majority did their peer reviews and their self assessments. Some missed the notifications from Canvas. Some students submitted reports late. To ensure peer reviews took place in these two cases we had to re-send the invitations ourselves and to make manual assignments respectively. For most students however the system worked well.
I was really surprised at how detailed and informative the comments were, critical in a constructive way. Comparing the peer marks and comments with my own, I found a close match. Clearly the students and I had a shared understanding of the marking scheme and how to apply it. They were prepared to take time to analyse the work and make suggestions to enable improvement. I think they wanted useful feedback, and so gave useful feedback. Interestingly the correlation was not so strong with the self-assessment, where self-interest was stronger, and some students were quite up front about their intentions here.
What we don’t have a handle on at the moment is whether or not it helped students with the next report any more than conventional feedback from their teacher. It would be good to devise a study to investigate that. Also looking ahead, we could make the peer review formative, with drafts of the assignment. We could have students giving peer review in groups. Students could report their observations back to the cohort, through Canvas. What we can say is that the perception of the process amongst students and for me was that it was fun, and instructive, and not that hard to do.
Please see the Peer Feedback section of the Remote Teaching Resource: https://canvas.bham.ac.uk/courses/42894/pages/peer-feedback?module_item_id=1477444