MicroCPD: How to make marking easier and better: building a rubric with drop-down comments

This week Mike Shulver talks through building a rubric with drop-down comments in Canvas

Building a rubric with drop-down comments

In the text below I will show an example of application of the principles discussed in the video. I will show:

  • an example of a rubric;
  • how I convert one of the rubric criteria into a set of drop-down comments;
  • how I associate points with the rubric comments, and;
  • an example of a drop-down comment list with marks.

The SpeedGrader application allows you to set up rubrics with drop-down comments under each of your assessment criteria. The assessment criteria you will have already, in that you currently say what you want of students in assessed submissions.

In the table below is an example of my assessment criteria for an applied essay:

 Criteria Criteria Discription Pts
Application of Service Management Concepts Work earning higher marks will clearly demonstrate, by appropriate and correct application, an understanding of the Service Management theory, concepts, tools, techniques etc, used in the submitted work. 20.0
Methodological Rigour Work earning higher marks will have a transparent methodology, which is justified, and is a good fit with the subject of analysis. 20.0
Argument Work earning higher marks will be underpinned by a transparent and clear logical argument. 20.0
Effectiveness Work earning high marks will be revealing. The analysis will clearly identify issues and challenges for the subject operation, and make recommendations for ways in which these challenges can be addressed. 20.0 
Critical Ability A student earning high marks would demonstrate application of theory that goes beyond the merely descriptive. Work earning low marks would be reportage ... very descriptive. Work earning higher marks will demonstrate that the student understands, for example: the limits to applicability of theory; the context specificity of theory. 20.0 
Substantiation  Work earning high marks will be well supported by reference to the module notes, videos and textbook. 15.0
Wider Reading Work earning higher marks will show clear evidence of wider reading of peer-reviewed articles in the Service Management literature. Any material referenced should be applied in the work. It is not sufficient to merely "pepper" the work with additional references.  15.0
Expression Work earning high marks will be engaging, well structured, make good use of graphics, tables and other presentational devices. The best work will be accessible to the reader; the reader should not have to hunt for arguments, or supporting references etc. The work should be an enjoyable read! 10.0 
Overall The criteria listed above are used to separately assess different aspects of the submitted work, but there is also a need to make some comment on the work ‘in the round’. How does the work ‘hang together’? 10.0

The criteria are expressed mostly as statements about how a very good submission would address each criterion. In building the list of drop-down comments I modify the excellent statement to make it appropriate to associate with work that does not address that criterion. I am explaining why I am taking marks away in respect of that criterion. When I have written 8 to 10 rubric comments I have a … kind of, pyramid of comments. The bottom items in the pyramid are extensive and explain how a poorly performing student (under that criterion) can improve their work.

Then for each of the comments I attach a mark. These can be set up in several ways in SpeedGrader but the easiest is to set points from 0 to 10. (Easy because the alternative, using say 15 points for each criterion, requires more mental attention when marking and will slow you down.) If you have 10 criteria in your rubric you have 100 point rubric.

Here is an example of a list of comments and marks for the “argument” criterion:

Argument

10/10 The logic underpinning your work is hard to fault.

09/10 The logic underpinning your work is exceptionally robust. Not perfect; just the odd gap here and there.

08/10 The logic underpinning your work is extremely robust. It gets very hard to earn extra marks when you are up in the '8s', but you could take the 'argument' mark into the '9s' with just a little more attention to the development of your ideas. Think of ways to help the reader / viewer more easily follow as you build your ideas to a conclusion.

07.5/10 Your work is underpinned by a very robust logic. Things get more difficult when you are up in the '7.5s', but you could take the 'argument' mark into the '8s' with a little more attention to that 'red thread'. Think of ways to help the reader / viewer more easily follow as you build your ideas to a conclusion.

07/10 Your work is underpinned by a highly robust logic. You could take the 'argument' mark higher with a little more attention to that 'red thread' so that the reader can more easily follow as your ideas build.

06.5/10 Your work is underpinned by a robust logic. However, you could take the mark well above average with more attention to the development of the argument in your work. You need to do a bit more to strengthen that 'red thread' so that the reader can easily follow as your ideas build. The viewer should not have to make too many inferences … connections amongst your various statements and ideas. More to the point, if it is not written down (or drawn), we can’t give you marks for it!

06/10 Your work is underpinned by a moderately robust logic. However, you could easily lift your mark into the high '6s and 7s' with more attention to the arguments in your work. We are not looking for reams of 'deathless prose', but you need to do a bit more to forge a thread that the reader can follow. The viewer should not have to make so many connections amongst your various statements and ideas. If you don’t write it down (or draw it), we can’t mark it.

05.5/10 The logic underpinning your work is fairly robust but you could easily lift your mark into … and so on.

(Note. The leading zeros ensure that SpeedGrader sorts the comments in the correct order.)

Some colleagues use a marks range associated with each rubric comment. This approach can give flexibility to the marker, but it does mean that there is potential to agonise over the exact mark to be associated with that comment. Obviously, we also introduce inconsistency with marks ranges. If you do use a range of marks with each comment then you need to clearly manage expectations with students.

Finally, if you want to make individual comments, then you can always do that by annotating the student’s work in the DocViewer application.

Further Resources

The mechanics of setting up Canvas rubrics is covered in the guide here: https://community.canvaslms.com/docs/DOC-12861-4152724129

Learn more about Online Marking in our online course which is open to all University of Birmingham staff - https://www.birmingham.ac.uk/university/professional-development-gateway/assessment/how-to-do-online-marking-2.aspx