Using Comparative Judgement to Rank the Importance of Concepts

If you haven’t heard of comparative judgement (CJ), it is the latest fashionable way for judging the quality of student work (see here and here) – although it’s not really new. I think it has great potential for judging longer written answers (or even short answers) beyond just right and wrong – some right answers (and some wrong answers) are better than others and this should be recognised and explored.

I had the idea of trying our CJ by ranking energy statements into order of importance for understanding energy. I took the statements from the ASEs Big Ideas in Science energy section here.

10 physics teachers ranked the statements using the CJ engine at Making 25 comparisons (each comparison taking, on average, less than 10s), the correlation was surprisingly high (0.82). The top 5 are:

  1. When energy is transferred from one object to others the total amount of energy in the universe remains the same; the amount that one object loses is the same as the other objects gain.
  2. Energy cannot be created or destroyed.
  3. Objects can have stored energy (that is, the ability to make things change) either because of their chemical composition (as in fuels and batteries), their movement, their temperature, their position in a gravitational or other field, or because of compression or distortion of an elastic material.
  4. An object at a higher temperature heats the surroundings or cooler objects in contact with it until they are all at the same temperature.”
  5. The transfer of energy in making things happen almost always results in some energy being shared more widely, heating more atoms and molecules and spreading out by conduction or radiation.

The full bank of statements, ranked, is here.

I’d be really interested if anyone has tried this with written exam answers.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s