We are very good at assessing retrieval through multiple choice questions (MCQs): they are easy to write and quick to mark. These retrieval questions are useful – knowing stuff is important. But knowing stuff is only a route to more important aims of education: developing deeper thinking such as problem solving and creativity. 

Can we ask questions which assess higher order thinking? Yes, but it’s harder. 

Item-writing is an art. It requires an uncommon combination of special abilities. It is mastered only through extensive and critically supervised practice. It demands, and tends to develop, high standards of quality and a sense of pride in craftsmanship.(Ebel, 1951, p. 185)

Daisy Christodoulou (here) describes an example of an effective higher order thinking history question. The question is excellent and examples are really helpful. I wanted to share our process of developing higher order MCQs in science. 

At Paradigm Trust, we use MCQs as an important part of our subject assessment. We have a bank of effective retrieval questions which we’ve refined over the last 18 months. Now we are writing ‘higher-order thinking’ questions and we have developed a couple of techniques which help. 

Similar/Different Questions

These questions allow us to probe the understanding of similar concepts. E.g.

How is melting similar to dissolving?

a) Both can pass through filter paper. 

b) Both need to be hot.

c) You can get back the solid by heating.

How is electric current different to electric charge?

a) A current is made of particles, but a charge is not. 

b) Current moves, but charges do not.

c) Current happens when charges move.

What is one difference between a chemical story of energy and a gravativational store?

a) A gravitational store can last indefinitely, but a chemical store always runs out.

b) The particles in a chemical store need to be touching, but the particles in a gravitational store can be far apart. 

c) The energy stored in chemical bonds is called a ‘chemical store’ whereas energy stored in the gravitational field between masses is called a ‘gravitational’ store. 

What If….. Questions

This type of question allows us to assess whether our learners are able to use their understanding of phenomena to predict what would happen if something changed. E.g. 

What would happen if liquid water could only evaporate at its boiling point?

a) Washing on the line would never dry. 

b) A kettle could never boil.

c) It would rain more often.

What would happen if there were no gravity on the Moon?

a) If an astronaut dropped an object, it would not fall. 

b) Astronauts would need extra heavy boots to stay on the surface. 

c) Craters would be bigger. 

Apply it Questions

These questions put a taught concept into a new context to see whether the learner can apply it. E.g.

On a hot day, a dog is lying in the shade, panting. The dog pants so that:

a) evaporation cools its tongue.

b) evaporation dries its tongue.

c) the dog gets more oxygen.

Some migrating birds carry seeds in their digestive tracts over very large distances before excreting them. This is an advantage to plants because:

a) plants can colonise new habitats.

b) more seeds are wasted by being excreted in unsuitable places. 

c) the habitat where the parent plant is growing is unsuitable. 

 

Please note – previously we developed 4 answer MCQs. Reducing the number of options to three makes the questions significantly easier to write. There is evidence that reducing the number of options to three does not impact negatively on the questions difficulty or its ability to discriminate between pupils (see here).

These questions are still in the development stage – we would welcome feedback and comments. 

Thanks,

Ben