The Impossibility of Reliable Setting by Ability

In principle, I don’t like the idea of setting by ability. It feels like segregation. But I also don’t like the idea of teaching wide ranges of ability in my classes. Except that I already do.

Despite all the warm words and passion on both sides, setting by ability isn’t really possible: you can’t measure ability precisely enough – the error bars you should be asking for when you get the results (Standard Error of Measurement) are surprisingly big.

We all know you can have a good test or a bad test – not about nerves, or lack of sleep, just they asked the right/wrong questions for you. Luck is built in.

It is possible (not easy) to calculate the SEM for an test score. NoMoreMarking give it to you in the spreadsheet you download. Commercial assessments should be prepared to give it to you (they won’t always be delighted, because it makes their assessment look less accurate).

A rough guide – in a well designed assessment with 100 as the average score; 80 as very low and 120 as very high,  an ‘average’  pupil would score 100 +/- 5ish. That means, if an “100” pupil did multiple equivalent assessments, she’d get between 95 and 105 six times out of ten. Three times out of ten she’s get more or less than that. (This is about right for the KS2 SATs).

This is what it looks like in a chart ranking 90 pupils in an assessment:

Result for Individual Pupils with SEM

Each chunky bar shows the individual’s score with SEM included. The thin line represents 2 SEM – meaning 95% of the time, the pupil would likely score within this range. In other words, around 5 children in the cohort above are likely to be at least this far out.

This has a major implication for setting. You assessment can’t tell with enough precision which group a child should be in.

The impossibility of setting meaningfully

In the data above, a child you might place in the middle of the middle set (child 34) could easily be placed in the top set or the bottom set, depending on pure luck.

And if she is in the wrong group, and the groups do different work, it will become increasingly difficult to put matters right.

So, even if you think it is harder to teach mixed ability groups, you probably already are.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s