The Summer an Algorithm Set the Grades

What happened

In spring 2020 the pandemic closed schools, and England cancelled its exams. Teachers estimated a grade for each student and ranked the students in each subject. The exams regulator, Ofqual, then ran those estimates through a statistical model. For each school and subject, the model predicted a spread of grades, based mainly on the school’s results in earlier years and adjusted for this year’s students’ prior attainment. It then fitted the ranked students into that spread.

When results came out in August, 39.1 percent of the 718,276 teacher-assessed A-level grades had been lowered: 35.6 percent by one grade, 3.3 percent by two, and 0.2 percent by three or more.

Statistics are unreliable for small groups, so the model trusted teachers more where a subject had few entries: fully below an entry size of 5, and partly between 5 and 15. Small entries are more common in independent schools. In Ofqual’s own figures, the share of A grades and above rose by 4.7 percentage points at independent schools compared with 2019, against 2.0 at comprehensives and 0.3 at sixth-form and further education colleges.

Who spoke up

Students. Some lost university places they had been offered. On 16 August around 300 students marched from Parliament Square to the Department for Education and Downing Street, saying the model penalised students from disadvantaged backgrounds.

What changed

Scotland moved first. On 11 August its government directed the exams body to reissue downgraded awards based on teachers’ judgements, after about 76,000 candidates had at least one grade lowered. On 17 August Ofqual’s chair announced that students in England would receive their teacher-assessed grade, or the calculated grade if it was higher, and apologised for the distress the process had caused. Wales and Northern Ireland made the same switch. GCSE results, due three days later, were never issued under the model.

What to keep straight

• “Almost 40 percent” refers to grades, not students: 39.1 percent of teacher-assessed A-level grades were lowered.
• The small-entry rule used a subject’s entry numbers averaged across years, not literally the size of a class.
• Ofqual said it found no evidence of bias. The independent-school advantage appears in its own outcome data; the explanation through small entries comes from analysts at FFT Education Datalab.

Talk about it

• The model was built to keep national results in line with earlier years. Was that a reasonable goal? Who paid for it?
• A teacher’s estimate and a school’s history are both evidence about a student. Which should count more, and why?
• Students were told their grade, not how it was produced. What would you have wanted to see?