Society & AI — Academy Community Voices account societyandai.org/academy/community-voices

Teachers

A Score Nobody Could Check

Houston rated teachers with a value-added model whose code was a trade secret, and low scores helped end contracts. Teachers sued, a federal judge let their due process claim go forward, and the district settled.

WhereHouston Independent School District

WhenLawsuit 2014, settled 2017

SystemEVAAS value-added model

Who it reachedTeachers

All Community Voices

The question it leaves

If you cannot check how a score was calculated, can you ever really challenge it?

What happened

The Houston Independent School District rated teachers partly with EVAAS, a value-added model sold by the software company SAS. A value-added model estimates how much a teacher changed their students’ test scores. Low scores were used in decisions about teachers’ jobs: the court record lists 12 union members whose contracts ended over low value-added scores between 2012 and 2014.

SAS treated the model’s code as a trade secret. Neither the teachers nor the district could see it. The district admitted it did not verify or audit the scores, and an expert for the teachers could not reproduce them.

Who spoke up

Teachers. In 2014 the Houston Federation of Teachers and individual teachers sued the district in federal court. One of them, Daniel Santos, a sixth-grade teacher at a Houston middle school who had been rated ineffective, spoke publicly about his commitment to his students.

What changed

In May 2017, US Magistrate Judge Stephen Wm. Smith refused to dismiss the teachers’ claim that their right to due process had been violated. Teachers, he found, had no meaningful way to make sure their scores had been calculated correctly, which left them open to losing their jobs by mistake. He dismissed the teachers’ other claims.

In October 2017 the district settled. It agreed not to use value-added scores to terminate a teacher as long as the teacher cannot independently check or challenge the score, to set up a consultation panel on teacher appraisal, and to pay $237,000 in legal fees. It had already ended its contract with SAS in 2016.

What to keep straight

This was not a final ruling that the system was unconstitutional. The judge let one claim go forward, and the case settled before trial.
The settlement did not ban value-added scores. It barred using them to fire teachers while the scores cannot be verified.

Talk about it

Should a company be allowed to keep secret a formula used to decide whether someone keeps their job?
What would it take for a teacher to really check a score like this? Would seeing the code be enough?
Test scores measure students. Is it fair to use them to judge teachers?

Go further

Read the argument Blogs · Sai Gattupalli

Public-Interest AI

Principles, playbooks, and procurement for the common good

Read the article Teach it Classroom Kits · Ages 9–14

Explain Yourself

One student decides using a secret rule. Everyone else gets 20 yes-or-no questions to work out why — the actual skill of demanding an explanation from a system that won't just give you one.

View the kit