Face Not Found
A Black student in Amsterdam said exam-proctoring software kept failing to see her face. She took it to the Netherlands Institute for Human Rights, and the ruling was more complicated than the headlines.
When someone says a system failed them, who should have to prove what, and with whose data?
What happened
During the pandemic, Vrije Universiteit Amsterdam (VU) ran online exams with Proctorio, software that watches students through their webcam and checks that a face stays in view. Robin Pocornie, a Black master’s student, said it repeatedly failed to detect her, with messages such as “face not found” and “room too dark.” She said she had to shine a lamp on her face to sit her exams, and that her white classmates did not.
Who spoke up
She did, with support from the Racism and Technology Center. She filed a complaint with the Netherlands Institute for Human Rights, arguing that the software discriminated on grounds of race, and described her experience in interviews.
What changed
In December 2022 the Institute issued an interim ruling: she had put forward enough facts for a presumption of indirect race discrimination. That shifted the burden to the university, which was given ten weeks to back up its denial with data the Institute could verify.
In October 2023 the final ruling found that VU had not made a prohibited distinction by using Proctorio. The university’s log data showed her problems were no greater than other students’, and the Institute attributed them to her glasses and an unstable internet connection. It also found no prohibited distinction in VU’s failure to have the software checked independently before using it. But it ruled that VU had handled her complaint too slowly, and so not carefully enough, and that this was a prohibited distinction on grounds of race. VU apologised and said it would review its complaints procedure and re-evaluate remote testing.
What to keep straight
- The interim ruling was a presumption that moved the burden of proof, not a finding of discrimination. The final ruling did not “reverse” an earlier finding.
- The final ruling went mostly against her complaint about the software, and for her on how the complaint was handled.
- The Institute’s rulings are not legally binding.
Talk about it
- The university could make its case only because it had log data. What if it had kept none?
- Is “it fails for everyone about equally” the same as “it works”?
- Why might the way an institution handles a complaint matter as much as the original problem?
Go further
What Is Society-Centered AI?
Society-centered AI extends human-centered design into collective, civic, and institutional contexts—positioning affected communities as co-designers, co-owners, and co-stewards of artificial intelligence systems. This foundational article articulates the principles, processes, and practices that distinguish society-centered approaches from conventional AI development, arguing that legitimacy, not merely usability, must become the central criterion for AI systems that shape public life.
Read the article Teach it Classroom Kits · Ages 11–15The Human Sorting Machine
Write a rule before you see who it sorts. Then meet the people it excluded, and one thing about them the rule couldn't see.
View the kitSources
- Interim ruling 2022-146, College voor de Rechten van de Mens (Netherlands Institute for Human Rights) (7 December 2022).
- Ruling 2023-111, College voor de Rechten van de Mens (Netherlands Institute for Human Rights) (17 October 2023).
- Uitspraak College voor de Rechten van de Mens (statement, in Dutch), Vrije Universiteit Amsterdam (October 2023).
- Anti-cheating software could be racist, says human rights institute, Erasmus Magazine (14 December 2022).
- Cheating algorithm did not discriminate against black student, DutchNews (October 2023).
More accounts to read
All Community Voices-
Students
Labelled High Risk in Ninth Grade
A state system scored students on their chances of graduating, using race and income among its inputs. Most of its warnings were false alarms, and two students wrote about living with the label.
2012–2023 · Wisconsin, US Read the account -
Students
The Summer an Algorithm Set the Grades
With exams cancelled, a model lowered 39 percent of teacher-assessed A-level grades, and small classes fared best. Students marched; the government reversed within days.
2020 · England Read the account -
Teachers
A Score Nobody Could Check
Houston judged teachers with a score built on secret code that no one could verify. Teachers sued, and the district agreed not to fire anyone on an unverifiable score.
2014–2017 · Houston, US Read the account
Summarised from the public record. Follow the sources for the full story.