Society & AI — Academy Community Voices account societyandai.org/academy/community-voices

Students

Flagged as AI-Written

A chatbot asked whether it wrote students' essays, and detectors that wrongly flagged most essays by non-native English writers. What happens when a confident label stands in for evidence.

WhereTexas, and a Stanford study

When2023

SystemAI-text detectors, and a chatbot asked to judge itself

Who it reachedStudents, especially non-native English writers

All Community Voices

The question it leaves

When a tool cannot explain why it flagged your work, what should count as proof that you wrote it yourself?

What happened

In May 2023 an instructor at Texas A&M University–Commerce pasted his students’ final essays into ChatGPT and asked whether it had written them. When it said yes, students in the class were given a temporary incomplete grade. A chatbot has no reliable way to recognise its own writing, so its answer was not evidence of anything.

Purpose-built detectors have a different problem. Researchers at Stanford ran seven of them on 91 essays written by non-native English speakers for the TOEFL test and 88 essays by US eighth graders. The eighth graders’ essays were classified almost perfectly. The TOEFL essays were wrongly labelled AI-generated 61.22 percent of the time on average. All seven detectors flagged 18 of them, and at least one detector flagged 89 of the 91. When the researchers had ChatGPT make the TOEFL essays’ word choice more elaborate, the false-positive rate fell to 11.77 percent: plainer vocabulary was being read as a sign of a machine.

Who spoke up

Students in the Texas class disputed the accusation publicly, and the story spread from a post on Reddit before the university responded. The Stanford team published its findings in the journal Patterns and cautioned against using such detectors to judge students’ work, particularly the work of non-native English speakers.

What changed

The university said several students were cleared and had their grades issued, one admitted using ChatGPT in the course, and no student failed or was barred from graduating. It said it was developing policies on AI in the classroom and adopting detection tools of its own, which is exactly where the Stanford findings matter.

What to keep straight

Headlines that the instructor failed the whole class, or that diplomas were withheld, are wrong according to the university.
The instructor asked a chatbot, not a detector. That is a different mistake from the one the Stanford study measured.
The study used small samples and detectors as they were in early 2023. Tools change; the central point, that a confident label can be wrong in a patterned way, does not.

Talk about it

If a detector is right most of the time but wrong far more often for one group of students, should it be used at all?
What could you show to prove you wrote something yourself? Is it fair to have to?
When software raises a suspicion, who should carry the burden of proof: the student, or the school?

Go further

Read the argument Perspectives · Marvin Starominski-Uehara

Trace Pedagogy: Grade the Trace, Not the Guess

When AI can polish any final product, teachers need evidence of the reasoning behind it. Trace Pedagogy makes open, peer-visible work the evidence of learning and uses exams to confirm it.

Read the article Teach it Classroom Kits · Ages 7–11

Mystery Machine

One student silently applies a secret rule. The class has only yes-or-no answers to work out what it is — practice at reasoning about a system you can't see inside.

View the kit