How do AI detectors work

An AI detector does not know who wrote a text. It measures how much the text resembles the output of a language model, by one of four methods, and reports the resemblance as a score. Everything that follows about accuracy, false positives and what a teacher may conclude comes from that one fact: resemblance is not authorship. This guide explains the four methods, what each one measures, and where each one breaks.

Method 1: perplexity, or how predictable the words are

A language model writes by estimating, at every position, which word is most likely to come next, and picking one of the likely ones. The text it produces is therefore, by construction, a text the model finds unsurprising. Perplexity is the measure of that surprise: run a text through a model, ask at each word how probable the model thought that word was, and average. A low perplexity means the text kept choosing the words the model expected; a high perplexity means it kept surprising the model.

The first generation of detectors, in late 2022 and 2023, rested almost entirely on this. GPTZero's early description of its method paired perplexity with burstiness, the variation of perplexity across the text: human writers alternate predictable passages and surprising ones, and alternate long sentences with short ones, where a model is steadier. A text that is both low in perplexity and low in burstiness looked generated.

The flaw is in the word "looked". Low perplexity is a property of plain, conventional, correct prose, whoever wrote it. Liang and colleagues at Stanford tested seven widely used detectors on 91 essays written by non-native English speakers for the TOEFL exam and 88 essays by US eighth-graders. The detectors were near-perfect on the eighth-graders and misclassified more than half of the TOEFL essays as generated, an average false positive rate of 61.22 percent. The essays all seven detectors flagged had significantly lower perplexity than the others: limited vocabulary and safe constructions, which is what writing in a second language looks like, is also what a model's output looks like. When the researchers asked ChatGPT to enrich the word choices of the TOEFL essays, the false positive rate fell to 11.77 percent; when they asked it to simplify the eighth-graders' word choices, misclassification rose from 5.19 to 56.65 percent. Perplexity measures vocabulary range, and vocabulary range is not authorship.

Perplexity, worked through on one sentence

Google DeepMind's own explanation of how language models write uses a sentence worth borrowing: "My favourite tropical fruits are mango and..." The model holds a probability for every possible next word; "bananas" scores high, "airplanes" scores low. A model writing on its own picks from the high scores. A perplexity detector runs the finished text back through a model and asks, at each word, how high that word scored. "Bananas" costs almost nothing; "airplanes" costs a great deal; a text made of bananas has low perplexity and a text with an airplane in it does not. That is the whole mechanism, and it explains both of its failures at once. A writer whose vocabulary is small or careful chooses bananas too, and scores as a machine. A machine told to surprise chooses airplanes, and scores as a person. Liang and colleagues measured exactly these two moves: enriching non-native essays raised their perplexity and cleared most of them; simplifying native essays lowered it and condemned most of them.

Two practical consequences. Perplexity needs length: a tweet has too few words to average, which is why serious detectors refuse texts under a few hundred words and why ours says nothing under one hundred. And perplexity is measured against a particular model: a text that surprises a small model may not surprise a large one, so the same passage can score differently in two tools that both call themselves perplexity-based.

Method 2: a classifier trained on examples

The second generation added a model of its own: a classifier trained on large collections of texts labelled human or generated, which learns whatever features separate the two in its training set and outputs a probability. This is what Turnitin, Copyleaks, Originality and the current GPTZero describe, in general terms, as their core. GPTZero's technology page, for example, describes a multi-step pipeline and claims 96.5 percent accuracy on mixed documents and a false positive rate under 1 percent, figures we quote as the vendor's own.

A classifier sees more than perplexity does: sentence structure, discourse patterns, the fingerprints of a particular model. It inherits three limits. It knows the models it was trained on, so a new model or a fine-tuned one shifts the distribution under it. It knows the kinds of human writing it was trained on, so a style it saw little of (second-language writing, technical prose, a particular dialect) can land on the wrong side. And it is opaque: when it says 87 percent, nobody, including the vendor, can point at the sentence that earned the number. Vanderbilt University's reason for disabling Turnitin's detector in August 2023 included exactly this: the company said its tool looked for patterns common in AI writing but did not explain or define them.

Method 3: watermarks embedded at generation

The third method does not look at the text's style at all. It relies on the model that generated the text having left a mark. Google DeepMind's SynthID, the best-known implementation, describes it this way: a model assigns each candidate next word a probability score, and SynthID adjusts those scores as the text is generated, in a pattern that a reader cannot notice and that does not change the quality of the output, but that its detector can find afterwards. DeepMind applies it to text produced by the Gemini app and to its images, audio and video, where the mark is designed to survive cropping, filters and compression. A watermark detector can be far more reliable than a style detector, because it is testing for a signal that was deliberately planted rather than guessing from resemblance, and because the planting side and the detecting side share a secret that an outsider cannot imitate.

It also answers a narrower question. It can only find marks from models that embed them and for which it holds the key; text from any other model, or from a model whose provider does not watermark, returns nothing, which is not the same as "human". Editing and paraphrasing weaken the mark. And the provenance question it answers best, "did this particular service produce this text", is not the question most people ask a detector, which is "did a person write this".

Method 4: surface patterns a reader can check

The fourth method is the oldest, and it is what an experienced teacher or editor does without a tool: notice that every paragraph is the same length, that every sentence opens with "Furthermore" or "Additionally", that the vocabulary never leaves the safe middle, that the text is full of "delve", "tapestry", "testament to" and "navigate the complexities", that there is not one contraction or one concrete detail in six hundred words. These are the patterns the models of 2023 to 2025 produce when left to their defaults, and they are measurable.

The detector on this site measures seven of them and shows each measurement. Its advantage is transparency: a coefficient of variation of 0.28 for sentence length is a fact you can verify by reading. Its limits are the same as everyone else's: a formal human text shows the same patterns, and a lightly edited generated text does not. We publish the thresholds and no accuracy figure, because we have not measured one.

The threshold: the knob the operator holds

Whatever the method, the raw output is a number, and someone decides where the number becomes a flag. A perplexity detector flags below a certain surprise; a classifier flags above a certain probability; a watermark detector flags above a certain statistical confidence. Move the threshold up and fewer human texts are flagged, but more generated texts pass; move it down and the reverse. Every published accuracy figure was measured at one threshold, usually the one that made the figure look best, and the product in front of a teacher may be set at another. When a vendor quotes a false positive rate, the question "at what threshold, and is that the default" is not pedantry; it is the difference between 1 percent and several times that.

Why all four methods share one blind spot

Three of the four measure resemblance to a model's default style, and that style is also the style of careful, conventional human prose. Liang and colleagues showed the mirror image too: a one-line instruction to ChatGPT to "elevate the provided text by employing literary language" cut the detectors' catch rate on generated college essays from 100 percent to 13 percent. The patterns are real; they are just not attached to an author.

What happened to the detectors that tried

OpenAI released a classifier for its own models' output in January 2023 and withdrew it on 20 July 2023, with an addendum citing its low rate of accuracy, as TechCrunch reported; the company said it was researching provenance techniques instead. A month later Vanderbilt disabled Turnitin's AI detection for its whole campus, working out that Turnitin's own claimed 1 percent false positive rate would have meant roughly 750 wrongly flagged papers a year on its 75,000 submissions, and concluding that it did not believe AI detection software was an effective tool that should be used. The vendors that remain have improved their numbers on their own benchmarks and added features to reduce bias; GPTZero reports bringing its TOEFL false positive rate down to 1.1 percent. Whether those figures hold on your text, in your discipline, this year, is the question no vendor can answer for you, and the next guide looks at what the published tests say.

How to read any detector's output

Ask which of the four methods it uses; a vendor that will not say is asking for trust it has not earned. Ask what the score is a probability of: resemblance to a model's style, or presence of a specific watermark. Ask for the false positive rate on texts like yours, not on a benchmark of news articles, and remember that 1 percent of a large pile is a large number of people. Then read the text. A detector can tell you where to look; what you find when you look is the only evidence there is.