AI Detection: how it works, where it fails, and how to use it well

Overview

AI text detection is the practice of estimating the probability that a passage was generated by a large language model rather than written by a human. It's moved, in just a few years, from a research curiosity to a tool that institutions use to make consequential decisions - about academic integrity, editorial trust, and hiring. That shift has outpaced most people's understanding of how the underlying systems actually behave.

This guide exists to close that gap. Everything here is drawn from controlled, reproducible testing - including the dataset behind The State of AI Detection 2026, in which I ran 10,000 documents through eight leading detectors. Where a claim depends on a number, that number comes from testing I can show you.

How AI detectors work

Most production detectors rely on two families of signal. The first is statistical: measures like perplexity (how "surprised" a language model is by each word) and burstiness (the variation in sentence length and complexity). Human writing tends to be burstier and less predictable; model output is often smoother. The second is learned: a classifier trained on large corpora of human and machine text to recognize the subtle fingerprints of generation.

Detectors don't know anything. They estimate a probability, and that probability is only as good as the data they were trained on.

Watermarking is a third, emerging approach: some model providers embed an imperceptible statistical signal at generation time (Google's SynthID is the most prominent). Watermark detection is far more reliable than post-hoc statistical detection - but only works for content from participating models, which is a small fraction of what's in the wild.

Why detectors produce false positives

This is the most consequential failure mode, and the most misunderstood. Because detectors key on predictability, any human writing that happens to be predictable - simple vocabulary, formulaic structure, limited syntactic variety - can read as machine-generated. In practice this disproportionately affects non-native English writers (a bias Liang et al. documented in 2023), students writing in a constrained academic register, and anyone using assistive writing tools.

The practical implication is stark: a detector that is 98% accurate in aggregate can still flag one in eight pieces of legitimate non-native writing as AI. Deployed without this context, that's a fairness problem, not just a technical one.

Deploying detection responsibly

Detection is useful - but only inside a process that respects its limits. The workflows I help institutions build share four principles:

• Never a sole verdict. A score initiates a conversation; it doesn't conclude one.
• Calibrate to your population. A newsroom and a university with many international students need different thresholds.
• Preserve process evidence. Version history and drafts are stronger signals than any single score.
• Be transparent with the people you assess. Tell them detection is in use and how appeals work.

For the full operational playbook, see my case studies on detection rollouts in academic and editorial settings.