How AI Detectors Work and Where They Fail
AI detectors do not read for meaning or know who wrote a text. They measure patterns and return a probability. Here is what is under the hood, what each approach gets right, and how far to trust a score.
- 4 detection methods
- Strengths and weak spots
- Sourced research
How Does an AI Detector Work?
A detector cannot see who typed a text or which tool produced it. It only sees the finished words. So it asks a statistical question: does this text look more like the writing it has learned to associate with humans, or with language models?
Every detector follows roughly the same path from text to score. What differs is which features it measures and how it decides.
Text goes in
The detector receives your document, often split into paragraphs or sentences.
Features are measured
It measures properties of the text, such as how predictable the wording is and how much the sentences vary.
A model compares
The measurements are compared with what the detector expects from human and AI writing.
A probability comes out
You get a score or a label. It is an estimate, not a fact about who wrote the text.
Detectors differ and vendors do not publish every detail, so treat this as the general picture rather than a description of any one product.
The 4 Main Approaches to Detecting AI Text
Researchers group detectors into four families. Real products often combine them, and each has a characteristic weak spot.
Trained classifiers
A model is trained on many labeled examples of human and AI-written text and learns to tell them apart. Many commercial detectors are built this way, though vendors rarely publish the details.
Statistical signals
No labeled training is needed. The detector uses a language model to measure how predictable each word is (perplexity) and how much that varies (burstiness), then flags text that is uniformly predictable.
Watermarking
The AI provider subtly biases the model’s word choices so a hidden statistical pattern can be detected later by anyone who knows the key.
Retrieval and matching
The provider keeps a record of what its model generated, and the detector checks whether a submitted text closely matches something in that record.
Categories follow the taxonomy in Sadasivan et al., Can AI-Generated Text be Reliably Detected?
What a Detector Score Really Means
A score is a probability produced by a model that can be wrong in both directions. Published research and vendor figures show how large the margins are.
In a stress test of watermark-based detection on 300-token passages, the detection rate at a 1% false positive rate fell from 99.3% to 9.7% after the text was repeatedly paraphrased by a neural model, while text quality dropped only slightly.
The authors also describe a trade-off: making a detector avoid false positives tends to increase false negatives, and the reverse. They conclude that relying on detectors alone is risky. Source: Sadasivan et al., arXiv:2303.11156.
OpenAI retired its AI classifier, stating that “as of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy.”
Source: Search Engine Land.
Turnitin reports a document-level false positive rate of less than 1% for documents with 20% or more AI writing, and about 4% at sentence level.
Source: Turnitin.
The takeaway. Accuracy depends on the text length, how much of it is AI-written, whether it was edited, and where the threshold is set. A score is a starting point for a human look, not a verdict. For why human writing gets caught, read Why Human Writing Gets Flagged as AI.
How Our Free Checker Fits In
Our free AI text checker is deliberately simpler than a commercial detector. It does not use a trained model. It runs in your browser, counts four signals in your text, and shows them in a sentence view so you can see what to edit.
Sentence variation
How much sentence length varies. Very even pacing is one pattern associated with AI drafts.
Vocabulary diversity
How varied the word choice is across the text. A narrow range reads as repetitive.
Repeated phrases
How often the same three-word sequences return.
Transition density
How many formal connectors such as “moreover” appear per 100 words.
How to use a result well
- Use a score as a reason to read the text more closely
- Look at which sentences or signals drove the result
- Compare the result with drafts, notes and version history
- Ask for a human review before any decision
- Remember that short or mixed texts are the least reliable
How not to use it
- Treating one score as proof of who wrote a text
- Using a detector as the only basis for a grade or hiring decision
- Assuming two different tools should give the same number
- Trusting a result on a few sentences
- Editing a text only to push a score down
Want to see the four signals on your own text?Paste a draft of 80 words or more. Nothing is uploaded or stored.
Open the free checkerFrequently Asked Questions
How do AI detectors work?
Most measure statistical properties of a text, such as how predictable the wording is and how much sentence structure varies, and compare them with what they expect from human and AI writing. Some use classifiers trained on labeled examples, and some rely on watermarks added by the AI provider. All of them output an estimate.
What are perplexity and burstiness?
Perplexity measures how predictable a text is to a language model: low perplexity means the wording is very expected. Burstiness describes how much that predictability and the sentence length vary through the text. Human writing tends to vary more, but careful or simple human writing can score low on both.
Can AI detectors be accurate?
They can be useful on long, unedited text, but none is reliable enough to serve as proof. Researchers have shown that paraphrasing can sharply reduce detection rates, and that human text can be misclassified. OpenAI withdrew its own classifier in July 2023 because of its low rate of accuracy.
Why do different detectors give different results?
They use different methods, training data, thresholds and score definitions. A 40% on one tool and a 90% on another do not measure the same thing, so scores should not be compared across products.
What is the difference between a false positive and a false negative?
A false positive is human writing flagged as AI. A false negative is AI-assisted writing that is not flagged. Research suggests there is a trade-off: tuning a detector to avoid false positives tends to increase false negatives, and the reverse.
Does watermarking solve the problem?
Not by itself. A watermark only exists if the AI provider adds it, and text from other tools carries none. Research has also shown that repeated paraphrasing can weaken watermark detection.
How does this site’s checker work?
It runs in your browser and measures four simple signals: sentence-length variation, vocabulary diversity, repeated phrases and transition density. It does not use a trained model or upload your text, and it cannot prove who wrote anything.
Is this site connected to Proofademic?
No. proofademicaidetector.net is independent and not affiliated with, sponsored by, or endorsed by Proofademic. The explanations here are our own.
See how your own writing reads
Paste your draft into the free checker to see sentence rhythm, vocabulary variety, repeated phrases and transitions, then revise the flagged lines in your own voice.