Every AI detector claims 99% accuracy. Here's what independent testing actually shows.
Search for any of the major AI detection tools — GPTZero, Originality.AI, Copyleaks — and their marketing says the same thing. 99% accuracy. Sometimes it's "98%+." Occasionally a more confident "99.12%." The number varies slightly, but the message is consistent: these tools are nearly infallible.
Then you read the independent research.
In peer-reviewed benchmarks, AI detector accuracy on edited and humanized content consistently measures 85–92% — a 7–14 percentage point gap from vendor claims. On non-English text, that gap can reach 25 points.
This isn't a rounding error. It's a systematic gap between what vendors measure and what users actually encounter — and the people paying for it are students wrongly accused of cheating, freelancers losing clients, and SEO teams flagged on content they wrote themselves.
Section 1: The Accuracy Claims vs. Reality
Let's look at the numbers side by side. The "Claimed" column comes directly from vendor marketing pages and press releases. The "Verified" column represents performance measured by independent researchers, primarily the RAID benchmark (Ramponi et al., 2024), the Tian et al. Penn State evaluation, and user-reported testing across forums like r/education and r/SEO.
| Detector | Claimed Accuracy | Verified (Independent) | Gap | Non-English Accuracy |
|---|---|---|---|---|
| GPTZero | ~99% | ~85% | −14 pts | 79–88% |
| Originality.AI | 98–99% | 92–96% | −3–7 pts | Varies widely |
| Copyleaks | 99%+ | 91–92% | −7–8 pts | 74–84% |
| QuillBot | 90–93% | 80–88% | −5–10 pts | Limited data |
How vendors arrive at 99%
The methodology gap isn't hidden — it's just rarely explained. Vendors typically measure accuracy on clean, unmodified AI output. They feed raw GPT-4 or Claude responses directly into their detector and score the result. On this benchmark, yes, most detectors perform well. Pure, unedited AI text is relatively easy to catch.
The problem: that's not what's actually submitted to these tools in the real world.
Real-world AI-assisted content goes through editing. It's lightly proofread, rephrased for tone, blended with personal experience, or run through a grammar tool. It's been through at least one human pass. And it's precisely on this category — humanized AI content — that detection accuracy drops significantly, often to 60–75% in independent tests.
The non-English cliff
The English-language accuracy gap is significant. The non-English accuracy gap is alarming.
Copyleaks, which markets itself heavily on multilingual support (30+ languages), achieves 99.12% accuracy on English text by their own measure. Independent testing on Spanish, French, Portuguese, and Hindi content shows accuracy dropping to 74–84% — a 15–25 point cliff. That's not a feature working as intended. That's a different tool entirely.
GPTZero is more candid about this: their supported language list is limited to five, and they acknowledge reduced performance outside English. Most detectors aren't this transparent.
"The accuracy claims aren't technically false — they're accurate on the benchmark the vendor chose. The problem is that benchmark doesn't reflect what users actually submit."
Section 2: The False Positive Problem
Accuracy gaps matter in the abstract. But false positives cause real harm — to real people, right now.
A false positive is when a detector flags human-written content as AI-generated. Here's what the false positive rates look like across vendors, per independent testing:
| Detector | Claimed FP Rate | Observed FP Rate | Notes |
|---|---|---|---|
| GPTZero | 1–2% | ~9% | Higher on formal/academic writing |
| Originality.AI | ~1% | 5–15% | Spikes on humanized content |
| Copyleaks | 0.2% | 3–7% | Varies by language and domain |
Who gets hurt
Students. When a student submits a paper written under exam conditions — high-pressure, formal prose, probably using some grammatical structures that AI systems also favor — detection tools can flag it at rates as high as 9–15%. The student gets accused of cheating. The burden of proof falls on them. Some educators are understanding. Many are not.
Non-native English speakers. This is the worst-affected group. Academic writing by non-native English speakers often skews formal, avoids idiom, and uses sentence structures that train data labels as "low perplexity" — which is the main signal many text detectors use. A Chinese PhD student writing in precise, structured English gets flagged more often than a native speaker writing colloquially. The tool penalizes clarity and caution.
Content marketers and SEO teams. An SEO agency producing 200 blog posts per month can't tolerate a 5–15% false positive rate. That's 10–30 articles per month flagged — articles the team actually wrote — that now have to be defended to clients. Real companies have lost retainers over this. Not because they cheated, but because the tool said they did.
In r/education, threads about false positives regularly pull hundreds of upvotes. The pattern: formal academic writing, non-native English, or polished professional prose gets flagged at rates users never expected from "99% accurate" tools.
Section 3: Why Aggregated Scores Lie
To address accuracy limitations, some tools have started offering "multi-API" detection — running content through multiple underlying models and returning a single aggregated score. The pitch is compelling: if one model says 75% AI and another says 85% AI, the aggregate (80%) is more reliable than either alone.
In practice, this approach has a serious problem: averaging dilutes the signal from the model that actually detected something.
The math problem with averaging
Imagine three APIs: Model A detects the text as 94% AI-generated. Model B says 48%. Model C says 52%. The average is 64.7%. Based on a typical threshold of 70%, the content gets cleared.
But Model A found something real. The 94% score wasn't noise — it was a strong detection signal from a model specialized for that type of content. Averaging it with two models that were confused brought the aggregate below threshold. The content passes.
This is a systematic problem with mean aggregation. It's not a bug in a specific tool — it's a mathematical property. Averaging heterogeneous classifier outputs suppresses minority signals, even when those signals are the most informative ones.
What's worse: the "99% accuracy" is built on this flawed aggregation
Many vendor accuracy benchmarks test their aggregated score against clean AI content. The aggregated score does perform well there — because all models agree when the content is pure AI text. Accuracy on clean content is high. Accuracy on ambiguous, edited, or mixed content — where models disagree and the average suppresses the strongest signal — is where performance drops. And that's exactly the scenario that matters.
"Aggregating classifier scores sounds like science. If you use the wrong aggregation method, it produces worse results than just using the best individual model."
Section 4: RealCheck's Approach — Raw Scores, Min-Score Rule, Transparency
We built RealCheck with one rule: never hide a detection signal.
Raw scores from every API
When RealCheck scans text, you see each model's score individually. If Model A says 94% AI and Model B says 48%, you see both numbers. You also see which model has higher confidence on this content type, and what factors drove the score.
This isn't just transparency theater. It gives you the information you need to make a judgment call. A 94% from a model specialized in GPT-4o detection, on a piece of text that might have been GPT-4o assisted, is meaningful. Hiding it behind a 64% average is not helpful.
The min-score rule
For high-stakes detection decisions, RealCheck uses the minimum score rule as the primary signal: if any integrated model returns a score above the detection threshold, the content is flagged. Not the average — the maximum.
Why? Because the cost of a false negative (missing AI content) and a false positive (flagging human content) are not symmetric in most use cases. In academic integrity contexts, missing AI content is the critical failure. The min-score rule is conservative in the right direction. Users can adjust this threshold based on their risk tolerance.
We report our actual false positive rate
Our current measured false positive rate on human-written content (English, general domain) is <5%. On formal academic writing, it's higher — we're transparent about that. We're actively working to reduce it. We don't claim 0.2%.
RealCheck will always publish our actual measured accuracy numbers — not best-case benchmarks on clean AI text. If our performance on a specific content type is lower, you'll see it in the documentation.
The AI detection market has a credibility problem that's entirely self-inflicted. Vendors raced to claim 99% accuracy because it sells. The users paying the price are the students, teachers, marketers, and writers who trusted those numbers and got burned.
Honest accuracy is a competitive advantage, not a liability. We'd rather have users who trust our numbers than users who distrust our tool.
RealCheck is building the honest alternative.
Raw scores. Transparent methodology. No inflated benchmarks. Join the waitlist — early access is opening soon.