Accuracy and error rates
How accurate is Fileverity at detecting AI-written text?
Measured error rates for FVDOC V3, Fileverity's document text model, by language, document type and length, and the conditions under which detection is harder.
Last updated
Short answer
In held-out validation of FVDOC V3 (October 2026), 0.2% of human-written English texts received an AI finding, and 0.1–0.9% across the five supported languages, while 98–100% of fully AI-written documents in our held-out document sets were detected. Detection is much lower when AI rewrites, lightly edits or inserts short passages into a human document, so a negative result never clears a document.
Detection of fully AI-written documents
| Documents | Detected |
|---|---|
| English | 99.5% |
| French | 100% |
| German | 98% |
| Spanish | 100% |
| Chinese | 100% |
| OpenAI models | 100% |
| Anthropic models | 99.3% |
| Google models | 100% |
| DeepSeek models | 99.5% |
| AI models never used in development, per model | 98.4–100% |
| A whole model family withheld from development | 96.5–100% |
Held-out validation of FVDOC V3, October 2026
AI-written imitations of real document types
AI asked to write a document of a genuine type on the same topic, such as a complaint letter or a case summary.
| Documents | Detected |
|---|---|
| Five languages | 92.5–95.2% |
| English | 94% |
| Claims-type documents | 91% |
Held-out validation of FVDOC V3, October 2026
Review measured detection rates, error rates and evaluation conditions.
Offline regression covered 38 scenarios and 35 distinct file contents, including 10 public documents, plus report generation and presentation in five languages. It tests file handling, correct rejection and evidence presentation, not AI/OCR or fraud-detection accuracy. Existing model figures apply only to their stated datasets, languages, tasks and releases, not to scanned fields, signatures or all documents.
Web limits are 50 MB, 100 pages and 60,000 words, with further per-check resource limits. Unexamined, partial and indeterminate results are distinct. An absence of findings is not proof of authenticity. There is no promise to recover deleted history, find all visual redaction remnants, or certify full UTS #39 conformance or PAdES long-term validation.
Sources
- EPA: public training documentsAccessed October 30, 2026
- Section508.gov: public document training materialsAccessed October 30, 2026
Terms used on this page
False positive
A human-written text that receives an AI finding. The false-positive rate is the share of human-written texts in a test set that did. A 0.2% false-positive rate does not mean a finding is 99.8% certain: how often a finding is right also depends on how many of the documents you check involve AI.
Detection rate
The share of AI-written documents in a test set that received an AI finding. Every detection rate here names the kind of document it was measured on.
Inconclusive
The result returned instead of a finding when the evaluated text supports neither conclusion. It is not an indication that AI was used.
Share of evaluated text in flagged passages
For partly flagged documents, the approximate part of the evaluated text that lies in flagged passages. It describes extent, never probability.
Held-out test set
Documents kept aside and never used to develop FVDOC V3, so that they measure it fairly. Every figure on this page comes from held-out sets.
Words evaluated and coverage
The running text long enough to be assessed, and its share of the document. Forms, tables and very short paragraphs are not evaluated.
False positives on human writing, by language
| Language | AI finding |
|---|---|
| English | 0.2% |
| French | 0.9% |
| German | 0.2% |
| Spanish | 0.1% |
| Chinese | 0.7% |
Held-out validation of FVDOC V3, October 2026
English professional documents
Real documents of the kinds businesses receive, each set measured separately.
| Document set | AI finding |
|---|---|
| English real-world PDF documents | 0% |
| English contracts, filings and terms | 0% |
| English grant proposals | 0% |
| English corporate emails | 0% |
| US consumer complaint narratives (English) | 0% |
| Real-world PDF documents, five languages | 0% |
| Legal texts, five languages | 0% |
Results for French, German, Spanish and Chinese professional documents are published on the accuracy page in each of those languages.
Held-out validation of FVDOC V3, October 2026
False positives by length
| Length of text | AI finding |
|---|---|
| 150 to 300 words | 0.7% |
| 300 to 1,000 words | 0.4% |
| Over 1,000 words | 0.2% |
Shorter texts are not evaluated. Chinese is counted at about two characters per word.
Held-out validation of FVDOC V3, October 2026
Formatting and templates
Letterheads, numbering, signature lines and boilerplate are not what the check looks for.
| Human-written text | AI finding |
|---|---|
| Re-laid out as a letter or contract with numbering, page numbers and signature lines: five languages | 0.2–0.7% |
| The same, English | 0.7% |
| Filled-in templates: five languages | 0% |
| The same, English | 0% |
Held-out validation of FVDOC V3, October 2026
How to read a result
AI-written text indicated: the evaluated text shows the characteristics of AI-written text throughout the document. Parts show AI-writing signals: some passages do, and each is listed with its page or paragraph. No significant AI-writing signals: the evaluated text did not show those characteristics, which does not exclude AI involvement. Inconclusive: the text supports neither conclusion, and this is not an indication that AI was used.
Not enough text to evaluate, Language not supported and No readable text are not findings. No report is issued and nothing is charged.
FVDOC V3 is designed to prefer a missed detection over a false accusation. That is why detection is lower for rewrites and light edits, and why passages that are not flagged are not cleared.
How we test
Every figure on this page comes from documents held out from development: none was used to build or tune FVDOC V3. The AI-written sets cover several model families, including one withheld from development entirely and models never used in development. The human-written sets cover web pages and professional documents in each language, from emails and contracts to court decisions and real-world PDFs.
Every figure is tied to a model version and a validation date, and every report cites the validation it relies on.
The validation is Fileverity’s own, run on held-out sets. Researchers and prospective customers who want to evaluate FVDOC V3 independently can request access under agreement through our contact page.
Versions and updates
FVDOC V3 replaced earlier versions of Fileverity's document model in October 2026. The model is re-evaluated before each release, and this page is updated with the new figures and date.
Every report names the model and release that produced it and links to this page, so a report issued today can still be read against the figures that applied when it was issued.
Where detection is harder
Detection is lower in the conditions below. They are stated plainly so your team can weigh a negative result correctly: a negative result never clears a document.
- AI that writes its own version of a human document, keeping its facts, names and dates, is often not detected.
- AI that rewords a human document, makes it more professional or humanizes it, is often not detected.
- Light AI editing of human text is detected less reliably, and how reliably depends on the AI model.
- AI passages inside an otherwise human document, short or large, are detected less reliably. Flagged passages are a minimum, not a complete list, and passages that are not flagged are not cleared.
- Small, scattered AI edits through a human document are often not detected.
- AI-assisted text, and AI text that has been paraphrased or back-translated, is detected less reliably.
- AI passages inside German documents are detected far less often than in the other four languages, and every German report says so.
What a result cannot tell you
- Who wrote the document, or which AI tool was used.
- Whether the contents are true, or whether using AI was allowed.
- That a document was never touched by AI: a negative result does not clear it.
- Anything about forms, tables, scanned pages without a text layer or texts shorter than about 150 words, which are not evaluated.
Frequently asked questions
How often does it flag human writing as AI?
In held-out validation of FVDOC V3 (October 2026): English 0.2%, French 0.9%, German 0.2%, Spanish 0.1% and Chinese 0.7%, on web pages and professional documents. On English professional documents, the rate was 0% for real-world PDFs and 0% for contracts, filings and terms.
Does it flag formal, template-based or reformatted writing?
Rarely. Human text re-laid out as a letter or contract, with numbering, page numbers and signature lines, received an AI finding in 0.2–0.7% of cases across the five languages, and 0% of filled-in English templates were flagged in held-out validation of FVDOC V3 (October 2026). Documents that are mostly form fields or tables return “Not enough text to evaluate”.
Can edited or “humanized” AI text still be detected?
Not reliably. Detection is lower when AI rewrites or lightly edits a human document. When AI rewords a human document, makes it more professional or humanizes it, the text is often not detected, and AI-assisted and paraphrased text is detected less reliably. A negative result therefore never clears a document.
Why does it sometimes say “inconclusive”?
Because some texts support neither conclusion, and a guess in either direction would be wrong too often. Inconclusive is not an indication that AI was used, and no validation rate applies to it. If the document matters, rely on other evidence, such as the original digital file or the sender's account of how it was drafted.
Which AI models has it been tested against?
Fully AI-written documents from the OpenAI, Anthropic, Google and DeepSeek model families were detected in 99.3–100% of cases, documents from AI models never used in development in 98.4–100%, and documents from a whole model family withheld from development in 96.5–100%, in held-out validation of FVDOC V3 (October 2026). Models outside those families are not named here.
How often is it re-evaluated?
Before every release of the model. Each release is validated on held-out sets, and this page is updated with the new figures and the validation date. Reports always name the model and release that produced them.
Who validated these figures?
Fileverity, on held-out sets that were never used in development. Researchers and prospective customers who want to evaluate FVDOC V3 independently can request access under agreement through our contact page.
Check the documents behind every decision.
Review submitted documents before approving a claim, paying an invoice or relying on a written statement. Check file history, signatures, scanned fields and AI-written text, with evidence your team can follow back to the source.