Evaluating detection
Can ChatGPT Tell If an Image Is AI-Generated? The Image Forensics Principle
Why understanding an image does not establish its origin: information theory, source detection and the measured FVIMG V2 ROC curve.
Published · Updated · 5 min read
On this page
Can ChatGPT tell if an image is AI-generated? It may identify useful clues, but understanding what an image shows does not establish how it was made. The central problem in image forensics is preserving and testing information about origin, including information that a perfectly accurate description can omit.
Fileverity's detection workflow does not call LLMs. We use specialized machine learning and technical verification to examine files. This article explains one public principle behind that choice—information about content is not the same as information about source—without describing our internal models or implementation.
A correct description can leave the detection question unanswered
Imagine two images of a red car beside a brick wall. One is a camera photograph; the other is generated. Both can be described correctly as “a red car beside a brick wall.” Lighting, perspective and object identity might also be described correctly in both cases.
The description has succeeded at its task. Yet the question “camera or generator?” remains open. Once two inputs have been mapped to the same description, a downstream decision based only on that description cannot distinguish those particular inputs. More eloquent reasoning over the same sentence does not restore their discarded differences.
This is why a list of visible oddities is an incomplete foundation for AI image detection. An anatomically plausible hand does not establish camera origin. An unusual reflection does not establish generation. Such observations can be useful clues, but their diagnostic value has to be tested rather than assumed.
For the practical inspection sequence, see how to tell if an image is AI-generated. Here, we examine why the underlying detection task requires more than a plausible account of the scene.
The information bottleneck: what reaches the decision?
Let Y denote the source category, X the available image and T a representation derived from that image, such as a description. If the downstream process receives only T, the question is how much source information T retains.
The data-processing inequality in MIT's information theory notes gives the relationship I(Y; T) ≤ I(Y; X) for this processing chain: transforming the input cannot create additional information about its source.
Conceptual illustration, not a diagram of Fileverity's architecture. X is the observed image, Y its source category and T a derived representation. No model-specific features are shown.
The important word is “can.” A representation need not lose source information: an appropriately chosen one may retain exactly what a detection task needs. But a representation that is good for scene description is not automatically sufficient for source classification. Those are different optimization goals.
This principle does not prove that every multimodal LLM is incapable of detection. Such a model may receive image information directly, and a specialized training procedure can change its capabilities. It establishes the narrower, stronger point: semantic competence alone is not evidence of validated source-detection performance.
How a detector turns evidence into a testable decision
Consider a generic measurement E. A source detector asks whether the observed E is more characteristic of generated images or camera photographs. Conceptually, that comparison can be expressed as:
L(E) = p(E | generated) / p(E | camera)
This is a likelihood ratio, not an assertion that Fileverity implements this formula directly. It explains the target: distinguish source-conditioned patterns, rather than judge whether a scene makes sense. A high ratio favors one explanation relative to the other under the assumed distributions; it is not automatically a posterior probability that the individual file is fake.
A learned detection score can be assessed empirically without knowing the true distributions. Set a threshold, apply it to labeled examples and count the outcomes. Moving the threshold changes the trade-off between finding generated images and flagging real photographs. The resulting ROC curve makes that trade-off inspectable.
Public research offers examples of this distinction. Zero-Shot Detection of AI-Generated Images studies statistical predictability under a model of real images. We cite it as an example of a research question about source signals, not as a disclosure of Fileverity's method.
From theory to observations: one frozen evaluation
The following figure is computed from the saved per-image predictions of our FVIMG V2 evaluation dated November 1, 2026. Unlike the conceptual diagram, these are measured results on two public datasets: generated images from Synthbuster and camera photographs from RAISE-1k. FVIMG V2 itself is developed and validated on massive, exclusive, proprietary and professionally curated datasets; the public benchmark lets anyone check the test setup independently.
Empirical ROC, with tied scores grouped before integration. The marked operating point uses the evaluation's fixed threshold: false-positive rate 3.70%, true-positive rate 97.54%. The matrix shows the share of each source row. No smoothing or simulated observations are used.
| What was measured | FVIMG V2 result |
|---|---|
| Generated images detected | 97.54% |
| Real photographs correctly below the AI threshold | 96.30% |
| Balanced accuracy | 96.92% |
| AUROC | 0.99625 |
The 3.70% of real photographs that crossed the threshold is the false-positive rate. The score separation supports a concrete detection claim on this evaluation; it does not turn the model score into a universal probability of authenticity. The full benchmark report records the input handling and scope: the pixel detection core, without metadata or provenance added to the headline result. LLMs were not tested as comparators.
Why complementary evidence can help
The same information principle explains the value of multiple checks. After observing E1, another measurement E2 is useful if it tells us something further about the source. Repeating E1 in different wording adds no new measurement. A genuinely complementary observation may.
That is the distinction between collecting more opinions and examining more evidence. Correlated checks can repeat the same error; a larger number of components is not a mathematical guarantee. The practical test is whether their combination improves measured decisions on representative files.
Fileverity applies this evidence-led approach to image verification. FVIMG V2 and our document detection model FVDOC V3 have separate version identities and evaluation scopes. We do not disclose their internal feature choices or combinations here; the public result is an inspectable report and a performance record attached to a defined task.
What this means when selecting an AI image detector
A useful question is not “Can the system explain this image?” but “What evidence supports its source classification, and how was that classification evaluated?” Ask for a named version, a defined test population and false-positive behavior alongside detection rate. Our accuracy guide explains how to interpret those measures.
Understanding a scene is valuable. Professional source detection adds a different requirement: retain relevant evidence, turn it into a defined decision and demonstrate how that decision behaves. That is the foundation of Fileverity's specialized verification workflow. If you plan to connect a detector to your own systems, How to Choose an AI Detection API covers ten integration checks.
By Yang Zheng