Scorecard
Model v16, measured 2026-11-20. All numbers come from images the detector never trained on.
What we catch
Share of AI images we flag as AI, per image generator. These generators were held back from training, so this is how we do on a generator we have never seen.
| Generator | Images | Caught | Caught after compression |
|---|---|---|---|
| FLUX | 352 | 96% | 97% |
How often real photos are wrongly flagged
Share of real photos we wrongly call AI, per photo source. Lower is better.
| Source | Photos | Wrongly flagged | After compression |
|---|---|---|---|
| RAISE (raw camera photos) | 299 | 7% | 10% |
| COCO* | 355 | 5% | – |
| Open Images* | 402 | 6% | – |
| Open Images originals (full-size camera files)* | 32 | 0% | – |
| wikimedia_commons* | 300 | 3% | – |
* Validation photos. The detector never trained on them, but its cut-off was tuned so that about 5% of them are flagged, so these rows sit near 5% by design. "–" means not measured.
Clean vs compressed
Images shared online are usually re-compressed. We test each image as stored and after compression.
| All test images | Clean | Compressed |
|---|---|---|
| AI images caught when 5% of real photos are flagged | 95% | 93% |
| Ranking score (AUROC: 100% is perfect, 50% is a coin flip) | 99% | 99% |
| Images tested | 651 | 651 |
The visual check (Claude Haiku)
| Right answer (an "uncertain" counts as wrong) | 96% |
|---|---|
| Answered "uncertain" | 2% |
| Test images | 300 |
What the percentage means
When we say 80% AI, about 80% of such images are AI. On test images the gap between what we say and how often it is true averages 2.8 percentage points for the visual check and 3.5 for the detector. A number near 50% means the evidence is mixed.
Usage
| Checks since launch | 2 |
|---|---|
| Checks in the last 30 days | 2 |
Not enough feedback yet.
Feedback comes from the "Was this right?" box under each result. Only people who chose to answer are counted, and one person can answer for several checks, so this is not a measured accuracy. We show it once there are at least 20 answers.
Honest limits
- Only fully AI-generated still images. Edited, inpainted or upscaled photos are out of scope.
- Few test generators. A new generator can fool the detector until we train on it.
- Some kinds of real photos are wrongly flagged far more often than others (see the table above).
- A result is a probability, not proof. Do not use it alone to accuse anyone.