DEV Community: Han Lou

I ran a million Secret Santa draws through my generator. Here's what the paper hat gets wrong

Han Lou — Mon, 02 Nov 2026 02:09:31 +0000

I run a small Secret Santa generator. Before this year's rush I wanted two things: proof that its draw is actually fair, and real numbers for the problems people hit when they draw names from a hat. So I took the draw module the server runs (TypeScript, about 400 lines), ran it in Node on a laptop and counted what came out.

Every simulated draw below got a fresh 128-bit seed from crypto.getRandomValues, the same way the server seeds it. Where an exact answer exists I computed it separately, by formula or by listing every possible draw, and checked the simulation against it.

1. The hat fails about 63% of the time

Put everyone's name in a hat, let everyone pull one. The chance that at least one person pulls their own name:

People	Someone draws their own name	Average full draws until a clean one
3	66.7%	3.00
4	62.5%	2.67
5	63.3%	2.73
6	63.2%	2.72
10	63.2%	2.72
30	63.2%	2.72

From 6 people up the number stops moving. A draw where nobody gets themselves is a derangement, and the share of shuffles that are derangements converges to 1/e (36.8%) almost immediately, so the failure rate sits at 1 − 1/e and the expected number of full draws sits at e, about 2.72.

As a sanity check I shuffled 8 names 1,000,000 times with a plain Fisher–Yates shuffle. 632,287 shuffles had at least one person holding their own name: 63.23%, against an exact 63.21%.

I keep a version of this table, along with what to do when it happens, on the Secret Santa rules page.

2. "Just redraw that one person" is worse than it sounds

The usual fix at the table: if you pull your own name, put it back and draw again. Two things go wrong.

First, the last person can be left holding only their own slip. Second, even when it works, the pairings are no longer equally likely. People who draw early pick freely, people who draw late get what's left, so some complete pairings can be reached in many ways and others in very few.

I computed both exactly by walking every possible sequence of draws:

People	Last person stuck with their own name	Most likely pairing vs least likely
4	13.9%	3×
5	13.2%	4×
6	11.3%	5×
8	9.0%	7×

In each case the gap between the most and least likely full outcome was exactly n − 1. Nobody at a party will notice a 5× bias, but it's the kind of thing a generator should not have. The only paper method that stays fair is the boring one: if anyone gets themselves, everyone puts their slip back and you start over.

3. What the generator does instead: shuffle, check, start over

That boring method is also the right algorithm. Shuffle uniformly, check every rule, and if anything fails, throw the whole shuffle away. Conditioning a uniform distribution on "the result is valid" gives a uniform distribution over valid results, so no valid draw is favoured.

A stripped-down version:

// allowed(g, r): may g give to r? (g !== r, plus any exclusions)
function drawNames(n, allowed, rng) {
  const a = Array.from({ length: n }, (_, i) => i);
  for (let tries = 1; ; tries++) {
    for (let i = 0; i < n; i++) a[i] = i;
    let ok = true;
    for (let i = 0; i < n && ok; i++) {
      const j = i + rng.int(n - i);       // forward Fisher–Yates
      [a[i], a[j]] = [a[j], a[i]];
      ok = allowed(i, a[i]);              // a[i] is final now, so check it early
    }
    if (ok) return { receivers: a, tries };
  }
}

Bailing out at the first broken rule is safe because the rejected attempt is discarded whole. The real module also checks up front that a valid draw exists at all (more on that below) and caps the number of attempts, so it can't spin forever.

Results from 1,000,000 draws of 8 people through the real module, no exclusions:

Average attempts: 2.719 (exact: 2.718)
Worst case: 32 attempts
Time: about 7.5 µs per draw on my laptop
The slower fallback for very tight rules was never needed

Is it actually uniform? With 5 people there are exactly 44 valid draws. In 440,000 runs each one should show up about 10,000 times. The rarest came up 9,826 times and the most common 10,204. A chi-square test gives 27.8 on 43 degrees of freedom, p = 0.96: nothing to see. I ran the same test on 6 people with 2 couples kept apart (116 valid draws, 400,000 runs each time) six times: p = 0.04, 0.54, 0.73, 0.96, 0.24 and 0.74. One run in twenty lands under 0.05 by chance with a fair draw, and the reruns of that setup scattered the way they should.

4. What exclusions cost

Couples who shouldn't draw each other are the most common rule. Every rule removes valid draws, so the shuffle gets rejected more often:

Group	Valid draws / all orderings	Share that pass	Measured attempts
6 people, 3 couples	80 / 720	11.1%	8.97
8 people, 2 couples	8,132 / 40,320	20.2%	4.98
8 people, 4 couples	4,752 / 40,320	11.8%	8.46
10 people, 5 couples	440,192 / 3,628,800	12.1%	8.24 (exact)
12 people, 6 couples			8.06
200 people, 100 couples			7.58

For a computer, 8 attempts is nothing; the 200-person case took about 5 ms per draw, most of it the up-front feasibility check. For a group doing it by hand with slips, "everyone puts their slip back" nine times in a row is where the evening goes. This is the case the generator with exclusions exists for.

5. Households and the "half" limit

Families usually want more than couples apart: everyone under one roof. That has a hard limit, and it's a textbook result. A draw exists only if every group of givers has at least as many allowed recipients as it has members (Hall's condition). A household of k people needs k recipients outside the house, so a household can be at most half the group.

Exact counts for 10 people with one household kept apart:

Household size	Valid draws (of 3,628,800 orderings)	Expected attempts
3	585,900 (16.1%)	6.2
4	181,440 (5.0%)	20
5	14,400 (0.4%)	252 (measured: 252.2)
6	0	impossible

At exactly half, the draw is forced into two teams: every household member buys for someone outside and every outsider buys for someone inside, 5! × 5! = 14,400 ways. Rejection sampling still copes at 252 attempts. One more person in the household and there's no valid draw, so instead of looping the module returns the blocking group: in my 6-of-10 test, five household members who between them could only give to four people. I go through the family version of this, with several households and kids, on the family Secret Santa page.

6. Mutual pairs, and why there's a "one big circle" switch

A uniformly random valid draw often contains two people who buy for each other. That's legal, but some groups find it flat:

People	At least one mutual pair	Draw is one single loop
4	33.3%	66.7%
6	39.6%	45.3%
8	39.4%	34.0%
12	39.4%	22.6%
20	39.4%	13.6%

The million-draw run agreed: 39.42% of the 8-person draws had a mutual pair and 33.96% formed one loop.

If you want a single chain (A gives to B, B to C, and the last person back to A), the generator has a one-big-circle mode that uses Sattolo's algorithm instead of Fisher–Yates. It's the backward Fisher–Yates loop with one small change, j = rng.int(i) instead of rng.int(i + 1), and it produces every single loop with equal probability. No mutual pairs and no separate loops by construction; with no exclusions it always passes on the first attempt, and with 4 couples among 8 people it took 3.39 attempts on average. The chain is handy when the reveal happens on a call: each person opens their gift, says who they had, and that person goes next. I wrote up how that works for remote teams on the virtual Secret Santa page.

7. Two smaller things that turned out to matter

Seeds make draws auditable. Every draw stores its 128-bit seed and runs it through a small PRNG (sfc32). Same seed and same inputs give the same result, so if someone says "the draw was rigged", it can be replayed.
Tight rules need a fallback, but a labelled one. If rejection sampling hasn't found a valid draw within its budget, the module switches to a randomized search plus random swaps that keep every rule. That's only approximately uniform, so the result records which method produced it. In all the runs above it never triggered.

The short version

A paper hat draw fails about 63% of the time for any group of 6 or more.
Redrawing only the unlucky person can strand the last drawer (9 to 14% of the time in groups of 4 to 8) and makes some pairings up to n − 1 times likelier than others. Restart from scratch instead.
Couples cost you roughly 5 to 9 full draws. Fine for code, painful with slips.
No household can be more than half the group.
In groups of 6 or more, about 39% of random draws contain a mutual pair; a single-loop draw avoids that.

I built secretsantagenerator.xyz. An AI agent working for me ran these simulations on the generator's draw code and wrote this post from the results, with no line-by-line human edit, so it's marked as AI-generated. The exact counts don't depend on randomness; the simulated figures should land within a fraction of a percent if you rerun them.

I retested my AI image checker on GPT Image 2.5, Nano Banana 2.1, FLUX.3 and Midjourney V7

Han Lou — Thu, 29 Oct 2026 02:20:44 +0000

Two weeks ago I ran 100 images through four AI image detectors to pick the engine behind a small checker I built. Since then OpenAI shipped GPT Image 2.5 (September 8) and Google shipped Nano Banana 2.1 (October 6). A detector that was fine on last season's models can fall behind quickly, so I retested the checker as it actually runs today on the newest generators I could get clean samples from.

The question was narrow: how often does an image from a new generator come back as anything other than "Likely AI-generated", and how often is a real camera photo flagged?

Rules first, results second

I wrote the protocol down before the first detection call: which images, how they're picked, how answers are counted, and what I'd change on the site depending on the result. The one decision rule: if any generator is missed more than 1 time in 5, results that come back "Likely a real photo" or "Can't tell" get a short note saying how many images from that generator we missed.

A miss means anything other than "Likely AI-generated". The checker has three answers: Likely AI (detection score 0.8 or above), Likely a real photo (0.2 or below) and Can't tell (in between). So a Can't tell on an AI image counts against it.

No threshold tuning, no swapping out images after seeing a score. Spares were allowed only for download or format failures, and they were fixed before any score existed.

Where the images came from

Provenance was the hard part. A file called "midjourney.png" on the internet proves nothing, so each bucket needed a source that records the model:

GPT Image 2.5, Nano Banana (2.1, 2, 2 Lite, Pro), FLUX.3 and Flux.2 [dev]: made with Civitai's on-site generator, which writes down the model it called. Text-to-image only, no source image, no LoRAs, at most one image per user. 20 per family.
Midjourney V7: original PNG downloads from a CC0 archive on Hugging Face whose embedded prompt carries --v 7 and a Midjourney job id. 20 images, but all from one creator, so they share a style.
Midjourney, recent community posts: 20 r/midjourney showcase posts. Reddit strips the prompt, so the version can't be verified. I report this row separately and never pool it with the others.
Real photos: 40 camera originals from Wikimedia Commons "Taken with …" categories, shot in 2024 or later: 24 from recent phones (iPhone 17 Pro, 16 Pro, Pixel 10 Pro, Galaxy S25 Ultra and others) and 16 from mirrorless cameras.

That's 140 images. I kept links and fingerprints, not the images.

Two findings before any model ran

None of the 60 Civitai files carried Content Credentials. That includes the 20 GPT Image 2.5 images. In my first test, all 8 original OpenAI files carried OpenAI-signed credentials (6 I made through OpenAI's image API, 2 from a public dataset), and the checker settled them without calling a model. Files that come out of a third-party generator site don't necessarily keep them, so here the detection model had to decide. If you're trying to work out whether ChatGPT made a particular picture, I wrote up when OpenAI's credentials settle it and when they don't, and what OpenAI's own verifier can and can't confirm.

Labels anyone can write were common. 6 of the 20 Nano Banana files and all 20 Midjourney V7 files carried an AI label in their metadata. The checker doesn't trust that kind of label on its own, because anyone can add it to a real photo, so those images still went to the model. Only signed credentials from signers on the C2PA trust lists settle an answer: in this run that happened for 3 real photos, which carried Google Pixel camera credentials. For a single picture, the order the checker works through (signed credentials, then metadata, then the model, with your own eyes last) is written up on an is this image AI-generated page.

A small product bug turned up as well: two 4K PNGs from Civitai were over the site's 25 MB upload limit. On the live site those uploads fail with "too large".

Results

Run on October 28, 2026, through the same code path the site uses, with Sightengine as the detection model.

Generator	Images	Likely AI	Can't tell	Called a real photo	Missed (95% range)
OpenAI GPT Image 2.5	20	20	0	0	0 of 20 (0–16%)
Google Nano Banana 2.1, 2 and Pro	20	20	0	0	0 of 20 (0–16%)
FLUX.3 and Flux.2 [dev]	20	19	1	0	1 of 20 (1–24%)
Midjourney V7	20	19	1	0	1 of 20 (1–24%)
Midjourney, community posts (version not verified)	20	18	1	1	2 of 20 (3–30%)

78 of the 80 verified AI images came back Likely AI and none were called a real photo. Every miss in the main rows was a Can't tell: a Flux.2 [dev] image scored 0.79, just under the 0.8 cut-off, and a Midjourney V7 image scored 0.55. In the community row, one post scored 0.77 (Can't tell) and one scored 0.06, the only AI image in the run the checker called a real photo.

Real photos: 38 of 40 came back Likely a real photo. One was flagged as AI with a score of 0.99, and one got a Can't tell at 0.64. That's 1 of 40 flagged (2.5%, 95% range 0–13%). Both errors were recent iPhone shots, an iPhone 17 and an iPhone 17 Pro. Two photos aren't a pattern, but it's the case I'd watch, because a false alarm lands on a real person. If the image you're checking is a viral picture or a screenshot rather than an original file, the clues are different; I keep notes on checking those.

No generator crossed the 1-in-5 line, so the decision rule said no warning, and I didn't add one.

What the numbers don't say

Twenty images per generator is small. "0 of 20 missed" is still compatible with missing about 1 in 6, which is what the 95% range shows. The Midjourney V7 images come from a single creator. The community Midjourney row can't be tied to a version. Civitai's generator is one way to reach these models, not the only one, and files from other apps may behave differently, especially around credentials. And vendors update their models, so this is a snapshot of one week.

Disclosure

I built this checker and it uses Sightengine as its main engine, with Hive as a backup, so I have a stake in these numbers looking good. That's why the protocol was written first and why every row is published, misses included. No detection company paid for this or saw it in advance.

The full table, the method and the earlier 100-image comparison of four engines are on the accuracy page. If you have an image from one of these generators that it gets wrong, I'd like to see it.

An AI agent working for me wrote this post from the test logs, with no line-by-line human edit, so it's marked as AI-generated. The images, the runs and the numbers are real; the method and the full table are on the accuracy page.

I tested 8 AI song detectors on the same songs. Here's what actually works

Han Lou — Thu, 22 Oct 2026 04:20:26 +0000

"Is this song AI?" is a question a lot of people now type into a search box, and it's one a chatbot can't answer, because it can't listen to the track. I wanted to build a small checker for it, so first I had to find out which detection engine actually works. Vendor accuracy claims don't say how they were measured, so I ran my own test: 8 detectors on the same songs, and 65 songs in total for the one that came out on top.

It's a small test with real gaps, and I list them at the end. But the results were lopsided enough to be worth sharing.

The test set

25 AI songs from 17 acts that the press has identified as AI, mostly released in 2025. Among them: The Velvet Sundown, Xania Monet, Breaking Rust, Cain Walker, Sienna Rose, Aventhis, Solomon Ray, King Willonius ("BBL Drizzy") and TaTa ("Pulse", made by Timbaland's team with Suno). The Velvet Sundown is the clearest case, because the band now says so itself: since July 2025 its Spotify bio has called it a synthetic music project made with the support of AI. The timeline and our scores are on Is The Velvet Sundown AI?. Country, gospel and soul show up a lot, because that's where many of the AI acts in the news are. On the gospel side, Apple Music describes Solomon Ray as an AI entity made by rapper Christopher Townsend; the charts it reached are on Is Solomon Ray AI?. The full list of acts, who identified each one as AI and how every song scored is on my AI artists page. One act that isn't in this test set: Eddie Dalton, the blues persona whose album The Years Between went to No. 1 on Billboard's Blues Albums chart; Newsweek reported it was made with AI. The reporting and our scores are on Is Eddie Dalton AI?.

40 human songs, picked to be hard. I didn't want an easy pass from acoustic ballads, so the set includes:

heavy Auto-Tune and vocoders: Cher's "Believe", T-Pain, Bon Iver's "715 - CRΣΣKS"
electronic production: Daft Punk, Skrillex, Radiohead's "Everything In Its Right Place"
country, soul and gospel in the same styles as the AI songs: Chris Stapleton, Zach Bryan, Leon Bridges, Kirk Franklin, CeCe Winans
recent hits a model can't have memorised for long: "Espresso", "Not Like Us", "DtMF", "The Fate of Ophelia"

Audio: the official 30-second preview from Apple Music (AAC 256 kbps). That's what a checker can legally fetch when someone pastes a link, so that's what I tested on.

The pass line was fixed before I ran anything: at least 80% correct, and no more than 10% of human songs flagged as AI, on a core set of 40 songs (20 AI, 20 human). If no engine passed, I wasn't going to launch. Later I added 25 more songs (5 AI, 20 human) to narrow the error range on false alarms.

The results

All engines on the same 40 core songs, each with a simple 0.5 cut-off:

Engine	Correct	Human songs flagged as AI	AI songs missed
fakeprint (open-source, runs locally)	95%	0 of 20	2 of 20
Deezer's album AI label (35 songs found on Deezer)	94%	0 of 18	2 of 17
fakeprint CNN (second model, same project)	92.5%	2 of 20	1 of 20
Gemini 4 Argon, asked to listen (35 usable replies)	88.6%	3 of 19	1 of 16
Gemini 3.8 Flash, asked to listen	82.5%	1 of 20	6 of 20
Hive, general AI-generated audio detection	62.5%	0 of 20	15 of 20
SONICS 5-second model (ICLR 2025)	57.5%	1 of 20	16 of 20
SONICS 120-second model	50%	20 of 20	0 of 20

AI or Not would have been number nine, but its music model returned HTTP 403 on the free plan, so it isn't in the table.

A few notes so nobody over-reads this:

These are 30-second previews of 40 songs, not product reviews. For Hive I could only reach its general AI-audio and deepfake endpoint, not its music-specific model, so the 62.5% doesn't rate that product.
The SONICS 120-second model expects far more audio than a preview gives it. Feeding it the 30-second clip tiled to length helped (65%, but 9 of 20 human songs flagged). It's a research model trained on Suno and Udio songs from its own dataset, and it shows.
The Gemini numbers are flattering on the human side. Both models recognise famous songs, so "this is human" on Queen or Eminem may just be memory. Even so, they flagged human songs: Gemini 3.8 Flash called Eminem's "Lose Yourself" AI, and Gemini 4 Argon flagged "Lose Yourself", "Bohemian Rhapsody" and Holding Absence's "Afterlife". Argon also returned unusable replies on 5 of the 40.
The fakeprint CNN flagged Billie Eilish's "bad guy" and "Bohemian Rhapsody". Strong on AI songs, but that's two famous human songs called AI out of 20.

Then I ran the winner on all 65 songs:

fakeprint, all 65 songs	Result
Correct	63 of 65 (96.9%)
Human songs flagged as AI	0 of 40 (95% interval 0–8.8%)
AI songs caught	23 of 25
Adding Deezer's album label ("either says AI")	64 of 65; still 0 of 40 human songs flagged
Highest score on a human song	0.0007
Lowest score on a caught AI song	0.873

That last pair is the part I find most convincing. The scores don't sit near the line. Human songs, including the Auto-Tune and vocoder tracks, were all below 0.001; the caught AI songs were all above 0.87.

Why I care more about false alarms than accuracy

"Accuracy" mixes two very different mistakes, and they don't cost the same.

A missed AI song is a shame. A human song called AI lands on a person: a singer who gets accused in the comments, an indie artist whose track gets pulled from a playlist, a label that has to defend a release. If most songs people check turn out to be human, which seems likely, even a small false-alarm rate does real damage.

Here's the arithmetic with made-up but plausible numbers. Say 1 in 10 songs people check is really AI, and a detector catches 95% of AI songs but also flags 10% of human ones. Out of 1,000 checks it makes 95 correct "AI" calls and 90 wrong ones. Nearly half of its "AI" answers would be accusations against real musicians, even though its headline accuracy looks fine.

Two more things worth knowing:

Zero isn't zero. 0 of 40 human songs flagged is a good result from a small test, but with only 40 songs the 95% interval still runs up to about 8.8%. Treat any "likely AI" verdict as one piece of evidence.
Give the detector a way to say "I don't know". The checker I built answers "Likely AI-generated" at 0.8 or above, "Likely human-made" at 0.2 or below, and "Can't tell" in between. None of the 65 Apple previews landed in the middle, but some re-encoded copies did, which is exactly what that middle band is for.

Does compression break it?

I re-encoded every preview three ways and also scored Deezer's own MP3 previews:

Copy	AI songs called "Likely AI"	Human songs called "Likely human"
Apple preview (AAC 256k), the main test	23 of 25	40 of 40
Re-encoded to MP3 128 kbps	21 of 25	40 of 40
Re-encoded to Opus 64 kbps	20 of 25	40 of 40
Downsampled to 22 kHz, then AAC 96 kbps	19 of 25	40 of 40
Deezer's MP3 128k previews (58 songs found there)	18 of 22	36 of 36

Heavy compression costs some AI catches; those songs mostly slid into "Can't tell". But no human song moved towards AI in any version, so the model is reacting to the generator, not to the codec. One odd case: Breaking Rust's "Livin' On Borrowed Time" scored 0.998 on Apple's preview and 0.0006 on Deezer's. I don't know why; Deezer may carry a different master. That's why the checker fetches Apple's preview first. Billboard calls Breaking Rust an AI-powered country act, and its "Walk My Walk" topped Country Digital Song Sales; the sources and our test scores are on Is Breaking Rust AI?.

How the fingerprint works (and one porting trap)

AI music generators build a song as a compressed code and then turn it into sound with a neural audio decoder. The decoder's upsampling layers leave a row of evenly spaced peaks in the spectrum, roughly between 1 and 8 kHz. You can't hear it, but you can measure it. Researchers at Deezer described this in a 2025 ISMIR paper (Afchar et al., "A Fourier Explanation of AI-music Artifacts").

The open-source fakeprint model (lofcz/ai-music-detector, MIT licence) turns that into a tiny detector: resample to 16 kHz, take an 8192-point FFT, average the spectrum over the clip, subtract its lower envelope so the music's own shape drops out, and feed the remaining peaks to a 15 KB logistic regression. In my runs it took about a tenth of a second per song and cost nothing.

The trap if you port it: the resampler has to match training, which used torchaudio's default sinc filter. A different resampling filter can create spectral peaks of its own, and then you're measuring your resampler instead of the generator. The model's author ships alignment tests for exactly this. I ported it to run in the browser, so the first pass on an uploaded file happens on the device, and checked the port against the Python scores on all 65 songs: the largest difference was under 0.0005, and no verdict changed.

What this test can't tell you

New generators can slip past it. The model card covers Suno up to v5 and Udio up to v1.5. Suno released v6 on September 9, 2026, after these songs were made, and I haven't tested it, or ElevenLabs Music, Google Lyria or Mureka. Expect misses until it's retested.
Part-AI songs usually pass as human. TaTa's "Pulse" uses an AI voice with human production and scored below 0.001. Hive, both Gemini models and Deezer's label missed it too. The check answers "was this whole track generated?", not "was AI used anywhere?".
The licensing has a grey area. The model's code and weights are MIT, but its training data includes the SONICS dataset, licensed CC BY-NC 4.0 for research, plus data the author doesn't publish. Deezer has also filed patent applications on AI-detection methods; I haven't checked whether they cover this one.
It's small and I picked it. 25 AI songs, all from acts named in the press and mostly from 2025, and 40 human songs I chose. A different set could score differently, and the range on the AI miss rate is still wide (2.2–25%).
Deezer's label comes from an undocumented interface that can change, and it only applies to whole albums. I also looked at it while choosing AI songs, so its AI-side result isn't blind. Its human-side result (0 of 36 flagged) is.
I'm not neutral. I built a checker on top of fakeprint and Deezer's label, so I have an interest in good numbers. No vendor paid for this or saw it first. Hive ran on free trial credit and Gemini through OpenRouter at standard prices (about $0.003 per check).

If you want to check one song

Look at who's behind it. An artist with no live shows, no interviews and a burst of releases in a few months is a reason to look closer, not proof.
Check the platform's own labels. Deezer tags albums its detector found to be AI; it's worth a look if the song is there.
Run a detector as one more clue, preferably one that shows its score, its reasons, and is allowed to say "can't tell".
Don't use a detector's answer alone to accuse a real musician. Not mine, not anyone's.

A longer version of this list, nine clues with how much each one tells you (Spotify's AI Persona badge and AI credits included), is on How to tell if a song is AI.

I put this into a small free checker: paste a Spotify, Apple Music, YouTube or Deezer link, or upload a file. The first pass runs in your browser; for the deeper check a short excerpt (30 to 60 seconds) is sent to the server and then discarded, and no audio is stored. It's at isthissongai.app (1 free check a day, no sign-up). Every song, every score, the re-encoding results and the full list of limits are on the accuracy page. If it gets one of your songs wrong, I'd really like to hear about it.

Sources: Afchar et al., A Fourier Explanation of AI-music Artifacts, ISMIR 2025; lofcz/ai-music-detector model card; SONICS dataset.

I used an AI assistant to help draft and edit this post. The test set, the runs and the numbers are from my own test, published in full on the accuracy page.

AI画像判定ツール4つを100枚で検証してわかった、正解率より大事なこと

Han Lou — Wed, 21 Oct 2026 13:57:20 +0000

X で拡散している画像や、LINE で回ってきたスクショを見て「これ、生成AIで作ったものでは？」と思ったことがある人は多いと思います。AI画像の判定ツールはいくつもあり、どのサービスも正解率をうたっています。ただ、その数字をどう測ったのかはどこにも書かれていないので、比べようがありません。

私はいま小さなAI画像判定ツールを作っていて、裏側で使う検出エンジンを選ぶ必要がありました。そこで自分でテストしてみました。規模は小さく、抜けもあります（後でまとめて書きます）。それでも、この種のツールに対する見方はかなり変わりました。

テストに使った画像

100枚。すべて2026年10月13日に実行しました。

本物の写真50枚。 大半は Wikimedia Commons の CC0 またはクリエイティブ・コモンズ・ライセンスの画像です。うち38枚は元ファイルで、スマホのスナップ、ポートレート、風景、商品写真です。50枚中37枚は EXIF にカメラの機種名が残っていて、iPhone、Pixel、Galaxy のものでした。残りの12枚は、「出回った写真」に見えるように私が作ったコピーです。スマホ風のスクショ4枚、チャットアプリでよくあるように強く再圧縮したもの4枚、軽くレタッチしたもの4枚です。

AI画像50枚。 6つの画像生成AIのシリーズから集めました。OpenAI GPT Image、Google Nano Banana、Black Forest Labs FLUX.2、Midjourney（v6〜v8.1）、ByteDance Seedream、Stable Diffusion XL です。42枚は2025年以降に公開された生成AIによるものです。AIの元ファイル40枚のうち、27枚はこのテストのために自分で生成し（API料金は約1.42ドル）、13枚は Hugging Face 上のオープンライセンスのデータセットから取りました。最後の10枚はAI画像をスマホ風にスクショしたもので、PNG で保存し、メタデータは一切ありません。

採点のしかた

すべてのエンジンに同じファイルを渡しました。回答はエンジンごとに決めた固定のルールで「AI」か「本物」に変換しています。Sightengine はスコア0.5以上、Hive は公式推奨のしきい値0.9、AI or Not と WasItAI は各社自身の判定をそのまま使いました。「不明」という回答は不正解として数えています。

数字を見る前に一つ注意があります。100枚すべてを見たのは2つのエンジンだけです。無料トライアルの範囲内で済ませたため、AI or Not は層化抽出した20枚、WasItAI は9枚だけで実行しました。

結果

エンジン	テスト枚数	正解率	本物の写真をAIと誤判定	検出できたAI画像
Sightengine	100	99%	50枚中0枚	50枚中49枚
Hive	100	95%	50枚中0枚	50枚中45枚
AI or Not	20（サンプル）	85%	10枚中1枚	10枚中8枚
WasItAI	9（サンプル）	78%	5枚中2枚	4枚中4枚

下の2行はランキングとして読まないでください。20枚と9枚では少なすぎます。AI or Not のサンプルは全体より難しく、スクショや再圧縮したコピーの割合が高めでした。唯一の誤判定も、軽くレタッチした写真です。4つのエンジンすべてが見た9枚に限ると、Sightengine と Hive は9枚中9枚、AI or Not は9枚中8枚、WasItAI は9枚中7枚が正解でした。目安にはなりますが、結論ではありません。

生成AI別に見ると、いちばん難しかったのは FLUX です。Sightengine は FLUX の画像9枚中8枚、Hive は9枚中6枚を検出しました。Midjourney の画像は、両エンジンとも全部検出しています。

正解率より誤判定を見る理由

「正解率」には、性質のまったく違う2種類の間違いが混ざっています。私のテスト用プログラムには、どんな画像にも「AI」と答える偽のエンジンがあります（配線の確認にだけ使うものです）。このテストセットでは、それでもAI画像50枚を全部当てて、正解率は50%になります。本物の写真50枚もすべてAIと判定しているからです。

この2つの間違いは、重さが違います。AI画像を見逃すのは残念なことです。でも本物の写真をAIと言われると、その被害は実在の人に向かいます。フリマアプリで本当に不良品が届いて返金を求めた購入者、線を一本一本自分で描いたイラストレーター、マッチングアプリでただ写真がうまいだけの人。チェックする画像の大半が実は本物なら、誤判定率が小さくても、正しい指摘より間違った疑いのほうが多くなりえます。

ほかにも、予想していなかったことが2つあります。

しきい値は「選択」です。 Hive の推奨値0.9では、本物の写真の誤判定はゼロで、AI画像の90%を検出しました。0.5にすれば96%を検出できますが、本物の写真50枚中1枚を誤判定します。こうしたエンジンを使った製品は、どれもこの曲線上のどこかを選んでいて、たいていそれを教えてくれません。
ゼロは「ゼロ」ではありません。 Sightengine と Hive はどちらも、本物の写真50枚中0枚の誤判定でした。ただ50枚しかないので、95%信頼区間は0%から約7%まであります。小さなテストとしては良い兆候ですが、保証ではありません。

コンテンツクレデンシャル：多くの人が飛ばしている無料のチェック

いくつかのAIツールは、生成したファイルに C2PA という標準規格で署名するようになりました。一般にはコンテンツクレデンシャル（Content Credentials）と呼ばれています。記録にはどのツールで作ったかが書かれていて、暗号技術でファイルと結びついています。読み取りは無料で、ブラウザ上でもできます。

今回のセットでは、AI画像50枚中24枚にコンテンツクレデンシャルが付いていて、本物の写真には「AI生成」という記載のあるものは一枚もありませんでした。署名元は OpenAI、Google、Black Forest Labs、BytePlus（ByteDance）です。24枚はすべて元ファイルで、AIの元ファイル40枚のうち24枚に付いていたことになります。

後になってから本気で気にするようになった落とし穴が一つあります。署名が有効だとわかるのは「誰が署名したか」であって、「その署名者を信用してよいか」ではありません。24枚のうち、C2PA の公式信頼リストに載っている企業（OpenAI と Google）が署名していたのは14枚だけでした。残りの10枚は、リストにない証明書で署名されていました。自分で作った証明書なら、何にでも署名できます。そのため私のツールでは、この10枚と、Midjourney のような単なるメタデータのラベルは「自己申告」として扱い、検出モデルも実行しています。

注意点をまとめます。

Midjourney のファイルには一度も付いていませんでした（元ファイル7枚中0枚）。Stable Diffusion XL も同じです（5枚中0枚）。
スクショにすると消えます。 10枚のスクショには一枚も付いていませんでした。スクショはまったく新しいファイルなので、署名は引き継がれません。
私のファイルの一部は、それ以前に失っていたかもしれません。 公開データセットから取ったものがいくつかあり、保存し直したコピーでは記録が落ちることがあります。数が少ないからといって、その生成AIが署名しないとは言い切れません。
カメラも署名します。 Google Pixel 9 と Pixel 10 で撮った本物の写真2枚には、カメラのクレデンシャルが付いていました。クレデンシャルが本物の写真をAIと示したことは一度もありません。

つまり、コンテンツクレデンシャルは、あればほぼ証拠になり、なくても何もわかりません。X で拡散している画像をスクショで見かけたなら、見つからないものと思っておいたほうがいいでしょう。

上の表は、エンジン同士を比べるために単純な固定の線（Sightengine なら0.5）で判定しています。ツール自体はもっと慎重で、0.8を超えたらAI、0.2未満なら本物、その間は「判定できません」と答えます。クレデンシャルを先に読み、残りだけを Sightengine に送って、この2本の線で判定した結果は、100枚中98枚が正解、1枚が「判定できません」（本物の写真のスクショで、スコア0.21）、1枚が誤り（FLUX のスクショで、スコア0.18、本物と判定）でした。本物の写真の誤判定は、やはり50枚中0枚です。16枚は信頼できるクレデンシャルだけで決着し、モデルを呼ぶ必要がありませんでした（AI画像14枚と、本物の撮影として署名された Pixel の写真2枚）。残りの84枚がモデルに回りました。

スクショは、モデルにとっても少し不利です。AIの元画像とそのスクショのペア10組で比べると、Sightengine は検出数が10から9に、Hive は10から8に下がりました。

第3の答え：「AIで編集済み」

テストセットを作っている途中、ある Pixel の写真のメタデータに「Edited with Google AI」と書かれていました。完全な本物でも、完全な生成画像でもないので、本物の写真のグループからは外して差し替えました。主要な2エンジンはどちらも本物と判定していて、それはそれで妥当です。ただ、多くのスマホ写真にとって「本物かAIか」という問い自体がずれてきていることがわかります。役に立つ判定ツールには、3つ目の答えが必要です。

このテストでわからないこと

規模が小さいです。100枚、1日だけです。
4つのエンジンのうち2つは、サンプルしか見ていません。
マッチングアプリの自撮り、強いフィルターをかけたポートレート、手描きの作品、合成画像や部分的なAI編集は入っていません。イラストも数枚だけです。
本物の写真1枚だけは、1日の無料枠がリセットされた後、ほかより数時間遅れて Sightengine と Hive にかけました。
各社はモデルを更新するので、これはある時点のスナップショットです。
私は中立ではありません。私が作ったツールはいま Sightengine を使い、Hive を予備にしています。どの事業者もこのテストにお金を出しておらず、事前に内容を見てもいません。4つとも無料トライアルのクレジットで実行しました。

1枚の画像を確かめるなら

ある1枚の画像について知りたいだけなら、私はこの順番で確かめます。

コンテンツクレデンシャルを探す。 Content Authenticity Initiative の無料ツール Verify（contentcredentials.org/verify）で読み取れます。画像生成AIの名前が出れば、ほぼそこで終わりです。何も出なくても次へ進んでください。それが普通です。
いちばん古いコピーを探す。 Google レンズ、TinEye、Bing で画像検索（逆画像検索）をします。その画像が最初に出てきた場所から、誰がなぜ作ったのかがわかることがよくあります。
場面のつじつまが合っているかを考える。 制服は合っているか、道路標識は合っているか、天気はその日付と合っているか。このチェックは、生成AIが進歩しても弱くなりません。
それから細部を見る。 小さな文字、物を持つ手、映り込み、影、人混みの中で繰り返される顔。これらは手がかりにすぎません。おかしな細部が一つあってもAIの証明にはならず、完璧な画像だからといってカメラで撮った証明にもなりません。見分け方のコツは「AI画像の見分け方」にもまとめています。
判定ツールは手がかりの一つとして使う。 スコアの幅と理由を示してくれて、「判定できません」と言えるものを選ぶのがおすすめです。できれば元のファイルを使ってください。スクショは判定が難しくなります。

何がわかったとしても、判定ツールの答えだけを根拠に、実在の人を責めないでください。

このクレデンシャル優先の流れを、無料の小さなツール「AI画像判定」にしました。テストの方法の全体、生成AIごとの数字、限界は正確性についてのページ（英語）に載せています。うまく判定できない画像があれば、ぜひ見せてください。

この記事の英語版は dev.to に掲載しています：I tested 4 AI image detectors on 100 images

この記事の下書きと編集には AI アシスタントの手を借りました。テストセット、実行、数字はすべて私自身のものです。

KI-Bild oder echt? Ich habe 4 Detektoren mit 100 Bildern getestet

Han Lou — Wed, 21 Oct 2026 13:56:09 +0000

Irgendwann landet so ein Bild im Familienchat: eine WhatsApp-Weiterleitung, schon etwas verpixelt, und darunter die Frage „Ist das echt?“. KI-Detektoren gibt es inzwischen reichlich, und jeder Anbieter wirbt mit einer Trefferquote. Wie die gemessen wurde, schreibt keiner dazu. Vergleichen lassen sich die Zahlen also nicht.

Ich baue gerade einen kleinen KI-Bild-Checker und musste mich irgendwann für die Erkennungs-Engine dahinter entscheiden. Also habe ich selbst getestet. Der Test ist klein und hat Lücken (die liste ich unten auf), aber er hat verändert, wie ich über diese Tools denke.

Die Testbilder

100 Bilder, alle am 13. Oktober 2026 durchgelaufen.

50 echte Fotos, größtenteils von Wikimedia Commons unter CC0- oder Creative-Commons-Lizenz. 38 davon sind Originaldateien: Handyfotos, Porträts, Landschaften, Produktfotos. Bei 37 der 50 stand noch ein Kameramodell in den EXIF-Daten, von iPhone-, Pixel- und Galaxy-Handys. Die übrigen 12 sind Kopien, die ich so aufbereitet habe, dass sie wie herumgereichte Fotos aussehen: 4 Screenshots im Handy-Stil, 4 stark nachkomprimierte Kopien, wie Messenger sie erzeugen, und 4 leicht retuschierte Versionen.

50 KI-Bilder aus sechs Generator-Familien: OpenAI GPT Image, Google Nano Banana, Black Forest Labs FLUX.2, Midjourney (v6 bis v8.1), ByteDance Seedream und Stable Diffusion XL. 42 stammen von Generatoren, die 2025 oder später erschienen sind. Von den 40 KI-Originaldateien habe ich 27 extra für diesen Test erzeugt (rund 1,42 US-Dollar an API-Kosten), 13 stammen aus offen lizenzierten Datensätzen auf Hugging Face. Die letzten 10 sind Screenshots von KI-Bildern im Handy-Stil, als PNG gespeichert, ganz ohne Metadaten.

So habe ich gewertet

Jede Engine bekam dieselben Dateien. Jede Antwort habe ich mit einer festen Regel pro Engine in „KI“ oder „echt“ übersetzt: bei Sightengine ein Score ab 0,5, bei Hive der vom Anbieter selbst empfohlene Schwellenwert von 0,9, bei AI or Not und WasItAI das Urteil des Anbieters. Ein „unbekannt“ zählte als falsch.

Eine Einschränkung vorab: Nur zwei Engines haben alle 100 Bilder gesehen. Ich bin innerhalb der kostenlosen Testkontingente geblieben, deshalb lief AI or Not nur auf einer geschichteten Stichprobe von 20 Bildern und WasItAI auf 9.

Die Ergebnisse

Engine	Getestete Bilder	Genauigkeit	Echte Fotos als KI markiert	Erkannte KI-Bilder
Sightengine	100	99 %	0 von 50	49 von 50
Hive	100	95 %	0 von 50	45 von 50
AI or Not	20 (Stichprobe)	85 %	1 von 10	8 von 10
WasItAI	9 (Stichprobe)	78 %	2 von 5	4 von 4

Bitte lies die unteren beiden Zeilen nicht als Rangliste. 20 und 9 Bilder sind zu wenig. Die Stichprobe von AI or Not war außerdem schwerer als der Gesamtsatz, mit mehr Screenshots und nachkomprimierten Kopien, und ihr einziger Fehlalarm war ein leicht retuschiertes Foto. Bei den 9 Bildern, die alle vier Engines gesehen haben, kamen Sightengine und Hive auf 9 von 9, AI or Not auf 8 von 9 und WasItAI auf 7 von 9. Ein Hinweis, kein Urteil.

Nach Generator war FLUX am schwierigsten: Sightengine erkannte 8 von 9 FLUX-Bildern, Hive 6 von 9. Jedes Midjourney-Bild haben beide Engines erkannt.

Warum mich Fehlalarme mehr interessieren als die Genauigkeit

„Genauigkeit“ wirft zwei sehr unterschiedliche Fehler in einen Topf. In meinem Test-Setup gibt es eine Fake-Engine, die bei jedem Bild „KI“ sagt; die nutze ich nur, um die Pipeline durchzutesten. Auf diesem Testsatz erkennt sie alle 50 KI-Bilder und kommt trotzdem auf 50 % Genauigkeit, weil sie eben auch alle 50 echten Fotos markiert.

Die beiden Fehler kosten nicht dasselbe. Ein übersehenes KI-Bild ist schade. Ein echtes Foto, das als KI abgestempelt wird, trifft einen Menschen: die Käuferin, die zu Recht ihr Geld zurückverlangt, den Zeichner, der jeden Strich selbst gesetzt hat, jemanden in einer Dating-App, der einfach gute Fotos macht. Wenn die meisten Bilder, die du prüfst, am Ende echt sind, kann schon eine kleine Fehlalarmquote mehr falsche Beschuldigungen produzieren als richtige.

Zwei Dinge, mit denen ich nicht gerechnet hatte:

Der Schwellenwert ist eine Entscheidung. Mit dem von Hive empfohlenen Schwellenwert von 0,9 wurde kein echtes Foto markiert, und 90 % der KI-Bilder wurden erkannt. Bei 0,5 wären es 96 % gewesen, dafür aber 1 von 50 echten Fotos als Fehlalarm. Jedes Produkt, das auf diesen Engines aufsetzt, wählt einen Punkt auf dieser Kurve, meistens ohne es dir zu sagen.
Null ist nicht null. Sightengine und Hive haben beide 0 von 50 echten Fotos markiert. Bei nur 50 Fotos reicht das 95-%-Konfidenzintervall trotzdem von 0 % bis etwa 7 %. Ein gutes Zeichen aus einem kleinen Test, keine Garantie.

Content Credentials: der kostenlose Check, den die meisten auslassen

Manche KI-Tools signieren ihre Dateien inzwischen nach dem C2PA-Standard, öffentlich bekannt als Content Credentials. Der Eintrag nennt das Tool und ist kryptografisch an die Datei gebunden. Ihn auszulesen kostet nichts und geht direkt im Browser.

In meinem Testsatz trugen 24 der 50 KI-Bilder Content Credentials, und kein einziges echtes Foto trug eine KI-Angabe. Als Unterzeichner tauchten OpenAI, Google, Black Forest Labs und BytePlus (ByteDance) auf. Alle 24 waren Originaldateien: 24 der 40 KI-Originale hatten Credentials.

Einen Haken habe ich erst später ernst genommen: Eine gültige Signatur sagt dir, wer unterschrieben hat, nicht, ob du dem trauen solltest. Nur 14 der 24 waren von Firmen signiert, die auf der offiziellen C2PA-Vertrauensliste stehen (OpenAI und Google). Die anderen 10 waren mit Zertifikaten signiert, die dort nicht stehen, und ein selbst ausgestelltes Zertifikat kann alles signieren. Mein Checker behandelt diese 10, genauso wie reine Metadaten-Labels wie das von Midjourney, deshalb nur als Behauptung und lässt trotzdem das Erkennungsmodell laufen.

Die Haken im Einzelnen:

Midjourney-Dateien hatten nie welche (0 von 7 Originalen), Stable Diffusion XL ebenfalls nicht (0 von 5).
Screenshots verlieren sie. Keiner der 10 Screenshots trug Credentials. Ein Screenshot ist eine komplett neue Datei, die Signatur kommt nicht mit.
Manche meiner Dateien haben sie vielleicht schon vorher verloren. Einige stammen aus öffentlichen Datensätzen, und eine neu gespeicherte Kopie kann den Eintrag verlieren. Eine niedrige Zahl beweist also nicht, dass ein Generator nie signiert.
Auch Kameras signieren. Zwei echte Fotos, von einem Google Pixel 9 und einem Pixel 10, trugen Kamera-Credentials, und Credentials haben nie ein echtes Foto als KI markiert.

Heißt: Wenn Content Credentials da sind, sind sie fast ein Beweis. Fehlen sie, sagt das gar nichts. Wenn dich ein virales Bild als Screenshot erreicht, rechne damit, keine zu finden.

Die Tabelle oben nutzt eine einfache feste Grenze (bei Sightengine 0,5), damit sich die Engines vergleichen lassen. Das Tool selbst ist vorsichtiger: Es nennt ein Bild über 0,8 KI, unter 0,2 echt und sagt dazwischen „kann ich nicht sagen“. Credentials zuerst lesen, nur den Rest an Sightengine schicken, mit diesen beiden Grenzen: 98 von 100 richtig, 1 „kann ich nicht sagen“ (ein Screenshot eines echten Fotos mit Score 0,21), 1 falsch (ein FLUX-Screenshot mit 0,18, als echt eingestuft), und weiterhin 0 von 50 echten Fotos markiert. 16 Bilder waren allein über vertrauenswürdige Credentials geklärt, ganz ohne Modellaufruf (14 KI-Bilder plus 2 Pixel-Fotos, die als echte Aufnahmen signiert waren). Die anderen 84 gingen ans Modell.

Screenshots kosten auch die Modelle etwas. Bei 10 Paaren aus KI-Original und dazugehörigem Screenshot fiel Sightengine von 10 erkannten auf 9, Hive von 10 auf 8.

Eine dritte Antwort: „mit KI bearbeitet“

Beim Zusammenstellen hatte ein Pixel-Foto in den Metadaten den Vermerk „Edited with Google AI“. Es ist weder ganz echt noch ganz generiert, also habe ich es aus den echten Fotos herausgenommen und ersetzt. Beide Haupt-Engines hatten es als echt eingestuft, was fair ist. Es zeigt aber, dass „echt oder KI“ für viele Handyfotos zunehmend die falsche Frage ist. Ein brauchbarer Checker braucht eine dritte Antwort.

Was dieser Test nicht leisten kann

Er ist klein: 100 Bilder, ein Tag.
Zwei der vier Engines haben nur Stichproben gesehen.
Es fehlen Dating-App-Selfies, stark gefilterte Porträts, handgemachte Kunst sowie Composings oder teilweise mit KI bearbeitete Bilder. Illustrationen sind nur wenige dabei.
Ein echtes Foto lief ein paar Stunden nach den anderen durch Sightengine und Hive, nachdem sich deren tägliche Gratis-Kontingente zurückgesetzt hatten.
Anbieter aktualisieren ihre Modelle, das hier ist eine Momentaufnahme.
Ich bin nicht neutral. Mein Checker nutzt inzwischen Sightengine, mit Hive als Backup. Kein Anbieter hat für den Test bezahlt oder ihn vorab gesehen, und alle vier liefen über kostenlose Testguthaben.

So würde ich ein einzelnes Bild prüfen

Wenn du nur wissen willst, was mit einem bestimmten Bild los ist, würde ich in dieser Reihenfolge vorgehen:

Nach Content Credentials schauen. Das kostenlose Verify-Tool der Content Authenticity Initiative (contentcredentials.org/verify) liest sie aus. Nennt es einen KI-Generator, bist du im Grunde fertig. Zeigt es nichts an, mach weiter; das ist normal.
Die früheste Kopie finden. Mach eine Bilder-Rückwärtssuche mit Google Lens, TinEye oder Bing. Wo ein Bild zuerst aufgetaucht ist, verrät oft, wer es gemacht hat und warum.
Fragen, ob die Szene stimmt. Passen die Uniformen, die Verkehrsschilder, das Wetter zum Datum? Dieser Check wird nicht schwächer, wenn die Generatoren besser werden.
Dann genau hinschauen. Kleine Schrift, Hände, die etwas halten, Spiegelungen, Schatten, sich wiederholende Gesichter in einer Menschenmenge. Das sind Indizien: Ein seltsames Detail beweist keine KI, und ein makelloses Bild beweist keine Kamera. Worauf man im Detail achten kann, habe ich unter Woran erkennt man KI-Bilder? gesammelt.
Einen Detektor als weiteres Indiz nutzen. Am besten einen, der eine Spanne und seine Gründe zeigt und auch „kann ich nicht sagen“ antworten darf. Nimm wenn möglich die Originaldatei, Screenshots sind schwerer.

Egal, was du findest: Beschuldige keinen echten Menschen allein aufgrund der Antwort eines Detektors.

Diesen Ablauf mit Credentials zuerst habe ich in einen kleinen kostenlosen Checker gepackt: KI-Bild erkennen. Die vollständige Methode, alle Zahlen nach Generator und die Grenzen des Tests stehen auf der Seite zur Genauigkeit (auf Englisch). Wenn du ein Bild hast, bei dem er danebenliegt, würde ich es gern sehen.

Die englische Originalfassung ist auf dev.to erschienen: I tested 4 AI image detectors on 100 images

Beim Entwerfen und Redigieren dieses Beitrags hat mir ein KI-Assistent geholfen. Die Testbilder, die Durchläufe und die Zahlen sind meine eigenen.

實測 4 款 AI 圖片偵測工具（100 張圖）：比準確率更該看的是誤判

Han Lou — Wed, 21 Oct 2026 13:47:34 +0000

LINE 群組裡又轉來一張截圖，可能是某個名人「推薦」投資群組的廣告，可能是哪裡淹水的現場照片，你想知道它是不是 AI 做的。上網一查，AI 圖片偵測工具一大堆，每家都寫著自己的準確率有多高，卻沒有一家說清楚是怎麼量出來的，根本沒辦法比。

我最近在做一個小小的 AI 圖片檢查工具，總得替它選一個偵測引擎。既然各家的數字沒辦法比，我就自己測。規模不大，也有不少缺口（後面會一條條列出來），但測完之後，我看這類工具的方式整個變了。

測試用的圖

100 張圖，全部在 2026 年 10 月 13 日跑完。

真實照片 50 張，大多來自 Wikimedia Commons，授權是 CC0 或創用 CC。其中 38 張是原始檔：手機隨手拍、人像、風景、商品照。50 張裡有 37 張的 EXIF 還留著相機型號，來自 iPhone、Pixel 和 Galaxy 手機。另外 12 張是我刻意做出來、模仿「被轉傳過」的副本：4 張手機風格的螢幕截圖、4 張像聊天軟體那樣重度壓縮過的檔案、4 張輕度修圖的版本。

AI 圖片 50 張，來自六個生成器系列：OpenAI GPT Image、Google Nano Banana、Black Forest Labs FLUX.2、Midjourney（v6 到 v8.1）、ByteDance（字節跳動）Seedream 和 Stable Diffusion XL。其中 42 張出自 2025 年以後推出的生成器。40 個 AI 原始檔裡，27 張是我為這次測試自己生成的（API 費用大約 1.42 美元），13 張取自 Hugging Face 上開放授權的資料集。最後 10 張是 AI 圖的手機風格截圖，存成 PNG，完全沒有中繼資料。

怎麼計分

每個引擎拿到的檔案完全一樣。每家的回答，我都用一條固定規則轉成「AI」或「真實」：Sightengine 分數 0.5 以上算 AI；Hive 用官方建議的門檻 0.9；AI or Not 和 WasItAI 直接採用廠商自己的判定。回答「不確定」一律算錯。

看數字之前先講一個前提：只有兩個引擎看過全部 100 張。我只用免費試用額度，所以 AI or Not 只跑了分層抽樣的 20 張，WasItAI 只跑了 9 張。

結果

引擎	測試張數	準確率	真實照片被判成 AI	抓到的 AI 圖
Sightengine	100	99%	0/50	49/50
Hive	100	95%	0/50	45/50
AI or Not	20（抽樣）	85%	1/10	8/10
WasItAI	9（抽樣）	78%	2/5	4/4

下面兩列請不要當成排名。20 張和 9 張實在太少。AI or Not 那份樣本也比整體難，截圖和重新壓縮的比例比較高，它唯一一次誤判是一張輕度修圖的照片。四個引擎都看過的那 9 張裡，Sightengine 和 Hive 都是 9 張全對，AI or Not 對 8 張，WasItAI 對 7 張。這只能當參考，不是定論。

按生成器來看，FLUX 最難抓：Sightengine 9 張抓出 8 張，Hive 9 張抓出 6 張。Midjourney 的圖，兩個引擎都全部抓到。

為什麼我比較在意誤判，而不是準確率

「準確率」把兩種性質很不一樣的錯混在一起。我的測試程式裡有一個假引擎，不管什麼圖都回答「AI」，只用來確認整套流程有接通。在這組圖上，它 50 張 AI 圖全部抓到，準確率還是有 50%，因為它也把 50 張真實照片全部判成 AI。

這兩種錯的代價不一樣。漏掉一張 AI 圖，可惜而已。把真照片說成 AI，倒楣的是一個真實的人：網拍真的收到瑕疵品、申請退款的買家；一筆一筆自己畫出來的繪師；在交友軟體上只是很會拍照的人。如果你檢查的圖大多數其實是真的，就算誤判率很低，冤枉的人也可能比真正抓到的還多。

還有兩件我沒料到的事：

門檻是一種選擇。 用 Hive 建議的 0.9，它沒有誤判任何一張真實照片，抓到 90% 的 AI 圖。如果改成 0.5，可以抓到 96%，但 50 張真實照片會誤判 1 張。每個用這些引擎做出來的產品，都在這條曲線上選了一個點，通常不會告訴你選在哪裡。
零誤判不等於真的零。 Sightengine 和 Hive 都是 50 張真照 0 誤判。但只有 50 張，95% 信賴區間仍然從 0% 到大約 7%。這是小規模測試裡的好兆頭，不是保證。

內容憑證：大多數人跳過的免費檢查

現在有些 AI 工具會用 C2PA 標準替產出的檔案加上數位簽章，一般叫做「內容憑證」（Content Credentials）。紀錄裡寫著是哪個工具做的，而且用密碼學跟檔案綁在一起。讀它不用錢，在瀏覽器裡就能讀。

在我的測試組裡，50 張 AI 圖有 24 張帶著內容憑證，真實照片沒有任何一張帶著「AI 生成」的聲明。我看到的簽署者有 OpenAI、Google、Black Forest Labs 和 BytePlus（字節跳動）。這 24 張全部是原始檔，也就是 40 個 AI 原始檔裡有 24 個帶憑證。

有個重點我後來才認真看待：簽章有效，只告訴你是誰簽的，不代表你該信任它。24 張裡，只有 14 張的簽署者在 C2PA 官方信任清單上（OpenAI 和 Google）。另外 10 張用的數位憑證不在清單上，而自己做的憑證什麼都能簽。所以我的工具把這 10 張，還有像 Midjourney 那種單純寫在中繼資料裡的標籤，都只當成「聲明」，照樣跑偵測模型。

要注意的地方：

Midjourney 的檔案從來沒有（7 個原始檔 0 個），Stable Diffusion XL 也沒有（5 個 0 個）。
截圖會把它弄丟。 10 張截圖沒有一張帶憑證。截圖是一個全新的檔案，簽章不會跟著過去。
我有些檔案可能早就弄丟了。 有幾張來自公開資料集，重新存過的副本可能把紀錄拿掉，所以數量少，不能證明某個生成器從來不簽。
相機也會簽。 兩張真實照片，分別來自 Google Pixel 9 和 Pixel 10，帶著相機的內容憑證；而且內容憑證從來沒有把任何一張真實照片標成 AI。

所以，有內容憑證的時候，它幾乎就是證據；沒有的時候，什麼也說明不了。如果你在 LINE 群組裡看到的瘋傳圖片是一張截圖，就別期待找得到。

上面那張表為了方便比較，用的是一條簡單的切線（Sightengine 0.5 以上算 AI）。工具本身比較保守：分數高於 0.8 判 AI，低於 0.2 判真實，中間就說「無法判斷」。先讀內容憑證，剩下的才送 Sightengine，用這兩條線，結果是 100 張中 98 張正確、1 張「無法判斷」（一張真實照片的截圖，分數 0.21）、1 張判錯（一張 FLUX 截圖，分數 0.18，被判成真實），真實照片依然是 0/50 誤判。有 16 張靠可信的內容憑證就有答案，不用呼叫模型（14 張 AI 圖，加上 2 張簽成真實拍攝的 Pixel 照片），其餘 84 張才交給模型。

截圖也會讓模型掉分。拿 10 組「AI 原圖＋它的截圖」來比，Sightengine 從抓到 10 張掉到 9 張，Hive 從 10 張掉到 8 張。

第三種答案：「經 AI 編輯」

整理測試組的時候，有一張 Pixel 照片的中繼資料寫著「Edited with Google AI」。它不算完全真實，也不算完全生成，所以我把它從真實照片裡換掉了。兩個主要引擎都判它是真的，這也說得過去，但這說明了一件事：對很多手機照片來說，「是真的還是 AI」漸漸變成一個問錯的問題。好用的檢查工具需要第三種答案。

這個測試沒辦法告訴你的事

規模小：100 張圖，一天。
四個引擎裡有兩個只看了抽樣。
缺了交友軟體自拍、重度濾鏡人像、手繪作品，也沒有合成圖或局部 AI 修改的圖。插畫只有幾張。
有一張真實照片比其他圖晚了幾個小時才送進 Sightengine 和 Hive，因為要等它們的每日免費額度重置。
廠商會更新模型，這只是某一天的快照。
我不中立。我做的工具現在用 Sightengine，Hive 當備援。沒有任何廠商付錢或事先看過這份測試，四家用的都是免費試用額度。

如果只想查一張圖

如果你只是想弄清楚某一張圖，我會照這個順序：

先找內容憑證。 Content Authenticity Initiative 的免費 Verify 工具（contentcredentials.org/verify）可以讀。如果它寫出某個 AI 生成器，大致就可以確定了。如果什麼都沒有，就往下走，這很正常。
找最早的版本。 用 Google 智慧鏡頭、TinEye 或 Bing 以圖搜圖。一張圖最早出現在哪裡，常常就能看出是誰做的、為什麼做。
想想畫面合不合理。 制服對不對、路牌對不對、天氣跟那天對不對得上。這一關不會因為生成器變強而失效。
然後再仔細看。 小字、手拿東西的地方、倒影、影子、人群裡重複出現的臉。這些都只是線索：一個怪細節不能證明是 AI，一張完美的圖也不能證明是相機拍的。看圖的細節我另外整理在如何分辨 AI 圖片。
把偵測工具當成另一條線索。 盡量選會給出分數範圍和理由、而且允許說「無法判斷」的工具。能用原始檔就用原始檔，截圖比較難判。

不管查到什麼，都不要只憑偵測工具的一個答案就指控一個真實的人。如果那張圖是要你匯款或加入投資群組，與其自己糾結真假，直接打 165 反詐騙諮詢專線問最快。

我把這套「先讀內容憑證」的流程做成了一個免費小工具：AI 圖片檢測。完整的方法、每個生成器的數字和測試限制，都寫在準確度說明頁（英文）。如果你有它判錯的圖，很歡迎丟給我看看。

本文的英文原版發表在 dev.to：I tested 4 AI image detectors on 100 images

這篇文章有請 AI 助手協助起草和編輯。測試用的圖、實際執行和所有數字都是我自己的。

How to run Secret Santa without collecting anyone's email

Han Lou — Mon, 19 Oct 2026 01:20:38 +0000

Most online Secret Santa tools start by asking the organizer for everyone's email address. That's a fair design if you want automatic reminders. But it's a lot to ask at work, where nobody wants another account, and in families, where some relatives barely check email and an invite can land in spam.

You don't need anyone's email to run a fair draw. Here are three ways to do it, what each is good at, and how to handle the two things that trip groups up: exclusions and remote colleagues.

Option 1: paper slips, everyone in one room

Write every name on a slip, fold them, and let everyone draw at the same time.

It's simple and fun, with one known flaw: people pull their own name a lot. With six or more people, there's a 63% chance that at least one person draws their own slip. On average you'll need almost three full draws before one comes out clean.

The tempting fix is to let that person put the slip back and draw again. Don't. The last person can end up holding only their own name (a 7.6% chance with 10 people, 13.9% with 4), and even when it works, some pairings become more likely than others, up to five times as likely with 6 people. The fair fix is boring: collect every slip and start over.

Slips also make two things awkward. Keeping a couple apart means redrawing every time they get each other. And everyone has to be in the room.

Good for: a family dinner or a classroom, small groups, no special rules.

Option 2: a group chat and one trusted helper

Ask someone who isn't taking part, like a partner or a friend outside the group, to draw names and message each person their match privately. A simple way to draw: list the names in a spreadsheet, sort them into a random order, and have each person give to the next one down, with the last person giving to the first.

This works remotely, and the single circle makes the reveal easy if you open gifts in order. If two partners land next to each other, shuffle again.

The catch is that the helper knows everything. If the organizer does it, the organizer can't really play. And it all depends on one person sending a dozen private messages without a copy-paste slip.

Good for: small groups where someone is happy to sit out.

Option 3: an online draw with one private link per person

The online version of "nobody sees the whole list" is a tool that gives each person their own private link instead of emailing them. You send each link the way your group already talks: a direct message, a text, a QR code at the party, or a printed slip for someone without a phone.

What to look for:

No accounts for anyone, including the organizer.
One link per person, not one shared link. If everyone uses the same link and types their name to see their match, anyone can type someone else's name.
Exclusions that work both ways. Some tools block "Ana gives to Ben" but still let Ben draw Ana.
An organizer view that shows who has opened their link but doesn't show who has whom unless the organizer turns on teacher mode, which everyone's page then shows.
A way to add or remove someone after the draw without starting over.
Automatic deletion after the exchange.

Well-known tools like DrawNames and Elfster handle exclusions and wishlists, and they ask for an email address or an account, which is what you trade for reminders. Full disclosure: I built one that follows this checklist, Secret Santa Generator (https://secretsantagenerator.xyz/secret-santa-generator-with-exclusions/), but any tool that ticks these boxes will do.

Good for: offices, remote teams, families spread across several households.

Setting exclusions so the draw stays fair

Exclusions are where well-meaning organizers break their own draw.

Couples and housemates: block both directions. If Maya and Leo live together, neither should draw the other.
Last year's pairs: block one direction. The point is that Alex doesn't buy for Sam again. Sam can still draw Alex.
Keep the list short. Every rule removes options for everyone. With three couples (six people), only 80 of the 720 possible draws keep every couple apart and nobody drawing themselves. Add last year's pairs and 17 to 20 are left.
Know when it's impossible. Three people where two are a couple can't be drawn at all, because both partners would have to give to the third person. A household can be at most half the group: 6 people out of 12 is fine, 7 isn't.
Watch small groups. With two couples there are only 4 valid draws, so everyone can guess their Santa is in the other couple. Add last year's pairs and exactly one draw is left, which means everyone can work out who has whom.
Redraw, don't swap. If a draw breaks a rule, throw it away and draw again. Swapping two people to patch it makes some pairings more likely than others.

For a practical walkthrough, see the Secret Santa generator with exclusions.

Remote and international teams

The draw is the same. Everything after it changes.

Addresses: don't collect them in a shared spreadsheet. Let each person share their address only with whoever drew them, or ask for something digital instead.
Timing: draw names about three weeks before the exchange so people have time to ship; two weeks is enough if everyone is in one office. Set an order-by date at least a week before the call.
Borders: buying from a store in the recipient's own country avoids international shipping and most customs surprises. Gift cards are often tied to one country and currency, so check before you buy. When the gift ships straight from a store, the card is the only personal part, so I keep 60 short Secret Santa messages for coworkers, friends and family.
Budget: decide up front whether shipping counts toward the cap.
Names: colleagues in Australia or Canada may call it Kris Kringle, and in Ireland Kris Kindle. It's the same draw, so use the name the group knows; my Kris Kringle generator page is the same tool under that name, with the budget starting in AUD, NZD or EUR from the browser's region.
Time zones: send a calendar invite rather than typing the time into a message. In 2026, Europe turns its clocks back on October 25 and the US on November 1, so for that week London is 4 hours ahead of New York instead of 5. If your team spans the US, Europe, India and Australia, no single hour works for everyone: run two short calls, or let people unwrap on their own time and share a photo.

A quick checklist

Make it opt-in, and set one budget cap that everyone can afford.
Pick a draw method: slips, a helper, or private links.
Add only the exclusions you really need.
Share each match privately.
Write down the exchange date, the order-by date and where the reveal happens.

If teenagers are in a family draw, the cap matters more than usual. My list of Secret Santa gifts for teens has 24 ideas sorted by budget, from under $10 to under $50, mostly things they'd use every day.

For the announcement itself, I keep copy-and-paste Secret Santa invitation templates for a work email, a group chat and a family, with the budget, the dates and an easy opt-out line already in.

That's all it takes. No email list required.

I used an AI assistant to help draft and edit this post. The test set, the runs and the numbers are my own.

I tested 4 AI image detectors on 100 images. Here's what I'd tell a friend.

Han Lou — Thu, 15 Oct 2026 03:30:59 +0000

I've been building a small AI image checker, and at some point I had to pick the detection engine behind it. Every vendor publishes an accuracy figure. None of them says how it was measured, so I couldn't compare them. I ran my own test instead. It's small and it has gaps, which I list below, but it changed how I think about these tools.

The test set

100 images, all run on October 13, 2026.

50 real photos, mostly from Wikimedia Commons under CC0 or Creative Commons licenses. 38 are original files: phone shots, portraits, landscapes, product photos. 37 of the 50 still had a camera model in their EXIF, from iPhone, Pixel and Galaxy phones. The other 12 are copies I made to look like photos that have travelled: 4 phone-style screenshots, 4 heavily recompressed copies of the kind chat apps produce, and 4 lightly retouched versions.

50 AI images from six generator families: OpenAI GPT Image, Google Nano Banana, Black Forest Labs FLUX.2, Midjourney (v6 to v8.1), ByteDance Seedream and Stable Diffusion XL. 42 come from generators released in 2025 or later. Of the 40 original AI files, I generated 27 for this test (about $1.42 in API calls) and took 13 from openly licensed datasets on Hugging Face. The last 10 are phone-style screenshots of AI images, saved as PNG with no metadata at all.

How I scored it

Every engine got the same files. I turned each answer into "AI" or "real" with one fixed rule per engine: a Sightengine score of 0.5 or higher, Hive's own recommended threshold of 0.9, and the vendor's own verdict for AI or Not and WasItAI. An "unknown" answer counted as wrong.

One caveat before the numbers: only two engines saw all 100 images. I stayed within free trial allowances, so AI or Not ran on a stratified sample of 20 and WasItAI on 9.

The results

Engine	Images tested	Accuracy (one 0.5 cutoff)	Real photos flagged as AI	AI images caught
Sightengine	100	99%	0 of 50	49 of 50
Hive	100	95%	0 of 50	45 of 50
AI or Not	20 (sample)	85%	1 of 10	8 of 10
WasItAI	9 (sample)	78%	2 of 5	4 of 4

Please don't read the bottom two rows as a ranking. Twenty and nine images are too few. AI or Not's sample was also harder than the full set, with a bigger share of screenshots and recompressed copies, and its one false alarm was a lightly retouched photo. On the 9 images that all four engines saw, Sightengine and Hive got 9 of 9, AI or Not 8 of 9 and WasItAI 7 of 9. That's a hint, not a verdict.

By generator, FLUX was the hardest: Sightengine caught 8 of 9 FLUX images and Hive 6 of 9. Both engines caught every Midjourney image. Stable Diffusion XL was the thinnest bucket: 5 original files from a public dataset plus 1 screenshot, none with Content Credentials or an AI label in the metadata, so I wrote up separately what this test does and doesn't say about Stable Diffusion images. Eleven of the images, AI and real side by side with what each file carried and what fooled the software, are on AI vs real images.

Why I watch false alarms more than accuracy

"Accuracy" mixes two very different mistakes. My test harness has a fake engine that answers "AI" for every image, used only for wiring things up. On this set it catches all 50 AI images and still scores 50% accuracy, because it also flags all 50 real photos.

The two mistakes don't cost the same. A missed AI image is a shame. A real photo called AI lands on a person: a buyer with a genuine refund claim, an artist who drew every line, someone on a dating app who just takes good photos. If most of the pictures you check turn out to be real, even a small false-alarm rate can produce more wrong accusations than right ones. If you're checking a photo for one of those cases, I wrote separate notes on damage photos in refund claims and on profile and dating photos.

Two more things I didn't expect:

The threshold is a choice. At Hive's recommended cutoff of 0.9, it flagged no real photos and caught 90% of the AI images. At 0.5 it would have caught 96%, but flagged 1 real photo in 50. Every product built on these engines picks a point on that curve, usually without telling you.
Zero isn't zero. Sightengine and Hive both flagged 0 of 50 real photos. With only 50 photos, the 95% confidence interval still runs from 0% to about 7%. That's a good sign from a small test, not a guarantee.

Content Credentials: the free check most people skip

Some AI tools now sign the files they produce, using the C2PA standard, known publicly as Content Credentials. The record names the tool and is cryptographically tied to the file. Reading it costs nothing and can happen in the browser.

In my set, 24 of the 50 AI images carried Content Credentials, and none of the real photos carried an AI claim. The signers I saw were OpenAI, Google, Black Forest Labs and BytePlus (ByteDance). All 24 were original files: 24 of the 40 AI originals carried credentials. One catch I only took seriously later: a valid signature tells you who signed, not whether to trust them. Only 14 of those 24 were signed by companies on the official C2PA trust list (OpenAI and Google). The other 10 were signed with certificates that aren't on it, and a self-made certificate can sign anything, so my checker treats those, and plain metadata labels like Midjourney's, as a claim and still runs the detection model.

The catches:

Midjourney files never had them (0 of 7 originals), and neither did Stable Diffusion XL (0 of 5).
Screenshots lose them. None of the 10 screenshots carried credentials. A screenshot is a brand-new file, so the signature doesn't come along.
Some of my files may have lost them earlier. Several came from public datasets, where a re-saved copy can drop the record, so a low count doesn't prove a generator never signs.
Cameras sign too. Two real photos, from a Google Pixel 9 and a Pixel 10, carried camera credentials, and credentials never marked a real photo as AI.

So credentials are close to proof when they're there, and their absence tells you nothing. If you meet a viral image as a screenshot, expect to find none. I wrote up how I'd check a picture you only saw online, screenshot or not.

The table uses one simple cutoff at 0.5 so the engines can be compared. The tool itself is more cautious: it calls an image AI above 0.8, real below 0.2, and says "can't tell" in between. Reading credentials first and sending only the rest to Sightengine, with those two lines, the result was 98 of 100 correct, 1 "can't tell" (a screenshot of a real photo scoring 0.21), 1 wrong (a FLUX screenshot scoring 0.18, called real), and still 0 of 50 real photos flagged. 16 images were settled by trusted credentials without a model call (14 AI images plus 2 Pixel camera photos signed as real captures); the other 84 went to the model.

Screenshots cost the models a little too. On 10 pairs of an AI original and its screenshot, Sightengine went from 10 caught to 9, and Hive from 10 to 8.

A third answer: "edited with AI"

While I was building the set, one Pixel photo's metadata said "Edited with Google AI". It isn't fully real or fully generated, so I swapped it out of the real photos. Both main engines had called it real, which is fair enough, but it shows that "real or AI" is becoming the wrong question for a lot of phone photos. A useful checker needs a third answer.

What this test can't tell you

It's small: 100 images, one day.
Two of the four engines only saw samples.
It's missing dating-app selfies, heavily filtered portraits, hand-made artwork, and composites or partial AI edits. There are only a few illustrations.
One real photo went through Sightengine and Hive a few hours after the rest, once their daily free quotas had reset.
Vendors update their models, so this is a snapshot.
I'm not neutral. The checker I built now uses Sightengine, with Hive as a backup. No vendor paid for the test or saw it first, and all four ran on free trial credits.

How I'd check a single image

If you just want to know about one picture, this is the order I'd use:

Look for Content Credentials. The Content Authenticity Initiative's free Verify tool (contentcredentials.org/verify) reads them. If it names an AI generator, you're mostly done. If it shows nothing, move on; that's normal.
Find the earliest copy. Run a reverse image search with Google Lens, TinEye or Bing. The first place an image appeared often tells you who made it and why.
Ask whether the scene makes sense. The right uniforms, the right road signs, the right weather for the date. This check doesn't get weaker as generators improve.
Then look closely. Small text, hands holding things, reflections, shadows, repeated faces in a crowd. Treat these as clues: one odd detail doesn't prove AI, and a flawless picture doesn't prove a camera. The longer version of this step is a visual checklist for telling if an image is AI-generated.
Use a detector as one more clue. Prefer one that shows a range and its reasons, and that is allowed to say "can't tell". Use the original file if you can, because screenshots are harder.

Whatever you find, don't use a detector's answer alone to accuse a real person.

I put this credentials-first flow into a small free checker, and the full method, every number by generator and the limits are on its accuracy page: https://aiimagedetection.app/ai-image-detector-accuracy/. If you have an image it gets wrong, I'd like to see it.

I used an AI assistant to help draft and edit this post. The test set, the runs and the numbers are my own.