DEV Community: Khadim Hussain

Is SynthID reliable? I tested Google's AI image check on screenshots, stripped files and real photos

Khadim Hussain — Mon, 12 Oct 2026 18:40:48 +0000

Originally published on khadim.tech.

In short: Mostly, for what it is built to do. In my tests on October 12, 2026, Google's SynthID check in the Gemini app flagged a Gemini-generated image after its metadata was removed and again after a screenshot, and found nothing on a real photo. It missed the same AI image once it was pasted small into a real photo, and the new SynthID Detector website failed on every upload that day. A clean result only means no watermark from Google or its partners was found.

Google opened the SynthID Detector to everyone on October 7, 2026. Most of the coverage so far repeats the launch post, so I generated some images and tested the questions people search for. Does it survive a screenshot? Does it need the file's metadata? Will it flag a real photo?

What SynthID Detector is

SynthID is Google DeepMind's invisible watermark for AI-generated images, video and audio. Google's launch post says it has watermarked over 180 billion images and videos and 240,000 years of audio. The Detector reads watermarks from Google and its partners: OpenAI, NVIDIA, Kakao and, soon, Apple. You sign in with a Google, Apple or OpenAI account, then upload a file.

It is not a general AI detector. It answers one question: does this file carry a SynthID watermark? The same check is built into Search, Chrome and the Gemini app, which together handle over a million verification requests a day, according to Google.

How I tested it

I generated two photorealistic images with Gemini 3 Pro Image through the Gemini API (2400 × 1792 px). Google's image generation docs say "All generated images include a SynthID watermark." From one of them I made these test files:

the original file, untouched
the same pixels with the metadata removed (a pixel diff against the original is exactly zero)
a real screenshot: the image shown in Chrome, captured from the screen and saved as a 2048 × 1523 PNG
a real photo from NASA's public image library, taken at Kennedy Space Center in 2023
a second NASA photo with only the sky changed by Gemini
the AI image pasted small into the corner of the real photo

The Detector was the plan, but on October 12 all four of my uploads, between 4:36 and 5:33 pm UTC, ended in "Something went wrong! Please wait and try again later." That included a 166 KB file, so it wasn't file size. So I used the SynthID check built into the Gemini app: a fresh chat for each file, my personal account, and the question "Is this AI-generated?"

I kept to things a normal user would check. The Detector's terms forbid using its results to work out which transformations break the watermark, so I didn't build a grid of crops, filters and compression levels.

The results

Content Credentials (the C2PA standard) are signed metadata that record how a file was made. SynthID is the watermark in the pixels. Here is what Gemini said about each file:

File	Content Credentials in file	Gemini's answer	What it used
Gemini image, untouched	Yes, signed by Google LLC	"created using artificial intelligence by Google LLC, followed by subsequent editing using AI tools"	C2PA metadata
Same pixels, metadata removed	No	"Most or all of this image was edited or generated with Google AI." (2 of 2 runs)	SynthID
Screenshot of it (PNG)	No	"Most or all of this image was edited or generated with Google AI." (2 of 2 runs)	SynthID
Real photo (NASA, 2023)	No	"No reliable signals were detected indicating how the content was created"	Nothing found
Real photo, sky edited in Gemini	Yes, signed by Google	"created as a generative AI creation and subsequently edited using generative AI tools"	C2PA metadata
AI image pasted small into the real photo	No	"No reliable signals were detected"	Missed

Metadata removed, pixels untouched: Gemini still finds Google's watermark.

Does SynthID work on screenshots?

It did in mine. A screenshot changes the image: it's rescaled to fit the screen, re-rendered by the browser and saved in a different format. Gemini flagged it both times. The second answer also said "No reliable C2PA content credentials were detected for this file", so the detection came from the pixels.

A screenshot of the image, saved as PNG: flagged, with no C2PA credentials left in the file.

Not every screenshot test agrees. On October 8, Digital Information World reported that the Detector website flagged their image as a standalone file but not as a screenshot. Mine was cropped to the image and checked in Gemini, and theirs used the Detector, so the two aren't directly comparable. Either way, a screenshot is a weaker test than the original file.

Google's own claim is narrower. Its SynthID page says the image watermark is "designed to stand up to modifications like cropping, adding filters, changing frame rates, or lossy compression." Screenshots aren't on that list.

SynthID vs C2PA: the two labels inside a Gemini image

Every Gemini image I made carried two separate labels, and the check reads both.

	C2PA Content Credentials	SynthID
Where it lives	Signed metadata in the file (about 6 KB here)	In the pixels
What it records	Who made it, which tool, edit history	Only that a Google or partner model made it
Survives a re-save or screenshot	No	Yes, in my tests
Who can read it	Anyone, with open tools	Google's tools only

Removing the metadata, or re-saving the image as a 1080 px JPEG, deleted the C2PA manifest completely. The file then relies on SynthID alone. You can read the credentials yourself with c2pa-python (version 0.38.0 here):

import json
from c2pa import Reader

reader = Reader("gemini-image.jpg")
store = json.loads(reader.json())
manifest = store["manifests"][store["active_manifest"]]
print(manifest["signature_info"]["issuer"])
for assertion in manifest["assertions"]:
    if assertion["label"].startswith("c2pa.actions"):
        for action in assertion["data"]["actions"]:
            print(action["action"], action.get("digitalSourceType"))

Google LLC
c2pa.created http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia
c2pa.edited http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia

trainedAlgorithmicMedia is the IPTC term for media created by a generative model. If you'd rather not run code, contentcredentials.org/verify reads the same manifest in the browser.

With the metadata intact, Gemini answers from the C2PA credentials and doesn't mention SynthID.

Is SynthID reliable? What a result tells you

A positive result is strong evidence. Gemini identified the untouched file from its metadata, and the stripped copy and the screenshot from the watermark, while the real photo came back clean. One photo isn't a false-positive rate, though.

A negative result is weak evidence:

It only knows its partners. An image from a generator outside the SynthID scheme will always come back clean.
It missed a composite. The same AI image, pasted small into the corner of a real photo, came back "No reliable signals were detected." Digital Information World saw the same on the Detector website: an AI image covering more than half of a new image came back as unlikely to be from a SynthID partner.
The check has slipped before. In July, Lead Stories found that within one chat, Gemini repeated its verdict on the first file when asked about a second one, for images and videos. Google said on July 16, 2026 that a fix had rolled out globally. I still used a fresh chat for every file.

An AI image pasted small into a real photo (background: NASA, public domain): not detected.

How you ask matters

My first prompt was "Was this image created or edited with Google AI? Please check it for a SynthID watermark." On the metadata-stripped file, Gemini answered that it couldn't check for a watermark at all. Asked "Is this AI-generated?", which is Google's own suggested wording, it found the watermark both times. That was one run against two, but if Gemini says it can't check, ask again with the plain question in a new chat.

How to check an image yourself

In the Gemini app, upload the image in a new chat and ask "Is this AI-generated?" In my tests it answered from C2PA when the file had it, and from SynthID when it didn't.
On the SynthID Detector, sign in at synthid.com and upload the file. It covers Google's partners as well, and shows which parts of the image are most likely watermarked.
For Content Credentials, use contentcredentials.org/verify, or the code above, to see the signed history, if the file still has it.
Read a clean result as "unknown", not "real".

What about AI-generated text?

The image watermark is closed, but SynthID Text is open source. It ships in Hugging Face Transformers 4.46 and later as a watermarking config passed to generate(), with a Bayesian detector. Text detection is probabilistic, and I didn't test it here.

If you build image pipelines

Any step that decodes and re-encodes an image (resizing, thumbnails, format conversion) drops the C2PA manifest, as the 1080 px re-save above showed. If provenance matters to your product, carry the manifest through with the C2PA SDK rather than relying on the watermark alone. SynthID survived the re-save and the screenshot in my tests, but only Google's tools can read it.

FAQ

Does SynthID work on screenshots?

Sometimes. My screenshot of a Gemini image, cropped to the image and saved as a PNG, was flagged by Gemini's check in both runs, from the watermark in the pixels. Digital Information World's screenshot test on the SynthID Detector website failed on October 8, 2026. Use the original file whenever you can get it.

Is SynthID the same as C2PA Content Credentials?

No. C2PA Content Credentials are signed metadata stored in the file, and a normal re-save or upload removes them. SynthID is a watermark in the pixels themselves. Gemini images carry both, and Gemini's check reads whichever one it finds.

Can SynthID detect Midjourney or Stable Diffusion images?

Only if the generator embeds SynthID. Google lists itself, OpenAI, NVIDIA and Kakao as the companies whose watermarks the Detector reads, with Apple to follow. Anything else comes back as no watermark found, which says nothing either way.

Does SynthID have false positives?

A real photo came back clean in my test, but that is one photo, not a false-positive rate. Treat a positive result as strong evidence the image passed through Google's or a partner's AI tools, and a negative one as unknown.

Is SynthID open source?

The text watermark is: SynthID Text ships in Hugging Face Transformers 4.46 and later, with a reference implementation on GitHub. Image, video and audio detection is only available through Google's tools, such as the SynthID Detector and the Gemini app.

Fine-tuning gpt-oss-20b with Unsloth and running it in Ollama

Khadim Hussain — Sun, 11 Oct 2026 21:08:28 +0000

Originally published on khadim.tech.

In short: gpt-oss-20b fine-tunes with QLoRA on a single consumer GPU (Unsloth puts it at about 14 GB of VRAM). Shipping it is where the chat template matters: gpt-oss uses OpenAI's Harmony format, so the GGUF needs a Harmony template with <|return|> as a stop token, or Ollama gives gibberish or never stops.

I fine-tuned gpt-oss-20b for STEM reasoning and published it three ways: a LoRA adapter, a merged 16-bit model and GGUF files. Below are the settings and versions behind it, the export code, and the Modelfile that makes the GGUF behave in Ollama.

QLoRA settings for gpt-oss-20b

The published run, as recorded on the model card:

Setting	Value
Base model	openai/gpt-oss-20b (21B parameters, mixture of experts)
Method	QLoRA, 4-bit base weights
LoRA rank / alpha	32 / 64
Learning rate	1e-4
Batch size	1, with gradient accumulation 16
Optimizer	AdamW 8-bit
Epochs	1
Data	4,260 training and 474 evaluation examples of STEM Q&A with chain-of-thought
Result	train loss 1.087, eval loss 0.837
Framework	Unsloth + TRL

The full training config, with target modules and scheduler, is in configs/gpt_oss_20b.yaml in kllm, the small toolkit I use for these runs.

How much VRAM does fine-tuning gpt-oss-20b need?

QLoRA keeps the base weights in 4-bit and trains only a small adapter, so a 20B model fits on one consumer card. Unsloth's gpt-oss guide lists about 14 GB of VRAM for gpt-oss-20b with QLoRA and recommends at least 16 GB for stable runs. LoRA on unquantized BF16 weights needs about 44 GB, which is why QLoRA is the practical choice on one card.

The environment kllm is verified on:

Component	Version
GPU	RTX 5090 (32 GB)
OS	Ubuntu 24.04
CUDA	12.8 toolkit (13.0 driver)
Python	3.12.3
PyTorch	2.9.1+cu128
Unsloth	2026.1.4
transformers	4.57.3
vLLM	0.15.0

Pin transformers. vLLM 0.15.0 needs exactly transformers 4.57.3, and other versions fail with import errors. Install in this order, and force the pin last.

uv pip install -U vllm --torch-backend=cu128   # cu121 for RTX 40 series
uv pip install unsloth unsloth_zoo bitsandbytes
uv pip install --force-reinstall "transformers==4.57.3"

Export to LoRA, merged and GGUF with Unsloth

After training, the same model object gives all three formats. These are the calls kllm makes:

# 1. LoRA adapter (~61 MB): for people who already run the base model
model.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)

# 2. Merged 16-bit model (~41 GB): for Transformers or vLLM, no adapter handling
model.save_pretrained_merged(f"{output_dir}-merged", tokenizer, save_method="merged_16bit")

# 3. GGUF for llama.cpp, Ollama and LM Studio, one call per quantization
model.save_pretrained_gguf("gguf/gpt-oss-20b-stem", tokenizer, quantization_method="q4_k_m")

quantization_method takes f16, q8_0, q4_k_m and the other llama.cpp types listed in Unsloth's GGUF docs. The GGUF files for this model came out at:

File	Size
f16	41.9 GB
Q8_0	22.3 GB
Q4_K_M	15.8 GB

If the GGUF step runs out of memory, Unsloth's docs suggest lowering maximum_memory_usage from its 0.75 default to around 0.5.

The Harmony chat template is what breaks

gpt-oss doesn't use ChatML. It was trained on OpenAI's Harmony format, where every message is wrapped in special tokens and the assistant writes on separate channels:

analysis for its chain of thought, which end users shouldn't see
final for the answer
commentary for tool calls

A final answer ends with <|return|>, which OpenAI's docs call "a valid stop token indicating that you should stop inference." Messages kept in the conversation history end with <|end|> instead, while supervised training targets should end with <|return|>. OpenAI's model card is blunt about it: the models "should only be used with the harmony format as it will not work correctly otherwise."

So the template you ship has to match the one used in training. Unsloth's docs name a mismatch as the most common reason a model gives gibberish or endless output on other platforms. And <|return|> has to be a stop token, or generation runs past the answer.

This exact question, how to template a fine-tuned gpt-oss for Ollama, is still open on OpenAI's forum.

A working Ollama Modelfile for fine-tuned gpt-oss

This is the Modelfile that ships with the GGUF repo:

FROM ./gpt-oss-20b-finetuned-q8_0.gguf

TEMPLATE """<|start|>system<|message|>You are a helpful assistant trained by OpenAI.

Reasoning: medium

# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>user<|message|>{{ .Prompt }}<|end|><|start|>assistant"""

PARAMETER stop "<|return|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 4096
PARAMETER num_predict 2048

Setting reasoning effort

Reasoning: medium in the system message sets how much the model thinks in its analysis channel before it answers. Harmony accepts low, medium and high, and medium is the default. To change it, edit that line in the TEMPLATE and run ollama create again. Ollama shows the analysis channel as "Thinking...", so low is the setting for short, fast answers.

ollama run hf.co or a Modelfile?

With the Modelfile, the template is the one you wrote:

ollama create gpt-oss-20b-stem -f Modelfile
ollama run gpt-oss-20b-stem

Ollama can also pull straight from Hugging Face:

ollama run hf.co/khadim-hussain/gpt-oss-20b-stem-reasoning-GGUF:Q4_K_M

That route doesn't read the Modelfile in the repo. Per Hugging Face's Ollama docs, it uses the chat template stored in the GGUF's tokenizer.chat_template metadata, or a file named template in the repo. The quantization tag is case-insensitive, and without a tag Ollama picks Q4_K_M. If you publish a Harmony model, put a template file in the repo or point people to the Modelfile.

Which format people actually download

All-time Hugging Face downloads as of October 2026:

Format	gpt-oss-20b	Qwen3-14B
GGUF	513	274
Merged 16-bit	70	42
LoRA adapter	46	51

Most people want a file they can run on their own machine. If you only publish one format, make it a GGUF with a working template. The adapter is worth adding for anyone who wants to keep training.

Troubleshooting

The failures documented in kllm's troubleshooting guide:

Symptom	Fix
Training process dies with no error	Out of memory. Check `dmesg \
{% raw %}`UnboundLocalError` mentioning dropout	Set `lora_dropout: 0.0`
`ImportError: cannot import name ... from 'transformers'`	`uv pip install --force-reinstall "transformers==4.57.3"`
`Python.h: No such file or directory`	`sudo apt-get install python3.12-dev`

The full project, with the Qwen3-14B run and the download numbers, is in the STEM reasoning LLMs case study.

Frequently asked questions

How much VRAM do you need to fine-tune gpt-oss-20b?

With QLoRA, Unsloth's gpt-oss guide puts gpt-oss-20b at about 14 GB of VRAM and recommends at least 16 GB for stable runs. LoRA on BF16 weights needs about 44 GB. kllm, the toolkit I trained with, is verified on an RTX 5090 with 32 GB.

Can I run a fine-tuned gpt-oss-20b in Ollama?

Yes. Export it to GGUF, then either run ollama create with a Modelfile that carries the Harmony template, or pull it with ollama run hf.co//:Q4_K_M. The second route uses the chat template embedded in the GGUF file, not the Modelfile in the repo.

Why does my fine-tuned gpt-oss model output gibberish or never stop?

Usually the chat template. The template at inference has to match the one used in training, and Harmony ends a final answer with <|return|>. If that token isn't a stop token, generation runs past the answer.

Which GGUF quantization should I use for gpt-oss-20b?

Q4_K_M (15.8 GB for this model) is the smallest file and the one Ollama picks by default. Use Q8_0 (22.3 GB) if you have the memory and want output closer to the 16-bit model.

How do I set reasoning effort for gpt-oss in Ollama?

Harmony reads it from the system message as Reasoning: low, medium or high, and medium is the default. In a Modelfile, change that line inside the TEMPLATE and run ollama create again.