DEV Community: CoreMesh

claimcheck: Gemma 4 fact-checks my AI blog drafts against their own sources

CoreMesh — Sat, 17 Oct 2026 07:49:37 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

I run CoreMesh, a small tech blog where an AI pipeline writes the first draft of every article and I approve or reject it. That is how a sentence like this got into a draft about upgrading to Python 3.15:

The stdlib now returns frozendict from EnumType.__members__ and errno.errorcode instead of MappingProxyType or dict.

It reads like a fact, but it isn't one. PEP 814 lists those two only as possible candidates for frozendict in the stdlib. A reader who trusted that line would write code that breaks.

So I built claimcheck: a small CLI that checks an AI-written draft against the sources the draft itself links to, using an open-weight model (Gemma 4), before anything goes live.

Who would pay for it: anyone who publishes AI-assisted writing under their own name and can't afford to be wrong in public:

solo bloggers and newsletter writers;
small content agencies;
docs and DevRel teams;
people selling technical courses.

The tool itself is MIT. What people would pay for is the service around it: a pre-publish check per article, a CI gate on a docs repo, or a weekly re-check of old posts as the docs they cite change.

Demo

A real run on that Python 3.15 draft, with the bad sentence put back in:

$ python3 claimcheck.py examples/python315-draft.md --exclude coremesh.online
28 claims, 4 sources
  [1/28] supported    Python 3.15 uses UTF-8 as the default text encoding for files, stdio and pipes.
  ...
  [18/28] not_found    In Python 3.15, `EnumType.__members__` and `errno.errorcode` return `frozendict`.

The report puts problems first and shows the draft's words next to the source's words:

## 5. [CHECK] In Python 3.15, `EnumType.__members__` and `errno.errorcode` return `frozendict`.
> Draft: "The stdlib now returns `frozendict` from `EnumType.__members__` and `errno.errorcode` instead of `MappingProxyType` or `dict`"

## 7. [OK] Python 3.15 uses UTF-8 as the default text encoding for files, stdio and pipes.
> Draft: "Python now uses UTF-8 as the default text encoding for files, stdio and pipes"
> Source (https://peps.python.org/pep-0686/): "With this change, Python consistently uses UTF-8 for default encoding of files, stdio, and pipes."

The exit code is 1 when anything needs a human, so it can block a publish step or a CI job.

Code

indra112f / claimcheck

Fact-check AI-written drafts against their own sources with an open-weight model (Gemma). Evidence quotes are verified word for word.

claimcheck

Fact-check an AI-written draft against its own sources with an open-weight model, before you publish it.

$ claimcheck examples/python315-draft.md --exclude coremesh.online
28 claims, 4 sources
  [1/28] supported    Python 3.15 uses UTF-8 as the default text encoding for files, stdio and pipes
  [18/28] not_found    In Python 3.15, `EnumType.__members__` and `errno.errorcode` return `frozendict`.

That last line is a real mistake our blog's AI writer made: PEP 814 lists those two as possible candidates, not as shipped changes. claimcheck flagged it.

How it works

Claims. The model lists the checkable claims in the draft: numbers, versions, limits, defaults, dates, "X supports Y". Every claim must quote the draft. A claim whose quote isn't in the draft is dropped.
Retrieval. Every source page linked in the draft (plus any --source) is fetched and split into passages. A small IDF-weighted lexical index picks the closest passages for each claim, with no…

View on GitHub

It's one file, standard-library Python, with offline tests that use a fake model.

How I Built It

The pipeline has four steps. The interesting part is that the model is never trusted on its own word.

Extract claims. Gemma 4 lists the checkable claims in the draft (numbers, versions, limits, defaults, "X supports Y") and has to quote the draft for each one. If the quote isn't actually in the draft, the claim is dropped.
Retrieve. Every linked source page is fetched and cut into overlapping passages. A tiny IDF-weighted lexical index picks the closest six passages per claim. There are no embeddings and no vector DB. Rare tokens like load_module or 3.15 carry the weight, which is what you want for technical claims.
Judge. Gemma sees one claim and those passages only, and must answer supported, contradicted or not_found and copy one evidence sentence word for word.
Verify the verifier. This is plain code, not the model:
- Quote check: the evidence sentence must exist in the fetched page (after normalizing quotes, dashes and whitespace). If it doesn't, the verdict is thrown away.
- Number check: every number in the draft's sentence must appear in the evidence or in the text around it. I added this after a real run marked "you'll need LLVM 21" as supported with a quote that never mentions 21.

Model and infra: gemma-4-31B-it on DigitalOcean serverless inference, through the OpenAI-compatible endpoint, with temperature: 0. A full article took 3 to 5 minutes and about 70k input tokens. At DigitalOcean's listed Gemma 4 serverless rate ($0.18 per 1M input tokens and $0.50 per 1M output tokens, as of October 2026), that is about 1.4 cents per article.

I built it in one sitting with an AI agent (Claude) doing most of the typing while I steered. That fits the project: an AI wrote it, and the tests and the quote check are why I trust it.

Why Does Open Innovation Matter?

The unit economics only work with cheap inference. A check is around 30 model calls. At about 1.4 cents per article, I can run it on every draft and every old post, every week, and still charge a few dollars a month. That's a business.
The judge shouldn't be a black box I rent. A fact-checker whose model changes behind an API without notice is a liability. With open weights I can always run the exact same model myself, so the verdicts can be reproduced and audited.
Client drafts are private until they're published. The tool talks to any OpenAI-compatible endpoint, so the same CLI can point at Gemma running locally with Ollama and the draft never leaves the machine. For an agency checking a client's unpublished piece, that's a hard requirement. I measured the numbers here on DigitalOcean only.
No lock-in. Swapping Gemma for another open model is one flag. If a better open checker ships next month, I switch the same day.

My First User

My first user is my own blog, so here's what it found, good and bad.

The planted mistake: flagged, as shown above.
A false alarm, and why the report format matters. In one run it marked "all four models have a 1M-token context window" as contradicted, quoting Anthropic's pricing page: "Claude 4.6 and later models (except Claude Haiku 5.5) … include the full 1M token context window at standard pricing." I checked the models overview: all four list 1M tokens. The draft was right; that sentence is about price, not size. Because the report shows the draft's words next to the source's words, it took me about 30 seconds to see it. A later run left the same claim as "not found", so the model isn't fully deterministic even at temperature 0, which is one more reason a human makes the final call.
My best-ranking article ("Claude Code usage limit reached", page 1 on Google) got 0 of 12 claims backed by its sources. It links only to documentation home pages. Re-run against the right support and docs pages, 3 of 10 were backed: the 5-hour and weekly limits, the limits being shared with chat, and /usage. Most of the rest are my own measurements (build and review times) and a description of my own product, which no doc can confirm, and flagging them is correct. The fix there is better links, not a better model.

Nobody has paid yet. The next step is to offer a free check of one article to three bloggers who publish AI-assisted posts, and ask whether they'd pay for it weekly.

Prize Categories

Best Use of Gemma: Gemma 4 is both the claim extractor and the judge.
Best Use of DigitalOcean: it runs on DigitalOcean serverless inference.

This post and claimcheck were written with an AI agent (Claude) working under my direction. Every number above comes from real runs, and the reports are in the repo.

Upgrade to Python 3.15: What Breaks and How to Fix It

CoreMesh — Sat, 17 Oct 2026 06:35:46 +0000

Originally published on CoreMesh.

Upgrading to Python 3.15 is mostly painless, with one exception. Python now uses UTF-8 as the default text encoding for files, stdio and pipes (PEP 686). If you run on Windows, or you read data written in a legacy encoding, any open() call without an explicit encoding= can start raising errors or producing garbled text. Test for that first, then upgrade.

The short version: on your current Python, run your test suite with -X warn_default_encoding to find every implicit-encoding call and fix them, check that your dependencies support 3.15, create a fresh virtual environment on 3.15, and run the suite again. Below are the details for each step, the removed APIs that now raise instead of warn, and the new features (frozendict, lazy imports, unpacking in comprehensions) worth using once you're there.

Should you upgrade to Python 3.15 right now

Python 3.15.0 shipped on October 9, 2026 as a stable release. For a new project, I'd start on 3.15 without hesitation. For an existing codebase, the answer depends on two things: whether your code relies on the platform default encoding, and whether your dependencies have caught up.

Two packaging changes are easy to miss. The official Windows 64-bit binaries now use the tail-calling interpreter, and the official macOS binaries now install free-threading support by default. Both are worth knowing if you file bug reports or compare performance.

One platform warning from the release notes: on macOS 27.0, IDLE and other tkinter applications may hang when a menu command opens a dialog. This is an operating system behavior change that affects all current Tk versions, not something specific to 3.15. If you depend on Tk apps on a Mac, hold off on macOS 27.0 or test your workflow first.

The UTF-8 default is the change most likely to bite

Before 3.15, the default text encoding depended on the locale. Most Unix systems already use a UTF-8 locale, and Python already switched on UTF-8 mode when the locale was C or POSIX, so Linux and macOS users rarely noticed. Windows was different: the default was the legacy code page, so a bare open("report.txt") read the system's ANSI code page (cp1252 on most English-language Windows installs).

In 3.15, UTF-8 mode is on everywhere. The same bare open() now reads UTF-8. That's the right default, and it matches Node.js, Go, Rust and Java, but if the file was written by an older tool in the legacy code page, you get one of three outcomes: a UnicodeDecodeError, mojibake, or silent data corruption on round trips.

Here's what the mojibake case looks like. The bytes are the same; only the decoder's assumption changes:

text = "café costs 5€"
legacy_bytes = text.encode("cp1252")   # written by an old Windows tool

print("decoded as utf-8: ", legacy_bytes.decode("utf-8", errors="replace"))
print("decoded as cp1252:", legacy_bytes.decode("cp1252"))

The fix is to stop relying on the default at all. Three patterns cover almost every case:

# 1. You know the file is UTF-8 (JSON, TOML, Markdown, source code). Say so.
with open("config.json", encoding="utf-8") as f:
    config = f.read()

# 2. The file was written by a local tool in the system's locale encoding.
with open("legacy_export.csv", encoding="locale") as f:
    rows = f.read()

# 3. You were calling locale.getpreferredencoding() to pick an encoding.
import locale
enc = locale.getencoding()   # locale encoding, ignores UTF-8 mode

The encoding="locale" option is honored even in UTF-8 mode (that was fixed in 3.11 specifically so this upgrade could happen). And locale.getencoding() exists since 3.11 for the same reason: locale.getpreferredencoding(False) returns UTF-8 under UTF-8 mode, so it can no longer tell you what the system locale is.

The same rule applies to subprocesses. If you call subprocess.run(..., text=True) and parse the output of a Windows command-line tool, the pipe is now decoded as UTF-8. Pass encoding="utf-8" or encoding="locale" explicitly. PEP 686 considered using PYTHONIOENCODING as the pipe default and rejected it, so there's no shortcut. If you already wrap commands with timeouts, see the subprocess timeout examples for the surrounding pattern.

If you need time, you can turn the old behavior back on with PYTHONUTF8=0 in the environment or -X utf8=0 on the command line. Treat that as a migration aid, not a fix.

How to test for encoding problems before you upgrade

PEP 686 spells out a procedure, and it works on the Python you're running today. The idea is to make every implicit-encoding call visible, fix it, then run under UTF-8 mode to confirm nothing changed.

# Step 1: find every place the default encoding is used.
# EncodingWarning is raised by open() and locale.getpreferredencoding().
python -X warn_default_encoding -W error::EncodingWarning -m pytest

# Step 2: fix each call site with encoding="utf-8" or encoding="locale".

# Step 3: run the suite with UTF-8 mode forced on, still on your current Python.
PYTHONUTF8=1 python -m pytest

Turning the warning into an error is deliberate. A warning scrolls past; an error gives you a traceback at the exact open() call. Run it once with -W error to get the list, then fix them in bulk.

Step 3 is the real preview. If the suite passes with PYTHONUTF8=1 on 3.14, the encoding change in 3.15 is a non-event for you. If it fails, the failures point at files or pipes that carry legacy-encoded data, and those need encoding="locale" or a conversion step.

On Windows, run this on a real Windows machine or CI runner. Testing only on Linux proves nothing, because Linux was already UTF-8.

Removed APIs that now raise errors

Everything in this table was deprecated earlier and is gone in 3.15. The full list is in the What's new in Python 3.15 page.

Removed or changed	What to use instead
AST node constructors raise `TypeError` for missing required args or unknown keyword args	Pass every required field; drop keywords that aren't fields (was a `DeprecationWarning` since 3.13)
`datetime.strptime()` with `%d` but no year directive raises `ValueError`	Add a year directive to the format string
`glob.glob0()` and `glob.glob1()`	`glob.glob()` with the `root_dir` argument
`http.server.CGIHTTPRequestHandler` and the `--cgi` flag	Removed; What's new lists no direct replacement
`Loader.load_module()` is no longer called by the import system	Implement `exec_module()` on custom loaders
`importlib.resources.files(package=...)` keyword	Pass the package positionally: `files("mypkg")`
`pathlib.PurePath.is_reserved()`	`os.path.isreserved()` on Windows
`ctypes.SetPointerType()`	Removed, undocumented
`collections.abc.ByteString` dropped from `__all__`	Still importable until its removal in 3.17; stop using it now

The strptime one catches people who parse log lines like "%m/%d %H:%M". That worked with a warning on 3.13 and 3.14. On 3.15 it's a ValueError.

The custom-loader change is easy to miss because it lives in code you rarely touch: the import system no longer calls load_module(), so any custom loader that only implements the old method needs an exec_module() before you upgrade.

New syntax and built-ins you can start using

Once the suite is green, these four features are the ones most likely to change how you write day-to-day code.

frozendict is a new built-in immutable mapping (PEP 814). It's not a dict subclass; it inherits from object and implements the Mapping protocol. It preserves insertion order, pickles, and is hashable when its values are hashable, which makes it usable as a dict key, a set member, a safe function default, and an argument to functools.lru_cache()-decorated functions.

DEFAULTS = frozendict(retries=3, timeout=30)

def fetch(url, options=DEFAULTS):      # safe default, no mutable-default bug
    opts = {**DEFAULTS, **options}       # merge into a new dict
    ...

frozendict({"a": 1})                     # from a dict
frozendict([("a", 1), ("b", 2)])         # from key/value pairs
frozendict({"a": 1}, b=2)                # both

Construction from a dict is a shallow O(n) copy. frozendict.copy() returns the same object in CPython, and mutable values stay mutable, so use copy.deepcopy() if you need isolation. eval() and exec() accept a frozendict for globals.

sentinel is also a new built-in type (PEP 661), listed in the release highlights. The release notes don't spell out the API in the summary, so check the What's new page before using it.

Explicit lazy imports add a lazy soft keyword (PEP 810). The module is loaded on first use instead of at the import statement, which cuts startup time for CLIs and tools that import heavy libraries they only sometimes need.

lazy import json
lazy from decimal import Decimal

def main(args):
    if args.pretty:
        print(json.dumps(data, indent=2))   # json is loaded here, not above

Two things to know. First, lazy from module import Class still executes the whole module on first access; there's no way to run part of a file. Second, for code that must also run on older versions, use the __lazy_modules__ global, which 3.15 honors and older versions ignore:

# Lazy on 3.15+, eager on older Python
__lazy_modules__ = ["json", "decimal"]
import json
from decimal import Decimal

Unpacking in comprehensions (PEP 798) lets you write [*xs for xs in items] where before you needed a double loop or itertools.chain.from_iterable. It works in list, set and dict comprehensions and in generator expressions, with * for iterables and ** for mappings. Unpacking is restricted to the top level of the expression. This example runs on any Python 3 and uses the new syntax when it's available:

import sys

nested = [[1, 2, 3], [], [4, 5]]
try:
    flat = eval("[*xs for xs in nested]", {"nested": nested})
    how = "PEP 798 syntax"
except SyntaxError:
    flat = [x for xs in nested for x in xs]
    how = "double loop (Python %d.%d has no PEP 798)" % sys.version_info[:2]

print(flat, "via", how)

One readability trap: f(*x for x in y) means f((*x for x in y)), a single generator argument, not unpacked arguments. Add the parentheses yourself so the next reader doesn't have to think about it.

What the experimental JIT does and doesn't give you

The JIT is still experimental and is not on in a default build. Per the What's new page, the pyperformance suite reports a 7-8% geometric mean improvement for the JIT over the standard interpreter on x86-64 Linux, and an 11-12% speedup over the tail-calling interpreter on AArch64 macOS. The spread across individual benchmarks goes from roughly a 15% slowdown to over 100% speedup, so your workload may land anywhere in that range.

The practical upshot: don't upgrade for the JIT. If you build CPython yourself with the JIT enabled, you'll need LLVM 21 at build time (end users running the resulting binary don't need LLVM). The new tracing frontend records real execution paths and handles object creation, overloaded operations and generators partially, which is why more code speeds up than in 3.14. GDB and GNU backtrace() can now unwind through JIT frames on supported Linux ELF platforms, so crash reports from JIT builds are no longer dead ends.

How to install Python 3.15 on Windows, macOS and Linux

Installers come from the python.org download page for the 3.15.0 release.

Windows. The download page points to the Python install manager, and the Windows release manifest is installed with:

py install 3.15

Traditional installers also exist: 64-bit (recommended), 32-bit and ARM64, plus embeddable packages for each if you're bundling Python into an application.

macOS. The macOS installer supports macOS 10.15 and later and, new in 3.15, installs free-threading support by default.

Linux (including Ubuntu). The download page lists source tarballs (gzipped and XZ compressed) but no Linux binaries. Your options are your distribution's packages once they ship 3.15, a tool that fetches prebuilt interpreters, or building from the tarball. The release announcement itself shows uv in action:

uvx --python 3.15 whatsnewt   # a TUI tour of the new features, run on 3.15

Whichever route you take on Linux, don't pip install into the system interpreter. If you hit the externally-managed-environment error, the safe fix for that pip error is a virtual environment, which is where the upgrade should live anyway.

After installing, create a clean environment with the new interpreter rather than pointing an old one at it:

python3.15 -m venv .venv
source .venv/bin/activate
python -m pip install -e . && python -m pytest

If you need a refresher on activation across shells, see how to create and activate a Python virtual environment.

Common mistakes when upgrading

Setting PYTHONUTF8=0 and forgetting it. It silences the problem on one machine and moves the crash to CI or a colleague's laptop. Use it only while you fix call sites.
Assuming encoding="locale" means UTF-8. It means the system locale encoding, which on Windows is the legacy code page. Use it only for files that really were written that way.
Testing only on Linux. Linux was already UTF-8, so a green Linux run tells you nothing about Windows behavior.
Expecting frozendict.copy() to isolate mutable values. It returns the same object. Reach for copy.deepcopy().
Making a module lazy when its import has side effects. Registration, logging setup or monkeypatching that used to happen at startup now happens on first use, or never. Keep those imports eager.
Parsing dates with %d and no year. This is the removal most likely to appear in a production traceback, because it was only a warning before.
Reusing the old virtual environment. Compiled extensions built for 3.14 won't load. Build a new one and reinstall.
Expecting a JIT speedup from the stock installer. The JIT is experimental and the reported gains are for JIT-enabled builds.

If your code is async-heavy, the same rules apply inside coroutines; nothing in the UTF-8 change is specific to asyncio, but file and pipe reads in async code are just as affected.

Upgrade checklist

On your current Python, run the suite with -X warn_default_encoding -W error::EncodingWarning and fix every reported call with an explicit encoding=.
Run the suite again with PYTHONUTF8=1 set. Do this on Windows if you ship to Windows.
Grep for strptime, glob0, glob1, load_module, is_reserved, CGIHTTPRequestHandler and ByteString and replace them using the table above.
Check that each dependency with compiled extensions publishes a 3.15 build.
Install 3.15 and create a fresh virtual environment with python3.15 -m venv.
Install your project into it and run the full suite.
Only then adopt new features: frozendict for constants and defaults, lazy import for slow optional imports, [*xs for xs in items] for flattening.
Leave PYTHONUTF8=0 out of your production configuration.

FAQ

How do I install Python 3.15 on Ubuntu or another Linux distribution?

The python.org download page provides source tarballs but no Linux binaries. Use your distribution's packages when they ship 3.15, a version manager that fetches prebuilt interpreters such as uv, or build from the tarball, and then put your project in a virtual environment.

How do I install Python 3.15 on Windows?

Use the Python install manager and run py install 3.15, or download the 64-bit installer, which is the recommended one. The official 64-bit Windows binaries now use the tail-calling interpreter.

How do I install Python 3.15 on a Mac?

Download the macOS installer, which supports macOS 10.15 and later and installs free-threading support by default. If you use IDLE or other tkinter apps, note that macOS 27.0 has a Tk dialog hang that affects every Python version.

Should I update to Python 3.15?

Yes for new projects. For existing code, first run your tests with -X warn_default_encoding and then with PYTHONUTF8=1 on your current Python; if both pass and your dependencies support 3.15, upgrading is low risk.

Is Python 3.15 faster?

The experimental JIT shows a 7-8% geometric mean improvement on x86-64 Linux and 11-12% on AArch64 macOS in the pyperformance suite, but individual results range from roughly a 15% slowdown to over 100% speedup and the JIT is not enabled in default builds.

The original, with any later corrections, is on CoreMesh.

This article was drafted by an AI system, fact-checked against the official sources linked above, and reviewed by a human before publishing.

Claude Opus 5.5 Fast Mode: Enable It With Request Examples (No Beta Header Needed)

CoreMesh — Sat, 17 Oct 2026 06:18:48 +0000

Fast mode is officially out of research preview for Claude Opus 5.5. You no longer need to join a waitlist or pass the old beta header to get faster token streaming.

Here is the minimal setup, what changed in the October 2026 release notes, pricing details, and the breaking parameter gotchas to watch out for when migrating older Opus requests.

💡 *Originally published on CoreMesh * — Practical AI and coding guides, tested fixes, and runnable prompts.

The Smallest Request That Turns On Fast Mode

The request body is a standard Messages API payload with one extra field, "speed": "fast":

{
  "model": "claude-opus-5-5",
  "max_tokens": 1024,
  "speed": "fast",
  "messages": [
    {
      "role": "user",
      "content": "Summarize this stack trace in two sentences."
    }
  ]
}

Three things to check in that body:

model is claude-opus-5-5, the official Claude API ID.
speed is "fast". Leave the field out and you get standard speed.
There is no thinking field. Passing older thinking parameters on Opus 5.5 will trigger a 400 error (see the gotchas section below).

What Changed Recently

The latest release notes bundle several updates for Opus 5.5 fast mode:

No waitlist: Fast mode is out of research preview for Claude Opus 5.5.
No beta header required: Requests to Opus 5.5 no longer require the fast-mode-2026-02-01 beta header (though requests that still send it will keep working). Note: If you are still calling claude-opus-5 or claude-opus-4-8, keep sending the beta header.
Default rate limits: Opus 5.5 now has a default fast mode rate limit across the Start, Build, and Scale usage tiers.
Azure support: Available on the first-party Claude API and on Claude in Microsoft Foundry for deployments hosted on Azure (not yet available on AWS/Bedrock or Batch API).

Pricing & When to Use It

Opus 5.5 Fast Mode: $8 USD / MTok (Input) · $40 USD / MTok (Output)
Fast mode pricing applies across the full context window (including requests over 200k input tokens), and prompt caching or data residency multipliers apply on top.
Rule of thumb: Since fast mode is not available with the Batch API, save "speed": "fast" for interactive paths where a user is actively watching tokens stream in, and route offline jobs to standard/batch.

Common Gotchas When Migrating Old Requests

Opus 5.5 introduces behavior changes unrelated to fast mode that often break copied-over requests:

The thinking parameter returns a 400 error: Both thinking: {"type": "disabled"} and thinking: {"type": "enabled", ...} will fail. Omit thinking entirely and use the effort parameter (low, medium, high) to control reasoning depth.
tool_choice types any and tool return a 400 error: Use auto with strict tool use instead.
Computer use toolset version: On the Claude API and Google Cloud, computer use now requires computer_toolset_20260801 (the older computer_20251124 returns a 400).

Read the full breakdown

The complete side-by-side curl/SDK examples, the model comparison table and the rate limit checklist are in the original article on CoreMesh: Claude Opus 5.5 Fast Mode.

This article was drafted by an AI system. Facts were checked against Anthropic's official documentation, which the original article lists as sources, and a human reviewed it before publishing.