DEV Community: Maksim Danilchenko

MarkItDown vs Docling vs Marker: PDF to Markdown for LLMs

Maksim Danilchenko — Sun, 03 May 2026 08:36:33 +0000

TL;DR

If you're feeding PDFs into a RAG pipeline or an LLM context window in 2026, three open-source tools own the space: MarkItDown (Microsoft, fast and shallow), Docling (IBM, slow and structurally rich), and Marker (Vik Paruchuri / Datalab, GPU-hungry and accuracy-first). None is universally best. Pick MarkItDown when your inputs are clean digital PDFs you control. Docling earns its keep when tables, formulas, or multi-column academic layouts dominate. Marker is the right call when you have GPU budget and need the highest fidelity you can get without paying a vendor.

Why bother comparing these three

Every team building on top of a language model hits the same wall eventually: most of the source material lives in PDFs. Contracts, research papers, datasheets, regulatory filings, internal SOPs all ship as PDF and don't paste cleanly into a context window. Even with the long-context tricks I covered in Recursive Language Models, you still need clean text on the way in — garbage tokenization is garbage retrieval. Markdown is the lowest-common-denominator format that an LLM actually reads well: headings, tables, lists, and code, without HTML's tag noise or PDF's positional spaghetti.

I've spent the last three weeks rebuilding a RAG ingestion pipeline that pulls roughly 4,000 PDFs from a regulatory archive: a mix of scanned 1990s circulars, recent EU directive PDFs with embedded tables, and academic papers with two-column layouts and inline math. The pipeline previously used pdfplumber plus a hand-rolled table heuristic, and it was a mess. So I sat down and tested the three tools that keep coming up in 2026 RAG threads on Reddit and HN. Here's what I found, what surprised me, and which one I shipped.

This is a comparison post, not a tutorial, but each tool gets a runnable snippet so you can reproduce the smoke test on your own corpus before committing.

The contenders, briefly

MarkItDown is Microsoft's official converter, MIT-licensed, currently at v0.1.5 (released February 20, 2026). It supports a long tail of formats (PDF, DOCX, PPTX, XLSX, HTML, images, audio, even YouTube URLs and EPUBs) and dumps everything to Markdown. The architecture is a thin wrapper around format-specific Python libraries (pdfminer.six for PDFs, python-pptx, mammoth, etc.). No models. No GPU. pip install and you're done in about ten seconds.

Docling is IBM Research's MIT-licensed converter, currently at v2.92.0 (released April 29, 2026, four days before this post). It uses a layout-detection model and an optional Visual Language Model called GraniteDocling (258M params) to preserve document structure. It runs on CPU by default but supports MLX acceleration on Apple Silicon and CUDA on NVIDIA. Output is a structured DoclingDocument you can export to Markdown, JSON, or HTML.

Marker is Datalab's GPL-3.0 converter (model weights under a custom Open RAIL-M license, free for personal and startup use under $2M revenue). Currently at v1.10.2 (released January 31, 2026). It bundles three of Datalab's own models (Surya for OCR + layout, Texify for formulas, and a layout/order model) into a tightly-tuned PDF pipeline. Peak VRAM is 5GB per worker. Datalab claims 122 pages/second on an H100, which translates to roughly 0.18s/page.

How I tested

Three input documents, picked to stress different parts of each tool:

A 14-page EU regulation PDF (digital, multi-column, dense tables) — the realistic ingestion case.
A 1996 scanned circular (300 DPI, blurry, OCR territory) — the worst case.
A 22-page arXiv paper (LaTeX-rendered, two-column, inline math, figures with captions) — the academic case.

Hardware: a Hetzner CPX31 (4 vCPU, 8GB RAM, no GPU) for the CPU runs, and a local M2 Pro MacBook with 32GB unified memory for the MLX/Apple-Silicon runs. No H100, so I can't reproduce Marker's GPU benchmark numbers; those stay flagged as reported by Datalab.

I scored each output on three axes: wall-clock speed, table fidelity (does the markdown table match the visual table cell-for-cell?), and structural sanity (do headings come through as ##, do lists stay as lists, do figure captions survive?).

MarkItDown: the fast, shallow workhorse

from markitdown import MarkItDown

md = MarkItDown(enable_plugins=False)
result = md.convert("eu-regulation.pdf")
print(result.text_content)

That's the whole API. There's no model to download, no GPU to provision, no config knobs that matter. On the 14-page EU regulation, MarkItDown finished in 0.6 seconds on the Hetzner box. On the 22-page arXiv paper, 1.1 seconds. On the scanned 1996 circular, it produced almost no usable output. pdfminer.six can't OCR, and MarkItDown doesn't run OCR by default.

The structural fidelity is where it falls apart. Tables in the EU regulation came out as run-on paragraphs of cell content with no pipe characters, no row breaks, nothing a downstream parser could recover. The arXiv paper's two-column layout interleaved left and right columns sentence by sentence, which is exactly what you don't want when chunking for retrieval. Headings sometimes survived as ## Heading, sometimes came through as bold text, sometimes vanished into the body.

Where MarkItDown shines is the rest of its format support. Throw it a PowerPoint deck and it produces clean Markdown with one slide per heading. Hand it a Word doc and it preserves nested lists and tables. The PDF path is the weak link, not the tool itself. If your corpus is 80% PowerPoint and 20% PDF, MarkItDown is the right answer. If it's the other way around, you're going to spend more time post-processing than you save.

One detail Microsoft buries in the README: MarkItDown can call Azure Document Intelligence as an OCR backend if you set the docintel_endpoint argument. That promotes it from "useless on scans" to "competitive on scans," but you're now paying Azure per page (roughly $1.50 per 1,000 pages on the read tier as of last check, with volume discounts above 1M pages), which is a different conversation.

Docling: slow, model-heavy, structurally accurate

from docling.document_converter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("eu-regulation.pdf")
print(result.document.export_to_markdown())

Same shape. Underneath, the first call downloads roughly 600MB of model weights from Hugging Face into your ~/.cache. Subsequent runs are faster but never as fast as MarkItDown. On the Hetzner CPX31, the EU regulation took 41 seconds. On the M2 Pro with MLX, it dropped to 9 seconds. The arXiv paper took 78 seconds CPU, 14 seconds MLX. The scanned 1996 circular finally produced legible Markdown at 52 seconds, because Docling's layout model can route scanned regions through its OCR path automatically.

Tables are where Docling earns its keep. The EU regulation's three-row, six-column tariff schedule came out as a clean Markdown table with the right cells in the right rows. The arXiv paper's results table preserved its column headers and row labels exactly. I didn't have to write a single regex to clean up output. That alone justifies the 50× wall-clock penalty for my use case.

Docling's DoclingDocument intermediate representation is more useful than I expected. You can export to Markdown, but you can also walk the document tree programmatically and pull out figures with their captions, tables as structured cells, or extract just the abstracts of academic papers without parsing the Markdown twice. For an ingestion pipeline that needs to chunk by section heading, this is a real win.

The downside, beyond speed: install size. The base wheel pulls in PyTorch, Transformers, and several CV libraries. A clean pip install docling in a fresh Docker image weighs in around 2.4GB. If you're packaging this for AWS Lambda, you're going to have a bad day. ECS Fargate or a real container runtime is the realistic deployment story.

Marker: GPU-hungry, accuracy-first

from marker.converters.pdf import PdfConverter
from marker.models import create_model_dict
from marker.output import text_from_rendered

converter = PdfConverter(artifact_dict=create_model_dict())
rendered = converter("eu-regulation.pdf")
text, _, images = text_from_rendered(rendered)

Three lines instead of two, but the API is still small. The first call downloads Datalab's Surya, Texify, and layout models (about 1.1GB). On the Hetzner CPX31 (CPU only), Marker took 2 minutes 14 seconds on the EU regulation, 4 minutes 30 seconds on the arXiv paper. CPU is not Marker's preferred surface. On the M2 Pro with MPS, those dropped to 38 seconds and 71 seconds, which is still slower than Docling-MLX but produced visibly better math output on the arXiv paper.

Where Marker pulls ahead: inline LaTeX. The arXiv paper's equations came through as $\hat{y} = \mathbf{W}x + b$ -style spans inside the Markdown, which is exactly what you want if you're handing the result to GPT or Claude. Both render LaTeX internally and reason about equations more accurately when the structure is preserved. Docling rendered most equations as image references with garbled OCR'd text. MarkItDown skipped them.

Marker's structural recall on tables was a tie with Docling on simple grids and slightly worse on nested headers (a multi-row column header in the EU regulation came out flattened). On figures, Marker has the cleanest behavior of the three: it extracts each figure as a separate PNG, references it from the Markdown with a relative path, and pulls the caption from the surrounding text. For a RAG pipeline that wants to embed image regions separately, this is a big quality-of-life upgrade.

Don't skip the license fine print. Marker's code is GPL-3.0, which is fine for most server-side workloads. The model weights are under Datalab's modified Open RAIL-M: free for personal use, research, and startups under $2M annual revenue/funding. Above that threshold, you need a commercial license from Datalab. If you're a Series-B-and-up company, factor in the procurment conversation before standardizing on Marker.

Head-to-head: the numbers

All wall-clock numbers below are from my own runs, not vendor benchmarks. The H100 column for Marker is reported by Datalab and not independently verified.

Dimension	MarkItDown	Docling	Marker
License	MIT	MIT	GPL-3.0 + Open RAIL-M (weights)
Install size	~80MB	~2.4GB	~1.5GB + 1.1GB models
Stars (May 2026)	120k	59k	34.6k
GPU required?	No	Optional (helps a lot)	Recommended
EU reg, CPU	0.6s	41s	2m 14s
arXiv paper, MLX/MPS	1.1s (CPU)	14s	71s
Scanned 1996 PDF	Empty	Legible	Legible
Tables (simple)	Broken	Excellent	Excellent
Tables (nested headers)	Broken	Excellent	OK
Inline math	Skipped	Image+OCR	LaTeX preserved
Figures + captions	Lost	Caption only	Image extracted + caption
Reported H100 throughput	n/a	n/a	122 pages/sec

Three takeaways from this matrix:

MarkItDown is in a different speed class from the other two. If your PDFs are clean and your downstream consumer doesn't care about table structure, MarkItDown buys you a 50–100× speedup over the other two. That gap is the difference between processing a 10K-document corpus in an afternoon and a week.
Docling and Marker are close on accuracy and far apart on dependencies. Docling is the easier deploy. Marker is the better GPU citizen.
Nobody ships table-fidelity Markdown without a model. The 2024-era pure-Python parsers (pdfplumber, pdfminer) do not produce LLM-grade output on real-world documents, and MarkItDown is essentially a polished wrapper around those parsers.

When to pick which

A short decision matrix, based on what I actually shipped:

Pick MarkItDown if your PDFs are digital-native and structurally simple, your corpus skews toward Office formats, you need to deploy to a constrained environment (Lambda, edge), or you're prototyping and don't yet know if PDF quality will be a bottleneck. I keep MarkItDown around for the PowerPoint and Word path even when Docling handles the PDFs.
Pick Docling if tables, formulas, or multi-column layouts dominate your corpus, you don't have a GPU, you want a clean intermediate representation you can walk programmatically, or you're on Apple Silicon and want MLX acceleration. This is what I shipped for the EU regulatory archive.
Pick Marker if you have GPU budget, your corpus is heavy on academic papers with inline math, you need clean per-figure extraction for downstream image embedding, or you're below the $2M revenue threshold for the model-weights license. For a research-paper pipeline at any reasonable scale, Marker is the strongest answer.

If you're building something general (a Notion-style "drop a PDF, get clean Markdown" feature, say), I'd run a tiered pipeline: MarkItDown first, fall back to Docling if MarkItDown's output looks structurally broken (zero tables detected, very low headings-to-body ratio), and fall back to Marker only for the documents that contain math. Most documents land in the fast path; the slow path only fires when it's worth the cost.

What the hosted alternatives offer

Two closed-source services keep coming up in the same threads, and they belong in any honest comparison even though this post focuses on open source:

Mistral Document AI is a hosted endpoint priced around $2 per 1,000 pages at last check (about half that with batch discounts). Reported quality on tables and math sits between Docling and Marker, with the operational benefit of zero local compute. I haven't run it on the same corpus as the open-source three, so treat that as second-hand impression rather than a measured ranking.
Reducto is more expensive (roughly $15 per 1,000 pages on the base tier) and is reportedly the strongest option on truly nasty inputs (handwritten annotations, multi-column scientific PDFs with inline formulas). Same caveat: I haven't paid for it on this corpus, so the framing is based on third-party benchmarks and a couple of recent HN threads, not my own runs.

If you care about time-to-market more than unit economics, paying a vendor is a perfectly defensible choice. If your corpus is large enough that the per-page bill would dominate your budget, the open-source path wins on cost even after you account for engineering time.

Getting started

The fastest path to evaluating all three on your own corpus:

If your usual stack is uv instead of plain pip (worth it — see uv vs pip vs Poetry for the case), swap the install command for uv pip install. The rest is identical.

# fresh venv
python3 -m venv .venv && source .venv/bin/activate

# install all three
pip install 'markitdown[all]' docling marker-pdf

# point them at the same PDF
python -c "from markitdown import MarkItDown; print(MarkItDown().convert('test.pdf').text_content)" > out_markitdown.md
python -c "from docling.document_converter import DocumentConverter; print(DocumentConverter().convert('test.pdf').document.export_to_markdown())" > out_docling.md
python -c "from marker.converters.pdf import PdfConverter; from marker.models import create_model_dict; from marker.output import text_from_rendered; r = PdfConverter(artifact_dict=create_model_dict())('test.pdf'); t,_,_ = text_from_rendered(r); print(t)" > out_marker.md

Diff the three Markdown outputs against your eyeballs. Whichever one you stop arguing with first is your tool. If you end up arguing with all three, you probably need a hosted service or a custom layout model, and that's a different post.

For deployment, my opinionated default in 2026: Docling in a slim Python container, with MarkItDown as the fast-path fallback for clean digital PDFs. Marker stays in a GPU pool for the academic-paper subset, called only when the document's first page contains LaTeX-shaped tokens. If you're exposing the converter as a tool for an LLM agent rather than a batch job, wrap it as an MCP server — see Build a real MCP server with FastMCP for the Python pattern I use for exactly this kind of glue.

FAQ

Which is better, MarkItDown or Docling?

For PDFs specifically, Docling produces materially better output on tables, formulas, and multi-column layouts. MarkItDown is roughly 50–100× faster on simple digital PDFs but loses structural information that downstream RAG retrieval depends on. For non-PDF formats (PPTX, DOCX, EPUB), MarkItDown is the better tool because Docling's PDF-first model architecture isn't applied there.

What is the fastest PDF-to-Markdown tool for LLMs?

MarkItDown, by a wide margin: it's a thin wrapper around pdfminer.six and runs in well under a second per page on CPU. The price is structural fidelity: it produces unusable output on tables, broken column ordering on multi-column PDFs, and nothing at all on scanned documents.

Does Docling work without a GPU?

Yes. Docling runs on CPU by default and is the only one of the three I'd recommend for CPU-only environments where accurate output still has to hold up. CPU runs are slower (40–80 seconds per multi-page document in my tests), but the output quality is the same. Apple Silicon with MLX cuts wall-clock by 3–5× without needing a discrete GPU.

Is Marker free to use commercially?

The code is GPL-3.0 and free to use, including commercially. The model weights are under Datalab's modified Open RAIL-M license: free for research, personal use, and any startup under $2M in annual revenue/funding. Above that threshold, you need a commercial license from Datalab.

How do I convert a PDF to Markdown for a RAG pipeline?

Pick the converter that matches your accuracy and compute budget: MarkItDown for clean digital PDFs and constrained compute, Docling for tables and CPU-only deploys, Marker for math and GPU-equipped pipelines. Then chunk the resulting Markdown by heading (split on ^##), embed each chunk with a sentence-transformer or a hosted embedding API, and store in your vector DB of choice. The converter quality directly determines retrieval quality, so it's worth A/B-testing two or three options on a representative slice of your corpus before committing.

Sources

MarkItDown — github.com/microsoft/markitdown — official Microsoft repo, MIT license, v0.1.5 release notes
Docling — github.com/docling-project/docling — official IBM Research repo, MIT license, v2.92.0 release notes
Marker — github.com/VikParuchuri/marker — official Datalab repo, GPL-3.0 + Open RAIL-M weights, v1.10.2 release notes
Docling whitepaper — arXiv:2408.09869 — IBM's technical report on the Docling architecture
Mistral Document AI — hosted alternative referenced for pricing context

Bottom line

Three usable tools, three honest tradeoffs. MarkItDown wins on speed and Office-format coverage. Docling wins on table fidelity and CPU-friendliness. Marker wins on math and figure handling, if you can spare the GPU. Pick the tool whose weakness you can live with rather than the one with the flashiest benchmark. Your bottleneck is downstream retrieval quality, not converter throughput, and the converter you pick is the input to that quality.

For my regulatory-archive job: Docling, MLX-accelerated on the M2 Pro for nightly batch ingestion, with MarkItDown as a fast-path optimization for the documents I already know are clean. The 4,000-PDF backfill ran over a weekend. The downstream retrieval got measurably better the day I switched off the old pdfplumber script, which was the whole point of the rebuild.

Python t-strings (PEP 750): A Practical Tutorial With Real Examples

Maksim Danilchenko — Mon, 27 Apr 2026 08:35:20 +0000

TL;DR

Python 3.14 ships t-strings (PEP 750), a new string literal that looks like an f-string but returns a Template object instead of a finished str. You get the static parts and the interpolated values separately, so a library author can sanitize, escape, parameterize, or defer the rendering. I rewrote a small SQLite logger I keep on my laptop using t-strings and the diff was about ten lines, but the SQL injection class of bug is now structurally impossible. Library authors will get the most use out of them; application code will mostly read t-strings rather than write them.

Why f-strings stop being enough

I have been writing Python since 2.6 and f-strings, introduced in 3.6, were a clear win. They replaced % formatting and .format() for almost everything I do. The catch is that f-strings evaluate immediately: the moment you write f"... {x} ...", Python calls str.__format__ on each interpolated value and concatenates the result. There is no hook, no transform, no chance for a library to inspect what got plugged into the gaps.

That sounds academic until you watch a junior engineer write cursor.execute(f"SELECT * FROM users WHERE name = '{name}'") for the third time. The "use parameterized queries" lecture is technically correct and operationally ignored, because the f-string syntax is too inviting. The Python 3.14 release notes from the Python 3.14.4 page call this out indirectly: PEP 750 lists "domain-specific languages that need string-like syntax with safe interpolation" as the headline use case.

T-strings close that hole. Instead of producing a str, the literal t"..." produces a string.templatelib.Template instance. The library author decides what happens next.

Setup: getting Python 3.14 on your machine

You need Python 3.14 or newer. The current stable as of this post is 3.14.4 (April 7, 2026). On macOS I use uv because it manages interpreter installs without touching the system Python (I compared uv against pip and Poetry here if you want the long version):

$ uv python install 3.14
$ uv python pin 3.14
$ uv run python --version
Python 3.14.4

If you prefer pyenv or the official installer, both work. The point is that t-strings are syntax. There is no from __future__ import to backport them. A t"..." literal is a SyntaxError on 3.13 and earlier.

Already on Python 3.14? See my walkthrough of the free-threaded build for the GIL story that shipped alongside t-strings.

The shape of a Template object

Open a 3.14 REPL and try this:

>>> name = "Pythonista"
>>> site = "danilchenko.dev"
>>> template = t"Hello, {name}! Welcome to {site}!"
>>> type(template)
<class 'string.templatelib.Template'>
>>> template.strings
('Hello, ', '! Welcome to ', '!')
>>> template.interpolations
(Interpolation('Pythonista', 'name', None, ''),
 Interpolation('danilchenko.dev', 'site', None, ''))
>>> template.values
('Pythonista', 'danilchenko.dev')

That output is the whole secret. Three observations:

strings is a tuple of the literal fragments around your interpolations. There is always exactly one more string than there are interpolations (some may be empty).
interpolations is a tuple of Interpolation objects, each with four fields: value, expression, conversion, and format_spec.
The order is implicit: the template alternates strings[0], interpolations[0], strings[1], interpolations[1], .... To walk the alternation explicitly you iterate the template directly: for item in template: ....

The Interpolation class deserves a closer look because the expression field is what makes structured logging click:

>>> i = template.interpolations[0]
>>> i.value
'Pythonista'
>>> i.expression
'name'
>>> i.conversion        # 'a', 'r', 's', or None
>>> i.format_spec       # '' here, '04d' in t"{n:04d}", etc.
''

The library author can read i.expression to learn the source code of the placeholder, not just its evaluated value. That single attribute makes structured logs, SQL placeholder names, and i18n catalog keys trivial to build. None of that was reachable from f-strings.

A SQL helper that makes injection structurally impossible

Here is the shortest practical example I keep around. The function turns any t-string into a (query, params) pair compatible with sqlite3.execute():

# safe_sql.py
import sqlite3
from string.templatelib import Template

def parameterize(template: Template) -> tuple[str, tuple[object, ...]]:
    if not isinstance(template, Template):
        raise TypeError("safe_sql expected a t-string")
    parts: list[str] = []
    params: list[object] = []
    for item in template:
        if isinstance(item, str):
            parts.append(item)
        else:
            parts.append("?")
            params.append(item.value)
    return "".join(parts), tuple(params)


def query(conn: sqlite3.Connection, template: Template):
    sql, params = parameterize(template)
    return conn.execute(sql, params)

Now use it:

>>> import sqlite3
>>> from safe_sql import parameterize, query
>>> conn = sqlite3.connect(":memory:")
>>> conn.execute("CREATE TABLE users (name TEXT, age INT)")
>>> conn.execute("INSERT INTO users VALUES (?, ?)", ("Anna", 33))

>>> evil = "'; DROP TABLE users;--"
>>> sql, params = parameterize(t"SELECT * FROM users WHERE name = {evil}")
>>> sql
'SELECT * FROM users WHERE name = ?'
>>> params
("'; DROP TABLE users;--",)
>>> list(query(conn, t"SELECT * FROM users WHERE name = {evil}"))
[]
>>> list(query(conn, t"SELECT * FROM users WHERE age > {30}"))
[('Anna', 33)]

The injected payload lands in the parameter tuple. SQLite escapes it correctly because the SQL itself never contains the value — it contains a ?. Compare against the f-string version that everyone has typed at 11 PM:

# Don't do this. Ever.
sql = f"SELECT * FROM users WHERE name = '{evil}'"
conn.execute(sql)
# sqlite3.OperationalError: near "DROP": syntax error
# (and on a different DB it would have happily dropped the table)

The structural win is that parameterize only accepts a Template. If a junior writes query(conn, f"..."), your type checker of choice catches it at the type boundary, and at runtime the isinstance check raises immediately. The unsafe path requires affirmative effort to reach.

I tried this on a small budget tracker that lives in ~/code/buckets. The before-state was a smattering of f"UPDATE accounts SET balance = {amount} WHERE id = '{acct}'" calls written for an audience of one (me) but written badly enough that I would not run it as a service. After porting to t-strings the diff was 8 lines of changed source plus a 14-line safe_sql.py helper. Every place that used to take a string now takes a Template. The class of bug went away because the wrong shape no longer typechecks.

HTML escaping with the same pattern

The exact same skeleton produces an HTML helper. The PEP 750 reference and Real Python's t-strings tutorial both show this; here is my version with the imports tightened:

# safe_html.py
from html import escape
from string.templatelib import Template

def render(template: Template) -> str:
    if not isinstance(template, Template):
        raise TypeError("safe_html expected a t-string")
    out: list[str] = []
    for item in template:
        if isinstance(item, str):
            out.append(item)
        else:
            out.append(escape(str(item.value), quote=True))
    return "".join(out)

In use:

>>> from safe_html import render
>>> bad = "<script>alert('xss')</script>"
>>> render(t"<p>Hello, {bad}!</p>")
"<p>Hello, &lt;script&gt;alert(&#x27;xss&#x27;)&lt;/script&gt;!</p>"

The static <p>...</p> passes through untouched because it is part of template.strings. The interpolated bad lands in template.interpolations, gets escaped, and only then concatenated. A reader cannot accidentally introduce XSS by writing user input into the template — the escaper sees user input as user input, not as a string fragment.

A more capable HTML library could special-case attribute interpolation, dict-of-attrs syntax, and component-style nesting. The PEP itself gestures at this with the t"<img {attributes} />" example where attributes is a dict.

Logging without paying for the format string

Python's logging module has a long-standing performance trick: pass a format string and the args separately, like log.info("user %s logged in", user_id), so that %-formatting only runs if the log line actually fires. F-strings break this — the format runs at the call site whether or not INFO is enabled.

T-strings give you the trick back, plus structured context:

# t_log.py
import json
import logging
from string.templatelib import Template


class LazyTemplate:
    """A logging-safe wrapper that defers rendering."""
    def __init__(self, template: Template):
        if not isinstance(template, Template):
            raise TypeError("LazyTemplate expected a t-string")
        self._template = template

    def __str__(self) -> str:
        parts: list[str] = []
        for item in self._template:
            if isinstance(item, str):
                parts.append(item)
            else:
                value = format(item.value, item.format_spec)
                parts.append(value)
        msg = "".join(parts)
        ctx = {
            i.expression: i.value
            for i in self._template.interpolations
        }
        return f"{msg} | {json.dumps(ctx, default=str)}"


def info(template: Template) -> None:
    logging.info("%s", LazyTemplate(template))

Used like this:

>>> import logging, t_log
>>> logging.basicConfig(level=logging.INFO, format="%(message)s")
>>> user, latency = "anna", 42.7
>>> t_log.info(t"login complete for {user} in {latency:.1f}ms")
login complete for anna in 42.7ms | {"user": "anna", "latency": 42.7}

When the level is raised to WARNING, the __str__ call never runs and the JSON dict is never built. You get human-readable messages and machine-readable context from one literal, with no extra cost when the log line is suppressed.

f-strings vs t-strings — a side-by-side cheat sheet

Aspect	f-string (`f"..."`)	t-string (`t"..."`)
Return type	`str`	`string.templatelib.Template`
Evaluated when?	Immediately at the literal	Whenever the consumer iterates it
Where to use	Application code, print, simple formatting	Library APIs that take user-controlled values
Can a library hook in?	No — already concatenated	Yes — via `template.strings` and `template.interpolations`
Knows the source expression?	No	Yes — `interpolation.expression`
Can replace any `str`?	Yes	No — needs a renderer first
Backportable?	No (3.6+)	No (3.14+ syntax)
Raw variant?	`rf"..."`	`rt"..."` or `tr"..."`

The "Can replace any str?" row is the source of every gotcha. Because a Template is a separate type, you cannot pass it to print and expect formatted output, you cannot send it to a function that calls len() on it, and t"hi" + " there" raises TypeError. The library author has to provide a renderer, which is by design and which surprises people on the first day.

Caveats and gotchas worth knowing

A few things tripped me up the first week, in order of how much time each one cost me.

You cannot mix f and t prefixes. ft"..." is a SyntaxError. If you need both behaviors in one file, write two literals. The accepted prefix combinations are t, T, rt, Rt, rT, RT, tr, tR, Tr, TR. No others.

Template does not implement __len__ or __contains__. This is deliberate — the value can change once you render it, and a library author may render to something other than a string. If you want length, render first.

isinstance(x, Template) is the right check, not isinstance(x, str). I wasted thirty minutes on a function that did if not x: on a template, which calls __bool__, which is always truthy for templates, so type-check explicitly.

Empty static segments are still in template.strings. A literal t"{a}{b}" produces strings = ("", "", ""). Direct iteration over the template silently drops the empties, so for item in template: already does the right thing for renderers; the empties only show up if you read template.strings directly.

The expression field is the source text, not a variable lookup. t"{a + b}" gives an Interpolation whose expression is "a + b" and whose value is the evaluated result. Useful for debug logs; do not try to round-trip the expression back through eval.

There is no f-string to t-string converter. A linter could rewrite trivial cases, but in general the migration is a behavior change and has to be reviewed by hand. I ported the SQL spots first because the security argument made the priority obvious; the rest can wait until the helpers exist for them.

Subprocess support is still a draft. PEP 787 proposes letting subprocess.run(t"...", shell=True) shell-quote interpolated values automatically. As of 3.14.4 it is deferred — the authors plan to revise after experimental implementations land in the 3.14 beta cycle, with a target of 3.15. For now, write your own shlex.quote renderer if you need one.

When not to use t-strings

I keep seeing developers reach for t-strings everywhere because the security framing is compelling. Most code does not need them.

Application code that builds a one-shot human-readable message (a print statement, an exception text, a debug log) should keep using f-strings. The reason f-strings are so popular is that they are the right tool for the boring 90% of string formatting. T-strings only pay for themselves when there is a consumer of the literal that needs to inspect it. If the consumer is print, an f-string is shorter, faster, and easier to read.

The rule of thumb I am using: t-string the API, f-string the body. Library boundaries take templates; everything inside the function uses regular strings.

FAQ

What are t-strings in Python?

T-strings are a new string literal in Python 3.14, introduced by PEP 750. The syntax mirrors f-strings — t"hello {name}" — but the literal evaluates to a string.templatelib.Template instance instead of a str. The Template exposes the static fragments and interpolated values separately, so library code can intercept and transform them before final rendering.

How are t-strings different from f-strings?

F-strings produce a str immediately. T-strings produce a Template object. F-strings are convenient for application code; t-strings are designed for library APIs that need to sanitize, escape, parameterize, or defer the interpolation. You can iterate a Template to walk the alternation of static strings and Interpolation objects; you cannot do that with an f-string because the f-string is already collapsed into a flat string.

How do t-strings prevent SQL injection?

They do not prevent it on their own — they make a safe API expressible. Because the library function only ever sees the user input as interpolation.value, never as part of the SQL fragment, you can replace each interpolation with a ? placeholder and pass the values through the database driver's parameter binding. The driver does the actual escaping. The structural change is that the unsafe path (raw f-string concatenation) is no longer the path of least resistance.

What Python version supports t-strings?

Python 3.14, released October 7, 2025, with the latest patch being 3.14.4 on April 7, 2026. T-strings are a syntactic feature, so there is no backport. A t"..." literal will raise SyntaxError on 3.13 and earlier.

Can you pass a t-string anywhere a string is expected?

No. Template is not a subclass of str. Passing one to print() will print the repr of the Template, not the rendered text. Concatenation with + raises TypeError. The library that consumes the t-string has to provide a renderer. This is by design. Silently coercing to str would defeat the security guarantees t-strings are built for.

Will t-strings replace f-strings?

No. F-strings remain the right tool for application-level string formatting. T-strings target library and DSL authors. Most Python users will write t-strings only when calling SQL, HTML, logging, i18n, or shell helpers, and will consume them rarely.

Sources

PEP 750 — Template Strings — the accepted proposal that introduced t-strings, with the full motivation, rationale, and reference implementation.
string.templatelib — Python 3.14.4 documentation — official module reference for Template and Interpolation.
Python 3.14.4 release notes — the patch release used for examples in this post.
What's new in Python 3.14 — full changelog including t-strings, free-threading, and the experimental JIT.
Real Python — Python 3.14: Template Strings — secondary tutorial with additional examples used to cross-check the SQL and HTML helpers.
PEP 787 — Safer subprocess usage using t-strings — deferred proposal for subprocess and shlex integration.

Bottom line

T-strings are a small syntax change with most of the impact concentrated in library APIs. Your daily print(f"hello {name}") keeps working as before. But over the next few years, expect sqlite3, psycopg, httpx, subprocess, and the structured logging libraries to grow t-string-aware constructors. The code samples in this tutorial are short on purpose: once you understand template.strings and template.interpolations, every other helper is a variation on the same loop. Try it on the next SQL or HTML hot spot in your codebase. The diff is small, and the class of bug it removes is large.

Hetzner vs DigitalOcean 2026: Real Numbers After the Price Hike

Maksim Danilchenko — Sun, 19 Apr 2026 02:14:11 +0000

TL;DR

Hetzner raised most cloud server prices by 30–37% on April 1, 2026 (steeper on some US tiers). Despite that, it's still 50–70% cheaper than DigitalOcean for equivalent CPU and RAM, and it includes 4–5× more bandwidth on the same tier. Recent migration write-ups land on roughly the same number: about $14K saved per year on a mid-sized stack. Switching is worth it if you're running your own MySQL/Postgres and Nginx; it isn't worth it if you depend on managed databases, App Platform, or Spaces. I run two production boxes on Hetzner from Cyprus and one droplet on DigitalOcean for a US-only side project, so the rest of this comes straight from current bills.

What changed on April 1, 2026

Hetzner announced the price adjustment in late February and rolled it out a month later. The company cited rising hardware acquisition costs; Tom's Hardware framed it against a 171% year-over-year jump in DRAM. The change applies to both new orders and existing products. There was no grandfathering.

The increases aren't uniform:

Cloud servers in Germany and Finland: +30% to +37% depending on tier
Cloud servers in the US: broadly similar, with some tiers seeing larger jumps
Dedicated servers: smaller bumps, mostly in setup fees
Storage Box and bandwidth pricing: largely unchanged

DigitalOcean hasn't raised pricing in 2026. The gap narrowed, but it didn't close.

Current pricing: side by side

This is a head-to-head on the tiers that come up the most in real billing tickets: small workhorse VMs, mid-sized API servers, and "I just want a Postgres host" boxes. All numbers are post-April-1 Hetzner pricing, converted at €1 = $1.07.

Tier (Hetzner SKU)	RAM / vCPU / Disk	Hetzner Cloud (FSN/HEL)	DigitalOcean Basic	Hetzner Bandwidth	DO Bandwidth
Entry (CPX22)	4 GB / 2 vCPU / 40 GB NVMe	€7.99 / mo (~$8.55)	$24 / mo	20 TB	4 TB
Mid (CPX32)	8 GB / 4 vCPU / 80 GB NVMe	€13.99 / mo (~$14.97)	$48 / mo	20 TB	5 TB
Workhorse (CPX42)	16 GB / 8 vCPU / 160 GB NVMe	€25.49 / mo (~$27.27)	$96 / mo	20 TB	6 TB
Beefy (CPX52)	32 GB / 16 vCPU / 240 GB NVMe	€36.49 / mo (~$39.04)	$188 / mo (DO General Purpose, 8 vCPU)	20 TB	6 TB

The pre-April Hetzner CPX21 (the spiritual ancestor of the entry tier) cost €5.83/mo, so €7.99 represents a +37% jump. Even after that bump, the Hetzner column is roughly a third of DO Basic at the low end, and at the Workhorse tier you get 2× the vCPUs on top of the price gap. You also get 4–5× the included bandwidth across every tier.

The bandwidth point is the one that flips ROI for video, image-heavy SaaS, and game servers. DigitalOcean charges roughly $10/TB over the included quota (priced as $0.01/GB); Hetzner charges €1/TB. On a workload pushing 10 TB/month over the included tier, that's $100/month versus about €10/month, roughly $1,000/year in bandwidth savings on top of the base price gap.

Performance: closer than you'd guess

Hetzner runs newer silicon. The CPX line uses AMD EPYC 7002 and 7003 (Rome and Milan); the dedicated AX line is on EPYC 9004 (Genoa). DigitalOcean's Premium AMD droplets run EPYC Milan and Genoa too, but the Basic droplets (the ones most people are actually paying for) sit on older Skylake and Cascade Lake Xeons.

From benchmarks I've run myself and cross-checked against VPSBenchmarks: Hetzner CPX is ~25–40% faster on single-core CPU and 2× faster on disk IOPS than a same-priced DigitalOcean Basic droplet. Network throughput within the same datacenter is comparable on both; cross-region latency from Hetzner Falkenstein to a US-East user runs about 110ms, versus ~25ms on a DO NYC droplet.

The latency number is the one that decides things for anyone outside Europe. If your audience is US-only, the Hetzner US datacenters in Ashburn and Hillsboro are real options now, but they're smaller and the EU-tuned support muscle doesn't fully reach them yet. For a Cyprus or EU-focused product, Falkenstein is the obvious win.

Real migration numbers from the last six months

These are from public write-ups by people who actually moved production traffic, not promo posts.

Isa Yeter documented a full migration: 30 MySQL databases (248 GB), 34 Nginx vhosts, GitLab EE, Neo4j, hundreds of thousands of mobile users, going from $1,432/month on DigitalOcean to $233/month on a Hetzner AX162-R dedicated server with 48 cores and 256 GB DDR5. That's the headline $14K/year number making the rounds.

The Talk Python infrastructure swap reported a similar pattern: a decade on DigitalOcean, then about $1,500/year saved by moving the same workload to Hetzner Cloud. byteiota's writeup landed at 60% off. The shape of the savings is consistent: half to two-thirds off regardless of stack size, because the underlying euro-per-vCPU-per-month math is the same whether you're running one box or twenty.

Where DigitalOcean still wins

Migration breakeven depends on more than the raw bill. DigitalOcean's PaaS layer is the part you actually pay for, and Hetzner doesn't have an equivalent.

Managed Databases: DO's managed Postgres, MySQL, Redis, MongoDB, and Kafka are turnkey with point-in-time recovery, read replicas, and automatic failover. Hetzner gives you a bare VM and an apt install postgresql problem.
App Platform: Heroku-style git-push deploys with autoscaling, build pipelines, and edge routing. Hetzner has Cloud Console; you bring your own CI/CD.
Spaces (S3-compatible object storage): Hetzner has Storage Boxes (FTP/SFTP/SMB), which aren't the same thing. If you need S3 semantics in Europe, you're looking at OVHcloud Object Storage, Scaleway, or a self-hosted MinIO on a Hetzner box.
One-click apps and droplet snapshots that work like AMIs: DigitalOcean has invested in this for a decade. Hetzner snapshots work but feel less polished.
24/7 chat support with US business hours coverage: DO has it. Hetzner has email tickets and a community forum.

If your team is two people and the half-day per month spent on database ops would otherwise go into shipping features, paying ~$60/month for DigitalOcean Managed Postgres on the smallest production tier is a defensible call. If you have a dedicated SRE or you genuinely enjoy pg_basebackup, Hetzner wins on every other axis.

Migration playbook: the zero-downtime version

The pattern that keeps showing up in successful migrations is the same six-phase outline. This is the abridged version; if you are moving real traffic, Isa Yeter's full guide is the most thorough recent reference.

# 1. Drop DNS TTL to 300s a week before the cutover
# (Doing this on day-of is too late — caches lag.)
dig +short example.com
# Verify TTL on registrar side, set to 300

# 2. Provision the Hetzner box and bring it to parity
# OS, packages, configs, secrets, deploy keys
# Use a configuration tool you already trust — Ansible, Pulumi, or shell

# 3. Set up MySQL/Postgres replication from DO → Hetzner
# Old box = primary, new box = replica, async streaming
# For MySQL: GTID-based replication
# For Postgres: physical or logical replication

# 4. Cut traffic by flipping DNS A records
# Old box keeps running as a fallback for 24h

# 5. Convert the old DO box to a reverse proxy
# Anything still hitting the old IP gets forwarded to Hetzner
# This handles cached resolvers without dropping a single request

# 6. Tear down the DO box after 7 days of clean logs

Two things break in real migrations and never make it into the marketing case studies.

First, MySQL mysql.user schemas drift between minor versions, and a 5.7→8.0 jump will fail the slave promotion if you haven't done mysql_upgrade --force and rebuilt the sys schema. Test this on a staging copy.

Second, application users that you granted SUPER to during some emergency three years ago will quietly bypass read_only = 1 on the replica and write to the wrong master. Check SHOW GRANTS for every account before you cut traffic, and revoke SUPER from anything that isn't an admin. The Yeter writeup hit this on 24 accounts.

GitLab webhooks are the third one if you are running GitLab. They store the absolute IP, not the hostname, and you have to do a bulk API rewrite after the cutover.

Cyprus and the EU: latency, residency, and the boring win

Hetzner is a German company with datacenters in Falkenstein, Nuremberg, and Helsinki. From Cyprus, latency to FSN runs about 60–80ms versus 110ms+ to DO Frankfurt. From any EU country, you're getting GDPR-clean data residency by default: no DPA acrobatics, no Standard Contractual Clauses for a US sub-processor, no awkward conversation with your enterprise customer's legal team.

For startups based in Cyprus, Estonia, Portugal, or anywhere on the Blue Card / digital nomad track, this is a quietly useful side benefit. The EU AI Act and the data sovereignty pieces of the Digital Services Act both nudge companies toward keeping inference and customer data inside the EU. A Falkenstein box is the cheapest way to be compliant on day one without rearchitecting your stack later on.

You might also like the Polars vs Pandas comparison if you're squeezing more out of a single Hetzner box on a data workload.

FAQ

Is Hetzner cheaper than DigitalOcean?

Yes. Even after the April 1, 2026 price increase of 30–37%, Hetzner cloud servers cost roughly 50–70% less than equivalent DigitalOcean droplets on the same RAM and vCPU. The 4 GB / 2 vCPU tier is €7.99/month on Hetzner versus $24/month on DigitalOcean. Hetzner also includes 20 TB of bandwidth versus 4 TB on DO, which widens the gap further for traffic-heavy sites.

Is Hetzner reliable?

In production usage, yes. Hetzner runs three EU datacenters (Falkenstein, Nuremberg, Helsinki) and two US ones (Ashburn, Hillsboro), with a published uptime track record comparable to DigitalOcean. The differences are at the SLA paperwork layer (DigitalOcean publishes a 99.99% SLA, Hetzner's is less prominent) and at the support layer, where Hetzner is email-ticket-only versus DO's chat support.

How do you migrate from DigitalOcean to Hetzner with zero downtime?

The proven pattern: drop DNS TTL to 300 seconds a week ahead of the cutover, provision and configure the Hetzner box to full parity, set up MySQL/Postgres replication with the old box as primary, flip DNS, and convert the old box to a reverse proxy for cached-resolver traffic for 24 hours. Tear down the old box only after 7 days of clean logs. Real migrations of 30+ databases have completed in 24 hours with zero downtime using this exact sequence.

Why is Hetzner so cheap?

Three reasons. They own and operate their own datacenters in lower-cost regions of Germany and Finland (cheap power, cheap real estate). They run a flat catalog with no managed-service margin layered on top. And they've historically chosen newer-but-cheaper AMD EPYC silicon over the brand-name Intel Xeon parts that hyperscalers default to. After the April 2026 price hike they're still cheaper, just less dramatically so.

Is Hetzner good for production workloads?

For self-managed stacks: yes, and a lot of European startups have been on it for years. For workloads that lean heavily on managed services (managed databases, S3-compatible object storage with full API compatibility, autoscaling app platforms, edge networks), DigitalOcean, AWS, or GCP are still the right call. Hetzner is a "you do the ops" platform. That's both why it's cheap and why it isn't for everyone.

Does the April 2026 price hike change the migration math?

It compresses the payback period but doesn't eliminate the savings. If you were saving $1,000/month at the old prices, you're saving $700–800/month at the new prices on the same workload. A typical migration that took 40 engineering hours to execute now pays back in 3–5 months instead of 2–3. Still worth it for any stack where the original DO bill is over $200/month.

Sources

Hetzner — Statement on price adjustment as of April 1st 2026 — official announcement
Tom's Hardware — German data center giant hikes prices up to 37% — independent reporting on the price hike
Isa Yeter — DigitalOcean to Hetzner migration: $1,432 to $233/month — full zero-downtime playbook with real numbers
byteiota — DigitalOcean to Hetzner: $14K Saved, 60% Cost Cut (2026) — second migration story corroborating the savings ratio
Hetzner Cloud Pricing — current per-tier pricing referenced in the comparison table
DigitalOcean Pricing — current droplet pricing referenced in the comparison table

Bottom line

The April 2026 price hike was Hetzner closing a gap that was always going to close: they were too cheap for the global semiconductor cycle they were absorbing. Even at the new prices, the math on a self-managed stack still lands in the same place: half to two-thirds off your DigitalOcean bill, with better silicon and more bandwidth thrown in. The catch: you have to like running your own databases, and you have to be okay with email-only support. If those two things are acceptable, the migration is one of the cleanest infrastructure wins of 2026. If they aren't, pay the DigitalOcean tax and ship features instead.

Python 3.14 Free-Threading: Real Benchmarks, Real Breakage, Real Code

Maksim Danilchenko — Mon, 13 Apr 2026 02:15:25 +0000

TL;DR

Python 3.14 makes free-threading officially supported. You get true thread-level parallelism for CPU-bound work, with up to 3.5x speedups on 4 cores. The single-threaded penalty dropped from ~40% in 3.13 to roughly 5-10%. But the library support isn't fully there yet: any C extension that hasn't opted in will silently re-enable the GIL. Here's how to install it, what actually works, and when it's worth the switch.

The GIL Is Finally Optional

For over three decades, CPython's Global Interpreter Lock has been the answer to "why can't Python use all my cores?" The GIL ensures only one thread executes Python bytecode at a time. That keeps things simple but means CPU-bound code can't use multiple cores.

Python 3.13 introduced an experimental free-threaded build. Python 3.14, released October 2025, promoted it to officially supported status via PEP 779. The implementation described in PEP 703 is now complete. Temporary workarounds in the interpreter have been replaced with permanent solutions, and the single-threaded performance hit has been slashed.

Two things to know upfront:

Free-threading is supported but not the default build. You still have to opt in.
If you import a C extension that hasn't declared itself thread-safe, the interpreter quietly re-enables the GIL for the entire process. Your threads keep running, but they won't run in parallel.

How to Install the Free-Threaded Build

The free-threaded interpreter ships as a separate binary: python3.14t (note the t suffix).

With uv (fastest method):

uv python install 3.14t
uv venv --python 3.14t
source .venv/bin/activate
python --version  # Python 3.14.x (free-threading build)

If you just read our uv vs pip vs Poetry comparison, you already know uv handles Python version management. The 3.14t variant is a first-class citizen.

With the official installers (macOS/Windows):

Download from python.org/downloads. On macOS, the installer has an optional checkbox for the free-threaded build. On Windows, use py install 3.14t.

Building from source (Linux):

git clone https://github.com/python/cpython.git
cd cpython
git checkout v3.14.3
./configure --disable-gil --prefix=$HOME/.local/python3.14t
make -j$(nproc)
make install

Verify it works:

import sys
print(sys._is_gil_enabled())  # False = free-threading active

If that prints True, a C extension re-enabled the GIL. More on that in the breakage section.

Benchmarks: The Numbers That Matter

I ran three CPU-bound benchmarks comparing python3.14 (GIL build) and python3.14t (free-threaded) on a 4-core machine.

Test 1: Prime counting

import sys
import time
from concurrent.futures import ThreadPoolExecutor

def count_primes(start, end):
    count = 0
    for n in range(start, end):
        if n < 2:
            continue
        for i in range(2, int(n**0.5) + 1):
            if n % i == 0:
                break
        else:
            count += 1
    return count

def bench_threads(num_threads, limit=500_000):
    chunk = limit // num_threads
    ranges = [(i * chunk, (i + 1) * chunk) for i in range(num_threads)]

    start = time.perf_counter()
    with ThreadPoolExecutor(max_workers=num_threads) as pool:
        results = list(pool.map(lambda r: count_primes(*r), ranges))
    elapsed = time.perf_counter() - start

    print(f"Threads: {num_threads}, Primes: {sum(results)}, "
          f"Time: {elapsed:.2f}s, GIL: {sys._is_gil_enabled()}")

for t in [1, 2, 4]:
    bench_threads(t)

Test 2: SHA-256 hashing

import hashlib
import time
from concurrent.futures import ThreadPoolExecutor

def hash_work(n):
    data = b"benchmark" * 1000
    for _ in range(n):
        hashlib.sha256(data).hexdigest()

start = time.perf_counter()
with ThreadPoolExecutor(max_workers=4) as pool:
    pool.map(hash_work, [100_000] * 4)
elapsed = time.perf_counter() - start
print(f"4 threads, 400K hashes: {elapsed:.2f}s")

Results

Benchmark	GIL build (1 thread)	GIL build (4 threads)	Free-threaded (1 thread)	Free-threaded (4 threads)
Prime counting (500K)	2.31s	2.28s	2.45s	0.68s
SHA-256 (400K hashes)	4.12s	4.09s	4.34s	1.18s
Matrix multiply (pure Python)	1.87s	1.85s	1.98s	0.57s

With the GIL, adding threads to CPU-bound Python code does nothing. Free-threaded, you get near-linear scaling up to your core count. The single-threaded overhead (about 6% in my tests) comes from the atomic operations CPython now uses instead of the GIL lock.

What Breaks (and the Silent GIL Trap)

When the free-threaded interpreter loads a C extension module that hasn't been marked as safe for concurrent use, it automatically re-enables the GIL for the entire process. There's no warning or error message — your code keeps running, but threads take turns instead of running in parallel.

You can detect this at runtime:

import sys
import numpy  # might re-enable the GIL

if sys._is_gil_enabled():
    print("GIL was re-enabled by an extension module")
else:
    print("Free-threading is active")

This is a backwards-compatibility safeguard. CPython can't know whether an extension's internal state is thread-safe, so it assumes the wrost. Extension authors need to explicitly opt in by setting Py_mod_gil in their module definition.

Library Compatibility Right Now

I checked the py-free-threading tracker and ft-checker.com in April 2026. Major library status:

Library	Free-threaded wheels?	GIL re-enabled?	Notes
NumPy 2.3+	Yes	No	Improved in 2.3, still some edge cases
pandas	Yes	Partial	Some operations re-enable GIL
scikit-learn 1.8+	Yes	No	Free-threaded wheels on all platforms (ongoing optimization)
SciPy	Yes	Partial	Core routines work, some submodules lag
Matplotlib	Yes	Yes	Plotting re-enables GIL (expected, not thread-safe)
PyArrow	Yes	No	Good support since 18.0
Pydantic	Yes	No	Works with free-threaded builds since v2.11
FastAPI / Uvicorn	Yes	Mostly no	ASGI event loop + threads works
requests	Yes	No	I/O-bound, GIL irrelevant anyway
SQLAlchemy	Yes	Partial	Connection pools need care

The Quansight Labs team and Meta's Python runtime group have been doing the heavy lifting on library compatibility. But if your stack includes niche C extensions — custom Cython modules or anything with hand-written CPython API calls — test before you deploy.

When Free-Threading Actually Helps

Free-threading shines when your bottleneck is CPU-bound Python code running across multiple cores. Good use cases:

Data processing pipelines where you transform chunks in parallel
Pure-Python numerical computation (though you should probably use NumPy)
Web servers handling CPU-heavy request processing alongside async I/O
AI inference preprocessing: tokenization, feature extraction across batches

It doesn't help when:

Your code is I/O-bound (async/await is still the right tool)
You're already using NumPy/pandas for the heavy lifting (those release the GIL internally)
Your C extensions re-enable the GIL anyway
You need isolation between workers (use multiprocessing or the new InterpreterPoolExecutor)

The New InterpreterPoolExecutor

Python 3.14 also shipped concurrent.futures.InterpreterPoolExecutor (PEP 734). Each worker gets its own interpreter with isolated state: no shared memory, no GIL contention. Think of it as a lighter-weight multiprocessing without the serialization overhead of IPC.

from concurrent.futures import InterpreterPoolExecutor

def cpu_work(n):
    return sum(i * i for i in range(n))

with InterpreterPoolExecutor(max_workers=4) as pool:
    results = list(pool.map(cpu_work, [10_000_000] * 4))

This is a better fit when you need true isolation. No worrying about thread safety at all.

Other Python 3.14 Features Worth Knowing

Free-threading gets the headlines, but 3.14 packed in several other changes:

Template strings (PEP 750) let you write t"Hello {name}" — like f-strings but for custom processing. Build SQL queries, HTML templates, and log messages with proper escaping.

Deferred annotation evaluation (PEP 649) means annotations are no longer eagerly evaluated. Forward references just work. If you've ever fought from __future__ import annotations, this fixes it properly.

There's also compression.zstd in the stdlib (PEP 784) — Zstd compresses faster than gzip at similar ratios. And official macOS/Windows binaries now include a copy-and-patch JIT compiler. Early days, but it shows where CPython is headed.

FAQ

Is Python 3.14 free-threading production-ready?

For CPU-bound workloads where you control the dependency stack, yes. For complex applications with many C extensions, test thoroughly. The "officially supported" label means CPython commits to maintaining it, but third-party library coverage is still catching up.

Will free-threading become the default?

PEP 703 laid out a three-phase plan. Phase 1 (experimental, 3.13) and Phase 2 (supported, 3.14) are done. Phase 3 would make free-threading the default build, but no specific version has been committed to. The timeline depends on how fast libraries adopt free-threaded builds.

How much slower is single-threaded code?

About 5-10% compared to the GIL build, down from ~40% in 3.13. The overhead comes from atomic reference counting and per-object locks that replace the GIL's coarse-grained protection.

Can I use free-threading with Django/Flask?

Yes, with caveats. ASGI servers like Uvicorn can benefit from mixed async + thread workloads. But web frameworks rarely bottleneck on CPU-bound Python code. Most of the time is spent waiting on databases and external APIs. Profile before optimizing.

What happens if I mix free-threaded and GIL-requiring packages?

The GIL gets re-enabled for the whole process. You won't get an error. Your code just runs single-threaded like regular Python. Check sys._is_gil_enabled() after imports to verify.

Bottom Line

The GIL removal is real, and it works. I've been running CPU-bound batch jobs on python3.14t for a few months now, and the multi-core speedups are exactly what Python has needed for decades. The 6% single-threaded overhead is a reasonable trade.

But don't rip out your multiprocessing code just yet. Most libraries need another 6-12 months before most developers can switch without hitting the silent GIL re-enable. Check your deps with sys._is_gil_enabled(), verify with the compatibility tracker, and start with isolated workloads where you control the stack.

Free-threading works. Libraries just need time to catch up.

How to Run Gemma 4 Locally With Ollama, llama.cpp, and vLLM

Maksim Danilchenko — Sat, 11 Apr 2026 22:40:26 +0000

TL;DR

Google Gemma 4 dropped on April 2 under Apache 2.0 and it's genuinely good: the 31B dense model hit #3 on the Arena AI leaderboard, beating models 20x its size. You can run it locally with Ollama in about two minutes, or go the llama.cpp / vLLM route if you want more control. But there are real bugs right now, especially on Apple Silicon and with tool calling. This guide covers all three options, what hardware you actually need, and the workarounds for the issues I've hit so far.

Why Gemma 4 Is Worth Running Locally

I've been running local models since the Llama 2 days, and Gemma 4 is the first time an open model has made me reconsider whether I need API access to frontier models for everyday coding tasks.

Look at the benchmarks. Gemma 4 31B scores 89.2% on AIME 2026 (math), 80.0% on LiveCodeBench v6 (coding), and 84.3% on GPQA Diamond (science). Gemma 3 scored 20.8%, 29.1%, and 42.4% on those same tests. Every metric roughly tripled in one generation.

The family comes in four sizes:

Model	Parameters	Active Params	Min VRAM (Q4)	Best For
E2B	2.3B	2.3B	~1.5 GB	Mobile, Raspberry Pi
E4B	4.5B	4.5B	~3 GB	Quick local tasks
26B MoE	26B	3.8B	~14 GB	Best bang per VRAM GB
31B Dense	31B	31B	~18 GB	Maximum quality

The 26B MoE model is the sleeper hit here. It only activates 3.8B parameters per token but delivers reasoning quality close to the full 31B, and it fits in 14 GB of VRAM at Q4 quantization. If you're on a 16 GB GPU or a MacBook Pro with 18 GB unified memory, go with that one.

All four variants ship under Apache 2.0. No usage restrictions, no commercial limitations, no weird "you can't use this to compete with Google" clauses that plagued earlier open model releases. (If you're on a Mac and want to explore Apple's built-in local AI too, see my Apfel review — different beast, but it's free and already on your machine.)

Option 1: Ollama (Easiest)

Ollama is the fastest way to get Gemma 4 running. Two commands and you're chatting.

Install Ollama

On macOS:

brew install ollama

On Linux:

curl -fsSL https://ollama.com/install.sh | sh

On Windows, download the installer from ollama.com.

You need Ollama v0.20.0 or later for Gemma 4 support. Check with:

ollama --version

Pull and Run a Model

# The 26B MoE — best quality-to-VRAM ratio
ollama run gemma4:26b

# The small but capable 4B
ollama run gemma4:4b

# The full 31B dense (need 20+ GB VRAM)
ollama run gemma4:31b

# Tiny model for edge devices
ollama run gemma4:2b

That's it. Ollama handles downloading the GGUF, quantization selection, and memory management automatically. By default it picks a quantization that fits your available memory.

Pick Your Quantization

If you want more control over the quality/memory tradeoff:

# Higher quality, more memory
ollama run gemma4:26b-q8_0

# Lower memory, slightly less quality
ollama run gemma4:26b-q4_K_M

# Middle ground
ollama run gemma4:26b-q5_K_M

For the 31B model, Q4_K_M is the sweet spot. It keeps quality high while fitting in ~18 GB. Going to Q8 pushes you to ~28 GB, which means you need a 32 GB GPU or Mac with 32+ GB unified memory.

Use the API

Ollama exposes an OpenAI-compatible API on port 11434:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:26b",
    "messages": [{"role": "user", "content": "Write a Python function to merge two sorted arrays"}]
  }'

This works with any OpenAI SDK client. Just point the base URL to http://localhost:11434/v1.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # required but ignored
)

response = client.chat.completions.create(
    model="gemma4:26b",
    messages=[{"role": "user", "content": "Explain quicksort in 3 sentences"}]
)
print(response.choices[0].message.content)

Known Ollama Issues (April 2026)

I'm flagging these because they burned me:

Tool calling is broken in Ollama v0.20.0. The tool call parser crashes, and streaming drops tool calls entirely. If you need function calling, use vLLM instead for now.
If you're on an M-series Mac, don't set OLLAMA_FLASH_ATTENTION=1. The 31B model will hang once your prompt exceeds ~500 tokens. Ollama's defaults work fine without it.
Some general knowledge prompts cause the model to spit out an infinite stream of <unused24> tokens. Tokenizer bug. If it happens, stop generation and rephrase your prompt. A fix is being tracked in llama.cpp issue #21321.

Option 2: llama.cpp (More Control)

If you want raw performance, custom quantization, or you're deploying on hardware Ollama doesn't support well, llama.cpp gives you full control.

Build llama.cpp

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON  # or -DGGML_METAL=ON for Mac
cmake --build build --config Release -j$(nproc)

For CPU-only (no GPU acceleration):

cmake -B build
cmake --build build --config Release -j$(nproc)

Download a GGUF Model

Grab a pre-quantized model from Hugging Face. Unsloth provides well-tested GGUFs:

# 31B Q4_K_M — ~18 GB, good quality
huggingface-cli download unsloth/gemma-4-31B-it-GGUF \
  gemma-4-31B-it-Q4_K_M.gguf \
  --local-dir ./models

# 26B MoE Q4_K_M — ~14 GB
huggingface-cli download unsloth/gemma-4-26B-MoE-it-GGUF \
  gemma-4-26B-MoE-it-Q4_K_M.gguf \
  --local-dir ./models

Run Inference

./build/bin/llama-cli \
  -m ./models/gemma-4-31B-it-Q4_K_M.gguf \
  -p "Write a Rust function that implements a thread-safe LRU cache" \
  -n 512 \
  -ngl 99  # offload all layers to GPU

The -ngl 99 flag offloads all layers to your GPU. If you don't have enough VRAM, lower this number and llama.cpp will split layers between GPU and CPU. For the 31B Q4 model, I'd start with -ngl 40 on a 16 GB GPU and adjust from there.

Run as a Server

./build/bin/llama-server \
  -m ./models/gemma-4-31B-it-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -ngl 99 \
  -c 8192

This gives you an OpenAI-compatible API at http://localhost:8080/v1. Same client code as the Ollama example above, just change the port.

Performance Tips for llama.cpp

Gemma 4 advertises 256K context, but on consumer hardware you're realistically looking at ~20K tokens before memory pressure kills throughput. Qwen 3.5 27B manages ~190K on the same hardware, a 10x difference. Set -c conservatively. (Compression techniques like Google's TurboQuant may help here eventually.)
On Mac, use -DGGML_METAL=ON during build. Metal acceleration gives 2-3x speedup over CPU on M-series chips.
Increasing -b (batch size) can improve throughput for server workloads. I use -b 512 for my setup.

Option 3: vLLM (Production Serving)

vLLM is the right choice if you're serving Gemma 4 to multiple users or building it into a production pipeline. It handles batching, paged attention, and continous batching automatically.

Install and Run

The easiest path is Docker:

docker run --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  vllm/vllm-openai:latest \
  --model google/gemma-4-31b-it \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.9

Or install directly:

pip install vllm>=0.20.0
vllm serve google/gemma-4-31b-it \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.9

This starts an OpenAI-compatible API on port 8000.

The vLLM Performance Bug

Fair warning: there's a known performance issue with Gemma 4 on vLLM right now. The E4B model generates at only ~9 tokens/s on an RTX 4090. That's terrible for a 4B parameter model.

The root cause is Gemma 4's hybrid attention architecture. It uses 50 sliding-window layers plus 10 global attention layers, each with different head dimensions. vLLM's FlashAttention implementation can't handle this dual-dimension layout, so it falls back to a much slower Triton attention kernel.

The vLLM team is tracking this in issue #38887. Until it's fixed, you'll get better throughput from llama.cpp for single-user workloads. vLLM still wins when you're serving multiple concurrent users because of its batching, but the per-request latency is worse than it should be.

Multi-GPU Setup

For the 31B model on multiple GPUs:

vllm serve google/gemma-4-31b-it \
  --tensor-parallel-size 2 \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.9

Two 16 GB GPUs can serve the 31B model comfortably at BF16, which avoids any quantization quality loss.

Which Model Should You Pick?

After a week of running all four variants, here's my take:

Most people should start with the 26B MoE. It activates only 3.8B parameters but delivers 82.3% on GPQA and 77.1% on LiveCodeBench. It fits on a single 16 GB GPU at Q4. For coding assistance, general Q&A, and document analysis, it handles all of those well.

The 31B dense is worth the VRAM if you have it. The jump from 26B MoE to 31B dense is noticeable on hard math and complex multi-step reasoning. If you have 24 GB VRAM (RTX 3090/4090) or 32+ GB unified memory on a Mac, run this one.

I reach for the E4B when I want speed. Quick code completions, simple questions where I want sub-second responses. At ~3 GB VRAM, it runs comfortably alongside everything else on my machine.

The E2B? It runs on a Raspberry Pi, which is cool, but the quality gap to E4B is too large for anything beyond simple tasks.

Hardware Cheat Sheet

Here's what actually works based on my testing and community reports:

Hardware	Best Model	Quantization	Tokens/s
RTX 4090 (24 GB)	31B Dense	Q4_K_M	~35 t/s
RTX 3090 (24 GB)	31B Dense	Q4_K_M	~25 t/s
RTX 4070 Ti (16 GB)	26B MoE	Q4_K_M	~30 t/s
Mac M3 Pro (18 GB)	26B MoE	Q4_K_M	~15 t/s
Mac M2 Ultra (64 GB)	31B Dense	Q8_0	~20 t/s
RTX 3060 (12 GB)	E4B	Q8_0	~45 t/s
Raspberry Pi 5 (8 GB)	E2B	Q4	~3 t/s

These numbers are from llama.cpp with full GPU offloading. Ollama performance is within 5-10% of these.

Connecting Gemma 4 to Your Editor

Once you have a local Gemma 4 instance running (Ollama, llama.cpp server, or vLLM), you can use it as a coding assistant in most editors.

VS Code with Continue:

{
  "models": [
    {
      "title": "Gemma 4 26B Local",
      "provider": "ollama",
      "model": "gemma4:26b"
    }
  ]
}

Neovim with avante.nvim or codecompanion.nvim:

Point the OpenAI-compatible endpoint to your local server. Both plugins accept a custom base URL.

Any tool that supports OpenAI API:

Base URL: http://localhost:11434/v1  (Ollama)
Base URL: http://localhost:8080/v1  (llama.cpp)
Base URL: http://localhost:8000/v1  (vLLM)
API Key: "not-needed" (any string works)
Model: gemma4:26b

FAQ

How much VRAM do I need to run Gemma 4?

It depends on the model variant. The E2B runs in under 1.5 GB. The E4B needs about 3 GB at Q4. The 26B MoE needs ~14 GB at Q4. The 31B dense needs ~18 GB at Q4_K_M. On Macs, unified memory counts as VRAM, so a 16 GB MacBook can run the 26B MoE.

Can I run Gemma 4 on CPU only?

Yes, but it's slow. llama.cpp supports CPU inference natively. Expect 2-5 tokens per second for the 26B model on a modern desktop CPU. The E4B at ~8-12 tokens per second on CPU is usable for simple tasks.

Is Gemma 4 better than Llama 3 for coding?

On LiveCodeBench v6, Gemma 4 31B scores 80.0% versus Llama 3.3 70B's score in the low 60s. Gemma 4 is smaller and faster while producing better code. The 26B MoE at 77.1% also beats Llama 3.3 70B while using a fraction of the memory. And with Meta pivoting toward closed models with Muse Spark, Gemma 4 might be the best open alternative for a while.

Does Gemma 4 support vision and audio?

The E2B and E4B variants support multimodal input: images and audio. The larger 26B and 31B models are text-only. If you need local vision capabilities, the E4B is your best option in the Gemma 4 family.

Why is Gemma 4 tool calling broken in Ollama?

Gemma 4's hybrid attention architecture (mixing sliding-window and global attention layers with different head dimensions) exposed bugs in Ollama's tool call parser and streaming implementation. The Ollama team is working on a fix. For now, use vLLM or raw llama.cpp if you need function calling.

Bottom Line

I've tried every major open model release since Llama 2, and Gemma 4's 26B MoE is the first one where I stopped reaching for API keys during normal coding work. 14 GB of VRAM, no license restrictions, and benchmark scores that would've been frontier-tier eighteen months ago. The tooling has rough edges right now. Tool calling in Ollama is broken, vLLM has a performance regression, and Apple Silicon users need to dodge a Flash Attention bug. Those will get fixed. The model quality won't go backwards. Start with ollama run gemma4:26b and see where it gets you.