Mustafa ERBAY

Posted on Sep 10 Originally published at mustafaerbay.com.tr

What Free-Threaded Python Was Waiting For: abi3t

#python #performance #operations #sistemmimarisi

The idea of removing Python's global interpreter lock has been discussed for so long that it stopped being news and became folklore. Then it quietly happened: with Python 3.14 the free-threaded build — the one without the GIL — stopped being experimental and became an officially supported option. And almost nothing changed.

The reason nothing changed is not the interpreter. Most of the interpreter work was already done. The jam was in packaging: every C extension that wanted to run on the GIL-free interpreter had to be compiled and shipped as a separate wheel, and that matrix was heavier than the ecosystem was willing to carry. Python 3.15, due on 1 October 2026 — rc2 is what we have as I write this — aims at exactly that lock: abi3t.

This post is not the GIL's obituary. It is an answer to "when should I actually move," plus the story of a trap almost everyone walks past.

I measured it first: same machine, same version, two interpreters

I don't like taking numbers from someone else's slide deck. So I installed two interpreters with uv on the M4 Pro in front of me (10 performance + 4 efficiency cores, macOS 26.6.2): CPython 3.14.4 and its free-threaded twin, 3.14.4t. Same version, same compiler, same machine — the GIL is the only variable.

uv python install 3.14.4 3.14.4t

I pinned the patch version deliberately: write 3.14 and uv installs whichever patch its version list knows about, and the whole point of the comparison is that both interpreters sit on the same patch.

The workload is deliberately dumb: counting primes in pure Python. It doesn't represent the real world, it represents the GIL — because it never releases the lock. Each number is the best of three runs.

Threads	3.14.4 (with GIL)	3.14.4t (free-threaded)	Gain
1	0.393 s	0.430 s	0.91×
2	0.778 s	0.453 s	1.72×
4	1.568 s	0.617 s	2.54×
8	3.098 s	0.724 s	4.28×

Look at the GIL column: as the thread count doubles, so does the wall time. You handed eight cores eight jobs and they all queued. We knew this already, but watching it in your own terminal lands differently.

The free-threaded column is not linear either. Eight threads bought me 4.28×, not 8×. I didn't chase the cause — efficiency cores, memory bandwidth and the cost of the newly freed reference counting are all in play at once, and I won't assign a cause I didn't measure. The direction, though, is not in doubt: for the first time, adding threads actually finishes work.

Then there is that first row, the one where the "gain" is 0.91. On a single thread the free-threaded build is slower. I measured 9.4%. Comparing that against the documented figure calls for some honesty: the free-threading howto gives the pyperformance average as "about 1% on macOS aarch64 to 8% on x86-64 Linux systems" — roughly 1% for the exact platform I measured on. Mine is nine times that. Not a contradiction, a difference in definition: pyperformance measures a mixed application profile, while my loop is pure Python that never releases the lock. The documented number is the penalty for an average application; mine is the penalty for the worst case. The 3.14 release notes putting it at "roughly 5-10%" by platform lands somewhere between the two. Memory usage goes up as well.

So this is a trade, not an upgrade. And buying the bad half of that trade by accident is far easier than you'd think.

The worst configuration: pay the tax, skip the benefit

The free-threaded build can turn the GIL back on at runtime, via PYTHON_GIL=1 or -X gil=1. (Don't drop the value; a bare -X gil doesn't work.) I ran the same benchmark that way too:

Threads	3.14.4 (with GIL)	3.14.4t + `PYTHON_GIL=1`
1	0.393 s	0.438 s
4	1.568 s	1.708 s
8	3.098 s	3.406 s

Slower than the normal build on every row. No parallelism, but the penalty is still charged. It is the most pointless cell you can land in: you pay for the free-threaded build and receive the behaviour of the GIL build.

Nobody does this on purpose. The problem is that you don't have to do it on purpose.

One legacy extension is enough

When a C extension that hasn't declared itself free-thread safe gets imported, the interpreter re-enables the GIL by itself. The docs cover this in a single sentence, and I wanted to watch it happen, so I wrote a fifteen-line extension in the old style that declares nothing:

static PyMethodDef Metotlar[] = {
    {"selam", selam, METH_NOARGS, "selam"},
    {NULL, NULL, 0, NULL}
};
/* Note: no Py_mod_gil slot -> the interpreter treats this as unsafe */
static struct PyModuleDef modul = {
    PyModuleDef_HEAD_INIT, "eski", NULL, -1, Metotlar
};
PyMODINIT_FUNC PyInit_eski(void) { return PyModule_Create(&modul); }

I compiled it and imported it. I captured the output to a file, so buffering puts the warning line first:

RuntimeWarning: The global interpreter lock (GIL) has been enabled to load
module 'eski', which has not declared that it can run safely without the GIL.
To override this behavior and keep the GIL disabled (at your own risk),
run with PYTHON_GIL=0 or -Xgil=0.

gil before import: False
hello from the old world
gil after import: True

Three lines apart, False turned into True. One RuntimeWarning. A warning, not an error. Your application keeps running, your tests pass, the container comes up — and you are sitting in the worst cell of the table above. It doesn't even have to be your code that does it; a neglected package four levels down your dependency tree will do just fine.

⚠️ Don't let a silent fallback stay silent

If you move to the free-threaded build, export sys._is_gil_enabled() at startup as a log line or a metric. That one line makes the "why aren't we seeing any speedup" meeting unnecessary before it is ever scheduled.

For the record: numpy 2.5.3 and PyYAML's C-backed CLoader did not re-enable the GIL in the same environment. The visible half of the ecosystem is ready. The invisible half was always the problem.

The real bottleneck was the wheel matrix

Everything above was already true in 3.13 and 3.14. So why did nobody move?

Because if you publish a C extension, free-threaded support added one more column to your build matrix. Python's stable ABI, abi3, has solved this for years: compile once, load on every later 3.x. But abi3 did not cover the free-threaded build. Result: one wheel in the abi3 world, one wheel per version in the free-threaded world.

PEP 779 — the document that made the free-threaded build officially supported — names the adjacent deadlock without flinching: a "chicken and egg problem." You can't tell whether the Stable ABI is good enough without packages using it, and once problems surface you can no longer remove things from a stable ABI. Packagers waited on the interpreter; the interpreter waited on packagers.

PEP 803 is what broke it.

What abi3t actually changes

abi3t is a new variant of the stable ABI defined for free-threaded builds. Its status is Final and it targets 3.15. Mechanically, three things happen:

PyObject becomes an opaque box. For an extension targeting abi3t, structures like PyObject, PyVarObject and PyModuleDef become incomplete types, and macros such as PyObject_HEAD go away. You can't reach into fields directly; you go through functions like Py_TYPE(). That is the price of letting the interpreter change its object layout. This is also where the real porting pain lives: instead of making PyObject part of the instance struct, you move to PEP 697's negative basicsize and PyObject_GetTypeData().

Module entry changes. Instead of the classic PyInit_ function you use the PyModExport_* hook introduced by PEP 793. As the 3.15 release notes add, the new PySlot structure from PEP 820 comes along with it. Declaring a static PyModuleDef variable is over.

One file serves both worlds. This is the most important part and the most commonly misread. PEP 803 says that on systems where Stable ABI extension filenames end in .abi3.so, "all builds of CPython – GIL-enabled and free-threaded – will load extensions with the abi3t tag." So .abi3t.so is not a file specific to the GIL-free build; it is the file that works on both. Packages supporting both ABIs declare it with the compound tag abi3.abi3t — for example cp315-abi3.abi3t. (If both are present, GIL-enabled builds prefer "their" .abi3.so.)

For packages that can move to abi3t, the columns merge. That was the missing piece all along.

It isn't free, of course. The 3.15 release notes state plainly that moving to abi3t "usually requires some non-trivial changes to the source code." The ecosystem will not be ready tomorrow morning; but this is now a one-time investment rather than a chore repeated every release.

And 1 October is also a breaking date

If you've read this far as good news, let's turn the coin over — because this is what you'll hit first in production.

The free-threaded build in 3.15, unlike 3.14, no longer loads extensions tagged abi3. PEP 803 is explicit: in free-threaded builds PyModuleDef_Init() will detect extensions using the non-free-threading Stable ABI, emit an informative message when one is loaded, and raise an exception. Not a warning. An exception.

See the difference? On 3.14t an old abi3 extension still loaded — silently re-enabling the GIL, but loading. On 3.15t the same extension blows up on the import line.

The road isn't closed for those that can't move: the 3.15 release notes say extensions that cannot switch to abi3t should keep building for the existing abi3 and the free-threading version-specific ABI (cp315t) separately. For those packages the matrix stays exactly as it was.

The short version: 1 October is a lock opening, and a break. That's the reason to redo your dependency inventory after that date — not curiosity.

There is a quieter signal too: in 3.15 the PyGILState_Ensure() / PyGILState_Release() family was soft-deprecated. No removal is planned, but no new APIs will be added either. When a C API family is handed a pension, the message is clear enough.

Is the toolchain ready?

A specification is lovely, but if pip install doesn't recognise the tag, none of it matters. So I checked.

The change adding abi3.abi3t support to the packaging library was merged on 7 April 2026, in time for the 26.1 release. pip picks it up through its vendored copy of packaging. In my own environment:

pip 26.2.1 | vendored packaging 26.2
lines mentioning abi3t: 15

uv is in the same position: its 0.11.3 release notes carry "Implement support for PEP 803" and "Emit abi3t tags for every abi3 version," dated 1 April 2026.

The install side is ready. Next up is extension authors making those "non-trivial changes."

There's also something arriving quietly: according to the 3.15.0rc2 announcement, the official macOS installers now ship free-threading support by default. Which means a GIL-free interpreter is about to be sitting on a lot of developer machines, whether or not anyone asked for one.

There is no phase III, and that's deliberate

Let me be blunt, because the opposite is circulating: the free-threaded build is not Python's default, and there is no announced timeline for when it will be.

PEP 703 defined three phases: experimental, officially supported but optional, and default. It was PEP 779 that tied them to versions — phase I started during 3.13, phase II landed in 3.14. The same document set criteria for the second phase — a 15% ceiling on the performance penalty, 20% on memory — and set none at all for the third. In its own words those are requirements "for entering phase II only"; the decision to make free-threading the default is expected to revolve around "community support, willingness, and showing clear benefit."

Translated: the decision is social, not technical. And social decisions don't run on schedules.

The practical consequence is that moving to free-threaded Python in 2026 means making a deliberate choice rather than drifting with the current. If you haven't measured a gain, don't.

Will your code break without the GIL?

Don't trust anyone who answers "no," and the ones who say "yes" are overselling it. The honest answer: the GIL spent years accidentally providing a safety it never promised you, and that accident is now ending.

The howto underlines two concrete items. First, it is not safe to access frame.f_locals on a frame that is currently executing in another thread, and doing so may crash the interpreter — one for the people writing debuggers and profilers. Second, accessing the same iterator object from multiple threads concurrently is generally not thread-safe, and threads may see duplicate or missing elements.

And this may be the most practical advice in the whole post: rather than relying on the internal locks of built-in types, use threading.Lock or another synchronization primitive. Every piece of code built on years of "appending to a list is atomic" folklore is the intended audience of that sentence.

If it were me, I'd start the migration with a single question: where does this application keep shared mutable state? If the answer doesn't fit on one page, your migration problem isn't performance. It's design.

A decision frame

The critical box here is F. Every other question can be answered at your desk; that one you cannot know without measuring.

Checklist

Before you let the free-threaded build anywhere near production:

Measure first. Compare the same version with and without the GIL, on your workload and your hardware. My table is correct for my workload, not yours.
Export sys._is_gil_enabled(). Log it once at startup or publish it as a metric. A silent fallback is the most expensive class of bug.
Add a CI run with -W error::RuntimeWarning. That turns an extension-triggered GIL fallback from a warning into a red test.
Inventory your dependencies. List every package containing a C extension: do they publish free-threaded wheels, are they maintained, when did the last release ship?
Check your memory budget. The free-threaded build typically uses more memory; for services sitting flush against a container limit, that is an OOM problem before it is ever a performance problem.
Lock your shared state. Find the code leaning on built-in types' internal locks and move it to explicit synchronization.
Don't silence the warning with PYTHON_GIL=0. The warning text itself offers that escape "at your own risk." An extension not declaring itself safe is not the same as an extension being unsafe — but suppressing the difference without knowing it removes the only layer of protection you had.
Test your toolchain separately. Profilers, debuggers and APM agents are C extensions too; check their free-threaded support independently of your application. Anything built on greenlet is its own chapter.
Derive thread counts from the cgroup quota in containers. os.cpu_count() tells you the machine's cores, not the CPU allotted to your container. Without the GIL, that gap means you will, for the first time, genuinely oversubscribe.
Put 3.15 on the calendar — as a break, not an opening. The 1 October 2026 final is when the ecosystem can start publishing abi3.abi3t wheels; it is also the day the free-threaded build starts rejecting abi3 extensions. Redo your inventory after that date.

Conclusion

The part of this story I find most interesting isn't technical. Removing the GIL was discussed for years as an interpreter problem; the actual resistance showed up in the distribution layer, the place nobody gives conference talks about. The interpreter was ready in 3.14. The ecosystem could not be ready until an ABI tag existed.

I see this pattern often in infrastructure: you make the engine bigger, then discover the bottleneck was in the gearbox. When I wrote about moving from Python to Rust or Go, I treated the GIL as a fixed law of nature. It isn't anymore — but the distance between "it isn't anymore" and "your problem is solved" is exactly one wheel matrix wide.

The practical summary: measure today, rescan your dependencies after 1 October, and ask sys._is_gil_enabled() now and then. Don't assume the door is open just because the lock is gone. Somewhere deep in your dependency tree, a C extension whose name you don't know may be closing it on your behalf — politely, leaving a single line of warning.

DEV Community