DEV Community: Joseph Boone

A Secret I Will Never Reveal

Joseph Boone — Fri, 15 May 2026 02:17:23 +0000

My Secret:

I may or may not know the exact way to release the GIL on python 3.10+

I might also know how to recreate this minimally and easily without C extensions

It's not a magic trick, or some weird off beat code that doesn't work.

It functional, typed, code. What it really is I'll never actually reveal directly. Because I already did by releasing TokenGate, the abstraction itself is simple readable and traceable in that codebase. (Once you see it, you know why.)

When I first unlocked the GIL I tested it and traced the exact requirements. You can do it with or without serialization (serializing makes it easier for obvious reasons once you know but the physics doesn't change). Threading can output ~45x operations across 32 workers on my system in varied normal task distributions. Take a look at these results:

Wave   Tokens    OK      Fail    Time      Tok/s     Lat(ms)   Conc    Overlap
1      4         4       0       0.003s    1386.2    0.721     1.00×   1.44×
2      8         8       0       0.003s    2391.2    0.418     1.72×   2.48×
3      16        16      0       0.006s    2744.8    0.364     1.98×   4.82×
4      32        32      0       0.011s    2812.7    0.356     2.03×   11.32×
5      64        64      0       0.022s    2880.0    0.347     2.08×   22.01×
6      128       128     0       0.044s    2907.6    0.344     2.10×   29.78×
7      256       256     0       0.090s    2846.8    0.351     2.05×   37.98×
8      512       512     0       0.182s    2811.5    0.356     2.03×   41.81×
9      1024      1024    0       0.364s    2813.9    0.355     2.03×   44.18× <-
10     2048      2048    0       0.775s    2644.3    0.378     1.91×   44.86× <-
11     4096      4096    0       1.454s    2816.3    0.355     2.03×   38.34×
12     8192      8192    0       2.905s    2819.9    0.355     2.03×   32.64×
13     16384     16384   0       5.925s    2765.0    0.362     1.99×   27.92×
14     32768     32768   0       12.102s   2707.7    0.369     1.95×   24.96× <-
15     65536     65536   0       23.494s   2789.5    0.358     2.01×   24.21× <- 

Wave 9+10, 1024-2048 tasks at any given moment = 40+ times faster

Wave 14 + 15, Massive overload and still holding GIL free status.

TOTAL  131,068   131,068  0      89.091s
Avg latency : 0.386 ms/token
Peak overlap: 44.86×

All tokens submitted instantly per-batch.

I'll just say this: There are domains, there is physics, the CPU must obey these. So think about that and read TokenGate.

(Bonus hint: There is exactly 3 components required to unlock the GIL.)

Clarifications:

Why is my parallelism higher than my worker count:

It's a matter of perspective around synchronous and concurrent processes. Coders often don't realize parallelism in a varied application is not lateral. When 1 operation costs 3 seconds but 36 operations just occurred in the same period, parallelism is technically a higher value then the workers because the unit time to completion is not a line. It's a curve along the edge of time to complete all tasks.

When Stability Improves Performance (Threading)

Joseph Boone — Sat, 09 May 2026 17:33:56 +0000

The common assumption in concurrent systems is that stability and performance pull in opposite directions. You add safety mechanisms, locks, routing constraints, and you pay for them in throughput. This post is about a case where that assumption turned out to be wrong.

The Premise

TokenGate is a token-managed concurrency system. Decorated functions return tokens instead of executing immediately. Those tokens are admitted through a wrapped decorator, routed to per-core mailboxes by weight class, and executed on thread pool workers.

This system also includes a successful separation over async coordination and threaded execution, which is a common source of complexity in concurrent systems.

TokenGate aims to ease this by using tokens as a bridge between the async event loop and a thread pool. The async event loop manages the routing and coordination of tokens, while the thread pool handles the execution. The routing model assigns tokens to cores by weight and storage speed.

Ops are handled in separation making them distinct at every stage.

The weight classification sets the core viability of a task. This is useful
to allow "front-back-fill" scheduling patterns where light work occupies the
latter cores while heavy work utilizes the first core and falls back toward
the others as load increases.

Each weight class has a defined core range:

HEAVY → All Cores
MEDIUM → Core 2 +
LIGHT → Core 3 +

Within those ranges, a staggered position counter distributes tokens across workers in FIFO with the ability to interleave retry tokens.

What Was Built

Sticky Token Registry

Tokens are marked when seen to have related args: Pinning tokens with matching (operation_type, args) keys to the core that first receives them keeping data locality clean.

When a token arrives, sticky_registry.mark() creates the sticky anchor that automatically sees these related parts.

@task_token_guard(
    operation_type="my_op",
    tags={"weight": "medium", "sticky_anchor": "my_domain"},
)
def my_operation(n: int) -> int:
    ...

This is reactive it catches collisions as they arrive. It handles the case where two tokens with identical logical identity are submitted concurrently and would otherwise chain across other core domains.

Hash Conductor

The second layer is proactive. Instead of waiting to see if the collision would happen, the conductor anchors an entire call chain to a domain before any child token is even routed, it's a heavier pattern but is a secure way to ensure all related data is interpreted on the same core.

When a lead token is decorated with external_calls, a SHA-256 seed is
generated from the token ID and the call list:

seed = SHA-256( token_id + ":" + freeze(external_calls) )

Full 64-character hex digest. The token ID is included so two leads with
identical call lists still get independent domains.

That seed is pinned to whichever core the lead lands on. Any token spawned
during the lead's execution inherits the seed and is routed to the same core automatically. No configuration needed inside call-sites. No explicit passing. The seed propagates through a thread-local set in the executor thread before the lead function runs while in token routing.

The most important thing to consider is that workers within the core domain may still execute the tokens in parallel. Respecting the workers staggered routing. It's not a "nerf" to the capability of the system - while operating under saturated load conditions noticeable performance gains were observed.

@task_token_guard(
    operation_type="lead_op",
    tags={"weight": "medium", "external_calls": ["child_op"]},
)
def lead_operation(n: int) -> list:
    # These children inherit the seed and land on the same core
    return [child_op(n + i) for i in range(4)]

The pending lead operations count starts at 1 and increments for each subsequent external call at creation time, and decrements on every completion. When it reaches zero the seed is released.

The Benchmark

15 doubling waves. 131,068 tokens total.

Wave   Tokens    Tok/s     Lat(ms)   Overlap
1      4         1386.2    0.721     1.44×
2      8         2391.2    0.418     2.48×
3      16        2744.8    0.364     4.82×
4      32        2812.7    0.356     11.32×
5      64        2880.0    0.347     22.01×
6      128       2907.6    0.344     29.78×
7      256       2846.8    0.351     37.98×
8      512       2811.5    0.356     41.81×
9      1024      2813.9    0.355     44.18×
10     2048      2644.3    0.378     44.86× ← peak overlap
11     4096      2816.3    0.355     38.34×
12     8192      2819.9    0.355     32.64×
13     16384     2765.0    0.362     27.92× ← better sustained performance
14     32768     2707.7    0.369     24.96× ←
15     65536     2789.5    0.358     24.21× ←

Zero failures. Avg latency 0.386ms/token.

The previous ceiling was around 17× overlap after saturation. This
run hit 44.86× at wave 10 and descended gracefully from there.
Latency moved 0.04ms across the entire run from wave 3 to wave 15.

Why Stability Produced More Concurrency

The 17× previous ceiling wasn't a capacity ceiling. It was a friction ceiling.

At that overlap level the old routing was generating cross-domain
traffic. Related tokens landing on different cores meant cache lines being
written back and refilled across the interconnect. The scheduler was spending a growing proportion of its time on coordination rather than execution.

Domain anchoring removed the wasted power. Tokens that belong together stay together. The cache lines loaded for a lead token's data are still warm when its children execute on the same core. The cross-core traffic that was expanding with overlap now barely exists for conducted chains. The scheduler has more execution headroom compared to coordination cost. The overlap ceiling rises.

This is why the overlap column reaches over 20× and holds flat latency while doing it. The system isn't working harder. It's working cleaner, scaling better.

Calling Production Crews

If you run concurrent Python workloads task queues, async pipelines, anything with related operations that currently route freely try hooking some ops up, I'd like to know what you see looking at this and any poking at my work is helpful.

As a self taught developer I'm open to criticism and would love to learn from some trained or learned folks.

Registering calls for anchoring

The sticky registry and hash conductor are opt-in. Existing code routes normally.

Hashed domain anchoring and sticky tokens are aiming to be the first "production ready features" for this.

The cases I'm most interested in:

Anything that hits an unexpected concurrency ceiling without obvious cause.

The repo is public. Issues and observations welcome. GitHub

Leave some feedback! Tavari

TokenGate represents my journey through hobbyist coding for nearly 4000 hours, times change, I'm now opening up as business. Stay tuned for the future of Tavari.

What Code is About (IMO)

Joseph Boone — Wed, 06 May 2026 19:38:58 +0000

I want to talk about what code is about and what it did to my life and mind, not as a career or hobby, but as a way of thinking that I didn't expect and won't let go of.

I've been building TokenGate and learned what I know from many prototype applications over 1000's of hours of code. I started working with LLMs - mostly Claude Sonnet, to help me understand things I didn't know, debug things I couldn't see, and make decisions about architecture I had no formal training for. And somewhere in that process, something shifted.

Code is not a language. It's a lens.

People talk about code like it's a skill, like typing or driving, or something you learn. But that framing misses something important. Code is closer to a way of perceiving. Once it starts working on you, you start seeing structure. Systems, dependencies, state changes, feedback loops, and contracts.

You start asking "what are the rules here?" about things that have nothing to do with a terminal, things that don't relate to feedback loops, things that are part of our life start to gain structure.

I'd call this structural truthiness. Not truth like a fact, but truth like: does this actually hold together? Is the logic real, or am I just convincing myself? Code is ruthless about this in a way that most thinking isn't. It either runs or it doesn't. The compiler doesn't care about your vibes.

That ruthlessness, weirdly, becomes freeing. Because once you trust the structure, you can go anywhere inside it. I've found myself reading things about physics, economics, biology, Lagrangian formulas - fields I didn't learn explicitly but got to "taste" the essence of because of code while actually following the logic because I learned to look for it. Code widened something in me that I had lost as a person: "Perspective".

Working with AI made it weirder and better

Working with an LLM like Sonnet changes the dynamic in an interesting way. You're not just writing code, you're describing what you want, and then negotiating with something that can tell you if your description is coherent. If you're fuzzy, the output is fuzzy. If you're precise, it's surgical.

That forced me to get better at knowing what I actually wanted. Not what I thought I wanted. Not what sounded right. What I could actually articulate as an intention with a structure behind it. That's a skill that transfers everywhere.

The thing that actually matters

Here's the point I keep coming back to:

A calculator computes what must be. A computer equates what you want.

A calculator is deterministic and bounded. You give it inputs, it gives you the only possible output. There's no room for intention, only execution of fixed rules. A computer is different. A computer runs your model of something. It doesn't know what you're trying to do. It just faithfully executes whatever you describe which means the quality of what comes out is a direct reflection of the quality of your thinking going in.

That's not a technical distinction. That's a philosophical one. Code is the medium where intention becomes testable. Where you stop saying "I think this is how it works" and start saying "let's find out". You're not looking up an answer, you're building a small version of your understanding and seeing if it holds.

For some people - myself included, that's the clearest and most satisfying way to think. Not because it's easier, but because it's honest in a way that feels like you're challenging your ability to see the truth in the structure.

If you're on the fence about going deeper into code: Don't think of it as learning a tool. Think of it as picking up a new way to think and challenge what you know.

It's worth it.