1m Tokens (& WebSocket)

Joseph Boone — Thu, 19 Mar 2026 21:32:25 +0000

Greetings readers, I made a threading engine with many optimizations (including ML) and WebSocket task controls per operation.

Even when computing a slow moving series like Leibniz PI at 1 million token executions the tasks all resolved as expected in ~200 seconds.

# ── LAYER 0: TERM TOKENS ──────────────────────────────────────────────────────
@task_token_guard(operation_type='pi_term', tags={'weight': 'light'})
def compute_pi_term(n: int) -> str:
    """
    Compute a single Leibniz term: (-1)^n / (2n + 1)
    Returns as string to preserve Decimal precision across token boundary.
    Light weight — 1,000,000 of these fire simultaneously.
    """
    getcontext().prec = DECIMAL_PRECISION
    sign = Decimal(-1) ** n
    term = sign / Decimal(2 * n + 1)
    return str(term)

# ── LAYER 1: CHUNK TOKENS ─────────────────────────────────────────────────────
@task_token_guard(operation_type='pi_chunk', tags={'weight': 'light'})
def sum_chunk(term_strings: List[str]) -> str:
    """
    Sum a batch of Leibniz terms.
    Receives resolved term strings from Layer 0 tokens.
    Light weight — 1,000 of these, each summing 1,000 terms.
    """
    getcontext().prec = DECIMAL_PRECISION
    total = sum(Decimal(t) for t in term_strings)
    return str(total)

# ── LAYER 2: PARTIAL TOKENS ───────────────────────────────────────────────────
@task_token_guard(operation_type='pi_partial', tags={'weight': 'medium'})
def sum_partial(chunk_strings: List[str]) -> str:
    """
    Sum a batch of chunk sums.
    Receives resolved chunk strings from Layer 1 tokens.
    Medium weight — 10 of these, each summing 100 chunks.
    """
    getcontext().prec = DECIMAL_PRECISION
    total = sum(Decimal(c) for c in chunk_strings)
    return str(total)

Leibniz is intentionally the slowest converging PI series, it needs ~10 million terms for 7 correct digits. That makes it a good stress test: maximum token volume, minimum mathematical payoff.

(Note: 64 workers with SMT enabled is only ~7% faster on a 7800X3D — more workers doesn't always mean more throughput, especially for micro-ops where execution port contention becomes the real ceiling.)

Tokens move through async admission and resolve on pinned workers, CPU heavy tasks stay on core 1, light tasks distribute across the rest. Failure nets, duplication safety, and WebSocket controls prevent runaway processes at the process level.

Take a look at the repo:

TavariAgent / Py-TokenGate

Experimental Python concurrency model using token-managed routing

TokenGate

Welcome to the TokenGate repository.

What it is:

A small experimental system for routing decorated synchronous functions
through a token-managed concurrency model. It is intended to operate as
its own concurrency workflow rather than alongside normal threading patterns.

What it is not:

It is not presented as production code.

Overview:

TokenGate is an exploration of token-managed concurrency: a
concept for coordinating async orchestration with thread-backed
work in a structured way.

This repository is a proof of concept, not a finished product.
It is experimental, still evolving, and shared in the spirit of
exploration.

If you'd like the fuller overview, please start here:

Proof of Concept

If anything here is useful, interesting, or sparks an
idea, that already makes this project worthwhile.

How to Use (Two Versions, Two Decorators)

Note: Do not attempt to decorate an async function.

The token decorator uses asyncio, but the decorated function itself should
…

View on GitHub

Threading Async Together

Joseph Boone — Mon, 16 Mar 2026 22:39:11 +0000

Hello readers,

I built a proof-of-concept application I call TokenGate. It’s a high performance async/threaded event bus, with control mechanisms designed to be extremely minimalist.

The core concept is to produce parallelism in concurrent operations through async token gathering and coordinated threading workers.

Here's what "TokenGate" uses to thread an operation:

# -- Python 3.12 -- #
from token_system import task_token_guard
from operations_coordinator import OperationsCoordinator

# 1. Decorated standard synchronous function for threading
@task_token_guard(operation_type='string_ops', tags={'weight': 'light'})
def string_operation_task(task_data):
    # This function is now threaded
    return result

# 2. Starts the coordinator (through a running loop)
coordinator = OperationsCoordinator()
coordinator.start()

# 3. finally or an exception stops on close
coordinator.stop()

Task tokens are generated by using a wrapped decorator.

Here's some test results on operations in a "release mechanism" that dispatches batches of mixed tasks incrementally:

CONCURRENCY BURST: Medium x8 | release 1464 (8 tasks)
======================================================================
  Submit spread (barrier jitter): 0.19ms
  Overall wall-clock:             0.009045s
  Min task duration:              0.007818s
  Max task duration:              0.008432s
  Mean task duration:             0.008148s
  Stdev (clustering indicator):   0.000218s

  Duration per task (tight clustering = true concurrency):
    Task 00: 0.007928s  
    Task 01: 0.008000s  
    Task 02: 0.008136s  
    Task 03: 0.008209s  
    Task 04: 0.008432s  
    Task 05: 0.008300s  
    Task 06: 0.008362s  
    Task 07: 0.007818s  

  Serial estimate (sum):  0.065186s
  Actual wall-clock:      0.009045s
  Concurrency ratio:      7.21x  (concurrent)

CONCURRENCY BURST [Medium x8 | release 1464] PASSED
======================================================================
CONCURRENCY WINDOW: Sustained mixed releases (30s)
======================================================================
  Releases:                       1484
  Total tasks:                    11872
  Overall wall-clock:             30.070291s
  Min task duration:              0.001157s
  Max task duration:              0.105874s
  Mean task duration:             0.014970s
  Stdev (clustering indicator):   0.025983s

  Serial estimate (sum):          177.728067s
  Actual wall-clock:              30.070291s
  Sustained concurrency ratio:    5.91x  (concurrent)

CONCURRENCY WINDOW [Sustained mixed releases (30s)] PASSED

CONCURRENCY SUITE COMPLETE.

(Concurrency ratios of up to 7.21x were witnessed on an 8 core CPU with ~32 dynamic workers in ideal conditions, which is roughly 90% of the 8x concurrent operation ceiling.)

I've tested a wide variety of normally threaded operations with result delivery as expected.

It's still a just a proof, however I've used it in various side-projects with good results.

For anyone interested here's my project on GitHub (with proofs):

Repo link - https://github.com/TavariAgent/Py-TokenGate

DEV Community: Joseph Boone