DEV Community: Ahmed Trading

Building Bots That Survive a Breach: Rate Guards, Kill Switches, and October 29

Ahmed Trading — Sun, 01 Nov 2026 18:39:53 +0000

Building Bots That Survive a Breach: Rate Guards, Kill Switches, and October 29

When a platform gets breached, most trading bots fail in one of three boring ways: they keep hammering endpoints that are being drained, they hold session keys nobody can revoke, or they have no way to stop themselves. October 29, 2026 — when a hijacked script drained $2.73M from 737 Polymarket accounts and Binance lost $6.3B the same day — was a live drill for all three failure modes.

This is the bot-survivability checklist I run after every incident. It assumes you already have a working bot; if you don't, the Polymarket bot build guide covers the foundation.

1. Rate guards: your bot is a client, treat it like one

After a breach, platforms throttle, pause, or reshuffle endpoints while they triage. A bot without request budgeting hammers through the throttle window and gets its key flagged — sometimes permanently, sometimes along with the account.

Concretely: every outbound call goes through a token-bucket limiter keyed by endpoint class, with exponential backoff on 429/5xx and a circuit breaker that trips after N consecutive failures. My rate-guard reference implementation shows the pattern; the same post covers per-endpoint budgets for the current API.

2. Session keys with expiry, never raw keys

The wallets drained on October 29 were not bot wallets — but the lesson transfers. A bot holding your raw private key is a bot that can lose everything your key can lose. A bot holding a scoped session key with an expiry is a bot that can lose a bounded amount for a bounded time.

Issue session keys with the narrowest scope your strategy allows, rotate them on a schedule, and rehearse revocation before you need it. The session keys guide covers issuance and rotation on Polymarket specifically.

3. A kill switch you can hit from your phone

Every trading bot needs a component that stops it — not a config change, not a redeploy, a switch. Mine is a dead-simple endpoint check: the bot polls a local flag file every loop, and if the flag says "halt", it cancels open orders, closes nothing (closing is a decision, not a reflex), and goes idle.

On October 29, the bots that survived were the ones whose operators could halt them in one action from a phone, without SSH-ing into anything. If your kill switch requires infrastructure access during an incident, you don't have one — you have a hope.

4. Breach-day behavior is different from normal behavior

The checklist above is table stakes. Breach-day behavior is a separate mode, and it should be explicit in your code:

On circuit-breaker trip: stop quoting, cancel resting orders, hold positions. Don't "wait it out" with live orders in a book that's being repriced by panic.
On session-key anomaly (any signature request you didn't initiate): halt, revoke, rotate. Assume compromise first, investigate after.
On platform announcement of an incident: assume the announcement lags reality by minutes to hours. Your risk window starts at first anomaly, not first announcement.

The first-24-hours playbook is written for humans with drained wallets, but the ordering logic — move what survives first, revoke second, document third — maps directly onto bot priorities.

5. The audit that costs ten minutes

After any incident: every approval on every wallet the bot touches, every session key, every standing allowance — reviewed and trimmed. Unlimited approvals don't survive my audits anymore, and neither should yours. It's the same revoke discipline human traders need, applied to machine traders.

October 29 didn't create new risks. It priced the ones everyone was already carrying.

Background: the full October 29 double-breach analysis, and API authentication mechanics if you're reviewing your key handling. General reading on the Binance side: 2026 Binance hack.

Exponential Backoff and Request Budgets for Trading APIs in Python

Ahmed Trading — Tue, 20 Oct 2026 16:33:28 +0000

Trading APIs fail in a predictable rhythm: fine for hours, then a burst of 429s exactly when your bot is busiest. The fix is not "retry harder" — it is a request budget with jittered exponential backoff, enforced in one place.

This post shows the minimal pattern I use for Polymarket CLOB/Data API work: a token bucket for steady-state pacing, backoff with jitter for bursts, and a hard ceiling so one bad hour does not take the scanner offline all day. The design rationale and observed limits are documented separately: https://brainresearchhub.net/research/sdk/rate-limit-design-patterns

The three components

Token bucket — enforces average rate with burst allowance, centrally.
Backoff with jitter — handles 429s without re-synchronizing your workers.
Retry cap + kill switch hook — fails loud instead of hammering forever.

import time, random, threading

class TokenBucket:
    """rate = tokens per second; capacity = max burst."""
    def __init__(self, rate: float, capacity: float):
        self.rate, self.capacity = rate, capacity
        self.tokens, self.ts = capacity, time.monotonic()
        self.lock = threading.Lock()

    def take(self, n: float = 1.0) -> float:
        with self.lock:
            now = time.monotonic()
            self.tokens = min(self.capacity, self.tokens + (now - self.ts) * self.rate)
            self.ts = now
            if self.tokens >= n:
                self.tokens -= n
                return 0.0
            return (n - self.tokens) / self.rate


def call(fn, bucket: TokenBucket, max_retry: int = 6,
         base: float = 0.5, cap: float = 45.0):
    """fn() returns an object with .status_code and .headers (requests-style)."""
    for attempt in range(max_retry + 1):
        wait = bucket.take()
        if wait > 0:
            time.sleep(wait)
        resp = fn()
        if resp.status_code != 429:
            return resp
        if attempt == max_retry:
            raise RuntimeError(f"budget exhausted after {max_retry} retries")
        ra = resp.headers.get("Retry-After")
        delay = float(ra) if ra else min(cap, base * 2 ** (attempt + 1))
        delay += random.uniform(0, delay * 0.2)      # jitter: de-synchronize workers
        time.sleep(delay)

Wiring it in

import requests

bucket = TokenBucket(rate=5.0, capacity=20)   # 5 rps sustained, 20 burst

def get_markets(slug: str):
    return call(
        lambda: requests.get(
            f"https://data-api.polymarket.com/markets?slug={slug}"),
        bucket,
    )

Rules I follow that the code cannot enforce:

One bucket per process, not per module. Two components each "within limits" is how you exceed limits.
Never retry non-idempotent writes blind. An order POST that times out may have succeeded; reconcile from the order channel instead of re-posting.
Subscribe instead of polling. The market WebSocket channel kills most of your REST load; the stream architecture is covered here: https://brainresearchhub.net/research/sdk/websocket-stream-architecture
The retry loop calls the kill switch, not the logger. When retries exhaust, halt order placement: https://brainresearchhub.net/research/sdk/kill-switch-design

Why jitter is non-negotiable

Without jitter, all your workers that hit the 429 at the same moment retry on the same beat, re-synchronize, and trip the limit again. Uniform jitter in delay * (1 ± 0.2) breaks the lockstep. It looks trivial and it halves your ban windows in practice.

Background reading: the generic mechanics of rate limiting (token bucket, leaky bucket) are well covered on Wikipedia: https://en.wikipedia.org/wiki/Rate_limiting — and for a regulated-venue comparison, the Kalshi API guide: https://brainresearchhub.net/research/sdk/kalshi-api-guide

Educational purposes only. Not financial advice.

Building a Polymarket Perps Trading Bot: Architecture Guide (October 2026)

Ahmed Trading — Sun, 11 Oct 2026 11:04:47 +0000

When I came to America from Dubai with nothing but a suitcase and a plan, I didn't know those plans would lead through MIT, Jump Trading, and Alameda Research. But the engineering patterns I learned along the way are exactly what you need to build a production-grade trading bot for Polymarket Perps. This guide is the architecture I wish someone had written down in October 2026.

The big picture

A perps trading bot has four layers, and every layer deserves its own process:

Data ingestion — subscribe to websocket feeds for orderbook, trades, and index prices. Never poll REST in a loop for data you can stream.
Strategy engine — takes normalized market state, decides actions. Pure function in, orders out. No hidden state.
Execution — signs and submits orders, handles partial fills and retries with idempotency keys.
Risk management — position limits, drawdown limits, kill switch. This layer overrides everything else, always.

When I was at Jump, the desk principle was simple: the risk layer must be able to cancel every order in one operation, without asking the strategy engine for permission. Build that first.

Data ingestion details

For Polymarket Perps you want:

a persistent websocket connection per market group, with sequence numbers checked on every message
a REST fallback for snapshots on reconnect
a clock-sync routine so you can reason about latency honestly

Reconnect logic is where hobby bots die. Exponential backoff, snapshot-then-delta recovery, and a watchdog that restarts the whole ingestion layer if staleness exceeds your threshold.

Strategy engine: keep it boring

Market making on perps is quoting two-sided prices around a fair value you compute from the underlying prediction market. Your fair value model can start simple: index price plus a skew adjustment from orderbook imbalance.

The complexity belongs in inventory management, not in signals. Track net position, skew quotes away from inventory extremes, and cap notional per market. A boring strategy with good inventory control beats a clever strategy with none.

Execution: idempotency or bust

Every order submission carries a client-generated idempotency key. On timeout, query order state before retrying — never blind-retry submissions. Partial fills reduce remaining size atomically.

Rate limits are part of the API surface. Budget your calls like memory in embedded systems: fixed allocation, no leaks.

Risk management is the product

Max position per market — absolute notional cap.
Max drawdown per day — when hit, flatten and stop until next session, automatically.
Kill switch — one operation cancels all quotes and orders. Test it weekly.
Sanity checks — reject any order that moves your position beyond limits, no matter what the strategy says.

At Alameda I watched risk discipline decay gradually and then suddenly. The lesson: automation of limits is not optional; humans are terrible at cutting their own losses in the moment.

What's next

In the next posts I'll walk through the funding-rate mechanics of Polymarket Perps (what the docs say, term by term), then margin modes and the liquidation waterfall with real numbers.

The full architecture with code lives in my GitHub repo: polymarket-mm-bot. It's MIT licensed — take it, fork it, build on it.

In America, if you work hard and smart, you win. Not financial advice — trading involves substantial risk of loss.

polymarket #algorithmictrading #marketmaking #crypto #python