Fix Broken US Stock Backtests: Dynamic WebSocket Subscriptions To Eliminate Tick Gaps

Intro

If you’re building US stock order book reconstruction, backtesting scripts, or live algo trading tools, you’ve probably run into a brutal development pitfall: restarting your WebSocket every single time you add or remove tickers from your watchlist.

I made this mistake early on when building my market data pipeline, and it created a cascade of reliability issues during high-volatility trading hours. Every reconnect introduces permanent blank gaps in your tick stream, misaligns bid/ask price levels, and messes up millisecond-level transaction timestamps. The end result? Your backtest results become untrustworthy, and your high-frequency strategy fires invalid signals.

Lost tick data from disconnects can’t be recovered with incremental tick feeds alone. I ended up forced to build bulky snapshot compensation and validation logic just to patch the holes, which bloated my code and added unnecessary latency.

We’ll use AllTick API as a practical real-world example here. It natively supports dynamic ticker adjustments over one persistent WebSocket connection—no full reconnect required. In this post, I’ll break down all the flaws of the naive reconnect pattern, share a validation reference table, and drop fully functional Python code you can drop straight into your project.

The 4 Major Issues With Restarting WebSockets To Switch Tickers

Most new quant devs take the shortcut of closing and reopening sockets to swap symbols. This creates four persistent data consistency bugs that tank backtest accuracy:

1. Triple performance overhead + unrecoverable tick loss Each reconnect triggers a new handshake, authentication request, and full resubscription for every active stock. All ticks generated mid-disconnect vanish forever. Your order book has to rely entirely on snapshots to fill missing depth, which cripples precision for short-term trading strategies.
2. Unfiltered duplicate tick flooding Sending the same subscribe command multiple times makes the data server send duplicate tick records. Your local order book inflates volume numbers, making bid/ask depth completely unreliable for signal logic.
3. Race conditions create ghost subscriptions Rapid sequential add/remove ticker calls send unordered commands to the server. Your local tracked symbol list falls out of sync with the data feed: unwanted stocks spam you with ticks, while the tickers you care about receive zero data.
4. Corrupted depth from malformed tick packets Network flakiness often returns tick payloads with empty or zero price/volume fields. Without filtering these broken messages, you’ll overwrite valid order book tiers and create artificial depth breaks.

Core Solution: Dynamic Ticker Management On A Single Persistent WebSocket

What even is dynamic subscription?

Dynamic subscription lets you adjust your watchlist inside the lifetime of one open WebSocket connection. You send standardized API commands with lists of tickers to add or remove—no socket shutdown, no wasteful REST polling. A local set tracks your active symbols to keep client and server state perfectly synced.

Validation Reference Table (Great for unit testing)

Workflow Scenario	Common Dev Headache	Subscription Command Config	Pass Benchmark
Bulk ticker load on app startup	Reconnect logic creates massive handshake latency	cmd_id=22004, action="subscribe", code=["NASDAQ:AAPL","NASDAQ:TSLA"]	Local symbol set matches input list; only requested tickers receive ticks
Add tickers mid-trading session	Rebuilding sockets creates permanent data gaps	cmd_id=22004, action="subscribe", code=["NASDAQ:MSFT"]	Existing tick streams run uninterrupted, zero order book drift
Remove unused tickers from watchlist	Idle symbols waste bandwidth & compute power	cmd_id=22004, action="unsubscribe", code=["NASDAQ:TSLA"]	Target tick data stops streaming, symbol removed from local cache
Accidental duplicate subscribe calls	Repeated tick delivery inflates volume metrics	cmd_id=22004, action="subscribe", code=["NASDAQ:AAPL"]	Duplicate requests blocked client-side before sending to API
Empty ticker list payloads	Void commands waste server resources	cmd_id=22004, action="subscribe/unsubscribe", code=[]	Empty lists intercepted locally, no outbound API request sent

Full Working Python Implementation

# Dedicated WSS endpoint for US equities, following official API specs
import websocket
import json
from collections import defaultdict

# In-memory local order book storage
order_book = {
    "bids": defaultdict(float),
    "asks": defaultdict(float)
}
# Track active tickers to prevent ghost subscriptions & state desync
subscriptions = set()
# US stock market WebSocket endpoint
WS_STOCK_URL = "wss://quote.alltick.co/quote-stock-b-ws-api?token=YOUR_TOKEN"
# Fixed command ID for tick subscription control
CMD_TICK_SUB = 22004

def send_sub_cmd(ws, action: str, code_list: list):
    """Wrapper for subscription commands with pre-flight validation"""
    if not code_list or len(code_list) == 0:
        return
    unique_codes = list(set(code_list))
    payload = {
        "cmd_id": CMD_TICK_SUB,
        "action": action,
        "code": unique_codes
    }
    ws.send(json.dumps(payload))

def add_subscribe(ws, code_list: list):
    """Incrementally add tickers & sync local watchlist cache"""
    new_codes = [c for c in code_list if c not in subscriptions]
    if len(new_codes) == 0:
        return
    send_sub_cmd(ws, "subscribe", new_codes)
    for c in new_codes:
        subscriptions.add(c)

def remove_subscribe(ws, code_list: list):
    """Unsubscribe tickers & clean local tracking set"""
    del_codes = [c for c in code_list if c in subscriptions]
    if len(del_codes) == 0:
        return
    send_sub_cmd(ws, "unsubscribe", del_codes)
    for c in del_codes:
        subscriptions.discard(c)

def update_order_book(tick_data):
    """Update market depth, discard malformed tick packets entirely"""
    price = tick_data.get("price")
    qty = tick_data.get("quantity")
    side = tick_data.get("side")
    code = tick_data.get("code")

    # Filter invalid zero/empty tick entries to avoid broken depth tiers
    if not all([price, qty, side, code]) or float(price) <= 0 or float(qty) <= 0:
        return
    price = float(price)
    qty = float(qty)
    if side == "buy":
        order_book["bids"][price] += qty
    else:
        order_book["asks"][price] += qty

def on_open(ws):
    """Run bulk initial subscription once WebSocket handshake completes"""
    init_codes = ["NASDAQ:AAPL", "NASDAQ:TSLA"]
    add_subscribe(ws, init_codes)
    print("Initial subscription complete, active tickers:", subscriptions)

def on_message(ws, message):
    """Route tick data only for symbols present in local watchlist"""
    try:
        msg = json.loads(message)
        tick_code = msg.get("code")
        if tick_code and tick_code in subscriptions:
            update_order_book(msg)
    except Exception:
        # Ignore malformed JSON without terminating active connection
        return

def on_error(ws, error):
    print("WebSocket connection fault detected:", error)

def on_close(ws, close_code, close_msg):
    print("Market data stream disconnected, clearing subscription cache")
    subscriptions.clear()

if __name__ == "__main__":
    ws_app = websocket.WebSocketApp(
        WS_STOCK_URL,
        on_open=on_open,
        on_message=on_message,
        on_error=on_error,
        on_close=on_close
    )
    # 10s heartbeat interval to silently dead connections early
    ws_app.run_forever(ping_interval=10)

4 Common Production Bugs & Their Fixes

1. Tick callback backlogs during volatility spikes

Symptom: Mass tick surges block synchronous callback processing, delaying order book updates and scrambling timestamps.
Detection: Flag any single tick processing runtime over 1ms.
Fix: Separate network I/O and depth calculation with a thread pool; WebSocket callbacks only enqueue raw tick data for async processing.

2. Silent dead WebSockets from temporary network jitter

Symptom: Brief packet loss breaks heartbeat exchanges without triggering on_close, freezing market depth indefinitely.
Detection: Mark connection invalid after two consecutive unacknowledged heartbeats.
Fix: Add heartbeat timeout reconnection logic, re-subscribe all cached tickers after reconnecting.

3. Race conditions from rapid watchlist edits

Symptom: Fast add/remove requests send out-of-order commands, desyncing local symbol tracking with server streams.
Detection: Cross-check incoming tick symbols against local set after every subscription command.
Fix: Wrap all watchlist operations in thread locks to serialize subscription commands.

4. Silent failed subscriptions from incomplete ticker IDs

Symptom: Raw ticker strings like AAPL (missing exchange prefix) receive zero ticks with no error logs. Valid format: NASDAQ:AAPL.
Detection: Flag tickers with no incoming ticks 5s post-subscription request.
Fix: Build a formatter to auto-prepend NASDAQ / NYSE exchange prefixes to all ticker inputs.

Final Thoughts & Implementation Limits

This single persistent WebSocket architecture delivers gap-free continuous tick streams and eliminates order book drift caused by frequent reconnects. The four layers of safeguards — local watchlist tracking, malformed packet filtering, thread locking, and heartbeat monitoring — resolve nearly all data consistency issues you’ll face building US equity depth pipelines.

Two key limitations to plan for in production:

1. Watchlist state can’t sync across multiple independent WebSocket connections; logic only works within one single stream.
2. Real-time tick feeds don’t support historical backfilling. Pair this system with periodic order book snapshots to maintain long-term reconstruction accuracy.

If you’re building intraday backtesting tools or live algorithmic trading bots, this pattern fully replaces error-prone reconnect workflows and drastically reduces false signals from corrupted tick data.

Wrap-Up

Have you run into any weird WebSocket or tick consistency bugs while building US stock order book tools? Share your pain points and workarounds in the comments below—I’d love to exchange solutions with other quant devs.