DEV Community

Emily
Emily

Posted on

Crypto Quant Systems: Polling vs WebSocket for Trading Pair Updates — and Why Hybrid Wins

#tutorial

The problem we don’t think about

When you’re building a digital asset quant system, you obsess over tick speed, order book depth, and backtesting accuracy. But there’s a quieter, sneakier issue that can undermine everything: your symbol list. As a data science lead for a crypto quant desk, I’ve seen multiple live incidents where a strategy fails with “symbol not found” because the local list of trading pairs was stale. This post digs into why trading pair updates are a non-trivial data engineering challenge and how we solved it.

The real cost of stale symbols

Crypto exchanges continuously adjust their instrument offerings. New coins get listed, dormant pairs get suspended, and sometimes pairs are delisted entirely. If you initialize your system once and never refresh, you create a growing divergence between your data view and reality. The damage is twofold: new pairs are invisible to your strategies, and invalid pairs generate noise that wastes compute and triggers false alerts. In a small setup with 20 symbols, you might never notice. Scale to 500 symbols across five exchanges, and it becomes a daily headache.

Inefficiency of naive approaches

Manual whitelisting breaks down immediately at scale. Timed polling every few hours reduces toil but introduces a latency window where the system is blind. I’ve seen new tokens rally 10% before our polling job caught up. And even when the job runs, you need logic to compare what’s changed — a raw list dump alone doesn’t tell you what’s new or removed.

Our technical solution: diff plus push

We resolved this by combining a lightweight diff engine with a push-based update stream.

The diff engine runs on every full list fetch:

# Existing cache
old_symbols = set(["BTCUSDT", "ETHUSDT"])
# Incoming list
new_symbols = set(["BTCUSDT", "ETHUSDT", "WIFUSDT", "PEPEUSDT"])

added = new_symbols - old_symbols
removed = old_symbols - new_symbols

print("New pairs:", added)
print("Removed pairs:", removed)
Enter fullscreen mode Exit fullscreen mode

For the push component, we use WebSocket events. Data services like Alltick provide a WebSocket API that can deliver both tick-level market data and symbol change notifications through a single stream.

import websocket
import json

def on_message(ws, message):
    data = json.loads(message)
    if data.get("type") == "symbol_update":
        print("Symbol universe update:", data["symbols"])
    if data.get("type") == "tick":
        print(data["symbol"], data["price"])

ws = websocket.WebSocketApp(
    "wss://quote.alltick.co/stream",
    on_message=on_message
)
ws.run_forever()
Enter fullscreen mode Exit fullscreen mode

This setup turned symbol updates from a background afterthought into a first-class event in our pipeline.

Changes in our workflow and architecture

We now keep symbol metadata in a dedicated service with a clear schema:

Field Meaning
symbol Pair identifier
status Whether it’s tradable
update_time Last status change
source Origin of the data

This allows us to propagate status changes (e.g., active → suspended) to all dependent services. A lot of developers focus only on additions and removals, but status transitions are more dangerous — they can silently block orders while the symbol still looks valid in a cached list.

We’ve converged on a three-layer pattern: in-memory cache for low-latency lookups, periodic full sync to correct any drift, and WebSocket push for immediate awareness. It’s a hybrid that gives us both reliability and speed. The lesson? In digital asset quant engineering, the symbol list is not boring plumbing — it’s the foundation your strategies stand on. Invest in it accordingly.

Top comments (0)