Fixing WebSocket Silent Disconnects for Financial Market Data

Intro

If you work with real-time financial tick data via WebSocket long connections, you’ve probably run into frustrating hidden issues:
Connections show as online, but data stops flowing. Adding/removing trading symbols triggers connection storms. Minor network glitches break your data pipeline without obvious error logs.

Today I’ll share a practical Python solution built around single-connection dynamic subscription and heartbeat timeout detection. We use alltick api as our example throughout this post. I’ll cover real production pain points, architecture, full runnable code, common bugs and optimization results. All code can be directly used in your projects.

Real Production Pain Points & Background

My team maintains real-time data pipelines for stocks, forex and cryptocurrencies. A core requirement is to add or remove monitored trading symbols on demand.

At first, we created one independent WebSocket connection for each symbol. This simple approach caused a lot of problems in production:

• Bulk symbol updates create frequent connection creation and destruction, leading to reconnection storms and high network load.
• Ghost subscriptions: Data keeps coming even after you send an unsubscribe request.
• False-alive sockets: The connection status stays connected, but tick data stops completely. Your analysis logic runs on invalid data.

We refactored the architecture to use one single persistent WebSocket connection for all symbols, plus an independent thread for heartbeat monitoring. After the upgrade, all hidden connection issues are gone, and the system runs stably under high-frequency data traffic.

4 Common WebSocket Issues in Fintech

Let’s break down the most frequent problems when using WebSocket for financial streaming.

1. Connection Flood & Reconnection Storms

Rebuilding connections every time you update symbols causes endless handshakes and closures. It wastes system resources and makes the entire data stream unstable.

2. Undetectable False-Alive Sockets

Network jitter or server rate limits won’t trigger a WebSocket close event. The link is actually broken, but logs show normal status. These silent failures are hard to debug.

3. Desynchronized Subscription States

Rapid subscribe/unsubscribe requests cause race conditions. Your local symbol list no longer matches the server’s real subscription status, leaving many invalid subscriptions.

4. Missing Edge Case Validation

Duplicate requests, empty symbol lists and invalid codes are not filtered in advance. Accumulated useless requests increase API parsing pressure.

Complete Technical Solution

What is Dynamic Subscription

Dynamic subscription means updating your symbol list via API commands inside an existing WebSocket long connection. You never destroy the active connection. This is a standard pattern for high-frequency real-time data, different from traditional REST polling.

Overall Architecture

We split the solution into two decoupled modules to improve stability:

1. Dynamic Subscription: Manage all symbols on one long connection to avoid reconnection storms.
2. Heartbeat Monitoring: Run detection in a separate daemon thread. It works independently even when data processing logic is blocked.

Common Use Cases & Rules

Here are the most common scenarios you will meet during development, with API settings and validation rules.

Initial bulk subscription for multiple symbols

Creating a connection per symbol wastes resources.
Use cmd_id=22004 + action=subscribe, and fill code with your symbol list like [NASDAQ:AAPL,BTCUSDT].
Validation: Only one WebSocket connection is created, and local storage syncs all symbol codes.

Incrementally add new symbols

New connections will cause stream instability.
Use cmd_id=22004 + action=subscribe, set code to new symbols such as [EURUSD].
Validation: Original connection remains active; client only receives data from new symbols.

Unsubscribe from specific symbols

Redundant data still arrives after unsubscription.
Use cmd_id=22004 + action=unsubscribe, write target symbols to the code field like [NASDAQ:AAPL].
Validation: Remove codes from local list; server stops pushing data for these symbols.

Send duplicate subscription requests

Repeated requests increase server load.
When using cmd_id=22004 + action=subscribe for existing symbols like [BTCUSDT], add client-side deduplication. Do not send duplicate requests to server.

Send subscription with empty list

Empty requests trigger runtime errors on both sides.
Block empty code: [] requests locally before sending over the network.

Full Python Code Implementation

This code includes connection callbacks, local state management, heartbeat monitoring, dynamic subscription and data validation. We use Alltick API as the example, and it works in all standard Python environments.

import websocket
import json
import time
import threading

# Manage subscription status and deduplication
subscriptions = set()
# Record the last heartbeat timestamp
last_heartbeat_time = time.time()

# Independent thread for heartbeat monitoring
def heartbeat_monitor():
    global last_heartbeat_time
    while True:
        current_ts = time.time()
        time_gap = current_ts - last_heartbeat_time
        # Timeout: 20s | Check interval: 5s
        if time_gap > 20:
            print("【Alert】Heartbeat timeout, connection link abnormal")
            # Extend: close connection / trigger reconnection / pause tasks
            break
        time.sleep(5)

# Triggered when WebSocket connects successfully
def on_open(ws):
    global subscriptions
    init_codes = ["NASDAQ:AAPL", "BTCUSDT"]
    subscriptions.update(init_codes)
    sub_req = {
        "cmd_id": 22004,
        "action": "subscribe",
        "code": init_codes
    }
    ws.send(json.dumps(sub_req))
    print("Initial symbol subscription completed")

# Handle incoming messages
def on_message(ws, message):
    global last_heartbeat_time
    if not message:
        return
    try:
        data = json.loads(message)
        # Refresh heartbeat time
        if data.get("type") == "heartbeat":
            last_heartbeat_time = time.time()
            return
        # Process tick data and filter invalid values
        if data.get("type") == "tick":
            code = data.get("code", "")
            price = data.get("price", 0)
            open_24h = data.get("open_24h", 0)
            if not code or price <= 0 or open_24h <= 0:
                return
            print(f"Symbol {code} Latest Price: {price}")
    except Exception as e:
        print(f"Message parsing error: {str(e)}")

# Connection error callback
def on_error(ws, error):
    print(f"Connection error occurred: {error}")

# Connection close callback
def on_close(ws, close_status_code, close_msg):
    global subscriptions
    subscriptions.clear()
    print(f"Connection closed, status code: {close_status_code}")

# Add new symbols to subscription
def add_subscribe(ws, code_list):
    global subscriptions
    new_codes = [code for code in code_list if code not in subscriptions and code]
    if not new_codes:
        return
    subscriptions.update(new_codes)
    req = {
        "cmd_id": 22004,
        "action": "subscribe",
        "code": new_codes
    }
    ws.send(json.dumps(req))

# Unsubscribe selected symbols
def cancel_subscribe(ws, code_list):
    global subscriptions
    remove_codes = [code for code in code_list if code in subscriptions and code]
    if not remove_codes:
        return
    for code in remove_codes:
        subscriptions.discard(code)
    req = {
        "cmd_id": 22004,
        "action": "unsubscribe",
        "code": remove_codes
    }
    ws.send(json.dumps(req))

if __name__ == "__main__":
    # Stock market WebSocket endpoint
    stock_wss_url = "wss://quote.alltick.co/quote-stock-b-ws-api?token=YOUR_TOKEN"
    # Forex & Crypto WebSocket endpoint
    common_wss_url = "wss://quote.alltick.co/quote-b-ws-api?token=YOUR_TOKEN"

    ws_app = websocket.WebSocketApp(
        common_wss_url,
        on_open=on_open,
        on_message=on_message,
        on_error=on_error,
        on_close=on_close
    )
    # Start heartbeat monitor thread
    threading.Thread(target=heartbeat_monitor, daemon=True).start()
    # Built-in ping detection every 10 seconds
    ws_app.run_forever(ping_interval=10)

Common Pitfalls & Fixes

Here are the most frequent bugs we met during development and production, with simple solutions.

1. Issue: High-volume tick data blocks callbacks and delay heartbeat detection
   Fix: Run heartbeat logic in a separate thread. Sort data by priority and process core data first.

2. Issue: Network jitter causes false-alive sockets without close events
   Fix: Use heartbeat time gap to judge timeout. Close broken connections and reconnect orderly; pause data tasks during reconnection.

3. Issue: Frequent add/remove requests lead to unsynchronized subscription states
   Fix: Use set to manage symbols, enable deduplication and limit request frequency.

4. Issue: Wrong codes or empty strings cause silent subscription failure
   Fix: Add local validation for empty and invalid codes; auto-resubscribe for abnormal items.

Feature Boundary

• ✅ Supported: Dynamically add and remove symbols within one WebSocket connection.
• ❌ Not supported: Cross-connection subscription sync, historical tick data query, custom commands other than cmd_id=22004.

Optimization Results

After deploying this solution:

1. Lower resource usage: Single long connection reduces TCP handshake and teardown overhead for both client and server.
2. Stronger code robustness: Pre-validation blocks invalid requests and corrupted data.
3. Faster debugging: Heartbeat and subscription exceptions are fully logged. Silent disconnections are easy to locate.
4. More stable business: No more connection flutters, duplicate data or calculation errors for financial analysis.

---

Discussion

Have you ever dealt with WebSocket false-alive issues in fintech or real-time projects? What solutions do you use for connection keep-alive? Feel free to leave a comment below! 😊