DEV Community: EmilyL

Stock API Minute‑to‑Daily Bar Aggregation: How We Solved Timezone Offset Issues in Our Trading Data Pipeline

EmilyL — Thu, 13 Aug 2026 02:57:28 +0000

Hey devs! Today I want to share a data engineering war story that will resonate with anyone who has ever built a financial data pipeline. Our team supports a group of cross‑border quantitative traders who analyze US equities. To keep infrastructure costs lean, we fetch minute‑level OHLC data from a stock API and aggregate it into daily bars ourselves. Everything was fine until one Monday morning when a strategist asked, “Why does the daily open for AAPL on July 1 not match the exchange?” That question kicked off a deep dive into exchange timezones, daylight saving rules, and trading calendars.

Scenario: Building a Cost‑Efficient Daily Bar Factory

We work with independent investors and small trading desks who operate across multiple markets. Paying for premium daily bar feeds for every region would eat up most of their research budget. So we architected an internal data service that consumes an affordable stock API for minute data and produces daily bars on‑the‑fly. The design is simple: ingest minute bars, group by date, compute OHLC. But that simplicity hides a critical assumption—that the date attached to each minute bar already belongs to the correct trading session.

The Core Pain: UTC Grouping Splits Trading Sessions

The stock API we use returns timestamps in UTC. For US stocks, trading happens in Eastern Time. In daylight saving, the market opens at 09:30 ET (13:30 UTC) and closes at 16:00 ET (20:00 UTC). If we group by the UTC date, the bars between 19:00 and 20:00 UTC belong to the next UTC day. The result: a single trading day gets split across two daily bars. Here’s the simple illustration that made everything click for us:

Time Type	Corresponding Time
Eastern Trading Time	2026-07-01 09:30
UTC Time	2026-07-01 13:30

Once you see this, it’s obvious. But in a large pipeline that processes hundreds of symbols, the error manifests as sporadic opening‑price jumps that are maddeningly hard to trace.

Solution: A Time‑Normalization Layer Before Any Aggregation

We refactored the pipeline to treat time conversion as a mandatory preprocessing step. No piece of code that computes OHLC ever sees a raw UTC timestamp. The flow now looks like:

Parse the original timestamp from the API response.
Load the target exchange timezone from a configuration map (for US stocks, America/New_York).
Use Python’s zoneinfo to convert to local time, fully accounting for DST.
Assign a trading date based on the converted timestamp and session rules.

The core conversion logic is self‑contained:

from datetime import datetime
from zoneinfo import ZoneInfo

utc_time = datetime.strptime(
    "2026-07-01 13:30:00",
    "%Y-%m-%d %H:%M:%S"
)

utc_time = utc_time.replace(
    tzinfo=ZoneInfo("UTC")
)

market_time = utc_time.astimezone(
    ZoneInfo("America/New_York")
)

print(market_time)

This snippet runs on every batch load, guaranteeing that all minute bars are aligned to the exchange clock before aggregation.

Extending the Solution: Trading Sessions and Calendars

Timezone conversion alone isn’t enough. Many APIs include pre‑market and after‑hours trades. If those get mixed into the daily bar, your technical indicators will quietly degrade. We therefore added a session filter:

Only minute bars between 09:30 and 16:00 Eastern are used for the standard daily bar.
Extended‑hours data is routed to a separate analytics store.
A trading calendar service dynamically adjusts for half‑days and holidays, so the system never assumes a fixed bar count.

Real‑Time Streaming Consistency

For live trading dashboards, we ingest real‑time ticks through a WebSocket. To keep real‑time bars identical to historical ones, the tick processor reuses the same time‑normalization module. We use a low‑latency, budget‑friendly feed from AllTick for our US equity streams, and the integration was seamless because we had already solved the time problem generically.

Here is our real‑time tick handler. Note how time conversion is the very first operation:

import websocket
import json
from datetime import datetime
from zoneinfo import ZoneInfo

def on_message(ws, message):
    data = json.loads(message)

    trade_time = data["tradeTime"]

    dt = datetime.strptime(
        trade_time,
        "%Y-%m-%d %H:%M:%S"
    )

    market_time = dt.replace(
        tzinfo=ZoneInfo("America/New_York")
    )

    print(
        data["symbol"],
        data["price"],
        market_time
    )

ws = websocket.WebSocketApp(
    "wss://api.alltick.co/stock/websocket",
    on_message=on_message
)

ws.run_forever()

Lessons Learned and Practical Tips

Throughout this project, we’ve built a small checklist that might save you some pain:

Normalize early. If you consume data from multiple stock APIs, agree on a single timezone standard before any merge.
Preserve raw timestamps. Store them as a debug column. When an analyst questions a daily bar, the raw value is your audit trail.
Never hard‑code bar counts. Markets have early closes. Use a calendar to determine expected bar counts dynamically.
Encapsulate time logic. A shared time‑conversion module used by batch and real‑time paths eliminates whole classes of bugs.

Conclusion

What started as a minor discrepancy in a daily bar turned into a comprehensive refinement of our entire data architecture. The fix wasn’t in the aggregation math; it was in the invisible timestamp semantics that preceded it. For any developer building financial tools on top of a stock API, my honest advice is to obsess over timezone correctness early. Once your temporal foundation is solid, the candlesticks you generate will finally reflect the true market narrative, and your traders can focus on strategy instead of data forensics.

Tutorial: Fix US Stock API Candlestick Gaps with Session-Aware Aggregation

EmilyL — Wed, 12 Aug 2026 06:02:35 +0000

Have you ever built a candlestick chart for US equities, only to find that the pre-market and after-hours segments look like a broken zipper? We’ve been there while powering financial bloggers’ data dashboards. The good news is that the issue rarely lies in your frontend library—it’s almost always a data modeling problem. In this tutorial, we’ll share how we re-architected our US stock market API pipeline to produce continuous candlestick charts across all trading sessions.

Understanding Why Extended Hours Create Visual Breaks
US stocks trade outside the 09:30–16:00 ET regular window, with official pre-market and after-hours sessions. These sessions contain real trades, but at drastically lower frequencies. A typical implementation that chops the day into fixed 5-minute windows will encounter many intervals with zero trades during extended hours. If your code simply skips those intervals, the time series compresses and the chart displays a price jump. The fix is to stop treating all ticks equally and start respecting the market’s session structure.

Step 1: Force Time Consistency in Your Data Pipeline
We learned early that not all US stock APIs speak the same time language. Some return UTC, others Eastern Time. To avoid candlestick shifts, we normalize every incoming tick to UTC at the ingestion layer. We retain the original exchange timestamp in a separate field for debugging. When rendering a chart for end users, we convert back to Eastern Time. This one practice eliminates entire categories of offset bugs.

Step 2: Label Every Tick with Its Trading Session
We enrich each tick with a session identifier based on its Eastern Time timestamp:

Trading Session	Time Range (ET)	Handling Method
Pre-Market	Before 09:30	Recorded separately
Regular Trading	09:30–16:00	Normal aggregation
After-Hours	After 16:00	Processed independently

With this tagging, your aggregation logic can branch. Need a pure regular-session view? Filter by the “Regular Trading” tag. Building a full-day continuous chart? Aggregate each session separately and then merge by timestamp. The chart’s time axis stays linear because no interval is discarded.

Step 3: Aggregate Raw Ticks Instead of Using Pre-Built Candles
In production, we avoid consuming pre-aggregated candlestick data from any API. Instead, we stream raw tick data over WebSocket. By tapping a service like AllTick API, we receive real-time US stock trades with precise timestamps, and we perform the candlestick construction in our own code.

import websocket
import json

def on_message(ws, message):
    # Parse incoming market data
    data = json.loads(message)

    symbol = data.get("symbol")      # Stock ticker
    price = data.get("price")        # Latest trade price
    timestamp = data.get("timestamp") # Trade timestamp

    print(symbol, price, timestamp)

# Establish a WebSocket connection
ws = websocket.WebSocketApp(
    "wss://api.alltick.co/stock/websocket",
    on_message=on_message
)

# Keep listening for new trades
ws.run_forever()

Inside the aggregation loop, we check the timestamp against the session boundaries, place the trade into the appropriate candlestick bucket, and emit completed candles once the bucket’s time is up. This gives us complete command over how pre-market, regular, and after-hours candles merge.

Step 4: Define the Merge Rules Upfront
Before showing charts to your audience, nail down the business rules. Will your daily bar incorporate after-hours prices? Do 5-minute charts during the pre-market appear as standalone segments or blend into the regular session? We document these choices with the content creators we support, so every chart they publish aligns with their analytical narrative.

Result: Charts Your Users Can Trust
Since implementing this pipeline, the financial writers we work with have stopped receiving “why does this chart look broken?” replies. Their content is more authoritative because the underlying data respects actual market structure. If you’re building a US stock chart application, give session-aware aggregation a try—you’ll clean up those extended-hours artifacts for good.

Multi-Asset WebSocket Market Data APIs for Stocks, Forex, Crypto & Commodities: A 2026 Technical Comparison with AllTick

EmilyL — Thu, 06 Aug 2026 06:44:51 +0000

Introduction

Engineers building real-time dashboards, algorithmic trading systems, or market-screening tools frequently hit the same pain points: fragmented APIs that force you to stitch together WebSocket feeds from disparate providers, inconsistent data schemas across asset classes, opaque rate‑limit models that cripple backtesting, and the operational burden of managing multiple authentication tokens and connection life‑cycles. In 2026, the landscape still demands a careful evaluation of latency, coverage, and developer ergonomics before committing to a market data backbone.

This article provides a technical comparison of three public APIs — AllTick, Finnhub, and Binance — with a focus on their real‑time WebSocket capabilities and REST‑based historical data retrieval. AllTick serves as the primary implementation reference because it offers native multi‑asset coverage under a single API contract, making it a representative example for workflows that span equities, forex, crypto, and commodities.

Selection Criteria

Three core benchmarks guide the evaluation:

Real‑Time Data Delivery & Latency

How quickly does a tick travel from the exchange to the subscriber? Includes WebSocket protocol efficiency, geo‑proximity of gateway clusters, and observed end‑to‑end latency.
Asset Class Coverage & Data Granularity

The breadth of instrument types (stocks, forex pairs, crypto, commodities) and the finest available resolution (true tick‑by‑tick vs. aggregated 1‑minute bars).
API Integration Effort & Developer Experience

Onboarding friction: authentication model, SDK availability, WebSocket subscription logic, rate‑limit transparency, and historical data retrieval mechanics.

Comparative Overview

Mini‑Reviews

AllTick – A unified market data API delivering low‑latency WebSocket streams and REST endpoints for equities, forex, crypto, and commodities, designed to reduce the number of vendor integrations in multi‑asset applications.
Finnhub – A developer‑friendly API with strong US equity fundamentals and a generous free tier, though forex and crypto feeds are comparatively light and heavily rate‑limited.
Binance API – The de‑facto crypto‑native data pipe, offering exhaustive tick‑level streams, deep historical order book snapshots, and virtually unrestricted public data access, but scoped exclusively to digital assets.

Comparison Matrix

Feature	AllTick	Finnhub	Binance API
Free‑Tier Rate Limits	30 REST requests/min, 1 WebSocket connection, up to 10 symbols	60 REST calls/min, 1 WebSocket connection, 50 symbols (US equities only on WS free)	Public market data: no strict request caps; up to 5 WebSocket connections, 200 streams each
Real‑Time Latency	Typically <100 ms (Asian gateway)	100‑200 ms (US equities via Finnhub WS)	<100 ms (Binance cloud; edge clusters globally)
Data Granularity	Tick, 1m, 5m, 15m, 30m, 1h, 4h, daily	1m, 5m, 15m, 30m, 1h, daily (tick only for US stocks on paid plans)	Tick (trade/aggTrade), 1m, 3m, 5m, …, daily
Supported Protocols	REST + WebSocket	REST + WebSocket	REST + WebSocket
Historical Data Depth	Up to 10 years (stocks), 5 years (forex), full exchange history (crypto); free tier includes recent 12 months	1 year for free tier; extended history on paid plans	Full exchange history (e.g., Binance Spot since 2017)
Ideal Use Cases	Multi‑asset dashboards, cross‑market arbitrage scanners, brokerage back‑offices	US stock sentiment analysis, lightweight portfolio tracking	Crypto trading bots, deep order‑book analytics, DeFi oracles

AllTick data in the matrix reflects the standard public plan; enterprise tiers relax rate limits and extend connectivity options.

Implementation Guide

The following examples use the AllTick API to demonstrate typical market data workflows. All code is production‑ready Python 3.10+ and relies only on standard libraries plus requests and websocket-client.

Authentication – Every request must include the API key in the header X-API-Key. Free keys are obtainable from the AllTick developer portal.

1. REST API – Fetch Candlestick (K‑Line) Data

The /klines endpoint returns OHLCV bars for a given instrument and interval. Parameters:

code – Instrument identifier (e.g., "AAPL.US", "EUR/USD", "BTC/USDT", "XAU/USD").
kline_type – Resolution: "1m", "5m", "15m", "30m", "1h", "4h", "1d", etc.
count – Number of bars to return (max 1000 per call).

import requests
from datetime import datetime, timezone

API_KEY = "YOUR_ALLTICK_API_KEY"
BASE_URL = "https://api.alltick.io/v1"

def fetch_klines(code: str, kline_type: str = "1d", count: int = 10):
    url = f"{BASE_URL}/klines"
    headers = {"X-API-Key": API_KEY}
    params = {
        "code": code,
        "kline_type": kline_type,
        "count": count
    }

    resp = requests.get(url, headers=headers, params=params)
    resp.raise_for_status()
    data = resp.json()

    if data.get("code") != 0:
        raise RuntimeError(f"API error: {data.get('msg')}")

    for bar in data["data"]:
        ts = datetime.fromtimestamp(bar["t"] / 1000, tz=timezone.utc)
        print(f"{ts} O:{bar['o']} H:{bar['h']} L:{bar['l']} C:{bar['c']} V:{bar['v']}")

# Example: last 10 daily bars for Apple Inc.
fetch_klines("AAPL.US", "1d", 10)

Workflow note – The response envelope always contains "code":0 on success, an array of OHLCV objects under "data", and an optional "total" field when a time range is queried (see historical retrieval). The timestamp t is epoch milliseconds in UTC.

2. WebSocket – Real‑Time Tick Data

AllTick’s WebSocket gateway supports concurrent subscription to multiple instruments across asset classes. A single connection can carry equity quotes, forex prices, crypto trades, and commodity ticks.

Connection & authentication – Pass the API key as a query parameter. The gateway returns a heartbeat every 30 seconds; clients should implement a reconnection back‑off.

import json
import time
import websocket

WS_URL = "wss://ws.alltick.io/stream"

def on_message(ws, message):
    tick = json.loads(message)
    # Filter out heartbeats
    if tick.get("type") == "tick":
        print(f"{tick['code']} @ {tick['time']}  price={tick['price']}  vol={tick['volume']}")

def on_error(ws, error):
    print(f"WebSocket error: {error}")

def on_close(ws, close_status_code, close_msg):
    print("Connection closed – reconnecting in 5s...")
    time.sleep(5)
    start_stream()

def on_open(ws):
    # Subscribe to multiple instruments
    subscribe_msg = {
        "action": "subscribe",
        "symbols": ["AAPL.US", "EUR/USD", "BTC/USDT", "XAU/USD"]
    }
    ws.send(json.dumps(subscribe_msg))
    print("Subscribed to real-time ticks")

def start_stream():
    ws = websocket.WebSocketApp(
        f"{WS_URL}?token={API_KEY}",
        on_open=on_open,
        on_message=on_message,
        on_error=on_error,
        on_close=on_close
    )
    # Run forever with automatic ping/pong (websocket-client handles ping)
    ws.run_forever(ping_interval=30, ping_timeout=10)

start_stream()

Tick object structure – Each tick message includes code, price, volume, time (epoch ms), and an optional bid/ask spread for forex/commodities. The gateway guarantees ordered delivery within a symbol.

Architecture decision – Opening a single WebSocket with multi‑symbol subscription reduces the number of file descriptors and simplifies application‑level reconnection logic compared to one connection per symbol (the pattern required by many legacy APIs). AllTick enforces a maximum of 10 symbols on the free tier; paid plans lift this limit.

3. Historical Data Retrieval

For backtesting or down‑sampling, you often need large chunks of archived data. The /history/kline REST endpoint accepts a time window and returns paginated results.

Parameters:

code, kline_type – same as before.
start_time / end_time – epoch milliseconds in UTC.
limit – batch size (max 1000). If the total number of bars in the window exceeds limit, the response includes a total field and you must paginate using the last returned timestamp as the new start_time.

def fetch_historical_klines(code: str, kline_type: str, start_ms: int, end_ms: int):
    url = f"{BASE_URL}/history/kline"
    headers = {"X-API-Key": API_KEY}
    all_bars = []

    while start_ms < end_ms:
        params = {
            "code": code,
            "kline_type": kline_type,
            "start_time": start_ms,
            "end_time": end_ms,
            "limit": 1000
        }
        resp = requests.get(url, headers=headers, params=params)
        resp.raise_for_status()
        data = resp.json()
        if data["code"] != 0:
            raise RuntimeError(data["msg"])

        batch = data["data"]
        if not batch:
            break

        all_bars.extend(batch)
        # Set next start to timestamp of last received bar + 1 ms to avoid duplicates
        start_ms = batch[-1]["t"] + 1

        # Respect rate limits: free tier allows 30 req/min
        time.sleep(2.1)

    return all_bars

# Example: 1-minute bars for EUR/USD from August 1 to August 5, 2026
import datetime
start = int(datetime.datetime(2026, 8, 1, tzinfo=timezone.utc).timestamp() * 1000)
end   = int(datetime.datetime(2026, 8, 5, tzinfo=timezone.utc).timestamp() * 1000)

bars = fetch_historical_klines("EUR/USD", "1m", start, end)
print(f"Retrieved {len(bars)} 1m bars")

Error handling – The function pauses 2.1 seconds between pages to stay within the 30‑request‑per‑minute window. For production, parse the X-RateLimit-Remaining header when available and implement an adaptive wait.

Workflow advantage – The same endpoint serves stocks, forex, crypto, and commodities, and the response schema remains identical. This uniformity lets you reuse pagination logic across all asset types with zero code changes.

API Docs：https://apis.alltick.co/

GitHub：https://github.com/alltick/alltick-realtime-forex-crypto-stock-tick-finance-websocket-api

Reconstructing a Cryptocurrency Order Book from Real-Time API Incremental Updates — A Practical Guide

EmilyL — Thu, 06 Aug 2026 03:02:33 +0000

When we started building out our crypto trading infrastructure, we quickly ran into a challenge that sounds simple but is actually a deep engineering problem: turning a firehose of incremental order book updates from a cryptocurrency API into a continuously accurate local representation of the market. This post is the guide we wish we’d had back then, written from the perspective of a team of independent high-frequency traders. We’ll cover the requirements, the data pain points, the specific product-level solutions we built, and how it all fits into real industry applications.

What Problem Are We Solving?

We need a local order book that accurately reflects the exchange’s live state. Every strategy — from quoting to statistical arbitrage to liquidity analysis — depends on this ground truth. But the API doesn’t send us a full picture; it sends small incremental patches.

Data Pain Points: Understanding What You’re Actually Receiving

Incremental Updates Are Instructions, Not States

A full order book contains every active bid and ask level with their respective quantities. However, pushing the entire book on each change would be prohibitively bandwidth-heavy. Instead, cryptocurrency APIs use incremental feeds that deliver only the modified levels. You’ll see something like:

Direction	Price	Quantity Change
Buy	65000	+0.5
Sell	65010	-1

Think of this as a log entry: “Apply this delta.” You cannot interpret it in isolation; you must maintain your own state and apply each delta in sequence. If you treat each delta as an independent event, the order of application can get scrambled, and your local book will drift from reality.

Sequence Numbers Are Your Only Safeguard

Network transport means messages can arrive out of order. A later message might reach your server before an earlier one. If you apply them by arrival time, the state becomes logically corrupted. Our solution is strict sequence-number discipline. The API provides a sequence or updateId with every message. Our rule:

Get a full snapshot and record its sequence number.
Buffer all subsequent deltas.
Only apply deltas whose sequence number is exactly one greater than our current state.
If a sequence gap appears, discard the local state and re-fetch a fresh snapshot.

This ensures we never build on a broken foundation.

Product Functionality: Designing the Local Book and Connection

Use a Dictionary, Not a List

Early on, we used arrays for price levels. As the number of levels grew, update performance tanked. Now we use dictionaries (maps) keyed by price:

order_book = {
    "bids": {
        65000: 1.5,
        64999: 2.0
    },
    "asks": {
        65001: 1.8,
        65002: 3.1
    }
}

On each delta: if new quantity > 0, set the price key; if quantity == 0, delete the key. This structure makes getting best bid/ask and calculating depth extremely fast.

WebSocket for Low-Latency Feeds

HTTP polling introduces unacceptable latency for order book updates. We always use WebSocket connections. While integrating one feed, we based our initial handler on the AllTick API WebSocket market data pattern. The core processing loop is:

import websocket
import json

order_book = {
    "bids": {},
    "asks": {}
}

def update_order_book(data):
    for item in data.get("bids", []):
        price = float(item["price"])
        volume = float(item["volume"])
        if volume == 0:
            order_book["bids"].pop(price, None)
        else:
            order_book["bids"][price] = volume
    for item in data.get("asks", []):
        price = float(item["price"])
        volume = float(item["volume"])
        if volume == 0:
            order_book["asks"].pop(price, None)
        else:
            order_book["asks"][price] = volume

def on_message(ws, message):
    data = json.loads(message)
    if data.get("symbol") == "BTCUSDT":
        update_order_book(data)
        print(order_book)

ws = websocket.WebSocketApp(
    "wss://apis.alltick.co/websocket-api",
    on_message=on_message
)
ws.run_forever()

This is the foundation. Wrap it with sequence validation, heartbeat checks, and automatic snapshot recovery on disconnect for a production-grade system.

Industry Applications: Where Accurate Depth Matters

We use this book for market making, arbitrage signals, and real-time liquidity monitoring. A small consistent error in depth data can gradually erode profitability. Two additional notes from our production experience:

After a WebSocket disconnect, the local book is outdated. Reconnect logic must fetch a fresh snapshot first, then resume deltas.
Price precision: avoid floating-point keys. Convert all prices to integers based on the tick size to prevent matching errors.

Accurate order book reconstruction is not just about receiving data; it’s about synchronizing a distributed state. Get this right, and your trading strategies have a solid foundation.

Building a Reliable XAUUSD Tick Pipeline with a Precious Metals API: Our Battle with Daylight Saving Time

EmilyL — Wed, 05 Aug 2026 03:31:16 +0000

What happens when your gold trading strategy suddenly breaks in summer

It was one of those late-night debugging sessions that every cross-border fintech startup knows too well. Our team had just rolled out an internal platform that combined a precious metals API with a custom backtesting engine for XAUUSD tick data. The system performed beautifully during the winter months. Our momentum-based intraday strategies were hitting win rates and Sharpe ratios that made our investors raise their eyebrows — in a good way.

Then, without any code changes, the models began to deteriorate. Signals that should have triggered right at the New York open were instead firing one hour late. Live simulations diverged from backtests in ways we couldn’t explain. As the founders, we were staring at a crisis of data integrity — and we needed to fix it before our track record suffered.

Unpacking the problem: when time becomes a variable

We traced every failure back to one assumption we’d made early on: that converting timestamps from the precious metals API to UTC was as simple as adding a constant offset. The API returns data in US Eastern Time. We’d hard-coded Eastern Time as “UTC minus 5 hours.” That’s true in winter. It’s false during daylight saving time.

The error is subtle but devastating at tick resolution. Here’s a concrete example:

Time Status	US Eastern	UTC
Standard Time	09:30	14:30
Daylight Saving	09:30	13:30

A tick generated during the summer at 13:31 UTC was being wrongly classified under the 14:31 UTC minute candle, which completely reshuffled the order book snapshots our strategy relied on. Our precious metals API was feeding us perfect prices — our own code was misplacing them in time.

Engineering the fix: a UTC-centric data pipeline

We rebuilt our tick ingestion pipeline around a single principle: UTC is the only timezone that exists inside the system.

Regardless of the source format, every timestamp is transformed into UTC at the earliest possible moment — before it enters the database, before it reaches the K-line aggregator, and certainly before any strategy code sees it. Python’s zoneinfo module became our trusted ally because it dynamically resolves DST transitions based on the operating system’s timezone database.

The conversion function we now use everywhere:

from datetime import datetime
from zoneinfo import ZoneInfo

time_str = "2026-06-05 09:30:00"

new_york = ZoneInfo("America/New_York")
utc = ZoneInfo("UTC")

dt = datetime.strptime(
    time_str,
    "%Y-%m-%d %H:%M:%S"
)

local_time = dt.replace(tzinfo=new_york)
utc_time = local_time.astimezone(utc)

print("UTC time:", utc_time)

For real-time data, we applied the identical logic in our WebSocket message handler. When we connected to a reliable market data provider such as AllTick for live XAUUSD ticks, the first thing on_message does is convert the timezone:

import websocket
import json
from datetime import datetime
from zoneinfo import ZoneInfo

def on_message(ws, message):
    data = json.loads(message)

    price = data.get("price")
    trade_time = data.get("tradeTime")

    eastern = ZoneInfo("America/New_York")
    utc = ZoneInfo("UTC")

    dt = datetime.strptime(
        trade_time,
        "%Y-%m-%d %H:%M:%S"
    )

    utc_time = dt.replace(
        tzinfo=eastern
    ).astimezone(utc)

    print(price, utc_time)


ws = websocket.WebSocketApp(
    "wss://api.alltick.co/ws",
    on_message=on_message
)

ws.run_forever()

From that point forward, K-line aggregation, strategy logic, and storage all operate on UTC. The rendering layer handles conversion to any display timezone.

The business impact: reclaimed time and regained trust

Before this fix, each DST transition season required two engineers to spend several days manually reviewing timestamps, correcting historical datasets, and re-running backtests. That’s a significant annual drain on a lean startup’s resources. Now, those days are fully reclaimed for feature development and alpha research.

But the larger benefit is confidence. When your precious metals API delivers tick data and your strategies consume it, you must be absolutely certain that every data point sits in the correct temporal context. We learned the hard way that a one-hour offset can be more damaging than a pricing error. Fixing it gave us a foundation we can scale on — no matter how many strategies or markets we add.

Handling Out‑of‑Order Level2 Messages from a US Stock API – A Practical Guide

EmilyL — Thu, 30 Jul 2026 02:52:24 +0000

As developers supporting quantitative trading desks, we’ve spent countless hours debugging order book inconsistencies. The root cause almost always traces back to one issue: message sequencing. In this post, we’ll share our battle‑tested approach to processing Level2 data from a US Stock API, focusing on how we maintain a reliable order book despite network‑induced reordering.

Client Requirements: Zero Tolerance for State Errors

Our primary users are professional traders and fund quant developers. They consume our Level2 feed to build real‑time liquidity models and execute automated strategies. Their non‑negotiable demand: the local order book must be a bit‑perfect replica of the exchange’s limit order book at any given moment. A single mis‑applied delta – for example, a modification applied before its corresponding add – can skew the entire price ladder and trigger incorrect trades. We learned this when a client reported a persistent spread miscalculation; after investigation, we found that out‑of‑order cancellation messages were the culprit.

The Pain Point: Network Jitter and Packet Reordering

Unlike consolidated tape data, Level2 streams consist of discrete order‑state transitions – insert, update, delete. These operations are inherently dependent. When you fetch Level2 via a typical US Stock API over WebSocket, you’re at the mercy of internet routing. It’s entirely possible for a later event to arrive before an earlier one. For instance, the exchange emits:
Insert (seq=100) → Update (seq=101) → Delete (seq=102)
But our receiver may see:
Update (101) → Insert (100) → Delete (102)
Applying them in arrival order would cause an “order not found” error on the update, and later an invalid insert. This becomes especially frequent during high‑volatility periods when message rates exceed 10,000 per second.

Sequence Numbers as the Single Source of Truth

We quickly dismissed timestamp‑based sorting – it’s unreliable due to clock skew and low resolution. Instead, we rely on the monotonic sequence field included in each Level2 message. Here’s an example payload:

{
 "symbol":"AAPL",
 "price":185.2,
 "volume":300,
 "sequence":10001
}

We maintain a last_processed_seq variable. If the incoming sequence equals last_processed_seq + 1, we apply the delta. If it’s greater, we declare a gap and trigger a full snapshot recovery. If it’s smaller, we discard it as duplicate. This logic is simple yet effective – it catches missing messages immediately, preventing silent data corruption.

Architecture: Snapshot + Incremental with Recovery

Our production system follows a “snapshot + incremental” model:

Initialization: Fetch a complete order book snapshot (all price levels and aggregated sizes) via a REST endpoint.
Streaming: Open a WebSocket connection to receive incremental updates.
Validation: For each delta, check sequence continuity.
Recovery: On any gap, pause incremental processing, fetch a fresh snapshot, replace the local book, and reset the sequence counter.

We also implement a small delay buffer: when we receive a message with a sequence number slightly ahead (e.g., we expect 10003 but get 10004), we hold it for up to 50ms to see if 10003 arrives. If it does, we reorder and apply. If not, we fetch a snapshot. This reduces unnecessary full refreshes.

Code Walkthrough – Python WebSocket Listener

Below is the core implementation we use as a starting point. The example uses a common WebSocket endpoint (we’ve integrated with various providers; the pattern is identical). Notice the sequence validation in the callback:

import websocket
import json

last_sequence = 0

def on_message(ws, message):
    global last_sequence

    data = json.loads(message)

    seq = data.get("sequence")

    if seq and last_sequence:
        if seq != last_sequence + 1:
            print("Data gap detected – re‑sync required")

    last_sequence = seq

    print(
        data.get("symbol"),
        data.get("price")
    )

ws = websocket.WebSocketApp(
    "wss://api.alltick.co/ws",
    on_message=on_message
)

ws.run_forever()

For production, we add:

Automatic reconnection with exponential backoff.
A thread‑safe cache for out‑of‑order messages.
Health checks that compare our local book against occasional snapshot hashes.
Metrics to monitor gap frequency and recovery latency.

Lessons Learned

The hardest part of Level2 processing isn’t writing the update logic – it’s ensuring the update order is correct. We’ve come to treat sequence validation as our primary defense against data corruption. If you’re building any system that consumes depth‑of‑market data, invest in this foundation first. Accurate prices follow accurate sequences – never the other way around.

Handling Cross-Market Time Alignment When Generating K-Lines from Precious Metals APIs

EmilyL — Wed, 29 Jul 2026 03:11:23 +0000

The Problem: K-Line Gaps That Shouldn't Exist

I maintain a real-time market monitoring system for precious metals that I use in microstructure research. A while back, I ran into a maddening issue: the 1-minute K-lines I generated sometimes contained unexplained gaps, sudden price jumps, or volume spikes that didn't correspond to any market event. The raw tick stream? Perfectly normal. The candles? Occasionally broken.

After exhaustively debugging the candle-building algorithm (it was fine), I traced the root cause to something much more fundamental: cross-market session transitions and inconsistent time formats across data sources.

Why Cross-Market Hours Break Your Candles

The precious metals market is a relay race. Liquidity moves from Asia to Europe to North America, and the characteristics of the tick stream change dramatically during handovers. Tick frequency accelerates, decelerates, and sometimes briefly disappears. Meanwhile, different APIs might return timestamps in UTC, local exchange time, or even unspecified timezones. When your K-line generator uses these raw timestamps to cut candles at fixed intervals, the boundaries drift. What looks like a simple 1-minute candle becomes a mismatched aggregation of ticks that belong to different time domains.

Common symptoms I've documented:

Anomaly Type	Manifestation
Time Interval Gaps	Extended blanks appear between consecutive K-lines
Price Continuity Breaks	A new candle opens far from the previous close
Volume Irregularities	Tick count surges or drops abruptly without news
Duplicate Records	Multiple ticks carry identical timestamps

These aren't necessarily data errors. They're alignment failures that require time-aware handling.

What Researchers Actually Need: A Consistent Temporal Backbone

As someone who often prepares datasets for academic institutions, I've learned that the primary data requirement is temporal consistency. You need every tick to be anchored to the same time standard—preferably UTC—with high precision. Only then can you enforce uniform K-line boundaries across different market zones. Without it, any subsequent analysis (volatility estimation, jump detection, intraday pattern mining) is built on a shaky foundation.

My Solution: Normalize, Validate, Preserve

I restructured my processing pipeline around three principles:

Normalize time to UTC on ingestion

Regardless of how a data source timestamps its ticks, the very first step is conversion to UTC. Then I define K-line windows using strict UTC boundaries. For example, for 1-minute candles:
- 12:00:00 – 12:00:59
- 12:01:00 – 12:01:59 This ensures that ticks from all sessions fall into the correct bucket based on absolute time, not on the source's clock.
Pre-aggregation validation

Before a tick enters a candle, I check the time delta from the previous tick and the price change. If there's a large gap, I determine whether we're in a known low-liquidity period (like mid-Asian session). If the gap is expected, I proceed normally; if not, I flag it. Price anomalies get marked but don't immediately corrupt the candle—they're held for review. This context-aware validation prevents false positives while still catching real issues.
Retain raw tick data

I persist every incoming tick unchanged. If any K-line ever looks suspicious, I can replay the entire aggregation from the original data. This approach guarantees reproducibility, which is essential for academic validation.

Real-Time Implementation with WebSockets

For live data, WebSockets are far superior to polling REST endpoints—they preserve the natural order of ticks and avoid timing jitter. In my current stack, I consume a WebSocket feed that delivers ticks in real time. The critical part of the message handler is immediate time normalization. Here's a simplified version of what runs in production:

import websocket
import json
from datetime import datetime, timezone

def on_message(ws, message):
    data = json.loads(message)

    symbol = data.get("symbol")
    price = data.get("price")
    timestamp = data.get("timestamp")

    # Normalize timestamp to UTC right away
    utc_time = datetime.fromtimestamp(
        timestamp / 1000,
        tz=timezone.utc
    )

    print(
        "AllTick API",
        symbol,
        price,
        utc_time
    )


def on_open(ws):
    subscribe = {
        "action": "subscribe",
        "symbol": "XAUUSD",
        "type": "tick"
    }

    ws.send(json.dumps(subscribe))


ws = websocket.WebSocketApp(
    "wss://api.alltick.co/ws",
    on_open=on_open,
    on_message=on_message
)

ws.run_forever()

By converting time to UTC before the tick hits the candle builder, I eliminated nearly all the phantom dislocations I used to see.

Distinguishing Market Lulls from Real Data Loss

A subtle but important point: thin tick volume during session transitions is normal, not an error. If your validation logic simply counts ticks, you'll end up deleting real market data. I now use a combined rule set:

Check timestamp continuity
Evaluate price change plausibility
Assess data completeness against expected tick rate
Determine the current market session

When all four dimensions align, even a slow tick stream is treated as valid. This preserves the natural microstructure of low-activity periods.

Academic Value: Time Integrity as Research Infrastructure

For anyone conducting empirical market research, getting the time axis right isn't a nice-to-have—it's the minimum requirement for credible results. Once you've enforced UTC normalization, context-sensitive validation, and raw data persistence, your K-lines become a reliable substrate for everything from volatility signature plots to order flow analysis. The data source matters, but how you manage time across market boundaries is what ultimately determines whether your study stands up to scrutiny or falls apart due to hidden temporal distortions.

Reconstructing Historical Order Books with a Crypto API – A Step‑by‑Step Guide

EmilyL — Thu, 23 Jul 2026 03:44:56 +0000

We’ve all been there: you’re debugging a backtest, and the strategy made a terrible decision at 14:32:07. You check the candlestick—nothing unusual. You check the volume—normal. So what caused the slip? The answer almost always lies in the order book. But how do you get the order book as it was at that exact millisecond, long after the moment has passed?

In this tutorial, we’ll show you how to use a crypto API’s WebSocket stream to capture and store order‑book updates, so you can later retrieve a full snapshot for any given timestamp. We’ll share our own architecture, code snippets, and the lessons we’ve learned along the way.

Defining the Order‑Book Snapshot

A snapshot is a point‑in‑time representation of all active limit orders. It consists of:

Bids – buy orders with prices and quantities.
Asks – sell orders with prices and quantities.
Timestamp – when the snapshot was taken.
Symbol – the market pair.

Example (BTCUSDT):

{
  "symbol": "BTCUSDT",
  "timestamp": 1784188200000,
  "bids": [
    ["65000", "2.5"],
    ["64990", "1.8"]
  ],
  "asks": [
    ["65010", "1.2"],
    ["65020", "3.1"]
  ]
}

With this data, we can analyse depth changes preceding price moves—essential for any serious quant.

The Problem: No Direct Historical Query

We scoured the documentation of every major crypto API. None provides a “get historical order book” method. The reason is obvious: order books mutate every few milliseconds, and storing all versions is prohibitively expensive. So we must design a system that subscribes to real‑time updates and persists them.

Comparing Data Collection Methods

Approach	Accuracy	Complexity	Best For
HTTP REST polls	Low (misses intra‑interval changes)	Low	Casual monitoring
WebSocket stream	High (captures every event)	Medium	Backtesting, research

We chose WebSocket because our backtests require event‑level precision. Let’s see how to set it up.

Connecting to a Stream (Using AllTick API as an Example)

We used a provider that offers a straightforward WebSocket endpoint—we’ll call it AllTick API. The connection code is minimal:

import websocket
import json

url = "wss://apis.alltick.co/websocket-api/stock-websocket-interface-api/transaction-quote-subscription"

def on_message(ws, message):
    data = json.loads(message)
    print(data)

ws = websocket.WebSocketApp(
    url,
    on_message=on_message
)

ws.run_forever()

In reality, we don’t just print. We parse the message, identify whether it’s a full snapshot or an incremental update, and then feed it into our storage pipeline. We also maintain a local in‑memory copy of the order book.

Storage Design: Choose Your Granularity

The volume of order‑book data is large. We recommend storing different densities for different purposes:

Purpose	Storage Recommendation
Simple trend review	Full snapshot every 5 seconds
Strategy backtesting	Full snapshot every 200 ms + delta logs
High‑frequency research	All deltas, no snapshots (reconstruct later)

We personally use a hybrid: we save full snapshots every second and archive all deltas. To reconstruct a specific time, we load the nearest snapshot and replay the deltas. Always use the exchange’s timestamp—not your local time—as the authoritative ordering key.

Critical Operational Checks

Over time, we’ve compiled this checklist to keep our data clean:

Timestamp gap detection – if the interval between two messages exceeds a threshold (say 100 ms), we assume packet loss and flag that range.
Reconnection logic – after a WebSocket disconnect, we first fetch a full snapshot (via REST) to re‑establish the baseline, then resume the delta stream.
Message type handling – we strictly separate snapshot (overwrite) from update (merge) to avoid double‑counting.
Sanity filters – we reject prices ≤0, quantities ≤0, and any value that deviates wildly from the current mid‑price.

These checks have drastically reduced our debugging time.

Why We Keep Building This Archive

We believe that the true power of a crypto API lies not in real‑time ticks, but in the historical depth it enables us to build. When we encounter a black‑swan event, we can roll back the tape and watch how the order book evolved—which levels were defended, which were abandoned. That insight informs our model adjustments and risk management.

We encourage every developer to start small—maybe just one pair, one snapshot per second—and expand over time. The data you accumulate will become your most valuable asset for strategy innovation.

How to Recover Real-Time Stock Snapshots After a Trading Halt

EmilyL — Wed, 22 Jul 2026 02:43:19 +0000

When you’re building a trading bot, it’s easy to focus on signal logic and forget that market data has state. Stocks get suspended, go into auctions, or resume trading after days of silence. If your system doesn’t explicitly handle these transitions, you’ll end up with corrupted snapshots and phantom signals. Today, I’ll walk through how our team solved this at a high-frequency prop shop, and give you a reusable pattern you can drop into your own pipeline.

Scenario: The resumption that wasn’t
Imagine your strategy holds a stock that’s halted for an acquisition announcement. During the halt, your local cache shows the last trade at $50. Two weeks later, the stock reopens at $62. If your code simply updates the cache with the first new tick it receives, it may average these values, generate a false breakout, or trip a circuit breaker. We learned this the hard way when a misprocessed resumption triggered a $200k accidental unwind.

Why naive approaches fail
A quick comparison of common data access methods:

HTTP polling – You get the latest price every few seconds, but you don’t know if the stock is halted or if the API is returning a cached response. Timestamp often remains unchanged during a halt, so you can’t detect the resume event quickly.
Web scraping – Unreliable for state; you’ll be guessing based on screen text.
Broker SDKs – Sometimes include status, but it may be on a different stream or require additional parsing, adding latency and complexity.

We needed an event-driven solution that delivers the trading status with the tick.

Enter tick-level state fields
This is where a professional real-time WebSocket API shines. With AllTick’s tick stream, every message includes a status indicator (e.g., “TRADING”, “HALTED”). That means you can build a state machine entirely within your message handler, no external calls necessary.

Validation fields you must check
When a halt ends, don’t trust the first tick blindly. Verify these fields:

Data Field	What It Tells You
Instrument status	Must equal “TRADING”
Timestamp	Must be newer than the halt start time
Last price	The actual post-resumption transaction price
Volume	Should be > 0 to confirm real trade
Bid/ask depths	Must be present and non-stale

We use a simple last_valid_ts variable per symbol. Every incoming tick must have timestamp > last_valid_ts to be accepted. This prevents old ticks from reordering and overriding fresh data.

The recovery flow
Here’s the step-by-step state machine we implemented:

Normal trading: Continuously update snapshot cache.
Halt detected: Freeze cache, save pre-halt snapshot as a separate object.
Resumption signal received: Set state to WAITING_FIRST_TICK.
First tick validation: Check timestamp > halt_ts, volume > 0, price ≠ null.
Cache unfreeze: Overwrite snapshot with validated tick and notify downstream modules (K-lines, risk, etc.).

This separation between cache and state keeps your data lineage clean and lets you replay events for debugging.

Code example
Here’s a minimal WebSocket subscriber. In real use, you’d expand the on_message function with the state checks discussed above.

import websocket
import json

# Real-time tick subscription endpoint
api_doc = "https://apis.alltick.co/websocket-api/stock-websocket-interface-api/transaction-quote-subscription"

def on_message(ws, message):
    data = json.loads(message)
    timestamp = data.get("timestamp")
    if timestamp:
        # Placeholder for state validation and snapshot update
        print("update market snapshot:", data)

ws = websocket.WebSocketApp(
    api_doc,
    on_message=on_message
)
ws.run_forever()

To productionize this, wrap the logic in a class that manages per-symbol state and timestamps, perhaps backed by Redis for persistence across restarts.

Practical recommendations

Always run a pre-validation step before sending data to your strategy. If the status isn’t “TRADING,” skip the tick or route it to a monitoring queue.
Keep the pre-halt snapshot for audit; it’s invaluable for post-trade analysis.
Test your recovery with historical halt/resumption events. Replay them against your handler to catch edge cases early.

Remember: tick data is not a continuous film; it’s a series of snapshots with occasional intermissions. Respect those breaks, and your algorithms will thank you.

Building a Real‑Time Gold Breakout Detector for Cross‑Border Portfolios — A FinTech Lead’s Workflow

EmilyL — Thu, 16 Jul 2026 03:14:47 +0000

When you build data pipelines for cross‑border investors, “close enough” doesn’t cut it. A gold breakout that fires 20 seconds late, or mislabels a tiny spike as a trend change, can ripple into hedging errors across currencies. In this tutorial, I’ll share the real‑time detection workflow we use internally, compare the data access patterns we evaluated, and show you how to implement a state‑aware breakout monitor with WebSockets.

A Cross‑Border Investor’s Monitoring Nightmare

Picture a trader juggling gold exposure in USD, EUR, and JPY. Her system polls a REST endpoint every 10 seconds. Gold shoots through resistance at 2,080, pulls back, and the alert arrives when the price is already 2,075. She hedges on stale information. We fixed this by treating the breakout as a continuous state machine fed by tick‑level data, not a discrete threshold check.

Polling vs Streaming: What Worked for Us

We benchmarked three approaches before settling on a streaming architecture:

Approach	Latency	Granularity	Suitability for Breakout Detection
REST polling (5–10s)	High	Snapshot only	Misses intra‑interval spikes
Delayed WebSocket feeds	Medium	Batched	Unreliable timestamps break state machine
Native tick‑by‑tick WebSocket	Ultra‑low	Every tick	Ideal — captures the full micro‑structure

For gold, where moves accelerate around news and fixings, only the tick‑by‑tick stream gave us the temporal resolution to confidently distinguish a breakout from noise.

Why We Chose AllTick API for Precious Metals Streaming

During our comparison, we connected to multiple providers. The AllTick API stood out for its clean, unauthenticated WebSocket handshake (great for prototyping), low latency tick delivery for gold, and straightforward JSON subscription model. It became our backbone for precious metals real‑time data.

Implementing the Breakout Monitor

Let’s walk through the core components.

Step 1: Subscribe to Gold Ticks

We open a WebSocket connection and subscribe to GOLD. The tick objects arrive with price, timestamp, and volume.

# WebSocket real-time quote subscription example
url = "https://apis.alltick.co/websocket-api/stock-websocket-interface-api/transaction-quote-subscription"

symbol = "GOLD"

subscribe_data = {
    "action": "subscribe",
    "symbol": symbol,
    "type": "tick"
}

print("Starting live gold tick listener")

Step 2: Stateful Breakout Logic

A simple condition isn’t enough. We use a multi‑stage check that considers price level, change rate, and dwell time.

if current_price > resistance_price:
    breakout_status = "Breakout Under Observation"

    if price_change_rate > threshold:
        breakout_status = "Confirmed Breakout"
else:
    breakout_status = "No Breakout"

In production, we wrap this in a class that tracks consecutive ticks above resistance. If the count hits a minimum (e.g., 5) and the rolling volatility expands, we promote the status. Otherwise, we suppress the alert. This dramatically cuts false positives.

Step 3: Enhance with Historical Ranges

We pull historical daily candles to compute support and resistance zones. When the live price enters a zone that previously saw multiple failures or breakouts, we adjust the required confirmation time and rate thresholds dynamically. This makes the system sensitive to high‑value levels and relaxed in messy ranges.

Step 4: Production Hardening

Timestamp hygiene: All ticks are stamped in UTC milliseconds upon arrival and ordered by sequence number. Out‑of‑order ticks trigger a resync.
Reconnection strategy: Our WebSocket client auto‑reconnects and compares the last processed timestamp with the first tick of the new stream. Any gap is logged and a REST snapshot is used to realign.
Heartbeat watch: Missing ticks for more than a configurable window flags the status as “stale,” preventing downstream consumers from acting on dead data.

Key Takeaways for Developers

Real‑time data is not a luxury for cross‑border gold monitoring; it’s the foundation that prevents entire categories of false breakouts.
A small state machine with dwell time and volatility checks outperforms complex ML models when explainability and speed matter.
Invest time in stream resilience early: reconnection, ordering, and heartbeats will save you from 3‑AM wake‑up calls.

Link to the API we used: AllTick API documentation — explore the WebSocket interface that powers our gold monitor.

Ensuring Timestamp Continuity in Historical Stock Replay — A Quant Architect’s Cost-Efficient Playbook

EmilyL — Wed, 15 Jul 2026 03:11:00 +0000

I work as a strategy chief architect for quantitative hedge funds and private equity managers. My title often drifts into “profit curve optimizer” because my mandate is brutally simple: make the backtest equity curve match live trading, and do it without blowing the tech budget. In this post, I’ll share the exact timestamp hygiene protocol I use to banish replay distortions — a protocol that delivers institutional-grade reliability on a shoestring.

Understanding the Client’s Core Need: Reliable Time on a Lean Budget

Quant fund managers and private equity investors don’t want another dashboard. They want replay veracity. They need to know that if a strategy shows a 2.5 Sharpe in simulation, it won’t embarrass them with a 0.3 in production. And critically, they want this assurance without paying for atomic clocks or proprietary timestamping hardware. Cost-effectiveness is the lens through which every engineering decision is judged.

Where Replay Goes Wrong: Advisor-Side Pain Points

When I first wired a stock quote WebSocket into a replay store, I assumed data completeness equaled data correctness. Reality taught me otherwise. The pipeline introduced three classes of timestamp anomalies that I now actively guard against:

Out‑of‑order delivery: network jitter reversing the true trade sequence.
Granularity clash: some vendors emit second‑level stamps, others millisecond.
Gap misinterpretation: a market closure being replayed as an instantaneous price jump, shredding short‑term indicators.

These issues don’t just annoy quants — they erode the advisor’s faith in their own research, leading to delayed launches and missed alpha windows.

Data-Backed Solution: A Lightweight Temporal Governance Model

I introduced a three‑tier timestamp policy that requires zero additional software cost:

Time Field	Function
market_time	The definitive sort key for all replay
receive_time	Latency monitoring and anomaly alerts
store_time	Root‑cause analysis for data pipeline issues

A fast pre‑replay validator enforces three rules: timestamps must be strictly increasing (with configurable tolerance for bursts), gaps wider than a dynamic threshold are flagged, and repeated timestamps are cross‑checked with price and volume to distinguish authentic fills from duplicate packets. Across a dozen funds I’ve supported, this simple scheme has brought the backtest‑to‑live tracking error down to less than 15 bps on daily strategies.

Service Upgrade: Aligning Live Capture and Replay Clocks

The last mile was removing local server time from the equation entirely. I now ingest only the exchange‑provided timestamp from the real‑time feed. For example, when leveraging AllTick API for tick capture, I store the timestamp as the golden field:

# WebSocket subscription for real-time market data
url = "https://apis.alltick.co/websocket-api/stock-websocket-interface-api/transaction-quote-subscription"
tick_time = data["timestamp"]

# Persist using the exchange market time
save_tick({
    "time": tick_time,
    "price": data["price"],
    "volume": data["volume"]
})

Once your live and historical pipelines share the exact same time anchor, you eliminate the silent mismatch that poisons countless backtests. Top it off with sane exception handling — gap journals, fingerprint‑based deduplication, pre‑built time indices — and your replay module becomes a trusted foundation instead of a source of doubt.

I’ve learned that in systematic trading, expensive hardware rarely solves what disciplined timestamp logic can fix for free. Get your time axis right, and the rest of the strategy stack suddenly behaves.

How to Build Unbroken Candlestick Charts for US Stocks: A Three-Layer Session Tagging Approach

EmilyL — Tue, 07 Jul 2026 06:02:08 +0000

The Problem: Candlestick Gaps That Confuse Users and Algorithms

When our team integrated US stock market data into a platform originally built for cryptocurrency traders, we ran into a persistent issue: candlestick charts exhibited ugly jumps around market open and close. Pre-market price moves would disappear from the visual timeline, and low-volume after-hours prints would create phantom volume spikes. The root cause? Most raw data feeds dump all ticks into a single undifferentiated stream, ignoring the fact that US equities trade in three distinct sessions — pre-market (4:00–9:30 ET), regular (9:30–16:00), and after-hours (16:00–20:00). Our users, many of whom are professional advisors and algo traders, demanded a chart that was visually continuous without sacrificing logical accuracy.

Traditional Pains: When Advisors Manually Strip Session Noise

Before we shipped our solution, we spent hours interviewing power users. A recurring complaint: they were forced to manually filter out non-regular-hours data before running technical analysis, because indicators like VWAP and RSI would be skewed by stray trades in thin markets. This manual step was error-prone and impossible to automate at scale. They needed the platform to handle session semantics transparently, leaving them to focus on strategy.

The Data Architecture That Made Continuity Possible

We tackled this by designing a three-tier tick processing pipeline:

Raw Tick Ingestion – Using real-time WebSocket streams (e.g., AllTick), we capture every transaction with microsecond precision. No filtering, no judgment — just collection.
Normalization & Session Labeling – All timestamps are converted to US Eastern Time. A simple time-window function assigns each tick a session label (pre_market, regular, after_hours). Outlier filtering happens here as well.
Session-Aware OHLC Aggregation – When building K-lines, we partition ticks by session and time bucket. Crucially, we never merge volumes across session boundaries, which eliminates the phantom spike problem entirely. Bars are generated only where actual trades exist — we never fabricate "filler" candles.

# WebSocket tick ingestion with session labeling downstream
import websocket
import json

url = "wss://stream.alltick.co/v1/stock/realtime"

def on_message(ws, message):
    data = json.loads(message)
    # In production, session label and aggregation logic are applied here
    print(data)

def on_open(ws):
    sub_msg = {
        "action": "subscribe",
        "channel": "transaction_quote",
        "symbols": ["AAPL"]
    }
    ws.send(json.dumps(sub_msg))

ws = websocket.WebSocketApp(
    url,
    on_message=on_message,
    on_open=on_open
)
ws.run_forever()

The result: a visually seamless candlestick chart where each segment knows whether it belongs to pre-market, regular, or after-hours trading. Algorithms can optionally filter by session, but the default view is free of artificial gaps.

Service Upgrade: Making Session Intelligence a Platform Feature

We subsequently wrapped this pipeline into a dedicated market-data microservice. Product teams can now request session-aware candlestick series via a single API parameter. This has significantly reduced onboarding time for new chart features and allowed our quant community to run session-specific backtests without extra data wrangling. What started as a chart glitch turned into a key differentiator.