Handling Cross-Market Time Alignment When Generating K-Lines from Precious Metals APIs

The Problem: K-Line Gaps That Shouldn't Exist

I maintain a real-time market monitoring system for precious metals that I use in microstructure research. A while back, I ran into a maddening issue: the 1-minute K-lines I generated sometimes contained unexplained gaps, sudden price jumps, or volume spikes that didn't correspond to any market event. The raw tick stream? Perfectly normal. The candles? Occasionally broken.

After exhaustively debugging the candle-building algorithm (it was fine), I traced the root cause to something much more fundamental: cross-market session transitions and inconsistent time formats across data sources.

Why Cross-Market Hours Break Your Candles

The precious metals market is a relay race. Liquidity moves from Asia to Europe to North America, and the characteristics of the tick stream change dramatically during handovers. Tick frequency accelerates, decelerates, and sometimes briefly disappears. Meanwhile, different APIs might return timestamps in UTC, local exchange time, or even unspecified timezones. When your K-line generator uses these raw timestamps to cut candles at fixed intervals, the boundaries drift. What looks like a simple 1-minute candle becomes a mismatched aggregation of ticks that belong to different time domains.

Common symptoms I've documented:

Anomaly Type	Manifestation
Time Interval Gaps	Extended blanks appear between consecutive K-lines
Price Continuity Breaks	A new candle opens far from the previous close
Volume Irregularities	Tick count surges or drops abruptly without news
Duplicate Records	Multiple ticks carry identical timestamps

These aren't necessarily data errors. They're alignment failures that require time-aware handling.

What Researchers Actually Need: A Consistent Temporal Backbone

As someone who often prepares datasets for academic institutions, I've learned that the primary data requirement is temporal consistency. You need every tick to be anchored to the same time standard—preferably UTC—with high precision. Only then can you enforce uniform K-line boundaries across different market zones. Without it, any subsequent analysis (volatility estimation, jump detection, intraday pattern mining) is built on a shaky foundation.

My Solution: Normalize, Validate, Preserve

I restructured my processing pipeline around three principles:

1. Normalize time to UTC on ingestion
   
   Regardless of how a data source timestamps its ticks, the very first step is conversion to UTC. Then I define K-line windows using strict UTC boundaries. For example, for 1-minute candles:

   • 12:00:00 – 12:00:59
   • 12:01:00 – 12:01:59 This ensures that ticks from all sessions fall into the correct bucket based on absolute time, not on the source's clock.

2. Pre-aggregation validation
   
   Before a tick enters a candle, I check the time delta from the previous tick and the price change. If there's a large gap, I determine whether we're in a known low-liquidity period (like mid-Asian session). If the gap is expected, I proceed normally; if not, I flag it. Price anomalies get marked but don't immediately corrupt the candle—they're held for review. This context-aware validation prevents false positives while still catching real issues.

3. Retain raw tick data
   
   I persist every incoming tick unchanged. If any K-line ever looks suspicious, I can replay the entire aggregation from the original data. This approach guarantees reproducibility, which is essential for academic validation.

Real-Time Implementation with WebSockets

For live data, WebSockets are far superior to polling REST endpoints—they preserve the natural order of ticks and avoid timing jitter. In my current stack, I consume a WebSocket feed that delivers ticks in real time. The critical part of the message handler is immediate time normalization. Here's a simplified version of what runs in production:

import websocket
import json
from datetime import datetime, timezone

def on_message(ws, message):
    data = json.loads(message)

    symbol = data.get("symbol")
    price = data.get("price")
    timestamp = data.get("timestamp")

    # Normalize timestamp to UTC right away
    utc_time = datetime.fromtimestamp(
        timestamp / 1000,
        tz=timezone.utc
    )

    print(
        "AllTick API",
        symbol,
        price,
        utc_time
    )


def on_open(ws):
    subscribe = {
        "action": "subscribe",
        "symbol": "XAUUSD",
        "type": "tick"
    }

    ws.send(json.dumps(subscribe))


ws = websocket.WebSocketApp(
    "wss://api.alltick.co/ws",
    on_open=on_open,
    on_message=on_message
)

ws.run_forever()

By converting time to UTC before the tick hits the candle builder, I eliminated nearly all the phantom dislocations I used to see.

Distinguishing Market Lulls from Real Data Loss

A subtle but important point: thin tick volume during session transitions is normal, not an error. If your validation logic simply counts ticks, you'll end up deleting real market data. I now use a combined rule set:

• Check timestamp continuity
• Evaluate price change plausibility
• Assess data completeness against expected tick rate
• Determine the current market session

When all four dimensions align, even a slow tick stream is treated as valid. This preserves the natural microstructure of low-activity periods.

Academic Value: Time Integrity as Research Infrastructure

For anyone conducting empirical market research, getting the time axis right isn't a nice-to-have—it's the minimum requirement for credible results. Once you've enforced UTC normalization, context-sensitive validation, and raw data persistence, your K-lines become a reliable substrate for everything from volatility signature plots to order flow analysis. The data source matters, but how you manage time across market boundaries is what ultimately determines whether your study stands up to scrutiny or falls apart due to hidden temporal distortions.