Originally published on my Hashnode blog.
"Latency arbitrage windows in retail FX are 50–200 ms" is a number you will find in a lot of places. I wanted a measured number instead of a folk range, so I measured it: XAUUSD at three retail brokers, each recorded next to a fast reference feed. The results are in the full paper. This post is about the part that usually gets skipped: how the data is recorded and prepared, because every number afterwards depends on it.
The one rule: both feeds on the same clock
A stale-quote window is a time difference between two price streams. If the two streams are timestamped by two different clocks, you are measuring the clocks as much as the market. A 5 ms drift between two machines is invisible in a chart and fatal for a result that lives in tens of milliseconds.
So every tick in the study gets one timestamp: its arrival time at the recording server, from one clock, for both sources.
That choice has a consequence worth stating plainly. The timestamp includes each feed's own delivery delay. This is not a flaw. It is exactly what a trading program on that server would see: the moment the fast price is visible, and the moment the broker's price is visible. The study measures stale quotes as seen from one machine, not some abstract "true" market time.
If you ever have to record on two machines, you need serious clock synchronisation (PTP, or at least well-monitored NTP) and you should report the remaining offset as an error bar. Recording both feeds on one machine avoids the problem entirely.
What to record
Per tick, per source, you need very little:
| Field | Meaning |
|---|---|
t_ms |
arrival time at the recorder, milliseconds, one clock for both feeds |
bid, ask
|
the displayed top-of-book quote |
src |
fast or broker
|
Record raw ticks, not bars or snapshots. A one-second snapshot hides every window shorter than a second, which is nearly all of them.
For scale, here is what three gold sessions produced:
| Broker A | Broker B | Broker C | |
|---|---|---|---|
| Session length | 7 h 19 min | 5 h 19 min | 7 h 56 min |
| Fast feed updates | 327,588 (12.4/s) | 538,465 (28.1/s) | 457,011 (16.0/s) |
| Broker updates | 71,936 (2.7/s) | 177,133 (9.2/s) | 178,627 (6.2/s) |
| Median broker spread | 10 points | 12 points | 46 points |
(1 point = 0.01 USD per ounce.) Notice the update rates first: the broker publishes several times less often than the fast feed. That ratio alone tells you stale quotes will exist. How long they last is the question.
Merging two streams into one timeline
The analysis needs, at every moment, the latest quote from both sources. The simplest way: put all ticks on one sorted timeline and carry each source's last quote forward.
import pandas as pd
fast = pd.read_csv("fast.csv", names=["t_ms", "bid", "ask"])
broker = pd.read_csv("broker.csv", names=["t_ms", "bid", "ask"])
fast["src"], broker["src"] = "fast", "broker"
# one timeline, sorted by arrival time; stable sort keeps file order within the same millisecond
ev = (pd.concat([fast, broker], ignore_index=True)
.sort_values("t_ms", kind="stable")
.reset_index(drop=True))
is_fast = ev["src"].eq("fast")
for side in ("bid", "ask"):
ev["f_" + side] = ev[side].where(is_fast).ffill() # latest fast quote
ev["b_" + side] = ev[side].where(~is_fast).ffill() # latest broker quote
# rows before both feeds have printed at least once are useless
ev = ev.dropna(subset=["f_bid", "b_bid"]).reset_index(drop=True)
Every row is now "the state of both quotes right after this tick arrived". Nothing is resampled or interpolated, so no window is created or destroyed by the merge.
One honest caveat: at 1 ms resolution, a fast tick and a broker tick can share the same timestamp, and their order within that millisecond is arbitrary. That limits how precisely the shortest windows can be measured, so treat anything under a couple of milliseconds as noise.
The gap: compare across the spread
The tempting comparison is mid against mid. It is wrong for trading purposes, because you never trade at the mid. You buy at the broker's ask and sell at the broker's bid.
So the study uses a cross-side gap, spreads included:
- buy opportunity:
fast bid − broker ask(the market already bids above what the broker asks) - sell opportunity:
broker bid − fast ask(the broker still bids above what the market asks)
The gap at each tick is the larger of the two:
POINT = 0.01 # XAUUSD: 1 point = 0.01 USD
buy_gap = (ev["f_bid"] - ev["b_ask"]) / POINT
sell_gap = (ev["b_bid"] - ev["f_ask"]) / POINT
ev["gap"] = pd.concat([buy_gap, sell_gap], axis=1).max(axis=1)
ev["side"] = (buy_gap >= sell_gap).map({True: "buy", False: "sell"})
Because both spreads are inside the formula, a broker with a wide spread produces fewer and smaller gaps. That is exactly right: a wide spread really does eat the opportunity. Broker C, with a 46-point median spread, shows it in the results.
Check the data before you measure anything
Three checks catch most bad recordings.
# 1. crossed or locked quotes: bid >= ask should not happen in a clean feed
for p in ("f", "b"):
bad = (ev[p + "_bid"] >= ev[p + "_ask"]).sum()
print(p, "crossed/locked rows:", bad)
# 2. update rate per source: a sudden drop means the recorder or the feed stalled
per_min = ev.groupby([ev["t_ms"] // 60_000, "src"]).size().unstack()
print(per_min.describe())
# 3. spread distribution: a broker spread that jumps around a lot needs a closer look
print(((ev["b_ask"] - ev["b_bid"]) / POINT).describe())
These are not formalities. In the study, a second fast feed I tested produced 1,927 windows per hour against the same broker, far more than the main feed, and looked great on a count. It also had 3,044 export rows with crossed or zero-spread quotes, and its average result per signal was close to zero. A large number of windows from a noisy feed is not an opportunity. Check the feed before you trust the count.
A first look at windows
With a clean merged timeline, finding windows takes a few lines. A window opens at the first tick where the gap exceeds a threshold and closes at the first tick where it does not:
THRESHOLD = 30 # points
open_now = ev["gap"] > THRESHOLD
was_open = open_now.shift(fill_value=False)
starts = ev.loc[open_now & ~was_open, "t_ms"].to_numpy()
ends = ev.loc[~open_now & was_open, "t_ms"].to_numpy()
# a window still open when the recording stops has no end: leave it out
durations_ms = ends - starts[:len(ends)]
print(pd.Series(durations_ms).describe(percentiles=[0.5, 0.75, 0.9]))
On the three gold sessions with a 30-point threshold, this gives median windows of 50 ms, 34 ms and 56 ms, with 22–35% of windows closing within 20 ms and 19–31% lasting past 100 ms. That is inside the folk range, but the spread between brokers matters more than the range itself. The next part goes through these windows properly.
Limits of this setup
- Displayed quotes only. No commission, slippage, swap, markup, rejections or last look. Everything measured from this data is a gross upper bound.
- Arrival time, not exchange time. By design, as explained above, but it means results describe one server's view.
- Three sessions on three different days. Prices, days and possibly feed instances differ, so broker differences are not a ranking.
- Disclosure: I work at Westernpips, which develops the software used for these recordings. This is a vendor case study, written so it can be repeated and checked, not an independent audit.
The full study, with method, tables and charts: How Long Do Stale Quotes Last? (DOI 10.5281/zenodo.23364414).
Code for this post: github.com/AlexanderVinsky/stale-quote-windows
Top comments (2)
Recording both feeds on one machine is the right call. The part I would add a check for is that "arrival time at the recorder" is not the same delay for the two sources: the fast feed and the broker stream almost certainly arrive over different connections, and a broker stream that is batched (several quotes written into one TCP segment) gets several ticks stamped within the same millisecond or two. That makes the broker look stale for a stretch and then catch up in a burst, which inflates short windows.
Two cheap diagnostics: plot the broker's inter-arrival gaps and look for clusters at 0-1 ms followed by a long gap, and rerun the window count with the same-millisecond ties ordered both ways (fast first, broker first). If the 1-5 ms bucket moves between the two orderings, that bucket is tie-break, not market.
Do you have a number for how many ticks share a millisecond with a tick from the other source in the three sessions?
I would also end a quoted opportunity when it switches from buying to selling. With a 30-point threshold, buy/sell gaps of (40, -60) followed by (-60, 40) both qualify, but each points to the opposite trade. Joining them gives either trade direction a duration it never had. Measure the two directions separately before comparing how long an order could have used the quote.