Every week, publicly listed stocks in the US have their off-exchange ("dark pool") trading volume disclosed per symbol, per venue, under a market-transparency rule that predates most people's interest in market data APIs. The catch: it's published as raw, paginated compliance data, not a friendly product. Nobody looks at it directly.
I spent an afternoon turning that into a signal: is this week's off-exchange volume for a given ticker unusual compared to its own recent history?
The interesting engineering bits
Partition-key gotchas. The API enforces that any sort operation must filter ALL its partition keys with an exact-match comparator, or you get a 400 - though the error message thankfully tells you exactly which keys are missing. Once you know that, filtering instead of sorting sidesteps the whole problem.
Comparator naming isn't obvious. "greater or equal" isn't spelled the way you'd guess - it's a short enum name you basically have to discover by trial. A handful of curl calls with different guesses got there faster than reading the docs end to end.
JSON vs CSV by header, not URL param. Same endpoint, same body - send an Accept: application/json header and you get typed JSON back instead of a quoted-CSV text blob you'd otherwise have to parse by hand.
The actual signal
Rather than an absolute threshold (which punishes small-caps and ignores mega-caps equally), each ticker's most recent reported week gets compared against its OWN trailing average over the prior weeks. A stock trading 3x its own normal off-exchange volume is a much more useful signal than "over 1 million shares," which means something completely different depending on the ticker.
Only tickers that clear both a minimum volume floor and a minimum spike ratio get returned. Empty runs are never billed, so there's no incentive to pad results with noise - a design pattern I try to keep across everything I ship.
Packaged it as an Apify actor (pay-per-result): Dark Pool Pulse - https://apify.com/0xgollum/dark-pool-pulse
Always curious what data-anomaly patterns other people are scanning for - drop them in the comments.
Top comments (0)