I pointed a DTW scanner at the official Binance Spot ETHUSDT one-minute archive for July 2023.
The reference window was:
2023-07-01 12:13–12:23 UTC
The best non-overlapping candidate in the monthly scan was:
2023-07-27 17:45–17:55 UTC
The scanner returned:
distance=9.967796
path-steps=11
stretch-steps=2
Fine.
Is that a match?
No honest answer yet.
It is the smallest distance found under one preprocessing rule, one local cost, one Sakoe–Chiba window, and one slope penalty. The scanner found an argmin. It did not magically produce a trading decision.
That distinction is where most DTW demos stop being useful.
I also tried to use two Binance UI screenshots as provenance. I rejected them.
The visible price scales did not match the CSV rows used by the scanner: the reference file contains roughly 1925–1927, while the screenshot showed a different price region; the candidate file contains roughly 1864–1867, while its screenshot was around 1923–1929.
Wrong date, wrong market, wrong timezone, or a chart that never actually jumped to the requested archive — it does not matter. A pretty screenshot pointing at different candles is worse than no screenshot.
The figure below is rebuilt directly from the exact OHLC rows fed into the scan.
The score got four times smaller. It still did not become a probability.
The two windows live at different nominal price levels:
reference: around 1,926 USDT
candidate: around 1,865 USDT
Comparing raw closes would mostly measure that price-level difference. The scanner converts each window to close-to-close returns in basis points:
returnBPS := ((currentClose - previousClose) / previousClose) * 10000
That transformation is visible in the code. lvlath/dtw does not silently normalize the sequence behind the caller's back.
Hidden normalization is the sort of “help” that makes a demo look clean and a production incident miserable to reconstruct.
With strict index pairing, the ten return samples accumulate an absolute cost of:
39.893841
The selected DTW path accumulates:
9.967796
under the same absolute local cost and a 0.05 penalty for every horizontal or vertical move.
The objective fell by roughly 75%.
That is not a 75% similarity score. It only says that allowing a constrained time deformation explains the two return sequences much better than forcing sample i to match sample i.
A percentage without calibration is just a confident UI element.
What the real path says
The scanner called:
dtw.Align(candidateSignal, referenceSignal, ...)
The public coordinate contract is therefore:
Coord.I → candidate index
Coord.J → reference index
The selected path was:
{0,0}
{0,1} horizontal
{1,2}
{2,3}
{3,4}
{4,5}
{5,6}
{6,7}
{7,8}
{8,9}
{9,9} vertical
Most of the path runs diagonally with a one-step offset.
Two cells carry the temporal deformation:
{0,0} → {0,1}
The candidate's first return is held while the reference advances once.
At the other end:
{8,9} → {9,9}
The candidate advances while the final reference phase is held.
That is much more useful than “9.97”. It tells the reviewer exactly where the clocks were stretched.
The path also shows what DTW did not fix. The largest local contribution on the selected route is 3.578995, where candidate return -3.321707 is paired with reference return -6.900703. Warping can move time; it cannot turn a genuinely different price move into the same move.
This is why a warping path is evidence, not a permission slip.
“Best candidate” is not “accepted match”
The first scanner version printed only the minimum.
That is enough for ranking, not classification.
Suppose the best monthly distance is 9.967796. Is that excellent, mediocre, or awful?
The answer needs context the DTW recurrence does not own:
- a labeled positive/negative corpus;
- a threshold selected against false-positive and false-negative cost;
- distance distributions for the same preprocessing and policy;
- perhaps a second-stage rule that inspects path shape, stretch count, or domain constraints.
A threshold copied from a synthetic fixture would be especially bad. The deterministic repository example reports 0.89; that does not make 1.50 a valid production cutoff for this Binance corpus.
A cutoff from the synthetic fixture does not survive a change of data distribution.
The updated article scanner therefore uses two passes and can export every candidate score.
The architecture is simple:
1. preprocess explicitly
2. scan every window distance-only
3. rank / apply a calibrated threshold
4. rerun selected windows with path tracking
No need to allocate an O(n×m) backtracking surface for every candidate and then act surprised when the pod gets OOMKilled.
The current lvlath/dtw API
The package exposes three canonical facades because “time series” is not one data type.
Align: scalar sequences
Use Align when one scalar represents one time step:
res, err := dtw.Align(
candidateReturns,
referenceReturns,
dtw.WithWindow(2),
dtw.WithSlopePenalty(0.05),
)
The default local cost is absolute difference. Callers can select squared cost or provide a deterministic custom CostFunc.
This surface fits returns, temperatures, single-axis sensors, latency traces, or other scalar signals.
AlignMatrix: multivariate features
Use AlignMatrix when each time step has several features:
res, err := dtw.AlignMatrix(
breakoutTemplate,
liveWindow,
dtw.WithWindow(4),
dtw.WithSlopePenalty(0.15),
dtw.WithReturnPath(true),
)
Rows are time steps. Columns are features. The two matrices must have the same feature dimension.
The facade builds pairwise local costs with squared L2 distance between rows.
It does not normalize the features.
Leaving preprocessing to the caller is an ownership decision.
OHLC, returns, z-scores, volatility-adjusted values, sensor channels, and embeddings are different mathematical inputs. A library that silently rescales them has changed the problem while pretending to save the caller a few lines.
AlignCostMatrix: bring your own local model
Use AlignCostMatrix when another system already produced the n×m pairwise cost surface.
The voice-command example feeds a 10×14 acoustic local-cost matrix. An embedding model, phoneme recognizer, frame matcher, or domain-specific rule engine could own the same layer.
DTW then does one job: find the cheapest admissible monotone path through that surface.
The local costs still have a contract. They must be finite and non-negative.
Window and slope are not decorative tuning knobs
The window
WithWindow(w) defines the Sakoe–Chiba band:
w = -1 no band constraint
w = 0 strict diagonal
w >= 0 only cells with |i-j| <= w
0 does not mean “disabled”. It means no temporal deformation at all.
A finite band blocks ridiculous alignments: the first phoneme should not match the end of a command; the onset of a vibration event should not match recovery.
The policy can also make alignment impossible. If the band contains no valid path:
Reachable=false
Distance=+Inf
That is a legitimate model result, not a panic.
The slope penalty
WithSlopePenalty(p) adds p only to horizontal and vertical steps.
Those are exactly the moves that hold one sequence while the other advances.
A positive penalty makes temporal stretching cost something. Zero leaves it free except for local mismatch.
Neither choice is universally correct.
A voice command may tolerate a held vowel. A machinery signature may treat an extra peak frame as suspicious. A market scanner may allow a one- or two-candle delay but reject a pattern smeared across half an hour.
The number belongs to the domain, not to DTW folklore.
The path is deterministic on purpose
lvlath/dtw returns one deterministic representative optimal path.
type Coord struct {
I int
J int
}
type Path []Coord
Several predecessor cells can have equal accumulated cost. The package uses a fixed backtracking order:
diagonal
then vertical/up
then horizontal/left
This does not claim the mathematical optimum is unique.
It means a stable distance also produces a stable explanation.
That matters when tests assert phase anchors, incident reports compare paths, or downstream logic counts stretch steps. Without a tie-break law, an internal refactor can leave the scalar unchanged and quietly rewrite the witness.
Ask for a path and the bill changes. It should.
The benchmark suite measures separate DTW contracts rather than pretending they are one workload.
On this machine:
darwin/amd64
Intel Core i9-9880H @ 2.30GHz
10 benchmark runs
The saved output did not include go version. Add the actual toolchain before publication; guessing it later would undermine the reproduction block.
benchstat reported:
| Contract | Workload | Time/op | B/op | allocs/op |
|---|---|---|---|---|
Scalar distance, TwoRows
|
500×500 |
2.169 ms | 8.156 KiB | 4 |
Reachable band, w=10
|
500×500 |
527.8 µs | 8.156 KiB | 4 |
Strict-diagonal no path, w=0
|
500×501 |
455.0 µs | 8.156 KiB | 4 |
FullMatrix + path |
250×250 |
1.232 ms | 1004.2 KiB | 9 |
AlignMatrix, squared L2, d=8
|
200×200 |
1.935 ms | 1007.3 KiB | 20 |
This is not a speed ranking. The dimensions and returned artifacts differ.
It does expose three useful facts.
First, the finite w=10 band ran about 4.1× faster than unconstrained scalar 500×500 distance on this fixture while keeping the same rolling-row allocation profile.
Second, the smaller 250×250 path workload allocated roughly 123× more bytes per operation than scalar 500×500 distance-only mode.
That is the cost of retaining accumulated state and recovering a witness. The lower runtime is not a contradiction; the path benchmark has only one quarter as many grid cells.
Third, AlignMatrix also sits around one megabyte per operation at 200×200×8 because the facade materializes the multivariate local-cost surface.
So the honest memory statement is not:
DTW uses O(m) memory.
It is:
Scalar distance-only
Aligncan keep accumulated DP state in O(m). Path recovery needs O(n×m), andAlignMatrixhas an additional O(n×m) local-cost surface.
Returning nil and making the caller play detective is lazy API design
The result carries the state needed to interpret it:
type Result struct {
Distance float64
Reachable bool
Path Path
PathTracked bool
Window int
SlopePenalty float64
MemoryMode MemoryMode
Accumulated *matrix.Dense
LocalCost *matrix.Dense
}
PathOrError() distinguishes:
nil result
→ ErrNilResult
no admissible alignment
→ ErrNoPath
finite alignment, path not requested
→ ErrPathNotTracked
A distance-only scan is a successful operation. It should not look like an alignment failure merely because no path slice was allocated.
A too-tight window is a different outcome. Caller logic may widen the band, reject the candidate, or record a policy failure.
Different decisions deserve different states.
Crypto is only one workload
The same result model appears in two other runnable examples.
Voice command
distance=0.83
path-steps=14
stretch-steps=4
decision=accepted with stretched pronunciation
The local acoustic-cost surface is computed elsewhere; AlignCostMatrix recovers the monotone correspondence.
Vibration signature
distance=0.0776
path-steps=17
stretch-steps=3
peak={6,8}
decision=signature match
Here AlignMatrix compares three sensor features per time step and accepts a delayed-but-normal spindle impact.
Other plausible uses follow the same shape:
- gesture and motion alignment;
- production-cycle timing;
- latency-profile comparison;
- ECG or respiratory phase alignment;
- video-frame or embedding correspondence;
- event-sequence template matching.
The API does not make those domains equivalent. It gives each one the same honest set of artifacts:
distance
path
local cost
accumulated cost
reachability
memory policy
Reproduce the real-data scan
The source dataset is the official Binance Spot monthly kline archive:
ETHUSDT
1-minute candles
July 2023
UTC
The article package includes the exact selected windows, path CSV, screenshots, and an updated two-pass scanner.
Run:
go run ./dtw_binance_real_scan_v2.go \
-csv ETHUSDT-1m-2023-07.csv \
-reference-start 2023-07-01T12:13:00Z \
-candles 11 \
-window 2 \
-slope 0.05 \
-exclude 30 \
-top 10 \
-out-prefix ethusdt-dtw
The scanner deliberately prints:
classification=not-configured
until the caller supplies a calibrated threshold.
That line is not missing functionality. It prevents the monthly argmin from being smuggled into business logic as a verified pattern match.
The useful boundary
A distance answers:
Which candidate is closest under this preprocessing and policy?
A path answers:
Which time steps correspond, and where did the clock stretch?
A local-cost matrix answers:
Which feature pairs created the mismatch?
An accumulated matrix answers:
Why did dynamic programming choose this route through the cost surface?
A threshold answers another question entirely:
Is this candidate good enough for my application?
One float should not pretend to answer all five.
Source and reproduction material
- Package: https://github.com/lvlath/go/tree/main/dtw
- Go Reference: https://pkg.go.dev/github.com/lvlath/go@v0.1.0/dtw
- Current API: https://github.com/lvlath/go/blob/main/dtw/api.go
- Current options: https://github.com/lvlath/go/blob/main/dtw/options.go
- Current result types: https://github.com/lvlath/go/blob/main/dtw/types.go
- Runnable examples: https://github.com/lvlath/go/blob/main/dtw/example_test.go
- Deterministic long-stream scanner: https://github.com/lvlath/go/blob/main/examples/dtw_signal_alignment.go
- Playground: https://go.dev/play/p/96eGjJQxauW
- Benchmarks: https://github.com/lvlath/go/blob/main/dtw/bench_test.go
Top comments (0)