DEV Community

finaltype
finaltype

Posted on Originally published at finaltype.github.io

Redefining the Performance Benchmark - Replacing a Period-Mismatch Illusion With a Stock-Selection Metric

My live-account report looked like it was badly trailing the index, but that was an artifact of measuring the numerator and denominator over different periods. After redefining the benchmark as a per-lot like-for-like comparison, an honest measurement surfaced a more uncomfortable question — the stock-selection edge is essentially zero.

This is the English version of a post originally written in Korean for my algorithmic trading system devlog(new tab).

The numbers against the index looked far too bad

As the market kept falling, the "vs benchmark" line in my daily live-account performance report(new tab) was printing a strikingly large negative figure.

The overall account loss already stung, but if I was also trailing the index by a wide margin, that would mean my stock picking itself was terrible.

So I handed the question — "why does only the live account look this bad against the index?" — to another AI as a root-cause investigation.

It was an illusion — the numerator and denominator spanned different periods

The conclusion was that this wasn't a difference in skill. It was an artifact of measuring two numbers on two different bases.

The report's "total return" was the full profit and loss accumulated over two months. It included realized losses from positions already sold and closed — a long-horizon cumulative figure.

The "index" number sitting next to it, however, only weighted the index movement from the moment I bought each currently held position up to today. Since much of my buying was concentrated recently, that window was far shorter.

So the numerator spanned "the full two months" while the denominator spanned "the last few weeks." In a falling market, the gap between those two is structurally bound to blow out. The index declines that the already-sold positions lived through were missing from the denominator entirely.

When I recomputed everything on the same window, same index, and same NAV definition, the live account came out roughly in line with my pre-registered benchmark. Most of the loss came not from stock selection but from the broad market decline — beta.

Redefining the benchmark as a "stock-selection effect"

The original definition of this metric was something I'd specified a few months back, so changing it was my call too. Once I'd confirmed the cause of the illusion, I decided to redefine the "vs benchmark" line entirely as a stock-selection effect.

The new definition is a per-lot, apples-to-apples comparison. For each buy lot, it compares that stock's return against the index movement over exactly the same holding period. Entry and exit are anchored to the actual round timestamps the orders went out on, and it rolls up both already-sold lots and still-held lots on a cost-weighted basis.

This isolates the one thing I actually want to see: on top of the market (beta), did the stocks I picked beat the index? The period-mismatch illusion disappears.

After implementing it, I wrapped the edge cases in tests — the example case the owner raised, order-timestamp handling, FIFO lot handling, an empty ledger. I then confirmed that the value computed against the real live ledger matched an independent recomputation done in a scratchpad outside the repo, and deployed. I also restarted the resident processes that import this metric's code.

Measuring honestly left a more uncomfortable question

What I faced after fixing the benchmark was a harder question.

Separately that same day, I asked another AI advisory to lay out whether it even makes sense to run this system around the clock for months, once you factor in electricity, hardware wear, and my own effort.

Measured honestly, the stock-selection edge was essentially zero. The lots that beat the index and the lots that lost to it were close to an even split — statistically indistinguishable from a coin flip. Splitting it into up and down regimes gave the same conclusion.

And nearly all of the loss was the market falling — beta. The current design is long-only and stays almost fully invested, leaving barely any cash. A structure like that simply cannot produce an absolute (positive) return in a falling market.

What this exposed is that it isn't a problem you fix by tuning the signal. The goal of "positive even when the index falls" and the design principle of "don't leave cash idle" are fundamentally incompatible. Choosing between them is a decision for me to make, not something code can resolve.

Why it matters

Today's core lesson is that measuring your own performance honestly is harder than it sounds, and getting it wrong leads you to the wrong conclusion entirely.

A badly aligned benchmark made the system look far more incompetent than it was in a down market. Conversely, some metrics can trick you into reading "lost less because it was defensive" as "did well," even with no real skill.

Only after fixing the metric could I see the real question. It isn't about shaving the signal further — it's whether running the system at this structure and this scale is justified on return logic at all, or whether it should be treated as a research and learning asset with a budget attached.

Fixing the measurement is tedious, but skip it and every judgment you make on top of it stacks on an illusion. Today was about relaying that foundation.

Top comments (0)