DEV Community

Cover image for The Trap of Stale Docs: How I Almost Rewrote a "Rejected" Trading Strategy I Found in an Old Book
oji - building AI in public
oji - building AI in public

Posted on

The Trap of Stale Docs: How I Almost Rewrote a "Rejected" Trading Strategy I Found in an Old Book

#ai #automation #buildinpublic #python

Hey, it's ya boi, the developer dad. I'm 38, an engineer at an IT company in Tokyo by day, and a humble builder of AI-driven trading bots by night. My dev time usually kicks off around 10 PM, right after the kids are tucked in. This particular evening, I was doing my usual thing, tweaking an FX bot that's been in operation.

The next item on my task list: refining the exit strategy (take profit/stop loss rules). I had a promising idea brewing, so I opened up my dev docs.

I have a file called CHECK_LOGIC.md where I jot down ideas for trading methods and potential validation candidates. Buried in there, I found this line:

G8: 10xATR Trailing Exit Strategy ◎Evaluated, Not Implemented

This was a strategy I'd come across in a popular investment book, one that dynamically adjusts the stop-loss line based on volatility. When I did my literature review, I marked it with a "◎" (meaning "looks promising") and added it to my backlog.

"Alright, let's tackle this one tonight," I thought.

I opened my editor, fingers poised to start coding, when a powerful sense of déjà vu washed over me.

"Wait... didn't I backtest this logic before?"

I tried to recall, but the memory was elusive. Still, something nagged at me. These gut feelings are usually right. I paused my development and started digging through my old validation logs.

The "Almost Broke Even" Validation Log

My personal dev logs are like my own archaeological dig. A few weeks back, I'd uncovered a promising file:

prereg_runner_exit_batch2_2026-08-05.md

The filename suggested it was a batch result from pre-validating an exit strategy. I opened it and gasped.

// prereg_runner_exit_batch2_2026-08-05.md:52
// E8 = 0.97〜0.99 → ❌ "Barely breaks even on median" 

// G8 in CHECK_LOGIC.md still quotes the "◎Evaluated, Not Implemented" from the literature review 
// and does not reflect the subsequent rejection based on actual measurements.
Enter fullscreen mode Exit fullscreen mode

There it was. I'd completely forgotten.

"E8" was the validation ID for that very 10xATR trailing strategy. The profit factor (total profit ÷ total loss) was a dismal 0.97 to 0.99. This meant that the more trades I made, the more money I'd lose. Yikes.

And to add insult to injury, there was a neat handover note for my future self: "CHECK_LOGIC.md doesn't reflect these results." My past self had genuinely saved my present self. I was seconds away from spending two hours implementing a logic I'd already sent to the trash bin.

Why Did a "Rejected" Task Resurface?

The reason is simple: failure in document freshness management.

My dev process usually goes like this:

  1. Literature Review: Read books and papers, jot down promising strategies in CHECK_LOGIC.md with notes like "◎Evaluated, Not Implemented."
  2. Backtesting: Validate the noted strategies with my own data, record results in a separate file.
  3. Implementation: Only incorporate strategies with good validation results into the live bot code.

This time, after step 2 produced a "barely breaks even" result and I rejected the strategy, I forgot to feed that result back into the step 1 document, CHECK_LOGIC.md.

As a result, only the old information, "◎Evaluated, Not Implemented," persisted.

The word "Not Implemented" itself was tricky. I misinterpreted it with a positive nuance, thinking it was a valuable task still to be done, when in fact, it was an already rejected idea.

In solo dev, without peer reviews, such document inconsistencies can easily sit unnoticed for weeks or months. That's dangerous.

The Discipline of Differentiating "Not Implemented" and "Rejected"

To prevent this kind of rework, I decided to revise my document status management rules.

I used to use the ambiguous term "Not Implemented." I've abolished it. Instead, I'll use clearer statuses:

  • [TODO]: Tasks to be validated/implemented.
  • [DONE]: Tasks validated and implemented.
  • [REJECTED]: Tasks judged as unfavorable (rejected) after validation.

That's it. If I open CHECK_LOGIC.md and see [REJECTED] 10xATR Trailing Exit Strategy, there's no way I'll ever think, "Should I implement this?" It's crystal clear at a glance.

# (Reference) Excerpt from actual validation code.
# This is how I calculated PF by varying parameters.
def run_backtest(df, atr_multiplier=10.0):
    # ... omitted for brevity ...
    total_profit = result['profit'].sum()
    total_loss = abs(result['loss'].sum())

    if total_loss == 0:
        return float('inf') # PF is infinite if no losses

    profit_factor = total_profit / total_loss
    return profit_factor

# Validation result:
# pf = run_backtest(my_data, atr_multiplier=10.0)
# print(f"Profit Factor: {pf:.2f}") # -> Profit Factor: 0.98
Enter fullscreen mode Exit fullscreen mode

Lessons Learned

There are two main things I learned from this screw-up.

First, methods deemed effective in academic papers or literature are not absolute. There's no guarantee they'll work the same way in my market, my timeframe, or with my dataset. The extra step of validating them myself, in my own environment, is crucial to prevent fatal losses.

Second, documents should serve as "monuments of judgment." Especially in personal development, a record of why something wasn't pursued—a record of rejection—becomes a lifeline that protects your future self's time. "Not implemented" and "rejected" are worlds apart in meaning. Skipping this distinction will lead to repeating the same mistakes and burning through your already limited side-hustle hours.

It was incredibly lucky that I caught this right before implementation. Past self, thank you for leaving those logs.

Alright, time to reset and start validating another exit strategy.

Top comments (0)