My Stock Buyback Bot Was Crushing It... Thanks to 220 Phantom Signals from Stock Consolidations

Hey there, it's your friendly neighborhood Old Man Dev. (That's my online persona, not my actual age – mostly.)

I spend my weekday evenings and weekends building AI agents and automated trading bots. Today, I want to share a massive backtesting pitfall I recently stumbled into. The TL;DR? A strategy that looked incredibly profitable turned out to be a mirage, all thanks to unadjusted data.

The Glitch in My "Cannibal Strategy" Backtest

I'd been developing a strategy to invest in companies that consistently reduce their outstanding shares – what I call "Cannibal" stocks. When a company buys back its own shares from the market, it tends to increase the value per share. It’s a pretty straightforward concept.

I ran a backtest using 10 years of historical data, and to my surprise, it showed exceptionally good performance. Honestly, I was pretty excited, thinking, "Hey, this might actually be a winner!"

But as I dug into the transaction logs, something felt off. Around 2018, in particular, there was an unnatural surge of buy signals, concentrated in a short period. It didn't make sense for so many companies to start aggressive buybacks all at once when the market wasn't experiencing any major systemic shifts. This inconsistency was my first clue.

The Culprit: Stock Consolidations

The logic for my Cannibal strategy was simple: "If outstanding shares decreased quarter-over-quarter, issue a buy signal." My bot interpreted every instance of a reduced share count as a "stock buyback."

And that was the mistake.

Stock buybacks aren't the only reason for a decrease in outstanding shares. There's another significant event: stock consolidations (reverse stock splits).

For example, if a company announces a "2-for-1 stock consolidation," its outstanding shares are simply cut in half. But this only means shareholders now hold half the number of shares; the company's overall value hasn't changed. And crucially, it's not a stock buyback.

My bot, bless its heart, misinterpreted every share reduction due to consolidation as an "ultra-massive stock buyback," leading to a flurry of false buy signals.

So why the concentration around 2018?

A quick investigation revealed the answer: In October 2018, the Tokyo Stock Exchange completed a system change to unify trading units for all listed companies to 100 shares. Companies that previously traded in 1,000-share units, for instance, often performed consolidations (e.g., "10 shares into 1 share") to align with the new 100-share standard.

This wave of regulatory-driven stock consolidations made those companies look like the "best Cannibal stocks" to my bot. Brutal.

Eliminating 220 False Positives

Once the cause was identified, the fix was straightforward. I added a process to fetch corporate action data (history of stock splits and consolidations) and correctly adjust the time-series data for outstanding shares.

Instead of simply comparing the current and previous period's share counts, I now check for any consolidations or splits in between. If found, I adjust the previous period's share count proportionally before making the comparison. It's a fundamental data cleaning step that I had completely overlooked.

// The Cannibal implementation had a critical bug: unadjusted stock split/consolidation data.

// A 1:2 stock consolidation halves share count without any buyback, leading to false positives.

// Consolidation adjustments removed 220 false positives (25-30% of signals), aligning with the TSE's 2018 100-share unit standardization rush.

After implementing this fix, 220 false positive signals vanished from my backtest. That's a whopping 25-30% of the original signal set. Crazy. The performance after removing this noise was, unsurprisingly, nowhere near the illusory gains I initially saw.

Key Takeaways from This Fiasco

I learned three critical lessons from this experience:

1. Fundamental Data Needs Adjustment Too: It's common practice to adjust stock price data for splits. But fundamental data like outstanding shares also needs similar corporate action adjustments; otherwise, your results will be completely skewed, as mine were.
2. Regulatory Changes Are Minefields: Always be aware of how market-wide regulatory changes, like the 2018 trading unit unification, might impact your backtests. These macro events can unexpectedly break the continuity of your data.
3. Always Suspect Good Results: This might be the most important lesson. When a backtest shows unusually good results, you must question it: "Why is this working so well?" If I had blindly deployed this to live trading based on premature celebration, I could have been burned badly.

So, my Cannibal strategy is back to square one. But finding and squashing this critical bug myself feels like a significant step forward.

Working on bot development as a solo side-hustle often means falling into these kinds of traps and climbing out on your own. I hope this post helps anyone else doing automated trading or data analysis. Until next time!