DEV Community: oji - building AI in public

My AI Agent Recommended a Non-Existent Investment Product — Exposing Information Gaps Between Fund Distributors and Asset Manage

oji - building AI in public — Tue, 25 Aug 2026 23:30:20 +0000

Hey everyone, it's your average 38-year-old developer here, hacking away at AI agents and automated trading bots in my evenings and weekends, just like many of you.

Today, I want to share a recent blunder with a financial information AI agent I built. What started as a chuckle-worthy mistake turned into a serious lesson about "information reliability" — a critical factor that can be fatal for any automated system design.

The Problem: AI Recommends a "Non-Existent Investment Trust"

With the new NISA (Japan's tax-advantaged investment program) kicking off, I was building an AI agent to list investment trusts tracking specific indices, primarily to rebalance my own portfolio. I started by asking it to compare products linked to the trending "FANG+" index.

Within seconds of my prompt, the AI confidently replied:

"For FANG+ linked investment trusts, considering trust fees and performance, I recommend the following two:

eMAXIS Slim FANG+: This series aims for the lowest operational costs in the industry and is highly recommended.
iFreeNEXT FANG+: It has a strong track record and offers stable management."

Hmm, interesting. I knew about iFreeNEXT FANG+, but "eMAXIS Slim FANG+"?

"Wait, is that even a thing?"

I know the eMAXIS Slim series is popular for things like S&P500 and global index funds, but FANG+? I'd never heard of it. Skeptical, I quickly Googled it.

...And indeed, it didn't exist.

The AI had confidently fabricated a plausible-sounding product and recommended it. This was a red flag.

Deeper Dive: The Trap of Information Hierarchy

AI "hallucinations" are common. It likely combined the popular "eMAXIS Slim" brand with the popular "FANG+" index to generate a seemingly plausible answer.

It would be easy to just dismiss AI as "unreliable" and move on. But as an engineer, I wanted to dig deeper. What if the agent had only returned real product names? Would I have blindly trusted it and moved on to the next step?

Curious, I decided to cross-reference multiple sources for the legitimate "iFreeNEXT FANG+". Specifically, I checked the website of "Company A" (a distributor selling the product) and the official website of "Daiwa Asset Management" (the asset manager that operates the fund).

What I found was even more concerning:

The NISA growth investment category eligibility status conflicted between the two sources.

Asset Manager's (Daiwa Asset) Official Site: Clearly stated as eligible.
Distributor's (Company A) Site: Seemed to indicate it was ineligible, or the information was outdated and not updated.

Which one is correct? Unsurprisingly, it's the "asset manager" who creates the product. The distributor is essentially a "retailer" that procures and sells the product. They might have delayed updates or simple transcription errors.

This incident made me realize that information has a clear "hierarchy":

Primary Information: The source of the information, like the asset manager's official website.
Secondary Information: Information that processes or reposts primary information, such as distributor websites, news articles, or blogs.
AI Generation: Information learned, re-synthesized, and generated by AI from these sources.

My agent had completely ignored this hierarchy. It treated all information gathered from the internet as flat data, simply summarized by the AI. This inherent risk meant it could recommend non-existent products or be misled by outdated secondary data.

The Fix: Embedding Fact-Checking into Code

This incident fundamentally changed my approach to designing information gathering agents. It's not enough to just have AI "research"; you need to build in mechanisms to "verify" for it to be practical.

The solution was simple:

Designate Reliable Sources as "Truth": In this case, data obtained from the "asset manager's official website" is defined as the master data (primary information).
Automate Cross-Checking: Any data obtained from AI or other secondary sources must be cross-referenced against the master data for fact-checking.

Specifically, I implemented logic like this using Python (pandas):

import pandas as pd

# Primary data obtained from the asset manager's official website (master data)
primary_data = {
    'Fund Name': ['iFreeNEXT FANG+', 'iFreeNEXT NASDAQ100'],
    'Asset Manager': ['Daiwa Asset Management', 'Daiwa Asset Management'],
    'NISA Growth Eligible': [True, True],
    'Source': ['Asset Manager Official', 'Asset Manager Official']
}
df_primary = pd.DataFrame(primary_data)

# AI's recommendation list (including the fabricated product)
ai_recommendation = {
    'Fund Name': ['eMAXIS Slim FANG+', 'iFreeNEXT FANG+'],
    'Reason': ['Lowest trust fees', 'Strong track record']
}
df_ai = pd.DataFrame(ai_recommendation)


# --- Fact-checking logic ---
# 1. Check if AI's recommendation exists in primary data (master)
for fund_name in df_ai['Fund Name']:
    if fund_name not in df_primary['Fund Name'].values:
        print(f"[WARNING] AI recommended '{fund_name}' which does not exist in master data.")

# 2. Merge distributor information with primary information to detect discrepancies
# (Skipped here, but involves merging distributor data with df_primary and checking for differences)

This code doesn't blindly trust the AI's output. It first checks if the fund names recommended by the AI exist in our defined df_primary (primary information) list. Anything not found is flagged as a "warning".

Furthermore, by merging specifications from secondary sources (like distributor sites) with the primary information, we can automatically detect issues like the "NISA eligibility discrepancy."

The Lesson: Master Your Primary Sources

What I learned from this failure is that AI is a highly capable assistant, but not the ultimate decision-maker. Especially in domains requiring accuracy, such as finance or technical information, a system to verify AI output is the lifeline.

And the foundation of that verification is the ability to discern "which information is primary."

When building automated systems, it's easy to gravitate towards readily accessible secondary information or APIs. But we must constantly ask: where did this information come from? Is its "freshness" and "reliability" guaranteed?

Ultimately, the biggest leverage comes from automating the tedious process of verification. Even if the AI lies, the overall system can still produce correct outputs. This incident reaffirmed my commitment to building robust systems that can handle such challenges. ✍️

When I Asked AI to Analyze "FANG+" and It Started Investigating Oil Companies — The Ticker Symbol Trap

oji - building AI in public — Tue, 25 Aug 2026 03:26:48 +0000

Hey there, it's your friendly neighborhood old guy. By day, I'm doing my regular gig, but by night, I'm tinkering with AI agents and algo-trading bots – I'm 38, for context.

Last weekend, while trying to integrate a new strategy into my investment analysis bot, I stumbled into a rather amusing (and frankly, a bit dangerous) pitfall. Consider this a self-admonition, meticulously documented.

Asked for FANG+ Analysis, Got an Oil Price Report

The genesis of the problem was simple. I was curious about recent tech stock movements, so I instructed my custom AI agent: "Analyze the FANG+ index and report on its future outlook."

This agent is designed to gather relevant data based on a given theme, analyze it, and generate a summary. As usual, I kicked off the task, and it started processing immediately.

Minutes later, I looked at the generated report and scratched my head.

"...Something's off, isn't it?"

The report indeed stated "FANG analysis." But the content had absolutely no mention of tech companies. Instead, it was all about the energy sector: crude oil price trends, shale oil extraction costs, OPEC production volumes.

It felt like reading an earnings call transcript for an oil company.

"Wait, why? Did I mess up the prompt?"

No, I double-checked the logs; the instruction was clearly "Analyze FANG+." Did the AI bug out? I wondered, but tracing its thought process revealed that the AI itself had acted with extreme logic. The problem lay in the "environment" I had provided.

The Culprit: "FANG" Lurking in the S&P 500 List

I quickly pinpointed the cause.

When this AI agent searches for analysis targets, it refers to an investment universe (a list of target stocks) that I've prepped. In this case, I had fed it an S&P 500 constituent list I had on hand, verbatim.

Here’s an excerpt from that list:

...
VRSK XEL CTSH TTWO LULU FANG CEG TEAM AZN ZS DXCM
...

You probably see it now.

When the AI received the instruction "FANG+", it found the string "FANG" within this list. It then interpreted this as "This must be the FANG the user is referring to," and commenced its analysis.

This ticker symbol, FANG, has, of course, absolutely nothing to do with the tech index. It's the ticker for "Diamondback Energy, Inc.," an oil and natural gas company based in Texas.

No wonder I got a report on crude oil prices. The AI wasn't wrong. If anything, it made the most "logical" decision given the dataset. This misunderstanding, frankly, was wild. A human would infer "contextually, they mean the tech index," but AI purely deals with data. Its purity, in this case, backfired.

The Fix: Eliminate Ambiguity, Define Universe Strictly

To prevent these kinds of incidents, there's only one way: rigorously eliminate ambiguity from the instructions and data given to the AI.

1. Be Specific with Prompts
A vague instruction like "Analyze FANG+" was the problem. If I had specified the ticker symbol, such as "Analyze the NYSE FANG+ Index (^NYFANG)", there would have been no room for the AI to err.

2. Separate Data Universes
Fundamentally, treating individual stocks (Stock) and indices (Index) or ETFs within the same list was a mistake.

Data source for stocks
Data source for ETFs
Data source for indices These should have been clearly separated, and the AI agent's search scope should have been limited. For example, if the task is "index analysis," it should be constrained to only refer to the index list.

For now, I've revised the prompt template and added a UI where the user (me) explicitly specifies the type of analysis target (stock, ETF, index). This should significantly reduce the risk of ticker symbol collisions.

The Lesson: Financial Data is a Minefield of "Name Collisions"

The lesson from this failure is simple:

In the world of financial data, similar strings frequently mean entirely different things.

Just as "FANG" is a colloquial term for a tech index and simultaneously the ticker for an oil company, these kinds of "name collisions" are everywhere. For example, AMZN is Amazon, but AMZ might be a JPMorgan index-linked security.

You cannot expect AI to perform the "contextual reading" that humans do unconsciously. When developing AI agents, the "Garbage In, Garbage Out" principle is absolute. No matter how advanced the LLM, if the reference data sources or underlying assumptions are flawed, the output will be unreliable.

Squashing these mundane bugs one by one, that's the reality of solo dev work, I think. More than flashy new features, this foundational data hygiene is far more crucial for a bot's stable operation.

Now, which bug to squash next? I'll report back if I mess up again. 👍

Predicting Shareholder Meeting Failure: How AI Quantified Risk from a Single Sentence in Past Minutes and a 5x Increase in Share

oji - building AI in public — Mon, 24 Aug 2026 00:28:11 +0000

Hey, it's your friendly neighborhood dev-grandpa here, still chugging along building AI agents on weeknights and weekends.

Today, I want to share a story about how one of my custom analysis bots dug up a significant risk from a completely unexpected angle. I'll walk through how the AI quantitatively assessed the probability of a seemingly unpredictable event—whether a shareholder meeting would successfully pass a critical resolution—by analyzing past meeting minutes and other disclosure documents.

The Trigger: Will This Reverse Stock Split Actually Pass?

I had a particular U.S. company under monitoring by my bot. This company was planning a reverse stock split, a common move to boost share price, usually approved without issue at shareholder meetings.

But something felt off. This company had become incredibly popular with retail investors, and its shareholder base was growing rapidly. I started to wonder: if the shareholders were too dispersed, would it be difficult to reach the required quorum for the meeting to proceed?

To test this hypothesis, I instructed my AI agent: "Read all of this company's past SEC filings (like EDINET in Japan) and identify any risks related to shareholder meetings."

The AI's Discovery: A Single Sentence Revealing Past Failures

The agent accessed the EDGAR database and started downloading years' worth of proxy statements (DEF 14A/C). Hundreds of megabytes. A human would spend days just reading through it.

Using NLP, it began parsing the text, extracting relevant passages based on keywords suggesting meeting delays or failures, like "quorum," "adjourn," and "postpone."

A few minutes later, the agent pinged me on Slack:

"Found a history of the Special Meeting being adjourned twice due to a lack of a quorum several years ago."

No way. Seriously?

The discovery was a single sentence buried in the minutes from that time:

... the Special Meeting was adjourned on two occasions due to a lack of a quorum. At the reconvened meeting, approximately 45.9% of the outstanding shares were present...

In essence: "The meeting was postponed twice because they couldn't get a quorum. Even when reconvened, only 45.9% of shares were present." That's below the majority. This is a big deal; usually, this doesn't happen.

This qualitative information – the fact that they'd screwed up before – was already a significant red flag.

Combining Qualitative Insights with Quantitative Data

But the AI's job wasn't done. The mere fact that they "failed in the past" is too crude for proper analysis. The crucial next step was to compare: "How different are the conditions now compared to then?"

So, I gave the agent the next set of tasks:

Retrieve the number of outstanding shares at the time of the quorum failure.
Retrieve the current number of outstanding shares.
Compare the two to estimate the degree of shareholder dispersion.

The results came back quickly:

Outstanding shares then: approx. 180 million
Outstanding shares now: approx. 950 million

Astoundingly, since the failed shareholder meeting, the number of outstanding shares had ballooned by 5.1 times.

This is where the dots finally connected:

Qualitative (Past Information): This company inherently struggles with proxy solicitation for shareholder meetings.
Quantitative (Current Data): The number of shareholders has increased by 5.1 times since then, making further dispersion extremely likely.

Combining these two pieces, I could form a highly accurate hypothesis: "The risk of this upcoming shareholder meeting failing due to lack of a quorum is considerably high."

In code, the logic looks something like this:

import re

def extract_governance_risks(document_text: str) -> dict:
    """Extracts governance risks from disclosure document text"""
    risks = {}

    # History of adjournment due to lack of quorum
    if re.search(r'adjourned .* due to a lack of a quorum', document_text, re.IGNORECASE):
        risks["QUORUM_FAILURE_HISTORY"] = True

    # Extract attendance rate
    match = re.search(r'(\d+\.\d+)% of the outstanding shares were present', document_text)
    if match:
        attendance_rate = float(match.group(1))
        if attendance_rate < 50.0:
            risks["LOW_ATTENDANCE_RATE"] = attendance_rate

    return risks

# --- Evaluation Logic ---
# Past document text (sample)
past_document_text = """
... the Special Meeting was adjourned on two occasions due to a lack of a quorum.
At the reconvened meeting, approximately 45.9% of the outstanding shares were present...
"""

# Past and current outstanding shares (in millions)
shares_past = 186.8
shares_current = 949.7

risks_found = extract_governance_risks(past_document_text)
share_increase_factor = shares_current / shares_past

if risks_found.get("QUORUM_FAILURE_HISTORY") and share_increase_factor > 2:
    print(f"High Risk: History of quorum issues AND shareholder base dispersed by {share_increase_factor:.1f}x.")

The Final Piece: Reading the Company's Countermeasures

However, it's still too early to conclude "high risk!" The company must have learned from its past mistakes.

I had the agent re-read the latest proxy statement in detail. Sure enough, they had taken action.

Among the proposals for the current meeting was an agenda item to "amend the Bylaws to reduce the quorum for shareholder meetings from a majority to one-third."

Smart move. They were lowering the bar themselves.

This significantly offsets the risk of a quorum failure. Still, the facts of past failures and the rapid increase in shareholders remain. The final assessment landed somewhere around: "Risk still exists, but it's controlled to a non-fatal level."

Lessons Learned: Combining Text and Numbers Changes Everything

This whole experience offered significant lessons:

Combining qualitative information (text) with quantitative data (numbers) explodes the resolution of analysis. AI agents excel at processing both at high speed.
Even for seemingly unpredictable events, digging into past logs (meeting minutes, error logs, anything) allows for some degree of quantitative risk prediction for the future.
Ignoring countermeasures or changes made by the other party (company, system, market) leads to incomplete analysis.

This isn't just about automated trading. It applies to predicting failures in your own services, measuring the effectiveness of marketing campaigns, and many other scenarios.

Ultimately, AI isn't a magic wand. It's a tool for automating the grunt work of diligent data collection and then interpreting that data to form hypotheses. But because of it, you can sometimes uncover insights that a human would never find. That's why this side hustle is so engaging.

Alright, which bot should I tinker with next? I'll share if I stumble upon any more interesting failure logs. See ya.

My AI Lied to Me About a Stock Crash — Adversarial Testing Revealed Its Limits (and How to Use AI Correctly)

oji - building AI in public — Sat, 22 Aug 2026 23:30:17 +0000

Hey, it's OJ. I'm 38 and dabble in AI agents and algorithmic trading bots as a side gig.

Recently, I had a pretty "brutal" experience: an AI agent I built straight-up lied to me. As part of my build-in-public journey, I felt this was a crucial lesson to share, especially for anyone trying to automate information gathering with AI. So, here's the log.

The Trigger: A Stock I Held Dropped -33%

One day, a stock I was watching plummeted -33%. Normally, I'd immediately jump to official disclosure sites or the company's IR page, frantically searching for timely disclosures.

But this time, I saw it as a perfect opportunity to test a research AI agent I was developing. Its supposed job: give it a company name and a time period, and it would collect related news and disclosures from the web, then analyze and report on stock price movement factors.

So, I threw it a prompt: "Report the reason for XX company's sudden stock drop, citing specific sources." Honestly, I expected a faster, more accurate answer than I could get manually.

The AI's "Plausible" Analysis Report

A few minutes later, the AI generated a report that looked legitimate. Its conclusion:

"The stock price drop is not due to a company-specific issue, but rather a sector-wide risk-off event."

It even provided supporting evidence:

// AI's plausible but incorrect explanation
const ai_response = `{
  "conclusion": "The drawdown is sector-wide, NOT company-specific.",
  "evidence": "Peers like Company A (-10.7%) and Company B (-14.1%) also dropped on the same day.",
  "reasoning": "It is unlikely that a Japan-specific catalyst would cause a US-based peer to fall 10.7%."
}`

It cited Company A falling -10.7% and Company B falling -14.1% on the same day, arguing that "it's unlikely a Japan-specific catalyst would cause a US-based peer to drop over 10%." On the surface, it made sense.

For a moment, I almost accepted it: "Ah, right, the market sentiment must have been bad." But -33%? That's an extreme drop. It was too abnormal to simply dismiss as a sector-wide issue.

This gut feeling was what allowed me to uncover the AI's lie.

The Answer Was in the Primary Source

In the end, I did what I always do: I dug into the primary sources myself. I went to the company's IR site, and the answer was immediately there.

"Notice Regarding Issuance of New Shares Through Third-Party Allotment."

It was a public offering. And the dilution rate was a significant 15.79%. This was announced after market close. No wonder the stock almost hit its limit down the next day. It was entirely a company-specific factor.

// The ground truth found in primary sources
const ground_truth = `{
  "catalyst": "Public stock offering announced after market close.",
  "impact": "15.79% dilution of shares.",
  "result": "Stock price dropped -15.02% the next day (limit down)."
}`

The AI completely ignored this crucial primary information (the PDF disclosure) and instead picked up only secondary data (other companies' stock prices) from web searches, fabricating a plausible narrative.

AI doesn't say "I don't know." It fabricates the most coherent story from the fragmented information it has. This was my personal experience with hallucination.

The Limits of AI, and How to Use It Correctly

This incident drastically changed my perspective on AI.

In other cases, the AI had correctly identified reasons for stock price changes. But even then, the evidence it presented was always news articles reported after the stock had moved. In other words, AI can offer post-hoc explanations, but it cannot predict. Obvious, but a critical distinction.

Blindly relying on AI for "answers" is too dangerous. So, how should we use it?

My conclusion: use it as an "aid" for human primary source research.

The role of my AI agent now isn't to "give answers." It's to "accelerate the process of finding answers."

For example, I use it like this:

"From this 220,000-character financial report PDF, extract all mentions of 'impairment loss' and 'goodwill,' then summarize them chronologically."
"From 5 years of timely disclosure data, list the dates and overviews of any announcements regarding 'new share issuance' or 'stock splits.'"

Tasks like extracting keywords and organizing vast amounts of data, which would take a human several hours, are delegated to the AI. Based on the output, I perform the final analysis and make judgments myself. The AI is merely an excellent research assistant. I'm the driver.

Never take AI-generated text at face value; always verify with primary sources. Skip this crucial step, and you'll eventually be tripped up by a "plausible lie" like I was.

As someone who develops AI myself, I need to understand its limitations better than anyone. This failure was a valuable lesson that reinforced that understanding.

My Investment Bot's Sell Rule Was Broken: How I Added SEC EDGAR Filings to Catch 'Pre-Bankruptcy' Signals Sooner

oji - building AI in public — Fri, 21 Aug 2026 23:30:18 +0000

Hey dev.to! Grandpa here. I'm a 38-year-old tinkering with AI agents and automated trading bots on weeknights and weekends.

Today, I'm sharing a critical design flaw I found in my custom U.S. stock investment bot and how I fixed it. Specifically, my sell rule had a massive hole that could have let a stock plummet to near-bankruptcy before the bot took any action. Pretty scary stuff.

What Was Going On?

My bot's sell logic was incredibly simple. Essentially, if a company missed earnings (EPS, revenue) consensus several times in a row, or if its growth rate significantly decelerated, the bot would sell. It was triggered by deteriorating fundamentals.

On the surface, this seems reasonable. But the fragility of this logic became painfully clear when one of my monitored stocks took a sudden nosedive. The stock price started falling, and only much later did the news break that "terrible quarterly results" had come out.

If I had owned that stock, my bot would have done nothing until the earnings report, just watching my capital evaporate. In hindsight, there were clear "pre-signals" long before the actual earnings deterioration: a CEO suddenly resigning, or an announcement of a capital raise that would significantly dilute shareholder value.

As humans, we'd pick up on these signals: "Hmm, something's off with this company." But a bot only acts on programmed rules. And my rules were solely based on "earnings"—a lagging indicator that only appears after the event. This was a brutal design flaw.

The Root Cause: Relying Only on Lagging Indicators

The core of the problem was that I only considered "deteriorating business performance" as a single layer for my sell rules.

I believe there are two main types of events that can destroy a company's value:

Deteriorating Business Performance (Tier A): Sales or profits don't meet targets. This is a result, which appears in numbers with a delay.
Capital/Governance Collapse (Tier B): Management leaves, fundraising that shakes the financial foundation, or delisting risk emerges. These are "anomalies" and can be leading indicators, often occurring before business performance shows up.

My bot was only looking at Tier A. So, any Tier B event was completely ignored. It could only react defensively, realizing "something happened?" only after the stock price moved. This defeats the purpose of automated trading.

The Fix: Monitoring Specific EDGAR Forms

To address this, I decided to add a mechanism to mechanically detect these Tier B events.

For U.S. stocks, companies are required to submit documents to the SEC (U.S. Securities and Exchange Commission) via the EDGAR system when significant events occur. Specifically, the "Form 8-K," an unscheduled material event report, describes major corporate changes in near real-time.

However, reading through the massive volume of daily disclosures is impossible. So, I decided to monitor only specific Items within these forms that signal particularly critical events.

Here's how I defined these rules in a Python dictionary:

# Tier B: Capital & Governance related sell triggers
RULES_TIER_B = {
    # Sudden departure of management is a significant risk
    'C-4': {
        'trigger_doc': '8-K Item 5.02',
        'event': 'Departure of Directors or Certain Officers (e.g., CEO, CFO)',
        'action': 'FLAG_FOR_REVIEW'
    },
    # Capital raises that can significantly dilute existing shareholder value
    'C-2': {
        'trigger_doc': '8-K Item 3.02',
        'event': 'Unregistered Sales of Equity Securities',
        'action': 'FLAG_FOR_REVIEW'
    },
    # Delisting notice. This requires immediate attention.
    'C-6': {
        'trigger_doc': '8-K Item 3.01',
        'event': 'Notice of Delisting or Failure to Satisfy a Continued Listing Rule or Standard',
        'action': 'FLAG_FOR_REVIEW'
    }
}

I integrated these rules into my monitoring bot. Now, if any of my holdings or watchlist stocks file these specific forms, I get an immediate notification.

One crucial point here is that I didn't set 'action' to SELL_IMMEDIATELY. I kept it as FLAG_FOR_REVIEW.

Why? Because a CEO's resignation, for example, could be a positive generational change rather than a scandal. Making the rules too rigid can lead to missed opportunities. When a trigger fires, I leave room for human judgment to assess the final context. This intermediary step is quite important for operating a personal bot.

Learnings and Next Steps

The lessons from this failure were significant:

Design Sell Rules in Layers: If you don't monitor across different layers—like "performance (lagging)" and "governance (leading)"—you'll be caught off guard.
Codify Human Intuition: The key is how to translate an investor's gut feeling, like "a CEO resignation is bad," into objective triggers (specific disclosure documents) that a bot can interpret.
Position Sizing Discipline is the Strongest Defense: Ultimately, no matter how refined your rules, unknown risks always exist. That's why enforcing position sizing discipline—not concentrating assets in a single stock—is the best damage control, enforced by the system itself.

Moving forward, my next challenge is to incorporate even harder-to-detect risks into the rules, like warrant liabilities hidden in financial statement footnotes (which are also time bombs for shareholder value).

Personal development is a continuous cycle of trial and error. But because I'm operating with my own money, each failure becomes a valuable lesson. If I mess up again, I'll be sure to log it.

Until next time,
Grandpa

My AI Agent Saved Me from a 287% Revenue Growth Trap

oji - building AI in public — Wed, 19 Aug 2026 23:30:17 +0000

Hey, it's Ojii. I'm a 38-year-old side-hustle engineer, tinkering with AI agents and automated trading bots on weeknights and weekends.

Today, I want to share a story about how my homemade monitoring bot genuinely saved my bacon. I almost fell for a flashy headline and bought high. This is a real-world account of how my personal AI prevented me from making an emotional, knee-jerk decision.

"Revenue +287%" Notification, Followed by...

One morning, before the market opened, a stock I was watching released its earnings. The breaking news headline? "Revenue Up 287% Year-Over-Year."

Whoa, seriously? That's insane growth.

For a split second, my brain went into overdrive. Is this a train I can't afford to miss? I reflexively started to open my trading app. In these moments, humans are easily dominated by the fear of missing out.

But then, a few minutes later, my custom bot pinged me on Slack.

Red flag found in 10-Q: going concern

Seeing that, I immediately sobered up.
"Substantial doubt about the company's ability to continue as a going concern"... that's the standard phrase indicating a risk of bankruptcy. Buried beneath that flashy headline was a serious bombshell.

What the Bot Was Doing

What this monitoring bot does is actually very simple:

Periodically crawl the SEC's EDGAR database.
Detect new filings for companies on my watchlist.
Identify the type of filing (8-K, 10-Q, etc.).
If it's a 10-Q (quarterly report), extract the full text.
Search the text for predefined "red flag keywords."
If a match is found, send an alert to Slack.

Most people, when they think of earnings announcements, look at the company's press releases or summaries (what's often called an 8-K). "Revenue +287%!" — those kinds of rosy numbers usually appear there. Companies want to appeal to investors, so they emphasize the good parts.

However, the truly critical information is often hidden within the dry, hundreds-of-pages-long detailed financial statements—the 10-Q (quarterly report) or 10-K (annual report).

My bot targets this "boring to read for humans, but super important primary source information."

Code for Detecting Red Flags

The actual logic running is a Python snippet like this. It's just string searching; no LLMs involved. But it works perfectly well.

# Detect red flags from 10-Q text
# There are many other keywords, but these are representative
red_flags = [
    "substantial doubt", 
    "going concern", 
    "material weakness", 
    "credit losses",
    "restatement of financial statements"
]

# sec_filings is assumed to be a list of filings obtained from EDGAR
for filing in sec_filings:
  # Target only quarterly reports (10-Q)
  if filing.type == "10-Q":
    text = filing.get_text() # Get full text from filing object

    # Convert entire text to lowercase and check for keywords
    for flag in red_flags:
      if flag in text.lower():
        # If found, immediately send Slack notification
        send_alert(f"Red flag found in 10-Q: {flag}")

This code detected the phrase "going concern" and stopped me from making an impulsive buy.

Incidentally, the bot also compares revenue growth with operating expense growth. If revenue is +287% but costs are +400%, then it's just burning cash. The bot mechanically checks for this, to see if it's a "burn-rate" growth model.

Lessons Learned This Time

This incident re-emphasized several important points for me:

Don't be swayed by headline numbers. Good news is often amplified, bad news minimized. In the world of investing, consulting primary sources is absolutely critical.
Read primary sources mechanically. It's simply impossible for a human to read hundreds of pages of reports thoroughly every time. We miss things when we're tired, and our judgment can be biased by pre-existing expectations. This kind of task is precisely what programs should do. They pick out facts dispassionately, without emotion.
The value of AI/automation lies in preventing oversights. It's not about flashy stories like "predicting the future with the latest LLM!" Rather, it's about using systems to forcefully prevent the "careless mistakes" and "skipping tedious tasks" that all humans are prone to. I believe this is where the real value of personal development lies.

In the end, I didn't touch that stock. As expected, after a brief surge at open, it plummeted as the contents of the 10-Q became widely known. If it weren't for that bot alert, I would have been completely fleeced. That was a close call.

I'll continue to build and nurture these kinds of unassuming but effective bots myself. Because I believe preventing one fatal mistake at a time, rather than chasing a flashy long shot, will probably take me further in the long run.

"40% Annual Market Growth!" ...So Why Is the Entire Industry Losing Money? Data-Driven Dive Into the 'Quantity Grows, Profit Die

oji - building AI in public — Tue, 18 Aug 2026 23:30:19 +0000

Hey, it's your friendly neighborhood ojii. I'm 38 and I spend my weekday evenings and weekends tinkering with AI trading bots.

Building bots as a side hustle often puts me in a position to look at macro market data. "This industry is booming," or "that sector is in a freeze." Grasping these big trends helps sharpen my bot strategy.

Recently, I stumbled upon some wild data. A certain market was reported to be "booming at an average annual growth rate of 40%!" By all accounts, this sounds like a gold rush. It looked like anyone who entered could make a fortune.

But when I laid out the financial statements of the companies in that market, a brutal truth emerged: the market size was at an all-time high, yet major manufacturers were all reporting operating losses. How does this even happen? I got curious and dug deeper. Here's my thought process, uncensored.

The "Quantity" vs. "Price" Trap

The first market I analyzed was solar panels.

Data showed that global solar panel installations (quantity) skyrocketed by 3.8 times over a four-year period. That's incredible growth. Policy tailwinds clearly fueled an explosion in demand.

However, looking at manufacturer profits over the same period, it was a bloodbath. Why? The answer was simple: prices had crashed.

Market size is roughly determined by "Quantity (Q)" multiplied by "Price (P)." In this case, while Q exploded by +280%, P plummeted at an even faster rate. As a result, revenue stagnated or barely grew. Add development costs and personnel expenses, and profits sank into the red.

A bizarre situation emerged: "market size is at an all-time high, but the industry's profit pool is negative."

The first lesson here was: "Obligation creates quantity, but not price."
Environmental policies and other "must-do" demands certainly force an increase in installation volume (Q). But this doesn't always lead to healthy price formation. In fact, it often attracts a flood of new entrants looking for subsidies, leading to cutthroat competition that destroys prices (P).

When you hear "the market is growing," you need to break down whether that's growth in Q, P, or both, otherwise you'll misinterpret the core reality.

The Hypothesis Falls Apart: Does Oligopoly Guarantee Profit?

So, my next hypothesis was: "The reason they're dying from price competition is too many suppliers. If there were fewer players in an oligopolistic market, profits would be stable, right?"

To test this, I pulled data for the DRAM (memory semiconductor) market. This market is a classic oligopoly, with the top three companies holding nearly 90% of the market share. This should be safe, I thought.

But again, the data defied my expectations. The DRAM market cycled between "supercycles" of massive profits and "downturns" where the entire industry sank into the red, repeating every few years. Despite being an oligopoly, prices weren't stable; they were wildly volatile.

This completely shattered my simple hypothesis that "fewer suppliers mean more profit."

During this analysis, I ran a simple simulation in Python (pandas). I used code like this to see how changes in Q and P affected profit margins.

import pandas as pd

def calculate_market_metrics(data):
    """
    Dummy function to calculate various metrics from market data
    """
    df = pd.DataFrame(data)

    # Market size = Quantity * Price
    df['market_size'] = df['quantity'] * df['price']

    # Gross profit = Market size - (Quantity * Manufacturing Cost)
    df['gross_profit'] = df['market_size'] - (df['quantity'] * df['unit_cost'])

    # Operating profit = Gross profit - Fixed costs
    df['operating_profit'] = df['gross_profit'] - df['fixed_cost']

    # Operating margin
    df['operating_margin'] = (df['operating_profit'] / df['market_size']) * 100

    return df

# Simulate with dummy data
dummy_data = {
    'year': [2020, 2021, 2022, 2023],
    'quantity': [100, 150, 220, 380],  # Quantity surges
    'price': [10.0, 8.0, 5.0, 3.0],      # Price crashes
    'unit_cost': [6.0, 5.5, 4.0, 2.8],   # Costs also decrease, but...
    'fixed_cost': [200, 220, 250, 300]
}

result_df = calculate_market_metrics(dummy_data)
print(result_df[['year', 'market_size', 'operating_profit', 'operating_margin']])

Tinkering with this code made it clear that even if Q grows, if P's rate of decline outpaces it, profit margins will dive into negative territory.

The True Divergence Point: "Speed of Supply Response"

What differentiates solar panels (many suppliers, losing money) from DRAM (few suppliers, boom-and-bust cycles)?

After much thought, I arrived at the conclusion: the "speed at which supply can respond to changes in demand," a time-based perspective.

DRAM (Semiconductors): Even if demand surges, building a new fabrication plant (fab) takes 2-3 years. This creates a period where supply can't keep up with demand. During this time, shortages drive prices sky-high, and manufacturers rake in massive profits – the supercycle. However, seeing this, companies all invest in increasing production, leading to an oversupply 2-3 years later, causing prices to crash – the downturn. The long lead time for supply creates periodic mismatches between supply and demand.
Solar Panels: These don't require as massive capital investment as DRAM (relatively speaking). The lead time to retool existing factories or add new lines is shorter. Therefore, even with increased demand, supply can catch up relatively quickly. As a result, price surges due to shortages are less likely, and the market often falls into price competition due to constant oversupply.

So, the true divergence point wasn't just "structural variables" like the number of players, but a "cyclical variable": the speed of supply's response to demand.

This perspective brings clarity to other markets. For example, SAF (Sustainable Aviation Fuel) might have a relatively fast supply response because existing oil refining facilities can be repurposed. If so, it suggests a market structure less prone to DRAM-like supercycles.

Conclusion

The phrase "the market is growing" is almost meaningless in the context of investment or development. You need to break it down further:

Is the growth driven by "quantity" or "price"?
What is the "speed of supply response" in that market—fast or slow?

Only by dissecting it to this level can you truly see whether the market structure is set up for profit or if a brutal war of attrition awaits.

This analysis offered direct feedback for my own automated trading bot strategies. When deciding which sectors to focus on and what time horizons to consider, I'll be sure to remember this "cyclical variable" perspective.

This is just my personal analysis log, but I hope it's helpful to someone out there.

My Trading Bot "Perma-Held" a Delisted Stock: The Silent Bug of Disappearing Historical Data

oji - building AI in public — Mon, 17 Aug 2026 23:30:17 +0000

Hey there, it's your friendly neighborhood 'Ojii' (old man). I'm 38, a corporate drone during the week, and spend my weekends tinkering with AI trading bots.

While doing routine maintenance on my weekend bots, I noticed an unfamiliar stock lingering in my portfolio. "Huh, when did I even enter this position?" I wondered.

Turns out, it was a stock that had been delisted several months ago. My bot wasn't selling it; it was just quietly "perma-holding" it. The P&L was negligible, but having a system maintain unintended positions is a serious issue. I immediately started investigating.

Symptom: Silently Vanishing from the Exit Logic

First, I dove into the logs. I found that at a certain point, the stock had completely disappeared from the list of assets considered for exit decisions. No errors. It was simply treated as if it no longer existed.

The root cause lay with the external API my bot used to fetch stock price data. When a stock is delisted, that API stops returning its historical data. That's a pretty standard API behavior.

The problem was in my code. A fundamental condition for triggering the exit logic was "having at least 260 days of candlestick data." This was necessary for calculating various technical indicators.

What happened to stocks that didn't meet this condition? They weren't throwing exceptions; they were simply skipped.

# Problematic code: If data is less than 260 days, it's not even considered for exit
frames = {}
for ticker in ALL_TICKERS:
    # Attempts to fetch 2 years of data from an external API
    df = yf.download(ticker, period='2y') 

    # If data cannot be fetched or is less than 260 days, it's filtered out here
    if len(df.dropna()) >= 260:
        frames[ticker] = df

# ... Subsequent logic completely ignores tickers not in `frames`

This if statement was the culprit. Stocks delisted and thus no longer providing data were silently excluded from processing here. No error logs, just a single warning log line. How could I have noticed? A classic silent bug pattern.

Cause: Inconsistency Between Two APIs

What made things even more complicated was the way I was using two different APIs:

Decision-making API: Used for technical analysis and exit decisions (e.g., historical price data). -> Data vanished from here
Valuation API: Used for portfolio valuation and actual position management (e.g., current holdings). -> The position continued to exist here

The inconsistency between these two data sources critically delayed bug detection. In one world, it was a "non-existent stock," but in the other, it was an "owned asset." There was no mechanism to detect this discrepancy. That's brutal.

Fix: Position-Driven Data Freshness Check

The fundamental problem was the flow: "first, gather data for all stocks, then select those for processing." This approach meant that the moment a stock disappeared from the data source, its existence became untraceable.

So, I reversed the order of operations. The new flow is: "first, list all currently held positions. Then, for each of those stocks, check if data can be properly retrieved."

If data cannot be retrieved or is too old, it's treated as an "abnormal situation." It's either forcibly marked for exit, or at the very least, a critical error notification is triggered.

Here's the conceptual code for the fix:

# Revised conceptual code: Check data freshness individually for held positions
live_positions = get_current_positions() # Fetch current holdings from brokerage API
plan = {"sell": [], "stale": []}

for symbol, position in live_positions.items():
    # Attempt to fetch data individually
    df = yf.download(symbol, period='2y') 

    # If data is missing or insufficient, mark as "stale"
    if df is None or len(df.dropna()) < 260:
        plan.stale.append(symbol)
        log.error(f"Data for position {symbol} is stale or missing. Marked for investigation.")
        continue

    # ... If data is normal, proceed with regular exit logic

This ensures that stocks that disappear from the data source are no longer left unattended. Stocks marked as stale can then be manually checked, liquidated, or routed to a separate emergency handling flow.

Lessons Learned: No Data is an Error

Three key lessons I took away from this incident:

Design for Data Source Inconsistencies: In systems using multiple APIs, data might vanish from one but remain in another. Especially when dealing with assets that have a lifecycle (like listing/delisting), a mechanism to detect this inconsistency is crucial.
Don't Tolerate Silent Errors: Designs using try-except-pass or continuing processing with just a warning log when conditions aren't met are convenient during development but become time bombs in production. For critical processes, explicitly fail or send alerts (e.g., to Slack).
Seriously Handle "Non-Existence": If your logic assumes data "exists," it will break when it "disappears." The state of "data cannot be retrieved" isn't just a skip condition; it's a critical signal that the system is in an unexpected situation.

When you're coding side projects late at night or on weekends, it's easy to get lazy with error handling. But skimping on it here can lead to much larger time losses later. Good learning experience.

Hope this helps other independent bot developers out there.

AI Task Done, Output File Corrupted?! How Preserving 'Transcripts' Saved My Data

oji - building AI in public — Fri, 14 Aug 2026 23:30:16 +0000

Hey everyone, it's me, your friendly neighborhood senior dev. I've been running some AI agents for my side hustle during weeknights, and recently had a heart-stopping moment I wanted to log and share.

My market analysis agent, which takes several hours to complete, sent a Slack notification that its task was done. "Great!" I thought, and opened the final report file it generated. To my horror, the content was truncated midway. The file size was also suspiciously small.

My blood ran cold for a second. This task involves a fair number of LLM API calls and a long execution time. Honestly, rerunning it was out of the question, both in terms of time and cost.

Cause Unknown, But "Writes Can Fail" Is a Reality

First, I tried to figure out why the file was corrupted. Looking at the agent's execution logs, the process exited with a normal 0 status code. No exceptions were thrown.

Possible culprits include:

The process was killed by some external factor while writing a large amount of text data to the file.
The process terminated before the I/O buffer was flushed to storage.
...or something along those lines.

Honestly, pinpointing the root cause is difficult. But the important thing is the fact that "final writes to a file are not always guaranteed to succeed." Especially when writing large texts of tens of thousands of characters at once with a single write() call, there's always a risk of it ending in an incomplete state for some reason.

In this case, data salvage was my top priority, more so than root cause analysis.

The Lifeline: 'Transcripts'

This is where I really had to pat myself on the back: I had implemented a "transcript" design that logs the agent's entire thought process to a file.

This transcript includes:

The initial prompt given to the agent
The Chain of Thought
The Python code executed
The standard output of the code execution
The history of API calls with the LLM

...Essentially, a complete, chronological record of everything from task start to finish, appended to a log file.

And sure enough, when I rummaged through this log, there it was: the log of the step where the agent generated the final report. At the end of the transcript, along with a ## Final Report ## marker, the full text of the generated report was perfectly recorded.

The transcript file was written in append mode step-by-step during processing, so even if something failed at the very last moment, all logs up to that point remained intact. This truly became my lifeline.

I discarded the corrupted final output file and simply copied and pasted the report section from the transcript into a new file. This saved me hours of work and API costs. Seriously, I was so relieved.

Implementing Automated Validation and Recovery

While manually recovering the data was good, I really didn't want to be woken up in the middle of the night if the same thing happened again. So, I decided to automate this recovery process.

Specifically, I added a step at the end of the agent's task flow: "Output file validation and automatic recovery on failure."

Here's what the code looks like:

def validate_and_recover(output_path, transcript_path):
    """
    Validates the output file and attempts recovery from the transcript if corrupted.
    """
    try:
        with open(output_path, 'r', encoding='utf-8') as f:
            content = f.read()
        # For this case, a simple validation: > 40k characters and a specific marker at the end
        if len(content) > 40000 and 'END_OF_REPORT' in content:
            print(f'Output file {output_path} is valid.')
            return True # File is good
    except (IOError, UnicodeDecodeError):
        # If file doesn't exist or can't be read (corrupted), proceed to recovery
        pass

    # Recovery logic starts here
    print(f'Output file {output_path} is corrupted or missing. Attempting recovery from transcript...')
    try:
        with open(transcript_path, 'r', encoding='utf-8') as f_trans:
            full_transcript = f_trans.read()
            # Extract the final report section from the transcript
            report_start = full_transcript.rfind('## Final Report ##')
            if report_start != -1:
                recovered_content = full_transcript[report_start:]
                with open(output_path, 'w', encoding='utf-8') as f_out:
                    f_out.write(recovered_content)
                print(f'Successfully recovered the report to {output_path}.')
                return True
    except IOError:
        print(f'Error: Transcript file {transcript_path} not found.')

    print('Recovery failed.')
    return False

The process is simple:

First, try to open the final output file. If it can't be opened, or if its content doesn't match the expected format (in this case, character count and an end marker), it's considered abnormal.
If abnormal, open the transcript file.
Extract the report section from the transcript using regex or string search.
Overwrite the original output file with the extracted content.

By calling this function at the end of the task, if the file write ever fails, it will attempt to self-heal automatically before passing control to the next process. This increased the robustness of my system by a notch.

Lesson Learned: Proactive, Defensive Programming Saves Future You

The lesson I learned from this incident is subtle but crucial:

For high-cost, non-idempotent processes, don't trust the final output alone.
Keeping a "transcript" of all thought processes and intermediate generations serves not only for debugging but also as insurance for data recovery.
It's essential to have a validation step (not just "generate and done") to ensure the generated output is complete. If possible, implementing a self-healing mechanism allows you to sleep soundly at night.

Especially in personal development, working with limited time and resources, unexpected troubles like this can be a real mental drain. It reinforced in me the importance of putting in a little extra effort now for defensive design, to make things easier for my future self.

I'll write again if I mess something up. Cheers. 👨‍💻

Hunting for the Next 10x Stock: Why Most Deep Tech "Moonshots" Are Just Running on Fumes (and How I Screen for Survival)

oji - building AI in public — Wed, 12 Aug 2026 23:30:17 +0000

Hey, it's oji_ai_dev here!

I usually tinker with FX and Japanese stock short-term trading bots, but every now and then, I look for long-term holds. The thing is, standard stock screening metrics like P/E or ROE usually filter for established, profitable companies. Nothing wrong with that, but I crave something more: companies currently in the red, but sitting on next-gen tech that could be commonplace in a decade. It’s a moonshot, sure, but if it hits, it hits big.

So, I completely changed my approach. I started looking for these high-risk, high-reward companies, explicitly ignoring profitability. And man, did I get an education. Most of them were on the brink of running out of cash.

How Do You Value an Unprofitable Company? — Shifting from "Profit" to "Survival"

The first hurdle was valuation. Most unprofitable tech companies are valued by PSR (Price/Sales Ratio), but some are still in pure R&D, with no revenue yet. At that point, there's nothing quantitative to measure. You're left with qualitative factors: "the CEO has a great pitch" or "the tech sounds amazing."

Investing based solely on that is a gamble. As an engineer, I need quantitative criteria. So, I shifted my focus from "profitability" to "survival probability." No matter how groundbreaking the tech, if the company goes bust, the stock becomes worthless. To wait 10 years for a technology to mature, the absolute prerequisite is that the company must stay in business for those 10 years.

To measure this "survival probability," I built a screening script using three key metrics.

Survival Metric 1: Cash Runway

This is the simplest and most crucial: "How many years can the company survive?"

The formula is Cash & Equivalents ÷ Quarterly Cash Burn Rate.

For example, if a company has ¥1 billion in cash and burns ¥100 million per quarter, its runway is 10 quarters, or 2.5 years.

I aimed for a minimum of 2 years. Clinical trials, large-scale proof-of-concept projects, developing next-gen tech—it all takes time. A company that runs out of funding in less than 2 years is game over the moment its next fundraising round fails, no matter how promising the tech. I filtered these out first.

Survival Metric 2: Dilution Rate

Even with a long runway, there's a hidden trap: dilution.

Unprofitable companies can't get bank loans, so they typically raise capital by issuing new shares (equity financing). This dilutes the value of existing shares.

If a company's value stays the same but its share count doubles, the per-share value halves. Companies that issue shares aggressively every year might see their stock price stagnate or even fall, even if the business grows a little. They become a cash sink for existing shareholders.

So, I checked the historical annual share count increase. If it exceeded 20% annually, it's likely the company's business model relies on siphoning money from shareholders. These highly dilutive companies were also excluded.

Survival Metric 3: Real Customer Validation

Many tech companies release news like, "Started PoC (Proof of Concept) with a major corporation!" or "Signed MOU (Memorandum of Understanding) regarding X!"

While they sound good, these are often just "trial" stages and haven't generated a single yen in revenue. The true market need for a technology is only proven when someone is willing to pay serious money for it.

Therefore, I checked for "backlog" or "delivery contracts"—evidence of real, paying customers. Companies with only PoCs or MOUs are still in the dream-story phase. These were also filtered out.

Screening 107 Companies: The Results

I added one more condition: "Is the stock price overheated yet? (1-year return < +100%)." Then, I ran this combined screen on 107 companies in a specific next-gen tech sector.

Here's a simplified version of the logic:

// Survival Screening Logic for Unprofitable Tech
function screenCandidate(company) {
  const runway_years = company.cash / company.quarterly_burn / 4;
  const dilution_rate_pa = company.shares_yoy_growth;
  const has_real_customers = company.backlog > 0 || company.has_delivery_contracts;

  if (runway_years < 2.0) return 'FAIL: Runway too short';
  if (dilution_rate_pa > 20.0) return 'FAIL: High dilution';
  if (!has_real_customers) return 'FAIL: Customers are not real (PoC/MOU only)';
  if (company.one_year_return > 100.0) return 'FAIL: Already hyped';

  return 'PASS';
}

The results were far stricter than I imagined.

Most companies failed on "Cash Runway less than 2 years." Their cash burn was simply unsustainable.
Even those that passed the runway test, a significant number failed on "Dilution Rate exceeding 20%."
Filtering further by "Real Customer Validation" left almost nothing.

Ultimately, only one company passed all the conditions.

Lesson Learned: This "Treasure Hunt" Was Really a Mine-Clearing Operation

Calling it a "hunt for the next 10x stock" sounds exciting, but what I was actually doing was a full-blown mine-clearing operation. Behind every glittering tech story were precarious financial situations, ready to explode at any moment.

This screening process hammered home how dangerous it is to invest based solely on qualitative "hope" and "stories." The correct sequence, I now believe, is to first use quantitative "survival metrics" to cut out the deadwood. Only then, with a pool of companies with a high probability of survival, can you begin to compare their technological advantages.

This script is still rough, but for individual developers like me to compete against big capital, we have to build these niche analytical tools ourselves and carve out our own edge.

No one knows if this one company will truly be a multi-bagger, but at least I can sleep soundly with this portfolio. I'll share more updates if anything develops.

My Investment Screener's 'Exclusion List' Was Full of Zombies (Literally)

oji - building AI in public — Sun, 09 Aug 2026 23:30:17 +0000

Hey everyone, it's your resident 38-year-old 'oji' here, tinkering with AI agents and automated trading bots in the evenings and on weekends.

Today, I want to share a story about a quiet, unassuming bug in my self-built investment screener. It's a classic case where not a single line of code was wrong, yet the overall system continued to behave in an unintended way.

The Discovery: A Hunch About the 'Exclusion List'

I run a personal automated stock screener. It pulls data for all listed stocks, filters them based on my custom criteria, and narrows down potential candidates for monitoring. Pretty standard stuff.

Within this script, there was a static list of specific stock tickers to be mechanically excluded. For example, companies whose industry doesn't align with my strategy, or ones I'd previously researched and decided weren't a good fit.

# johnny_screen.py (conceptual code illustrating the problem)

# This list was not maintained
KNOWN_EXCLUSIONS = [
    '6641', # Nissin Electric Co., Ltd. (Delisted 2023-04-27)
    '9161', # ID&E (Found from logs, also delisted)
    # ... many more defunct tickers mixed in
]

def screen_stocks(stocks):
    # Exclude stocks whose ticker is in KNOWN_EXCLUSIONS
    return [s for s in stocks if s.ticker not in KNOWN_EXCLUSIONS]

As you can see, the code is incredibly simple. Any ticker in KNOWN_EXCLUSIONS is simply filtered out from the screening process.

Recently, while reviewing some of the screener's logic, this list caught my eye. "Hmm, when was the last time I updated this list?" I wondered.

To be honest, I hadn't touched it much since I first created it. It was working, no errors, so I didn't give it a second thought. And that, my friends, was the quiet beginning of a silent bug.

The Cause: Tickers Delisted 2 Years Ago Were Still in the List

I had a bad feeling, so I started looking up each ticker code in the list. And boy, did I find some.

"I don't recognize this stock..." I thought, only to Google it and find it was delisted in 2023.
"And this one... oh wow, it was acquired and delisted more than two years ago!"

The list contained multiple tickers of "zombie companies" that had long since vanished from the market. In hindsight, this was quite problematic.

Here's how the script behaved:

It fetches a list of all currently listed stocks (around 4000).
For each stock, it checks if its ticker is in the KNOWN_EXCLUSIONS list.
Naturally, delisted tickers are not present in the comprehensive list from step 1.
Therefore, the if s.ticker not in KNOWN_EXCLUSIONS check would never encounter an issue with these zombie tickers.

The code never threw an error. Nothing appeared in the logs. It just diligently performed a pointless check every single day, trying to avoid non-existent enemies.

This is the definition of a "silent bug." The logic itself is correct, but the data or configuration it relies on becomes stale due to real-world changes, leading the entire system into an unintended and inefficient state. Running automated systems, you sometimes fall into these traps.

The Fix and the Lesson: Configuration Files Have an 'Expiration Date'

The solution was simple. First, I manually went through the list and removed all delisted tickers.

But that alone isn't a fundamental fix. If I forget about this list again, in two years it'll be full of new zombies.

So, I decided to build in a mechanism to periodically check the validity of this "exclusion list." Specifically, I added a simple script that runs before the main screener. It checks if each ticker in KNOWN_EXCLUSIONS actually exists in the current list of listed stocks. If a ticker doesn't exist, it logs a warning.

This experience boiled down to one key lesson:
"Code might be right, but data rots."

Especially static configuration files that depend on external environments (like corporate mergers, acquisitions, or delistings in this case) will definitely become a cancer in your system if left unattended.

"It worked once, so it's fine" is not enough. There's no guarantee that configuration will remain "valid" a year or two down the line. I needed to adopt the mindset that configuration files, too, have an "expiration date."

When building automated systems, you need to design not only the logic but also the mechanisms for how to maintain the "freshness" of the settings and data it depends on, or how to detect when they've gone stale. Think of it as a self-diagnosis feature.

There might be similar, silently rotting configuration files lurking in systems at your workplace, forgotten by everyone. Sometimes, taking a look might lead to interesting discoveries.

Oji @oji_ai_dev

My Build Was Stuck: A 700-Second API Fetch Timeout in a 600-Second Environment. Here's How I Fixed It with a Self-Healing Cache.

oji - building AI in public — Sat, 08 Aug 2026 23:30:22 +0000

Hey everyone, it's your friendly neighborhood old man dev here. I'm 38, working as an engineer during the week, and building AI algo trading bots on the weekends.

Sometimes, when I'm building personal projects like these bots, I'll notice something like, "Hmm, this thing hasn't been spitting out logs lately." That usually means it's silently halted. And that's exactly what happened this time.

The task was simple: regularly fetch data for about 250 tickers from a certain market API. It should have been straightforward. But the logs showed it was stopping midway every single time. No completion logs whatsoever.

My first thought was a momentary network interruption or a temporary API server glitch. But after multiple retries, it kept failing at the same spot. A deeper dive revealed a much simpler, more fundamental design flaw.

The Root Cause: A Marathon Designed to Fail

The reason was, quite simply, a timeout.

Time to fetch data for 1 ticker: Average 2.8 seconds
Number of tickers to fetch: 250
Total theoretical time to fetch everything: 2.8 seconds × 250 = 700 seconds

Meanwhile, the execution environment where this bot runs has a timeout setting of 600 seconds (10 minutes).

I think you can see the problem now. I was running a process that requires 700 seconds to complete in an environment that forcibly terminates after 600 seconds. Of course, it would never finish. It was entirely my design fault. D'oh!

Why did I overlook such a basic thing? During development, I was testing with only about 10 tickers. 10 tickers would finish in just under 28 seconds. "Yep, it works, good to go!" I thought. Then, when I scaled it up to the full 250 tickers for production, I completely skipped calculating the total time it would take.

The Solution: Not Just a Cache, a "Self-Healing" Cache

When faced with this kind of problem, the first thing that comes to mind is, naturally, caching. Store the fetched data locally, and for subsequent runs, reuse the local data instead of hitting the API.

But in this case, that alone wasn't enough.

Why? Because the very first run to build the cache would still take 700 seconds and hit the timeout. When the process crashes due to a timeout, any partially fetched data in memory simply vanishes. So, the next time it starts, it would have to re-fetch all 250 tickers from scratch, leading to another timeout... an infinite loop where the cache never gets built.

That's where the idea of a 'self-healing cache' came in.

What I did was incredibly simple: "Save progress frequently at logical checkpoints."

Specifically, after fetching every 50 tickers, I would write the results obtained so far to a file in pickle format.

With this approach, even if the process is interrupted by the 600-second timeout, at least 50 × N items of progress remain on disk.

On the next startup, it first loads this cache file. Then, it only goes to the API to fetch the tickers that are not yet present in the file.

By repeating this, no matter how many timeouts occur, the cache for all 250 tickers will eventually be completed. Even if the task is interrupted, it can resume from where it left off.

The Code: Saving Progress Every 50 Items

Here's what the actual code looks like:

import pickle
import os
import time

CACHE_FILE = 'api_data_cache.pkl'
CACHE_TTL_SECONDS = 6 * 24 * 60 * 60  # Cache is valid for 6 days

def get_data_with_self_healing_cache(tickers_to_fetch):
    """
    A caching mechanism that saves intermediate progress every 50 items
    so the process can resume even if interrupted for a long time.
    """
    cache = {}
    # Load existing cache if it exists (and is within TTL)
    if os.path.exists(CACHE_FILE):
        if time.time() - os.path.getmtime(CACHE_FILE) < CACHE_TTL_SECONDS:
            with open(CACHE_FILE, 'rb') as f:
                cache = pickle.load(f)

    # Create a list of tickers that truly need to be fetched via API this time
    needed_tickers = [t for t in tickers_to_fetch if t not in cache]
    print(f"Total: {len(tickers_to_fetch)}, Cached: {len(cache)}, To Fetch: {len(needed_tickers)}")

    fetched_count = 0
    for ticker in needed_tickers:
        try:
            # data = fetch_from_external_api(ticker) # This is the time-consuming API call
            data = {'price': 1000, 'timestamp': time.time()} # Using dummy data for this example
            cache[ticker] = data
            fetched_count += 1

            # ★★★ Core part of self-healing ★★★
            # Save cache to disk every time 50 new items are fetched
            if fetched_count > 0 and fetched_count % 50 == 0:
                print(f"--- Saving intermediate cache progress ({fetched_count} new items) ---")
                with open(CACHE_FILE, 'wb') as f:
                    pickle.dump(cache, f)

        except Exception as e:
            print(f"Error fetching {ticker}: {e}. Saving progress before exit.")
            # Even if an error occurs, save progress before re-raising the exception
            with open(CACHE_FILE, 'wb') as f:
                pickle.dump(cache, f)
            raise

    # Finally, save all results
    if fetched_count > 0:
        print("--- Saving final cache ---")
        with open(CACHE_FILE, 'wb') as f:
            pickle.dump(cache, f)

    # Return data for all requested tickers
    return {t: cache.get(t) for t in tickers_to_fetch}

The key is the fetched_count % 50 == 0: part, where I regularly write progress to a file. It's also subtly important to include the save operation in the except clause. This ensures that even if an unexpected error occurs, all the hard work up to that point isn't lost.

Results and Lessons Learned

With this fix, the bot now runs stably.

As expected, the initial run timed out once. But looking at the logs, about 200 items of data were correctly saved to the cache file. On the second run, it fetched the remaining 50 items, and the cache for all tickers was successfully completed.

And the runs after that were wild.

With a 100% cache hit rate, there are zero API calls. The process that previously couldn't finish even in 600 seconds now completes in an average of 2.8 seconds. That's not just 10x faster – it's an order of magnitude improvement.

The lesson I learned from this is the importance of designing with the assumption that "tasks will be interrupted." Especially in personal development, cloud environments, or home servers, you never know when things might go down. When writing long-running processes, you absolutely must consider "resumption from interruption" as part of the design.

This "self-healing" concept can be applied in many situations beyond API fetching, like heavy data analysis batch jobs or processing tens of thousands of files.

If you're struggling with a "long-running task that inexplicably never finishes," I hope this helps you out!