My AI Lied to Me About a Stock Crash — Adversarial Testing Revealed Its Limits (and How to Use AI Correctly)

Hey, it's OJ. I'm 38 and dabble in AI agents and algorithmic trading bots as a side gig.

Recently, I had a pretty "brutal" experience: an AI agent I built straight-up lied to me. As part of my build-in-public journey, I felt this was a crucial lesson to share, especially for anyone trying to automate information gathering with AI. So, here's the log.

The Trigger: A Stock I Held Dropped -33%

One day, a stock I was watching plummeted -33%. Normally, I'd immediately jump to official disclosure sites or the company's IR page, frantically searching for timely disclosures.

But this time, I saw it as a perfect opportunity to test a research AI agent I was developing. Its supposed job: give it a company name and a time period, and it would collect related news and disclosures from the web, then analyze and report on stock price movement factors.

So, I threw it a prompt: "Report the reason for XX company's sudden stock drop, citing specific sources." Honestly, I expected a faster, more accurate answer than I could get manually.

The AI's "Plausible" Analysis Report

A few minutes later, the AI generated a report that looked legitimate. Its conclusion:

"The stock price drop is not due to a company-specific issue, but rather a sector-wide risk-off event."

It even provided supporting evidence:

// AI's plausible but incorrect explanation
const ai_response = `{
  "conclusion": "The drawdown is sector-wide, NOT company-specific.",
  "evidence": "Peers like Company A (-10.7%) and Company B (-14.1%) also dropped on the same day.",
  "reasoning": "It is unlikely that a Japan-specific catalyst would cause a US-based peer to fall 10.7%."
}`

It cited Company A falling -10.7% and Company B falling -14.1% on the same day, arguing that "it's unlikely a Japan-specific catalyst would cause a US-based peer to drop over 10%." On the surface, it made sense.

For a moment, I almost accepted it: "Ah, right, the market sentiment must have been bad." But -33%? That's an extreme drop. It was too abnormal to simply dismiss as a sector-wide issue.

This gut feeling was what allowed me to uncover the AI's lie.

The Answer Was in the Primary Source

In the end, I did what I always do: I dug into the primary sources myself. I went to the company's IR site, and the answer was immediately there.

"Notice Regarding Issuance of New Shares Through Third-Party Allotment."

It was a public offering. And the dilution rate was a significant 15.79%. This was announced after market close. No wonder the stock almost hit its limit down the next day. It was entirely a company-specific factor.

// The ground truth found in primary sources
const ground_truth = `{
  "catalyst": "Public stock offering announced after market close.",
  "impact": "15.79% dilution of shares.",
  "result": "Stock price dropped -15.02% the next day (limit down)."
}`

The AI completely ignored this crucial primary information (the PDF disclosure) and instead picked up only secondary data (other companies' stock prices) from web searches, fabricating a plausible narrative.

AI doesn't say "I don't know." It fabricates the most coherent story from the fragmented information it has. This was my personal experience with hallucination.

The Limits of AI, and How to Use It Correctly

This incident drastically changed my perspective on AI.

In other cases, the AI had correctly identified reasons for stock price changes. But even then, the evidence it presented was always news articles reported after the stock had moved. In other words, AI can offer post-hoc explanations, but it cannot predict. Obvious, but a critical distinction.

Blindly relying on AI for "answers" is too dangerous. So, how should we use it?

My conclusion: use it as an "aid" for human primary source research.

The role of my AI agent now isn't to "give answers." It's to "accelerate the process of finding answers."

For example, I use it like this:

• "From this 220,000-character financial report PDF, extract all mentions of 'impairment loss' and 'goodwill,' then summarize them chronologically."
• "From 5 years of timely disclosure data, list the dates and overviews of any announcements regarding 'new share issuance' or 'stock splits.'"

Tasks like extracting keywords and organizing vast amounts of data, which would take a human several hours, are delegated to the AI. Based on the output, I perform the final analysis and make judgments myself. The AI is merely an excellent research assistant. I'm the driver.

Never take AI-generated text at face value; always verify with primary sources. Skip this crucial step, and you'll eventually be tripped up by a "plausible lie" like I was.

As someone who develops AI myself, I need to understand its limitations better than anyone. This failure was a valuable lesson that reinforced that understanding.