My AI Agent Recommended a Non-Existent Investment Product — Exposing Information Gaps Between Fund Distributors and Asset Manage

Hey everyone, it's your average 38-year-old developer here, hacking away at AI agents and automated trading bots in my evenings and weekends, just like many of you.

Today, I want to share a recent blunder with a financial information AI agent I built. What started as a chuckle-worthy mistake turned into a serious lesson about "information reliability" — a critical factor that can be fatal for any automated system design.

The Problem: AI Recommends a "Non-Existent Investment Trust"

With the new NISA (Japan's tax-advantaged investment program) kicking off, I was building an AI agent to list investment trusts tracking specific indices, primarily to rebalance my own portfolio. I started by asking it to compare products linked to the trending "FANG+" index.

Within seconds of my prompt, the AI confidently replied:

"For FANG+ linked investment trusts, considering trust fees and performance, I recommend the following two:

1. eMAXIS Slim FANG+: This series aims for the lowest operational costs in the industry and is highly recommended.
2. iFreeNEXT FANG+: It has a strong track record and offers stable management."

Hmm, interesting. I knew about iFreeNEXT FANG+, but "eMAXIS Slim FANG+"?

"Wait, is that even a thing?"

I know the eMAXIS Slim series is popular for things like S&P500 and global index funds, but FANG+? I'd never heard of it. Skeptical, I quickly Googled it.

...And indeed, it didn't exist.

The AI had confidently fabricated a plausible-sounding product and recommended it. This was a red flag.

Deeper Dive: The Trap of Information Hierarchy

AI "hallucinations" are common. It likely combined the popular "eMAXIS Slim" brand with the popular "FANG+" index to generate a seemingly plausible answer.

It would be easy to just dismiss AI as "unreliable" and move on. But as an engineer, I wanted to dig deeper. What if the agent had only returned real product names? Would I have blindly trusted it and moved on to the next step?

Curious, I decided to cross-reference multiple sources for the legitimate "iFreeNEXT FANG+". Specifically, I checked the website of "Company A" (a distributor selling the product) and the official website of "Daiwa Asset Management" (the asset manager that operates the fund).

What I found was even more concerning:

The NISA growth investment category eligibility status conflicted between the two sources.

• Asset Manager's (Daiwa Asset) Official Site: Clearly stated as eligible.
• Distributor's (Company A) Site: Seemed to indicate it was ineligible, or the information was outdated and not updated.

Which one is correct? Unsurprisingly, it's the "asset manager" who creates the product. The distributor is essentially a "retailer" that procures and sells the product. They might have delayed updates or simple transcription errors.

This incident made me realize that information has a clear "hierarchy":

1. Primary Information: The source of the information, like the asset manager's official website.
2. Secondary Information: Information that processes or reposts primary information, such as distributor websites, news articles, or blogs.
3. AI Generation: Information learned, re-synthesized, and generated by AI from these sources.

My agent had completely ignored this hierarchy. It treated all information gathered from the internet as flat data, simply summarized by the AI. This inherent risk meant it could recommend non-existent products or be misled by outdated secondary data.

The Fix: Embedding Fact-Checking into Code

This incident fundamentally changed my approach to designing information gathering agents. It's not enough to just have AI "research"; you need to build in mechanisms to "verify" for it to be practical.

The solution was simple:

1. Designate Reliable Sources as "Truth": In this case, data obtained from the "asset manager's official website" is defined as the master data (primary information).
2. Automate Cross-Checking: Any data obtained from AI or other secondary sources must be cross-referenced against the master data for fact-checking.

Specifically, I implemented logic like this using Python (pandas):

import pandas as pd

# Primary data obtained from the asset manager's official website (master data)
primary_data = {
    'Fund Name': ['iFreeNEXT FANG+', 'iFreeNEXT NASDAQ100'],
    'Asset Manager': ['Daiwa Asset Management', 'Daiwa Asset Management'],
    'NISA Growth Eligible': [True, True],
    'Source': ['Asset Manager Official', 'Asset Manager Official']
}
df_primary = pd.DataFrame(primary_data)

# AI's recommendation list (including the fabricated product)
ai_recommendation = {
    'Fund Name': ['eMAXIS Slim FANG+', 'iFreeNEXT FANG+'],
    'Reason': ['Lowest trust fees', 'Strong track record']
}
df_ai = pd.DataFrame(ai_recommendation)


# --- Fact-checking logic ---
# 1. Check if AI's recommendation exists in primary data (master)
for fund_name in df_ai['Fund Name']:
    if fund_name not in df_primary['Fund Name'].values:
        print(f"[WARNING] AI recommended '{fund_name}' which does not exist in master data.")

# 2. Merge distributor information with primary information to detect discrepancies
# (Skipped here, but involves merging distributor data with df_primary and checking for differences)

This code doesn't blindly trust the AI's output. It first checks if the fund names recommended by the AI exist in our defined df_primary (primary information) list. Anything not found is flagged as a "warning".

Furthermore, by merging specifications from secondary sources (like distributor sites) with the primary information, we can automatically detect issues like the "NISA eligibility discrepancy."

The Lesson: Master Your Primary Sources

What I learned from this failure is that AI is a highly capable assistant, but not the ultimate decision-maker. Especially in domains requiring accuracy, such as finance or technical information, a system to verify AI output is the lifeline.

And the foundation of that verification is the ability to discern "which information is primary."

When building automated systems, it's easy to gravitate towards readily accessible secondary information or APIs. But we must constantly ask: where did this information come from? Is its "freshness" and "reliability" guaranteed?

Ultimately, the biggest leverage comes from automating the tedious process of verification. Even if the AI lies, the overall system can still produce correct outputs. This incident reaffirmed my commitment to building robust systems that can handle such challenges. ✍️