DEV Community: Yash Lohade

How I Debugged My AI Trading Agent Token Overflow, Memory Leaks & Silent NaN Predictions

Yash Lohade — Mon, 27 Jul 2026 06:56:39 +0000

Project Overview

Algomaya is an AI-powered algorithmic trading education platform I'm building solo. It lets users learn algo trading, build strategies with a no-code builder, backtest against historical data, and practice with paper trading — all powered by an AI agent called Maya that autonomously chains together tools (backtests, signals, quotes, news) to answer complex trading questions.

Stack: React Native (Expo SDK 54) + FastAPI + PostgreSQL + Groq LLM (Llama 3.3 70B) + LSTM neural network for price prediction

The app has 89+ screens, 335+ commits, and a production backend on Google Cloud Run. It's a real product heading toward Play Store launch — which is exactly why these bugs mattered.

Bug Fix or Performance Improvement

I fixed 6 critical bugs across the AI agent pipeline, LSTM prediction model, and frontend — the kind of bugs that silently corrupt data, leak memory, and crash production.

Bug 1: AI Agent Token Overflow — Context Window Bomb

File: backend/app/services/maya_agent.py

Maya's ReAct loop accumulates messages with every LLM call and tool result. Each tool response is truncated to 2000 chars, but with 15 max steps, the message array can balloon past Groq's context window limit. The result: 400 Bad Request errors that kill the agent mid-analysis, or worse — silently truncated context that makes the LLM hallucinate.

The fix — sliding window token management:

# BEFORE: Messages grew unbounded
messages: List[dict] = [
    {"role": "system", "content": MAYA_SYSTEM_PROMPT},
    {"role": "user", "content": goal},
]
# ... messages.append() on every iteration, no limit check

# AFTER: Token-aware trimming keeps context within budget
@staticmethod
def _estimate_token_count(messages: List[dict]) -> int:
    """Rough token estimate (~1.3 tokens per word)."""
    total = 0
    for m in messages:
        content = m.get("content", "")
        if isinstance(content, str):
            total += len(content.split())
    return int(total * 1.3)

@staticmethod
def _trim_messages(messages: List[dict], max_tokens: int = 6000) -> List[dict]:
    """Keep system prompt + user goal + most recent messages within budget."""
    if MayaAgent._estimate_token_count(messages) <= max_tokens:
        return messages
    # Always preserve system prompt (index 0) and user goal (index 1)
    trimmed = [messages[0], messages[1]]
    remaining = messages[2:]
    tail: List[dict] = []
    budget = max_tokens - MayaAgent._estimate_token_count(trimmed)
    for msg in reversed(remaining):
        msg_tokens = int(len(str(msg.get("content", "")).split()) * 1.3)
        if budget - msg_tokens < 0:
            break
        tail.insert(0, msg)
        budget -= msg_tokens
    return trimmed + tail

Impact: Prevents context window overflow on complex multi-tool agent runs. The agent now gracefully drops older tool results while always preserving the system prompt and user's original goal.

Bug 2: Agent Run Hangs Indefinitely — No Outer Timeout

File: backend/app/services/maya_agent.py

Each Groq API call has a 30s timeout, but the entire agent loop had no outer timeout. If the LLM kept requesting tools in a loop (tool → LLM → tool → LLM...), the backend task could run for minutes or hours. The frontend times out at 30s, but the backend keeps burning CPU and API credits.

# BEFORE: No timeout guard on the loop
while step_number < max_steps:
    response = await _call_groq(messages, tools=MAYA_TOOLS)
    # ... could run forever if LLM keeps calling tools

# AFTER: Hard 5-minute timeout
MAX_RUN_TIMEOUT_SECONDS = 300

while step_number < max_steps:
    elapsed = (datetime.now(timezone.utc) - start_time).total_seconds()
    if elapsed > MAX_RUN_TIMEOUT_SECONDS:
        logger.warning(f"Maya run {run_id} exceeded {MAX_RUN_TIMEOUT_SECONDS}s timeout")
        run.summary = "Analysis timed out. Please try a simpler query."
        run.status = "completed"
        db.commit()
        return
    # ... rest of loop

Impact: Prevents runaway agent executions from burning Groq API credits and Cloud Run CPU. Users get a clean timeout message instead of an infinite spinner.

Bug 3: Tool Result Truncation Destroys LLM Context

File: backend/app/services/maya_agent.py

When a backtest returns 500 trades, the old _truncate_result() replaced the entire array with the string "[500 items omitted]". The LLM then received this useless string instead of actual data, making it impossible to analyze results properly.

# BEFORE: Replace arrays with useless strings
if k in ("chart_data", "trades", "equity_curve"):
    truncated[k] = f"[{len(v)} items omitted]"  # LLM can't parse this

# AFTER: Keep first + last N items so LLM has real data to reason over
if k == "trades":
    if isinstance(v, list) and len(v) > max_items:
        truncated[k] = v[:5] + v[-5:]  # First 5 + last 5 trades
        truncated["_trades_total"] = len(v)  # Total count for context

Impact: The LLM can now actually analyze backtest results — seeing both early and recent trades, real equity curve data points, and accurate totals. Before this fix, Maya would often say "I ran the backtest but can't see the results" because the data was replaced with placeholder strings.

Bug 4: LSTM Model Silently Predicts NaN — No Scaler Guards

File: backend/app/services/lstm_model.py

The LSTM price prediction model's _scale_sequences() and predict() methods had zero protection against NaN/Inf values. When market data had gaps (common with Indian stocks on holidays), the scaler would produce NaN, the model would predict NaN, and the frontend would silently show "NEUTRAL" for every stock — with no error logged anywhere.

# BEFORE: No NaN protection — silent data corruption
def _scale_sequences(self, X_seq):
    flat = X_seq.reshape(-1, n_features)
    scaled = self.scaler.transform(flat)  # NaN in → NaN out, silently
    return scaled.reshape(n_samples, seq_len, n_features)

# AFTER: Detect, log, and handle NaN at every stage
def _scale_sequences(self, X_seq):
    if self.scaler is None:
        raise ValueError("Scaler not fitted. Call train() or load() first.")

    flat = X_seq.reshape(-1, n_features)

    if np.isnan(flat).any():
        logger.warning("Input contains NaN values before scaling — forward-filling")
        flat = pd.DataFrame(flat).ffill().fillna(0).values

    scaled = self.scaler.transform(flat)

    if np.isnan(scaled).any() or np.isinf(scaled).any():
        logger.warning("Scaling produced NaN/Inf — clamping to safe range")
        scaled = np.nan_to_num(scaled, nan=0.0, posinf=3.0, neginf=-3.0)

    return scaled.reshape(n_samples, seq_len, n_features)

Impact: Stock predictions no longer silently fail. NaN values from missing market data are forward-filled (a standard financial data technique), and extreme values that break the scaler are clamped to a safe range. Users now see actual predictions instead of blanket "NEUTRAL" signals.

Bug 5: TensorFlow Memory Leak in Prediction Loop

File: backend/app/services/lstm_model.py

Every call to predict() allocated X_scaled and X_seq numpy arrays but never cleaned them up. On Cloud Run with limited memory, repeated predictions (e.g., scanning 50 stocks for signals) would accumulate hundreds of MB of orphaned arrays until the container OOM-killed.

# BEFORE: Arrays allocated, never freed
X_scaled = self.scaler.transform(X)
X_seq = self.create_sequences_predict(X_scaled)
probabilities = self.model.predict(X_seq, verbose=0).flatten()
# X_scaled and X_seq still in memory

# AFTER: Explicit cleanup in finally block
try:
    probabilities = self.model.predict(X_seq, verbose=0).flatten()
    predictions = (probabilities >= 0.5).astype(int)
finally:
    del X_scaled, X_seq
    import gc
    gc.collect()

Impact: Prevents OOM crashes on Cloud Run during batch prediction operations. Memory is explicitly freed after each prediction call, keeping the container's footprint stable even when scanning dozens of stocks.

Bug 6: Sentry AI Agent Monitoring Integration

File: backend/app/services/maya_agent.py, backend/app/main.py, backend/app/core/config.py

Beyond fixing bugs, I integrated Sentry for full AI agent observability. Each Maya agent run now creates a Sentry transaction with spans for every LLM call and tool execution, complete with token usage tracking and error capture.

# Sentry transaction wraps the entire agent run
with sentry_sdk.start_transaction(
    op="ai.agent", name="maya_agent_run", description=goal[:100]
) as txn:
    txn.set_tag("tier", tier)
    txn.set_tag("run_id", str(run_id))

    # Each LLM call gets its own span with token tracking
    with sentry_sdk.start_span(
        op="ai.chat_completions.create", description=f"groq/{GROQ_MODEL}"
    ) as llm_span:
        llm_span.set_data("ai.model_id", GROQ_MODEL)
        response = await _call_groq(messages, tools=MAYA_TOOLS)

        usage = response.get("usage", {})
        llm_span.set_data("ai.prompt_tokens_used", usage.get("prompt_tokens", 0))
        llm_span.set_data("ai.completion_tokens_used", usage.get("completion_tokens", 0))

    # Each tool execution gets a span
    with sentry_sdk.start_span(
        op="ai.tool", description=f"tool:{tool_name}"
    ) as tool_span:
        tool_span.set_data("ai.tool.name", tool_name)
        result = await _execute_tool(tool_name, tool_args, tier=tier)

What this enables:

Conversation traces — See the full agent reasoning chain (LLM call → tool → LLM → tool → summary) as a waterfall in Sentry
Token usage monitoring — Track prompt/completion tokens per run to detect context overflow before it happens
Tool failure tracking — Pinpoint which tools fail most often and why
Performance profiling — Identify slow tool executions (backtests taking 10s+) vs fast ones (quotes <1s)

Code

All code changes are in the diffs above. Summary of files modified:

File	Change	Impact
`backend/app/services/maya_agent.py`	Token management, timeout guard, truncation fix, Sentry AI tracing	Prevents crashes, hangs, and data loss in AI agent
`backend/app/services/lstm_model.py`	NaN guards, memory cleanup	Prevents silent prediction failures and OOM crashes
`backend/app/main.py`	Sentry SDK initialization	Error tracking and performance monitoring
`backend/app/core/config.py`	Sentry config variables	Environment-specific monitoring config
`backend/requirements.txt`	Added`sentry-sdk[fastapi]`	Sentry dependency

My Improvements

These fixes address the hardest class of bugs in AI applications — the ones that don't crash loudly but silently corrupt your data pipeline:

Token overflow was making Maya hallucinate on complex queries because the LLM lost context. The sliding window approach keeps the most recent tool results while always preserving the original goal.
The missing timeout was burning real money — each Groq API call costs tokens, and a runaway agent could chain 15 calls over several minutes with no kill switch.
NaN propagation is the classic ML pipeline bug. One missing data point in market data → NaN features → NaN scaled values → NaN predictions → "NEUTRAL" everywhere. The fix adds guards at every stage of the pipeline.
The Sentry integration isn't just monitoring — it's AI agent observability. Each agent run becomes a trace with spans for LLM calls and tool executions, token usage metrics, and error capture. This is exactly what you need to debug why an AI agent gave a bad answer.

How I found them: I conducted a systematic audit documenting 71 bugs across 4 severity levels. The AI pipeline bugs were the most dangerous because they failed silently — no crash, no error log, just wrong results served to users.

Best Use of Sentry

I integrated Sentry (sentry-sdk[fastapi]>=2.10.0) into my FastAPI backend specifically for AI agent monitoring:

Setup:

Added SENTRY_DSN, SENTRY_TRACES_SAMPLE_RATE, and SENTRY_PROFILES_SAMPLE_RATE to my Pydantic settings config
Initialized Sentry in main.py before middleware with enable_tracing=True
Used Sentry's AI-specific span operations (ai.agent, ai.chat_completions.create, ai.tool) for proper categorization

How Sentry helps debug my AI agent:

Conversation Traces — Each Maya agent run creates a Sentry transaction. I can see the full reasoning chain as a waterfall: which tools were called, how long each LLM call took, and where failures occurred.
Token Usage Tracking — Every LLM span records ai.prompt_tokens_used and ai.completion_tokens_used. I can now detect token overflow before it hits the context limit by monitoring token growth across runs.
Tool Failure Monitoring — When a tool fails (e.g., Yahoo Finance API timeout), Sentry captures the exception with full context: which stock, which strategy, what the LLM was trying to do. This lets me prioritize which tool integrations need hardening.
Performance Profiling — Sentry's continuous profiling shows that backtests are the bottleneck (5-15s per run), while stock quotes are fast (<500ms). This informed my decision to add the timeout guard — a 15-step run with 10 backtests could easily exceed 2 minutes.
Error Alerting — Sentry alerts on new error types. The first time I deployed, it immediately caught a json.JSONDecodeError from malformed LLM tool arguments that I'd been silently swallowing as empty {} — a bug I didn't even know existed.

I got tired of blowing up my trading account, so I built Algomaya 🇮🇳📈

Yash Lohade — Mon, 20 Apr 2026 08:43:24 +0000

Quick question before you scroll:

Have you ever had a "genius" trading idea at 2 AM, deployed it with real money the next morning, and then watched your capital disappear faster than your weekend plans? 🫠

Yeah. Same. That's literally why Algomaya exists.

The 2 AM moment that started it all

I was sitting on my couch, chart open, convinced I had found the pattern. You know the one, the setup that's going to change everything.

I didn't paper trade it. I didn't backtest it. I just… sent it. 💸

Spoiler: the pattern did not, in fact, change everything. (Well, it changed my P&L. Not in a good way.)

That night I went looking for a tool where I could:

Test my strategy with real Indian market data (NSE/BSE)
Not spend 3 weekends configuring Python + broker APIs + data feeds
Move from paper to live trading without rewriting everything
Actually afford it as a retail trader 😅

I couldn't find it. So I built it.

Meet Algomaya 👋

Algomaya is an algo trading + paper trading platform built specifically for the Indian stock market.

The core idea is stupidly simple:

Idea > Paper trade it > See if it actually works > Automate it > Sleep better

No more "I think this strategy works." Either the numbers say yes, or they don't.

What it does (in plain English)

🧪 Paper Trading: Run your strategies on live market data with zero risk. Your ego might take a hit, but your bank account won't.

🤖 Algo Trading: Once a strategy proves itself on paper, flip the switch and let it run automatically.

🇮🇳 Built for India: NSE, BSE, Indian brokers, Indian market hours, Indian quirks. Not a Robinhood clone with the logo changed.

👨‍💻 Not just for quants: You don't need to be a Python wizard or have a CFA to use it.

The part where I need your help 🙏

I'm building this in public, and honestly the best feedback comes from devs and traders who've been in the trenches. So I'd love your take on this:

👉 Drop a comment and tell me:

Have you ever tried to automate your trading? What stopped you?
If you trade Indian markets, what tool do you currently use, and what do you hate about it?
What's the one feature that would make you actually try a new trading platform?

I read every comment. Seriously. Even the spicy ones. 🌶️

Check it out

🔗 algomaya.com

If you're a fellow indie hacker / trader / dev curious about the fintech + India intersection, hit me up. Happy to nerd out about the tech stack, the regulatory maze, or how I'm thinking about the market.

P.S. If this post saves even one person from their own 2 AM "genius idea," my work here is done. 😌

Follow along for more build-in-public updates, the good, the bad, and the "why did I think this would be easy."

I Built an AI-Powered Algo Trading App for the Indian Stock Market - Here's What I Learned

Yash Lohade — Thu, 16 Apr 2026 19:10:43 +0000

So, I've been quietly building something over the past year and I finally feel like it's time to talk about it openly. Not as a product pitch — more like a dev journal entry where I unpack what worked, what didn't, and what genuinely surprised me along the way.

The app is called Algomaya — a paper trading and algorithmic trading platform built specifically for the Indian stock market (NSE, NIFTY, BANKNIFTY, F&O). Check it out at algomaya.com or download it directly: Download Algomaya on Google Play

Why I Built This

I got into algo trading a few years back and quickly realized the Indian market had a weird gap. There were either super complex platforms meant for institutional traders, or overly simple apps that treated retail users like they couldn't handle real information. There was almost nothing in between for someone who actually wanted to learn algorithmic trading without risking real money first.

That's where the idea for Algomaya came from — what if someone could practice trading with live market data, zero risk, and also learn how to build trading strategies with AI assistance, all in one place?

What Algomaya Actually Does

Here's a quick breakdown of the core features:

📈 Paper Trading & Virtual Trading Simulator (FREE)

Real-time NSE/BSE market data during live market hours (9:15 AM – 3:30 PM)
Trade NSE stocks, NIFTY, BANKNIFTY, FinNIFTY, and F&O options with virtual cash
Full simulated P&L tracking, trade history, virtual portfolio management
No real money involved — zero risk, full market realism

🤖 No-Code Algo Trading Platform

Build and backtest trading strategies without writing a single line of code
AI-powered strategy builder using natural language — describe what you want, it builds the logic
Deploy strategies to paper trade automatically during live market hours
11+ free algorithmic trading courses built right into the app

💬 AI Trading Assistant

Ask questions, get real-time market analysis and strategy suggestions
Helps beginners understand concepts like moving averages, RSI, Bollinger Bands
Think of it like a knowledgeable trading buddy that's always available

🔗 Try Algomaya FREE on Google Play

The Technical Stack (The Part I Actually Want to Talk About)

Choosing the Right Stack

For an Android app dealing with live financial data, performance and reliability are non-negotiable. Here's what I landed on:

Kotlin for the Android frontend — coroutines saved my life when handling real-time data streams
Python backend for the AI and strategy execution engine
WebSockets for live market data feeds — REST APIs just don't cut it for tick-by-tick data
Firebase for auth and real-time sync
TensorFlow Lite on-device for latency-sensitive AI inference

The hardest part? Keeping the live data pipeline stable. NSE data feeds can be... temperamental. Building retry logic, fallback mechanisms, and graceful degradation was probably 30% of the total dev time.

Building the No-Code Strategy Builder

This was the most interesting engineering challenge. The goal: let a user describe a strategy in plain English and turn it into executable trading logic.

The approach I landed on:

User inputs a strategy description (e.g., "Buy when RSI crosses above 30 and price is above 200 EMA")
An LLM parses this into a structured strategy schema (conditions, entry/exit rules, indicators)
The schema compiles into a backtesting engine that runs against historical NSE data
User sees backtest results and can deploy to live paper trading

Getting step 2 right took way longer than expected. LLMs are great at understanding intent but they hallucinate indicator parameters constantly. I ended up building a validation layer that catches impossible or contradictory conditions before they hit the backtesting engine.

The AI Integration Layer

I used a combination of fine-tuned models and RAG (Retrieval Augmented Generation) for the trading assistant. The knowledge base includes:

SEBI regulations and guidelines
NSE/BSE trading rules and circuit breakers
Technical analysis documentation
Historical market patterns specific to Indian indices

One thing I learned the hard way: generic AI models trained on Western market data give terrible advice for Indian markets. NIFTY and BANKNIFTY behave very differently from S&P 500 constituents. The fine-tuning step was absolutely essential.

What I Got Wrong (Being Honest With Yourself)

1. I Underestimated Onboarding Complexity

Most early users were complete beginners. The UI I built assumed people knew what "F&O" or "BANKNIFTY" meant. Spoiler: they didn't. I had to rebuild a significant portion of the onboarding flow with progressive disclosure — start simple, layer in complexity.

2. The Backtesting Engine Was Too Optimistic

Early backtest results looked great. Almost suspiciously great. The issue was lookahead bias — I was accidentally allowing the strategy to "see" future data during backtesting. Classic mistake. Fixing it required a full rewrite of the data pipeline.

3. I Shipped Too Many Features at Once

Classic developer mistake. I thought more features = better app. Users thought it = confusing app. The current version is much more focused. When in doubt, cut.

AI Trends That Actually Matter for Fintech Apps Right Now

Since this post is partly about AI development, here's what I've found genuinely useful in the #GenerativeAI and #AIAgents space for fintech:

RAG over fine-tuning for domain knowledge — unless you have massive labeled datasets, RAG is more practical and easier to update
Structured output from LLMs — tools like function calling in the OpenAI API are game changers for reliable parsing
On-device ML for latency-sensitive features — TFLite for Android is underrated; it also keeps sensitive financial data off the cloud
Embeddings for pattern similarity — finding similar historical market patterns using vector search is genuinely powerful

Where Algomaya Is Headed

The roadmap includes:

Options strategy visualization
Multi-leg strategy builder
Community where traders can share and fork strategies
More conversational and context-aware AI assistant across sessions

If you're building in the fintech + AI space in India, or you're a developer interested in algo trading, I'd genuinely love to connect and talk shop.

Resources & Links

🌐 Algomaya Official Website — Learn more about the platform
🚀 Download Algomaya — FREE Paper & Algo Trading App
📚 Learn Algorithmic Trading — 11+ Free Courses on Algomaya
📖 What is Paper Trading? (Investopedia)
📊 NSE India — Official Market Data
🤖 LangChain RAG Documentation

If you've built something similar or have thoughts on AI + trading, drop a comment below. Always happy to geek out about this stuff.

Tags: #AI #MachineLearning #GenerativeAI #AIAgents #LLM #FinTech #AlgoTrading #AndroidDev #MobileDev #IndianStockMarket #PaperTrading #NoCode #RAG #TensorFlow #Kotlin #BuildInPublic #100DaysOfCode #StockMarket #DevJournal #OpenSource