Ayush Singh Tomar

Posted on Aug 10

I Built an AI Agent That Thinks Before It Answers — And Loops Back When It Doesn't Know Enough

#ai #python #langchain #agenticai

Most AI projects look like this:

User Input → LLM → Output

That's not an agent. That's autocomplete with a UI.

AgentLoop is different. Give it a research topic and it:

Checks if it already researched something similar (long-term memory)
Breaks the topic into targeted sub-questions
Decides per sub-question whether to call a live web search tool
Re-reads its own notes and routes itself back into research if it finds gaps
Only then writes a structured, sourced report
Saves the run to memory for next time The loop in the middle — where the agent judges its own output and decides to keep going — that's what makes it genuinely agentic. Not the LLM. The decision-making around it.

Live demo: agentloop.streamlit.app
GitHub: github.com/ayush-s-tomar/agentloop

What It Does

You type a research topic. The agent runs a 6-node pipeline:

[Recall] → [Plan] → [Research] → [Reflect] → [Synthesize] → [Persist]
                                      ↑               |
                                      └───────────────┘
                                    (loops back if gaps found)

Recall — checks SQLite long-term memory for related past research. If it finds something relevant, it loads those notes as context before planning. Avoids re-researching the same ground twice.

Plan — the LLM breaks the topic into 3–5 specific sub-questions. Not "tell me about X" — actual targeted questions like "what companies are deploying X in production?" and "what are the failure modes of X?"

Research — for each sub-question, the LLM decides whether to call the Tavily web search tool. Not every sub-question needs a search — sometimes the answer follows from earlier notes. This is genuine tool-use, not hardcoded search-every-time.

Reflect — the node I'm most proud of. The agent re-reads everything it's gathered and asks: is this complete? Are there gaps? If yes, it routes back into Research for another loop. If no, it moves forward. Bounded at 3 iterations so it can't loop forever.

Synthesize — writes a structured markdown report from all gathered notes and search results. Citations included.

Persist — saves the full run to SQLite: topic, sub-questions, sources, report. Available for recall on future runs.

Architecture

The 6-node graph, including the reflect → research loop-back.

Topic input, live trace, and the generated report — all in one Streamlit view.

Stack: LangGraph · Streamlit · Groq (llama-3.1-8b-instant) · Tavily · SQLite · Streamlit Cloud

How Each Part Works

1. The Reflect Node — Conditional Loop-Back

This is the decision that separates AgentLoop from a linear pipeline.

After Research runs, instead of immediately synthesizing, the agent hits the Reflect node:

def reflect_node(state: AgentState) -> AgentState:
    notes = "\n".join(state["notes"])
    sub_questions = "\n".join(state["sub_questions"])

    prompt = f"""You researched these questions:
{sub_questions}

Here are your notes so far:
{notes}

Are there significant gaps? Answer YES or NO, then explain."""

    response = llm.chat([{"role": "user", "content": prompt}], system="Be critical.")

    has_gaps = response.strip().upper().startswith("YES")
    iterations = state.get("reflect_iterations", 0)

    return {
        **state,
        "needs_more_research": has_gaps and iterations < 3,
        "reflect_iterations": iterations + 1
    }

The LangGraph conditional edge routes based on needs_more_research:

graph.add_conditional_edges(
    "reflect",
    lambda state: "research" if state["needs_more_research"] else "synthesize"
)

The iteration cap (iterations < 3) is non-negotiable. Without it, a loop-happy LLM spins forever on ambiguous topics. With it, the worst case is 3 research passes — still far more thorough than one.

2. Two-Layer Memory

Short-term (within a run): state["notes"] — a list that accumulates across all Research iterations. Each tool call appends its findings. The Reflect and Synthesize nodes see the full accumulated picture, not just the last search.

Long-term (across runs): SQLite with a simple schema — topic, sub-questions, notes, report, timestamp. The Recall node queries this by keyword similarity at the start of every run. Not vector search (that's the next step) — just SQL LIKE matching, which is good enough for a portfolio project and dead simple to reason about.

def recall_node(state: AgentState) -> AgentState:
    topic = state["topic"]
    words = [w for w in topic.lower().split() if len(w) > 4]

    past_runs = []
    for word in words[:3]:  # top 3 keywords
        results = db.search_by_keyword(word)
        past_runs.extend(results)

    context = format_past_runs(past_runs[:2])  # most recent 2 matches
    return {**state, "memory_context": context}

3. Tool-Calling — Decision Per Sub-Question

The LLM doesn't call web search blindly. It receives the tool schema and decides per sub-question:

TOOLS = [{
    "name": "web_search",
    "description": "Search the live web for current information",
    "parameters": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Search query"}
        },
        "required": ["query"]
    }
}]

For factual sub-questions about current events, it calls the tool. For sub-questions it can reason about from existing notes, it doesn't. That's the distinction between a real agent and a search wrapper.

What Broke (The Honest Part)

1. Render Kept Suspending the Live Demo — So I Moved to Streamlit

The original architecture was FastAPI + React, streaming the live trace to the frontend via Server-Sent Events — each node completion fired an SSE event, the UI updated in real time.

It worked perfectly locally. On Render's free tier, SSE died after 30 seconds — Render closes long-lived HTTP connections on the free plan, and a 6-node agent with multiple web searches takes longer than that. I patched around it with background tasks + polling (the frontend polling /api/research/status/{job_id} every 2 seconds instead of holding a stream open), and that solved the timeout.

But the deeper problem was the platform itself. Render's free web services spin down on inactivity and get suspended monthly, so the live demo link kept going cold between visitors regardless of the polling fix. That's not something you patch — it's the tier's actual model. So I migrated off FastAPI + React entirely onto a single-file Streamlit app, reusing the agent/ and memory/ modules unchanged, deployed to Streamlit Community Cloud. No backend process to suspend, no SSE-vs-polling tradeoff to manage — Streamlit re-runs the script and renders state directly on each interaction.

Lesson: design for the deployment environment, not just localhost — and sometimes the right fix isn't a smarter workaround, it's picking a host whose free-tier model actually matches how the project gets used (an occasional demo click-through, not a service that needs to stay warm).

2. Model Deprecations — Three Times

Groq deprecated models mid-development, more than once:

llama-3.3-70b-versatile → deprecated
llama3-70b-8192 → decommissioned
llama3-groq-70b-8192-tool-use-preview → tool-calling broken Each one failed silently or with a cryptic error. Ended up on llama-3.1-8b-instant — smaller, but stable and actively maintained.

Lesson: never hardcode a model string. It belongs in an environment variable:

MODEL = os.getenv("GROQ_MODEL", "llama-3.1-8b-instant")

One env var change, no redeployment needed when the next deprecation hits.

3. Rate Limits Hit During Demo Recording

Groq's free tier is 100k tokens/day. Between development testing, debugging, and demo runs, I exhausted the quota on the same day I tried to record the LinkedIn screenshot. Two hours lost waiting for reset.

Lesson: keep a separate Groq API key for demos. Never use the dev key for production.

4. Python 3.14 Broke the Old Render Build

Back when this was still FastAPI on Render, pydantic-core had no wheel for Python 3.14 — silent build failure, no clear error, just a broken deploy. The fix was one environment variable (PYTHON_VERSION = 3.11.9) pinning the runtime explicitly instead of trusting the host default.

It's moot now that the app is Streamlit-only with a much smaller dependency surface, but the underlying lesson traveled with the migration: pin your Python version explicitly on any host, don't trust the default.

What I'd Do Differently

Replace SQL LIKE matching with vector search. Right now the Recall node finds past runs by keyword — it misses semantically similar research with different wording. ChromaDB or Supabase pgvector would fix this. It's the most meaningful upgrade this project needs.

Add a second tool. Right now the only tool is web_search. A calculator or structured data lookup would demonstrate the LLM genuinely choosing between tools — not just "search or not." That's a stronger tool-use story.

Instrument the reflect loop. I don't log how often the agent actually loops back vs. goes straight to synthesis. That metric would tell me whether the reflect node is earning its latency or just adding overhead.

Define the LLM interface contract on day one. Half my debugging time came from graph.py and llm.py making different assumptions about function signatures and return types. One typed contract file written before any agent logic would have caught all of it.

Try It