DEV Community

Cover image for I Built an AI That Interviews You Like a Real Mentor — Here's What I Learned About Multi-Agent Systems
Isaac Natarajan
Isaac Natarajan

Posted on

I Built an AI That Interviews You Like a Real Mentor — Here's What I Learned About Multi-Agent Systems

A few weeks ago I set out to learn LangGraph. I ended up building something I actually use.

Here's the origin story, the architecture, and the mistakes that taught me the most.


The Problem With Interview Prep Tools

Most interview prep apps are the same thing wearing different skins: a list of questions, a text box, maybe a model answer if you're lucky. They don't react to you. They don't know if you're stuck, confused, or just had a brain fart and typed something completely unrelated. They grade you like a quiz, not coach you like a mentor.

I wanted to build something different — a system that feels less like a form and more like sitting across from someone who actually wants to see you improve.

And since I'd already worked with LangChain and LangSmith, this felt like the perfect excuse to finally learn LangGraph properly, by building something with real stakes: multiple agents, real conditional logic, and a genuinely useful end product.

Starting With the Wrong Architecture (On Purpose, Sort Of)

My first instinct was the "trendy" AI-app checklist: RAG, a vector database, the works. I was building this in the middle of a big shift in the ecosystem too — mid-project, Groq announced they were deprecating the exact models I'd been using (llama-3.3-70b-versatile and llama-3.1-8b-instant), pushing everyone toward their newer gpt-oss models. A good reminder that in this space, the ground moves under you constantly, and your architecture needs to be flexible enough to swap a model string without a rewrite.

But early on, a simpler realization changed the whole design: this system doesn't need a knowledge base. The questions aren't retrieved from documents — they're generated live, tailored to a role, an experience level, and an optional job description. Qdrant would have added infrastructure that solved a problem I didn't have.

So I cut it. MongoDB became the single source of truth for everything — sessions, users, performance history. One database, one mental model, zero unnecessary complexity. Sometimes the best architectural decision is the thing you don't build.

The Part That Actually Made It "Agentic"

Here's the distinction that took me the longest to internalize: using multiple LLM calls doesn't make a system agentic. Deciding which call to make, dynamically, based on context — that's what does.

My first version had a rigid pipeline: user answers → evaluator scores it → maybe a hint → done. It worked, but it was fragile. The moment a user typed something like "can we have more questions?" instead of an actual answer, the system would dutifully score it as a wrong answer. Because as far as the code was concerned, every input was "an answer."

That was the wake-up call. I built an Intent Classifier Agent that runs in front of everything else, reading the user's message and routing it to the right specialist:

  • Genuinely answering the question? → Evaluator Agent
  • Stuck and want help? → Hint Generator
  • Don't understand the question itself? → Question Clarifier (a new agent I hadn't planned for)
  • Want more detail on feedback you already got? → Answer Elaborator (another agent born from user testing, not the original design)
  • Talking about the session, not the question? → Off-Topic Handler
  • Want to skip? → Skip logic, no scoring, no judgment

Nine agents now, each with one job, coordinated through LangGraph's conditional edges instead of a fixed sequence. That's the difference between a chatbot with extra steps and an actual multi-agent system.

The Bugs That Taught Me the Most

A few humbling moments, because no build log is honest without them:

The Python scoping bug that looked like a networking issue. I had a function reassign a variable (state = hint_generator_agent(state)) inside what I thought was a safe scope. Python quietly decided the whole variable was local to the function from that point on — including before the reassignment — and threw an UnboundLocalError that manifested downstream as a browser CORS error. I spent twenty minutes checking my CORS config before realizing the backend had crashed entirely.

The evaluator that graded a session it never saw. Reload a chat mid-session, and the system would ask the exact same question the user had just answered — because the "current question" was only updated when explicitly requested, not the moment an answer came in. Small state-management oversight, big trust-breaking bug if left in.

The tone problem. My evaluator originally wrote feedback like a report: "The candidate demonstrated..." Clinical, third-person, cold. The fix wasn't more intelligence — it was a prompt rewrite asking the model to speak in first person, use the user's name occasionally, and react to what they specifically said rather than generic praise. One prompt change transformed the entire feel of the product from "quiz" to "conversation."

What "Agentic" Actually Buys You

By the end, the system could handle things I never explicitly coded for. Say "I don't get it" and it rephrases the question in its own words — as the same interviewer, not a third party summarizing someone else's question. Ask "can you elaborate?" after getting feedback, and it digs deeper into that specific model answer, not a generic explanation.

None of that was hardcoded as a feature. It emerged from giving the system the ability to classify intent and route accordingly. That's the part of agentic design that's easy to describe in a sentence and genuinely hard to appreciate until you've built the alternative first and watched it fail.

What's Next

Docker's wired up so anyone can clone the repo, drop in an API key, and run the whole stack — frontend, backend, and MongoDB — with one command. Langfuse traces every agent call, which turned out to be invaluable for actually seeing the routing decisions happen in real time, not just trust that they were happening.

From here: a proper deployment, resume-based personalization, and maybe a voice mode, because reading and typing answers isn't quite the same pressure as saying them out loud.

If you're thinking about learning LangGraph, my honest advice: don't build a toy example. Build something with a real, opinionated use case, let real usage expose the gaps in your first design, and be ready to throw away your first architecture when a simpler one reveals itself.

Check out the full project, architecture, and setup instructions on GitHub and feedback welcome.

Top comments (0)