Building an AI Devil's Advocate: A Multi-Agent Debate App with LangGraph & Streamlit #ai #python #opensource #showdev

Hey devs!
This is my first post on here, and I am sharing what I built

Have you ever wanted an AI that argues with itself? I recently built a project called "AI Devil's Advocate" — a multi-agent debate application that lets two AI agents argue a topic while a Judge AI delivers the final verdict.

But building it came with a major challenges like: API Rate Limits etc.

Here is how I built it, how it works, and how I solved the rate-limit issue using Automatic Fallback.

What is AI Devil's Advocate?

It's an autonomous multi-agent system where:

1. Agent A (For): Argues in favor of the topic.
2. Agent B (Against): Argues against the topic.
3. Judge AI: Listens to both sides and provides a structured verdict.

It's fully built with Python, LangGraph, and Streamlit, supporting multiple free-tier providers like Groq, Google Gemini, and OpenRouter.

Key Features

1. Multi-Provider Architecture: Switch between Groq, Gemini, and OpenRouter on the fly.
2. Automatic 429 Fallback (The Magic): If your active provider hits a rate limit or quota exhaustion, the app automatically switches to your next configured provider with zero interruption to the debate.
3. Polished UI & Animations: Custom CSS, Google Fonts (Inter/Poppins), fade-in animations, and stylized debater cards.
4. Transcript Exports: Download the entire debate as a styled PDF or Markdown file with 1 click.
5. Session API Call Counter:** Real-time sidebar counter to monitor your free-tier usage.

Tech Stack & Architecture

1. Frontend: Streamlit (with custom CSS for animations)
2. Orchestration: LangGraph (State machine for multi-agent flow)
3. LLM Factory: Custom FallbackLLMWrapper class that handles provider switching.
4. Providers: Groq (ultra-low latency), Google Gemini, OpenRouter.
5. Exports: ReportLab (for PDF generation).

How the Fallback Works
Instead of relying on one API, I built an llm_factory.py that wraps all providers. If the primary LLM throws a 429 Too Many Requests error, the wrapper catches it and immediately routes the prompt to the next available provider in the list. The debate never stops!

Try it Live!

I have deployed it on Streamlit Community Cloud. You can try it right now:

Live Demo Link

Open Source

The project is completely open-source. Feel free to fork it, add new providers, or improve the UI!

GitHub Repository

What I Learned

Building this taught me a lot about:

1. State Management in LangGraph for multi-agent conversations.
2. Error Handling in Production LLMs (the 429 fallback is a lifesaver).
3. UI/UX in Streamlit — you can make it look like a real SaaS product with a bit of custom CSS.

If you have any questions or feedback, drop a comment below! I'm currently exploring more Agentic AI workflows.

And this is my first post, so please let me know if I made any mistakes in writing or anywhere else. Also, if you like the repo, please consider giving it a star..

Happy coding!

Comments

[CarbonLayer]

Nice first post, Humam! The automatic 429 fallback is a practical addition, especially when working across free-tier providers with different quota limits.

I'm curious about your FallbackLLMWrapper: does it simply try providers in a fixed order, or did you consider choosing the fallback based on things like response time, remaining quota, or model cost?

It would be interesting to hear whether the fixed-order approach has been enough for your app or if you've run into situations where you wanted more control over which provider gets the next request.

Great project, and thanks for sharing the implementation!

[Humam Moin]

Thanks a lot for the thoughtful comment, CarbonLayer!

Currently, it's a fixed-order chain (Groq → Gemini → OpenRouter). Simple, predictable, and easy to debug, which matters when you're iterating fast. But you're right that it's not optimal: I've had cases where Groq was rate-limited while Gemini was idle, and the app wasted a round-trip before switching.

I'm planning to add a lightweight scoring layer next, response time, remaining quota (from headers), and model cost, so the wrapper picks the best provider instead of the next one. Baby steps, but your question just pushed it up my roadmap.

Appreciate you reading the whole thing and asking the right questions.

[CarbonLayer]

That's a great example of why even a lightweight routing layer can make a difference, Humam.

I actually like that you started with a fixed-order chain. It's predictable, easy to debug, and gives you a useful baseline before adding more sophisticated decisions.

The wasted round-trip when Groq is rate-limited is an interesting problem, especially when another provider has available capacity.

Your proposed scoring inputs—response time, remaining quota, and model cost—sound like a practical next step.

One thing I'm curious about: are you planning to account for model capability or response quality before comparing those operational metrics?

For a multi-agent debate app, I could imagine a cheaper or faster provider changing the quality of the arguments in ways that aren't immediately obvious from latency and cost alone.

I'd be interested to see how your routing approach evolves!