DEV Community

Cover image for Beyond Chatbots: Building an Observable Multi-Agent Study Coach with Gemini, Google ADK & Docker
Hastimal Jangid
Hastimal Jangid Subscriber

Posted on

Beyond Chatbots: Building an Observable Multi-Agent Study Coach with Gemini, Google ADK & Docker

Build. Learn. Evaluate. Observe.

What if an AI study assistant could do more than answer questions?

What if it could generate a personalized certification study plan, explain technical concepts using architecture diagrams and Python examples, evaluate learning through quizzes, and let developers see exactly which agents, tools, and Gemini calls executed behind the scenes?

That's what I set out to build with Gemini ADK Study Coach Lab—an open-source, hands-on multi-agent AI workshop powered by Google Gemini, Google Agent Development Kit (ADK), Python, Streamlit, and Docker.

In this article, I'll walk through the working demo and show how to combine agent orchestration, deterministic validation, and production-style observability.

🔗 GitHub: https://github.com/hastimal/gemini-adk-study-coach-lab

1. Architecture: One learning experience, multiple specialized agents

Rather than relying on one large prompt, the Study Coach separates responsibilities across specialist agents.

  • Study Planner Agent: Builds personalized study schedules within a defined daily time budget.
  • Topic Expert Agent: Explains complex concepts with learning objectives, architecture diagrams, practical Python, and curated resources.
  • Quiz Generator Agent: Creates knowledge checks and practice questions, while deterministic Python handles grading.
  • Document Agent: Supports document-grounded learning using uploaded PDF/DOCX study guides.

Google ADK provides the framework for coordinating the agents, while Gemini powers adaptive generation and explanations.

Architecture overview:

                  Student / Developer
                         |
                    Streamlit UI
                         |
               Google ADK Coordinator
                         |
        +----------------+----------------+
        |                |                |
   Study Planner    Topic Expert    Quiz Generator
        |                |                |
   Study Schedule   Lessons & Code    Questions
        |                |                |
   Python Rules    Learning Links    Python Grading
                         |
                    Document Agent
                         |
                 PDF/DOCX Retrieval

              OBSERVABILITY LAYER
                         |
                   OpenTelemetry
                         |
             +-----------+-----------+
             |                       |
         Prometheus                Tempo
          Metrics                  Traces
             |                       |
             +-----------+-----------+
                         |
                       Grafana
Enter fullscreen mode Exit fullscreen mode

The architectural lesson is straightforward:

Use Gemini for reasoning and content generation. Use deterministic Python for constraints, calculations, and grading. Use observability to understand execution.

2. Workshop setup: Run the entire lab locally

This workshop combines a locally running Python/Streamlit application with Docker-based observability services.

Start the application:

git clone https://github.com/hastimal/gemini-adk-study-coach-lab.git

cd gemini-adk-study-coach-lab

# Follow README instructions to configure
# Gemini credentials and install dependencies.

./scripts/run_local.sh --full-demo
Enter fullscreen mode Exit fullscreen mode

Start Prometheus, Tempo, OpenTelemetry Collector, and Grafana:

docker compose --profile observability up -d \
  --no-deps otel-collector tempo prometheus grafana
Enter fullscreen mode Exit fullscreen mode

The local workshop exposes several interfaces:

Component Local URL
Study Coach http://localhost:8501
Workshop Launcher http://localhost:8500
Google ADK Developer UI http://localhost:8000
Prometheus http://localhost:9090
Grafana http://localhost:3000
Tempo API http://localhost:3200
Application Metrics http://localhost:8001/metrics

For this demo, the Study Coach runs locally while the monitoring services run in Docker.

Running locally

Let's see how GUI of services shows -

All services running in  GUI

3. Workshop exercise: Generate a personalized 14-day study plan

Let's prepare for the Google Cloud Generative AI Leader certification.

My demo configuration:

  • Experience: Beginner
  • Study duration: 14 days
  • Daily availability: 90 minutes
  • Total planned study time: 1,260 minutes

The Study Planner generated a day-by-day learning schedule with time allocations, foundational topics, practice activities, and supporting resources.

Generated 14-day study plan

On Day 1, for example, the plan allocated time between core Generative AI concepts and practice activities.

The important engineering detail is that time-budget constraints are validated through Python rather than relying solely on the model's generated text.

Study Coach

This combination of generative AI and deterministic validation helps make the study plan more predictable and useful.

4. Workshop exercise: Learn LLM architecture with an expert agent

Next, I selected Large Language Models (LLMs) in the Topic Expert experience.

Instead of returning a plain-text explanation, the application provided multiple learning modes:

  • Learning objectives and concept summaries
  • Architecture diagrams
  • Step-by-step lessons
  • Runnable Python examples
  • Verified external learning resources
  • Practice questions

Topic Expert

For example, the Transformer architecture lesson visualizes tokenization, embeddings, transformer blocks, and autoregressive decoding.

Architecture diagram

The practical Python section demonstrates tokenization concepts using a small simulation. It is intentionally educational rather than pretending to implement a production tokenizer.

Hands-on Python

The app also integrates curated resources from official Google Cloud documentation and community educators such as Hugging Face and 3Blue1Brown.

For developer education, this makes an important difference: learners can move from a concept to a diagram, to code, and then to deeper reference material without switching between unrelated experiences.

5. Workshop exercise: Generate quizzes and validate answers

The Quiz Generator creates knowledge checks and formal practice questions.

In the demo, I deliberately selected an incorrect answer to a question about Transformer architecture.

The application identified the correct answer—self-attention—and provided an explanation.

Quiz evaluation

Why is this design important?

LLMs are useful for generating educational questions and explanations. However, scoring should be deterministic whenever the correct answer is already known.

In this implementation, Python handles the grading logic, making the result predictable and testable.

6. AgentOps: Observe what actually happens

Now for my favorite part of the workshop.

When an AI agent takes 20 or 40 seconds to complete a task, developers need more information than simply whether the request succeeded.

We want to know:

Which agent executed? Which tool was invoked? Which Gemini model call consumed time? Where did the latency occur?

To answer these questions, the lab integrates four observability technologies.

Technology Responsibility
OpenTelemetry Captures and exports execution telemetry
Prometheus Collects request, agent, tool, and latency metrics
Grafana Displays operational dashboards
Grafana Tempo Stores and explores distributed execution traces

Prometheus: What is happening?

The application exposes Prometheus metrics on port 8001.

We can query workflow activity using PromQL:

sum(study_coach_requests_total)
Enter fullscreen mode Exit fullscreen mode

To view specialized agent invocations:

sum by (agent) (study_coach_agent_calls_total)
Enter fullscreen mode Exit fullscreen mode

And to understand deterministic tool usage:

sum by (tool) (study_coach_tool_calls_total)
Enter fullscreen mode Exit fullscreen mode

During the captured demonstration, the Grafana dashboard displayed:

  • 5 workflow requests
  • 4 agent calls
  • 1 quiz submission

These are values from one local workshop run, not performance benchmarks.

Grafana UI

Grafana Tempo: Why did a request take so long?

Tempo gives us a more detailed execution view.

In the demo, a quiz-generation trace contained 16 spans over approximately 20.13 seconds.

It showed the sequence of operations including:

  • Study Coach request
  • ADK Quiz Generator invocation
  • Gemini model generation
  • Curriculum tool calls
  • Subsequent generation and processing

Grafana Tempo

This is the difference between monitoring that an agent was called and understanding how the agent actually executed.

Prometheus helps us understand patterns across requests. Tempo helps us investigate an individual request.

7. Real engineering lesson: A healthy UI does not mean healthy telemetry

During development, I encountered an interesting issue.

The Streamlit UI was online, but the application metrics endpoint on port 8001 was unreachable.

The problem involved Streamlit's execution lifecycle. The metrics exporter was initialized inside the Streamlit application script, which might not execute until a browser session connected.

We corrected the startup approach so that tracing and the Prometheus exporter initialize before the Streamlit application begins serving requests.

The exporter shares the same registry as the running agents and workflow instrumentation.

Key lesson: Observability needs to be part of the application lifecycle—not something assumed to work just because the UI loads.

This is exactly the kind of issue that developer workshops should expose: how to diagnose and improve real systems, not only demonstrate successful prompts.

8. Bonus: Document AI for personalized learning

The lab also supports uploading PDF and DOCX study guides.

The Document Agent retrieves relevant sections from uploaded material and supports document-grounded explanations and quizzes.

This extends the learning experience beyond a predefined certification curriculum.

Document AI

Document grounding is especially useful when learners want to study using their own course notes, certification documentation, or training material.

The engineering responsibility is to connect generated answers to retrieved evidence rather than relying on plausible-sounding responses.

9. Your turn: Build and extend the lab

The entire project is open source.

GitHub: https://github.com/hastimal/gemini-adk-study-coach-lab

Here is my challenge for developers:

  1. Clone the project and run the local workshop.
  2. Generate a personalized study plan using Gemini and Google ADK.
  3. Explore the Topic Expert, quizzes, and Document AI.
  4. Run a real request and inspect its metrics in Prometheus.
  5. Open Grafana Tempo and follow the agent/tool execution waterfall.
  6. Add a new specialist agent or deterministic tool—and instrument it.

The broader lesson goes beyond certification preparation.

The same engineering patterns apply to enterprise agentic AI: specialized agents, explicit tool boundaries, deterministic guardrails, evaluation, and observability.

Building an agent is only the beginning. Understanding, validating, and operating that agent is what makes the engineering interesting.

If you're experimenting with Gemini, Google ADK, or production-grade agentic applications, I'd love to hear what you're building.

⭐ Explore, fork, and contribute: https://github.com/hastimal/gemini-adk-study-coach-lab

Build. Learn. Evaluate. Observe.

GoogleCloud #Gemini #GoogleADK #AgenticAI #OpenTelemetry #Docker

Top comments (1)

Collapse
 
hjangid profile image
Hastimal Jangid •