DEV Community

Cover image for Gemma 4 GenAI Coach - GenAI Concepts Made Easy with an Interactive Playground
koushalya2001
koushalya2001

Posted on

Gemma 4 GenAI Coach - GenAI Concepts Made Easy with an Interactive Playground

#devchallenge #gemmachallenge #gemma

Gemma 4 Challenge: Build With Gemma 4 Submission

This is a submission for the Gemma 4 Challenge: Build with Gemma 4.

ModelX GenAI Interactive Playground: Learning Gemma 4, Grounding, and Agentic Design

What I Built

I built ModelX GenAI Interactive Playground, an opinionated learning portal that helps developers understand how to design, ground, and debug Gemma 4–powered chatbot and agentic applications.

Instead of being “just another chat UI”, the app is structured as a guided playground:

  • A multi‑turn chatbot vs agentic playground where you can flip between simple chat and tool‑using agent flows.
  • A grounding playground to compare:
    • prompt‑based grounding,
    • tool/MCP‑style grounding, and
    • RAG over a small in‑memory vector index.
  • A RAG & vector DB lab to reason about chunking, top‑k, token budgets, and context window pressure.
  • A prompt injection & safety lab where you can experiment with red‑team prompts and see simple safety heuristics in action.
  • A metrics & system‑design view that shows per‑turn token usage and latency for the current session, and generates a dynamic Mermaid diagram of the selected architecture.
  • A Gemma‑specific guide, coach, and quiz that walk the user through model choices and tradeoffs (26B A4B vs 31B, edge vs server), grounded in the official Gemma documentation.1
  • A Google AI Studio context caching demo that shows how to create cached context for Gemma 4 and compare usage/latency with and without caching.2

The goal is not just to “host a model”, but to make the design decisions around Gemma‑based systems visible and learnable.

Demo

  • Live app: https://modelx-genai-interactive-playground-ulwenmfdutovvcfyffv4x5.streamlit.app/
  • Recommended walkthrough:
    1. Start at “⭐ How to use this portal (start here)” in the left sidebar.
    2. Pick your backend (OpenRouter or Google AI Studio) and a Gemma 4 model.
    3. Go to Grounding playground to see how the same question changes under different grounding strategies.
    4. Move to Playground (LLM & Agent) to compare chatbot vs agentic flows and watch per‑turn metrics.
    5. Visit Prompt injection & safety lab and RAG & Vector DB lab for deeper experiments.
    6. Finish at Metrics scorecard & system design and Google context caching demo to inspect your telemetry and caching behavior.

(You can embed a Loom / YouTube walkthrough or GIF here once you record it.)

Code

Key modules:

  • app.py – Streamlit entrypoint, backend selection (OpenRouter vs Google AI Studio), navigation, and the persistent “active backend/model” header.
  • openrouter_client.py – Thin wrapper around OpenRouter’s OpenAI‑compatible chat API (Gemma 4 26B A4B / 31B free routes) with token/latency metrics.
  • google_ai_client.py – Thin client around the Gemini API generateContent endpoint for Gemma 4 models.
  • playground.py – Multi‑turn chatbot vs agent loop with configurable context strategies.
  • grounding_playground.py – Side‑by‑side grounding experiments (prompt grounding, tool/MCP‑style, and RAG).
  • rag_lab.py – RAG/vector DB design lab with token‑budget reasoning.
  • safety_lab.py – Prompt injection & safety heuristics.
  • metrics_scorecard.py – Per‑session summary and per‑turn metrics view (stored in st.session_state.interactions).
  • diagrams.py – Mermaid system diagram generator driven by user selections.
  • gemma_guide.py / gemma_coach.py – Gemma‑focused guide, coach, and quiz.
  • google_context_cache_demo.py – Google AI Studio context caching demo using cachedContents.
  • how_to_use.py, ui_flow_explainer.py, infra_explainer.py – documentation pages inside the app.

How I Used Gemma 4

Model choice

The playground is built around the Gemma 4 family, focusing on:

  • Gemma 4 26B A4B – Mixture‑of‑Experts configuration (26B total, 4B active per token), tuned for strong reasoning and long‑context workloads on small servers.
  • Gemma 4 31B – dense 31B model for maximum quality on server deployments, with strong reasoning, coding, and long‑context capabilities.

These sizes are a sweet spot for an educational portal:

  • Small enough to be usable via hosted APIs.
  • Large enough to clearly show the impact of prompt design, context trimming, grounding, and RAG on quality.

The UI lets you switch models in the sidebar, and the active backend + model are always shown at the top of the main area so users know exactly which configuration they’re exploring.

Backends: OpenRouter and Google AI Studio (user‑selectable)

The app supports both OpenRouter and Google AI Studio, but always uses exactly one backend at a time. The user:

  1. Chooses which backends to configure in the sidebar.
  2. Enters:
    • An OpenRouter API key (for easy access to free Gemma 4 26B A4B / 31B routes), or
    • A Google AI Studio key (for direct Gemini API calls to Gemma 4 models).
  3. Picks the active backend for this session via a radio button.

Under the hood:

  • When OpenRouter is active:

    • openrouter_client.py calls https://openrouter.ai/api/v1/chat/completions with the selected Gemma 4 model ID and normalizes token/latency usage.
    • This is a nice “batteries‑included” path for users who don’t want to set up Google AI Studio immediately.

  • When Google AI Studio is active:

    • google_ai_client.py calls https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent with the user’s AI Studio key.
    • The app surfaces usageMetadata fields such as promptTokenCount and candidatesTokenCount for metrics, and google_context_cache_demo.py shows how to use cachedContents to get cachedContentTokenCount for context caching.

This separation makes it easy for learners to switch between “router‑style” and “direct provider” patterns without changing any of the UI or agent logic.

Context caching (Google AI Studio)

The Google context caching demo page walks through the Gemini API’s context caching for Gemma 4:

  1. Create cached context

    • Calls POST /v1beta/cachedContents with a large, stable prompt (e.g. Gemma 4 documentation and system instructions).
    • Stores the returned cachedContents/... name in st.session_state.

  2. Compare calls

    • Call A (no cache): uses generateContent with full context + question in one prompt.
    • Call B (with cache): uses cachedContent: "cachedContents/..." and sends only the new question.

  3. Inspect usage and latency

    • The app shows usageMetadata for both calls:
      • promptTokenCount
      • candidatesTokenCount
      • totalTokenCount
      • cachedContentTokenCount (for the cached call)
    • Users can see how many tokens are reused from cache and how latency changes.

This gives a very concrete, visual way to understand why context caching matters for long prompts and how Gemma 4 can take advantage of it via the Gemini API.

Agentic patterns & grounding

The Playground (LLM & Agent) and Grounding playground expose agent plumbing in a transparent way:

  • Chatbot vs agentic mode:

    • Chatbot mode: forwards the multi‑turn messages list to Gemma.
    • Agentic mode:
    • Runs toy tools (math evaluator, pseudo search).
    • Injects tool outputs as labeled assistant turns at the end of the context.
    • Applies a context strategy:
      • Keep all turns.
      • Sliding window (last N turns).
      • Summarize older history when beyond a threshold.

  • Grounding playground:

    • Prompt‑based grounding: paste “docs” text into the system prompt and compare answers with/without that text.
    • Tool/MCP‑style grounding: simulate a docs/search MCP server by pasting tool output and injecting it either as:
    • an assistant turn, or
    • part of the system prompt.
    • RAG‑style grounding:
    • Uses an in‑memory TF‑IDF + cosine similarity index (lightweight stand‑in for FAISS) to retrieve top‑k chunks from a corpus.
    • Appends those chunks to the prompt, mirroring a full RAG pipeline without adding heavy dependencies.

The RAG & Vector DB lab then lets users play with chunk size and top‑k and explains how those choices affect token budgets, latency, and KV/cache pressure in Gemma‑based systems.

System Design

The portal is intentionally small and composable: a thin agentic shell around Gemma 4 with strong observability and multiple “labs” that share the same backend, model, and session state.

Architecture diagram

Components

  • UI (Streamlit)

    • Sidebar:
    • Backend selection (OpenRouter vs Google AI Studio).
    • Model selection per backend.
    • Context strategy.
    • Navigation between all playgrounds and docs pages.
    • Main area:
    • Chat/agent playground.
    • Grounding, RAG, safety, and caching labs.
    • Metrics table and system diagram.
    • “How to use this portal”, UI–Agent flow, and Infra/Serving docs.

  • Session state & controller

    • Shared st.session_state holds:
    • chat_history per page,
    • interactions (per‑turn metrics),
    • architecture_choices,
    • context_strategy,
    • backend_config.
    • Agent controller:
    • Builds messages from history.
    • Injects grounding/tool/RAG context as separate turns.
    • Applies context strategy before calling the active backend client.

  • Metrics & system design

    • Every model call appends an entry to interactions with tokens, latency, model, app type, framework, context strategy, and (optionally) safety flags.
    • metrics_scorecard.py:
    • Computes per‑session aggregates.
    • Displays a per‑turn table for the current session only.
    • diagrams.py:
    • Generates a Mermaid diagram based on architecture_choices so the UI always reflects the user’s current design.

  • Labs

    • Grounding, RAG, safety, and caching labs all use the same backend client and session state, but with different wiring patterns. This keeps the mental model consistent: change the wiring, not the model.

How I Used this Portal for Learning

To help new users, I added a dedicated page:

⭐ How to use this portal (start here)

It explains:

  • How to select backends/models.
  • A recommended learning path:
    • Grounding → Playground → Safety → RAG → Metrics → Gemma guide/coach → Context caching.
  • Where to find deeper technical details:
    • UI–Model–Agent flow shows exactly how user input, session state, agent logic, and model calls fit together.
    • Infra & Serving 101 maps these patterns to OpenRouter, Google AI Studio, and vLLM/local.

By the end, users don’t just know how to call Gemma 4; they have an experiential understanding of how to architect, ground, observe, and iterate Gemma‑powered systems across multiple serving options.

  1. Based on Gemma 4 docs and overviews. ↩

  2. Based on the Gemini API context caching docs. ↩

Top comments (0)