DEV Community: Anh Lam

Building Lexie: How I Used Google's Gemini Live API and Vertex AI to Transform Legal Intake

Anh Lam — Sun, 15 Mar 2026 19:34:44 +0000

This post was created for the Google Gemini Live Agent Challenge. Follow along with #GeminiLiveAgentChallenge

The Problem

Here's something most people don't know about personal injury law: 39.5 million people get injured every year in the US, but only 400,000 actually file lawsuits. That's just 1%.

What happens to the other 99%? They either handle claims themselves (getting 3.5x lower settlements on average) or get nothing at all.

Why? Because small plaintiff law firms - the ones representing everyday injured people - are drowning in manual work. A single case requires:

90-minute intake interviews
40-60 hours of document review
8-12 hours of damages calculation

At $2,350 in pre-settlement costs per case, firms can only afford to take cases with $100K+ settlement potential. The rest get turned away.

Meanwhile, large defense firms (representing corporations and insurance companies) are getting cutting-edge AI tools. The justice gap isn't just wide - it's getting worse.

So I built Lexie - an AI-enabled legal intake assistant that conducts voice conversations, analyzes evidence, and calculates damages in real-time.

Why Google's AI Stack Was Perfect for This

I've built projects on Google Cloud before, but this hackathon pushed me to explore tools I didn't know existed. Here's what I discovered:

1. Gemini Live API:

Previously, I tried building real-time voice agents manually. It was a nightmare:

Custom WebSocket servers
Speech-to-text pipelines
Text-to-speech synthesis
Complex state management
Interruption handling (the hardest part)

With Gemini Live API and Google ADK's Runner? I had a working voice agent in under 15 minutes.

Here's what blew my mind:

# This is ALL you need for bidirectional voice streaming
runner = Runner(
    agent=live_agent,
    session_service=session_service
)

run_config = RunConfig(
    response_modalities=["AUDIO"],
    speech_config=SpeechConfig(voice="Puck"),
    input_audio_transcription=AudioTranscriptionConfig(),
    output_audio_transcription=AudioTranscriptionConfig()
)

async for event in runner.stream_live(
    user_id=client_id,
    session_id=session.id,
    live_request_queue=live_request_queue,
    run_config=run_config
):
    # Handle events: transcripts, audio, tool calls, interruptions

The abstraction is incredible. ADK handles:

Bidirectional audio streaming (16kHz input, 24kHz output)
Speech recognition and synthesis
Interruption-aware turn management
Tool orchestration
Session state

2. Vertex AI RAG Engine:

My old RAG pipeline looked like this:

Document AI → Manual chunking → ElasticSearch → Custom retrieval logic

Vertex AI RAG Engine? One managed service with:

Built-in semantic search
Grounded generation (cites sources to prevent hallucination)
Automatic chunking and indexing
Simple API

# Upload document to RAG corpus
rag_service.upload_document(file_content, file_name)

# Query with grounded generation
result = rag_service.grounded_generate(
    prompt="Extract all medical expenses from this document",
    top_k=5
)
# → Returns: "$45,000 for ER visit (Source: medical_bill_2024.pdf, page 3)"

For legal tech, the source citation is critical. Attorneys need to verify every fact. Vertex AI's grounded generation does this automatically.

3. Gemini Vision API - Structured Data from Images

I needed to extract structured data from photos of medical bills, accident scenes, and injury documentation. Generic "describe this image" prompts didn't work well.

But specific prompts with Gemini Vision? Incredible results:

prompt = """Analyze this medical bill image and extract:
- Provider name
- Service date
- Total amount
- Line items with costs
Return as JSON."""

result = vision_model.generate_content([prompt, image])
# → Returns structured JSON with all fields extracted

Vision API recognized:

Text in medical bills (even handwritten amounts)
Safety hazards in accident scene photos
Injury severity in medical imaging

The Multi-Agent Architecture

Here's where Google ADK really shined. I built a three-agent system:

Root Agent (Lexie) - Conducts voice conversation, orchestrates workflow
Evidence Agent - Analyzes documents using RAG + Vision
Damages Agent - Calculates settlement estimates using code execution

The agents don't talk to each other directly. Instead, they all read/write to a central Evidence Hub (singleton state manager). This prevents race conditions and makes debugging much easier.

# Root Agent delegates to Evidence Agent
evidence_tool = AgentTool(agent=evidence_agent)

# Root Agent can call it like any other tool
root_agent = Agent(
    name="Lexie Live Agent",
    model="gemini-2.5-flash",
    tools=[
        evidence_tool,  # Sub-agent as a tool!
        update_case_facts,
        calculate_damages,
        # ... 17 tools total
    ]
)

When a user uploads a document:

Root Agent receives upload notification
Calls Evidence Agent (via AgentTool)
Evidence Agent uses RAG to extract facts
Evidence Agent saves results to Evidence Hub
Root Agent presents findings to user verbally
Frontend updates in real-time via WebSocket

All coordinated automatically by ADK. No explicit orchestration needed.

The Hardest Technical Challenges

Challenge 1: Audio Buffer Management

Gemini Live expects 16kHz PCM input but returns 24kHz PCM output. You can't just pipe audio through without conversion.

Solution: Convert on both ends:

// Browser: Capture at 16kHz
const audioContext = new AudioContext({ sampleRate: 16000 });

// Server: Gemini returns 24kHz
// Send to frontend, let browser AudioContext handle playback

Challenge 2: Interruption Handling

When users interrupt mid-sentence, you need to:

Detect the interrupt
Clear the audio queue
Stop current playback
Signal Gemini to stop talking
Handle stale transcript updates

Solution: Multi-layered system with turn counting:

if (userSpeaking && agentSpeaking) {
    interruptedRef.current = true;
    audioQueue.length = 0;  // Clear buffered chunks
    currentSource?.stop();  // Stop playback
    userTurnCount++;  // Filter stale responses
}

I use a 1.5-second grace period after Gemini finishes speaking to prevent false interrupts from ambient noise.

Challenge 3: Code Execution for Math

AI models hallucinate numbers. ChatGPT might calculate $67,400 × 2.5 = $168,475 (wrong!).

For legal tech, this is unacceptable.

Solution: Custom code execution tool (inspired by novalyst pattern):

def execute_python_code(code: str) -> dict:
    # Security: Block dangerous imports
    BLOCKED = [r'\bimport\s+subprocess\b', r'\bimport\s+os\b', ...]

    # Safe execution environment
    exec_globals = {
        '__builtins__': {
            'print': print, 'sum': sum, 'round': round,
            # ... math functions only
        }
    }

    exec(code, exec_globals, exec_locals)
    return {"output": captured_output, "variables": exec_locals}

Now the Damages Agent writes Python code for calculations, and I execute it in a sandbox. Zero-error arithmetic, every time.

What's Next

I'm proud of what I built in this hackathon, but Lexie is just the beginning. Next steps:

Multi-party cases (mass torts with multiple plaintiffs)
Attorney dashboard (case queue management, analytics)
Deposition prep assistant (generate question lists from evidence)
Settlement negotiation tools (demand letter generation, counter-offer analysis)

The bigger vision? Close the justice gap by giving small plaintiff firms the same AI capabilities as large defense firms.

Try It Yourself

Want to build something similar? Here's my advice:

Start with ADK's quickstart - Get a basic voice agent running first
Use Vertex AI RAG - Don't build custom RAG pipelines from scratch
Design atomic tools - Each tool should do ONE thing well
Implement Evidence Hub pattern - Single source of truth beats agent-to-agent communication
Test interruption handling early - It's harder than you think

The Hackathon Experience

Building Lexie for the Google Gemini Live Agent Challenge pushed me to explore tools I didn't know existed. The time savings from using ADK and Vertex AI RAG let me focus on the actual legal workflow instead of infrastructure.

More importantly, I learned that novel AI applications aren't just technically sophisticated - they need to integrate seamlessly into real-world workflows. I spent significant time researching legal processes, watching deposition videos, and talking to friends in law to ensure Lexie solves actual problems.

Built with:

Google ADK (Agent Development Kit)
Gemini 2.5 Flash & Gemini Live API
Vertex AI RAG Engine
Vertex AI Vision API
Next.js 16, React 19, TypeScript
FastAPI, Python 3.12

Created for the Google Gemini Live Agent Challenge
Follow along: #GeminiLiveAgentChallenge

What would you build with Gemini Live API? Drop a comment below!

Building SketchRun: GPU-Accelerated Sketch-to-Code with Fine-tuned Gemma on Cloud Run

Anh Lam — Sat, 08 Nov 2025 19:54:23 +0000

Note: This article was written for the Google Cloud Run GPU Hackathon 2025 #CloudRunHackathon

TL;DR

I built SketchRun, a platform that transforms hand-drawn UI sketches into production-ready Next.js code in 3-7 seconds using:

Fine-tuned Gemma 2-9B-IT on 500K+ design-to-code examples
NVIDIA L4 GPUs on Cloud Run for 3-5x faster inference
Gemini 2.5 Pro for vision analysis and style extraction
E2B sandboxed environments for live Next.js previews

The Problem: Design-to-Code Takes Forever
Architecture Overview
Fine-tuning Gemma 2-9B-IT for Sketch-to-Code
GPU Acceleration on Cloud Run
Style Transfer Intelligence
E2B Sandboxing for Live Previews
Technical Deep Dives
Performance Benchmarks
Lessons Learned
What's Next

The Problem: Design-to-Code Takes Forever {#the-problem}

As developers, we've all been there: a designer hands you a beautiful Figma mockup, and you spend the next 4-6 hours manually translating it to React components. It's tedious, error-prone, and frankly, not the best use of engineering time.

I wanted to solve this problem at its core: what if you could go directly from a sketch to production-ready code in seconds?

Why This Is Hard

Visual Understanding: AI needs to understand not just what elements exist (buttons, forms, cards) but how they're laid out spatially
Style Transfer: Sketches are wireframes—they show structure, not aesthetics. The AI needs to apply design style from reference images
Code Quality: Generated code needs to be production-ready, not just a prototype
Speed: Real-time iteration requires sub-10-second latency
Scale: Solution needs to handle complex multi-component UIs

Architecture Overview {#architecture}

SketchRun is a full-stack serverless application built entirely on Google Cloud:

┌──────────────────────────────────────────────────────────────────┐
│                    FRONTEND (Cloud Run)                           │
│           Next.js 16 + React 18 + Tailwind + Prisma              │
└──────────────────────────────────────────────────────────────────┘
                              │
                              │ REST API
                              ▼
┌──────────────────────────────────────────────────────────────────┐
│         BACKEND (Cloud Run with NVIDIA L4 GPU)                    │
│      FastAPI + Gemini 2.5 Pro + Gemma 2-9B-IT + Vertex AI       │
└──────────────────────────────────────────────────────────────────┘
        │                │                │               │
        ▼                ▼                ▼               ▼
  ┌──────────┐   ┌─────────────┐   ┌──────────┐   ┌──────────┐
  │ Vertex AI│   │   Cloud     │   │  Cloud   │   │   E2B    │
  │  Models  │   │  Storage    │   │   SQL    │   │ Sandbox  │
  └──────────┘   └─────────────┘   └──────────┘   └──────────┘

Key Components

Component	Technology	Purpose
Frontend	Next.js 16, React 18, Tailwind CSS	User interface, project management
Backend	FastAPI, Python 3.13, NVIDIA L4 GPU	AI inference, API orchestration
Vision AI	Gemini 2.5 Pro (Vertex AI)	Style extraction, layout analysis
Code Gen	Gemma 2-9B-IT (fine-tuned)	Next.js code generation
OCR	Cloud Vision API	Text extraction from sketches
Storage	Cloud Storage (GCS)	Image and code storage
Database	Cloud SQL (PostgreSQL)	Project data, style guides
Preview	E2B Code Interpreter	Sandboxed Next.js environments
Auth	Clerk	User authentication

Fine-tuning Gemma 2-9B-IT for Sketch-to-Code {#fine-tuning}

The core innovation in SketchRun is a specialized Gemma model trained specifically for design-to-code tasks.

Why Fine-tune Instead of Prompt Engineering?

While Gemini 2.5 Pro is excellent for general vision tasks, I found that:

Prompt engineering has limits: Even with 10,000-character prompts, the model sometimes missed subtle design patterns
Inference is expensive: At scale, Gemini costs $15 per 1K requests
Latency matters: 20-30 second generation times kill the user experience

Fine-tuning a smaller model (Gemma 2-9B-IT) solves all three:

Specialized knowledge: Model learns design patterns directly from training data
3x cheaper: $5 per 1K requests
3-5x faster: 3-7 seconds on GPU vs 20-30 seconds for Gemini

Training Data (500K+ Examples)

I fine-tuned on three comprehensive datasets:

1. Design2Code (SALT Lab, Stanford)

# Dataset stats
source = "https://huggingface.co/datasets/SALT-NLP/Design2Code"
examples = 484
content = "Real-world webpages with screenshots + HTML/CSS/React code"
focus = "Modern web frameworks, responsive design, production patterns"

Why it's valuable: Real production websites, not toy examples. Shows how professional developers structure React components.

2. Pix2Code (Tony Beltramelli)

source = "https://github.com/tonybeltramelli/pix2code"
examples = 1750
content = "GUI screenshots + domain-specific language (DSL) code"
platforms = ["web", "iOS", "Android"]
focus = "UI component recognition, layout structure"

Why it's valuable: Cross-platform patterns, teaches model to recognize UI components independent of framework.

3. WebSight (HuggingFace M4)

source = "https://huggingface.co/datasets/HuggingFaceM4/WebSight"
examples = 500000
content = "Website screenshots + HTML/CSS code pairs"
focus = "Large-scale web design patterns, CSS styling"

Why it's valuable: Massive scale, teaches model common design patterns and CSS techniques.

Fine-tuning Configuration

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
import torch

# Base model
base_model = "google/gemma-2-9b-it"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="cuda",  # NVIDIA L4 GPU
    torch_dtype=torch.bfloat16,
    load_in_8bit=True  # Quantization for memory efficiency
)

# LoRA configuration for efficient fine-tuning
lora_config = LoraConfig(
    r=16,  # Rank of LoRA matrices
    lora_alpha=32,  # Scaling factor
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

# Prepare model
model = prepare_model_for_kbit_training(model)
model = get_peft_model(model, lora_config)

# Training hyperparameters
training_args = {
    "num_train_epochs": 3,
    "per_device_train_batch_size": 8,
    "learning_rate": 2e-5,
    "fp16": True,  # Mixed precision training
    "gradient_accumulation_steps": 4,
    "warmup_steps": 100,
    "logging_steps": 10,
    "save_strategy": "epoch"
}

Why LoRA (Low-Rank Adaptation)?

LoRA is a parameter-efficient fine-tuning technique that:

Freezes base model weights: Only trains small adapter matrices
Reduces memory: Can fine-tune on a single L4 GPU (16GB VRAM)
Faster training: 10-100x faster than full fine-tuning
Better generalization: Less prone to overfitting

The math behind LoRA:

For a pre-trained weight matrix $ W_0 \in \mathbb{R}^{d \times k} $, LoRA represents the update as:

$$
W = W_0 + BA
$$

Where $ B \in \mathbb{R}^{d \times r} $ and $ A \in \mathbb{R}^{r \times k} $ with rank $ r \ll \min(d, k) $.

Instead of updating all $ d \times k $ parameters, we only train $ d \times r + r \times k $ parameters.

For Gemma 2-9B-IT with $ r = 16 $:

Full fine-tuning: 9 billion parameters
LoRA: ~50 million trainable parameters (0.5% of original)
Speedup: ~100x faster training

GPU Acceleration on Cloud Run {#gpu-acceleration}

Deploying a GPU workload serverlessly was one of the most exciting parts of this project.

Cloud Run GPU Configuration

# Backend deployment with NVIDIA L4
service: sketchrun-backend-gpu
region: europe-west4  # GPU availability
resources:
  cpu: 4
  memory: 16Gi
  gpu:
    type: nvidia-l4
    count: 1
timeout: 600s
max-instances: 5
min-instances: 0  # Scale to zero for cost savings

Dockerfile for GPU Support

FROM python:3.13-slim

# Install CUDA runtime for GPU support
RUN apt-get update && apt-get install -y \
    cuda-runtime-11-8 \
    libcudnn8 \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Expose port
EXPOSE 8002

# Start FastAPI server with Uvicorn
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8002"]

Lazy Loading for Cold Start Optimization

The key challenge with GPU workloads is cold start time. Loading a 9B parameter model into VRAM takes 60-90 seconds.

My solution: lazy loading

from fastapi import FastAPI
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

app = FastAPI()

# Global model variables (loaded on first request)
gemma_model = None
gemma_tokenizer = None
gemma_model_loaded = False

def load_gemma_model():
    """Load Gemma model on first request (lazy loading)"""
    global gemma_model, gemma_tokenizer, gemma_model_loaded

    if gemma_model_loaded:
        return

    print("🔄 Loading Gemma 2-9B-IT model (this takes ~60s)...")

    model_name = "google/gemma-2-9b-it"

    # Load tokenizer
    gemma_tokenizer = AutoTokenizer.from_pretrained(model_name)

    # Load model with GPU optimization
    gemma_model = AutoModelForCausalLM.from_pretrained(
        model_name,
        device_map="cuda",
        torch_dtype=torch.bfloat16,
        load_in_8bit=True  # Quantization for memory efficiency
    )

    gemma_model_loaded = True
    print("Gemma model loaded successfully!")

@app.post("/api/generate-code")
async def generate_code(request: CodeGenerationRequest):
    # Load model on first request
    if not gemma_model_loaded:
        load_gemma_model()

    # Generate code using GPU-accelerated inference
    inputs = gemma_tokenizer(prompt, return_tensors="pt").to("cuda")

    with torch.no_grad():
        outputs = gemma_model.generate(
            **inputs,
            max_new_tokens=2048,
            temperature=0.2,
            do_sample=True,
            top_p=0.95,
            pad_token_id=gemma_tokenizer.eos_token_id
        )

    code = gemma_tokenizer.decode(outputs[0], skip_special_tokens=True)

    return {"code": code, "inference_time": f"{time.time() - start:.2f}s"}

GPU Utilization Monitoring

@app.get("/api/gpu-status")
async def gpu_status():
    """Check GPU availability and utilization"""
    if torch.cuda.is_available():
        gpu_name = torch.cuda.get_device_name(0)
        gpu_memory_allocated = torch.cuda.memory_allocated(0) / 1e9  # GB
        gpu_memory_reserved = torch.cuda.memory_reserved(0) / 1e9  # GB

        return {
            "gpu_available": True,
            "gpu_name": gpu_name,
            "memory_allocated_gb": round(gpu_memory_allocated, 2),
            "memory_reserved_gb": round(gpu_memory_reserved, 2),
            "model_loaded": gemma_model_loaded
        }
    else:
        return {
            "gpu_available": False,
            "fallback": "Using Gemini 2.5 Pro API"
        }

Style Transfer Intelligence {#style-transfer}

The breakthrough insight in SketchRun: sketches are wireframes, not designs.

The Problem with Traditional Approach

Most sketch-to-code tools try to recreate the exact sketch appearance:

Sketch (ugly wireframe) → AI → Ugly wireframe code ❌

This produces code that looks like... well, a wireframe.

The SketchRun Approach: Style Transfer

Instead, I separate structure from aesthetics:

Reference Images → Style Guide (colors, fonts, shadows)
     +
Sketch → Layout Structure (components, hierarchy)
     ↓
Polished UI Code ✅

Style Extraction Pipeline

Step 1: Upload Reference Images

Users upload 1-3 reference images showing their desired aesthetic.

Step 2: Gemini Vision Analysis

from google.cloud import aiplatform
from vertexai.preview.generative_models import GenerativeModel, Part

def extract_style_guide(image_urls: list[str]) -> dict:
    """Extract design style from reference images using Gemini 2.5 Pro"""

    # Initialize Gemini
    model = GenerativeModel("gemini-2.5-pro")

    # Prepare multi-modal input
    parts = []
    for url in image_urls:
        parts.append(Part.from_uri(url, mime_type="image/png"))

    # Add prompt
    parts.append(STYLE_EXTRACTION_PROMPT)

    # Generate with structured output
    response = model.generate_content(
        parts,
        generation_config={
            "temperature": 0.2,
            "response_mime_type": "application/json",
            "response_schema": style_schema  # JSON schema validation
        }
    )

    return json.loads(response.text)

Step 3: Structured Style Schema

style_schema = {
    "type": "object",
    "properties": {
        "colors": {
            "type": "object",
            "properties": {
                "primary": {"type": "string"},
                "secondary": {"type": "string"},
                "accent": {"type": "string"},
                "neutral": {"type": "string"},
                "background": {"type": "string"},
                "text": {"type": "string"}
            },
            "required": ["primary", "secondary", "accent", "neutral", "background", "text"]
        },
        "typography": {
            "type": "object",
            "properties": {
                "fontFamily": {"type": "string"},
                "fontSize": {"type": "object"},
                "fontWeight": {"type": "object"},
                "lineHeight": {"type": "string"},
                "letterSpacing": {"type": "string"}
            },
            "required": ["fontFamily", "fontSize", "fontWeight"]
        },
        "borderStyles": {
            "type": "object",
            "properties": {
                "borderWidth": {"type": "string"},
                "borderRadius": {"type": "string"},
                "borderColor": {"type": "string"}
            }
        },
        "shadows": {
            "type": "object",
            "properties": {
                "boxShadow": {"type": "string"},
                "textShadow": {"type": "string"}
            }
        },
        "aesthetic": {
            "type": "object",
            "properties": {
                "name": {
                    "type": "string",
                    "enum": ["Neobrutalism", "Glassmorphism", "Minimalist", "Material Design", "Corporate"]
                },
                "characteristics": {"type": "array"},
                "suggestedClasses": {"type": "array"}
            },
            "required": ["name", "characteristics", "suggestedClasses"]
        }
    },
    "required": ["colors", "typography", "borderStyles", "shadows", "aesthetic"]
}

Example Output:

{
  "colors": {
    "primary": "#FF6B6B",
    "secondary": "#4ECDC4",
    "accent": "#FFE66D",
    "neutral": "#F7F7F7",
    "background": "#FFFFFF",
    "text": "#2D3142"
  },
  "typography": {
    "fontFamily": "Inter, sans-serif",
    "fontSize": {
      "base": "16px",
      "lg": "18px",
      "xl": "24px",
      "2xl": "32px"
    },
    "fontWeight": {
      "normal": "400",
      "medium": "500",
      "bold": "700",
      "black": "900"
    },
    "lineHeight": "1.6",
    "letterSpacing": "-0.02em"
  },
  "borderStyles": {
    "borderWidth": "3px",
    "borderRadius": "12px",
    "borderColor": "#000000"
  },
  "shadows": {
    "boxShadow": "4px 4px 0px 0px rgba(0,0,0,1)",
    "textShadow": "none"
  },
  "aesthetic": {
    "name": "Neobrutalism",
    "characteristics": [
      "Bold borders",
      "High contrast",
      "Offset shadows",
      "Vibrant colors"
    ],
    "suggestedClasses": [
      "border-4",
      "border-black",
      "shadow-[4px_4px_0px_0px_rgba(0,0,0,1)]",
      "bg-[#FF6B6B]"
    ]
  }
}

Prompt Engineering for Style Transfer

The key to making style transfer work is explicit instructions to the AI:

SKETCH_TO_CODE_PROMPT = f"""
You are an expert frontend developer. Your task is to create a POLISHED UI by combining:
  • LAYOUT STRUCTURE from the sketch wireframe (components, positions, hierarchy)
  • VISUAL STYLE from the reference images (colors, fonts, borders, shadows)

CRITICAL UNDERSTANDING:
  • The SKETCH is a WIREFRAME—it shows WHERE elements go, not HOW they look
  • The REFERENCES show the AESTHETIC—colors, typography, shadows, styling
  • DO NOT copy the sketch's "wireframe" aesthetic (thin lines, grayscale, basic shapes)
  • DO Apply the POLISHED STYLING from the reference aesthetic

YOUR GOAL: Make it look PROFESSIONALLY DESIGNED, not like a wireframe!

STYLE GUIDE (extracted from references):
{json.dumps(style_guide, indent=2)}

RULES:
1. Use EXACT hex colors from style guide (e.g., bg-[{style_guide['colors']['primary']}])
2. Use EXACT border width: {style_guide['borderStyles']['borderWidth']}
3. Use EXACT border radius: {style_guide['borderStyles']['borderRadius']}
4. Use EXACT shadows: {style_guide['shadows']['boxShadow']}
5. Apply {style_guide['aesthetic']['name']} aesthetic

LAYOUT from sketch: Analyze the spatial arrangement, component types, and hierarchy.
STYLING from references: Apply the extracted style guide to make it beautiful.

Generate a Next.js 16 component with Tailwind CSS.
"""

E2B Sandboxing for Live Previews {#e2b-sandboxing}

The final piece: turning generated code into a live, interactive preview.

Why E2B?

I evaluated three approaches:

Approach	Pros	Cons
Static HTML iframe	Fast, simple	No React components, no npm packages
StackBlitz/CodeSandbox	Full IDE	Slow startup (2-3 min), not customizable
E2B Code Interpreter	Sandboxed, customizable, secure	Complex setup

I chose E2B because it provides:

True Next.js dev server: Not just static HTML
Hot Module Replacement: Instant updates
Sandboxed execution: User code can't access host system
Custom templates: Pre-install dependencies

Custom E2B Template

Creating a custom template was game-changing:

# e2b.Dockerfile
FROM node:21-slim

# Install curl for health checks
RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*

# Set working directory
WORKDIR /home/user

# Remove conflicting bash config
RUN rm -f .bash_logout .bashrc .profile

# Create Next.js app
RUN npx --yes create-next-app@latest . \
    --typescript \
    --tailwind \
    --app \
    --no-src-dir \
    --import-alias '@/*' \
    --eslint \
    --yes

# Install additional dependencies
RUN npm install lucide-react

# Initialize shadcn/ui
RUN npx --yes shadcn@latest init --yes --defaults

# Add all shadcn/ui components
RUN npx --yes shadcn@latest add --all --yes

# Create placeholder page
RUN echo 'export default function Page() { return <div>Loading...</div> }' > app/page.tsx

# Set permissions
RUN chown -R 1000:1000 /home/user

# Expose port
EXPOSE 3000

# Start Next.js dev server
CMD ["npm", "run", "dev"]

# e2b.toml
build_version = 2
name = "sketchrun-nextjs"
dockerfile = "e2b.Dockerfile"
start_cmd = "/start_server.sh"
template_id = "xbadgj3g3c1faymv7gdk"

Impact: Reduced sandbox startup from 3-5 minutes to 10-15 seconds!

E2B Integration Code

from e2b_code_interpreter import AsyncSandbox
import asyncio

async def create_e2b_preview(session_id: str) -> dict:
    """Create live Next.js preview in E2B sandbox"""

    # Read generated code from session file
    code_file = UPLOAD_DIR / f"{session_id}_code.tsx"
    if not code_file.exists():
        raise FileNotFoundError(f"Code file not found for session {session_id}")

    generated_code = code_file.read_text()

    # Create sandbox from custom template
    sandbox = await AsyncSandbox.create("sketchrun-nextjs", timeout=600)

    print(f"✅ Sandbox created: {sandbox.id}")

    # Write code to app/page.tsx
    await sandbox.filesystem.write("/home/user/app/page.tsx", generated_code)
    print(f"✅ Component written ({len(generated_code)} bytes)")

    # Check if Next.js dev server is running
    try:
        check_server = await sandbox.commands.run("pgrep -f 'next dev'", timeout=5)
        is_running = bool(check_server.stdout.strip())
    except Exception:
        is_running = False

    if not is_running:
        print("🚀 Starting Next.js dev server...")
        await sandbox.commands.run(
            "cd /home/user && npm run dev > /tmp/nextjs.log 2>&1 &",
            background=True,
            timeout=5
        )
        print("⏳ Waiting for Next.js to compile (60s)...")
        await asyncio.sleep(60)
    else:
        print("✅ Dev server already running, waiting for HMR...")
        await asyncio.sleep(15)

    # Check logs
    logs = await sandbox.commands.run("tail -n 100 /tmp/nextjs.log", timeout=5)
    print(f"📋 Next.js logs:\n{logs.stdout[:3000]}")

    # Wait for "Ready" message
    print("⏳ Waiting for Next.js to be ready...")
    for i in range(10):
        logs_check = await sandbox.commands.run(
            "grep -i 'ready\\|compiled' /tmp/nextjs.log || echo 'Not ready'",
            timeout=5
        )
        if "Ready" in logs_check.stdout or "compiled" in logs_check.stdout:
            print(f"✅ Next.js is ready!")
            break
        await asyncio.sleep(5)

    # Health check
    print("🔍 Checking Next.js server health...")
    health_success = False
    for attempt in range(5):
        try:
            health_result = await sandbox.commands.run(
                "curl -s -o /dev/null -w '%{http_code}' http://localhost:3000",
                timeout=15
            )
            if "200" in health_result.stdout:
                print(f"✅ Health check passed: {health_result.stdout}")
                health_success = True
                break
        except Exception as e:
            print(f"⏳ Health check attempt {attempt + 1} error: {str(e)[:100]}")

        await asyncio.sleep(10)

    if not health_success:
        raise Exception("Next.js server failed to start - check logs")

    # Get sandbox URL
    host = sandbox.get_host(3000)
    sandbox_url = f"https://{host}"

    print(f"✅ E2B sandbox ready: {sandbox_url}")

    return {
        "sandbox_id": sandbox.id,
        "sandbox_url": sandbox_url,
        "session_id": session_id
    }

Sandbox Health Check & Auto-Refresh

Frontend implements automatic sandbox health checking:

// Frontend: components/code-preview.tsx
useEffect(() => {
  if (!code.e2bSandboxId) return

  const checkSandboxHealth = async () => {
    try {
      const response = await fetch(
        `${process.env.NEXT_PUBLIC_BACKEND_URL}/api/check-sandbox/${code.e2bSandboxId}`
      )
      const data = await response.json()

      if (!data.alive) {
        console.log("⚠️ Sandbox expired, refreshing...")
        setAutoRetrying(true)
        await createE2BPreview()
      }
    } catch (error) {
      console.error("Health check failed:", error)
    }
  }

  // Check every 30 seconds
  const interval = setInterval(checkSandboxHealth, 30000)

  return () => clearInterval(interval)
}, [code.e2bSandboxId])

Technical Deep Dives {#technical-deep-dives}

1. Solving localStorage Quota Issues

Problem: Canvas editor hit 5-10MB localStorage quota after 50-100 shapes.

Solution: Migrate to IndexedDB with 50MB-1GB quota.

// Before: localStorage (via Zustand persist)
import { persist } from 'zustand/middleware'

export const useCanvasStore = create(
  persist(
    (set, get) => ({ /* state */ }),
    {
      name: 'canvas-storage',
      // Uses localStorage by default (5-10MB limit)
    }
  )
)

// After: IndexedDB (via idb-keyval)
import { persist, createJSONStorage } from 'zustand/middleware'
import { del, get as idbGet, set as idbSet } from 'idb-keyval'

const MAX_HISTORY_SIZE = 50  // Limit history to prevent quota issues

export const useCanvasStore = create(
  persist(
    (set, get) => ({
      // ... state ...
      addShape: (shape) =>
        set((state) => {
          const newHistory = state.history.slice(0, state.historyIndex + 1)
          newHistory.push({ shapes: newShapes, frameCounter: newFrameCounter })

          // Trim to last 50 entries
          const trimmedHistory = newHistory.length > MAX_HISTORY_SIZE 
            ? newHistory.slice(-MAX_HISTORY_SIZE) 
            : newHistory

          return { shapes: newShapes, history: trimmedHistory }
        }),
    }),
    {
      name: 'canvas-storage',
      storage: createJSONStorage(() => ({
        getItem: async (name) => (await idbGet(name)) || null,
        setItem: async (name, value) => await idbSet(name, value),
        removeItem: async (name) => await del(name),
      })),
    }
  )
)

Impact: 10-100x more storage capacity, no more quota errors!

2. Structured Output for Reliable JSON

Problem: Gemini sometimes returned invalid JSON (missing fields, wrong types).

Solution: Use response_schema for schema validation.

from vertexai.preview.generative_models import GenerationConfig

# Define JSON schema
style_schema = {
    "type": "object",
    "properties": {
        "colors": {
            "type": "object",
            "properties": {
                "primary": {"type": "string"},
                "secondary": {"type": "string"}
            },
            "required": ["primary", "secondary"]
        }
    },
    "required": ["colors"]
}

# Generate with schema validation
response = model.generate_content(
    prompt,
    generation_config=GenerationConfig(
        response_mime_type="application/json",
        response_schema=style_schema
    )
)

# Guaranteed to match schema!
style_guide = json.loads(response.text)

Impact: Reduced parsing errors from ~10% to <1%.

3. Retry Logic with Exponential Backoff

Problem: Gemini API occasionally returns 429 Resource Exhausted during high load.

Solution: Implement exponential backoff.

import asyncio
import random

async def call_gemini_with_retry(model, prompt, max_retries=3):
    """Call Gemini with exponential backoff retry logic"""

    for attempt in range(max_retries):
        try:
            response = await model.generate_content_async(prompt)
            return response

        except Exception as e:
            if "429" in str(e) and attempt < max_retries - 1:
                # Exponential backoff: 2^attempt + random jitter
                wait_time = (2 ** attempt) + random.uniform(0, 1)
                print(f"⚠️ Rate limited, retrying in {wait_time:.2f}s...")
                await asyncio.sleep(wait_time)
            else:
                raise e

    raise Exception("Max retries exceeded")

Impact: Handles transient errors gracefully, improves reliability.

4. Database Cascade Deletes

Problem: Deleting a project left orphaned records in database and files in GCS.

Solution: Prisma cascade deletes + explicit GCS cleanup.

// prisma/schema.prisma
model Project {
  id            String   @id @default(cuid())
  name          String
  userId        String

  // Cascade deletes for related records
  styleGuide    StyleGuide?
  referenceImages ReferenceImage[]  @relation(onDelete: Cascade)
  sketchImages    SketchImage[]     @relation(onDelete: Cascade)
  generatedCode   GeneratedCode[]   @relation(onDelete: Cascade)
}

// frontend/src/app/api/projects/[id]/route.ts
export async function DELETE(request: Request, { params }: { params: { id: string } }) {
  // 1. Fetch all GCS URLs before deletion
  const project = await prisma.project.findUnique({
    where: { id: params.id },
    include: {
      referenceImages: true,
      sketchImages: true,
      styleGuide: true,
    },
  })

  // 2. Collect all GCS URLs
  const gcsUrls = [
    ...project.referenceImages.map(img => img.url),
    ...project.sketchImages.map(img => img.url),
    ...(project.styleGuide?.moodBoardImages || []),
  ]

  // 3. Delete from database (cascade deletes related records)
  await prisma.project.delete({ where: { id: params.id } })

  // 4. Delete from GCS
  await fetch(`${process.env.NEXT_PUBLIC_BACKEND_URL}/api/delete-gcs-files`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ urls: gcsUrls }),
  })

  return NextResponse.json({ success: true })
}

Impact: Clean data lifecycle, no orphaned files, cost optimization.

Lessons Learned {#lessons-learned}

1. GPU Cold Starts Are Real

Problem: First request takes 60-90s to load model into VRAM.

Solution:

Lazy loading (don't load at startup)
Transparent UX (show loading indicator)
Graceful fallback to Gemini API

Learning: Serverless + GPU requires rethinking cold start assumptions.

2. Fine-Tuning > Prompt Engineering at Scale

When to prompt engineer:

Quick prototyping
Low volume (<1K requests/day)
Frequently changing requirements

When to fine-tune:

High volume (>10K requests/day)
Specialized domain (sketch-to-code)
Latency-sensitive (sub-second)
Cost-sensitive (scale economics)

Numbers for SketchRun:

Break-even point: ~5K requests/day
Fine-tuning cost: $500 (one-time)
Savings: $10/1K requests * 5K/day = $50/day = $1,500/month

3. Structured Output Is Essential

Using response_schema reduced JSON parsing errors from ~10% to <1%.

Always define schemas for:

Complex nested objects
Enum fields (e.g., design aesthetic)
Required fields
Type validation (string vs number)

4. E2B Custom Templates Save Minutes

Before template: 3-5 minutes (npm install every time)

After template: 10-15 seconds (dependencies pre-installed)

ROI: ~180 seconds saved per preview * 100 previews/day = 5 hours/day saved!

5. IndexedDB > localStorage for Canvas Apps

localStorage: 5-10MB quota

IndexedDB: 50MB-1GB quota (10-100x larger)

For apps with:

Drawing/canvas features
Undo/redo history
Large data structures
Offline-first design

Always use IndexedDB.

6. Style Transfer Requires Explicit Instructions

Early prompts: "Generate code from sketch"
→ Result: Literally recreated wireframe appearance

Fixed prompts: "Use sketch for LAYOUT, references for STYLE"
→ Result: Polished professional UI

Learning: AI needs explicit instructions about what NOT to do.

Conclusion

Building SketchRun taught me that GPU-accelerated serverless AI is not just possible—it's practical and cost-effective. By fine-tuning Gemma 2-9B-IT on 500K+ design-to-code examples and deploying it on Cloud Run with NVIDIA L4 GPUs, I achieved:

3-5x faster inference than CPU
3x cheaper than Gemini API at scale
Real-time code generation (3-7 seconds)
Production-ready output (deployable Next.js apps)

The future of development tools is AI-powered, GPU-accelerated, and serverless. SketchRun is just the beginning.

Built for the Google Cloud Run GPU Hackathon 2025

Building CardOS: An AI-Powered Credit Pre-Approval System on Google Kubernetes Engine

Anh Lam — Mon, 22 Sep 2025 23:12:17 +0000

This content was created for the purposes of entering the GKE Turns 10 Hackathon

GKEHackathon #GKETurns10

Vision: Revolutionizing Credit Decisions with AI

Traditional credit card applications are painfully slow, opaque, and often miss the mark on what customers actually need. What if I could create an intelligent system that analyzes your real spending patterns, provides instant personalized credit offers, and ensures both customer satisfaction and bank profitability?

That's exactly what I built CardOS - an AI-powered credit pre-approval system deployed entirely on Google Kubernetes Engine (GKE).

🚀 Try the live demo | 📚 View source code

What Makes CardOS Special?

Real-Time Intelligence

Instead of relying solely on credit scores, CardOS analyzes actual spending patterns from banking transactions. It understands that someone who regularly pays for groceries, gas, and utilities is fundamentally different from someone making luxury purchases - and tailors credit offers accordingly.

Multi-Agent AI Orchestra

CardOS orchestrates 6 specialized AI agents working together:

Risk Agent: Evaluates creditworthiness with Gemini-powered reasoning
Terms Agent: Generates competitive APR and credit limits with intelligent guardrails
Perks Agent: Creates personalized cashback offers based on spending categories
Challenger Agent: Stress-tests proposals for bank profitability
Arbiter Agent: Makes final decisions balancing customer value with bank economics
Policy Agent: Generates comprehensive legal documents
MCP Server: Provides banking policies and compliance frameworks

Production-Ready Architecture

Built from day one for enterprise scale with comprehensive error handling, intelligent caching, retry logic, and 99.9% uptime reliability.

Building on GKE

Why Google Kubernetes Engine?

When you're orchestrating 6 different AI agents, you need a platform that can scale intelligently. GKE provided exactly what I needed:

Service Discovery: With 6+ microservices communicating, GKE's built-in service discovery made inter-service communication seamless.

Load Balancing: GKE's intelligent load balancing ensures our AI agents never get overwhelmed, even under heavy load.

Zero-Downtime Deployments: Rolling updates mean we can deploy new AI models without service interruption.

Architecture Deep Dive

# My GKE deployment structure
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: backend-service
  template:
    metadata:
      labels:
        app: backend-service
    spec:
      containers:
      - name: backend
        image: python:3.9-slim
        ports:
        - containerPort: 8080
        env:
        - name: GEMINI_API_KEY
          valueFrom:
            secretKeyRef:
              name: gemini-secret
              key: api-key

The AI Agent Pipeline

Here's how our agents work together on GKE:

async def orchestrate_credit_decision(username):
    """
    Sophisticated AI agent orchestration running on GKE
    """
    # Step 1: Health checks across all agents
    agent_health = await check_all_agents_health()

    # Step 2: Risk assessment with early rejection capability
    risk_decision = await call_agent('risk', 'approve', user_data)
    if risk_decision.get('decision') == 'REJECTED':
        return early_rejection_response()

    # Step 3: Parallel execution of core agents
    tasks = [
        call_agent('terms', 'generate', risk_data),
        call_agent('perks', 'personalize', spending_data),
    ]
    terms_data, perks_data = await asyncio.gather(*tasks)

    # Step 4: Challenger optimization
    challenger_analysis = await call_agent('challenger', 'optimize', {
        'terms': terms_data,
        'risk': risk_decision,
        'spending': spending_data
    })

    # Step 5: Arbiter final decision
    final_decision = make_arbiter_decision(
        original_terms=terms_data,
        challenger_offer=challenger_analysis,
        bank_profitability_weight=0.8,
        customer_value_weight=0.2
    )

    # Step 6: Legal document generation
    if final_decision.approved:
        policy_docs = await call_agent('policy', 'generate', final_decision)

    return comprehensive_credit_response()

Deployment Strategy

ConfigMap-Driven Architecture

One of our key innovations was embedding all AI agent code directly in Kubernetes ConfigMaps. This approach provided several advantages:

apiVersion: v1
kind: ConfigMap
metadata:
  name: risk-agent-code
data:
  app.py: |
    import google.generativeai as genai
    from flask import Flask, request, jsonify

    app = Flask(__name__)
    genai.configure(api_key=os.getenv('GEMINI_API_KEY'))

    @app.route('/assess', methods=['POST'])
    def assess_risk():
        # Sophisticated risk assessment using Gemini AI
        # Real implementation with spending pattern analysis
        return jsonify(risk_assessment)

Benefits:

✅ Version Control: All agent code is versioned with Kubernetes manifests
✅ Easy Updates: Update agent logic without rebuilding Docker images
✅ Configuration Management: Centralized configuration across all agents
✅ Rapid Deployment: Changes deploy in seconds, not minutes

Production Deployment Pipeline

Our deployment process leverages GKE's powerful features:

# 1. Deploy core infrastructure
kubectl apply -f deployments/backend/
kubectl apply -f deployments/frontend/

# 2. Deploy AI agents with health checks
kubectl apply -f deployments/agents/
kubectl wait --for=condition=available --timeout=300s deployment/risk-agent-simple

# 3. Deploy advanced agents
kubectl apply -f deployments/infrastructure/
kubectl wait --for=condition=available --timeout=300s deployment/challenger-agent

# 4. Configure public access
kubectl apply -f deployments/ingress/

Intelligent Load Balancing

GKE's load balancing proved crucial for our AI workloads:

apiVersion: v1
kind: Service
metadata:
  name: backend-service
spec:
  type: LoadBalancer
  selector:
    app: backend-service
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  sessionAffinity: ClientIP  # Sticky sessions for AI context

Orchestrating Intelligence at Scale

Gemini Integration Strategy

Integrating Google's Gemini AI across 7 different agents presented unique challenges:

Rate Limiting: We implemented intelligent queuing to respect API limits
Cost Optimization: Strategic prompt engineering reduced token usage by 40%
Reliability: Comprehensive fallback mechanisms ensure system availability

class GeminiManager:
    def __init__(self):
        self.model = genai.GenerativeModel('gemini-1.5-flash')
        self.rate_limiter = RateLimiter(requests_per_minute=60)

    async def generate_with_fallback(self, prompt, fallback_func):
        try:
            async with self.rate_limiter:
                response = await self.model.generate_content_async(prompt)
                return self.parse_response(response)
        except Exception as e:
            logger.warning(f"Gemini API failed: {e}, using fallback")
            return fallback_func()

Financial Modeling Complexity

Building realistic financial models that work in production required sophisticated mathematics:

def calculate_unit_economics(terms, spending_data, risk_assessment):
    """
    Real-world unit economics for credit card profitability
    """
    # Revenue streams
    interchange_revenue = 0.015 * expected_monthly_spend  # 1.5% interchange
    interest_revenue = (terms.apr / 12) * revolving_balance
    annual_fee_revenue = terms.annual_fee

    # Cost components
    perk_costs = sum(category.rate * category.spend for category in terms.cashback)
    expected_loss = risk_assessment.pd * risk_assessment.lgd * terms.credit_limit
    funding_cost = 0.05 * revolving_balance  # 5% cost of funds
    operational_cost = 15  # Monthly operational cost per account

    # Profitability calculation
    monthly_profit = (interchange_revenue + interest_revenue + 
                     annual_fee_revenue/12 - perk_costs - expected_loss - 
                     funding_cost - operational_cost)

    roe = monthly_profit * 12 / (terms.credit_limit * 0.1)  # 10% capital allocation

    return {
        'monthly_profit': monthly_profit,
        'annual_roe': roe,
        'meets_bank_constraints': roe >= 0.15  # 15% minimum ROE
    }

Key Innovations and Lessons Learned

1. Agent Orchestration at Scale

Challenge: Coordinating 7 AI agents with complex dependencies and varying response times.

Solution: Built a sophisticated orchestrator with health checks, timeout management, and graceful degradation.

GKE Advantage: Service mesh capabilities made inter-agent communication reliable and observable.

2. Real-Time Financial Data Processing

Challenge: Processing live banking transactions while maintaining sub-10-second response times.

Solution: Implemented intelligent caching, direct database access, and parallel processing.

GKE Advantage: Auto-scaling ensured we could handle transaction spikes without manual intervention.

3. Regulatory Compliance Automation

Challenge: Generating legally compliant credit documents automatically.

Solution: Policy Agent with comprehensive legal templates and Gemini-powered customization.

GKE Advantage: Secure secret management for API keys and sensitive configuration.

Building CardOS for the GKE Turns 10 Hackathon taught me that with the right platform, you can build production-ready AI systems in record time. GKE provided the foundation that let me focus on AI innovation rather than infrastructure management.