DEV Community: Shir Meir Lador

Fine-Tuning Gemma 3 with Cloud Run Jobs: Serverless GPUs (NVIDIA RTX 6000 Pro) for pet breed classification 🐈🐕

Shir Meir Lador — Thu, 09 Apr 2026 13:07:00 +0000

Architectural worklow: fine tuning Gemma 3 27B on Cloud Run Jobs

Recently, I was inspired by a major new release on Google Cloud: the availability of NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs on Cloud Run Jobs. This launch is important because it unlocks the ability to tackle fine-tuning workloads for open models with the simplicity of a serverless batch job. To put this new hardware to the test in a fun way, I fine tuned a multi-modal model to identify a pet’s breed from a photo using The Oxford-IIIT Pet Dataset. This model could be used for a “Smart pet care” — an AI application that identifies a pet’s breed from a photo and provides tailored health and nutrition advice.

Image taken from The Oxford-IIIT Pet Dataset and showcase the images of cats and dogs and their corresponding breed — the classification label

Why Fine-Tuning?

In a recent Agent Factory episode, we discussed that while foundational models are a powerful ‘one-size-fits-all’ starting point, they essentially remain generalists. You should consider fine-tuning when you have a problem that requires high specialization that a generalist model might not excel in on its own, or when you need more control and cost-efficiency over your own hosting.

For this pet-care use case, distinguishing between 37 different breeds isn’t just about ‘knowledge’, it’s about taking that foundational reasoning and adding a specific capability based on a unique dataset. As we explored in the episode and as mentioned in this Nvidia paper, this kind of specialization is what allows smaller, focused models to become sufficiently powerful and economical for production agentic systems. Fine-tuning acts as the necessary bridge, transforming a broad reasoner into a high-precision classification expert.

Bridging Reasoning and Precision

For this project, I chose the multimodal breadth of Gemma 3 27B. While specialized vision models often provide superior accuracy for narrow identification tasks, I wanted to use a model capable of both identifying breeds and reasoning about the specific health and dietary needs associated with them. By leveraging the power of the new Blackwell GPUs, I was able to fine-tune this model to bridge the performance gap, all while keeping the setup reproducible, cost-effective, and entirely container-native.

From Batch to Production: Economically Efficient Hosting

The true ‘deploy and forget’ magic happens after the weights are saved. With high-performance inference now supported on Cloud Run, you can host your fine-tuned Gemma 3 27B model on the same NVIDIA RTX PRO 6000 Blackwell GPU without managing any underlying infrastructure. This setup delivers a highly economical production environment: Cloud Run automatically scales your GPU instances to zero when they aren’t in use, ensuring you only pay for the exact minutes your model is active.

In this guide, I’m excited to show you how this new hardware release transforms complex fine-tuning into a scalable, serverless experience without the need to manage complex clusters or maintain idle instances.

Simplifying 27B Fine-Tuning on Cloud Run

Fine-tuning an open model can seem like a daunting task that requires complex orchestration, from provisioning high-capacity VMs and manually installing CUDA drivers to managing tedious data transfers and scaling down manually to control costs. Cloud Run Jobs elegantly solves this by allowing you to package your training logic as a container, now backed by the fully managed environment of NVIDIA RTX PRO 6000 Blackwell GPUs and their 96GB of VRAM.

This setup delivers on-demand availability without the need for reservations, rapid 5-second startup times with drivers pre-installed, and automatic scale-to-zero efficiency that ensures you only pay for the minutes your model is training. By leveraging built-in GCS volume mounting for high-speed access to model weights, we can now move past infrastructure hurdles and focus on the core task: fine-tuning Gemma 3 27B to achieve high-precision results for Pet Breed Classification on the Oxford-IIIT Pet Dataset.

If you’d like to dive straight into the code, you can clone the repository here.

Prerequisites

Before you begin the fine-tuning process, ensure you have the following software and environment configurations in place.

Python 3.12+
uv (Python package manager): will be used to manage our local Python environment and speed up our Docker builds. Use curl to download the script and execute it with sh:

curl -LsSf https://astral.sh/uv/install.sh | sh

Google Cloud SDK (gcloud CLI) installed and authenticated.
A Google Cloud Project with billing enabled.
APIs Enabled Ensure the following APIs are active in your project: Cloud Run Admin API, Artifact Registry API, Cloud Build API, Secret Manager API, Compute Engine API (for GPU provisioning)
Hugging Face Token: A valid token with access to the Gemma 3 27B-IT model weights.

Access to gated models: Gemma 3 27B-IT is a gated model, which means you must explicitly accept the terms of use before you can download or fine-tune the weights.

Accept the License: Visit the Gemma 3 27B-IT model page on Hugging Face and click the “Agree and access repository” button.
Generate a Token: Once access is granted, ensure your Hugging Face Token has “read” permissions (or “write” if you plan to push your fine-tuned model back to the Hub) to authenticate your training job.

Step 1 — Setting the stage: Your environment

Step 1.1 — Prepare your Google Cloud environment

Set environment variables.

[!IMPORTANT] Regional Alignment is Critical: To use Cloud Storage volume mounting, your GCS bucket must be in the same region as your Cloud Run job. We recommend using europe-west4 (Netherlands) as it supports the RTX PRO 6000 Blackwell GPU and ensures zero-latency access to your model weights.

export PROJECT_ID=YOUR_PROJECT_ID
export REGION=europe-west4
export HF_TOKEN=YOUR_HF_TOKEN
export SERVICE_ACCOUNT="finetune-gemma-job-sa"
export BUCKET_NAME=$PROJECT_ID-gemma3-finetuning-eu
export AR_REPO=gemma3-finetuning-repo
export SECRET_ID=HF_TOKEN
export IMAGE_NAME=gemma3-finetune
export JOB_NAME=gemma3-finetuning-job

Step 1.2 — Get the code

Whether you’re running locally or on the cloud, you’ll need the code. After you open Cloud Shell or install your local Google Cloud CLI, you need to clone the repository. The finetune_gemma repository contains the finetune_and_evaluate.py script, a Dockerfile, and the requirements.txt file to your machine.

git clone https://github.com/GoogleCloudPlatform/devrel-demos
cd devrel-demos/ai-ml/finetune_gemma/

gcloud auth login

Set your Project:

gcloud config set project $PROJECT_ID

Create the service account and grant storage permissions:

gcloud iam service-accounts create $SERVICE_ACCOUNT \
  --display-name="Service Account for Gemma 3 fine-tuning"

gcloud storage buckets create gs://$BUCKET_NAME --location=$REGION

gcloud storage buckets add-iam-policy-binding gs://$BUCKET_NAME \
  --member=serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \
  --role=roles/storage.objectAdmin

Create an Artifact Registry repository and store your HF Token in Secret Manager:

gcloud artifacts repositories create $AR_REPO \
    --repository-format=docker \
    --location=$REGION \
    --description="Gemma 3 finetuning repository"

# Create the secret (ignore error if it already exists)
gcloud secrets create $SECRET_ID --replication-policy="automatic" || true

printf $HF_TOKEN | gcloud secrets versions add $SECRET_ID --data-file=-

gcloud secrets add-iam-policy-binding $SECRET_ID \
  --member serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \
  --role='roles/secretmanager.secretAccessor'

Step 2 — Staging the Model with cr-infer (Recommended)

To avoid downloading the model every time the job runs, we’ll stage the Gemma 3 27B weights in Google Cloud Storage. We’ll use cr-infer, which allows you to run model transfers directly via uvx without needing a local installation.

Before running the transfer, you must set up your Application Default Credentials. This is required for running scripts locally. In this case it allows the cr-infer tool to use your local identity to write the weights to your GCS bucket.

gcloud auth application-default login

Download Gemma 3 27B to GCS: Now, execute the transfer using uvx. This clones the model into gs://$BUCKET_NAME/google/gemma-3–27b-it/, allowing our Cloud Run job to mount the weights as a local volume and save gigabytes of container startup time

uvx — from git+https://github.com/oded996/cr-infer.git cr-infer model download \- source huggingface \
 - model-id google/gemma-3–27b-it \
 - bucket $BUCKET_NAME \
 - token $HF_TOKEN

Step 3 — Build and push the container image

Our Dockerfile leverages uv for fast dependency installation.

Option A: Use Google Cloud Build (Recommended — No local Docker needed)

This is the easiest way to build your image directly in the cloud and push it to Artifact Registry. (The build typically takes 10–15 minutes as it downloads large ML dependencies like PyTorch).

gcloud builds submit — tag $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$IMAGE_NAME:latest .

[!TIP] You can track the real-time progress of your build in the Cloud Build console.

Option B: Build locally with Docker

If you have Docker Desktop installed locally:

Install uv locally (if you haven’t already):

curl -LsSf https://astral.sh/uv/install.sh | sh

Build the image:

docker build -t $IMAGE_NAME .

Push to AR:

docker tag $IMAGE_NAME $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$IMAGE_NAME
docker push $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$IMAGE_NAME

Step 3.1 — Test locally (Optional)

I like to start with a quick local test run to validate the setup. It serves as a sanity check for your environment and scripts before moving the workload to Cloud Run. For this test, we use parameters optimized for speed and a smaller model, google/gemma-3–4b-it, to ensure the model correctly learns the task format:

python3 finetune_and_evaluate.py \
- model-id google/gemma-3–4b-it \
 - train-size 20 \
 - eval-size 20 \
 - gradient-accumulation-steps 2 \
 - learning-rate 2e-4 \
 - batch-size 1 \
 - num-epochs 3

On my Apple M4 Pro, running this on the CPU took about 20–30 minutes. If you want to see early signs of progress locally, you can increase the sample size — I found that a one-hour run on my Mac with 50 training and testing samples already yielded a 4% improvement in accuracy and a 3% boost in F1-score.

Results from a local run on my Mac with 50 train and 50 test samples

Inside the Fine-Tuning Script: How it Works

The finetune_and_evaluate.py script is designed to be a complete, self-contained pipeline, handling everything from data preparation to hardware-aware optimization and evaluation. Here is a look at the core logic that makes this possible:

1. Memory-Efficient Model Loading

To fit a 27B parameter model into the 96GB VRAM of the Blackwell GPU, the script uses 4-bit quantization via the bitsandbytes library. By setting low_cpu_mem_usage=True, it also ensures the model is loaded efficiently without exhausting the system RAM.

2. Vision-Language LoRA Configuration

Instead of updating all 27 billion parameters, we use LoRA (Low-Rank Adaptation). We target all the primary projection layers in the transformer blocks, allowing the model to adapt its internal representations to the visual nuances of the pet breeds while keeping the total trainable parameter count extremely low. More details on efficient GPU memory usage can be found in this blog.

3. The Custom Data Collator

This is a crucial part for fine-tuning vision-language models (VLMs). Because VLMs process a mix of image and text tokens, the data_collator ensures that the model only learns from the breed label (the model’s response). The turn marker is a structural boundary that signals the exact point where the user stops speaking and the model’s response begins. The script ensures the model learns only from the breed label by searching for the model’s turn marker in the token sequence and masking out the user’s prompt and image tokens, so they don’t contribute to the training loss.

4. Breed Extraction

Generative models often add conversational filler (e.g., “The animal in this image is a Samoyed”). Our evaluation logic includes a robust extraction heuristic that sorts class names by length. This ensures that if the model mentions “English Cocker Spaniel,” it correctly identifies the full breed rather than just matching “Cocker Spaniel”.

5. Automated GCS Archiving

Once the training completes and the final evaluation is calculated, the script doesn’t just stop. It bundles the fine-tuned LoRA adapters with the original model processor and automatically uploads the entire directory to your Google Cloud Storage bucket. This ensures your model is immediately ready for deployment or serving.

Step 4 — Create and execute the Cloud Run job

Now, we harness the power of the NVIDIA RTX PRO 6000 Blackwell GPU. Our container is built with CUDA 12.8 for full Blackwell/PyTorch 2.7 compatibility and uses an ENTRYPOINT configuration, allowing you to pass script arguments directly via the — args flag.

[!TIP] If the job already exists, use gcloud beta run jobs update instead of create.

gcloud beta run jobs create $JOB_NAME \
 - region $REGION \
 - image $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$IMAGE_NAME:latest \
 - set-env-vars BUCKET_NAME=$BUCKET_NAME \
 - set-secrets HF_TOKEN=$SECRET_ID:latest \
 - no-gpu-zonal-redundancy \
 - cpu 20.0 \
 - memory 80Gi \
 - task-timeout 60m \
 - gpu 1 \
 - gpu-type nvidia-rtx-pro-6000 \
 - service-account $SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \
 - add-volume name=model-volume,type=cloud-storage,bucket=$BUCKET_NAME \
 - add-volume-mount volume=model-volume,mount-path=/mnt/gcs \
 - network=default \
 - subnet=default \
 - vpc-egress=private-ranges-only \
 - args=" - model-id","/mnt/gcs/google/gemma-3–27b-it/"," - output-dir","/tmp/gemma3-finetuned"," - gcs-output-path","gs://$BUCKET_NAME/gemma3-finetuned"," - train-size","800"," - eval-size","200"," - learning-rate","5e-5"

Note on Execution Limits: Tasks using GPUs on Cloud Run Jobs currently have a maximum execution time of 60 minutes. To ensure this training job completes within the standard public limit, we have set the — num_epochs to 3 and restricted the — train-size to 800 samples. If your specific fine-tuning workload requires more time, you can sample your training dataset into segments that fit in under 60 minutes (like 800 samples in our case) and process them as a sequence of independent tasks while using checkpointing for the model training.

Understanding the Deployment Flags

To ensure a stable and production-ready environment, we use several specialized flags:

— gpu-type nvidia-rtx-pro-6000: Targets the NVIDIA RTX PRO 6000 Blackwell GPU. With 96GB of GPU memory (VRAM), 1.6 TB/s bandwidth, and support for FP4/FP6 precision, it provides the ample overhead and high-speed throughput needed for multimodal fine-tuning.
— memory 80Gi: We allocate high system RAM (scalable up to 176GB) to handle the low_cpu_mem_usage model loading and our memory-efficient streaming data generator.
— cpu 20.0: Cloud Run Jobs allows scaling up to 44 vCPUs per instance, ensuring that preprocessing and data loading never become a bottleneck for the GPU.
— add-volume & — add-volume-mount: This mounts your GCS bucket as a local directory at /mnt/gcs. Note: This requires the bucket and the job to be in the same region (europe-west4). It allows the script to read the base model weights at data-center speeds without copying them into the container’s writable layer.
— network & — subnet: Configures Direct VPC Egress, allowing the job to communicate securely with other resources in your VPC. To make sure this works you need to enable “Private Google Access”.
— vpc-egress=all-traffic: Ensures all outgoing traffic, including requests to Hugging Face, is routed through your VPC for enhanced security and monitoring.

[!TIP] If you skipped Step 2 and didn’t stage the model in your GCS bucket, you must change the — model-id in the — args to google/gemma-3–27b-it. This tells the script to download the weights directly from Hugging Face at runtime, though this will be significantly slower than using the GCS mount

Execute the job:

gcloud beta run jobs execute $JOB_NAME — region $REGION — async

Step 5 — Check Results and Evaluate Performance

Once your job finishes, you can jump into the Google Cloud Console to inspect the detailed logs. You’ll find your newly fine-tuned model waiting for you in your Cloud Storage bucket at gs://$BUCKET_NAME/gemma3-finetuned.

To rigorously quantify how well Gemma 3 learned to identify these breeds, we used Accuracy and Macro F1 Score as our primary metrics. While accuracy gives us a clear overall percentage, the F1 score ensures the model is accurate across all 37 breeds, not just the most common ones.

In my testing, I saw a clear progression as we scaled our data and compute:

Results with different sample size

79% Accuracy, 77% F1-score (1.1h run): Trained on 1,000 samples and evaluated against 200 test samples, this was a significant jump from the zero-shot baseline of 66%.
93% Accuracy, 91% F1-score (2.3h run): By scaling up to 2,500 training samples (and 1,500 test samples), the model reached nearly state-of-the-art performance.
94% Accuracy & 91.5% F1 (3.3h run): With a larger run on 3,600 training samples (evaluated against 3,500 test samples), the model effectively hit the state-of-the-art benchmark for this dataset.

Performance summary report for 3600 train samples and 3500 test sample — reached state of the art with 94% accuracy!

It is important to note that the standard public limit for GPU jobs is currently 60 minutes. As mentioned in step 4, sampling and checkpointing can help overcome this limitation.

These results prove that fine-tuning is the necessary bridge for generalist models, by leveraging serverless Blackwell GPUs, we’ve transformed a massive reasoner into a high-precision expert ready for production

Next Steps: Serving your fine-tuned model on Cloud Run

Now that you’ve fine-tuned Gemma 3, the next challenge is serving it efficiently for production-grade inference.

The true “deploy and forget” magic happens when you transition your saved weights into a serving environment. By hosting your fine-tuned model on Cloud Run with serverless Blackwell GPUs, you get a highly economical production environment where your GPU instances automatically scale to zero when they aren’t in use. This setup eliminates the operational toil of cluster management and manual maintenance, allowing you to serve massive models with no reservations, you only pay for the exact minutes your model is active.

To get started with inference, explore this codelab: Run inference using a Gemma model on Cloud Run with RTX 6000 Pro GPU.

To learn more about production serving, refer to the official guide on Running Gemma 3 on Cloud Run. The documentation provides a comprehensive roadmap for building a robust inference service, including:

Optimized Deployment: Instructions for serving Gemma models using GPU accelerators and loading model weights via high-speed Cloud Storage volume mounts.
Secure Interaction: Guidance on using IAM authentication to securely call your deployed service with the Google Gen AI SDK.
Performance Configuration: Best practices for setting concurrency to achieve optimal request latency and high GPU utilization

Special thanks to Sara Ford and Oded Shahar from the Cloud Run team for the helpful review and feedback on this article.

Agent Factory Recap: Supercharging Agents on GKE with Agent Sandbox and Pod Snapshots

Shir Meir Lador — Tue, 07 Apr 2026 13:04:00 +0000

In the latest episode of the Agent Factory, Mofi Rahman and I had the pleasure of hosting, Brandon Royal, the PM working on agentic workloads on GKE. We dove deep into the critical questions around the nuances of choosing the right agent runtime, the power of GKE for agents, and the essential security measures needed for intelligent agents to run code.

This post guides you through the key ideas from our conversation. Use it to quickly recap topics or dive deeper into specific segments with links and timestamps.

Why GKE for Agents?

Timestamp: 01:49

We kicked off our discussion by tackling a fundamental question: why choose GKE as your agent runtime when serverless options like Cloud Run or fully managed solutions like Agent Engine exist?

Brandon explained that the decision often boils down to control versus convenience. While serverless options are perfectly adequate for basic agents, the flexibility and governance capabilities of Kubernetes and GKE become indispensable in high-scale scenarios involving hundreds or thousands of agents. GKE truly shines when you need granular control over your agent deployments.

ADK on GKE

Timestamp: 06:58

We've discussed the Agent Development Kit (ADK) in previous episodes, and Mofi highlighted to us how seamlessly it integrates with GKE and even showed a demo with the agent he built. ADK provides the framework for building the agent's logic, traces, and tools, while GKE provides the robust hosting environment. You can containerize your ADK agent, push it to Google Artifact Registry, and deploy it to GKE in minutes, transforming a local prototype into a globally accessible service.

The Sandbox problem

Timestamp: 15:20

As agents become more sophisticated and capable of writing and executing code, a critical security concern emerges: the risk of untrusted, LLM-generated code. Brandon emphasized that while code execution is vital for high-performance agents and deterministic behavior, it also introduces significant risks in multi-tenant systems. This led us to the concept of a "sandbox."

What is a Sandbox?

Timestamp: 19:18

For those less familiar with security engineering, Brandon clarified that a sandbox provides kernel and network isolation. Mofi further elaborated, explaining that agents often need to execute scripts (e.g., Python for data analysis). Without a sandbox, a hallucinating or prompt-injected model could potentially delete databases or steal secrets if allowed to run code directly on the main server. A sandbox creates a safe, isolated environment where such code can run without harming other systems.

Agent Sandbox on GKE Demo

Timestamp: 20:25

So, how do we build this "high fence" on Kubernetes? Brandon introduced the Agent Sandbox on Kubernetes, which leverages technologies like gVisor, an application kernel sandbox. When an agent needs to execute code, GKE dynamically provisions a completely isolated pod. This pod operates with its own kernel, network, and file system, effectively trapping any malicious code within the gVisor bubble.

Mofi walked us through a compelling demo of the Agent Sandbox in action.We observed an ADK agent being given a task requiring code execution. As the agent initiated code execution, GKE dynamically provisioned a new pod, visibly labeled as "sandbox-executor," demonstrating the real-time isolation. Brandon highlighted that this pod is configured with strict network policies, further enhancing security.

The Future: Pod Snapshots

Timestamp: 29:39

While the Agent Sandbox offers incredible security, the latency of spinning up a new pod for every task is a concern. Mofi demoed the game-changing solution: Pod Snapshots. This technology allows us to save their state of running sandboxes and then near-instantly restore them when an agent needs them. Brandon noted that this reduces startup times from minutes to seconds, revolutionizing real-time agentic workflows on GKE.

Conclusion

It's incredible to see how GKE isn't just hosting agents; it's actively protecting them and making them faster.

Your turn to build

Ready to put these concepts into practice? Dive into the full episode to see the demos in action and explore how GKE can supercharge your agentic workloads.

Learn how to deploy an ADK agent to Google Kubernetes Engine and how to get your run agent to run code safely using the GKE agent Sandbox.

Connect with us

Shir Meir Lador → LinkedIn, X
Mofi Rahman → LinkedIn
Brandon Royal → LinkedIn

Agent Factory Recap: Reinforcement Learning and Fine-Tuning on TPUs

Shir Meir Lador — Tue, 31 Mar 2026 18:56:42 +0000

In our agent factory holiday special, Don McCasland and I were joined by Kyle Meggs, Senior Product Manager on the TPU Training Team at Google, to dive deep into the world of model fine tuning. We focused specifically on reinforcement learning (RL), and how Google's own infrastructure of TPUs are designed to power these massive workloads at scale.

This post guides you through the key ideas from our conversation. Use it to quickly recap topics or dive deeper into specific segments with links and timestamps.

When to Consider Fine-Tuning

Timestamp: 3:13

We started with a fundamental question: with foundational models like Gemini becoming so powerful out of the box, and customization through the prompt can often be good enough, when should you consider fine-tuning?

Fine tuning your own model is relevant when you need high specialization for unique datasets where a generalist model might not excel (such as in the medical domain), or when you have strict privacy restrictions that require hosting your own models trained on your data.

The Model Lifecycle: Pre-training and Post-training (SFT and RL)

Timestamp: 3:52

Kyle used a great analogy inspired by Andrej Karpathy to break down the stages of training. He described pre-training as "knowledge acquisition," similar to reading a chemistry textbook to learn how things work. Post-training is further split into Supervised Fine-Tuning (SFT), which is analogous to reading already-solved practice problems within the textbook chapter, and Reinforcement Learning (RL), which is like solving new practice problems without help and then checking your answers in the back of the book to measure yourself against an optimal approach and correct answers.

Why Reinforcement Learning (RL) is Essential

Timestamp: 5:50

We explored why RL is currently so important for building modern LLMs. Kyle explained that unlike SFT, which is about imitation, RL is about grading actions to drive "alignment." It’s crucial for teaching a model safety (penalizing what not to do), enabling the model to use tools like search and interact with the physical world through trial and error, and for performing verifiable tasks like math or coding by rewarding the entire chain of thought that leads to a correct answer.

The Agent Industry Pulse: Why 2025 is the year of RL

Timestamp: 8:33

In this segment, we looked at the rapidly evolving landscape of RL. Kyle noted that it is fair to call 2025 the "year of RL," highlighting the massive increase in investment and launches across the industry:

January: DeepSeek-R1 launched, making a huge splash with open-source GRPO.
Summer: xAI launched Grok 4, reportedly running a 200k GPU cluster for RL at "pre-training scale."
October: A slew of new tooling launches across Google, Meta, and TML.
November: Gemini 3 launched as a premier thinking model.
Recent: Google launched MaxText 2.0 for fine-tuning on TPUs.

The Hurdles of Implementing RL

Timestamp: 10:46

Following the industry trends, we discussed why RL is so difficult to implement. Kyle explained that RL combines the complexities of both training and inference into a single process. He outlined three primary challenges: managing infrastructure at the right balance and scale to avoid bottlenecks; choosing the right code, models, algorithms (like GRPO vs. DPO), and data; and finally, the difficulty of integrating disparate components for training, inference, orchestration, and weight synchronization.

To provide a solution across these dimensions of complexity, Google offers MaxText, a vertically integrated solution to help you perform RL in a highly scalable and performant fashion. MaxText provides highly optimized models, the latest post-training algorithms, high performance inference via LLM, and powerful scalability/flexibility via Pathways.

In contrast to DIY approaches where users assemble their own stack of disparate components from many different providers, Google’s approach offers a single integrated stack of co-designed components, from silicon to software to solutions.

The Factory Floor

The Factory Floor is our segment for getting hands-on. Here, we moved from high-level concepts to practical code with a live demo.

Why TPUs Shine for RL

Timestamp: 12:52

Before diving into the demo, Kyle explained why TPUs are uniquely suited for complex AI workloads like RL. Unlike other hardware, TPUs were designed system-first. A TPU Pod can connect up to 9,216 chips over low-latency interconnects, allowing for massive scale without relying on standard data center networks. This is a huge advantage for overcoming RL bottlenecks like weight synchronization. Furthermore, because they are purpose-built for AI, they offer superior price-performance and thermal efficiency.

Demo: Reinforcement Learning (GRPO) with TPU

Timestamp: 15:53

Don led a hands-on demonstration showing what RL looks like in action using Google's infrastructure. The demo showcased:

Using MaxText 2.0 as an integrated solution for the workload.
Leveraging models from MaxText and algorithms from Tunix.
Handling inference using vLLM.
Utilizing Pathways for orchestration and scaling to run GRPO (Group Relative Policy Optimization).

Conclusion

This holiday special was a great deep dive into the cutting edge of model fine tuning. While foundational models are getting better every day, the future of highly specialized, capable agents relies on mastering post-training techniques like RL, and having the right vertically integrated infrastructure, like TPUs, to run them efficiently.

Your turn to build

We hope this episode gave you valuable tools and perspectives to think about fine-tuning your own specialized agents. Be sure to check out the resources below to explore MaxText 2.0 and start experimenting with TPUs for your workloads. We'll see you next year for a revamped season of The Agent Factory!

Resources

Post-Training Docs https://maxtext.readthedocs.io/en/latest/tutorials/post_training_index.html

Google Cloud TPU (Ironwood) Documentation: https://docs.cloud.google.com/tpu/docs/tpu7x
Google Cloud open source code:
- MaxText - https://github.com/AI-Hypercomputer/maxtext
- GPU recipes - https://github.com/AI-Hypercomputer/gpu-recipes
- TPU recipes - https://github.com/AI-Hypercomputer/tpu-recipes
Andrej Karpathy - Chemistry Analogy: Deep Dive into LLMs like ChatGPT
Paper: "Small Language Models are the Future of Agentic AI" (Nvidia): https://arxiv.org/abs/2506.02153
Fine tuning blog: https://cloud.google.com/blog/topics/developers-practitioners/a-step-by-step-guide-to-fine-tuning-medgemma-for-breast-tumor-classification?e=48754805

Connect with us

Shir Meir Lador → https://www.linkedin.com/in/shirmeirlador/, X
Don McCasland → https://www.linkedin.com/in/donald-mccasland/
Kyle Meggs → https://www.linkedin.com/in/kyle-meggs/

My First Experience Creating Antigravity Skills

Shir Meir Lador — Fri, 20 Mar 2026 15:23:02 +0000

Experimenting with Agent skills for the first time, feeling empowered!

Last week, I was at an event where we taught developers how to build MCP servers, agents, and deploy open models to Google Cloud Run. After the session, one of the developers shared something that really stuck with me: he was already using our content to create specialized Skills to share with his entire team.

I got inspired and decided it was time to dive into Agent Skills. During my last project, the dev-signal agent, I had a lot of learnings about how to bring agents and AI applications to production in a robust and scalable manner. I thought, this is a great opportunity to give my favorite coding agent, Google’s Antigravity (Google’s “agent-first” IDE), those skills so that going forward, it will just do it for me!

In this post, I’ll walk through how I built the 13 production skills in this repository and the patterns behind them.

What are Agent Skills?

As Romin Irani explains in “Getting Started with Google Antigravity Skills”, skills represent a shift from monolithic context loading to Progressive Disclosure.

Agents get “overwhelmed” when providing them too many tools all at once (a phenomenon known as “Tool Bloat”), to solve for that, Skills allow the agent to “load” specialist knowledge only when needed. When you ask an agent to “evaluate a shadow revision,” it will figure out it will need to leverage the Shadow Deployer skill as context for this operation.

Workspace vs. Global Scope

In Antigravity, you can manage these skills in two distinct ways depending on how you want to use them:

Workspace Scope: Located in .agent/skills/ within your project root. These are specific to your project and can be committed to GitHub so your entire team can benefit from the same production patterns.
Global Scope: Located in ~/.gemini/antigravity/skills/. These are your personal utilities that stay with you across every project you work on.

How I built the skills

Following the principles in Daniela Petruzalek’s “Building Agent Skills with skill-creator”, I took a “methodology-first” approach. I used the existing dev-signal blog series I’ve been working on and the codebase itself as core context, asking Antigravity to identify and codify the unique skills needed to build a production agent on Google Cloud.

For some of the more specialized areas, I provided additional context with patterns I’d like to follow, such as the agent evaluation codelab and blog and the agent security codelab, both written by my awesome team.

These 13 skills provide Antigravity (or any developer using them) the crucial toolkit of a Google Cloud Production Engineer. I’m currently finalizing a detailed, step-by-step walkthrough of the dev-signal agent which will be published on the Google Cloud Blog very soon! (follow me for future updates)

In the meantime, you don’t have to wait — the full repository and the skills are available for you to explore and leverage in your own projects today.

Here is the full inventory of the skills:

🏗️ Production Agent

adk-memory-bank-initializer: Long-term state logic with Vertex AI Memory Bank.
agent-containerizer: Mixed-runtime Dockerfiles (Python + Node.js).
cloud-run-agent-architect: Least-privilege Terraform for Cloud Run.
gcp-production-secret-handler: In-memory secret fetching pattern (Secret Manager).
mcp-connector-generator: Standardized MCP connection logic.

📊 Evaluation

gcp-agent-eval-engine-runner: Parallel inference and reasoning trace capture.
gcp-agent-eval-metric-configurator: Setup for Grounding and Tool Use rubrics.
gcp-agent-golden-dataset-builder: Tools for building datasets with reference trajectories.
gcp-agent-shadow-deployer: “Dark Canary” deployment scripts with revision tagging.
gcp-agent-tool-trajectory-evaluator: Custom Python metrics for Precision and Recall.

🛡️ Security

gcp-agent-model-armor-shield: Intelligent firewall (Prompt Injection, RAI, Malicious URL filters).
gcp-agent-safety-gatekeeper: Python integration pattern (safety_util.py) for sanitizing user inputs.
gcp-agent-sdp-template-factory: Terraform for Sensitive Data Protection (PII/Secret redaction).

By codifying these patterns to production skills, Antigravity can now leverage these automatically in my day to day development. I hope you find these as helpful as I do!

Pro tip - self improving skills!

Because these skills were AI-generated, they might not work perfectly for your specific environment on the first try. But that’s actually the best part of working with an agentic IDE. If a skill doesn’t work well for you, don’t just manually fix the code, let the coding agent figure it out. Once it finds the solution, you can ask it to update the corresponding SKILL.md with the learned workflow. This will capture the corrected workflow for the future, ensuring the agent doesn’t repeat the mistake while saving you tokens and time on the next run. Think of these as living documents that actively improve as you build.

Ready to get started? Clone the repository and add these skills to your Workspace or Global Scope to start building your own production-ready agents. Learn more about Agent skills.

Follow me on LinkedIn and X for updates on my next blogs and videos.

How I Turned an Ugly Spreadsheet into an AI Assisted App with Antigravity

Shir Meir Lador — Wed, 18 Feb 2026 17:39:12 +0000

I have a confession to make.

Up until now, I wasn’t that much into “vibe coding.” I used AI all the time for Python coding, but I never really built a whole app from scratch in a language I knew nothing about.

That changed today. I encountered a really annoying problem: I had to review a massive amount of talk submissions for a conference. We’re talking about a massive spreadsheet. Staring at those tiny cells was literally making my eyes hurt.

My initial thought was, “Hey, let’s create a really sharp UI for the submission review.” But then I thought, why stop there? Why not let AI provide me valuable inputs from social media to help me with the review itself?

So, I decided to build TalkScout. And since I wanted to test drive Google Antigravity (Google’s new AI-powered coding agent), I figured this was the perfect opportunity.

Talkscout Dashboard (synthetic data)

Here is how I went from a painful CSV to a fully deployed Cloud Run app-without writing a single line of React code myself.

Step 1: The “Meta-Prompt” (Asking Gemini to Talk to Antigravity)

I didn’t start by coding; I started by chatting. I used meta-prompting to get started.

So, what is meta-prompting, you may ask? It’s actually when you go to Gemini 3 and ask it to write the prompt for the coding agent.

I explained my problem to Gemini 3 in simple words. Gemini 3 acted as my architect. It turned my “brain dump” requirements into a technical spec, defining the component structure and data model. I didn’t have to guess the right words, I just pasted that polished spec into Antigravity.

Step 2: Ditching the Spreadsheet for a Dashboard

With that prompt, Antigravity built the app of my dreams. It allowed me to:

Upload the CSV with all the conference talks.
Get a dashboard showing the status of each talk.
See a beautiful, high-contrast UI to review abstracts and demo plans without squinting at cells.

TalkScout submission review page with high contrast UI

The “Vibe” Fix: It wasn’t all smooth sailing — I actually hit a nasty React hydration error. This can take hours to debug, especially if you’re not a frontend developer… But I simply provided the error message to Antigravity and the coding agent pinpointed the mismatch in the DOM and fixed it in minutes.

Step 3: Integrating Grounded Intelligence

I didn’t just want a UI; I wanted to overcome my own bias. How do I know if a niche topic is actually hot?

I added a button to get an AI Assessment. But I didn’t want hallucinations. I used Google Search Grounding so the AI could search through Reddit, X (Twitter), and LinkedIn for real-world developer signals. That provided me inputs based on the current developer audience mindshare.

TalkScout submission review page with AI social media analysis

Step 4: Calibrating the “Strict” Reviewer

Initially, the AI was way too nice. It was giving high scores to anything with trendy keywords.

I used what’s called few-shot prompting to calibrate it. I gave examples of my scores vs. its scores and introduced what I call the “Marketing Fluff Penalty”.

If a submission reads like a documentation/marketing page? Points docked.
If the submission was way too short? We capped the score at a hard 2.
If it includes war stories and actual learnings — increase rating. After a few examples, it became more calibrated to my taste.

Step 5: The Pivot to Batch Mode

I realized it was taking me too long to ask the AI to evaluate each talk individually while I reviewed it.

So, I asked Antigravity to refactor the backend for Batch Mode. Now, TalkScout processes the entire submission pool in the background. By the time I grab a coffee, the “AI Draft” column is full of insights, allowing me to focus only on the final decisions.

Step 6: Sharing the Goodness (Deploy to Cloud Run)

TalkScout was working great for me, but I thought, “It would be great to share this with the other reviewers.”

This is where Antigravity really showed off. I simply asked it to deploy the app. It automatically recognized my Google Cloud Project ID, handled the containerization, generated the exact deployment commands, and deployed it to Cloud Run.

One simple ask, and minutes later, I had a URL to share with the team.

It Was Pretty Fun!

It was pretty fun to actually solve a real problem I had using Antigravity and vibe coding. I built a tool that handles ingestion, provides a distraction-free rating interface, and provides valuable inputs for my reviews.

I would love to hear from you all - have you recently solved a problem using vibe coding?

If you haven’t already - try playing around with Antigravity and easily deploy your apps to Cloud Run.

Decoding high-bandwidth memory: A practical guide to GPU memory for fine-tuning AI models

Shir Meir Lador — Thu, 15 Jan 2026 15:27:00 +0000

We've all been there. You've meticulously prepared your dataset and written your training script. You hit run, and your excitement builds, only to be crushed by the infamous error: CUDA out of memory.

This is one of the most common roadblocks in AI development. Your GPU's High Bandwidth Memory (HBM), is the high-speed memory that holds everything that's needed for computation, and running out of it is a hard stop. But how do you know how much you need?

To build a clear foundation, we'll start by breaking down the HBM consumers on a single GPU and we'll present key strategies to reduce HBM consumption on a single GPU. Later, we'll explore advanced multi-GPU strategies like data and model parallelism that can help relieve memory pressure and scale your training in the cloud.

Understanding HBM: What's using all the memory?

When you fine-tune a model, your HBM is primarily consumed by three things:

Model Weights: This is the most straightforward. It's the storage space required for the model's parameters—the "brain" that it uses to make predictions. A 7-billion parameter model loaded in 16-bit precision will take up roughly 14 GB before you even process a single piece of data.
Optimizer States and Gradients: This is the overhead that's required for learning. To update the model's weights, the training process needs to calculate gradients (the direction of learning) and the optimizer (like the popular AdamW) needs to store its own data to guide the training. In full fine-tuning, this can be the largest consumer of HBM.
Activations and Batch Data: This is the most dynamic part. When your data (images, text, etc.) flows through the model's layers, the intermediate calculations, or activations, are stored in HBM. The memory needed here is directly proportional to your batch size. A larger batch size means more activations are stored simultaneously, which leads to faster training but much higher memory usage.

Note: These calculations are theoretical minimums. Real-world frameworks add up to 30% overhead due to temporary buffers, kernel launches, and memory fragmentation.

Although it's impossible to get a perfect number without experimentation, you can estimate your HBM needs with this general formula:

Total HBM ≈ (Model Size) + (Optimizer States) + (Gradients) + (Activations)

Further reading: See this excellent JAX e-book that covers these topics in great detail and even has some "try it out yourself" test questions.

Example: Why full fine-tuning is so demanding

To see why running out of memory is such a common problem, let's walk through a real-world example that I recently worked on: fine-tuning the medgemma-4b-it model, which has 4 billion parameters. Our script loads it in bfloat16 precision (2 bytes per parameter).

First, let's calculate the static HBM footprint. This is the memory that's required just to load the model and prepare it for training, before you've even processed a single piece of data.

1. Model Size: The memory that's needed to simply hold the model on the GPU.

4 billion parameters × 2 bytes/parameter = 8 GB

2. Gradients and Optimizer States: The overhead for training every parameter with the AdamW optimizer.

Gradients: 4 billion parameters × 2 bytes/parameter = 8 GB Optimizer States (AdamW): 2 × 4 billion parameters × 2 bytes/parameter = 16 GB

Note: While AdamW is a popular optimizer, other optimizers, such as Adafactor and Lion, have different memory footprints.

Adding these together gives us our baseline HBM cost for a full fine-tuning attempt:

8 GB (Model) + 8 GB (Gradients) + 16 GB (Optimizer) = 32 GB

This 32 GB is the baseline just to start the training process. On top of this, the GPU needs additional memory for activations, which is a dynamic cost that grows with your batch size and input data size. This is why full fine-tuning of large models is so demanding and often reserved for the most powerful hardware.

Key strategies to reduce HBM consumption

The HBM requirement for a full fine-tune can seem impossibly high. But several powerful techniques can reduce memory consumption, making it feasible to train large models on consumer-grade or entry-level professional GPUs.

Parameter-Efficient Fine-Tuning (PEFT) with LoRA

Instead of training all the billions of parameters in a model, Parameter-Efficient Fine-Tuning (PEFT) methods focus on training only a small subset of parameters. The most popular of these is LoRA (Low-Rank Adaptation).

LoRA works by freezing the original model's weights and injecting a tiny number of new, trainable adapter layers into the model architecture. This means the memory-hungry gradients and optimizer states are only needed for these few million new parameters, not the full 4 billion.

The math behind LoRA's memory savings

LoRA doesn't remove the base model from your GPU. The full 8 GB of the original model's weights are still loaded and taking up HBM. They're just frozen, which means that the GPU isn't training them. All of the memory savings come from the fact that you no longer need to store the huge gradients and optimizer states for that massive, frozen part of the model.

Let's recalculate the static HBM footprint with LoRA, assuming it adds 20 million trainable parameters:

1. Model Size (unchanged): The base model is still loaded.

4 billion parameters × 2 bytes/parameter = 8 GB

2. LoRA Gradients & Optimizer States: We now only need overhead for the tiny set of new parameters.

Gradients: 20 million parameters × 2 bytes/parameter = 40 MB Optimizer States: 2 × 20 million parameters × 2 bytes/parameter = 80 MB

The new static HBM footprint is now:

8 GB (Model) + 40 MB (Gradients) + 80 MB (Optimizer) ≈ 8.12 GB

The training overhead has shrunk from 24 GB to just 120 MB. Your new baseline memory requirement is now just over 8 GB. This lower baseline memory requirement leaves much more room for the dynamic memory that's needed for activations, which lets you use a reasonable batch size on a common 16 GB or 24 GB GPU without running out of memory.

Model quantization

Besides training fewer parameters, we can also shrink the ones that we have by using quantization, which involves reducing the numerical precision of the model's weights. The standard precision for modern training is bfloat16 because it offers the dynamic range of float32 with half the memory footprint. But we can reduce HBM usage further by converting weights to lower-precision integer formats like int8 or int4.

Using lower-precision integer formats has a significant impact on HBM when compared to the standard bfloat16 baseline:

bfloat16 (standard): The baseline size (e.g., a 7B model requires ~14 GB).
8-bit precision: Halves the model size (e.g., 14 GB becomes ~7 GB).
4-bit precision: Reduces the model size by a factor of 4 (e.g., 14 GB becomes ~3.5 GB).

The reduction in size lets you fit much larger models into memory with minimal degradation in performance.

A word of warning from experience:
When I started experimenting in this area, my first attempt to load the model using the common float16 data type failed spectacularly. The model's outputs were NaN, and a quick check revealed that every internal value had collapsed into NaN (Not a Number) .

The culprit was a classic numerical overflow. The float16 data type has a tiny numerical range and it can't represent any number larger than 65,504. During training, intermediate values can easily exceed this limit, causing an overflow that creates a NaN. The fix was a simple one-line change to bfloat16, which has a massive numerical range that prevents these overflows and keeps training stable. For fine-tuning large models, always prefer bfloat16 for stability.

Combining LoRA and Quantization: These techniques work best together. Quantized LoRA (QLoRA) is a method that stores the massive base model in a highly efficient 4-bit format (specifically NF4 or NormalFloat 4), while adding small, trainable LoRA adapters in bfloat16. During the training process, the 4-bit weights are dequantized to bfloat16 for computation. Dequantizing in process lets you fine-tune very large models on a single GPU with the memory savings of 4-bit storage and the mathematical stability of 16-bit training.

FlashAttention: An algorithmic speed boost

Finally, FlashAttention is a foundational algorithmic optimization that significantly reduces HBM usage and speeds up training on both single and multi-GPU setups. The attention mechanism in transformers is a primary memory bottleneck because it requires storing a large, intermediate attention matrix. FlashAttention cleverly reorders the computation to avoid storing this full matrix in memory, leading to substantial memory savings and faster execution.

Best of all, enabling FlashAttention is often as simple as a one-line change. In the MedGemma fine-tuning script, this was done by setting the value attn_implementation="sdpa", which can automatically use more efficient backends like FlashAttention if the hardware supports it.

Scaling beyond a single GPU: Advanced strategies

Techniques like LoRA and quantization are useful for lowering HBM needs on a single GPU. But to train truly massive models or to really speed up the process, you'll eventually need to scale out to multiple GPUs. Here are some of the key strategies that can be used to distribute the load and overcome memory limitations.

Data parallelism

Data parallelism is the most common and intuitive approach to scaling. In a Distributed Data Parallel (DDP) setup, the entire model is replicated on each GPU. The key is that the global batch of training data is split, with each GPU processing its own mini-batch concurrently. After each forward and backward pass, the gradients from each GPU are averaged together to ensure that all of the model replicas learn from the entire dataset and they stay in sync. This method is excellent for speeding up training but it doesn't reduce the HBM that's required to hold the model itself, because every GPU needs a full copy.

Model parallelism

When a model is too large to fit into the memory of a single GPU, you must use model parallelism. Instead of replicating the model, this strategy splits the model across multiple GPUs. There are two primary ways to do this:

Tensor parallelism: This method splits a single large operation (like a massive weight matrix in a transformer layer) across several GPUs. Each GPU computes its part of the operation, and the results are combined.
Pipeline parallelism: This technique places different layers of the model onto different GPUs in a sequence. The data flows through the first set of layers on GPU 1, then the output is passed to GPU 2 for the next set of layers, and so on, like an assembly line.

These strategies are more complex to implement than data parallelism, but they're essential for models that are simply too big for one device.

Fully Sharded Data Parallelism (FSDP)

FSDP is a powerful and efficient hybrid strategy that combines the ideas of data parallelism and model parallelism. Unlike standard data parallelism where each GPU holds a full copy of the model, optimizer states, and gradients, FSDP shards (or splits) all of these components across the GPUs. Each GPU only materializes the full parameters for the specific layer that it's computing at that moment, dramatically reducing the peak HBM usage per device. FSDP makes it possible to train enormous models on a cluster of smaller GPUs.

By combining these hardware and software strategies, you can scale your fine-tuning jobs from a single GPU to a powerful, distributed cluster capable of handling even the most demanding AI models.

HBM sizing guide

HBM	Use case and explanation
16 GB	Sufficient for basic inference or fine-tuning with techniques like LoRA using a very small batch size (e.g., 1-2). Expect slower training times at this level.
24 GB	The recommended starting point for a good experience with 4-7 B parameter models. This capacity allows for a more effective batch size (e.g., 8-16) when using LoRA, providing a great balance of training speed and cost.
40+ GB	Necessary for maximizing training speed with large batch sizes or for working with larger models (in the 20+ B parameter range) now or in the future.

Encountering the CUDA out of memory error provides an important lesson in the trade-offs between model size, training techniques, and batch size. By understanding what consumes your HBM, you can make smarter decisions and keep your projects running smoothly.

I hope that this guide has helped clarify the CUDA out of memory error and that it's given you the tools to keep your projects running smoothly. When you're ready to take the next step, Google Cloud has the tools to accelerate your AI development.

Explore GPU configurations for your Cloud Run services and best practices for running Cloud Run jobs with GPU.
For maximum control: Spin up a Compute Engine instance with the latest NVIDIA H100 or A100 Tensor Core GPUs and take full control of your environment.
Looking to optimize your model hosting infrastructure? Take a look at The Ultimate Guide to xPU Inference Configuration.
For a deeper dive into scaling your model, check out How to Scale Your Model.
New to Google Cloud? Get started with the $300 free credit to find the perfect solution for your next project.

Special thanks to Jason Monden and Sayce Falk from the AI compute team for their helpful review and feedback on this post.

Agent Factory Recap: Can you do my shopping?

Shir Meir Lador — Fri, 19 Dec 2025 19:44:58 +0000

In episode #8 of The Agent Factory, Ivan Nardini and I are joined by Prateek Dudeja, product manager from the Agent Payment Protocol Team, to dive into one of the biggest hurdles for AI agents in eccomerce: trust, especially when it comes to money.

This post guides you through the key ideas from our conversation. Use it to quickly recap topics or dive deeper into specific segments with links and timestamps.

Introducing Agent Payment Protocol

Timestamp: [01:43]

What if an agent could buy concert tickets for you at a specific time that the tickets go on sale. You don't want to miss out! Maybe you want two tickets, and you don't want to spend more than $200. You definitely want to sit in a section with a great view of the stage. To have an agent act as your ticket buyer, you would have to trust that agent with all facets of your request and your credit card. How can you be sure that the agent won't buy 200 tickets or that it won't charge you for a lifetime supply of rubber duckies?

The potential for a messy outcome with this concert ticket request provides insight into a "Crisis of Trust" that can hold back agentic commerce. The good news is there's a way to move forward and build trust.

To solve the "Crisis of Trust," Google introduced the Agent Payment Protocol (AP2), a new open standard. It's not a new payment system; it’s a "trust layer" that sits on top of existing infrastructure. AP2 is designed to create a common, secure language for agents to conduct commerce, using role-based architecture and verifiable credentials.

Agent Payments and the Current Payment System

Timestamp: [02:29]

The current payment system was built for humans using trusted interfaces like browsers, not for autonomous agents, resulting in three main challenges for agents: authorization, agent error, and accountability.

The Agent Payment Protocol addresses these challenges by helping agents communicate securely with merchants and payment partners. The Agent Payment Protocol is available today as an extension for the A2A (Agent2Agent) protocol and relies on agents using the Model Context Protocol (MCP).

Deep Dive into the Agent Payment Protocol

Learn more about how this protocol works, including concepts and flow.

A Role-Based Ecosystem

Timestamp: [04:33]

The protocol is built on a "separation of concerns." Your agent doesn't have to do everything. There are specialized roles:

Shopping Agent: The AI agent you build, great at finding products.
Merchant Endpoint: The seller's API.
Credential Provider: A secure digital wallet (like PayPal, Google Pay, etc.) that manages payment details.
Merchant Payment Processor: The entity that constructs the final authorization message for the payment networks.

_Critical: Your shopping agent never touches the raw credit card number. It doesn't need to be PCI compliant because it delegates the payment to the specialized, secure providers._

Verifiable Credentials (VCs)

Timestamp: [06:15]

The "handshakes" between these roles in the Agent Payment Protocol ecosystem are secured by Verifiable Credentials (VCs). Think of credentials as protocolized, cryptographically signed digital receipts that prove what was agreed upon.

There are three types of verifiable credentials:

Cart Mandate: For "human-present" scenarios. The user reviews a final cart and cryptographically signs it as proof of approval.
Intent Mandate: For "human-not-present" scenarios (like the concert ticket example). The user signs an intent (e.g., "buy tickets under $200"), giving the agent authority to act within those guardrails.
Payment Mandate: Provides clear visibility to payment networks and banks that an AI agent was involved in the transaction.

A Contractual Conversational Model

Timestamp: [08:03]

The Agent Payment Protocol process creates a "Contractual Conversational Model," moving beyond simple API calls to a flow built on verifiable proof.

To understand this flow, we'll walk through a human-present scenario:

Delegation: You tell your agent, "Buy two concert tickets."
Discovery & Negotiation: The agent contacts the merchant's endpoint to prepare the cart.
Finalize Cart: The agent reaches out to your Credential Provider (e.g., your digital wallet). You select the payment method. The agent only gets a reference (like the last 4 digits), never the full credential.
Authorization with Mandates: The agent shows you the final, finalized cart.
You cryptographically sign the Cart Mandate. This is the non-repudiable proof, the "contract."
Purchase: The agent sends this signed mandate to the merchant. The merchant can now trust the purchase mandate is from you. The merchant's payment processor uses the mandate to securely get the payment token from the credential provider and complete the transaction.

This flow all hinges on trust. In the short term, this trust is built using manual allow lists of approved agents and merchants. In the long term, the plan is to use open web standards like HTTPS and DNS ownership to verify identities.

Q&A with Prateek Dudeja

Timestamp: [13:07]

With the concepts explained, the discussion moved to a Q&A with Prateek.

Why a New Protocol for Payments?

_Timestamp: [13:30]
_
Prateek gave a great analogy: HTTPS is a baseline protocol for browsing. Signing in requires stronger authentication. Making a payment requires an even higher level of trust. AP2 provides that "payments-grade security" on top of baseline protocols like A2A and MCP, ensuring the transaction is high-trust and truly from a human.

How Will Agents Find Trusted Partners?

Timestamp: [14:42]

In the short term, agents will use "decentralized registries of trust" (or allow lists) to find merchants they can interact with. Prateek noted that all the roles (merchant, credential provider, etc.) already exist in the payments industry today. The only new role is the Shopping Agent itself.

Accountability: What Happens When Things Go Wrong?

Timestamp: [16:03]

This is the big question. What if your agent shows you blue shoes, you wanted teal, but you click "approve" anyway?

Prateek explained that the signed Cart Mandate solves this. Because you biometrically signed a tamper-proof credential showing the blue shoes, the responsibility is on you. The merchant has cryptographic evidence that you saw and approved the exact product. This protects merchants from fraudulent chargebacks and users from unauthorized agent actions.

Demo: Reference Implementation

Timestamp: [18:04]

Prateek walked through a demo showing the human-present flow. It showed the user prompting the agent, the agent discovering products, and then the Credential Provider (PayPal) getting involved. The user selected their shipping and payment info from PayPal, and the agent only saw a reference. The user then signed the Cart Mandate, and the purchase was completed.

Compatibility and Getting Started

Timestamp: [19:43]

A key question was - Is this compatible with frameworks like LangGraph or CrewAI? Yes. Prateek confirmed the protocol is compatible with any framework. As long as your agent can communicate over A2A or MCP, you can use AP2.

To get started, Prateek directed developers to the GitHub repo. The first step is to see which role you want to play (merchant, credentials provider, etc.) and explore the sample code for that role.

The Future: Dynamic Negotiation

Timestamp: [21:13]

Looking ahead, Prateek shared an exciting vision for "dynamic negotiation." Imagine telling your agent: "I want that red dress that's out of stock. I need it by tomorrow... and I'm willing to pay 30% more".

A merchant's agent could see this "intent" and, if the dress becomes available, automatically complete the sale. What was a lost sale for the merchant becomes a completed order at a markup, and the user gets the exact item they desperately wanted.

Your turn to build

This conversation made it clear that building a secure payment infrastructure is a foundational step toward creating agents that can perform truly useful tasks in the real world. We're moving from a simple, programmatic web to a conversational, contractual one, and this protocol provides the framework for it.

We encourage you to check out the Agent Payment Protocol GitHub repo, think about which role you could play in this new ecosystem, and start building today!

Connect with us

Shir Meir Lador → LinkedIn, X
Ivan Nardini → LinkedIn, X
Prateek Dudeja → Linkedin

The Agent Factory podcast: 5 Episodes to Kickstart Your Journey to Production AI

Shir Meir Lador — Tue, 25 Nov 2025 21:22:16 +0000

We are so proud to announce that a project we're incredibly passionate about has grown into a full-blown resource for developers: The Agent Factory video podcast.

We started this show with a simple mission: to have the conversations developers need to be having about AI agents development. We wanted to move past the hype and focus on what really matters—building production-ready AI agents.

Fast forward to today, and we have 14 episodes published covering everything from architecture patterns to end to end vibe coding of advanced AI applications. To celebrate, we’re sharing our first 5 foundational episodes with the Dev.to community. If you are just starting to build agents or looking to harden your existing systems, this is the perfect place to start.

What to Expect:
We pack every episode with three core segments designed for developers:

🎙️ Agent Industry Pulse: We filter the noise and bring you the latest news you actually need to know.

🛠️ The Factory Floor: A technical deep-dive where we get our hands dirty with code, architectures, and patterns.

❓ Developer Q&A: We answer real questions from the community to help us learn together.

📺 The Starter Pack: Our First 5 Episodes
Here is the chronological journey to get you up to speed, starting from the very beginning.

1. Agents, their frameworks and when to use them (ft. Julia Wiesinger)
We kicked things off by tackling the big questions: What exactly is an agent? How do you choose between frameworks like LangChain, CrewAI, or the Agent Development Kit (ADK)? We were joined by Julia Wiesinger from the ADK team to guide us through building for production.

2. Multi-Agent Systems: Concepts & Patterns
Single agent or multi-agent? In this episode, we break down the architectural patterns that matter - from Supervisors to Swarms. We discuss exactly when you should transition from a single agent to a team of agents to handle complexity and improve reliability.

3. Building Custom Tools for Agents
Agents are only as good as the tools they can use. We dive into Model Context Protocol (MCP), function calling, and how to build secure, authenticated tools that let your agents interact with the real world safely.

4. Memory in Agents (ft. Kimberly Milam)
How do you stop your agent from acting like a goldfish? We chat with Kimberly Milam about implementing long-term memory, managing state, and the "Memory Bank" concept to create personalized experiences that persist across sessions.

5. Tackling the Hardest Questions (ft. Philipp Schmidt)
We sat down with Philipp Schmidt from Google DeepMind for a masterclass on the agent development workflow. We cover context engineering, evaluation strategies, and pro-tips for using the Gemini CLI to speed up your development cycle.

💬 Join the Conversation
We’re truly excited to continue building this community with you. Whether you're stuck on a specific bug or wondering about a new architecture, we want to hear from you.

What are you struggling with right now? Drop your questions in the comments below with #TheAgentFactory, and we might answer them in our next Q&A segment!

➡️ Listen & Subscribe: Google Cloud Tech on YouTube