DEV Community: Google Developer Experts

Dart 3.13 Primary Constructors + BlocSignal: Boilerplate-Free Reactive Architecture

Randal L. Schwartz — Fri, 14 Aug 2026 02:57:56 +0000

For years, one of the most common critiques of the BLoC pattern has been boilerplate.

Between declaring event classes, state hierarchies, constructor parameters, private fields, super-initializers, and event handler registries, you could easily write 50 lines of code before handling a single real-world user action.

With Dart 3.13, that all changes.

Dart 3.13 brings Primary Constructors, this constructor body blocks, and new/factory constructor shorthands. When combined with BlocSignal—the synchronous, signals-powered evolution of BLoC—the result is an ultra-concise, fully type-safe, and boilerplate-free state management workflow.

Let's explore how Dart 3.13 and BlocSignal fit together like hand in glove.

1. Zero-Boilerplate Events & States

In classic BLoC, defining a family of immutable events or states meant writing repeated constructor signatures and field definitions for every subtype.

🔴 Before Dart 3.13:

sealed class UserEvent {}

class UserFetchRequested extends UserEvent {
  final String userId;
  UserFetchRequested(this.userId);
}

class UserUpdated extends UserEvent {
  final String name;
  final int age;
  UserUpdated({required this.name, required this.age});
}

class UserLoggedOut extends UserEvent {}

🟢 With Dart 3.13 Primary Constructors:

sealed class UserEvent {}

class UserFetchRequested(final String userId) extends UserEvent;
class UserUpdated({required final String name, required final int age}) extends UserEvent;
class UserLoggedOut() extends UserEvent;

A whole sealed hierarchy of events or states can now be declared in just a few clean, expressive lines without losing type safety or exhaustiveness checking in switch expressions.

2. Streamlined Dependency Injection in `CubitSignal`

In CubitSignal, you typically inject repositories, API clients, or analytic trackers. In previous Dart versions, you had to declare each field, accept constructor arguments, and forward initial state to super.

With primary constructors, field declarations and super invocations live right in the class header.

🔴 Before Dart 3.13:

class UserCubit extends CubitSignal<UserState> {
  final UserRepository _repository;
  final AnalyticsService _analytics;

  UserCubit({
    required UserRepository repository,
    required AnalyticsService analytics,
    UserState initial = const UserInitial(),
  })  : _repository = repository,
        _analytics = analytics,
        super(initialState: initial);

  Future<void> loadUser(String id) async {
    emit(const UserLoading());
    try {
      final user = await _repository.fetchUser(id);
      _analytics.track('user_loaded', {'id': id});
      emit(UserSuccess(user));
    } catch (e, st) {
      onError(e, st);
      emit(UserError(e.toString()));
    }
  }
}

🟢 With Dart 3.13:

class UserCubit(
  final UserRepository repository,
  final AnalyticsService analytics, {
  final UserState initial = const UserInitial(),
}) extends CubitSignal<UserState>(initialState: initial) {

  Future<void> loadUser(String id) async {
    emit(const UserLoading());
    try {
      final user = await repository.fetchUser(id);
      analytics.track('user_loaded', {'id': id});
      emit(UserSuccess(user));
    } catch (e, st) {
      onError(e, st);
      emit(UserError(e.toString()));
    }
  }
}

No field re-declarations. No duplicate parameter names. The dependencies are immediately available across all methods.

3. Event Handler Registration via the `this` Block in `BlocSignal`

One of the most powerful features in Dart 3.13 is the this constructor body syntax. When using primary constructors, constructor body logic (such as registering event handlers with on<E>() or asserting preconditions) is placed inside a this { ... } block in the class body.

class SearchBloc(
  final SearchRepository repository, {
  final SearchState initial = const SearchInitial(),
}) extends BlocSignal<SearchEvent, SearchState>(initialState: initial) {

  // Dart 3.13 primary constructor body
  this {
    on<SearchQueryChanged>(
      (event, emit) async {
        if (event.query.trim().isEmpty) return emit(const SearchEmpty());

        emit(const SearchLoading());
        final results = await repository.search(event.query);
        emit(SearchSuccess(results));
      },
      transformer: restartable(), // Zero-stream event concurrency!
    );
  }
}

The header cleanly declares the class contract, and the this block sets up the event pipeline.

4. Immediate Reactive Wiring with `createEffect`

BlocSignal includes createEffect, which automatically tracks signal dependencies and manages teardown on container disposal. With primary constructor parameters in scope, derived cubits can synchronously wire up upstream state containers in the this block:

class CartSummaryCubit(final CartBloc cartBloc)
    extends CubitSignal<CartSummary>(initialState: const CartSummary.zero()) {

  this {
    // Automatically reacts to cartBloc.state signals synchronously:
    createEffect(() {
      final items = cartBloc.state.value.items;
      final total = items.fold<double>(0, (sum, item) => sum + item.price);
      emit(CartSummary(count: items.length, total: total));
    });
  }
}

5. Named Constructor Shorthands (`new`) for Testing & Seeding

Dart 3.13 also introduces constructor shorthands, allowing you to define secondary named constructors using new name() without repeating the class name:

class CounterCubit(var int count) extends CubitSignal<int>(initialState: count) {
  // Named constructor shorthands:
  new zero() : this(0);
  new seeded(int initial) : this(initial);

  void increment() => emit(state + 1);
  void decrement() => emit(state - 1);
}

This makes testing variations, mock seeds, and default configurations concise and readable.

🛠️ Enabling Dart 3.13 in Your Project

To take advantage of these features:

1. Set the SDK Constraint in `pubspec.yaml`:

environment:
  sdk: ^3.13.0

dependencies:
  bloc_signals: ^1.0.0
  bloc_signals_flutter: ^1.0.0

2. Enable Dart 3.13 Linter Rules in `analysis_options.yaml`:

include: package:very_good_analysis/analysis_options.yaml

linter:
  rules:
    - use_primary_constructors
    - use_declaring_parameters
    - unnecessary_type_name_in_constructor
    - unnecessary_primary_constructor_body

🚀 The Architectural Payoff

By combining Dart 3.13 language features with BlocSignal, you get:

0ms Synchronous Updates: State emissions propagate in the current frame without microtask delay.
Minimal Ceremony: Class headers declare fields and super initializers simultaneously.
Signal Graph Efficiency: Automatic == de-duplication and fine-grained UI rebuilding.
Standard BLoC Rigor: Clean event dispatching, state transitions, and OpenTelemetry observability.

💬 Over to You: What's Your Take?

We'd love to hear your thoughts in the comments below:

How do you feel about Dart 3.13's primary constructors? Does declaring fields directly in the class header match how you design your domain and state layers?
Are you planning to adopt primary constructors across your state management classes, or are there specific patterns where you still prefer classic constructors?
Have a boilerplate-heavy state class or Bloc? Drop a snippet in the comments, and let's see how much code Dart 3.13 and BlocSignal can shave off!

Ready to build boilerplate-free reactive apps?

Check out the full documentation, benchmarks, and interactive examples at blocsignal.dev or star the open-source repository on GitHub!

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

xbill — Thu, 13 Aug 2026 18:42:09 +0000

A field report on serving Google's Gemma 4 E2B on AWS EC2 **G5g* — a Graviton2 (aarch64)
host with an NVIDIA T4G (Turing, SM 7.5) GPU. Three obstacles: an arch list nobody
publishes for this combination, a version floor that only the newest vLLM clears, and
64 KiB of shared memory that stops the model dead. Plus the seven things I documented
wrong before I had a box.*


Model	`google/gemma-4-E2B-it` (reference bf16 release)
Hardware	AWS EC2 `g5g.4xlarge` — Graviton2 + 1x NVIDIA T4G, compute capability 7.5, 15,360 MiB
Base image	Deep Learning ARM64 AMI OSS Nvidia Driver GPU PyTorch 2.12 (Ubuntu 24.04)
Software	torch 2.12.0+cu132 · CUDA 13.2 · vLLM v0.27.2rc0 built from source for `sm_75`
Result	43.1 tok/s single-stream greedy, 329,579-token KV cache — after one patch to vLLM

G5g is the only instance AWS has ever shipped that puts an NVIDIA GPU behind a Graviton
host. It launched in 2020, it never got a successor, and Graviton is now on its fifth
generation without one.

That matters more than it sounds. The Arm-plus-CUDA world moved on to NVIDIA's own Arm CPU
— Grace, paired with SM 9.0 and 10.0 parts. Turing stayed well supported, on x86. G5g is
the only hardware that is aarch64 and compute capability 7.5, and almost nobody publishes
a build for that combination.

I put a rig on one anyway. The packaging problem was the quick part. Everything after it
— a compiler that was not there, a version floor I did not expect, and 32 KiB of shared
memory — took far longer, because none of it fails where you are looking.

No published build covers aarch64 and SM 7.5 together

Start with the obvious candidate. vllm/vllm-openai:v0.27.1 publishes both platforms under
one tag, and you can read the arch lists straight out of the image config without pulling a
layer:

docker buildx imagetools inspect vllm/vllm-openai:v0.27.1 --format '{{json .Image}}'

linux/amd64   7.5 8.0 8.6 8.9 9.0 10.0 12.0
linux/arm64       8.0 8.7 8.9 9.0 10.0 11.0 12.0

The one architecture this hardware needs is the only entry the two images disagree on. The
arm64 list is Ampere and up, because that is what ships as an Arm-plus-NVIDIA system: A100,
Jetson Orin, GH200, Blackwell. Turing is not on that list and never will be.

Normally a missing target degrades to JIT from embedded PTX. Not here. The Dockerfile says
so, with a comment:

# Do not add +PTX here: vLLM filters torch's top-level PTX flag when it
# converts global gencode flags into per-kernel arch lists.

So it does not run slowly. It fails outright, with no kernel image is available for execution on the device.

The rest of the ecosystem splits the same way. Check before you plan anything:

Artifact	7.5 on arm64	State
`vllm/vllm-openai` arm64	no	Current. Never had it.
`nvcr.io/nvidia/pytorch` arm64	through 24.10	Dropped by 24.12.
`drikster80/vllm-aarch64`	yes	Abandoned Sept 2024. vLLM 0.6.1, far too old for Gemma 4.
PyPI torch aarch64	no	Built for 9.0 / 10.0 / 12.0.
AWS ARM64 GPU DLAMI	yes	Maintained. PyTorch 2.2 through 2.12.

AWS ships the one PyTorch that still has Turing

This is the finding that saves the whole exercise, and I nearly wrote it off. I had assumed
PyTorch's aarch64 CUDA wheels lacked sm_75 and that a from-source PyTorch build was
coming. That is true of the PyPI wheels. It is not true of AWS.

Read on two different DLAMIs, on the box:

torch 2.7.0+cu128    ['sm_75', 'sm_90', 'sm_100', 'sm_120']
torch 2.12.0+cu132   ['sm_75', 'sm_80', 'sm_90', 'sm_100', 'sm_110', 'sm_120']

AWS sells G5g, so AWS keeps Turing in the build — right through PyTorch 2.12 on CUDA 13.2,
an image cut three months ago. PyTorch never needs building. Only vLLM's own kernels do,
and CMake takes the arch list without argument:

-- CUDA target architectures: 7.5
CMake Warning: Pytorch version 2.11.0 expected for CUDA build, saw 2.12.0 instead.

That warning is worth reading twice, and I come back to it below.

The PyTorch DLAMI has no compiler

Two things the DLAMI does not give you, neither of them documented anywhere I could find.

There is no nvcc. The image ships the driver and a torch built against CUDA, not the
toolkit. You need the keyring and cuda-toolkit-13-2 from NVIDIA's sbsa repo — not the
x86 one, which is an easy reflex to get wrong on an Arm box.

And vLLM now wants Rust. Its vllm-rs frontend needs setuptools_rust plus a toolchain,
and the failure is a bare ModuleNotFoundError: No module named 'setuptools_rust' thrown
from metadata generation, several minutes in.

The newest vLLM was the only one that worked

No vLLM tag pins torch 2.12. They go 2.11, then jump to 2.13. I reasoned that building older
code against a newer runtime was the safer direction, took v0.26.0, and spent an hour being
wrong about it.

It builds fine. It then dies on model load:

transformers.integrations.heterogeneity.configuration_utils.AmbiguousGlobalPerLayerAttributeError:
'head_dim' is a per-layer attribute and may vary across layers.

Gemma 4's head_dim is not one number, and current transformers refuses to hand out a
global value for it. vLLM's config converter was still doing a flat
getattr(config, "head_dim", 0). The per_layer_config handling that copes with it landed
in v0.27.2rc0 — not v0.27.1, which I also checked. The newest tag was the only one that
worked.

If you take one process lesson from this: reach for the latest release first, and make the
constraint say out loud what stopped you when you fall back.

Gemma 4's attention heads are not one size

With the build working the server still would not start, and this failure has nothing to do
with Arm or packaging. It is this model against this chip.

Gemma4 model has heterogeneous head dimensions
{'sliding_attention': 256, 'full_attention': 512}.
FA4 not available, forcing TRITON_ATTN backend.

Read that as a chain, because every link is load-bearing:

Gemma 4's sliding layers are 256 wide. Its global layers are 512.
Only FA4 or Triton support heterogeneous head dims at all.
FA4 is not available, so vLLM forces TRITON_ATTN.
That choice is not yours to make. VLLM_ATTENTION_BACKEND is not a recognised variable in v0.27 — it logs Unknown vLLM environment variable detected and carries on. I set it twice before I read the warning.
Triton's unified attention kernel at head_size=512 wants about 96 KiB of shared memory per block.

64 KiB is the whole problem

Turing's shared memory is two numbers, and both are real. The default static limit per block
is 48 KiB — that is what torch.cuda.get_device_properties().shared_memory_per_block reports,
49,152 bytes. A kernel that needs more has to opt in through the dynamic shared-memory
attribute, and even then it tops out at 64 KiB. Ampere and later have 164 KiB and up.

Triton opts in, so it is measuring against the 64 KiB ceiling. It still does not fit:

triton.runtime.errors.OutOfResources: out of resource: shared memory,
Required: 98304, Hardware limit: 65536

Refused outright. Not slow, not degraded — the kernel will not launch, and it takes the
engine down during CUDA graph capture, which is late enough that you have already watched
the weights load and the KV cache get sized.

The fix is small. Shrink the KV tile until the query block and the K/V tiles fit inside the
budget, and drop the software pipeline to one stage. Gate it on pre-Ampere so it is a no-op
on every other card:

if current_platform.get_device_capability()[0] < 8:
    _smem_budget = 60000
    _esz = q.element_size()
    def _fits(t): return (BLOCK_M + 2 * t) * head_size * _esz <= _smem_budget
    while TILE_SIZE_PREFILL > 16 and not _fits(TILE_SIZE_PREFILL): TILE_SIZE_PREFILL //= 2
    while TILE_SIZE_DECODE  > 16 and not _fits(TILE_SIZE_DECODE):  TILE_SIZE_DECODE  //= 2
    launch_num_stages = 1

With that in vllm/v1/attention/ops/triton_unified_attention.py, graphs capture, the engine
comes up in 76 seconds, and the model serves. This is not upstream. It lives on my
instance and has to be reapplied on any vLLM upgrade, which makes it the obvious thing to
send back.

Most of the build is kernels that can never load

67 minutes on a g5g.4xlarge at MAX_JOBS=12, and the majority of it is FlashAttention.
vLLM compiles FA2 and FA3 regardless of TORCH_CUDA_ARCH_LIST — I watched it grind
through hundreds of sm90 Hopper instantiations on a build targeting 7.5 only. FA2 needs
sm80, FA3 needs sm90. Neither can ever load on this card.

Constraining VLLM_FA_CMAKE_GPU_ARCHES should cut that dramatically. I did not try it,
because by the time I understood what I was looking at the build was 45 minutes in and
interrupting it would have cost more than finishing.

What I got wrong before I had hardware

I wrote the rig's documentation before provisioning anything. Seven claims in it were wrong,
and every correction came off the machine rather than out of an argument. This is the part I
would keep if I kept nothing else.

What I wrote	What the box said
PyTorch aarch64 lacks `sm_75`	AWS DLAMI has it, on both versions I checked
bfloat16 is a hard failure here	Torch upconverts; vLLM logs `Casting torch.bfloat16 to torch.float16` and proceeds
The backend is XFORMERS	`TRITON_ATTN`, forced, not selectable
`VLLM_ATTENTION_BACKEND` picks it	Not a recognised variable. I had shipped dead config.
w4a16 needs sm80+ Marlin	The build compiled `sm75_kernel_float16_u4b8_float16.cu.o`
The GPU has 16 GB	15,360 MiB
`/v1/completions` returns an empty body	It returns `': ok: ok: ok: ok'` — garbage, not silence

That last one has teeth. If you health-check by testing for an empty response, this endpoint
passes while producing nonsense. Use /v1/chat/completions and read the text.

One claim is still standing only because I never tested it: whether g5g.xlarge's 8 GiB of
host RAM can stage 9.5 GiB of weights. Safetensors loading is mmap-backed, so I suspect it
can. It is labelled untested rather than stated as fact, which is where it should have been
all along.

What it does once it runs

content: 'Site Reliability Engineering (SRE) is a discipline that applies
          software engineering principles to infrastructure and operations
          problems to create highly reliable, scalable, and efficient systems.'
finish_reason: stop      usage: 19 prompt / 32 completion / 51 total

Measure	Value
Throughput, single stream greedy	42.9 tok/s @ 64, 43.1 @ 256
KV cache	2.95 GiB, 329,579 tokens
Concurrency at 16k context	20.12x
GPU memory while serving	13,501 / 15,360 MiB
Engine init	76.4 s, graph capture 17 s
Memory bandwidth, measured	277.0 GB/s read · 234.3 GB/s copy (320.1 theoretical)

Before reading too much into 43 tok/s, note what the memory does. The T4G has GDDR6, not
HBM — 256-bit bus at 5,001 MHz, so 320 GB/s theoretical. I measured 277 GB/s on a
streaming read (87% of peak) and 234 GB/s on a read-modify-write. Decode is bandwidth-bound,
so 277 is the real ceiling. For scale, a TPU v5e is about 859 GB/s normalized and a v6e about
1,638 — this part has roughly a third of one and a sixth of the other. It is a bandwidth-limited
card behaving like a bandwidth-limited card.

Single run, single stream, no repeats and no variance figure. One sample per cell, and taken
with the clamped tiles, so it is a floor rather than a characterisation. My Inferentia port
measured about 44 tok/s for E2B on one core, which is the same neighbourhood — but that is a
different harness on different silicon and I would not put the two in one table.

Troubleshooting quick reference

Symptom	Cause
`no kernel image is available`	Stock arm64 image. No 7.5, no PTX. Build from source.
`OutOfResources: shared memory`	Turing's 64 KiB against a 512-wide head. Clamp the tiles.
`AmbiguousGlobalPerLayerAttributeError`	vLLM older than v0.27.2rc0.
`No module named 'setuptools_rust'`	Missing Rust toolchain for `vllm-rs`.
`nvcc: not found`	PyTorch DLAMI has no toolkit. Install `cuda-toolkit-13-2` (sbsa).
`Unknown vLLM environment variable`	You set `VLLM_ATTENTION_BACKEND`. It does nothing.
Healthy endpoint, nonsense output	You checked `/v1/completions`. Use chat completions.

The short version

Take the AWS ARM64 GPU PyTorch DLAMI — it is the only maintained aarch64 stack that still
carries sm_75. Add cuda-toolkit-13-2 from the sbsa repo and a Rust toolchain, because the
image ships neither. Build vLLM v0.27.2rc0 or newer from source with
TORCH_CUDA_ARCH_LIST=7.5 and use_existing_torch.py, and patch the Triton attention kernel
to fit Turing's shared memory before you try to start it. Serve with --dtype float16 and
--kv-cache-dtype auto.

Nothing here failed loudly, and nothing failed where I was looking. The packaging gap I built
the rig around was already solved by AWS; the thing that actually stopped me was 32 KiB of
shared memory and a model whose global attention heads are twice as wide as its sliding ones.
Hardware this far off the mainstream will keep producing that shape of surprise — the fix is
not to reason harder about it, but to get to a box sooner and let it tell you.

Measured on EC2 g5g.4xlarge spot, us-east-1a. NVIDIA T4G, compute capability 7.5,
15,360 MiB, driver 595.71.05. Deep Learning ARM64 AMI OSS Nvidia Driver GPU PyTorch 2.12
(Ubuntu 24.04). torch 2.12.0+cu132, CUDA 13.2. vLLM 0.27.2rc1.dev0+g7f7a32cfe built from
v0.27.2rc0.

Vibecoding EventMatch, built with Antigravity, ADK, and Gemini

Sireesha Pulipati — Thu, 13 Aug 2026 04:18:51 +0000

Every week I get invited to more AI events than I can attend. Bond AI, Founders Bay, AI Collective, half a dozen other newsletters, each one dropping a handful of events into my inbox with no way to tell, at a glance, which ones I’m actually free for. Public aggregators like Luma and Eventbrite don’t help either. The interesting events, the ones from niche community newsletters, never show up there in the first place. If you live in a city like New York or San Francisco, this is a familiar kind of overload, too many good options, scattered across too many places, with no single view of what actually fits your week.

So I built EventMatch. And I vibecoded it, using Antigravity, Gemini 3.5 Flash, and Google’s Agent Development Kit.

Code · Kaggle writeup

Here’s the walkthrough, what I wanted the app to do, how the agents are put together, and what building it actually taught me.

What I wanted the app to do

The core ask was simple to state and harder to deliver. Pull events from public platforms and from the private newsletters sitting in my inbox, check every one of them against my real calendar, and only show me what’s actually conflict free.

A few things went beyond that baseline. Preferences needed to be conversational, not a set of rigid tags, something like “a mix of tech networking, AI workshops, creative art showcases, and relaxed social gatherings,” not a checkbox list. And those preferences needed to stay stable. If I ask a one off question like “show me free events this weekend,” that shouldn’t silently overwrite my standing preferences. Only an explicit request to update them should.

The rest was polish that ended up counting for more than I expected. Price shown as a tier, free, , $ , $$$, instead of raw text pulled from the source. One register button per event, not two competing ones in the same modal.

The multi agent architecture

EventMatch runs on ADK 2.0, and the workflow alternates between plain Python nodes and LLM agents. That alternation is the whole design.

prepare_extractor (Python) sanitizes the incoming message and reads the current session state.

preference_extractor (LlmAgent) parses the user’s message to pull out location, and decides whether this is a persistent preference change or a one off search.

filter_events_node (Python) pulls public data from Luma and Eventbrite, triggers background fetches from Gmail and Google Calendar, and merges everything. This is where the actual grounding happens. It runs a millisecond level time overlap check against the live calendar and flags conflicts before anything reaches the model, and converts events to plain text descriptions to keep the token count down for the next step.

vibe_ranker (LlmAgent) takes the conflict free events and scores them against the user’s preferences, 0 to 100, with a short explanation for each one.

formatter_node (Python) closes the loop, and it’s the piece that holds the whole system together. The vibe_ranker only ever sees plain text descriptions and returns ranked IDs. formatter_node takes those IDs and re-joins them against the original database records. Registration URLs, host names, event locations, none of it passes through the model twice.

Python does the things Python is reliable at, filtering, matching, deduplicating. The LLM does the one thing it’s actually needed for, judging fit against a vague preference. An LLM asked to both evaluate and format a result will, often enough, paraphrase a URL wrong or invent a detail that sounds plausible. Keeping those two jobs separate removes that failure mode instead of trying to prompt it away.

Integrations

Underneath the agents is a fairly ordinary web app, FastAPI on the backend, Python 3.11, React on the frontend, built with Vite and Tailwind.

The integration that makes everything else possible is OAuth. EventMatch requests read only scopes for Gmail and Calendar through Google’s official client libraries. Once authorized, the backend caches the credentials and the UI shows the connected account in the navbar. Everything downstream, the newsletter parsing, the conflict checking, only works because that connection is live.

Public event data comes from Luma and Eventbrite, which meant working around scraping restrictions rather than clean APIs. Deployment is Docker on Cloud Run, provisioned with Terraform.

Building it, and what it taught me

I built this in Antigravity, following a spec driven process rather than jumping straight into code. Start with a product requirements document. Turn that into a high level design. Turn the design into a spec detailed enough for an agent to build from. Only then does actual development start.

Coding a working app over a weekend is genuinely exciting. It’s also where that process got tested, and where it broke down. The pattern showed up early and kept showing up. Fix one bug, move to the next, fix that one, and the first bug resurfaces. Back and forth, the same category of issue reappearing in a slightly different shape each time.

The spec was the actual problem. When the requirements document and the spec are vague about what the app needs to do and what it needs to be tested against, the agent has room to solve the same problem two different ways in two different places, and neither implementation knows about the other. Every fix becomes local. Nothing stays fixed.

The fix is spending more time upfront, at the PRD stage and the spec stage, reviewing what the agent actually produced there before writing a line of application code. Get the spec and the test plan tight enough that they cover what “done” actually means, and the back and forth during development drops off. Skip that step, and you pay for it later, one resurfacing bug at a time.

Do I Still Need a Monkey Patch for Gemini Live?

xbill — Thu, 13 Aug 2026 01:41:40 +0000

No. And deleting 187 lines of it was the single biggest benefit of moving to ADK 2.x — but it was not the only one, and it was not the last thing that needed fixing.

What Does This Agent Do?

The project is a biometric security scanner, built to exercise the parts of the Gemini Live API that a text chatbot never touches. A browser captures webcam and microphone, streams both to a FastAPI backend over a single WebSocket, and the backend forwards them to Gemini 3.1 Flash Live through the Agent Development Kit. The model watches the video feed, counts the fingers being held up, and calls a tool.

Three tools are registered:

report_digit(count) — the detected finger count, which drives the UI
trigger_system_error() — fired on an offensive gesture, which terminates the session
trigger_heavy_metal_mode() — fired on the "Devil's Horns", a secret override

The transport is deliberately plain. Binary WebSocket frames carry a 1-byte type prefix — 1 for audio, 2 for JPEG — with 16 kHz PCM going up and 24 kHz PCM coming back, played through an AudioWorklet so the main thread stays free. Everything runs locally with make run, or on Cloud Run behind make deploy.

git clone https://github.com/xbill9/way-back-home
cd way-back-home/level_3_new

Two versions live side by side in that repo: level_3, the original design, and level_3_new, the current one. Most of this article is the diff between them.

What Level 3 Needed to Work

The original build ran on google-adk 1.27.2, and it worked — but only because a file named patch_adk.py sat next to it, 187 lines long, applied at import time before anything else could run.

The problem it solved was real. Gemini 3.1 deprecated media_chunks, the field a 1.x ADK used to send realtime media. A 1.x ADK talking to a 3.1 Live model would send the deprecated shape and get nothing useful back. The patch monkey-patched three separate call sites to translate:

# level_3_gemini/backend/app/patch_adk.py
if hasattr(rt_input, "media_chunks") and rt_input.media_chunks:
    logger.info("[PATCH] Unrolling 'media_chunks' from realtime_input.")
    for chunk in rt_input.media_chunks:
        ...
        await self.send_realtime_input(audio=chunk)
        ...
        await self.send_realtime_input(video=chunk)

The three targets were live.AsyncSession.send_realtime_input, which unrolled media_chunks into the new typed keywords; GeminiLlmConnection.send_realtime, which routed each blob to audio=, video= or text= by mime type; and AudioCacheManager.cache_audio, which was guarded against a NoneType blob that would otherwise raise.

It worked, and it was a liability. Monkey patching a framework means every upgrade is a gamble — the patch either becomes redundant, becomes wrong, or silently stops applying because the method it wraps was renamed. The closing recommendation in the original write-up was to delete it the moment the ADK supported the model natively.

What ADK 2.x Handles Natively

That moment arrived with google-adk 2.6.3, and patch_adk.py was deleted outright. The framework now does the routing itself, detecting the model generation and dispatching on it:

# google/adk/models/gemini_llm_connection.py, send_realtime()
if isinstance(input, types.Blob):
  if self._is_gemini_3_x_live or self._is_gemini_3_5_live_translate:
    if input.mime_type and input.mime_type.startswith('audio/'):
      await self._gemini_session.send_realtime_input(audio=input)
    elif input.mime_type and input.mime_type.startswith('image/'):
      await self._gemini_session.send_realtime_input(video=input)
    else:
      logger.warning(
          'Blob not sent. Unknown or empty mime type for'
          ' send_realtime_input: %s',
          input.mime_type,
      )
  else:
    await self._gemini_session.send_realtime_input(media=input)

Note the third branch. Audio and image mime types are dispatched explicitly, and anything else is dropped with a warning rather than guessed at — so a blob sent with a missing or unexpected mime type goes nowhere, and the only evidence is a log line.

Text is handled the same way. A single-part text Content is routed to send_realtime_input(text=...) for 3.x models rather than going out as client content, which matches the Live API's own guidance that send_client_content is only for seeding history.

That is the whole first patch target and the whole second one, upstream, maintained, and tested by someone else. The third — the NoneType guard on cache_audio — was not carried over, because upstream still calls len(audio_blob.data) unguarded. No path in this application produces a blob with data=None, so it stays deleted rather than being reintroduced as a precaution.

The One Thing That Broke

The patch was hiding a bug in the calling code, and deleting it exposed the bug rather than causing it.

LiveRequestQueue.send_realtime() accepts types.Blob and nothing else. The old patch had used model_construct internally, which skips Pydantic validation, so passing a bare string worked by accident. Without the patch it raises a ValidationError.

The call site that matters is the keepalive. This project sends a text stimulus every ten seconds when the client goes quiet, and under 1.x that stimulus was a string handed straight to send_realtime(). Text has to go through send_content() instead:

def send_text_stimulus(live_request_queue: LiveRequestQueue, text: str) -> None:
    live_request_queue.send_content(
        types.Content(role="user", parts=[types.Part(text=text)])
    )

This is the removal most likely to take an agent off the air quietly. It does not fail at startup. It fails the first time the keepalive fires, ten seconds into a session that otherwise looks healthy. Anyone migrating a Live agent off 1.x should check that call site before touching anything else.

What Else Was Updated

The migration was the headline, but it was not the end of the work. Reading the Live API documentation with the source open beside it turned up several things that had been wrong the whole time, none of which any build, test or lint run had ever objected to.

Video was running at twice the documented maximum. The capabilities guide is specific:

Video frames are sent as individual images (e.g., JPEG or PNG) at a specific frame rate (max 1 frame per second).

The project ran at 2 FPS and permitted up to 5 through an environment variable. Nothing rejects the surplus frames, which is why it went unnoticed — but they are billed, and they consume the session budget twice as fast. VIDEO_FPS now defaults to 1.0 and is hard-clamped there, so VIDEO_FPS=3 yields 1.0 rather than being honoured. A documented limit that is not enforced is how the 2 FPS crept in to begin with.

Audio-plus-video sessions cap at two minutes. From the session management guide:

audio-only sessions are limited to 15 minutes, and audio-video sessions are limited to 2 minutes

Context window compression removes the cap entirely. RunConfig.context_window_compression defaults to None, so it has to be asked for:

context_window_compression=types.ContextWindowCompressionConfig(
    sliding_window=types.SlidingWindow(),
),

This application streams both continuously, so it had been on the two-minute clock since the first version. Short test sessions never reached it. A demo where someone works through five gestures does.

Interruptions were documented and unhandled. When a user talks over the model, the model stops generating — but the audio it already sent is sitting in the client's ring buffer and keeps playing. The Live API guidance is to stop playback and clear the queue on interruption. ADK surfaces interrupted on the event, and because the backend forwards whole events as JSON, the flag was already arriving in the browser with nothing reading it. The clearing machinery already existed too. Three lines connected them.

The Log Line That Never Ran

Both input and output audio transcription were enabled in RunConfig from the very first version of this project. Neither ever produced a line of output.

input_transcription = getattr(event, "input_audio_transcription", None)
if input_transcription and input_transcription.final_transcript:
    logger.info(f"USER TRANSCRIPT: {input_transcription.final_transcript}")

Two mistakes are stacked here. input_audio_transcription is the RunConfig field that enables transcription — it is not the field on the event that transcription produces. And final_transcript is not a member of types.Transcription at all; the fields are text, finished, language_code, speaker_label and words.

Either mistake alone raises AttributeError and gets fixed in minutes. Together, behind the default on getattr, they produce silence. The condition evaluates to None and ..., which is falsy, forever.

The correct field names:

input_transcription = getattr(event, "input_transcription", None)
if input_transcription and input_transcription.finished:
    logger.info(f"USER TRANSCRIPT: {input_transcription.text}")

Gating on finished is deliberate. ADK emits partial transcription events with finished=False and one accumulated event with finished=True, so this produces one clean line per turn instead of one per fragment. The run_live() docstring is the reference: partial and non-partial events are both yielded to the caller, but only non-partial ones are saved to the session.

The fix produces this on connect, which is the exact opening line the agent instruction specifies:

INFO - GEMINI TRANSCRIPT: Scanner Online.

Video That Stops When You Look Away

The original design captured frames on a timer:

intervalRef.current = setInterval(() => { /* capture, send */ }, 500);

The rewrite replaced that with requestAnimationFrame and a manual elapsed-time check. On paper it is the better primitive — frame-aligned, idle when the compositor has nothing to do, and paired with toBlob instead of toDataURL it keeps JPEG encoding off the main thread.

In a backgrounded tab, requestAnimationFrame is throttled to zero.

The microphone is not. It runs in an AudioWorklet on the audio thread, which browsers keep alive so capture and playback survive a tab switch. The result is an asymmetric, silent failure: switch tabs and video stops completely while audio streams on. The WebSocket stays open, the session stays billed, and finger detection — the entire point of the application — stops with no error on either side. A 65-second session logged 8,050 audio packets and zero video frames.

Timers are throttled in background tabs as well, but to roughly one second rather than to zero, so the fix is a self-rescheduling timeout:

const captureFrame = () => {
    if (ws.current?.readyState === WebSocket.OPEN) { /* capture, send */ }
    if (intervalRef.current !== null) {
        intervalRef.current = setTimeout(captureFrame, frameIntervalRef.current);
    }
};

Degrading to roughly 1 FPS beats stopping. Re-reading the interval on each tick also keeps the server's config frame authoritative, which the rAF version did and a plain setInterval would not.

Code With No Caller

Four things were deleted outright, for ninety lines removed and one added.

The base64 JSON media path decoded type: "audio" and type: "image" payloads out of JSON. It was not speculative — it was the original design's entire wire protocol, orphaned when media moved to binary frames with a type prefix. Its two tests went with it; they were pinning an implementation, not a contract anyone relied on.

The proactivity and affective_dialog query parameters were declared on the WebSocket endpoint, documented in the docstring, and read by nothing. In the original design they were real, feeding a conditional RunConfig. Gemini 3.1 Flash Live then shipped without support for either, the config was removed, and the parameters outlived it. The Live API reference lists both under limitations — "Proactive audio — Not yet supported in Gemini 3.1 Flash Live" and the same line for affective dialogue — each followed by an instruction to remove any configuration for the feature.

A function-response scan walked server_content.model_turn.parts looking for function_response. model_turn carries model output; a functionResponse is something a client sends. The list was structurally always empty.

A second notification channel, lastMessage, was set on every match, system error and heavy-metal trigger, exported from the frontend hook, and read by no component. The callbacks drive the UI.

Rules for Staying on ADK 2.x

ADK 2.0 moved agents onto a graph engine, and the new constraints fail quietly rather than loudly. Four are worth knowing before writing anything:

Use the plain Agent. Custom BaseNode subclasses and _run_async_impl() or generate_content() overrides are silently bypassed. No error, no warning, no effect.
Never hand-append events to the session. It circumvents the graph engine and breaks determinism; run_live() owns the session.
Watch broad exception handlers. The framework catches exceptions for retries and human-in-the-loop pausing. A broad except inside a node masks that, and catching BaseException breaks pausing outright.
run_live(session=...) is deprecated. Pass user_id and session_id.

This project satisfies all four, which is why the upgrade was uneventful: a plain Agent, InMemorySessionService, no hand-built events, and its broad handlers sitting in the transport layer rather than inside the graph.

One 2.x addition is worth adopting. Runner gained auto_create_session, and run_live() already calls its internal get-or-create helper — without the flag a missing session is a ValueError, which is why the code hand-rolled get-then-create:

runner = Runner(
    app_name=APP_NAME,
    agent=root_agent,
    session_service=session_service,
    auto_create_session=True,
)

One is worth skipping. Runner(app=App(...)) is now described as the recommended construction, but Runner(agent=..., app_name=...) is still supported and gets wrapped into an App internally, with no deprecation warning. "Recommended" and "required" are different words.

Whether any of this is still true after the next ADK release is checkable rather than a matter of reading changelogs:

python -W error::DeprecationWarning -m pytest -q

Zero ADK warnings today. That command is the check after any bump.

Two Traps Worth Knowing

ADK's own docstring names a field that does not exist. run_live() refers to RunConfig.save_live_model_audio_to_session. In 2.6.3 the real fields are save_live_blob and save_live_audio. Where documentation and installed source disagree, the source wins — it is the thing that runs.

A green test suite proves less than it looks like. The suites stub run_live(), which is what makes them hermetic: no API key, no network, no charge. It also means session creation, resumption and teardown are never exercised, and the suite passes whether or not they work. The two changes most capable of breaking every session — auto_create_session and context_window_compression — were verified against the real API with a throwaway WebSocket client instead.

What Did Not Change

The wire protocol, the agent instruction, the tool definitions and the deployment path all came through the migration untouched: binary frames with a 1-byte prefix, 16 kHz PCM in and 24 kHz out, the three server-side tools, Secret Manager for the API key, and the model-id fallback to gemini-2.5-flash under adk run, since the Live preview model still 404s on generateContent.

A major-version framework upgrade that touches only the compatibility layer is the outcome to want. It is also the argument for keeping shims isolated in one file with a name that says what it is.

So What Really Changed?

patch_adk.py is deleted — 187 lines and three monkey-patched call sites, replaced by native routing in ADK 2.6.3.
Text goes through send_content(), because send_realtime() takes types.Blob only. The keepalive is the call site that catches people out.
Video runs at 1 FPS with a hard ceiling, matching the documented maximum instead of doubling it.
Context compression is enabled, lifting a two-minute cap on audio-plus-video sessions.
Transcript logging works, after reading a RunConfig field name off the event since the original design.
The capture loop is a timer again, because requestAnimationFrame stops dead in a background tab while the microphone does not.
Barge-in clears the playback queue, connecting a flag that was already arriving to machinery that already existed.
Ninety lines with no caller are gone, along with the two tests that covered them.

Summary

The Agent Development Kit 2.x release removed the need for a compatibility patch that had been carried since the original build, and deleting it was almost entirely subtractive — the pleasant kind of upgrade. The one removal that bites is send_realtime() rejecting anything that is not a types.Blob, which surfaces as a keepalive failure ten seconds into an otherwise healthy session rather than as a crash at startup.

The wider lesson came after the migration. A framework upgrade tells you what stopped compiling; it says nothing about what still compiles and is wrong. A dead branch compiles. A no-op log line runs. A throttled callback returns cleanly. An undocumented frame rate is accepted by the server. Every one of these survived a migration, a test suite, a lint config and a demo that visibly worked, and each was found by reading the documentation next to the code rather than by running anything.

For anyone running a Live agent of their own, three checks cost about an hour between them: confirm the transcript handler has ever produced output, confirm video keeps flowing with the tab hidden, and read the session-limit page next to your RunConfig.

Riverpod to BlocSignal: Incremental Migration and Zero-Codegen Signals for Flutter

Randal L. Schwartz — Wed, 12 Aug 2026 23:05:09 +0000

Introduction: Combining Riverpod Safety with Zero Codegen

If you've been building Flutter apps with Riverpod, you already know the joy of compile-time provider safety, auto-disposal, and synchronous notification propagation via ProviderListenable. Riverpod fundamentally raised the bar for Flutter state management.

However, as projects grow, many Riverpod developers run into familiar friction points:

Code Generation Overhead: Depending heavily on riverpod_generator and waiting on build_runner watches during rapid UI iteration.
Complex Provider Trees: Managing nested ProviderScope overrides and .family cache eviction policies when scaling large teams.
All-or-Nothing Migration Concerns: Wanting to trial new reactive primitives or event-driven BLoC architectures without rewriting an entire codebase.

What if you could combine the compile-time safety and synchronous reactivity you love in Riverpod with zero code generation, fine-grained signal graph reactivity, and the ability to trial or migrate incrementally screen-by-screen?

Enter BlocSignal and bloc_signals_riverpod.

The Core Ergonomics: Riverpod `Notifier` vs. `CubitSignal`

Let's compare the classic Todos application—one of the benchmark examples in the official Riverpod monorepo—ported directly to BlocSignal.

The Riverpod Approach (Requires `@riverpod` & `build_runner`)

In modern Riverpod, creating a todo list with reactive filtering typically involves a generated Notifier and separate provider getters or ref.watch selectors:

@riverpod
class TodoList extends _$TodoList {
  @override
  List<Todo> build() => const [];

  void addTodo(String description) {
    state = [...state, Todo(id: DateTime.now().toString(), description: description)];
  }

  void toggle(String id) {
    state = [
      for (final todo in state)
        if (todo.id == id) todo.copyWith(completed: !todo.completed) else todo
    ];
  }
}

// Derived filter provider
@riverpod
List<Todo> filteredTodos(Ref ref) {
  final todos = ref.watch(todoListProvider);
  final filter = ref.watch(todoFilterProvider);
  return switch (filter) {
    TodoFilter.all => todos,
    TodoFilter.active => todos.where((t) => !t.completed).toList(),
    TodoFilter.completed => todos.where((t) => t.completed).toList(),
  };
}

The BlocSignal Approach (100% Pure Dart 3, No Codegen)

With BlocSignal, your state container is a standard handwritten Dart class (CubitSignal<List<Todo>>). Reactive derivations like filteredTodos and uncompletedCount are declared inline using signals computed():

import 'package:bloc_signals/bloc_signals.dart';
import 'package:flutter/foundation.dart';
import 'package:signals_core/signals_core.dart';

class TodosCubit extends CubitSignal<List<Todo>> {
  // Pass `equals: listEquals` to enforce value-based list equality for state de-duplication!
  TodosCubit([List<Todo> initialTodos = const []])
      : super(initialState: initialTodos, equals: listEquals) {
    // 1. Reactive filter signal
    filter = signal(TodoFilter.all);

    // 2. Synchronously derived computed signals
    filteredTodos = computed(() {
      return switch (filter.value) {
        TodoFilter.all => stateValue,
        TodoFilter.active => stateValue.where((t) => !t.completed).toList(),
        TodoFilter.completed => stateValue.where((t) => t.completed).toList(),
      };
    });

    uncompletedCount = computed(() => stateValue.where((t) => !t.completed).length);
  }

  late final Signal<TodoFilter> filter;
  late final ReadonlySignal<List<Todo>> filteredTodos;
  late final ReadonlySignal<int> uncompletedCount;

  void addTodo(String description) {
    emit([...stateValue, Todo(id: DateTime.now().toString(), description: description)]);
  }

  void toggle(String id) {
    emit([
      for (final todo in stateValue)
        if (todo.id == id) todo.copyWith(completed: !todo.completed) else todo
    ]);
  }

  void setFilter(TodoFilter newFilter) => filter.value = newFilter;

  @override
  Future<void> close() async {
    filter.dispose();
    filteredTodos.dispose();
    uncompletedCount.dispose();
    await super.close();
  }
}

What Changed?

No build_runner: No .g.dart generated files, no background watchers, no build step delays.
Built-In Custom Equality (equals: listEquals): Because Dart List instances don't override == by default, passing equals: listEquals configures the underlying signal graph to de-duplicate state emissions based on list content equality.
Fine-Grained Signal Graph: Updates to filter.value or calling emit(...) re-evaluate computed() derivations synchronously and notify only dependent widgets.
Explicit Container Lifecycles: Disposing the cubit disposes its internal signals automatically.

Zero Risk: Incremental Trial & Bidirectional Interop

You do not need to rewrite your application to try BlocSignal. With the bloc_signals_riverpod package, you can seamlessly bridge the two frameworks in both directions.

dependencies:
  bloc_signals: ^1.0.0
  bloc_signals_flutter: ^1.0.0
  bloc_signals_riverpod: ^1.0.0

Option A: Expose a `BlocSignal` / `CubitSignal` to Existing Riverpod Widgets

Want to write a new feature or state controller with BlocSignal, but keep your existing Riverpod UI layer (ConsumerWidget, WidgetRef)?

Simply call .toProvider():

import 'package:bloc_signals_riverpod/bloc_signals_riverpod.dart';

// Create your new BlocSignal or CubitSignal
final todosCubit = TodosCubit();

// Convert it directly into a Riverpod NotifierProvider!
final todosProvider = todosCubit.toProvider();

// Now consume it anywhere in existing Riverpod widgets:
class LegacyRiverpodWidget extends ConsumerWidget {
  const LegacyRiverpodWidget({super.key});

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    // Rebuilds reactively whenever todosCubit emits!
    final todos = ref.watch(todosProvider);

    return ListView.builder(
      itemCount: todos.length,
      itemBuilder: (context, i) => Text(todos[i].description),
    );
  }
}

Option B: Adapt an Existing Riverpod Provider into `BlocSignal`

Have a legacy Riverpod provider that you need to read from a new BlocSignalBuilder widget?

Use .toBlocSignal(ref):

import 'package:bloc_signals_riverpod/bloc_signals_riverpod.dart';

class NewFeatureWidget extends ConsumerWidget {
  const NewFeatureWidget({super.key});

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    // Adapt any Riverpod ProviderListenable into a BlocSignal container!
    // Automatically binds ref.onDispose to close the container when disposed.
    final todosBloc = legacyRiverpodProvider.toBlocSignal(ref);

    return BlocSignalBuilder<BlocSignalBase<List<Todo>>, List<Todo>>(
      bloc: todosBloc,
      builder: (context, todos) {
        return ListView.builder(
          itemCount: todos.length,
          itemBuilder: (context, i) => Text(todos[i].description),
        );
      },
    );
  }
}

Converting `AsyncValue` <-> `AsyncState`

bloc_signals_riverpod also provides extension methods to map seamlessly between Riverpod's AsyncValue and Signals' AsyncState:

// Riverpod AsyncValue to Signals AsyncState
final asyncState = riverpodAsyncValue.toAsyncState();

// Signals AsyncState to Riverpod AsyncValue
final asyncValue = signalsAsyncState.toAsyncValue();

Explore the Official Riverpod Ports on blocsignal.dev

To prove the DX gains and side-by-side equivalence, we've ported canonical state management examples directly from the official rrousselGit/riverpod monorepo into the BlocSignal open-source repository:

📝 Riverpod Todos (examples/riverpod_todos):
- Replaces @riverpod codegen and Notifier with CubitSignal<List<Todo>> and synchronous computed() signals for reactive filter tabs and stats.
🔍 Pub.dev Package Search (examples/riverpod_pub):
- Replaces Riverpod AsyncNotifier with BlocSignal and a streamless restartable() event transformer that automatically cancels in-flight API requests on keypresses without Rx streams.
🦸 Marvel Character Browser (examples/riverpod_marvel):
- Demonstrates API pagination, character search, and widget tree scoping via BlocSignalProvider.value.

Explore all 20 side-by-side benchmark ports across BLoC, Signals, and Riverpod live at blocsignal.dev/#ported-examples!

Built-In AI Agent Skills for Automated Migration

If you use AI coding assistants like Antigravity, Gemini, Cursor, or GitHub Copilot, BlocSignal publishes a dedicated Agent Plugin skill bundle (riverpod_migration.md).

When your AI assistant inspects a project with BlocSignal skills enabled, it automatically understands:

How to map StateNotifierProvider / NotifierProvider to CubitSignal;
How to preserve auto-disposal and cancellation contracts;
How to refactor ConsumerWidget rebuild boundaries to BlocSignalBuilder or SignalBuilder;
How to apply bloc_signals_riverpod interop adapters during multi-phase refactoring.

Summary

You don't need to throw away your existing architecture to enjoy the speed, simplicity, and zero-codegen elegance of reactive signals.

With bloc_signals_riverpod, you can trial BlocSignal on a single screen today, bridge your existing Riverpod providers seamlessly, and upgrade your developer experience at your own pace.

🌐 Website & Comparison Benchmarks: blocsignal.dev
📦 Pub.dev Packages: bloc_signals | bloc_signals_flutter | bloc_signals_riverpod
🐙 GitHub Repository: RandalSchwartz/BlocSignal

MCP Configuration for Looker with Codex

xbill — Wed, 12 Aug 2026 17:06:10 +0000

This article covers the MCP setup and configuration for using Looker with Codex to enhance and extend Looker operations over MCP.

Deja Vu — What is Old is New!

This paper is the third pass at the same idea. The original used Gemini CLI:

MCP Configuration for Looker with Gemini CLI

then Antigravity CLI:

MCP Configuration for Looker with Antigravity CLI

then Claude Code:

MCP Configuration for Looker with Claude Code

In this updated version, Codex is used to integrate Looker functionality. The Looker side of the stack does not change at all — that is the whole point of MCP. What changes is the client: how the server gets registered, how tool calls get approved, and where the agent reads its project instructions from.

What is Looker?

Looker is a cloud-based business intelligence (BI) and data analytics platform owned by Google Cloud that enables organizations to analyze, visualize, and share data in real-time. It uses a unique modeling language called LookML to define data relationships, offering a centralized “single source of truth” for metrics. Looker focuses on embedded analytics and live data exploration rather than storing data itself.

More information is available here:

Looker business intelligence platform embedded analytics

Key Features and Capabilities

LookML (Looker Modeling Language): A code-based modeling language that allows data analysts to define dimensions, aggregates, and calculations, ensuring consistent metrics across the organization.
Live Data Connection: Looker does not import data; it queries your data warehouse directly (e.g., BigQuery, Snowflake, Redshift) in real-time, ensuring data is always up to date.
Embedded Analytics: Looker can be embedded into other applications, websites, or portals, allowing businesses to provide data insights directly within their own tools.
Self-Service BI: Users can explore data, create visualizations, and build custom dashboards using a browser-based interface without needing deep SQL knowledge.
Workflow Integration: Actionable data insights can be sent directly to other applications, such as triggering an email based on specific business rules.

Looker? I thought Big Query Did everything!

Semantic layer is where all the cool kids hang out.

What is MCP?

Unless you have been living off grid without Internet- MCP is the new universal connector and next “Big Thing”.

More information is here:

What is Model Context Protocol (MCP)? A guide

Google MCP Strategy

Google has gone all-in for all the core Cloud services to provide connections over MCP. An overview is here:

Google Cloud MCP servers overview | Google Cloud Documentation

MCP Toolbox

MCP Toolbox is the “swiss army” knife that connects your data sources to MCP.

Use Looker with MCP, Gemini CLI and other Agents | Google Cloud Documentation

Looker MCP Setup

For a more detailed step by step setup instructions — there is a full codelab that goes through the setup:

Connect Gemini CLI to Looker with MCP Toolbox | Google Codelabs

and a further deep dive is here:

Use Looker with MCP, Gemini CLI and other Agents | Google Cloud Documentation

Codex

Codex is OpenAI's terminal-driven, agent-assisted coding CLI — the same category of tool as Gemini CLI, Antigravity CLI and Claude Code, and like all of them it ships a full MCP client.

Install it with npm:

npm install -g @openai/codex

or with Homebrew:

brew install codex

Then authenticate — Codex will open a browser to sign in with your ChatGPT account, or you can supply an API key:

codex login

Verify the install:

codex --version

Google Skills Repository

Google Skills give your MCP client well known approaches to work with the core Google products including Big Query.

The full details are here:

Level Up Your Agents: Announcing Google's Official Skills Repository | Google Cloud Blog

To install the Skills:

npx skills install github.com/google/skills

This vendors the skills into .agents/skills/ and records them in skills-lock.json. They are client-neutral markdown, so the same checkout serves Codex, Claude Code and Gemini CLI.

What you talkin ‘bout Willis?

That was a lot of setup! But wait- there is more! So what is different about this lab compared to all the others out there?

This demo is one of the first deep dives into configuring Looker for MCP with Codex. Codex provides a complete working environment with a full MCP client. Looker exposes the key features of the platform over the MCP layer.

The interesting wrinkle in the Codex version is approvals. Roughly half of the ~50 Looker tools mutate your live instance — make_look, make_dashboard, add_dashboard_element, the *_project_file family, the git and dev-mode tools. Codex has a first-class per-server approval mode, so this repo pins write tools behind a confirmation prompt while leaving discovery and querying to run freely. Read on.

Where do I start?

The strategy for configuring Looker with MCP is an incremental step by step approach.

First, the Looker configuration settings are retrieved. Then, these settings are used to configure Codex. Finally- Codex is used as a MCP client to the Looker environment. Several samples are run using the Looker MCP Tools directly from Codex.

Looker Admin Setup

For Looker (Google Cloud core) — Admins do not directly create keys for standard users; instead, they enable the permission for users to manage their own.

Navigate to the Looker Admin Users page (Admin > Users).

Click Edit next to the specific user.
Locate the API Keys field and toggle it to Enabled.
Once enabled, the user can generate their own keys by going to their personal Account settings page (User Icon > Account > API Keys).

Looker Instance URL

To connect to the Looker setup — you need to derive your Looker Base URL. Typically this will be the hostname in the Looker app domain.

For the test instance- this is an example of what the URL looks like (note the HTTPS prefix and no trailing slash):

https://780eb09e-7dab-4076-9ec1-ecf9d8414630.looker.app

Looker User Setup

First Login to your Looker User environment. Go to Profile->Account (in upper right hand side) and bring up the user settings:

If the API Key box is unavailable- contact your Admin to enable the API setup on a per user basis.

Once you have access to create API keys- the settings will look similar to this:

Then click the “Manage” button to setup the API Keys:

Click Create New API key to generate the API Key. Save the Client ID and Client Secret.

Setup the Basic Codex Environment

At this point you should have a working Shell environment and a working Codex installation. All of the relevant code examples and documentation is available in GitHub.

The next step is to clone the GitHub repository to your local environment:

cd ~
git clone https://github.com/xbill9/looker-mcp-codex
cd looker-mcp-codex

Then run init.sh from the cloned directory.

The script will attempt to determine your shell environment and set the correct variables:

source init.sh

This helper script will prompt for your Looker Instance details:

xbill@penguin:~/looker-mcp-codex$ source set_env.sh
Looker Base URL (e.g. https://your-company.looker.com): https://780eb09e-7dab-4076-9ec1-ecf9d8414630.looker.app
Looker Client ID:
Looker Client Secret:
Downloading MCP Toolbox binary...
Downloading from https://storage.googleapis.com/mcp-toolbox-for-databases/v1.6.0/linux/amd64/toolbox...
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 292M 100 292M 0 0 71.8M 0 0:00:04 0:00:04 --:--:-- 71.8M
Successfully installed MCP Toolbox binary (v1.6.0).
Environment successfully set up.

Current Environment (.env) — secret masked:
GOOGLE_GENAI_USE_VERTEXAI=True
GOOGLE_CLOUD_PROJECT=comglitn
GOOGLE_CLOUD_LOCATION=us-central1
LOOKER_BASE_URL=https://780eb09e-7dab-4076-9ec1-ecf9d8414630.looker.app
LOOKER_CLIENT_ID= **************
LOOKER_CLIENT_SECRET= ********
LOOKER_VERIFY_SSL=true
LOOKER_TOOLBOX=/home/xbill/looker-mcp-codex/toolbox

If your session times out or you need to re-authenticate- you can run the set_env.sh script to reset your environment variables:

source set_env.sh

One difference worth calling out versus the Claude Code write-up: with Codex you do not strictly need to source the script before every session. The launcher reads .env itself at process start. Sourcing is still the better habit, because it also puts LOOKER_* into your shell so you can drive toolbox (or the Looker CLI) by hand.

Codex MCP Configuration

Codex reads MCP servers from TOML. This repo ships a project-scoped .codex/config.toml:

[mcp_servers."looker-toolbox"]
command = "bash"
args = ["./start-looker-mcp.sh"]
cwd = "."
enabled = true
startup_timeout_sec = 30
tool_timeout_sec = 120
default_tools_approval_mode = "writes"

Four of those lines are the whole story:

command / args point at a small launcher script rather than at toolbox directly. That keeps the config file free of both secrets and shell quoting.
startup_timeout_sec = 30 — the toolbox binary is ~300 MB and does a real handshake against your Looker instance on boot. The stock timeout is tight enough that a cold start on a slow link can look like a broken server.
tool_timeout_sec = 120 — a run_dashboard against a ten-tile dashboard is ten warehouse queries. Two minutes is a realistic ceiling.
default_tools_approval_mode = "writes" — this is the important one. Discovery and query tools run unattended; anything that mutates the instance stops and asks. See the approvals section below.

The launcher, start-looker-mcp.sh, is deliberately boring:

#!/bin/bash

set -eu

PROJECT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

if [ -f "$PROJECT_DIR/.env" ]; then
    set -a
    # shellcheck disable=SC1091
    source "$PROJECT_DIR/.env"
    set +a
fi

LOOKER_VERIFY_SSL="${LOOKER_VERIFY_SSL:-true}"
export LOOKER_VERIFY_SSL

if [ -z "${LOOKER_BASE_URL:-}" ] || [ -z "${LOOKER_CLIENT_ID:-}" ] || [ -z "${LOOKER_CLIENT_SECRET:-}" ]; then
    echo "looker-toolbox: Looker credentials are missing. Run: source set_env.sh" >&2
    exit 1
fi

exec "${LOOKER_TOOLBOX:-$PROJECT_DIR/toolbox}" --stdio --prebuilt looker,looker-dev

Credential resolution order:

.env in the project root — used when present, and takes precedence.
Already-exported LOOKER_* variables — used as a fallback when there is no .env.
Neither — the launcher exits with a message telling you to run source set_env.sh, rather than dying with an opaque MCP connection error.

That third case is worth the eight lines it costs. Every MCP client, Codex included, reports a server that exits during startup as a generic transport failure. Failing loudly with a sentence of English turns a twenty-minute debug into a five-second one.

The repository also retains a Claude-compatible .mcp.json that launches the same binary through a bash -c wrapper. Both files contain only variable references, so both are safe to commit.

Trusting the Project

Codex will not load a project-scoped config from a directory it does not trust. On first launch inside the repo it will ask; approve it once and the setting sticks:

xbill@penguin:~/looker-mcp-codex$ codex

  You are running Codex in ~/looker-mcp-codex

  Since this folder is not version-control trusted, choose how to proceed:

  > 1. Yes, allow Codex to work in this folder
    2. No, exit

If you would rather register the server globally instead of per-project, put the same [mcp_servers."looker-toolbox"] block in ~/.codex/config.toml and use an absolute path for command/cwd. Project-scoped is the better default here — the launcher, the .env and the toolbox binary all live in the checkout, so the config travels with them.

Initial Connection

Start Codex from the project directory:

xbill@penguin:~/looker-mcp-codex$ codex

Then use /mcp to confirm the server came up:

> /mcp

  MCP Servers

  looker-toolbox   ✔ connected   45 tools
    command  bash ./start-looker-mcp.sh
    cwd      /home/xbill/looker-mcp-codex
    approval writes

You can also check without entering the TUI at all, which is handy in CI or when scripting a machine setup:

xbill@penguin:~/looker-mcp-codex$ codex mcp list
Name            Command  Args                   Env  Cwd  Status
looker-toolbox  bash     ./start-looker-mcp.sh  -    .    enabled

and confirm Codex actually parsed every option — a typo'd key is silently ignored, so this is the fast way to prove the approval mode took effect:

xbill@penguin:~/looker-mcp-codex$ codex mcp get looker-toolbox
looker-toolbox
  enabled: true
  transport: stdio
  command: bash
  args: ./start-looker-mcp.sh
  cwd: .
  env: -
  startup_timeout_sec: 30
  tool_timeout_sec: 120
  default_tools_approval_mode: writes
  remove: codex mcp remove looker-toolbox

Now ask for the inventory:

> list the looker mcp tools

Here are the Looker MCP tools available in this session, grouped by function:

Metadata / Discovery

- get_models — list LookML models
- get_explores — list explores
- get_dimensions — list dimensions in an explore
- get_measures — list measures in an explore
- get_filters — list filters
- get_parameters — list parameters
- get_dashboards — list dashboards
- get_looks — list Looks
- get_projects — list LookML projects

Querying / Running

- query — run a Looker query
- query_sql — run raw SQL against a connection
- query_url — build a query URL
- run_look — run a saved Look
- run_dashboard — run a dashboard
- run_lookml_tests / get_lookml_tests — LookML data tests

Connections / Database Introspection

- get_connections
- get_connection_databases
- get_connection_schemas
- get_connection_tables
- get_connection_table_columns

Content Creation                                    [approval required]

- make_look — create a Look
- make_dashboard — create a dashboard
- add_dashboard_element — add a tile to a dashboard
- add_dashboard_filter — add a filter to a dashboard
- create_view_from_table — generate a LookML view from a DB table
- generate_embed_url — create an SSO embed URL

LookML Project / File Management                    [approval required]

- get_project_files / get_project_file / create_project_file /
  update_project_file / delete_project_file
- get_project_directories / create_project_directory / delete_project_directory
- validate_project

Git (LookML version control)                        [approval required]

- list_git_branches / get_git_branch / create_git_branch /
  switch_git_branch / delete_git_branch
- dev_mode — toggle development mode

Health / Diagnostics

- health_analyze
- health_pulse
- health_vacuum

That's 51 tools total. The three groups marked [approval required] will prompt
before running, per default_tools_approval_mode = "writes" in .codex/config.toml.

Want me to run one of them (models, connections, or dashboards) to verify connectivity?

A Word About Approvals

This is where the Codex configuration earns its keep. Codex separates two things that other clients tend to conflate: the sandbox (what the agent may do to your filesystem and network) and tool approval (which MCP tools may fire without a human in the loop).

default_tools_approval_mode = "writes" means: run read-only tools freely, prompt before anything that changes state. A make_dashboard call surfaces as:

  ⚠ looker-toolbox › make_dashboard  wants to run

    title        VIP Customer Intelligence v2
    description  Real-time insights into top-performing revenue segments
    model        advanced_ecomm

  > 1. Yes, run it
    2. Yes, and don't ask again for make_dashboard
    3. No, and tell Codex what to do differently

Option 2 is worth understanding before you reach for it. Approving make_dashboard for the session is low-risk — it creates new content. Approving delete_project_file for the session is not. The Looker toolsets do not distinguish create from destroy in their naming, so read the tool name, not the category.

The blunter instruments still exist if you want them — --ask-for-approval never for a fully unattended run, --sandbox read-only to clamp the agent's filesystem access. Neither is a substitute for the per-tool mode: sandbox settings govern the agent's own shell, not what an MCP server does to a remote SaaS instance over HTTPS. A read-only sandbox will happily let a tool call drop a LookML file on your Looker instance.

The other half of the safety story is AGENTS.md, which Codex reads automatically from the project root. That is where this repo records the rules the config file cannot express — pass filter values bare, work LookML changes on a dev-mode branch, don't use mutating tools for routine verification, never print the client secret. Same role as CLAUDE.md for Claude Code and GEMINI.md for Gemini CLI; all three live side by side in the checkout.

All Right Already! Can you actually do something with Looker?

At this point- Codex is set up with the MCP Toolbox and has direct access to all the Looker MCP Tools. Any operations with the Tools come back directly into the Codex context.

To get an overview of Dashboards:

> get_dashboards

• looker-toolbox › get_dashboards

Here are the dashboards in your Looker instance (16 total):

  ID   Title                                       Description
  ──   ─────────────────────────────────────────   ──────────────────────────────────────
   1   2) Business Pulse - Intermediate            —
   2   1) Business Pulse - Basic                   —
   3   3) Business Pulse - Advanced                —
   4   Agreement for Shared Use of this Instance   —
   5   test                                        —
   6   test                                        —
   7   Inventory                                   —
   8   TestJC                                      —
   9   Self-serve industry                         —
  10   TestOrdersByCity                            —
  11   My Dashboard                                —
  12   test-orders                                 —
  13   E-commerce Overview                         Key e-commerce metrics
  14   💎 VIP Customer Intelligence                Advanced LTV and Ranking logic
  15   ☁️ Cloud Optimization Hub                    AI-driven cost savings, Recommender API
  16   Top Users & Countries Overview              Top spenders and high-performing countries

A few observations:
- Several test/scratch dashboards (5, 6, 8, 10, 11, 12) look like leftovers — cleanup candidates.
- The numbered "Business Pulse" set (1–3) are example/tutorial dashboards.
- The substantive content is 13–16.

Want me to run one of these (run_dashboard), or inspect the tiles on a specific one?

You can also get models:

> get_models

• looker-toolbox › get_models

Here are the LookML models in your instance (11 total):

  Model                    Label                 Project                             Connection(s)
  ──────────────────────   ───────────────────   ─────────────────────────────────   ─────────────────────────
  basic_ecomm              Z) Sample LookML      sample_thelook_ecommerce            default_bigquery_connection
  intermediate_ecomm       Z) Sample LookML      sample_thelook_ecommerce            default_bigquery_connection
  advanced_ecomm           Z) Sample LookML      sample_thelook_ecommerce            default_bigquery_connection
  extension-api-explorer   Extension API Expl.   marketplace_extension_api_explorer  default_bigquery_connection
  data_block_acs_bigquery  Census Data Block     marketplace_acs_census-v2           american-community-survey
  gcp_billing_block        Google Cloud Billing  marketplace_gcp-billing             billing-export-sample
  chrome-ux-block          Chrome UX Block       marketplace_chrome-ux-block         chrome-ux
  london_bicycles          London Bicycles       london_bicycles                     london-bicycles
  gde_rk                   Gde Rk                gde_rk                              (8 connections)
  data-dictionary          Data Dictionary       marketplace_extension_data_dict.    default_bigquery_connection
  bq_agent_analytics       Bq Agent Analytics    agent_events                        default_bigquery_connection

Notable groupings:
- theLook e-commerce samples (basic/intermediate/advanced_ecomm) — Looker's standard demo dataset.
- Marketplace blocks — Census/ACS, GCP Billing, Chrome UX, Data Dictionary, API Explorer.
- Public datasets — london_bicycles.
- gde_rk — a multi-connection catch-all model.

Want me to pull the explores for one of these? advanced_ecomm is the richest starting point.

And drill into the details — the advanced_example_ecommerce explore is where the interesting LookML lives:

> get_measures for advanced_example_ecommerce

• looker-toolbox › get_measures

20 measures, grouped by view:

order_items — core sales metrics
  order_items.count                 # of Order Items        count
  order_items.total_sale_price      Sales                   sum
  order_items.average_sale_price    Average Price           average
  order_items.first_order_date      First Order Date        date
  order_items.count_for_drill_*     drill-support counts    count

users / products / inventory_items
  users.count                       # of Users              count_distinct
  products.count                    Count Distinct Products count_distinct
  inventory_items.total_cost        Total Cost              sum
  inventory_items.average_cost      Average Cost            average

User-summary measures (filtered vs. lifetime symmetric-aggregate patterns)
  user_summary_filtered.total_sales_for_these_users      Sales (Filtered Summary)   sum_distinct
  user_summary_unfiltered.total_sales_for_these_users    Sales (Lifetime)           sum_distinct
  user_summary_filtered.total_items_for_these_users      Items (Filtered Summary)   sum_distinct
  user_summary_unfiltered.total_items_for_these_users    Items (Lifetime)           sum_distinct

Special measures — advanced ratio/comparison
  user_summary_and_order_items_fields.percent_of_lifetime_unfiltered_sales
  user_summary_and_order_items_fields.filtered_average_vs_unfiltered_average

Notes:
- The filtered vs. lifetime pairs showcase Looker's pattern for comparing a filtered
  subset against a user's full lifetime totals.
- The count_for_*_drill measures exist to support drill-down on dashboard tiles.

Let's see what VIP tiers look like:

• looker-toolbox › query

VIP Tiers — Lifetime Revenue Rank Group (advanced_example_ecommerce)

  Rank Group    Users    Total Sales   Order Items   Avg Sale Price
  ───────────   ──────   ───────────   ───────────   ──────────────
  1st 25%       15,569    $4,134,224        53,458           $77.34
  2nd 25%       16,554    $1,931,678        36,005           $53.65
  3rd 25%       16,557      $947,615        24,026           $39.44
  Top 1000 ⭐    1,000      $731,207         4,893          $149.44
  Bottom 25%    16,560      $382,605        17,800           $21.49

What this shows

- Steep revenue concentration. The top quartile drives $4.13M — about 55% of the
  ~$7.4M lifetime total — while the bottom 25% contributes only ~5%, despite having
  more users.
- The "Top 1000" is a distinct elite segment, not a quartile — it overlaps with the
  1st 25%. Just 1,000 users generate $731K, and their $149 average sale price is
  nearly 2× the top quartile's and 7× the bottom's.
- Basket value declines cleanly down the tiers: $77 → $54 → $39 → $21. Rank
  correlates strongly with per-item spend, not just order count.

Reading note: the quartile groups partition all purchasing users, while Top 1000 is
a separate flag sitting inside the 1st quartile — its dollars are also counted in the
1st 25% row. Don't sum the column.

The First Approval Prompt

Now build a Look — and this is where the writes approval mode fires for the first time:

> save that as a Look

  ⚠ looker-toolbox › make_look  wants to run

    model     advanced_ecomm
    explore   advanced_example_ecommerce
    title     VIP Tiers — Lifetime Revenue Rank Groups
    fields    user_summary_unfiltered.user_revenue_rank_group,
              users.count, order_items.total_sale_price,
              order_items.count, order_items.average_sale_price
    sorts     order_items.total_sale_price desc
    vis       looker_bar

  > 1. Yes, run it
    2. Yes, and don't ask again for make_look
    3. No, and tell Codex what to do differently

  [1]

• looker-toolbox › make_look

✅ Look created — "VIP Tiers — Lifetime Revenue Rank Groups"

- ID: 8
- URL: /looks/8
- Viz: horizontal bar chart, sorted by total sales, value labels on
- Saved to: your personal folder

Want me to add it to a dashboard, or create companion Looks (Top 1000 by
country/category)?

The prompt shows the full argument payload before anything touches the instance. That is the part worth having — you can see explore, fields and folder and catch a wrong model before it creates content, rather than after.

But Wait! There’s More!

Once the Tools are exposed by MCP they have full access to Looker resources. Time to update a Dashboard — three mutations, batched into one approval round:

> add it to dashboard 14 and create the companion Looks

I'll do all three — add the tile to dashboard 14, and create both companion Looks.

  ⚠ looker-toolbox › add_dashboard_element  →  dashboard_id 14      [1] approved
  ⚠ looker-toolbox › make_look  →  "Top 1000 VIPs by Country"       [1] approved
  ⚠ looker-toolbox › make_look  →  "Top 1000 VIPs by Category"      [1] approved

• looker-toolbox › (3 calls)

All three succeeded. Pulling the underlying data so you can see what they contain.

• looker-toolbox › (2 calls)

✅ Dashboard tile added
"VIP Tiers — Lifetime Revenue Rank Groups" is now a tile on 💎 VIP Customer
Intelligence (dashboard 14).

✅ Look 9 — Top 1000 VIPs by Country (/looks/9)

  Country          VIP Users   Total Sales   Avg Price
  ──────────────   ─────────   ───────────   ─────────
  China                  359      $259,129     $147.23
  United States          224      $163,185     $147.41
  Brasil                 143      $106,613     $155.64
  South Korea             53       $37,381     $129.34
  France                  52       $37,178     $149.31
  Spain                   47       $35,646     $163.51
  Germany                 43       $32,112     $158.19
  United Kingdom          37       $28,494     $178.09
  …6 more (AU, BE, JP, PL)

✅ Look 10 — Top 1000 VIPs by Product Category (/looks/10)

  Category              Total Sales   Items   Avg Price
  ───────────────────   ───────────   █████   ─────────
  Outerwear & Coats        $245,412     664     $369.60
  Suits & Sport Coats       $70,838     278     $254.81
  Jeans                     $58,421     423     $138.11
  Active                    $48,295     220     $219.52
  Sweaters                  $36,618     325     $112.67
  Dresses                   $35,942     185     $194.28
  …9 more

Notable insights
- Geography is concentrated: China + US + Brasil = ~726 of the 1,000 VIPs. The UK
  has the highest per-item spend ($178) despite few users.
- Outerwear dominates category spend — $245K, more than the next three categories
  combined, at a $369.60 average sale price. High-ticket outerwear is the signature
  purchase of your best customers.

This will look similar to:

And can also be accessed directly in Looker:

So what else can Codex do?

Health checks and instance maintenance — and the first real one surfaced a finding, just not the expected kind:

> run a full health sweep

Running usage analysis, six pulse checks, and cleanup candidates in parallel.

• looker-toolbox › (10 calls)


Meanwhile — a metadata-based audit does work

get_dashboards / run_dashboard don't touch System Activity, so here's a cleanup pass
from what we already gathered:

  ID     Title                          Flag
  ────   ────────────────────────────   ─────────────────────────────────────
  5, 6   "test" (×2, duplicate name)    Scratch — almost certainly deletable
  8      TestJC                         Scratch
  10     TestOrdersByCity               Scratch
  11     My Dashboard                   Default/placeholder name
  12     test-orders                    Scratch
  14     💎 VIP Customer Intelligence   Superseded by v2 (#17) — 7 broken tiles

~7 dashboards worth reviewing for deletion.

Note what Codex did not do here: it identified six deletion candidates and stopped. Deleting dashboards is a mutation, and the approval mode plus the AGENTS.md rule ("confirm before creating/altering/deleting Looker content") both point the same direction. The agent proposes; you dispose.

Show me the Money!

The key value shows up when you push the combination of Codex, the model, and the Looker tools over MCP:

> "Are we at risk of losing our best customers?"

I'll investigate using your Looker instance. Starting with what data is available,
then digging into retention signals.

• looker-toolbox › get_models

This is the thelook e-commerce dataset. Exploring the richest model.

• looker-toolbox › (3 calls)

Rich model — there's a lifetime revenue-rank on each user, so I can define "best
customers" precisely (Top Revenue Group / Top 1000) and track activity over time.
First, the data's date range, so I know what "recent" means.

• looker-toolbox › query

Data runs right up to today. Now the core analysis: the Top Revenue Group's monthly
spend and active-customer count over the last ~2 years.

• looker-toolbox › query

The business is growing overall, so a raw trend hides churn. The real risk signal is
recency — how many of our best customers have gone quiet.

• looker-toolbox › (4 calls)

Here's what the data says. I defined "best customers" as your Top Revenue Group —
the top 1,000 by lifetime valid revenue ($731K lifetime, ~$731 each). Then I looked
at whether they're still actually buying.

Short answer: the cohort's revenue is healthy, but there's a large dormant tail.

Aggregate spend from top customers is at record highs — valid revenue rose from
~$12–17K/month in late 2024 to $43K most recently, growing alongside the overall
business. The franchise isn't collapsing.

But recency tells a more worrying story. Of the 1,000 all-time best customers:

  Last purchased within   # still active   Share   → Gone silent
  ─────────────────────   ──────────────   ─────   ───────────────────────────
  90 days                            258     26%   74% (742) inactive 3+ months
  180 days                           419     42%   58% (581) inactive 6+ months
  365 days                           585     59%   42% (415) inactive 12+ months

So ~415 of your top-1,000 (42%) haven't placed a valid order in over a year —
effectively lapsed. At the cohort's average lifetime value (~$731), that's on the
order of ~$300K of historical revenue that has stopped repeating, plus another ~160
in the 6–12 month "cooling off" zone.

How to read this

- It's a tail problem, not a top-line collapse. A concentrated set of top customers
  is very active, while a long tail of former big spenders has quietly aged out.
  That's partly natural — "best customers" is a lifetime label.
- The actionable risk is the 6–12 month cooling band (~160 customers) — recently
  valuable, now slipping, still winnable.

Suggested next steps (I can do any of these — just say the word)

1. Pull the actual at-risk list — name, email, lifetime spend, days-since-last-order.
2. Build a "Best Customer Retention" dashboard so this is monitored ongoing.
3. Add a churn-risk segment to the LookML model so "at-risk best customers" becomes
   a reusable, governed field.

Note that #2 and #3 create or modify content in your live Looker instance — those
will come through as approval prompts, and I'll confirm specifics before running them.

Pull at-risk customers in real time, based on your actual data:

Your best customers who've gone quiet

The 5 highest-lifetime-value members of the Top-1,000 revenue group with no valid
order in 90+ days, ranked by lifetime spend. "Cooling" = last bought 3–6 months ago
(most winnable); "Dormant" = 6+ months silent.

  #  Customer          Country    Channel    Lifetime $  Orders  Status
  ─  ───────────────   ────────   ────────   ──────────  ──────  ──────────
  1  Craig Dickerson   France     Email          $1,849       8  🟡 Cooling
  2  Mark Nolan        USA        Organic        $1,826       7  🟡 Cooling
  3  John Hill         Brazil     Search         $1,554       4  🔴 Dormant
  4  Glenn Foster      USA        Facebook       $1,455       8  🟡 Cooling
  5  Thomas Davis      Australia  Search         $1,435       8  🔴 Dormant

Notable: your single highest-lifetime-value customer overall (Craig Dickerson,
$1,849) is on this list — cooling, not yet lost.

The full at-risk population (all 1,000 top customers)

- 🟢 258 active (bought in last 90 days) — healthy
- 🟡 161 cooling — last order 3–6 months ago  ← highest-priority win-back
- 🟠 166 slipping — last order 6–12 months ago
- 🔴 415 lapsed — no order in 12+ months

So 742 of your top 1,000 (74%) haven't purchased in 90+ days.

Method note: ranked by lifetime valid spend; "at-risk" = no valid order in 90 days,
cross-checked against the 90- and 180-day active sets. Status is bucketed by
90/180-day windows rather than exact per-customer last-order dates — the toolbox
couldn't compute that inline.

What-if scenarios:

> "What if we raised outerwear prices 10% — what happens to VIP revenue and would it
   dent volume?"

A pricing what-if on the VIP segment. Grounding it in actual numbers first.

• looker-toolbox › (2 calls)

Outerwear is the key finding: it's your VIPs' #1 category by a wide margin — $245K
(34% of all VIP lifetime revenue), 664 items at $370/item. This decision hits VIPs
where they're most concentrated. Pulling cost too, for profit impact.

Baseline — VIP outerwear today

  Metric             Value
  ────────────────   ───────────────────────
  Revenue            $245,412
  Units sold         664
  Avg price / unit   $369.60
  Cost / unit        $163.48
  Gross profit       $136,859 (55.8% margin)
  Profit / unit      $206.11

What a +10% price does

New price = $406.56 (+$36.96/unit; cost unchanged, so every extra dollar is margin).

  Volume response      Units (Δ)     Revenue   vs today   Gross profit   vs today
  ──────────────────   ──────────   ────────   ────────   ────────────   ────────
  No dip (inelastic)   664  (0%)    $269,956   +$24,544       $161,405   +$24,546
  −5%                  631 (−33)    $256,458   +$11,046       $153,335   +$16,476
  −10%                 598 (−66)    $242,960    −$2,452       $145,266    +$8,407
  −15%                 564 (−100)   $229,462   −$15,950       $137,197      ~flat
  −20%                 531 (−133)   $215,964   −$29,449       $129,127    −$7,732

The two numbers that matter

- Revenue break-even: ~9% volume loss. You can lose up to ~60 of the 664 units
  before the hike stops adding revenue.
- Profit break-even: ~15% volume loss. Because you keep the full $37 markup at zero
  added cost, profit holds even if revenue dips.

Would it dent volume? Expect a modest dip. For your most loyal customers buying a
premium considered-purchase category, elasticity is likely mild (roughly −0.3 to
−0.8) — the top two rows: revenue +$11–20K, profit +$16–22K, only ~20–50 fewer units.

One strategic caveat

Tying back to the last analysis: 74% of your VIPs are already cooling or lapsed, and
outerwear is their anchor category. A blanket 10% hike risks nudging cooling VIPs out
the door, and that churned lifetime value would swamp the ~$15–20K modeled gain.
- Raise on healthy/active VIPs; protect the at-risk ones.
- Apply to new-season inventory rather than a visible retroactive hike.
- Run it as a holdout test on a VIP subset first to get real elasticity.

Caveat on the model: elasticity is assumed, not measured — no price-experiment
history exists in this data.

Reading the LookML

Because looker-dev is loaded alongside looker, Codex can read the actual model source, not just the metadata — which is what makes "can I trust this measure?" an answerable question:

> "What does filtered_average_vs_unfiltered_average actually compute, and can I
   trust it?"

Let me read the LookML that defines it. First, locating the project files.

• looker-toolbox › (2 calls)

The measure lives in the user_summary_and_order_items_fields view. Reading it plus
the two views it depends on.

• looker-toolbox › (3 calls)

I have the full definition and its dependencies. Verifying my reading against live
data before giving a verdict — pulling the raw ingredients alongside the measure.

• looker-toolbox › query

The math reconciles exactly (Outerwear: 145.37 / 88.47 − 1 = 0.643 ✓). Full teardown:

What it computes

The percent difference between the average item price in the current query context
and the same shoppers' average item price over their entire lifetime. Positive =
they're buying pricier-than-usual items in this slice.

The actual SQL

  measure: filtered_average_vs_unfiltered_average {
    label: "Avg Price vs These Users' Avg Price (Lifetime)"
    type: number
    sql: (${order_items.average_sale_price} * 1.0
          / nullif(${user_summary_unfiltered.average_sale_price_for_these_users}, 0)) - 1 ;;
    value_format_name: percent_0
  }

Where each half comes from — the clever part

  Term          Field                                    Respects query filters?
  ───────────   ──────────────────────────────────────   ─────────────────────────
  Numerator     order_items.average_sale_price           Yes — rows in scope
  Denominator   user_summary_unfiltered.average_sale_    No — lifetime avg for
                price_for_these_users                    those same users

The denominator's "lifetime" behavior comes from one line:

  derived_table: {
    explore_source: advanced_example_ecommerce { ... bind_all_filters: no }
  }

That derived table is one row per user, joined back to order_items by user_id.
Because it's a 1-user-row joined to many order-item rows, Looker applies symmetric
aggregates so summing lifetime sales/items counts each user exactly once (no
fan-out). Its twin, user_summary_filtered, is identical but bind_all_filters: yes.

Empirically verified:

  Category            Filtered avg   Lifetime avg   Measure   Check
  ─────────────────   ────────────   ────────────   ───────   ──────────────────────
  Outerwear & Coats        $145.37         $88.47      +64%   145.37/88.47−1 = 0.643 ✓
  Suits                    $118.39         $77.38      +53%   ✓
  Blazers & Jackets         $92.51         $67.84      +36%   ✓

Can you trust it? ✅ Yes — with three rules

Correctly built: proper symmetric-aggregate handling, nullif(...,0) guards against
divide-by-zero, both sides on the same "Valid Orders only" basis. No bug here.

But it's a ratio-of-averages, so respect its nature:

1. Don't sum, average, or trust a Totals row for it. Ratio measures don't roll up.
2. The baseline is dynamic, not a global constant. Change a filter that changes which
   users appear and the baseline moves. This is the #1 way people misread it.
3. It's about per-item price mix, not spend or loyalty.

Bottom line: trustworthy for exactly one question — "in this slice, are these
shoppers trading up or down on per-item price versus their own norm?"

Note that this whole exchange used only read tools — get_project_files, get_project_file, query — so it ran without a single approval prompt. The mode is calibrated so that understanding your instance is frictionless and changing it is not.

Executive briefings

The dashboard existed already (dashboard 14, built earlier from advanced_ecomm). To
make it a proper briefing I added an executive layer on top of the existing charts:
a 4-KPI summary strip, and a "bottom line" callout.

The briefing in 30 seconds

  Metric                  Value    Meaning
  ─────────────────────   ──────   ─────────────────────────────────────
  Lifetime revenue        $7.40M   All purchasing users
  Top-1000 VIP revenue    $731K    $149 avg basket — ~7× the bottom tier
  Revenue concentration   56%      From the 1st customer quartile
  VIP geography           72%      Of VIP spend from China, US, Brasil

Bottom line for the exec team: revenue is highly concentrated — a 1,000-customer VIP
core spends ~7× the average shopper, is geographically narrow (three countries), and
is premium-led (Outerwear & Coats alone: $245K at a $370 avg item). The strategic
lever is retaining and expanding the VIP core, not broad acquisition.

One note on reading the charts: the Top 1000 tier is an elite flag inside the 1st
quartile — its dollars are already counted there, so don't sum the tier bars.

These are only high level examples of what can be done with the tech stack!

One More Thing: the Looker CLI

The MCP path is not the only way in. The repo also ships a Makefile target that installs the Looker CLI into the project root, checksum-verified:

make cli                              # latest release
make cli LOOKER_CLI_VERSION=v0.4.8    # pinned
make clean                            # remove downloaded binaries, keep credentials

It reads the same LOOKER_* variables, so source set_env.sh covers both. Useful for the deterministic, scriptable half of the work — CI checks, bulk operations — while MCP covers the exploratory half.

Troubleshooting

A short list of the things that actually go wrong:

Symptom	Cause	Fix
`/mcp` shows no servers	Project not trusted, so `.codex/config.toml` never loaded	Restart `codex` in the repo root and approve the trust prompt
Server fails immediately	No `.env` and no exported `LOOKER_*`	`source set_env.sh` — the launcher prints exactly this
Server times out on startup	300 MB binary + Looker handshake on a cold/slow start	Raise `startup_timeout_sec`
`run_dashboard` times out	Ten tiles = ten warehouse queries	Raise `tool_timeout_sec`, or run tiles individually
All `health_*` return Access Denied	API3 role lacks `see_system_activity`	Admin → Roles, add the permission
Filter returns nothing	Value was quoted	Pass values bare — `first_touch`, not `"first_touch"`
Tools run without asking	Approval mode not applied	`codex mcp get looker-toolbox` — if the key isn't echoed back it was misspelled and silently dropped. Also check you didn't pick "don't ask again" earlier in the session

Summary

Codex was configured as a Looker MCP client using the MCP Toolbox. The .codex/config.toml registration points at a secret-free launcher script that resolves credentials from .env at runtime, and pins write-capable tools behind default_tools_approval_mode = "writes" so discovery and analysis run unattended while anything that mutates the live instance stops and asks. The MCP connection was then used to explore the instance, read and verify LookML, build Looks and dashboards, and run open-ended business analysis against the governed semantic model.

The stack underneath is unchanged from the Gemini CLI, Antigravity CLI and Claude Code versions of this paper — same toolbox binary, same looker,looker-dev toolsets, same ~50 tools. That is the actual result worth noting: the same repository, with three client config files sitting side by side, serves all of them.

Preventing Quota Crashes via Antigravity CLI Agent Hooks

Tanaike — Wed, 12 Aug 2026 07:15:38 +0000

Solving the LLM quota monitoring paradox with zero-overhead local Connect RPC agent hooks.

Abstract

Google Antigravity CLI users using Google OAuth face abrupt task failures when API quota hits 0%, while account switching triggers unrecoverable signature errors. Querying quota via LLM tool calls creates a paradox by consuming the very tokens being monitored. We resolve this with antigravity-cli-check-usage-plugin, a CLI Agent Hook running outside the LLM execution turn. Directly querying local Connect RPC endpoints, it monitors quota with zero token overhead and injects proactive warning banners when threshold limits are reached.

1. Introduction

Developers relying on Google Antigravity CLI for autonomous pair programming frequently encounter a frustrating barrier: running out of API quota mid-session. When using Google OAuth authentication, your quota can silently hit 0%, causing task execution to halt abruptly with an unrecoverable quota error:

⚠ Individual quota reached. Please upgrade your subscription to increase your limits. Resets in 1h00m00s.
Error ID: 49a81c0f

To bypass this roadblock, developers often attempt to log out and switch to a paid Google Cloud project billing account. However, in Antigravity CLI v1.1.12, attempting to resume an active agent session after switching accounts triggers a critical signature mismatch failure:

⚠ Invalid thought signature.
Error ID: e2901f4c

This error prevents the session from continuing, forcing you to wait until the quota resets. While future CLI updates may resolve this session state issue, waiting for a patch is not a viable strategy when shipping code today.

The architectural divergence between standard tool-based monitoring and our agent hook model is illustrated in Figure 1. While developers can manually run the /usage slash command to view quota, AI agents executing multi-step autonomous tasks cannot trigger /usage programmatically. In traditional CLI workflows, invoking quota checks via LLM tool calls requires passing context back and forth through the inference API, depleting active model tokens. Conversely, the zero-overhead agent hook interceptor executes locally prior to prompt dispatch, querying the process socket silently and injecting status alerts only when remaining quota breaches configured safety bounds.

In this article, to overcome the limitation of agents being unable to trigger /usage, we walk through the engineering journey of building antigravity-cli-check-usage-plugin. By combining local Connect RPC inspection with proactive lifecycle hooks, this plugin automatically performs external quota checks with Zero Quota Consumption (0 LLM tokens), completely preventing mid-session crashes.

2. Repository

The plugin developed and discussed in this article is open-sourced and available on GitHub:

GitHub Repository: tanaikech/antigravity-cli-check-usage-plugin

This repository contains the dual-runner entrypoint (entrypoint.sh), Python script (check_quota.py), pure Bash fallback script (check_quota.sh), lifecycle hook manifest (hooks.json), and default threshold configuration (config.json), allowing instant one-command installation as an Antigravity CLI plugin across any developer environment.

3. Core Motivation

While Antigravity CLI provides the /usage slash command for developers to manually inspect quota limits, AI agents executing autonomous task loops cannot invoke /usage programmatically.

If we attempted to solve this by equipping the AI agent with a custom tool to query the internal RPC endpoint (/exa.language_server_pb.LanguageServerService/GetUserStatus), the tool invocation and context turns would consume LLM API tokens. This creates a fundamental paradox: using LLM context tokens to check remaining quota consumes the very quota you are trying to preserve.

In addressing this challenge, the solution built upon our previously published article, A Developer’s Guide to Agent Hooks in Antigravity CLI. Recalling the out-of-band execution mechanics of CLI Agent Hooks explored in that guide, we leveraged lifecycle events (PreInvocation and PostInvocation) to run local process checks completely outside the LLM inference turn—guaranteeing zero API token quota overhead.

Zero Token Overhead: During normal operation, quota checking runs entirely outside the LLM context (via local Python/Bash scripts) without invoking LLM tool calls.
Local RPC Interception: It automatically queries the CLI's internal status endpoint on 127.0.0.1 without external network calls.
Proactive Threshold Alerting: It notifies both the developer and the AI agent before quota hits 0%, preventing session corruption and hard crashes.

4. Connect RPC

Through reverse-engineering the Antigravity CLI local process architecture (originally explored in the antigravity-usage repository by skainguyen1412), we discovered that the running agy process hosts a local HTTPS server using the gRPC / Connect Protocol on 127.0.0.1.

By querying the internal endpoint /exa.language_server_pb.LanguageServerService/GetUserStatus, we can retrieve real-time model quota fractions and reset timestamps directly from the local process.

Because the agy process may open multiple listening sockets on 127.0.0.1 for IPC and WebSockets, a shell loop that probes each detected port until it receives a valid userStatus response is required:

# Scan listening sockets for the active 'agy' process on loopback (127.0.0.1)
for PORT in $(ss -tulpn 2>/dev/null | grep agy | awk -F'127.0.0.1:' '{print $2}' | awk '{print $1}' | sort -u); do
  # Post a Connect Protocol request to the internal GetUserStatus RPC endpoint
  RES=$(curl -k -s -X POST https://127.0.0.1:${PORT}/exa.language_server_pb.LanguageServerService/GetUserStatus \
    -H "Content-Type: application/json" \
    -H "Connect-Protocol-Version: 1" \
    -d '{"metadata":{"ideName":"antigravity","extensionName":"antigravity","locale":"en"}}')

  # Verify if the response contains the userStatus JSON key
  if echo "$RES" | grep -q "userStatus"; then
    echo "$RES" | jq .
    break
  fi
done

To execute this logic seamlessly and rapidly inside an agent hook outside the LLM invocation turn, we implemented a Python script using standard library components, alongside a pure Bash fallback script (check_quota.sh) and an entrypoint runner (entrypoint.sh) that automatically selects Python when available or Bash on systems without Python installed.

[!IMPORTANT]

Note on Scope: The GetUserStatus endpoint returns the Five Hour Limit Remaining fraction (remainingFraction) and ISO reset timestamp (resetTime) for active model pools. The long-term Weekly Limit Remaining is not exposed through this RPC endpoint.

5. Complete Agent Hook Workflow

Building upon the lifecycle concepts detailed in A Developer’s Guide to Agent Hooks in Antigravity CLI, the plugin integrates into the Antigravity CLI by registering PreInvocation and PostInvocation agent hooks in hooks.json. Because PreInvocation fires after the user submits input but before the prompt payload is dispatched to the LLM backend, it inspects local process state and dynamically injects steps prior to model inference.

As detailed in Figure 2, the final agent hook operates under two distinct execution patterns based on the configured warning threshold (default: 20%):

Pattern A: Normal Operation (Quota > Threshold)

When remaining quota is above the warning threshold, the hook outputs an empty step injection payload:

{
  "injectSteps": []
}

Impact: Zero Quota Consumption (0 Token Overhead). The hook executes silently in less than 50 milliseconds. No messages or extra context are injected into the LLM session, consuming absolutely zero model quota.

Pattern B: Warning State (Quota <= Threshold)

When remaining quota drops to or below the threshold, the hook injects a transient system message with mandatory agent directives:

{
  "injectSteps": [
    {
      "ephemeralMessage": "⚠️ [SYSTEM QUOTA WARNING] Model quota is below threshold (20%) (Active: gemini-3.6-flash-medium):\n - GEMINI Models [ACTIVE MODEL]: 20.0% remaining (Refreshes in 3h 00m)\n\n[MANDATORY INSTRUCTION FOR AGENT]: The model quota has dropped below the threshold. You MUST display a prominent Quota Warning banner at the very top of your response for THIS TURN ONLY! Do NOT display a warning banner on subsequent turns unless another quota warning is explicitly injected. In the warning banner, you MUST also inform the user that they can run the '/usage' command at any time to inspect detailed quota status."
    }
  ]
}

Impact: The AI agent immediately prepends a prominent Quota Warning banner to its response, advising the developer to run /usage or pause heavy multi-step automation before encountering a hard crash.

6. Installation & Dual Runtime

The complete implementation is published as an open-source Antigravity CLI plugin: antigravity-cli-check-usage-plugin.

Installation

Install the plugin directly via the Antigravity CLI:

agy plugin install https://github.com/tanaikech/antigravity-cli-check-usage-plugin

Dual Runtime Architecture: Python Primary + Pure Bash Fallback

The plugin features a multi-environment entrypoint (entrypoint.sh) producing 100% identical JSON outputs across both runtimes. The engineering rationale behind this dual design includes:

Python (Primary Runner): Requires zero external dependencies like jq, absorbs OS-specific syntax differences across Linux, macOS, and Windows, and guarantees type-safe date math.
Pure Bash (Fallback Safety Net): Ensures instant execution in minimal or containerized environments where Python is not pre-installed.

Configuration and Disabling

You can customize or completely disable the warning threshold (default: 20.0%) using environment variables, configuration files, or hook arguments.

Set Custom Threshold (e.g., 25%):

export QUOTA_THRESHOLD=25.0

Disable Quota Check Completely:
Setting QUOTA_THRESHOLD to -1 instructs the hook to skip all RPC queries immediately:

export QUOTA_THRESHOLD=-1

7. Real-World Testing & Verification

After installing the plugin, setting export QUOTA_THRESHOLD=80.0 and executing a live session test in Antigravity CLI v1.1.12 demonstrates the hook in action, as captured in Figure 3:

When the user enters a simple greeting (hello), the agent hook instantly detects that the active model's remaining quota (71.0%) has dropped below the configured threshold (80.0%). A prominent yellow Warning banner (Quota Warning: GEMINI Models quota is at 71.0% remaining...) is dynamically prepended at the top of the AI's response, alerting the developer and providing a reminder to inspect detailed limits via /usage.

8. Updating & Uninstalling

To update the plugin to the latest version or remove it from your environment:

Check installed plugins:

  agy plugin list

Uninstall the plugin:

  agy plugin uninstall antigravity-cli-check-usage-plugin

Reinstall the updated version:

  agy plugin install https://github.com/tanaikech/antigravity-cli-check-usage-plugin

Summary

In this article, we presented a zero-overhead solution to eliminate mid-session quota crashes and account-switching signature errors in Google Antigravity CLI. Drawing upon foundational concepts from A Developer’s Guide to Agent Hooks in Antigravity CLI and resolving the paradox where using LLM tool calls to query internal RPC endpoints consumes quota, we built native CLI Agent Hooks (PreInvocation / PostInvocation) running completely outside the LLM execution turn. Featuring a dual Python primary and pure Bash fallback architecture, the hook probes internal local Connect RPC endpoints with absolute zero token consumption during normal operation. By proactively injecting warning banners and /usage reminders when quota drops below threshold, it guarantees universal environment compatibility and eliminates task interruptions cleanly at the root.

Latency vs. Tokens: What I Learned Optimizing an Agent with Gemma (and What Didn't Work)

leslysandra — Wed, 12 Aug 2026 03:32:11 +0000

I'd been waiting for more than 30 minutes. The terminal just sat there, blinking, without returning a single word. I'd launched Gemma2 in its 9-billion-parameter version on my laptop (a regular Mac, the kind any professor or student would use) and the model simply wasn't responding.

It wasn't a bug. It was the most honest answer the experiment could have given me.

That frustrating wait ended up being, without exaggeration, the most interesting finding of the whole process. Because the question that brought me there wasn't "how big can a model get?" — it was a much more practical one: what actually happens when an agent you built in a tutorial has to survive in production?

I've been working with Gemma as a case study to understand that jump — from an educational prototype to something that can hold up under long conversations, limited hardware, and real users. This post is the honest summary of that process: what worked convincingly, what didn't work the way I expected, and why that "didn't work" turned out to be more useful than a clean result would have been.

The real problem: why tutorials are a little dishonest

Almost every conversational agent tutorial does the same thing, without saying so out loud: on every turn, it sends the model the entire previous history, all over again.

Imagine that every time you added a sentence to a conversation, you had to repeat everything said before it — every message, every reply — before you could say the new one. At first you don't notice. But if the conversation runs 30 or 50 turns, you're repeating an entire novel just to add one sentence.

This pattern is called linear context stacking, and it causes three concrete problems:

Memory saturation — every call to the model processes an increasingly large context.
Risk of hitting the token limit — every model has a maximum context window; sooner or later, you hit it.
Quality degradation — there's a documented phenomenon in NLP literature called "lost in the middle": when context gets very long, models pay less attention to information sitting in the middle of it, versus the beginning or end. In other words, it's not just slower — it gets worse.

This problem isn't unique to any one model, but it weighs differently depending on context. If you're using a closed API with a massive context window and pay-per-token billing, the cost of this problem is financial — you just pay more. But if you're running an open model locally, as is common in universities and research labs across Latin America, the cost is infrastructure: limited RAM, no dedicated GPU, no room to "just pay for more compute." An unbounded context isn't a minor optimization detail there — it's the difference between the agent working at all or not.

The experiment: design and decisions

To avoid staying purely theoretical, I ran a simple but controlled comparative experiment using Gemma 2 (2B), running locally with Ollama — no dependency on any paid external API.

The idea: simulate a typical technical conversation (a microservice troubleshooting case, where each turn adds new information) and run it against two different architectures:

Pipeline A (Naive): accumulates the entire history with no compression at all. This is, literally, what a tutorial-style agent looks like.
Pipeline B (Optimized): applies history pruning — instead of sending the whole conversation, it sends a compact summary of the latest state.

# Pipeline A — accumulates everything, no pruning
conversation_history += f"\nPrevious text {i+1}: {chunk}\n"
full_prompt = f"{conversation_history}\n{TASK_PROMPT}\n{chunk}"

# Pipeline B — only a compact summary of the latest state
full_prompt = f"Previous compact context: {compact_context}\n{TASK_PROMPT}\n{chunk}"

Three methodological decisions I almost overlooked, and which turned out to be key to making the results trustworthy:

1. The "cold start" nearly ruined everything.
In my first run, the first step of each pipeline came out suspiciously slower than the ones after it — several seconds off. It wasn't the prompt size: it was the cost of loading the model into memory the first time it's called. The fix was adding a throwaway "warm-up" call before starting to measure each pipeline, so both started on equal footing.

2. Real tokens, not estimated ones.
At first I was estimating tokens by counting words and applying an approximate conversion factor — a completely avoidable loss of precision. Ollama returns the real, exact count in every response (prompt_eval_count). Switching to that number made the charts far more defensible.

3. A single run isn't enough.
I ran each pipeline 3 times and averaged the results, with error bars included in the charts. This is what honestly revealed that one of my early results wasn't as solid as it first looked — more on that below.

Results: what held up cleanly, and what didn't

Tokens: the result that actually holds

The token pattern was consistent across all 3 runs, with no ambiguity. The naive pipeline grows linearly — from 107 to 266 tokens in just 4 steps, nearly tripling. The optimized pipeline flattens into a plateau, around 104 tokens.

That's a 61% reduction in input tokens by the final step. Active context management delivers exactly what it promises: it keeps the conversation's memory footprint from growing unchecked.

Latency: the result that forced me to rethink the hypothesis

This is where the experiment got genuinely interesting. The intuition says: fewer input tokens, faster response. The real data didn't back that up — at least not clearly. The error bars for the naive and optimized pipelines overlap in almost every step.

Why? Because with a 2B model, on relatively short conversations, total response time is dominated by how much the model has to generate as output — not by how much it has to read as input. Shrinking the context doesn't automatically speed up the generation of the response.

It's a "negative" result in the sense that it doesn't confirm the initial hypothesis, but it's honestly the most valuable finding of the whole experiment: context management and latency are related problems, but they're not the same problem, and optimizing one doesn't guarantee improving the other.

The failed attempt with Gemma2 9B (and why I'm not hiding it)

I wanted to push one step further and repeat the comparison with Gemma2's 9B version, to see whether a larger model would show a clearer latency advantage — the hypothesis being that processing a long prompt weighs more when the model itself is bigger.

I never got that data. Over 30 minutes running on my laptop, without a single complete response. I had to cancel it.

I could have left this out of the post. But it's a relevant data point in its own right, and honestly the one closest to my reality as a researcher in the region of Latin America: the barrier to experimenting with larger models isn't just a software optimization problem, it's a hardware access problem. If I, with intent and dedicated time, struggle to run a 9B model on a consumer laptop, that's exactly why this kind of work — optimizing efficient agents with small, accessible models — matters for universities, labs, and teams in the region that don't have dedicated GPUs on hand.

What this means in practice

If you're building, or thinking about building, an agent on a local open model, here's what I'm taking away from this experiment:

Measure before you optimize. My initial intuition about latency was not the correct one, and I only found out because I measured rigorously (3 runs, warm-up, real tokens) instead of trusting a single run.
Saving tokens doesn't automatically buy you latency. Depending on model size and conversation length, the real bottleneck might be somewhere else entirely.
Context pruning has trade-offs — it's not magic. My current implementation trims by length, not semantic relevance, which means there's real risk of losing important historical information. That's a limitation I'm naming, not hiding.
A failed experiment on real hardware is data, not a failure. I couldn't run 9B on my laptop. That data point ends up being as useful to the argument of this work as any chart.

Wrap-up

This experiment started from a simple question — how do you take a tutorial-style agent and make it survive production? — and ended up giving me a more nuanced answer than I expected: context management matters, a lot, but it doesn't solve every performance problem on its own, and hardware constraints are a legitimate part of the technical conversation, not just a logistics footnote.

All the code is available in the repository for anyone who wants to reproduce or adapt it — including both the successful results with Gemma2 (2B) and the documented limitation with the 9B model, because I believe transparency about what didn't work is as valuable as what did.

If you're working with open models in the region, I'd genuinely love to hear about your experience — what hardware you're running, what you've hit, what context management strategies have worked for you. Reach out on LinkedIn.

This work was also presented as a poster at the Second South American NLP School (Buenos Aires, August 2026).

The unofficial TPU migration guide: Cloud TPU API to Compute Engine

xbill — Tue, 11 Aug 2026 19:46:01 +0000

Cloud TPU resources in Compute Engine puts it plainly:

The Cloud TPU API is no longer under active development. This includes the Google Cloud CLI for the Cloud TPU API and the Cloud Client Libraries for the Cloud TPU API. The Cloud TPU API will receive bug fixes and security updates only.

New hardware generations, starting with TPU7x (Ironwood), are supported only through Compute Engine or Google Kubernetes Engine (GKE).

No sunset date is published, so nothing breaks on a deadline. But the second sentence is the forcing function: the API you are on today is the API your next chip will not support.

So I moved a rig over — a v6e-1 (Trillium) chip serving gemma-4-E2B-it under vLLM, rebuilt on gcloud compute instances. Same chip, same checkpoint, same serving flags, only the control plane changed.

The flag mapping was the quick part. Everything after it — the quota model, a dead boot, tooling that had silently gone blind — took far longer, because almost nothing on this path fails loudly. What follows is what changes, what bit me, and how to tell one failure from another.

What actually changes

The short version, so the rest makes sense.

Old:

gcloud alpha compute tpus queued-resources create vllm-gemma4-qr \
  --node-id=vllm-gemma4-qr-node \
  --zone=us-east5-b \
  --accelerator-type=v6e-1 \
  --runtime-version=v2-alpha-tpuv6e \
  --provisioning-model=flex-start \
  --valid-until-duration=2h

New:

gcloud compute instances create gce-vllm-v6e1-2b \
  --zone=europe-west4-a \
  --machine-type=ct6e-standard-1t \
  --image-family=ubuntu-accel-2204-amd64-tpu-v5e-v5p-v6e \
  --image-project=ubuntu-os-accelerator-images \
  --maintenance-policy=TERMINATE \
  --boot-disk-size=200GB \
  --scopes=cloud-platform \
  --metadata-from-file=startup-script=/tmp/startup.sh \
  --provisioning-model=FLEX_START \
  --request-valid-for-duration=2h \
  --max-run-duration=4h \
  --instance-termination-action=DELETE

Cloud TPU API	Compute Engine
`--accelerator-type=v6e-1`	`--machine-type=ct6e-standard-1t`
`--runtime-version=v2-alpha-tpuv6e`	`--image-family=ubuntu-accel-...` + `--image-project`
`--valid-until-duration`	`--request-valid-for-duration`
`--provisioning-model=flex-start`	`--provisioning-model=FLEX_START`
QR produces a node named `<id>-node`	the instance is the node
`gcloud compute tpus tpu-vm list`	`gcloud compute instances list`
`gcloud compute tpus tpu-vm ssh`	`gcloud compute ssh`

One documentation trap before you start, and it is a trap in both directions. Request TPU Flex-start VMs states:

You must use the queued resources API to use TPU Flex-start VMs.

That is true for v5e and out of date for everything else. The page sits in the deprecated API's doc set, describes flex-start within that API, and never mentions instances create. For v5e it is still correct — there is no Compute Engine path at all, as above. For v5p, v6e and TPU7x it will send you to the API you are trying to leave.

The Compute Engine provisioning models page is the one to believe for those three. It lists the flex-start machine series as "A4, A3, A2, G4, and G2" plus "TPU7x, TPU v6e, and TPU v5p"; instances create takes FLEX_START as a first-class value; and --request-valid-for-duration is its wait knob, capped at two hours for a standalone VM. Every flex-start instance in this article was created that way.

One caveat if you are planning ahead: flex-start on TPU7x is behind an allowlist, per a footnote on that page — contact your account team. v5p and v6e are ungated.

Two genuine wins while we are here. --max-run-duration is not flex-start's alone. On the TPU API, gcloud documents the flag as "Used with flex-start"; on Compute Engine I have used it on spot creates as well, so at least those two models can carry it. Pair it with --instance-termination-action=DELETE and a demo box cleans up after itself. And the two-object lifecycle collapses: no queued resource owning a node you did not name, no reconciling the two names, and teardown needs no --force — on the old path deleting an ACTIVE resource did.

Serving does not change. Same chip, same engine build, same flags: the KV cache allocation came out at 1,151,744 tokens on both control planes — the same integer, not merely close — and throughput matched to 0.6% on the control cells. Larger cells varied by a few percent in both directions, but that benchmark swings further than that on cache state alone, so I would not read anything into it. This is not a performance decision. Plan it as a refactor.

Now the parts that cost real time.

Not every chip has a Compute Engine path

Check before you plan anything, and do not check by looking in the machine-type catalog, because it will tell you yes when the answer is no.

v5e looks fine there:

$ gcloud compute machine-types list --filter="name~ct5lp"
NAME              ZONE           CPUS  MEMORY_GB  GUEST_ACCELERATOR_TYPE
ct5lp-hightpu-1t  us-central1-a  24    48.00      ['ct5lp']
ct5lp-hightpu-4t  us-central1-a  112   192.00     ['ct5lp']
ct5lp-hightpu-8t  us-central1-a  224   384.00     ['ct5lp']

Three shapes, 26 zones. Now create one:

ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - This user agent is not allowed to use the machine type [ct5lp-hightpu-1t].

Refused outright. Not a quota error, not a does-not-exist error.

GKE uses those exact machine type names — its TPU documentation describes ct5lp-hightpu-4t and its topologies directly. That gives the catalog entry a consumer who is not you, which is my best explanation for why the strings exist without a create path, though Google does not say so outright. The same reasoning covers the other things that look like a v5e path and are not: the image family is literally named ubuntu-accel-2204-amd64-tpu-v5e-v5p-v6e, and there is a compute.googleapis.com quota metric called TPU-LITE-PODSLICE-V5-per-project-zone. Whatever the reason, Compute-Engine-shaped artifacts exist for v5e without a Compute Engine create path, so do not treat any of them as evidence of one.

The public docs agree, if you read them closely. TPU machines in the accelerator-optimized family says "Compute Engine supports the following TPU versions: TPU7x, TPU v6e, TPU v5p" and does not mention v5e anywhere. And the TPU v5e page says v5e "is supported using Google Kubernetes Engine and the Cloud TPU API", with Compute Engine absent from that list.

Catalog presence is not creatability. Testing costs nothing: pick a zone where your quota is zero and try the create. A rejection is free and conclusive.

A primer on the four provisioning models

Compute Engine gives you four ways to ask for a chip, and the choice drives everything downstream — what you pay, which quota you spend, and how you fail. Worth ten minutes up front.

	how you get capacity	max run	how it ends	quota spent	v6e, europe-west4
`STANDARD` (on-demand)	immediately, if available	unlimited	when you say so	standard	$2.97/chip-hr
`SPOT`	immediately, if available	unlimited	preempted whenever Google wants it back	preemptible → standard	$1.78/chip-hr
`FLEX_START`	queues, up to a 2h wait	10 min – 7 days	at `--max-run-duration`	preemptible → standard	$1.35/chip-hr
`RESERVATION_BOUND`	reserved ahead, if approved	up to 90 days (calendar)	when the reservation ends	managed with the reservation	no list rate

Behaviour from the provisioning models page — the reservation-bound quota cell is a simplification, since that page says it varies by reservation type. Prices read live from the Cloud Billing Catalog on 2026-08-11.

Flex-start is the default worth reaching for, and it is the cheapest. That surprised me — it undercuts spot on v6e in both regions I priced ($1.35 against $1.78 in europe-west4, $1.40 in us-east5), and it is less than half on-demand. You trade immediacy for it: the request queues rather than failing, for up to two hours, and the instance self-terminates at a duration you set. For serving experiments and benchmarks that is the right shape.

Spot is not the cheap option here, despite the name. On v6e it costs more than flex-start, and it can be reclaimed at any time — with less warning than you might assume, since the preemption notice duration defaults to zero and the shutdown period is best-effort up to 30 seconds. I checked v5e as well, expecting the ordering to invert, and it does not: in us-west4 flex-start is $0.60 against spot's $0.607. Spot's one real advantage is that it does not queue — which makes it useful as a diagnostic, see below. Read the rate rather than assuming either way.

On-demand is for when you cannot tolerate a queue or a deadline. Twice the price, no run limit, and it is the only model that spends the family quota directly.

Reservation-bound is the one to know exists rather than the one to start with. You ask for capacity at a future date; if Google approves, you get a reservation and your instances bind to it. Calendar mode runs up to 90 days. It has no list rate in the billing catalog — what it costs is whatever the reservation was priced at — so if you have a cost tool, teach it to say "read the reservation" rather than falling back to the on-demand SKU. The old path had a counterpart, for what it is worth: queued-resources create takes a --reserved flag to schedule against reserved capacity.

The practical default: flex-start for anything time-boxed, on-demand when you need it now and unbounded, reservation-bound when you have a date and a budget, spot rarely on v6e.

Quota is the gate, and it is not the quota you expect

Three separate things went wrong for me here, so take them in order.

1. Your TPU API quota does not come with you

The two control planes meter against completely disjoint pools. My project holds 512 v6e chips in us-east5 on the TPU API and, on Compute Engine, held nothing at all in the same region for the same silicon. That is why my first create failed in the zone my rig had used happily for months.

2. Which quota you spend depends on the provisioning model

There are two v6e quotas on Compute Engine, and picking the wrong one to check is the easiest mistake to make:

Provisioning model	Quota id it spends
`FLEX_START`	`PREEMPTIBLE-TPU-V6E-per-project-region`, falling back to the family quota
`SPOT`	`PREEMPTIBLE-TPU-V6E-per-project-region`
`STANDARD` (on-demand)	`TPUS-PER-TPU-FAMILY-per-project-region`, `tpu_family=CT6E`

Flex-start spends the preemptible pool. That is counterintuitive — flex-start is not preemptible in behaviour, once granted it runs uninterrupted for up to seven days — and nothing in the flag names hints at it. The provisioning models page says so, and the second sentence matters as much as the first:

When you create a Flex-start VM, preemptible quota is consumed. If your project lacks preemptible quota, then standard quota is consumed.

So for flex-start, a region is usable if either pool has room. Check the preemptible metric first, because that is what gets spent, but do not write a region off on one listing alone.

I spent a day trying to establish this experimentally before finding it documented — and then, having found it, still got it wrong by quoting only the first sentence. Read the whole entry.

Note there is no non-preemptible v6e id at all — no TPU-V6E-per-project-region exists — which is why on-demand falls back to the generic family quota. v4, v5e and v5p each publish their own dedicated pair, so this fallback is a v6e and TPU7x quirk rather than a rule.

3. The obvious command does not show either of them

This is the part that cost me the most, because it answers confidently and wrongly:

$ gcloud compute regions describe us-east5 --format="value(quotas.list())" | tr ',' '\n' | grep TPU
TPU_LITE_DEVICE_V5               0.0
PREEMPTIBLE_TPU_LITE_DEVICE_V5   0.0
TPU_LITE_PODSLICE_V5             32.0
PREEMPTIBLE_TPU_LITE_PODSLICE_V5 1536.0

Four metrics, all v5e, none of which governs v6e. The regional quota view only carries the older metrics. v6e lives in the newer Cloud Quotas API and has to be asked for by name — once per metric:

$ gcloud alpha quotas info describe PREEMPTIBLE-TPU-V6E-per-project-region \
    --service=compute.googleapis.com          # flex-start and spot

$ gcloud alpha quotas info describe TPUS-PER-TPU-FAMILY-per-project-region \
    --service=compute.googleapis.com          # on-demand

Read both, because their defaults are opposite. A region absent from the family listing inherits 0. A region absent from the preemptible listing inherits 1536. So a region that looks dead in one listing may have plenty of headroom in the other — which is exactly the mistake I made, writing off regions as unusable when only their on-demand path was.

An unset value also reads identically to a zero one, so a blank does not tell you the hardware is missing. Check machine-types list for that.

What my project actually holds

After the requests below, for the twelve regions that publish ct6e-standard-1t:

region	flex-start / spot	on-demand
europe-west4, asia-east1, asia-northeast1, asia-south1, asia-southeast1, southamerica-east1, southamerica-west1, us-south1	1536	32
us-east1	1536	32
us-central1, us-west1	1536	0
us-east5	32	32
us-east4	0	0

Two things worth reading off that.

us-central1 and us-west1 look unusable if you only check on-demand, and are in fact fine for flex-start — they hold the full 1536 on the pool flex-start actually spends. That is the opposite-defaults trap doing real damage: I wrote both off for a day.

us-east5 sits at 32 where every other live region has 1536. I put it there by asking for 32, not realising the preemptible metric defaults to 1536. When I noticed and went back to ask for 1536, that request was denied — so the 32 was not the self-inflicted ceiling it looked like. us-east5 simply is not giving out more today, whatever number you put in the form.

Troubleshooting quota and capacity

These two produce the same symptoms and have different fixes, so this is the part worth having a routine for.

The symptom: your create sits in PENDING

PENDING means either no quota or no capacity, and from the outside they are identical. I produced both separately: a flex-start create in a region with zero quota queued indefinitely, and a flex-start create in a region with 1536 chips of quota and no hardware did exactly the same. In neither case did the create report the actual problem.

It is not even consistent. In a third zone the same create came back immediately with an explicit reason: stockout rather than queueing. So you cannot infer the cause from the behaviour, and "did my create succeed" is not a quota test.

The routine

Step 1 — probe capacity with a spot create. Spot does not queue; it fails fast and names the reason, which makes it a free capacity check that takes seconds:

$ gcloud compute instances create probe --zone=us-central1-a \
    --machine-type=ct6e-standard-1t --provisioning-model=SPOT ...

reason: stockout
zonesAvailable: ''
message: The zone '.../zones/us-central1-a' does not have enough resources
  available to fulfill the request.

A stockout means your flex-start request is queued behind real scarcity and no amount of quota will help. If spot provisions instead, capacity exists — delete it and go look at quota. (Spot and flex-start draw on the same preemptible pool, so this probes the zone rather than your entitlement.)

Step 2 — try the sibling zones, not just the region. Quota is regional; capacity is zonal, and they diverge sharply. In us-central1 I got a stockout in -a, a stockout in -c, and a working instance in -b, all within a few minutes and all against the same 1536-chip regional quota. If one zone is dry, the next one in the same region costs nothing to try.

Step 3 — check both quota metrics, not one. Covered above: flex-start spends the preemptible pool first and falls back to the family quota, and the two carry opposite defaults. A region that looks dead in one listing may be fine.

Step 4 — only then request more quota. One command per metric, and the dimension keys differ — the family quota takes region and tpu_family, the preemptible one takes region alone. Read them off gcloud quotas info describe <quota-id> rather than guessing:

gcloud quotas preferences create \
  --service=compute.googleapis.com --project=YOUR_PROJECT \
  --quota-id=PREEMPTIBLE-TPU-V6E-per-project-region \
  --dimensions="region=us-east5" \
  --preferred-value=32 \
  --preference-id=preemptible-tpu-v6e-us-east5 \
  --justification="..."

Check anything you file with gcloud quotas preferences list. And know what you are likely to get.

What quota requests actually do

I filed requests on both metrics across five regions, then retried the denials — all of them once at the same size, and the four that could be lowered again at 8 chips. Every decision came back within seconds, automated, with quotaConfig.stateDetail carrying the verdict:


approved	us-east5 preemptible → 32, us-east5 family → 32, us-east1 family → 32
denied	us-central1 family, us-west1 family, us-east4 family, us-east4 preemptible, us-east5 preemptible → 1536
denied again at 8 chips	us-central1 family, us-west1 family, us-east4 family, us-east4 preemptible
refused at submission	us-central1 / us-east1 / us-west1 preemptible

Three things fall out of that.

The size of the ask is not the variable. The same 0 → 32 request was approved in two regions and denied in three. Retried at 8 chips, the denials were identical. Three sizes tested — 8, 32, 1536 — and the outcome tracked the region every time. There is no magic number.

Denials may track capacity. Of the regions that denied me quota, the two I went on to probe — us-central1 and us-west1 — both refused a spot create for lack of capacity. I did not probe us-east4, so this is a suggestive pattern across two regions rather than a rule, and the API says nothing about its reasoning. It is at least a reason not to read a denial as a judgement about your project.

You cannot ask for less than you hold. Three requests never reached review:

FAILED_PRECONDITION: The quota override ... decreases effective quota unsafely

Those regions already sat at the 1536 default and I asked for 32. Because the two metrics carry different defaults, one blanket number is wrong about half the time — read the current value per metric first.

Quota is a ceiling, not an allocation

The thing to internalise: holding quota does not mean the hardware is there. Single v6e chips were scarce in most places I looked — europe-west4-a served me first time, and everywhere else was a fight.

zone	quota held	spot create
europe-west4-a	1536	provisioned
us-central1-a	1536	`reason: stockout`
us-central1-b	1536	provisioned, then stocked out a minute later
us-central1-c	1536	`reason: stockout`
us-west1-c	1536	`reason: stockout`

Three of five zones had full quota and no chips at all. us-central1-b is the one to remember: an instance came up there, I deleted it, and a request a minute later was refused for stockout. Availability moves faster than you can test against it, let alone plan around.

So treat quota as permission to ask, not as reserved hardware. Flex-start's queue is the mechanism that actually gets you a chip, because it waits rather than failing — which is worth more here than any amount of quota on paper.

The image is not the runtime version

The first boot of my migrated rig died 100 seconds in, for a reason no flag mapping would have caught:

+ sudo docker pull vllm/vllm-tpu:nightly
sudo: docker: command not found
...
ERROR: Failed to pull vLLM Docker image after multiple retries. Exiting.

ubuntu-accel-2204-amd64-tpu-v5e-v5p-v6e has no docker on PATH at first boot. The same script had worked unchanged for months on the TPU API's v2-alpha-tpuv6e runtime, which is why I assume that image ships Docker — I have not inspected it. Either way, the script came across verbatim and went straight for the pull.

Install it first:

if ! command -v docker > /dev/null 2>&1; then
  sudo apt-get update -qq
  sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker.io
  sudo systemctl enable --now docker
fi

And fix it in three places, not one: the startup script, any Docker command your tooling runs over SSH (the recovery tool you grab after a failed boot must not fail the same way), and any copy-pasteable deploy one-liner you emit.

The general form: your startup script was written against a runtime version that gave you things for free. Mine assumed Docker. Whatever yours assumes, the instance will sit there reporting RUNNING while it fails.

RUNNING does not mean ready

This is the most misleading signal on the new path.

A queued resource reached ACTIVE only once its node was up. An instance is RUNNING the moment the VM boots — before the startup script has pulled an image, loaded a model, or done anything at all. During the entire failed boot above, the instance list said:

NAME              ZONE            MACHINE_TYPE      STATUS
gce-vllm-v6e1-2b  europe-west4-a  ct6e-standard-1t  RUNNING

It said that indefinitely. Nothing distinguishes a dead boot from a healthy one except reading the startup log or curling the port. Any readiness check you ported that trusted ACTIVE is now wrong — by several minutes on a good day, and forever on a bad one.

Discovery and SSH move, and the old calls go quiet

A ct6e-* instance is an ordinary Compute Engine instance that happens to carry a TPU, so the old API cannot see it:

$ gcloud compute instances list --filter="name=gce-vllm-v6e1-2b"
NAME              ZONE            MACHINE_TYPE      STATUS
gce-vllm-v6e1-2b  europe-west4-a  ct6e-standard-1t  RUNNING

$ gcloud compute tpus tpu-vm list --zone=europe-west4-a
$

Empty. No error, no warning — your tooling simply believes nothing is running. Two field shapes move with it: status is status: RUNNING rather than state: READY, and the external IP moves from networkEndpoints[].accessConfig.externalIp to networkInterfaces[].accessConfigs[].natIP. Copy the old status check across and it will not throw; it will just sort every healthy instance to the bottom of your ranking, which you notice the day you have two.

SSH moves too, and this is the call site people miss:

gcloud compute tpus tpu-vm ssh <node>   # old
gcloud compute ssh <instance>           # new

Everything that manages your container, tails logs, reads journalctl or runs a benchmark has to move — and those are precisely the tools you reach for when something has already gone wrong.

I had a test asserting my rig was off the old API. It covered the discovery function. Four other tools were still calling tpu-vm ssh behind its back, plus several Makefile targets. Grep for the old command; do not trust one test over one function.

Three flags that fail late

--scopes=cloud-platform — required if your startup script reads a secret. Mine pulls a Hugging Face token from Secret Manager at boot. Without the scope the VM boots fine and then spins for 30 minutes before giving up, so the symptom is a slow startup followed by what looks like a token problem.

--boot-disk-size — the image default is 10 GB, which will not hold a vLLM TPU image. Fails after a clean boot, mid-pull, which is a long way from the flag you got wrong.

--maintenance-policy=TERMINATE — required, because a TPU instance cannot live-migrate.

Two more worth knowing, both from the provisioning models page. Flex-start instances run for a minimum of 10 minutes and a maximum of seven days, so set --max-run-duration explicitly rather than discovering the boundary. And you cannot suspend one — a standalone flex-start instance can be stopped, but suspend and recreate are unavailable, and anything created through a MIG resize request cannot be stopped either. Keep state you care about on a separate disk or in GCS.

Troubleshooting quick reference

Symptom	Likely cause	Check
`PENDING` for hours	quota or capacity — identical from outside	fire a SPOT create at the same zone; `stockout` means capacity, and usually it is
`This user agent is not allowed to use the machine type`	that generation has no Compute Engine path	use the Cloud TPU API for that chip
`RUNNING` but nothing serves	startup script died	read the startup log; curl the port
`docker: command not found`	the CE image ships no Docker	install `docker.io` before pulling
Out of disk mid-pull	10 GB image default	`--boot-disk-size`
Secret access hangs 30 min	missing `--scopes=cloud-platform`	recreate with the scope
Flex-start VM disappeared	it reached `--max-run-duration`, max seven days	set the duration explicitly; it is not unlimited
`tpu-vm list` returns nothing	wrong API for a `ct6e-*` instance	`gcloud compute instances list`
SSH says not found	wrong SSH surface	`gcloud compute ssh`, not `tpus tpu-vm ssh`
Quota looks fine but nothing works	reading `regions describe`, which shows v5 metrics only	Cloud Quotas API, by metric name
`decreases effective quota unsafely`	requesting less than you hold	read the current value first
`gcloud alpha` reported missing but works	apt install, component manager disabled by design	ignore it; alpha ships in the base package

The short version

Check whether your chip has a Compute Engine path at all, by trying a create rather than reading the catalog. Check your quota through the Cloud Quotas API rather than regions describe, and read it as an intersection with machine-type availability. Translate the flags. Then rewrite discovery and SSH instead of filtering them, and grep for the old commands afterwards.

And assume nothing fails loudly. A stuck request, a dead boot, a blind discovery helper and a missing SSH surface all present as silence or as a cheerful RUNNING. The flag mapping is the part gcloud checks for you; everything in this article is the part it does not.

There is nothing as constant as change. TPU7x is already Compute Engine only, so this will not be the last migration any of us does — but the next one should be cheaper, because the hard part was never the flags.

Gubernator v2.13.0: Google SRE SLOs, Native CoreDNS Suite & Caddy Ingress for Docker Compose

Mario Ezquerro — Tue, 11 Aug 2026 06:14:53 +0000

If you love the simplicity of Docker Swarm (native Compose files, lightweight single binary) but miss the advanced capabilities of Kubernetes (targeted label placement, SRE-grade observability, built-in DNS service discovery, and zero-trust ingress), meet Gubernator (gbnt).

We are excited to release Gubernator v2.13.0, introducing three massive feature suites natively integrated into a single binary and a modern Material Design 3 Flutter Web Dashboard:

Google SRE Multi-Burn-Rate SLO Engine & Interactive Suite
CoreDNS 4-Tab Management Suite & Interactive Dig Playground
Caddy Ingress & Zero-Trust Reverse Proxy Suite

Fun Fact: The entirety of Gubernator's codebase, multi-node deployment pipelines, and SRE features were designed, built, and pair-programmed using **Google Antigravity (AGY), Google DeepMind's agentic AI coding assistant!

Let's dive into what's new and how you can level up your self-hosted or production container clusters!

1. Google SRE Multi-Burn-Rate SLO Engine & Web Suite

Defining Service Level Objectives (SLOs) and tracking Error Budgets is the gold standard of Site Reliability Engineering. Until now, implementing SLOs meant running heavy Kubernetes CRDs (via tools like Sloth or Pyrra) or using costly SaaS platforms.

Gubernator v2.13.0 brings Google SRE Workbook (Chapter 5) compliant multi-burn-rate alerting straight to simple docker-compose.yml services:

version: "3.8"
services:
  payment-api:
    image: hashicorp/http-echo:latest
    labels:
      gbnt.slo.enable: "true"
      gbnt.slo.target: "99.9"
      gbnt.slo.window: "30d"
      gbnt.slo.template: "caddy-http"
      gbnt.slo.journey: "Checkout Flow"

What makes Gubernator's SLO Suite unique?

Google Multi-Burn-Rate Alerting: Automatically generates standard 4-window Prometheus recording and alert rules (Critical Page 1h/6h & Warning Ticket 3d/14d).
Dynamic "No-Code" Management: Click "+ Configure / Add SLO" in the Web UI or call POST /v1/slo/edit to create, edit, or disable SLOs on the fly without editing Compose files.
Composite User Journeys: Group multi-service SLOs into end-to-end flows (Checkout Flow: API Gateway + Payment + DB) and automatically identify the weakest-link bottleneck service.
Deployment Correlation Timeline: Cross-reference real-time burn rate spikes against stack updates and container restarts to answer "Did our last deploy burn the budget?".
PromQL Dry-Run Backtesting: Validate Compose YAML syntax and test PromQL queries against historical Prometheus metrics prior to deployment.
Automated Grafana Dashboards: Automatically generates /data/monitor/grafana/dashboards/slo_dashboard.json on rule sync.

SLO Ecosystem Comparison Matrix

Here is how Gubernator compares to other popular open-source and commercial SLO tools:

Feature / Capability	Gubernator	Sloth (`slok/sloth`)	Pyrra (`pyrra-dev/pyrra`)	OpenSLO / Nobl9
Native Runtime Environment	Docker Compose / Swarm / Bare Metal	Kubernetes / OpenSLO CLI	Kubernetes CRDs / Filesystem	Multi-Cloud / SaaS
Declarative Spec	`docker-compose.yml` labels (`gbnt.slo.*`)	K8s CRDs / Sloth YAML	Custom Resources (`ServiceLevelObjective`)	OpenSLO YAML Spec
SRE Calculation Engine	Google SRE Multi-Burn-Rate (via Sloth Engine)	Google SRE Multi-Burn-Rate (4 windows)	Prometheus Multi-Burn-Rate	Proprietary / Custom
Integrated Web Dashboard	Yes (Flutter Web 5-Tab Suite)	No (CLI / Operator only)	Yes (React/Go UI)	Yes (SaaS Console)
Dynamic Hot-Editing	Yes (Web UI Modal & REST API)	No (Requires re-applying YAMLs)	No (Read-only from K8s/Files)	Yes (SaaS Console)
User Journeys (Composite SLOs)	Yes (Aggregation & Bottleneck Analysis)	No	No	Yes (Related Services)
Deployment Correlation	Yes (Real-time Timeline of Stacks/Restarts)	No	No	Partial (CI/CD Webhooks)
Built-in SLI Templates	Yes (`caddy-http`, `http-status`, `latency-p99`, `grpc`)	Partial (Sloth Libraries)	No (Raw PromQL)	No
Dry-Run PromQL Backtesting	Yes (Pre-deploy Validation)	Partial (`validate` command)	No	No
RED Metrics Breakdown	Yes (RPS, Error Rate, P99 Latency Cards)	No	Partial (RPS & Errors)	Yes
Automated Grafana Provisioning	Yes (Auto-generates `slo_dashboard.json`)	Partial (Generic Rules)	No	No

2. Native CoreDNS 4-Tab Suite & Interactive Dig Playground

Internal container service discovery should "just work." In Gubernator, every deployed container automatically receives --dns <CoreDNS_IP>, enabling seamless *.gbnt internal resolution across multi-node clusters.

With v2.13.0, we are expanding CoreDNS into a full 4-Tab Management Suite:

+-------------------------------------------------------------------------+
|                      GUBERNATOR COREDNS SUITE                           |
|                                                                         |
|  [Tab 1: Auto-Discovered] [Tab 2: Custom Records] [Tab 3: DNS Playground] [Tab 4: Config]
+-------------------------------------------------------------------------+

Key Features:

Tab 1: Auto-Discovered Stacks (*.gbnt): Real-time table mapping running containers to <service>.<stack>.gbnt with copyable curl commands.
Tab 2: Custom Static DNS Records: Manage custom A, AAAA, CNAME, TXT, and PTR records stored in SQLite and merged into CoreDNS on the fly (POST /v1/coredns/custom-records).
Tab 3: Interactive Dig Playground: A built-in terminal console to run DNS queries against local CoreDNS (127.0.0.1:5354), benchmark query latency in milliseconds, and inspect raw nslookup output.
Tab 4: Upstream Forwarders & Corefile Editor: One-click upstream DNS presets (Cloudflare 1.1.1.1, Google 8.8.8.8, Quad9 9.9.9.9) and a live Corefile editor with container reload.

3. Caddy Ingress Suite & Zero-Trust Reverse Proxy

Gubernator packages Caddy as its default edge proxy, handling HTTPS certificate provisioning, reverse proxying, and access logging across multi-node setups.

Features in the Caddy Suite:

7-Tab Management Visualizer: Dashboard, Dynamic Routes Matrix, Corefile Preview, TLS Certs Inspector, Real-time Access Logs, Log Config, and Prometheus Metrics.
Root CA Trust Installation: One-click download of Gubernator's internal Root CA certificate for local TLS trust across your developer devices.
Automatic Ingress Label Routing: Simply add ingress.host=my-app.example.com to your Compose service, and Gubernator reconfigures Caddy route matrices across all cluster nodes automatically.

Getting Started in Under 60 Seconds

You can spin up a complete Gubernator cluster with full observability, CoreDNS, Caddy, and Prometheus/Grafana in seconds:

# 1. Download binary & start Gubernator Manager
curl -sSL https://raw.githubusercontent.com/mario-ezquerro/gubernator/main/install.sh | bash
gbnt serve

# 2. Deploy the SRE Monitoring Stack (Prometheus, Grafana, Loki, cAdvisor, Jaeger)
gbnt monitor init

3. Access Web Dashboards

Web UI Dashboard -> http://localhost:4001

Grafana -> http://localhost:3000

CoreDNS Playground -> http://localhost:4001 (CoreDNS tab)




---

## Built Autonomous with Google Antigravity

A special shoutout to **Google Antigravity (AGY)**! The entire architecture of Gubernator -- from Go backend REST APIs, SQLite ORMs, Caddy route management, CoreDNS hosts sync, Sloth SLO rule compilation, down to the 5-tab Flutter Web UI -- was built autonomously in pair-programming sessions with Google Antigravity AI.

---

## Conclusion & Open Source

Gubernator aims to make container orchestration **fast, resilient, and enjoyable** again -- without the steep operational overhead of Kubernetes.

- **GitHub Repository**: [mario-ezquerro/gubernator](https://github.com/mario-ezquerro/gubernator)
- **Documentation & Guides**: [https://mario-ezquerro.github.io/gubernator/](https://mario-ezquerro.github.io/gubernator/)
- **Give us a Star**: If you find Gubernator useful, drop a star on GitHub!

*What are your thoughts on native SLO tracking for Docker Compose? Let us know in the comments below!*

Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't

xbill — Tue, 11 Aug 2026 00:25:31 +0000

Serving Gemma 4 E2B on a TPU v6e-1

A Cloud TPU v6e-1 (Trillium) costs 2.25× a v5e-1 and returns 1.62–1.68× the throughput on workloads that fit in a v5e, and 2.32–2.77× on workloads that do not. Per output token that makes v6e 34–39% dearer in the first regime and 3–19% cheaper in the second — so the case for the bigger chip is narrower than the memory ratio suggests, and break-even sits at roughly 270,000 KV tokens.

v6e is not a general upgrade over v5e. It is a memory upgrade sold at a compute price: 32 GB against 16, a KV pool of 1,151,744 tokens against 321,376 (3.6×), for 1.907× the bandwidth. Where the extra memory does nothing, the workload pays 2.25× for 1.6×.

Two findings drive the rest:

There is no capacity knee at any occupancy tested. TTFT = −8542 + 265 × concurrency, R² = 0.999996, across 56% to 157% of the KV pool, with num_preemptions_total = 0 in every cell. A line fitted entirely below 100% occupancy predicts 157% to within 0.13%.
Spot is more expensive than flex-start on this chip — $1.4033 against $1.35/chip-hr in us-east5, reversing the v5e ordering. The cheaper option is also the preemption-free one that stops billing by itself.

Configuration: v6e-1 (ct6e-standard-1t, one Trillium chip), vllm/vllm-tpu:nightly, vLLM 0.26.1rc1.dev256+gf5bb701fa, tpu-inference JAX backend, google/gemma-4-E2B-it at bf16, TP=1, max_model_len 32768, max_num_batched_tokens 4096, kv_cache_dtype=auto, prefix caching on. OUTPUT_LEN 128 throughout. v5e-1 comparison figures are from the same model and engine family on v5litepod-1 and are not a controlled A/B — read them as shape, not delta.

Part 1 — Getting a v6e-1 at all

1.1 The gcloud spelling table

On v5e, v5e is spelled v5litepod to gcloud. On v6e the marketing name and the CLI value coincide — which teaches a habit that breaks on the next chip.

Context	v5e single chip	v6e single chip
Prose, directory names	`v5e-1` / `v5e1`	`v6e-1` / `v6e1`
`--accelerator-type`	`v5litepod-1`	`v6e-1`
Flex-start runtime version	`v2-alpha-tpuv5-lite`	`v2-alpha-tpuv6e`
`--type` / `--topology`	`v5litepod` / `1x1`	`v6e` / `1x1`
TPU API quota id	`TPUV5sLitepodPerProjectPerZoneForTPUAPI`	`TPUV6EPerProjectPerZoneForTPUAPI`
Spot quota id	`TPUV5sPreemptibleLitepodPerProjectPerZoneForTPUAPI`	`TPUV6EPreemptiblePerProjectPerZoneForTPUAPI`

Nothing in that table survives a retarget by analogy. The v6e quota ids drop the Litepod the v5e ids carry, and a stale quota id fails quietly — it matches no rows rather than erroring, producing a confident "no quota anywhere" that is a typo. v6e1, the directory spelling without the hyphen, is still not a valid gcloud value even though v6e is.

1.2 Provisioning clears three independent gates

A creation must pass three separate checks. They fail differently, and the one that is easiest to query carries the least information.

Gate 1 — does the zone have v6e-1 hardware? Of 37 zones reporting quota, only 18 offer the accelerator type:

gcloud compute tpus accelerator-types list --filter="type=v6e-1"

Google's regions-and-zones page names 8, a strict subset of what the API accepts. Read the API. This gate is provisioning-model-independent.

Gate 2 — does that zone offer that provisioning model for that accelerator type? Independent of both quota and hardware, and where a published price stops meaning anything. us-central1-b and us-south1-a have v6e-1 hardware, quota, and a published DWS Defined Duration V6e rate for their region, and both reject flex-start at the API:

FLEX_START provisioning model is not supported for accelerator type "v6e-1" in location "us-central1-b"

Confirmed accepting flex-start: us-east5-a, us-east5-b, europe-west4-a. This is the v6e analogue of the v5e result, where flex-start v5litepod-1 was accepted in exactly one zone out of 44. Note that europe-west4 inverts across generations: it rejected v5litepod-1 while quoting a rate for it, and accepts v6e-1.

Gate 3 — is there free capacity right now? Reachable only after the first two pass, and the one gate that is not a property of the zone. Requests in accepting zones sit at WAITING_FOR_RESOURCES for tens of minutes to hours before capacity is granted. That state is not a failure — it should not be recorded as one, and the request should not be torn down, because flex-start capacity can take up to two hours to come back once dropped.

A published rate is not an offer of capacity, and not even an offer of the provisioning model.
Quota is the first thing most people check and the last thing that should reassure them.

1.3 Provision

# Flex-start, via the Queued Resource API — the only model that accepts --max-run-duration,
# i.e. the only one that stops billing on its own.
gcloud alpha compute tpus queued-resources create gemma4-v6e \
  --node-id=gemma4-v6e-node --zone=us-east5-b \
  --accelerator-type=v6e-1 --runtime-version=v2-alpha-tpuv6e \
  --provisioning-model=flex-start --max-run-duration=4h

# Spot — preemptible with ~30s notice, no run limit, and on this chip not the cheap option.
gcloud alpha compute tpus tpu-vm create gemma4-v6e \
  --zone=us-east5-b --type=v6e --topology=1x1 \
  --provisioning-model=spot --version=v2-alpha-tpuv6e

--max-run-duration is flex-start-only. --valid-until-duration bounds the request, not the run, so it is shared by all three models. Spot and on-demand nodes bill until preempted or deleted.

The Hugging Face token belongs in Secret Manager, not in the startup script — the rendered script is uploaded as instance metadata, and anything baked into it is readable from the instance:

printf '%s' "hf_xxxxxxxxxxxx" | gcloud secrets create hf-token --data-file=- --project=YOUR_PROJECT

1.4 Serve

sudo docker run -d --name vllm-gemma4 --privileged --net=host \
  -v /dev/shm:/dev/shm --shm-size 10gb \
  -v ~/.cache/vllm:/root/.cache/vllm \
  -e HF_HOME=/dev/shm -e HF_TOKEN="$(cat ~/.hf_token)" \
  vllm/vllm-tpu:nightly \
  vllm serve google/gemma-4-E2B-it \
    --dtype bfloat16 \
    --kv-cache-dtype auto \
    --max-model-len 32768 \
    --max-num-batched-tokens 4096 \
    --tensor-parallel-size 1 \
    --enable-prefix-caching \
    --disable-chunked-mm-input \
    --limit-mm-per-prompt '{"image":4,"audio":1}' \
    --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4

Two deliberate differences from the equivalent v5e configuration:

--gpu-memory-utilization is absent. On v5e, 0.92 was a ceiling: 0.95 died after a 691-second compile while loading jit_structured_decode_fn, because compiled XLA programs live outside the knob's control. That is a 16 GB result with no bearing on 32 GB, and v6e's ceiling is unmeasured, so the flag is left underived rather than carrying another chip's limit forward.
--max-model-len is 32768, not the 16384 typical of v5e-era configs. §2.3 shows it is free.

Verify against the boot log, not against the flags:

sudo docker logs -f vllm-gemma4 2>&1 | grep -E "Memory statistics|GPU KV cache size|block_size"

Memory statistics | total_hbm_limit_gb=31.24GiB | total_hbm_limit_cap_gb=28.74GiB
                  | total_hbm_used_gb=8.97GiB   | total_hbm_avail_gb=19.77GiB
GPU KV cache size: 1,151,744 tokens

Smoke-test on /v1/chat/completions. Raw /v1/completions returns an empty string on -it models, which looks exactly like a broken deploy and is not.

Part 2 — The chip

2.1 On paper

Spec (per chip)	v5e	v6e (Trillium)	Ratio
HBM capacity	16 GB	32 GB	2.0×
HBM bandwidth	800 GiBps	1,638 GBps	1.907× — units differ
Peak bf16	197 TFLOPs	918 TFLOPs	4.66×
Peak Int8	393 TOPs	1,836 TOPs	4.67×
TensorCores	1 (4 MXUs, 128×128)	1 (MXUs 256×256, count unresolved)
ICI	400 GBps bidi, 4 ports	800 GBps bidi, 4 ports	2.0×
Machine type	`ct5lp-hightpu-1t`	`ct6e-standard-1t`
On-demand list	~$1.20/chip-hr	~$2.70/chip-hr	2.25×

The units trap costs 7%. Google quotes v5e HBM bandwidth in GiBps and v6e in GBps — on the v5e page, in the same table that uses GBps for ICI. Normalised, 800 GiBps = 858.99 GB/s, making the true ratio 1.907× rather than the 2.047× obtained by dividing the printed figures. The launch-blog claim that Trillium "doubled" HBM bandwidth is the naive reading. For bandwidth-bound work — decode is bandwidth-bound — that 7% separates a ratio that explains the measurement from one that does not.

The shape trap: 2.25× the price for 2× memory, ~1.9× bandwidth, and 4.7× the raw FLOPS. The 4.7× only pays for prefill-heavy or long-context work that burns the matrix units. For pure decode, v5e is priced close to right and v6e is not.

One row not to build on: Google's v6e page states each TensorCore has 2 MXUs, but two 256×256 arrays is exactly 2× v5e's four 128×128, against a published peak of 4.66× — which would require a 2.33× clock increase on top. Four 256×256 closes it almost exactly (262,144 MACs × 2 flops × 1.75 GHz = 917.5 TFLOPs against a published 918). The 918 figure is sound, cross-checking against the same page's 234.9 PFLOPs-per-Pod row. Treat peak compute as reliable and the MXU count as unresolved.

2.2 The memory budget

	v5e-1	v6e-1
Total HBM visible	15.75 GiB	31.24 GiB
Allocation cap	14.49 GiB (at 0.92)	28.74 GiB
E2B weights, resident	8.97 GiB	8.97 GiB
KV cache pool	5.52 GiB	19.77 GiB
KV tokens	321,376	1,151,744

3.58× the KV capacity for 2.25× the price. The weights are identical — E2B costs 8.97 GiB wherever it runs, consuming 62% of a v5e's usable budget and 31% of a v6e's. Every byte of the difference goes to KV.

The arithmetic closes independently on both chips, which is what distinguishes a real allocation from a log line: 19.77 GiB ÷ 18,432 B/token = 1,151,686, within 0.005% of the measured 1,151,744. The same division reproduces the v5e figure to 0.06%.

2.3 Longer context is free

`max_model_len`	`block_size`	KV pool
16,384	32	1,151,776 tokens
32,768	64	1,151,744 tokens
65,536	128	1,151,744 tokens

A 0.003% spread across a 4× range. The Pallas backend derives block_size to hold blocks-per-request constant at 512, so doubling the context doubles the page size, halves the block count, and arrives at the same token capacity. The same behaviour holds on v5e at a quarter of the pool.

Never set --block-size. Pinning it fights the derivation that keeps long context free.

2.4 Data types

format	native in the MXU?	v5e	v6e
bf16	✅	baseline	baseline
int8	✅ 2× bf16	the only compute win	still the only compute win
fp8	❌	storage/bandwidth only	❌ — unchanged
int4 / fp4	❌	footprint only	footprint only

Trillium does not bring fp8. Google's v6e page publishes exactly three peak-compute rows — bf16: 918 TFLOPs, Int8: 1836 TOPs, and a per-Pod bf16 figure — and no fp8 row anywhere. The int8 figure being exactly 2× bf16 is the signature of a native MXU path; the absent fp8 row is the tell in the other direction. v7/Ironwood is the first TPU with fp8 in the matrix units — no conclusion in this section carries forward to it.

The consequence is observable at boot. With --kv-cache-dtype auto — the flag never passed — the engine logs this twenty times:

Automatically using fp8_e5m2 for FP8 KV cache on TPU v6e

…and allocates regular_attn_dtype=bfloat16. The arithmetic is not close: 1,151,744 tokens against 19.77 GiB is the bf16 model to 0.01%, while the fp8 model is 50% off. The two hypotheses are far apart, making this a discriminator rather than a tolerance argument.

That is the sixth false fp8 signal recorded on this stack and the first on a second silicon generation. On v5e, --kv-cache-dtype fp8_e4m3 was accepted at the CLI, echoed in non-default args, praised by a log line, reported in /metrics, and allocated a genuinely float8_e4m3fn tensor — five independent signals of success — for a 1.000× capacity ratio, because the KV block layout is word-aligned: as the element width halves, the shape goes (32,1,2,256) → (32,1,4,256) and the byte count never moves.

Verify quantization from the boot allocation arithmetic — never from the flag being accepted, and
never from engine prose.

What v6e plausibly does unblock: on v5e, qwix int8 weight quantization died at RESOURCE_EXHAUSTED: HLO temporaries (16.23G) exceeds available HBM (15.75G), short by 0.48 G. v6e has 31.24 GiB, roughly 15 GiB of headroom over that same temporary peak. That failure was an HBM ceiling and this chip doubles it. Untested, and it fails fast — 2.5–4 minutes if it still does not boot.

Part 3 — Six properties of Gemma 4 E2B that decide the rest

None of these change with the chip, but their consequences land differently on 32 GB.

field	value
`num_hidden_layers`	35 (28 sliding / 7 full, `i % 5 == 4` is full)
`num_kv_shared_layers`	20 — only 15 layers own a cache
`num_attention_heads` / `num_key_value_heads`	8 / 1
`head_dim` / `global_head_dim`	256 / 512
`hidden_size` / `intermediate_size`	1536 / 6144
`vocab_size`	262,144 (tied embeddings)
`sliding_window`	512
resident at bf16	8.97 GiB

1. "E2B" is not a 2B model. ~2B effective against ~5B total, landing at 8.97 GiB resident. The E prefix is load-bearing: reading E4B as "a 4B model" understates its weights by roughly 2×, exactly the difference between fitting a 16 GB chip and not. On v6e this matters less for E2B than for what else becomes possible — E4B fits at bf16 here and does not on v5e.

2. There are two attention geometries. Sliding layers run at head_dim 256; the seven full-attention layers run at 512, applying to K and V, not just Q. Reading a single head_dim and applying it to all 35 layers under-counts the full layers by 2× — a 17% KV sizing error. It is also the root cause of the 2.9× capacity tax in Part 5.

3. Twenty of the thirty-five layers read another layer's cache. first_shared = 35 − 20 = 15, so layers 0–14 own KV and 15–34 share. The rule is "last preceding layer of the same attention type", and within 0–14 that means all twenty shared layers resolve to two source caches — layer 13 for the sliding ones, layer 14 for the full ones.

4. KV costs 18 KiB/token, and the boot log misreports why.

12 sliding cached layers × 1 KV head × 2 (K,V) × 256 × 2 B = 12,288 B
 3 full    cached layers × 1 KV head × 2 (K,V) × 512 × 2 B =  6,144 B
                                                    total  = 18,432 B = 18 KiB/token

The line describing the cache, regular_attn_shape=(num_blocks, (64, 1, 2, 256)), is a first-wins sample taken from layer 0, which is sliding, hence 256. It says nothing about layers 4, 9 and 14: count increments for all 15 tensors while shape is written once and never updated. The allocation is correct; the line is misleading. Size KV from the config geometry and check it against total_hbm_avail_gb.

5. One KV head means more chips make things worse. num_key_value_heads = 1 is full MQA, and a single head cannot be sharded — runtimes pad num_kv_heads up to a multiple of the TP size, so at TP=4 the same head is replicated at 4× the KV memory. A larger topology multiplies this model's KV cost rather than dividing it. The answer to "E2B needs more memory" is a bigger chip, not more chips.

6. The heads do not tile the hidden size. 8 × 256 = 2048 against hidden_size = 1536, so the Q projection is rectangular. Code computing head_dim = hidden_size / num_heads gets 192 and is silently wrong.

One further property explains performance rather than memory: 4.38 GiB of the 8.97 GiB resident is per-layer embedding tables (262,144 × 256 × 35), which are gathered per token, not streamed. Only ~3.15 GiB moves per decode step, which is why an 8.97 GiB model decodes as fast as it does, and it sets the bandwidth floor used in Part 4.

Part 4 — Results

Roles are sized against the v6e pool: control fits both chips trivially, bandwidth fits both but moves substantial KV per step, v6e_only exceeds v5e's entire pool, and long_ctx requires max_model_len > 16384 — impossible on v5e at any setting.

ctx	clients	role	KV needed	v5e tok/s	v6e tok/s	ratio	per-stream	median TTFT
128	1	control	256	123.3	202.9	1.65×	202.9	12 ms
128	8	control	2,048	738.3	1,195.1	1.62×	149.4	26 ms
1,024	16	control	18,432	896.1	1,508.0	1.68×	94.2	149 ms
4,096	64	bandwidth	270,336	585.9	1,360.0	2.32×	21.3	303 ms
8,192	32	bandwidth	266,240	307.8	758.4	2.46×	23.7	348 ms
8,192	64	v6e_only	532,480	314.4	870.0	2.77×	13.6	1,006 ms
16,000	32	v6e_only	516,096	166.8	432.6	2.59×	13.5	3,276 ms
16,000	64	v6e_only	1,032,192	166.7	446.0	2.68×	7.0	8,459 ms
32,000	16	long_ctx	514,048	—	242.6	—	15.2	2,233 ms
32,000	32	long_ctx	1,028,096	—	229.0	—	7.2	8,760 ms

There is correctly no v5e reference for the long_ctx cells: that configuration cannot exist on v5e.

A caution about the 4,096–8,192 band. Those three cells are the most sensitive in the matrix to how a sweep is ordered. Run at a shared --seed after a longer-context cell, they report 12–19% higher than they do with a distinct seed per cell, while every cell at 128, 1,024, 16,000 and 32,000 tokens moves by under 6.3% either way. The figures above are the clean-seed ones. Anything quoting this band from a single-seed sweep is quoting the high side.

4.1 The asymmetry

regime	cells	mean vs v5e
working set < 10% of v5e's pool	3	1.65×
working set ≥ 83% of v5e's pool	5	2.56×

Against a 1.907× bandwidth ratio and a 2.25× price ratio.

If v6e were simply faster, every cell would improve by roughly the same factor. Control cells move roughly with bandwidth and no more. Cells where v5e was over its pool — evicting and recomputing — move about 2.6×, because v6e is not doing that work. On decode throughput alone this chip is a poor deal. It pays for capacity, not speed — and note that even the memory-bound mean of 2.56× only just clears the 2.25× price ratio.

Single stream is the cleanest bandwidth read: TPOT 4.72 ms on v6e against 8.02 ms on v5e, 1.70× on a 1.907× bandwidth ratio. Decode moves ~3.15 GiB per step (derived from layer geometry), which at 1,638 GB/s is a 2.06 ms floor against 4.72 ms measured — 44% of peak, slightly worse utilisation than v5e's 49%. Roughly 2× of headroom sits in fixed per-step cost on both chips, not in memory bandwidth.

The 1.65× control figure sits below the bandwidth ratio and is unexplained. The MXU geometry change (4×128×128 → 256×256, a 4× larger minimum tile) is a plausible cause but is not demonstrated; separating it from "small batches do not saturate bandwidth" requires a batch-size sweep at fixed short context.

4.2 There is no capacity knee

A widely used v5e rule of thumb — keep clients × context under ~78% of the pool — scales to ~900,000 tokens on v6e. Tested directly at fixed 16,000 context with only concurrency varying:

clients	KV needed	% of pool	tok/s	median TTFT	preemptions
40	645,120	56%	452.7	2,088 ms	—
46	741,888	64%	465.9	3,659 ms	—
52	838,656	73%	470.6	5,273 ms	—
56	903,168	78%	467.7	6,309 ms	—
60	967,680	84%	471.6	7,373 ms	—
64	1,032,192	90%	446.0	8,459 ms	0
72	1,161,216	101%	458.5	10,544 ms	0
80	1,290,240	112%	459.1	12,689 ms	0
96	1,548,288	134%	472.0	16,937 ms	0
112	1,806,336	157%	465.5	21,229 ms	0

TTFT = −8542 + 265 × concurrency, R² = 0.999996 over all ten points. Throughput is flat at 446.0–472.0 tok/s (5.8%) across the range. TPOT sits at 66.2–67.4 ms and does not move. num_preemptions_total is 0 in every cell, including 157% occupancy.

The line is the most reproducible result in the matrix. Re-measured on a separate node, the 90% and 157% points land at +0.18% and +0.22% of what it predicts, with zero preemptions in both.

There is no knee at 78%, none anywhere in 56–157%, and crossing 100% of the pool is not an event. A line fitted entirely below 100% predicts the 157% point to within 0.13%.

The scheduler admission-controls rather than evicts. It admits what fits and queues the rest, so the working set never thrashes. Occupancy alone costs nothing; eviction would, and it never engaged. The v5e rule was a queueing curve read as a memory cliff.

Size by the latency you will accept, not by pool occupancy. Throughput saturates near
concurrency 40, and every further concurrent request buys 265 ms of TTFT and nothing else. The
pool bounds what is resident; it is not a performance threshold.

4.3 Two benchmarking traps on this stack

1. A config can silently fail to take effect, and the obvious check can miss it. vLLM prints max_model_len inside a dict repr ('max_model_len': 16384), so a regex written for the bare key matches nothing and returns no result rather than a mismatch — indistinguishable from a pass unless the check distinguishes them. The reliable signal is a different derived quantity downstream of the same setting: block_size reads 32 at 16384 and 64 at 32768. Verify a setting through two independent derivations.

2. --seed 0 plus prefix caching silently couples cells. vLLM defaults to enable_prefix_caching=True, and the random dataset is deterministic in the seed, so two cells at the same input_len draw overlapping prompts and the later one is served from cache. A 16000×32 cell run after 16000×64 took 1,539,904 cache-hit tokens against 1,536,000 input tokens — essentially all of it — and its 2,461 ms TTFT is not comparable to anything.

That artifact, combined with an unsampled gap between 46% and 89% of pool, is enough to manufacture an apparent 3.4× cliff: two points far apart with the lower one artificially fast reads as a cliff. Vary the seed per cell; the knee and overflow sweeps above measured 0.0% prefix hits throughout.

Part 5 — Why there is no cliff, and what it costs

E2B caches 15 of 35 layers: 12 sliding-window at head_dim 256 with a 512-token window, 3 full-attention at 512. tpu_inference sees two head dims, sets disable_sliding_window, and gives every layer full-length blocks — num_kv_cache_groups=1, 15 tensors, confirmed in the boot log.

Allocation and reads therefore diverge:

	12 sliding layers	3 full layers
allocated	full context — 12,288 B/token	6,144 B/token
read per decode step	6.29 MB, constant once L > 512	98.3 MB at L = 16,000

At 16,000 context, 94% of the KV bytes read come from 3 of the 15 cached layers. The pool is charged ~18 KiB/token where ~6.4 KiB would suffice.

This does not prevent a cliff — it moves the cliff closer. It is a capacity tax, not a latency shield; no cliff appears because nothing is ever evicted.

The tax is worth 2.91×: at 32,768 context, 576.0 MiB/seq allocated against 198.0 MiB/seq windowed. The trigger is disable_sliding_window = len(head_size_set) > 1, and Gemma 4's 256/512 head-dim split trips it family-wide, at every size. It is gated on an upstream TODO and is not settable from the serving side.

That single upstream fix is worth more than every available quantization flag combined — 2.91× the
effective KV capacity at zero quality cost.

Part 6 — Cost

Rates from the Cloud Billing Catalog, per chip-hour:

model	v5e (us-west4)	v6e (us-east5)	ratio
Spot	$0.5779	$1.4033	2.43×
Flex-start (DWS)	$0.6000	$1.3500	2.25×
On-demand	$1.2000	$2.7000	2.25×

On v6e, spot is dearer than flex-start, inverting the v5e ordering and the advice with it. On v5e, flex-start cost 3.8% more and bought preemption-freedom. On v6e flex-start is both cheaper and preemption-free, and it self-terminates via --max-run-duration, which the other two do not. There is no trade left to make.

Two catalog naming traps: flex-start is sold as "DWS Defined Duration" (Dynamic Workload Scheduler) and drops the Tpu prefix (DWS Defined Duration V6e), while spot is usageType: Preemptible spelled TpuV6e attached to Spot Preemptible VMs. The Reserved …, Commitment v1: … and Capacity Optimized TpuV6e … SKUs describe the same chip in the same region and three are also OnDemand, so anchor the match patterns.

Cost per million output tokens, flex-start both sides

ctx	clients	KV needed vs v5e pool	v5e @ $0.60	v6e @ $1.35	v6e vs v5e
128	1	0.1%	$1.352	$1.848	1.37× dearer
128	8	0.6%	$0.226	$0.314	1.39× dearer
1,024	16	6%	$0.186	$0.249	1.34× dearer
4,096	64	84%	$0.284	$0.276	0.97× — break-even
8,192	32	83%	$0.542	$0.494	0.91× — cheaper
16,000	32	161%	$0.999	$0.867	0.87× — cheaper
16,000	64	321%	$1.000	$0.841	0.84× — cheaper
8,192	64	166%	$0.530	$0.431	0.81× — cheaper
32,000	16	—	—	$1.545	v6e only

One chip 24/7: $438/month on v5e flex-start, $986/month on v6e flex-start.

Choosing

The rule follows from the table, and it is a memory question rather than a throughput one:

Determine whether the steady-state working set — clients × context — exceeds ~270,000 tokens,
roughly 84% of a v5e's pool. Below that, v5e is 26–29% cheaper per token. Above it, v6e is cheaper,
but by 3% at the boundary and never by more than 19%, because v5e can no longer hold the job and
starts recomputing it.

Two things follow that the memory ratio alone would not predict. The advantage saturates: once past the boundary, v6e settles at 0.81–0.87× and does not keep improving with working-set size — 16,000 × 64 runs at 321% of a v5e pool and is still only 16% cheaper. And break-even is not a comfortable margin. At 84% of pool the two chips are within 3% of each other, which is inside the run-to-run spread of the band those cells sit in.

This is not "v6e for long context". The 4096 × 64 cell is only 4K of context and reaches the boundary anyway, because 64 clients × 4,096 tokens is 270,336 KV tokens. Concurrency crosses the line as readily as context does. The threshold is a product, which is why the single-stream cell is v6e's worst showing at 1.37× dearer.

One effect dominates both columns: moving from a single stream to the best-measured concurrent cell is a 7.4× cost reduction per token on v6e ($1.848/1M at 128 ctx × 1 client → $0.249/1M at 1,024 ctx × 16). That is larger than the 2.25× between the two chips and larger than any other effect here. Tune concurrency before shopping for chips.

Limits

Nothing above 157% of pool, and nothing that forces preemption. Every cell was admission-controlled; the eviction regime — the condition that would actually produce a cliff — is untested.
The knee sweep varied concurrency at fixed context only. A context sweep at fixed concurrency loads the 3 full-attention layers differently and is not covered.
The 1.65× control figure is unexplained, sitting below the bandwidth ratio.
Single run per cell, on two independent nodes. Every cell in the results table was measured twice, on separate v6e-1 nodes in different zones. Seven of ten agreed to within 6.3%; the three in the 4,096–8,192 band diverged 12–19% and are reported at the clean-seed value. No within-node variance figure is established, so a difference under ~6% between any two cells here is not a result.
Cold-boot and compile timings are unmeasured on v6e. The v5e figures — 857 s cold, 685 s of it compilation, 497 s with the compile cache mounted — describe a mechanism that carries, not timings that do. v6e compiles its own kernels.
--gpu-memory-utilization has no established v6e ceiling.
qwix int8 weights, max_model_len 65536, max-num-batched-tokens 8192, VLLM_TPU_BUCKET_PADDING_GAP=128, and n-gram speculative decoding are untried on this chip.
Upstream flags E2B's correctness tests as failing — tpu-inference's support table marks gemma-4-E2B-it ✅ unit / ❌ correctness / ❓ performance, while the 26B and 31B pass all three. Quality probes on v5e were clean (8/9 byte-identical outputs, 3/3 needle retrievals at 2K/8K/14K).

Appendix: operational traps

A published price is not an offer of capacity, or even of the provisioning model. Three independent gates; quota is the weakest signal and the one most often checked first.
Google's regions-and-zones page undercounts v6e zones — 8 documented against 18 the API accepts.
WAITING_FOR_RESOURCES is not a failure. Do not record it as one and do not tear the request down; flex-start capacity can take two hours to return.
v6e quota ids drop the Litepod that v5e's carry, and a stale id matches no rows rather than erroring — indistinguishable from "no quota anywhere".
/v1/completions returns an empty string on -it models. Use /v1/chat/completions.
Endpoints are not stable. The external IP changes every time the node is recreated.
Google quotes v5e bandwidth in GiBps and v6e in GBps. Normalise before dividing.
31.24 GiB and 33.55 GB are the same number. XLA prints GiB; memory_analysis() returns bytes.
XLA compares temporaries alone against the whole chip and does not subtract resident weights. available HBM (31.24G) in an error message is not headroom.

Three Clouds, Three Native Agents

xbill — Mon, 10 Aug 2026 15:04:09 +0000

What is this project trying to do?

Three AI agents, each built with a different vendor's framework, each running on
that vendor's own hosting, all answering the same question at the same time:

Google — an ADK agent on Cloud Run
AWS — a Strands agent on Bedrock AgentCore Runtime
Azure — an Agent Framework agent on Container Apps

One coordinator calls all three over A2A v1.0 and takes the median of their
answers. And there is no long-lived credential stored anywhere in the running
system — every call is authenticated with a token minted at the moment it is
needed.

Everything is here:
github.com/xbill9/multicloud-adk-a2a-currency.
You can run the whole mesh on a laptop in about a minute; instructions are below.

The surprise wasn't the protocol. A2A worked. The surprise was that almost every
decision that mattered was made before a single A2A call happened.

Why bother? Just use a key

You have an agent on one cloud. Someone asks you to have it call an agent on
another.

The reflex is to create a service account key, drop it in a secret manager, and
move on. That works. It also means you now own a credential forever — rotating
it, scoping it, auditing it, and eventually explaining to somebody why
production contains a static key.

There is another way, and the interesting part is that it isn't harder. It is
just decided earlier.

The one decision that sets everything else

Here is the asymmetry the whole design falls out of.

Every agent you want to call can consume an external token. AWS IAM has OIDC
identity providers. Entra has Federated Identity Credentials. AgentCore accepts a
CUSTOM_JWT. All three will trust a token minted somewhere else, provided you
set the trust up correctly.

But only some runtimes can mint one. A runtime that can produce a workload
OIDC token — for an audience you choose — can federate outward to any of them.
A runtime that cannot is back to storing a credential.

So "where does my coordinator run?" is really "how many secrets will this system
have?"

Coordinator host	Legs it makes	Long-lived secrets
Cloud Run	GCP→AWS, GCP→Azure, GCP→GCP	potentially zero
AgentCore	AWS→Azure, AWS→GCP	at least one
Foundry	Azure→AWS, Azure→GCP	one or two, both unproven

Cloud Run wins here because its metadata server hands you an ID token for any
audience you name, which is exactly what the other two clouds' trust policies
want to see. Whether AgentCore can do the same is unconfirmed — I did not test
it. So "zero secrets" is a property of this topology, not a law about
cross-cloud agents.

Two things that choice costs you, worth saying out loud:

One leg stops being cross-cloud. The coordinator runs on Cloud Run, so the
GCP leg is Google calling Google. Two vendor boundaries get crossed, not three.
That belongs in the results, not in a footnote.

You cannot run it locally. A user credential cannot mint an
arbitrary-audience ID token at all — gcloud auth print-identity-token --audiences=... refuses outright, telling you it requires a service account.
There is no laptop version of this path. Once you choose federation, the only
place the system works is the place it is deployed.

Three legs, three mechanisms, one seam

The legs do not look alike:

GCP → GCP — an ID token whose audience is the target service's URL, plus roles/run.invoker.
GCP → AWS — mint that token, hand it to STS AssumeRoleWithWebIdentity, get temporary credentials back, sign the request with SigV4.
GCP → Azure — mint that token, present it to Entra as a client assertion against a Federated Identity Credential, get an access token back.

Two bearer tokens and a request signature. Different shapes entirely.

The move that made the rest tractable was putting all three behind one interface:
httpx.Auth. To httpx, a bearer header and a signature over the request body are
the same kind of object. All three vendor SDKs accept an httpx.AsyncClient. So
the credential attaches once, and everything through that client carries it.

auth = credentials_for(peer, endpoint)   # an httpx.Auth, or None
client = load_client(stack, endpoint, auth=auth)

Build that seam before your second cloud, not after your third. Get one leg
working with inline code and promise to generalise later, and you end up with
three error-handling styles and three places a token gets cached.

Worth noticing: an agent's card lives at /.well-known/agent-card.json,
and it sits behind the same authorization as the agent itself. Attach your
credential to the request instead of the client and discovery 403s while
the actual call would have worked. You get a protocol error pointing nowhere
near auth. Attaching to the client makes that impossible by construction.

Five traps that look exactly like working configuration

None of these are typos. Each is something you can get wrong while being careful.

Audience is not authorization. The caller picks the audience. So a trust
policy checking only audience proves that somebody in that IdP minted a token —
not that your identity did. Pin the subject too, and pin it to the immutable
numeric ID rather than the email, because emails can be released and re-bound to
someone else.

AWS and Azure invert the same step. AWS federates with accounts.google.com
natively — create an explicit IAM OIDC provider for it and you break it with
InvalidIdentityToken. Entra requires you to create the credential explicitly.
Same conceptual task, opposite prerequisites, and neither error tells you which
rule you are on.

The IAM condition keys do not hold what their names say.
accounts.google.com:oaud is the token's aud. accounts.google.com:aud is its
azp, which is a number. Put an audience string in :aud and you have written a
condition that can never match. The denial will not mention it.

Ask for the whole token. The GCP metadata mint takes format=full. Without
it, Google trims claims — including email — and any trust condition reading
that claim silently stops matching.

Two error codes are worth more than a day of logging. From STS,
InvalidIdentityToken means the token did not validate at all, which is a
provider-setup problem. AccessDenied means it validated fine and your
conditions did not match, which is a policy problem. Different afternoons.

Which leads to the one habit I would carry to any project like this: log the
raw provider response at every auth boundary. In an agent system an error comes
back as a tool result, and a model in the middle will cheerfully paraphrase
AccessDenied: condition accounts.google.com:sub did not match into "there was
an issue with the credentials." A raised message is not an observable.

Running it

Start local. Three agents on loopback, no cloud account, about a minute:

git clone https://github.com/xbill9/multicloud-adk-a2a-currency
cd multicloud-adk-a2a-currency

uv pip install --system "a2a-sdk[http-server]" google-adk \
  agent-framework-a2a agent-framework-core \
  pydantic httpx uvicorn pytest pytest-asyncio
uv pip install --system -e .

Bring up the three agents and ask them a question:

./infra/run_mesh.sh start          # :10001 :10002 :10003
python3 -m coordinator.cli 100 USD EUR JPY

Three vendors' agent stacks answering together:

participants: gcp, aws, azure

100 USD = 92 EUR @ 0.92 [3/3 clouds, agreed]
    gcp                  92 (164ms)
    aws                  92 (25ms)
    azure                92 (12ms)

The demo is the more interesting run, because it shows what happens when a
participant is wrong:

./infra/demo.sh

Four acts: three clouds answering, the 3×3 interop matrix, a cloud going
offline, and a cloud lying. The last two are the point — anything can show three
green ticks.

Deploying for real is one script per cloud, then one command to wire them
together:

./infra/deploy_aws.sh   deploy     # AgentCore Runtime + federated role
./infra/deploy_azure.sh deploy     # Container App
./infra/deploy_azure.sh fic        # Entra app registration + federated credential
./infra/deploy_azure.sh auth       # make the ingress actually demand it

./infra/deploy_gcp.sh deploy       # ADK service + coordinator job
./infra/deploy_gcp.sh wire         # fold the AWS and Azure legs in
./infra/deploy_gcp.sh run          # three-cloud consensus, from the cloud
./infra/deploy_gcp.sh verify       # the negative controls

Run verify twice. It is the part that decides whether any of the auth
claims mean anything, for a reason covered below.

Deployment decisions that aged well

Put deployment in the repo as verbs, not in a runbook. deploy, wire,
verify. Each cloud's identifiers live in exactly one place — the script that
created them — and the other scripts read them back rather than keeping copies.

I can tell you precisely what that buys, because I tore the entire mesh down and
rebuilt it from nothing to check.

The AWS runtime came back with a different ARN, and its invocation URL
contains that ARN. The Entra app registration came back with a different client
ID. The Container App came back on a different FQDN. Nothing was edited by
hand. wire read all three back out and the mesh returned:

100 USD = 92 EUR @ 0.92 [3/3 clouds, agreed]

Any copy of any of those identifiers stored anywhere else would have been stale
the moment it was written down.

Then the whole verification pass ran again against infrastructure that had not
existed an hour earlier: three consensus runs at 3/3 clouds, agreed, and all
eight auth probes — each leg answering with its credential, each leg denied
without it, an unauthenticated request rejected, and a right-identity
wrong-audience request rejected. Every number in this article comes from that
rebuilt mesh.

That teardown also found two bugs that no amount of redeploying would have,
because they live on code paths you can only reach from nothing:

A retry wrapper I had added to the AWS script made "no runtime exists" return an error instead of the string None. Under set -e, a first deploy died silently before ever creating the runtime. Every deploy since I wrote it had taken the update branch, so nothing ran the broken path.
Azure soft-deletes Cognitive Services accounts. Deleting the resource group does not purge them, so recreating by the same name fails with FlagMustBeSetForRestore — an error that never mentions deletion. destroy followed by deploy could not rebuild the Foundry account.

If you take one operational thing from this article: rebuild from nothing
at least once before you tell anyone it is reproducible.

Scale to zero, and label what it costs. Everything here idles at zero
replicas. Paying for idle capacity on three clouds to make a latency table look
tidier is paying to mislead. But it means the first call into a leg pays a cold
start — a cold Azure leg measured 27.8 seconds against 0.5 seconds warm.
Mix those two regimes in one table and every conclusion drawn from it is wrong.

Scaffolding worth stealing

Four structures did most of the work.

Structure	What it buys
One credential seam (`httpx.Auth`)	callers never know which of three mechanisms they are using
One participant interface (`convert()`)	a cloud is an implementation, not a branch
An instrument, not a demo	every failure typed by layer, not just red
Controls scoped to one leg	a degrading system cannot hide a denial from you

That last one is the one I would most want you to copy, because getting it wrong
is invisible.

The mesh takes a median across three clouds and degrades on purpose. Lose a
cloud, the other two still reach quorum, and the run exits 0. Now try testing
your auth by removing one leg's credential from a three-cloud run. It still exits

That reads as "no denial happened." What actually happened is "the denial was absorbed."

So every leg gets probed alone. Eight probes: each leg answering with its
credential, each leg denied without it, an unauthenticated request rejected, and
a right-identity-wrong-audience request rejected. Only then does an exit code
mean anything.

The general form: any system with graceful degradation needs its controls
scoped to a single component, or the degradation hides exactly the failure you
are testing for.

What it costs

Warm runs of the three-cloud consensus, after the rebuild:

	GCP (in-cloud)	AWS	Azure	elapsed
range	836–948ms	1027–1109ms	468–512ms	1711–1854ms

Elapsed lands roughly a second above the slowest single leg, and far below the
sum of all three. The legs are issued concurrently, so the sum was never the
right model — but neither is the slowest leg on its own. That extra second is the
coordinator's own fixed cost: container start, three agent-card fetches, three
credential mints.

Worth noticing: an earlier version of this claim quoted the slowest leg
alone and was wrong by 85% on the fastest run. That error only became
visible once there was more than one sample.

The federation itself is cheap. Token mints and exchanges are a small slice of
that fixed second. If the mesh feels slow, it is a cold start or a model — not
the identity work.

What this does not show

One deployment, one account, one region pair, one person, over a few days. These
are existence proofs: a thing worked, in a configuration. They are not
measurements of a population.

It is keyless in operation, not in bootstrap. Creating trust policies, app
registrations and federated credentials used ordinary operator credentials, as
provisioning always does.

And that claim needed checking, which is the honest part. The three A2A legs were
always keyless — but the Azure app pulled its container image using the
registry's admin password, stored as a secret in its own configuration. Not on
any agent-to-agent path, and still enough to make "no stored secrets" false as
written. Container Apps supports pulling by managed identity, so the fix was a
role grant and deleting the secret. An audit of all three deployments now shows
no stored credential in any of them.

The dull general point: image pull is part of your deployed system. A claim
about secrets has to cover all of it, not just the interesting part.

Token expiry and refresh are implemented and tested against a frozen clock, but
no token has ever expired in production — every run is a job that lives a few
seconds.

If you are starting one of these

Decide where the coordinator runs before anything else; it sets the secret count
for the entire system. Build the credential seam before the second cloud. Attach
auth to the client, not the request, so discovery is covered. Log the provider's
own words at every boundary, because you will spend more time reading auth
failures than writing auth code. Scope your controls to one component, because a
system built to survive failure will happily hide one from you.

And rebuild it from nothing once, before you claim it is reproducible.

Repo:
github.com/xbill9/multicloud-adk-a2a-currency
— the three agents, the coordinator, the interop matrix, the deploy scripts, and
the findings write-ups in docs/.

DEV Community: Google Developer Experts

Dart 3.13 Primary Constructors + BlocSignal: Boilerplate-Free Reactive Architecture

1. Zero-Boilerplate Events & States

🔴 Before Dart 3.13:

🟢 With Dart 3.13 Primary Constructors:

2. Streamlined Dependency Injection in CubitSignal

🔴 Before Dart 3.13:

🟢 With Dart 3.13:

3. Event Handler Registration via the this Block in BlocSignal

4. Immediate Reactive Wiring with createEffect

5. Named Constructor Shorthands (new) for Testing & Seeding

🛠️ Enabling Dart 3.13 in Your Project

1. Set the SDK Constraint in pubspec.yaml:

2. Enable Dart 3.13 Linter Rules in analysis_options.yaml:

🚀 The Architectural Payoff

💬 Over to You: What's Your Take?

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

No published build covers aarch64 and SM 7.5 together

AWS ships the one PyTorch that still has Turing

The PyTorch DLAMI has no compiler

The newest vLLM was the only one that worked

Gemma 4's attention heads are not one size

64 KiB is the whole problem

Most of the build is kernels that can never load

What I got wrong before I had hardware

What it does once it runs

Troubleshooting quick reference

The short version

Vibecoding EventMatch, built with Antigravity, ADK, and Gemini

What I wanted the app to do

The multi agent architecture

Integrations

Building it, and what it taught me

Do I Still Need a Monkey Patch for Gemini Live?

What Does This Agent Do?

What Level 3 Needed to Work

What ADK 2.x Handles Natively

The One Thing That Broke

What Else Was Updated

The Log Line That Never Ran

Video That Stops When You Look Away

Code With No Caller

Rules for Staying on ADK 2.x

Two Traps Worth Knowing

What Did Not Change

So What Really Changed?

Summary

Riverpod to BlocSignal: Incremental Migration and Zero-Codegen Signals for Flutter

Introduction: Combining Riverpod Safety with Zero Codegen

The Core Ergonomics: Riverpod Notifier vs. CubitSignal

The Riverpod Approach (Requires @riverpod & build_runner)

The BlocSignal Approach (100% Pure Dart 3, No Codegen)

What Changed?

Zero Risk: Incremental Trial & Bidirectional Interop

Option A: Expose a BlocSignal / CubitSignal to Existing Riverpod Widgets

Option B: Adapt an Existing Riverpod Provider into BlocSignal

Converting AsyncValue <-> AsyncState

Explore the Official Riverpod Ports on blocsignal.dev

Built-In AI Agent Skills for Automated Migration

Summary

MCP Configuration for Looker with Codex

Deja Vu — What is Old is New!

What is Looker?

Key Features and Capabilities

Looker? I thought Big Query Did everything!

What is MCP?

Google MCP Strategy

MCP Toolbox

Looker MCP Setup

Codex

Google Skills Repository

What you talkin ‘bout Willis?

Where do I start?

Looker Admin Setup

Looker Instance URL

Looker User Setup

Setup the Basic Codex Environment

Codex MCP Configuration

Trusting the Project

Initial Connection

2. Streamlined Dependency Injection in `CubitSignal`

3. Event Handler Registration via the `this` Block in `BlocSignal`

4. Immediate Reactive Wiring with `createEffect`

5. Named Constructor Shorthands (`new`) for Testing & Seeding

1. Set the SDK Constraint in `pubspec.yaml`:

2. Enable Dart 3.13 Linter Rules in `analysis_options.yaml`:

The Core Ergonomics: Riverpod `Notifier` vs. `CubitSignal`

The Riverpod Approach (Requires `@riverpod` & `build_runner`)

Option A: Expose a `BlocSignal` / `CubitSignal` to Existing Riverpod Widgets

Option B: Adapt an Existing Riverpod Provider into `BlocSignal`

Converting `AsyncValue` <-> `AsyncState`