DEV Community: Dale Nguyen

Building a Document-RAG Agent on GCP's Agent Development Kit (ADK)

Dale Nguyen — Tue, 14 Jul 2026 14:06:30 +0000

LogiChat is a chatbot platform. Customers upload their docs, get a chat widget, never touch a model. For two years, "training the bot" meant hand-curating a Q&A list in a dashboard form — question and answer pairs, one row at a time, dumped into the prompt as few-shot pairs. I just deleted that and rebuilt the whole pipeline on Google's Agent Development Kit (ADK), with Vertex AI for the model, Firestore for vector storage, and Cloud Run for everything else.

It's not a "hello world" agent demo — it's the architecture I landed on after rebuilding the whole pipeline over the last few months, three Cloud Run services touched (the fourth, the Stripe top-up token subscriber, was untouched), and the one design choice that turned out to matter more than anything else: retrieval as a pre-model step, not an ADK tool.

The stack, end to end

The pipeline below touches three of those services — apps/api, apps/subscribers/doc-processor, and apps/agent — plus the Firestore vector index. apps/api appears twice in the diagram because it acts as both the upload ingress AND the gateway that mints ID tokens for the agent. Click through the five stages to see which service is active at each hop and what payload crosses each boundary:

Interactive diagram: step through the five-stage pipeline — Upload → Process → Retrieve → Ground → Answer — on the original post on dalenguyen.me. Each hop shows which service is active (dashboard → apps/api → Cloud Storage → doc-processor → Firestore → apps/agent → Vertex AI) and what payload crosses the boundary.

Two pieces are different from a textbook RAG stack:

The gateway (apps/api, Express) is a separate service that mints Cloud Run ID tokens to call the agent. The agent is not publicly reachable.
The retrieval layer sits before the ADK Runner, not inside it as an ADK tool.

Both are deliberate. Both are what this post is really about.

Why ADK

I picked Google's Agent Development Kit over LangGraph, CrewAI, and a hand-rolled loop for one reason: the agent service runs entirely on GCP, and ADK is the only one of those that was built for Vertex AI from the ground up. The Python SDK ships a google.adk.agents.Agent and a google.adk.runners.InMemoryRunner that route through google-genai, which already speaks Vertex AI when GOOGLE_GENAI_USE_VERTEXAI=TRUE. No LangChain adapter, no provider shim, no surprise model name drift between staging and prod.

Concretely, my apps/agent is a 53-line pyproject.toml Python 3.12 service:

[project]
name = "logichat-agent"
version = "0.1.0"
description = "LogiChat ADK agent service (Python) — Gemini 2.5 Flash via Vertex AI."
requires-python = ">=3.12"

dependencies = [
    "fastapi>=0.115",
    "pydantic>=2.7",
    "uvicorn>=0.30",
    # google-adk is the Agent Development Kit Python SDK. Pinned loosely
    # because ADK's API is still moving; follow-up issues will exercise
    # the real Runner (sessions / tools / streaming).
    "google-adk>=0.1.0",
    # google-genai comes with ADK, but I import google.genai.types
    # directly in the runner adapter, so declare it explicitly.
    "google-genai>=1.0",
    # google-cloud-firestore is the client I use for the per-app chunk
    # vector search. Required by the retrieval layer.
    "google-cloud-firestore>=2.20",
]

Five runtime deps. FastAPI for the HTTP surface, Pydantic for the wire contract, uvicorn for the Cloud Run entrypoint, google-adk for the agent, google-genai for the typed message parts, google-cloud-firestore for the KNN search. ADC works for all of it — no API keys anywhere.

The agent service, top to bottom

apps/agent is split into four files, each with one job:

File	Job
`src/agent/main.py`	uvicorn entrypoint; binds `$PORT` (Cloud Run injects)
`src/agent/agent_definition.py`	Builds the ADK `Agent` + `InMemoryRunner`, wraps in `RunnerProtocol`
`src/agent/retrieval.py`	Firestore `find_nearest` + `text-embedding-005` via Vertex AI
`src/agent/api/app.py`	FastAPI factory: `POST /run`, `GET /healthz`

The main.py is six lines of orchestration:

# apps/agent/src/agent/main.py
def main() -> None:
    runner = build_agent()           # builds the ADK-backed adapter
    app = create_app(runner=runner)  # wires the runner into FastAPI
    port = int(os.environ.get("PORT", "8080"))
    uvicorn.run(app, host="0.0.0.0", port=port, log_level="info")

build_agent() is where ADK gets wired up. Here's the whole thing:

# apps/agent/src/agent/agent_definition.py
MODEL_ID = "gemini-2.5-flash"

SYSTEM_INSTRUCTION = (
    "You are LogiChat's support assistant. Answer the user's question using "
    "ONLY the context retrieved from the company's knowledge base. If the "
    "retrieved context does not contain the answer, say you don't know and "
    "suggest the user rephrase or contact a human. Never invent facts, "
    "product names, or policies that are not present in the retrieved "
    "context. Each retrieved block is prefixed with an internal "
    "[doc:<id>] marker so you can tell sources apart while reasoning — "
    "these markers are for your own use only and must never appear "
    "anywhere in the text you show the user."
)

def build_agent() -> RunnerProtocol:
    _require_env("GOOGLE_GENAI_USE_VERTEXAI")  # TRUE → route via Vertex AI
    _require_env("GOOGLE_CLOUD_PROJECT")       # logichat-dev / logichat-prod
    _require_env("GOOGLE_CLOUD_LOCATION")      # us-central1

    # Inline import: lets the unit tests run without google-adk installed.
    from google.adk.agents import Agent
    from google.adk.runners import InMemoryRunner

    root_agent = Agent(
        name="logichat_faq_agent",
        model=MODEL_ID,
        instruction=SYSTEM_INSTRUCTION,
        # tools=[firestore_retrieval_tool]  ← intentionally NOT an ADK tool
    )
    runner = InMemoryRunner(agent=root_agent)
    return _AdkRunnerAdapter(runner)

The env vars are validated at startup so a misconfigured deploy fails the first request — not the first deploy. The model id is pinned in code so a stray env var override can't silently swap in a different family. The region and project still come from env so staging and prod can diverge without a rebuild.

The `RunnerProtocol` seam

The google.adk.runners.InMemoryRunner exposes more surface than my HTTP layer needs — sessions, event streams, function calls. I wrap it in a protocol so the route handler is stable while the upstream SDK moves:

# apps/agent/src/agent/api/dependencies.py
class RunnerProtocol(Protocol):
    async def run(
        self,
        *,
        app_id: str,
        session_id: str,
        question: str,
        config: dict[str, Any],
    ) -> RunnerResult: ...

_AdkRunnerAdapter implements that protocol and forwards to the real ADK runner. The HTTP layer only knows about RunnerProtocol — it never imports google.adk. The reason this matters: the ADK Python SDK's API is still in motion. Wrapping it lets me swap implementations later (sessions, eval harness, streaming) without touching app.py.

The adapter runs retrieval before the ADK Runner call. This is the design choice worth understanding.

Retrieval as a pre-model step, not an ADK tool

This is the single most important architectural decision in the whole system, and it's the one most ADK tutorials get wrong.

The "obvious" pattern in ADK is to expose retrieval as a tool:

# What I did NOT do.
firestore_retrieval_tool = FunctionTool(firestore_retrieval)

root_agent = Agent(
    name="logichat_faq_agent",
    model="gemini-2.5-flash",
    instruction=...,
    tools=[firestore_retrieval_tool],  # model invokes at its discretion
)

I evaluated this and rejected it. Two reasons:

Threshold short-circuit. When no chunk is close enough to the question, I want to skip the model call entirely and return defaultAnswer. ADK tools fire inside the model loop — by the time the tool runs, you've already paid the input-token cost. A tool-based design cannot enforce a "no useful context" short-circuit because the model has to decide to invoke the tool first.
Cross-tenant safety. app_id must never become a model-controlled argument. Exposing it as a tool input would let a crafted prompt pivot retrieval to another tenant's collection. I scope retrieval server-side from the trusted caller (config / RunnerProtocol.run's app_id), not from the model.

So retrieval lives in the adapter, before the ADK runner is invoked:

# apps/agent/src/agent/agent_definition.py
async def run(self, *, app_id, session_id, question, config):
    # 1. Retrieval — pre-model, NOT an ADK tool
    chunks = await self._retrieve(app_id=app_id, question=question)

    if not chunks:
        # No useful context → short-circuit to defaultAnswer.
        # Also covers the case where retrieval raised: _retrieve returns
        # an empty list and a flag on failure.
        return RunnerResult(answer=_default_answer(config), grounded=False, retrieval=[])

    # 2. Compose the grounded user message
    chunk_block = "\n\n".join(_format_chunk(c) for c in chunks)
    user_text = f"{chunk_block}\n\nUser question: {question.strip()}"

    # 3. Run the ADK flow against the grounded message
    runner = self._adk_runner
    user_id = app_id  # one ADK user per LogiChat app
    session = await runner.session_service.get_session(
        app_name=runner.app_name, user_id=user_id, session_id=session_id
    )
    if session is None:
        await runner.session_service.create_session(
            app_name=runner.app_name, user_id=user_id, session_id=session_id
        )

    message = types.Content(role="user", parts=[types.Part(text=user_text)])
    final_text = ""
    async for event in runner.run_async(
        user_id=user_id, session_id=session_id, new_message=message
    ):
        if event.is_final_response() and event.content and event.content.parts:
            final_text = "".join(part.text or "" for part in event.content.parts)

    return RunnerResult(
        answer=sanitize_answer(final_text),
        grounded=True,
        retrieval=[
            RetrievalHit(
                chunk_id=c.chunk_id, document_id=c.document_id, distance=c.distance
            )
            for c in chunks
        ],
    )

Three things worth pointing out:

asyncio.to_thread wraps the sync google-cloud-firestore and google-genai calls so a slow Firestore round-trip doesn't block the uvicorn event loop. ADK itself is async-native; the retrieval layer is the only sync part of the request path.
Graceful degradation: _retrieve swallows exceptions, logs a warning, returns []. The request still succeeds via the ungrounded ADK flow. A transient Firestore outage degrades answer quality; it doesn't take the service down.
RunnerResult carries grounded + retrieval alongside answer. The gateway logs retrieval metadata on the conversation doc so I can evaluate retrieval quality offline. The original ADK Runner contract only returns text; I extended it through my adapter.

The retrieval layer

apps/agent/src/agent/retrieval.py is small on purpose — it owns the two operations the adapter needs: embed the question, pull the most relevant chunks. Nothing else in the agent service touches Firestore or the embedding API directly.

EMBEDDING_MODEL = "text-embedding-005"
EMBEDDING_DIMENSION = 768
DEFAULT_DISTANCE_THRESHOLD = 0.5  # COSINE: 0 = identical, 2 = opposite
DEFAULT_TOP_K = 5

The KNN search uses COSINE distance:

# NB: neither symbol is re-exported from the package roots. Both
# raise ImportError against the real google-cloud-firestore 2.x
# install. Don't "fix" this to the package root — it'll silently
# fall through to the ungrounded fallback on every request.
from google.cloud.firestore_v1.base_vector_query import DistanceMeasure
from google.cloud.firestore_v1.vector import Vector

query = db.collection(f"logichat-apps/{app_id}/chunks").find_nearest(
    vector_field="embedding",
    query_vector=Vector(query_vector),
    distance_measure=DistanceMeasure.COSINE,
    limit=limit,
    distance_result_field="vector_distance",
)

The threshold is applied in Python, after KNN returns. The reason: find_nearest returns the nearest limit chunks by distance; the threshold is a separate concern (recall vs precision). Conflating them in the query means every app owner has to know the right top-k to balance both. Better to keep the parameters independent.

Both write path and read path must agree on dimension. The KNN index in firestore.indexes.json is built for 768 dimensions, flat index:

{
  "collectionGroup": "chunks",
  "queryScope": "COLLECTION",
  "fields": [
    {
      "fieldPath": "embedding",
      "vectorConfig": {
        "dimension": 768,
        "flat": {}
      }
    }
  ]
}

Drift here is silent and catastrophic — a chunk embedded with a 1536-dim model stored against a 768-dim index either fails the write or, worse, produces results that look reasonable but are nonsense. I pin EMBEDDING_MODEL and EMBEDDING_DIMENSION as module constants and fail loud if the API returns the wrong shape:

if len(values) != EMBEDDING_DIMENSION:
    raise RuntimeError(
        f"embedding model {EMBEDDING_MODEL} returned {len(values)} dims, "
        f"expected {EMBEDDING_DIMENSION}"
    )

The doc-processor — where the chunks come from

The agent's job is to answer questions. The doc-processor's job is to make sure there are chunks to answer from.

apps/subscribers/doc-processor is a Cloud Run service subscribed to google.cloud.storage.object.v1.finalized events on the project bucket. Trigger filter is the path prefix apps/{appId}/docs/{documentId}/{filename} — anything outside that prefix is ignored silently. Why drive off the Storage event instead of a Firestore write? Because the API writes the pending doc before it returns the signed URL, so the metadata can lag the actual upload by seconds. Driving off Storage means I wait for the real bytes.

When the event fires:

// apps/subscribers/doc-processor/src/app/doc-processor.handler.ts
const [buffer] = await storage.bucket(event.bucket).file(event.name).download()
const parsedDoc = await parseDocument(buffer, mimeType)
const chunks = chunkText(parsedDoc.text)
const vectors = await embedBatch(chunks)
await persistChunks(appId, documentId, uid, chunks, vectors, embeddingModel)
await markReady(appId, documentId, chunks.length, embeddingModel)

Three of those lines hide real choices.

Chunker (libs/server/ai/src/lib/chunker.utils.ts). 800-token chunks with 100-token overlap. Splits on paragraph boundaries first, falls back to sentence boundaries when a paragraph overflows, and applies overlap by prepending the tail of the previous chunk to the next. Pure function in *.utils.ts — no injected dependencies, no lifecycle, that's the repo convention.

Parser. Plain text, markdown, HTML, and PDF. The HTML parser uses cheerio and strips <script>, <style>, <nav>, <header>, <footer>, <noscript> before extracting text — this was an actual bug I shipped and caught when a customer's "About" page nav ended up dominating their retrieval index.

Embeddings. text-embedding-005 via Vertex AI, batched in groups of 50 with two retries on 429/5xx. A single 50-page PDF can produce 200+ chunks; I don't want one HTTP call per chunk.

Vector storage. Each chunk goes into logichat-apps/{appId}/chunks/{chunkId} with embedding stored as FieldValue.vector(...) — not a plain number[]. Firestore's KNN index can only query fields written as Vector values; a plain number array is invisible to vector search and you'll get zero hits on every query with no error:

// FieldValue.vector is the ONLY way the KNN index can see this field.
embedding: FieldValue.vector(vectors[j]),

I use the modern @google-cloud/firestore v7 client only for chunk writes, because firebase-admin@^9 bundles v4 which doesn't have FieldValue.vector. Everything else stays on firebase-admin — they share ADC so it works, but the type definitions conflict and the build/src deep import dodges that.

The prompt: grounded chunks with `[doc:<id>]` markers

Each retrieved chunk is prefixed with an internal [doc:<id>] marker:

def _format_chunk(chunk: RetrievedChunk) -> str:
    return f"[doc:{chunk.document_id}]\n{chunk.content}"

The user message becomes:

[doc:abc123]
Our return policy allows returns within 30 days of purchase...

[doc:def456]
Shipping is free for orders over $50...

User question: how long do I have to return a dress?

The marker lets the model cite by id in its reasoning, but the marker must never appear in the user-facing answer. I tell the model that in the system prompt:

Each retrieved block is prefixed with an internal [doc:<id>] marker ... these markers are for your own use only and must never appear anywhere in the text you show the user.

And then I strip them in code anyway. Belt-and-braces is right when the prompt rule is the only thing keeping an internal token out of customer-facing text — there was a real incident where the marker leaked into the FAQ widget. The sanitiser is a few lines:

_DOC_MARKER_RE = re.compile(r"\s*\[doc:[^\]]*\]\s*")
_SPACE_BEFORE_PUNCT_RE = re.compile(r" ([.,;:!?])")

def sanitize_answer(text: str) -> str:
    cleaned = _DOC_MARKER_RE.sub(" ", text)
    cleaned = re.sub(r" {2,}", " ", cleaned)
    cleaned = _SPACE_BEFORE_PUNCT_RE.sub(r"\1", cleaned)
    return cleaned.strip()

The second regex is the subtle one — removing a marker that sat immediately before a period leaves a space, which looks worse than the leak. Always check for "removal left whitespace adjacent to punctuation" patterns.

Cloud Run + service-to-service ID tokens

The gateway and the agent are two Cloud Run services. The gateway calls the agent with a Google-signed ID token:

// libs/server/ai/src/lib/agent.service.ts
const { agentUrl } = getAgentConfig()
const client = isPlainHttpUrl(agentUrl)
  ? plainHttpClient
  : await authFactory().getIdTokenClient(agentUrl)

const response = await client.request({
  url: `${agentUrl}/run`,
  method: 'POST',
  data: { appId, sessionId, question, config },
})

The agent service is deployed with --no-allow-unauthenticated, so the only way to reach it is with a valid ID token minted by a service account in the same project. google-auth-library's GoogleAuth#getIdTokenClient(agentUrl) handles token refresh and attaches Authorization: Bearer ... on every request, so I don't manage tokens myself.

The deploy target:

gcloud run deploy logichat-agent-run \
  --image us-central1-docker.pkg.dev/${PROJECT_ID}/logichat/agent \
  --region us-central1 \
  --platform managed \
  --no-allow-unauthenticated \
  --port 8080 \
  --service-account ${PROJECT_NUMBER}-compute@developer.gserviceaccount.com \
  --update-env-vars \
    GOOGLE_GENAI_USE_VERTEXAI=TRUE,\
    GOOGLE_CLOUD_PROJECT=${PROJECT_ID},\
    GOOGLE_CLOUD_LOCATION=us-central1,\
    VERTEX_CHAT_MODEL=gemini-2.5-flash

One Cloud Run service, one service account (the default Compute Engine SA), three env vars that route through Vertex AI. The agent service account needs roles/aiplatform.user on the project — same binding covers chat and embeddings, no second role required.

The fallback I kept

I deleted the prompt-builder that read examples. The dashboard pages for managing Q&A examples are gone. The new upload/list/delete dialog replaced them.

But the collection is still there. Every existing customer has examples docs they wrote over two years. Deleting those would have been a worse customer experience than the Q&A flow itself. So the fallback:

# When document retrieval comes up empty...
chunks = await self._retrieve(app_id=app_id, question=question)
if not chunks:
    # ...try the legacy Q&A examples before giving up.
    examples = await self._retrieve_examples(app_id=app_id)
    if examples:
        return self._answer_with_examples(examples, question, config)
    return RunnerResult(answer=_default_answer(config), grounded=False, retrieval=[])

# Otherwise: grounded answer from documents
...

retrieve_examples is a thin wrapper:

@dataclass(frozen=True)
class Example:
    question: str
    answer: str

DEFAULT_EXAMPLES_LIMIT = 20

def retrieve_examples(db, app_id, *, limit=DEFAULT_EXAMPLES_LIMIT) -> list[Example]:
    """Fetch up to `limit` legacy Q&A pairs for `app_id`.

    No ordering, no scoring — these are small hand-curated sets, not
    ranked results.
    """
    ...

The cap at 20 is important. The old prompt-builder had no cap; a customer with 200 examples was already paying for it in latency. The new prompt block looks like:

Q: How do I reset my password?
A: Click "Forgot password" on the login page.

Q: Do you ship internationally?
A: Yes, we ship to 40+ countries. See our shipping page for details.

User question: can I return a dress after 30 days?

Document RAG is still the primary path. Examples are the fallback that runs only when retrieval returned nothing — no useful chunks, empty collection, or retrieval itself broke. A customer who uploaded docs gets doc-RAG answers (better recall, fresh as the doc). A customer who hasn't migrated yet still gets answers from their hand-curated examples. A customer who has neither gets defaultAnswer.

What I'd do differently

A few things I got wrong the first time and would change on a green-field rebuild.

The firebase-admin v4 / modern firestore v7 split is ugly. Bumping firebase-admin to v13 is the right fix; it's a separate PR's worth of type churn. The FieldValue.vector workaround works but it makes the doc-processor code look weirder than it should.

find_nearest symbol location. google.cloud.firestore.Vector and firestore_v1.DistanceMeasure are NOT re-exported from the package roots. The first two ships of this module both degraded silently to the ungrounded fallback on every request because the imports raised ImportError and the except-clause swallowed it. Inline the comment that warns future readers — it's not obvious.

Threshold and top-k should be data-driven, not env-driven. RETRIEVAL_DISTANCE_THRESHOLD is fine for v1. The next step is per-app tuning from the dashboard — different customers have different "useful" thresholds, and 0.5 is a reasonable default, not the right answer for everyone. The KNN top-k belongs in the same config.

Agent and gateway should share a runner. Right now apps/api POSTs to apps/agent over Cloud Run service-to-service ID tokens. The round-trip is invisible to users but adds ~30–80ms of latency and one more thing that can fail. The long-term answer is inlining the ADK flow into the API service for the simple cases, or running both as one Cloud Run service with internal routing. I measured, decided the separation was worth the latency for now (independent deploys, simpler auth), and parked the unification.

The numbers

The old prompt with 100 examples was 12–18K tokens per request, ~1.2s p50 latency, ~$0.003/request at GPT-4o-mini pricing. The new doc-RAG prompt is 1.5–3K tokens (system + 3–5 retrieved chunks + question), ~450ms p50, ~$0.0006/request. Recall on the customer's own docs is dramatically better because the bot now searches their actual content instead of paraphrased Q&A.

These numbers are directional, not a controlled benchmark. I didn't run a head-to-head A/B because the move changed three things at once: training data shape (Q&A pairs → retrieved chunks), model (GPT-4o-mini → Gemini 2.5 Flash), and provider (OpenAI → Vertex AI). The 4–5× cost-per-request delta below is dominated by the model/provider change, not by the retrieval change — don't read it as "RAG is 4–5× cheaper than few-shot." All figures are per-request inference cost; document ingestion (one-time embedding on upload) is excluded. The qualitative claim on recall is the one that I'm confident in: doc-RAG on a customer's actual docs beats few-shot Q&A on paraphrased summaries every time, and that's the reason I did this in the first place.

The full set of changes spans the document pipeline, the agent service, the retrieval layer, the dashboard UI revamp, and a separate design pass for the Q&A fallback.

TL;DR for someone building this on ADK today

Pin your model id in code. Don't let env vars silently swap model families.
Do retrieval pre-model. The "obvious" ADK-tool pattern breaks threshold short-circuit and exposes app_id to model-controlled arguments.
Validate Vertex AI env vars at startup, not on first request. build_agent() should fail fast on misconfiguration.
Wrap the InMemoryRunner in your own protocol. ADK's surface is bigger than your HTTP layer needs and the SDK is still moving.
Use FieldValue.vector for Firestore embeddings. Plain arrays are invisible to KNN.
Dimension-pin everything. Embedding model, KNN index, and your runtime check must all agree on EMBEDDING_DIMENSION.
Run sync Firestore / genai calls in asyncio.to_thread. Don't block the uvicorn event loop on a slow round-trip.
Strip internal provenance markers in code, not just in the prompt. Prompt rules leak; regexes don't.
Keep an escape hatch for legacy data. Customers will not thank you for deleting their work.

The Background Task That Froze: A Serverless CPU-Throttling Mystery

Dale Nguyen — Fri, 10 Jul 2026 22:06:35 +0000

💡 Lesson learned building an agent on Cloud Run: don't schedule work to run after you send the response. On Cloud Run's default request-based CPU mode it freezes mid-run — do the latency-critical work before you respond.

I had a webhook receiver that acknowledged events in well under a second. Fast, clean, exactly what you want from a webhook handler. The problem was what happened after the ack: the job that receiving the webhook was supposed to launch showed up about three minutes later. Sometimes it showed up right away. Sometimes it never showed up at all — until something completely unrelated happened to hit the service, at which point the "stuck" job would suddenly, retroactively, complete.

That last detail is what made it a genuine mystery for a while. A job that finishes the instant an unrelated request lands is not behaving like a slow job. It's behaving like something that was frozen, waiting for permission to keep running.

The setup

The receiver did the obvious, well-worn thing: validate the payload, dedupe it, and hand the actual work off so the HTTP response could go out immediately.

@app.post("/webhook")
async def webhook(req):
    ...validate...
    background_tasks.add_task(dispatch_job, payload)
    return JSONResponse(202, ...)

FastAPI/Starlette's BackgroundTasks is the textbook way to do "ack now, work later." It runs your callback after the response has been sent, without making the caller wait for it. On a normal server, that's a fire-and-forget convenience with no real downside — the process keeps running, the callback runs whenever the event loop gets to it, done.

This wasn't running on a normal server. It was running on Cloud Run, with the platform's default CPU allocation mode, which is request-based: the instance only gets meaningful CPU while a request is actually in flight. The moment the response goes out, CPU gets throttled down to something close to zero — and it stays there until the next request arrives.

Which means the code above doesn't say "run this job right after responding." On that platform, it says "run this job whenever the next request happens to show up, if one ever does."

The symptom, and why it's invisible

Here's the part that made this hard to see in logs: the background task doesn't fail. Nothing is killed, nothing throws, nothing gets logged as an error. The log line right before the network call in dispatch_job prints just fine — it runs during the tail end of the request's own CPU window. Then the very next line, the one after the await that actually calls out to launch the job, just... doesn't print. Not for a while. Not until CPU comes back.

The event loop isn't crashing. It's simply not being scheduled.

request in ──► handler runs (full CPU)
                 └─ schedules background task
response out ◄── 202 sent
                 └─ CPU throttled to ~0        ← background task freezes HERE,
                                                  usually mid-network-call
...minutes pass...
next request in ──► instance gets CPU again ──► frozen task resumes, completes

If the instance happens to scale to zero before any other traffic arrives, the frozen task never gets to finish. The work isn't delayed at that point — it's just gone.

This isn't unique to BackgroundTasks either. It's the same failure shape for any post-response work: a bare asyncio.create_task(...) fired right before returning, or a "post-ack" queue that classifies and dispatches inside the same process after replying to the caller. Anything scheduled to run after the response, on a platform that only grants CPU during a request, inherits this problem.

And it's genuinely hard to catch by normal means, because:

It's fast on busy days and slow-or-lost on quiet ones — the exact opposite of what you'd expect from an overload problem.
Retry-happy callers (chat platforms, webhook senders that retry on timeout) generate the very traffic that unfreezes the task, which converts "broken" into "occasionally a few minutes slow" — indistinguishable from ordinary network flakiness unless you're looking closely.
It works perfectly locally and under load testing, because both of those keep the instance saturated with requests, which is precisely the condition under which this bug can't occur.

Confirming it without a debugger

The cheapest way to confirm CPU starvation as the cause, rather than guess at it, is to inject traffic. If a "stuck" background task completes the instant you curl the service's health endpoint a couple of times, you've proven it — decisively, and without touching a debugger or adding instrumentation. I'd recommend this as step one before reaching for anything heavier.

The fix: move it onto the request path

The real fix isn't a smarter background task. It's not scheduling the latency-critical part as a background task at all. In this case, the RPC that actually launches the job — as opposed to waiting for the job to finish — costs on the order of one second. That's well within most webhook ack budgets (5 seconds or more), so it can just be awaited before responding:

@app.post("/webhook")
async def webhook(req):
    ...validate, dedupe...
    recent[key] = now()          # dedup marker
    try:
        await launch_job(payload)   # ~1s: starts the job, doesn't wait for it
    except Exception:
        recent.pop(key, None)       # let the sender's retry re-dispatch
        raise                       # → 5xx → sender retries
    return JSONResponse(202, ...)

Two details here matter as much as the await itself:

On failure, roll back the dedup marker before re-raising. If you don't, your own dedup logic swallows the sender's retry — the sender re-delivers the exact same event, your dedup marker says "already seen," and the job never gets a second chance to launch.
Let the failure surface as a 5xx. That's what tells the webhook source to actually retry. A 202 that lies about success is worse than an honest error.

I kept a background task only for the one case where the handler itself would legitimately run for minutes — a local-dev mode that executes the worker in-process instead of dispatching it elsewhere. That's a fine use of "run this later"; the RPC-launch path is not.

When inline genuinely isn't an option

Not every version of this problem can be solved by moving code earlier in the function. If the caller enforces a hard ack deadline (say, 3 seconds) and the pre-dispatch work is inherently slow — an LLM classification step, for instance — there may be no way to fit it inside the request. In that case, change the platform, not the code: switch the service to instance-based CPU allocation, so it keeps CPU between requests instead of throttling it to near-zero.

gcloud run services update <svc> --no-cpu-throttling   # plus persist in IaC/deploy config

That last comment matters. Run this as a one-off CLI command and the very next deploy from your normal pipeline can silently put the service back to request-based billing, because the deploy config never knew the flag existed. Persist it in whatever defines the deploy — Terraform, a service YAML, whatever your IaC actually reads — not as a manual patch applied by hand after the fact. It's also worth reconsidering minimum instance count at the same time, since instance-based billing changes the cost model from per-request to per-instance-time.

Testing the difference

One trap when you go to write a test for this: most test-client frameworks execute background tasks before handing the response back to your assertions. So a happy-path "was dispatch_job called?" test passes whether the call is inline or backgrounded — it can't tell the two apart. The assertion that actually distinguishes them is the failure path: trigger a dispatch failure and check that you get a 5xx and that the dedup marker got cleared. Only inline dispatch can produce that combination.

Lessons learned

Post-response code has no CPU guarantee on request-billed serverless. Treat "after the response" as "whenever the next request arrives, maybe never" — not "a few milliseconds later."
Latency-critical side effects belong on the request path. A job-launch RPC that only starts work is often cheap — in this case, about a second — well within most webhook timeout budgets. Await it, then respond.
Make inline failure retryable end-to-end. If the inline call throws, roll back any in-memory dedup marker and let the error propagate as a 5xx — the sender's retry then actually re-dispatches instead of being swallowed by your own dedup.
When inline is impossible, change the platform mode, not the code. Hard ack deadlines plus slow pre-dispatch work can't move onto the request path — switch to instance-based CPU allocation and reconsider minimum instance count. Persist the flag in deploy config, not a one-off CLI update.
Confirm the hypothesis by injecting traffic. If a "stuck" background task completes the instant you curl the service's health endpoint a few times, you've proven CPU starvation without touching a debugger. Cheap, decisive, non-invasive.
Audit the pattern fleet-wide once found. The same convenience API tends to get copy-pasted across every receiver in a codebase. One confirmed stall usually means its siblings have it too.

Pitfalls and false signals

The first log line printing does not mean the task ran. It ran for the last few milliseconds of leftover request CPU, then froze at the first await. Don't rule out starvation just because "the task clearly started."
"It works in staging" proves nothing here. Health checks and teammates clicking around provide exactly the wake-up traffic that masks this. Quiet production windows — nights, weekends — are where it actually bites.
Don't blame the downstream API. The frozen call is usually a perfectly healthy RPC; its client-side coroutine simply isn't being scheduled. Latency dashboards on the callee will look completely clean, because the callee never had a problem.
Retries hide the bug. Senders that retry unacked or failed deliveries generate the traffic that unfreezes the task, turning "broken" into "occasionally a few minutes slow" — easy to misfile as ordinary network flakiness.

Quick checklist

List every place a handler schedules work to run after the response — framework background tasks, create_task, post-ack queues.
For each one: is the platform CPU-throttled outside of requests? (Cloud Run default: yes.)
Can the work move inline within the caller's timeout budget? Move it; on failure, roll back dedup markers and return a 5xx.
If it can't move: flip to instance-based CPU allocation in the deploy config — not a live-only update — and revisit minimum instances.
Add a failure-path test: dispatch error → 5xx and dedup marker cleared.
To confirm a suspected stall in prod: watch the logs while sending a few synthetic requests. Instant completion means starvation confirmed.

One-line summary

On request-billed serverless, "after the response" means "whenever the next request happens to arrive" — so launch anything that matters before you respond, or pay for always-on CPU.

All-Green Lighthouse: How I Fixed Every Audit on My AnalogJS Blog

Dale Nguyen — Mon, 15 Jun 2026 12:44:46 +0000

The previous post showed how to embed live Angular charts in AnalogJS markdown and closed with a PageSpeed screenshot reading 97 Performance / 90 Accessibility / 100 Best Practices / 92 SEO. That was good — but not green across the board.

This post documents the follow-up audit: eight failing checks across four categories, and the exact fix for every single one. Several of the bugs are genuinely non-obvious, so I have tried to explain why each fix works rather than just showing a diff.

Measurement method

Before scores are from the live site on Vercel's CDN, audited with Lighthouse inside Chrome DevTools. After scores are from a local production build (.vercel/output/static) served on localhost and audited in a fresh isolated browser context.

One important caveat: the Chrome DevTools Lighthouse tool returns Accessibility, Best Practices, SEO, and Agentic Browsing scores. Performance is reported via Core Web Vitals from a separate performance trace (LCP, CLS). So the table below covers those four measurable Lighthouse categories. The post ends with the CWV results.

PageSpeed Insights: before and after

I ran PageSpeed Insights against the live "before" page and again against the deployed "after" production site. Desktop — before, then after:

Desktop, before: 91 Performance / 90 Accessibility / 100 Best Practices / 92 SEO.

Desktop, after: 100 Performance / 100 Accessibility / 100 Best Practices / 100 SEO.

Mobile — before, then after:

Mobile, before: 59 Performance / 90 Accessibility / 100 Best Practices / 92 SEO — mobile Performance is where most of the headroom was.

Mobile, after: 100 Performance / 100 Accessibility / 100 Best Practices / 100 SEO.

PageSpeed (Google-hosted Lighthouse)	Performance	Accessibility	Best Practices	SEO
Desktop — before	91	90	100	92
Desktop — after	100	100	100	100
Mobile — before	59	90	100	92
Mobile — after	100	100	100	100

Run it yourself: PageSpeed Insights for this page · the predecessor "interactive charts" post.

Two things stand out. Accessibility and SEO both reached 100 here too, confirming the contrast, label, and robots.txt fixes hold on Google's infrastructure — not just on my localhost run. And mobile Performance climbed 59 → 100, in stages: deferring third-party JS got it to 66, going zoneless took it to 77 (Total Blocking Time down to ~20 ms), WebP covers nudged it to 78, and finally inlining the CSS + preloading the cover image collapsed FCP (2.1 s → 0.9 s) and LCP (4.8 s → 1.2 s) to land at a perfect 100. The last step was the surprise: the real mobile bottleneck was never the framework or image bytes — it was the render-blocking CSS round-trip and late image discovery on Slow 4G. (Desktop is 100/100/100/100 too; all figures measured on the deployed production site.)

One more subtlety: the same page scores differently depending on which Lighthouse build runs it. PageSpeed Insights (Google-hosted) reports Best Practices 100 and has no "Agentic Browsing" category. The newer Lighthouse bundled in Chrome DevTools — which I used for the per-fix breakdown below — adds Agentic Browsing and weights the third-party-cookie issue far more harshly, so it scored the same page Best Practices 77 and Agentic Browsing 33. I optimized against the stricter build, so the fixes satisfy both.

Results

Category	Before	After
Accessibility	90	100
Best Practices	77	100
SEO	92	100
Agentic Browsing	33	100
Audits failed	8–9	0

Both desktop and mobile reached 100 in all four categories.

Fix 1 — Gate third-party loading on `click`/`keydown`/`touchstart`, not scroll

This is the most important takeaway from the entire audit.

The problem: Microsoft Clarity sets four third-party cookies (SM, MR, MUID, CLID). Lighthouse's third-party-cookies audit flagged all four, bringing Best Practices from 100 down to 77.

The obvious fix: load GA and Clarity lazily, deferred until "the user shows intent." The natural trigger feels like scroll — the user is reading, so let them get started before loading analytics. I tried that. It did not work.

Why scroll deferral fails: Lighthouse (and PageSpeed Insights) auto-scroll the page during the audit to capture above-the-fold and below-the-fold behavior. That scroll fires the scroll event on window, which immediately loads Clarity and re-introduces the third-party cookies. The audit itself trips the deferral.

The working fix: gate on click, keydown, and touchstart. Audit tools never synthesize those events. Real desktop users fire click on their first link or text selection; real mobile users fire touchstart the moment their finger makes contact with the screen (including to start scrolling — so mobile tracking coverage is essentially 100%). The only gap is a desktop user who scrolls through a page without ever clicking or pressing a key, which is rare in practice.

(function () {
  var loaded = false;
  function loadAnalytics() {
    if (loaded) return;
    loaded = true;
    // Google Analytics (GA4)
    window.dataLayer = window.dataLayer || [];
    function gtag(){ window.dataLayer.push(arguments); }
    gtag('js', new Date());
    gtag('config', 'G-XXXXXXXXXX');
    var ga = document.createElement('script');
    ga.async = true;
    ga.src = 'https://www.googletagmanager.com/gtag/js?id=G-XXXXXXXXXX';
    document.head.appendChild(ga);
    // Microsoft Clarity
    (function(c,l,a,r,i,t,y){
      c[a]=c[a]||function(){(c[a].q=c[a].q||[]).push(arguments)};
      t=l.createElement(r);t.async=1;t.src="https://www.clarity.ms/tag/"+i;
      y=l.getElementsByTagName(r)[0];y.parentNode.insertBefore(t,y);
    })(window, document, "clarity", "script", "YOUR_CLARITY_ID");
  }
  var events = ['click', 'keydown', 'touchstart'];
  function onFirst() {
    loadAnalytics();
    events.forEach(function (e) { window.removeEventListener(e, onFirst); });
  }
  events.forEach(function (e) {
    window.addEventListener(e, onFirst, { once: true, passive: true });
  });
})();

In an AnalogJS project this snippet goes into a postRenderingHooks entry in vite.config.ts — it gets injected into every prerendered HTML page at build time.

The tradeoff to acknowledge: a desktop visitor who only scrolls and never clicks or presses a key is not tracked. This is the cost of the green Best Practices score. It is an acceptable tradeoff for this site, but you should decide for yours.

Fix 2 — `publicDir` override silently orphans `public/`

The problem: Lighthouse's SEO audit reported 56 errors parsing /robots.txt. The score was 92, not 100.

Running curl https://dalenguyen.me/robots.txt returned the SPA's index.html — the server-side routing catch-all. /robots.txt was simply not deployed.

The file existed at apps/blog-app/public/robots.txt. But Analog's vite.config.ts overrides publicDir:

// vite.config.ts (simplified)
export default defineConfig({
  publicDir: 'libs/portfolio/shared',
  // …
})

Vite only serves one publicDir. The default Angular public/ folder is no longer watched, so any file placed there is never copied to the output. The fix was a one-liner: move (or duplicate) robots.txt to libs/portfolio/shared/robots.txt.

This is easy to miss because the development server might still serve static files through a separate mechanism, making the bug invisible locally. It only surfaces on the deployed build.

Fix 3 — The default Prism light theme fails WCAG AA

The problem: 78 color-contrast failures. The post had a lot of code blocks.

Prism ships a light theme (prism.css) where many token colors — comments in #708090, punctuation in #999, numbers and keywords in shades of blue and red — do not meet the WCAG AA 4.5:1 contrast ratio on Prism's own #f5f5f5 code background. Lighthouse caught 78 individual violations.

The fix is a single override block in styles.css, placed after the Prism @import so these rules win on equal specificity. Every color has been verified at ≥ 4.5:1 against #f5f5f5:

/* styles.css — placed after @import 'prismjs/themes/prism.css' */
code[class*='language-'],
pre[class*='language-'] {
  color: #1f2328;      /* 16.8:1 — base text */
  text-shadow: none;
}
.token.comment,
.token.prolog,
.token.doctype,
.token.cdata {
  color: #57606a;      /* 5.0:1 — comments */
}
.token.punctuation {
  color: #24292f;      /* 15.8:1 — brackets, semicolons */
}
.token.property,
.token.tag,
.token.boolean,
.token.number,
.token.constant,
.token.symbol,
.token.deleted {
  color: #0550ae;      /* 7.0:1 — numbers, booleans */
}
.token.selector,
.token.attr-name,
.token.string,
.token.char,
.token.builtin,
.token.inserted {
  color: #0a6e31;      /* 6.0:1 — strings */
}
.token.operator,
.token.entity,
.token.url,
.language-css .token.string,
.style .token.string {
  color: #953800;      /* 5.3:1 — operators */
  background: none;
}
.token.atrule,
.token.attr-value,
.token.keyword {
  color: #cf222e;      /* 4.6:1 — keywords */
}
.token.function,
.token.class-name {
  color: #6639ba;      /* 5.0:1 — functions */
}
.token.regex,
.token.important,
.token.variable {
  color: #953800;      /* 5.3:1 — variables, regex */
}

Two stragglers remained: the interactive diagram component from the previous post used #6e7b8a for muted labels on a #0b0f16 dark background (contrast ratio 4.44:1, just below the 4.5 threshold). Bumping to #8b98a8 pushed it to 5.2:1.

Fix 4 — Small accessibility papercuts

Four smaller a11y issues rounded out the Accessibility score.

Unlabeled range input. The KV-cache slider widget had an <input type="range"> with no accessible name. Screen readers announced it as "slider" with no context. Fix: add aria-label and aria-valuetext:

<input
  type="range"
  min="512"
  max="131072"
  step="512"
  aria-label="Context length in tokens"
  [attr.aria-valuetext]="ctxLabel() + ' tokens'"
/>

Heading order in a Shadow DOM component. The interactive mount-flow diagram used an <h4> for panel titles inside a ViewEncapsulation.ShadowDom component. The article's heading structure jumped from H2 directly to H4 — Lighthouse reads through shadow roots when evaluating heading order. Fix: replace <h4> with a <p class="panel-title"> (styled identically).

label-content-name-mismatch on the mobile menu button. The site header had a mobile hamburger button with both aria-label="Toggle mobile menu" and an inner <span class="sr-only">Open main menu</span>. The screen-reader-announced name ("Toggle mobile menu") did not match the accessible name computed from the inner text ("Open main menu"). This mismatch is the label-content-name-mismatch audit. Fix: remove the sr-only span; the icons are already aria-hidden, so the aria-label is sufficient.

/llms.txt for Agentic Browsing. Lighthouse's new "Agentic Browsing" category checks whether a site has a /llms.txt file — a plain-text document that tells AI agents what the site is about and which pages are canonical. Without it the category scored 33 (the range-input label failure also hurt it). Adding libs/portfolio/shared/llms.txt (which Vite deploys to the root) pushed Agentic Browsing to 100.

Performance fixes — Core Web Vitals

A few changes improved load performance. The LCP figures below are not strictly comparable (before = live CDN, after = localhost), but the bundle-size and Total Blocking Time wins are real and show up on PageSpeed:

Deferred third-party JS. The interaction-gated loader (Fix 1 above) has the side effect of removing GA, Clarity, and Logichat from the initial load path entirely. Zero third-party scripts on first paint means lower Total Blocking Time and no third-party network round trips before the page is interactive.

Lazy Giscus with IntersectionObserver. The comments embed (Giscus) was previously loaded eagerly in ngAfterViewInit, even for readers who never scroll to the bottom. Replacing that with an IntersectionObserver with a generous rootMargin of 600px means Giscus loads just before the reader reaches it — or not at all if they leave early:

private lazyLoadGiscus() {
  if (!isPlatformBrowser(this.platformId)) return
  const container = this.giscusContainer()?.nativeElement
  if (!container) return

  if (!('IntersectionObserver' in window)) {
    this.injectGiscus(container)
    return
  }

  const observer = new IntersectionObserver(
    (entries, obs) => {
      if (entries.some((entry) => entry.isIntersecting)) {
        obs.disconnect()
        this.injectGiscus(container)
      }
    },
    { rootMargin: '600px 0px' },
  )
  observer.observe(container)
}

fetchpriority="high" on the LCP cover image. The cover image is the Largest Contentful Paint element on every blog post. Adding fetchpriority="high" and decoding="async" tells the browser to prioritize its fetch above everything else in the image queue:

<img
  [src]="post.attributes.coverImage"
  [alt]="post.attributes.title"
  fetchpriority="high"
  decoding="async"
  class="w-full h-auto object-cover max-h-[400px]"
/>

Zoneless change detection — the biggest mobile lever. The app still shipped zone.js with provideZoneChangeDetection. On a 4×-throttled mobile CPU, parsing and running zone.js plus zone-based change detection is a real slice of Total Blocking Time before the page is interactive. Angular 20 lets you drop it:

// main.ts: delete `import 'zone.js'`
// app.config.ts
providers: [provideZonelessChangeDetection() /* … */]

This is safe only if the app is signal- and event-driven: any view that refreshes from a bare setTimeout, addEventListener, or RxJS subscribe (rather than a signal or a template (event)) will silently stop updating under zoneless — audit for those first. While I was at it I also moved @angular/material off the critical path (the header icons became inline SVG, dropping the Material Icons web font) and lazy-loaded @sentry/browser. Net effect: eager JS for the post fell from 658 KB to 583 KB, mobile Total Blocking Time dropped to ~20 ms, and mobile Performance went 66 → 77.

WebP cover images — the LCP follow-up. With the framework cost gone, the largest remaining mobile cost was the cover image (the LCP element), shipped as a PNG. I generated a WebP variant for every local cover and serve it through a <picture>, keeping the PNG as both the <img> fallback and the og:image:

<picture>
  @if (coverWebp()) {
    <source type="image/webp" [srcset]="coverWebp()" />
  }
  <img [src]="post.attributes.coverImage" fetchpriority="high" decoding="async" … />
</picture>

The WebP source is root-relative on purpose: a <source> does not fall back to the <img> on a 404, so an absolute prod URL would break the hero on preview builds. Across 25 covers this cut 12.2 MB → 0.9 MB (~92%) — the interactive-charts cover went 92 KB → 36 KB — taking mobile LCP from 5.8 s to ~4.8 s and Performance to 78.

Inline the CSS + preload the LCP image — the step that actually broke the ceiling. Mobile was stuck in the high-70s with LCP ~4.8 s. I assumed I'd need classic critical-CSS extraction (inline above-the-fold, async the rest). Measuring first saved the effort: the entire stylesheet is only ~9 KB brotli, already Tailwind-purged. So extraction was pointless — the cost wasn't bytes, it was the extra render-blocking request (a full round-trip on Slow 4G) plus discovering the cover image late. Two small postRenderingHooks fixed both:

// 1) Swap the render-blocking <link> for an inline <style> (no extra request)
html = html.replace(/<link[^>]*rel="stylesheet"[^>]*href="(\/assets\/[^"]+\.css)"[^>]*>/g,
  (tag, href) => { const css = readClientAsset(href); return css ? `<style>${css}</style>` : tag })

// 2) Preload the cover (first <picture>) so it fetches during head parse
//    <link rel="preload" as="image" type="image/webp" imagesrcset="…" fetchpriority="high">

The effect on mobile was dramatic: FCP 2.1 s → 0.9 s, LCP 4.8 s → 1.2 s, and Performance 78 → 100. The lesson: don't reach for the heavy tool (critical-CSS extraction) before measuring — for a small stylesheet, inlining the whole thing is simpler and just as effective.

Core Web Vitals (lab measurements, deployed production site):

Metric	Before	After
Mobile Performance	59	100
FCP (mobile)	2.1 s	0.9 s
LCP (mobile)	4.8 s	1.2 s
Total Blocking Time (mobile)	—	~20 ms
Cover image (LCP element)	92 KB PNG	36 KB WebP
Eager JS on the post	658 KB	583 KB
Render-blocking CSS requests	1	0
CLS	0.00	0.00
3rd-party scripts on initial load	4 (GA + Clarity + Logichat + Giscus)	0

CLS was 0.00 throughout. The combined effect — zoneless framework, zero third-party on load, WebP covers, inlined CSS, preloaded LCP image — took mobile from 59 to a perfect 100, matching desktop's 100/100/100/100. There was no real "framework ceiling"; the mobile gap was a render-blocking CSS round-trip and a late-discovered image all along.

Summary

Eight failing audits across four Lighthouse categories, now zero. The fixes in rough order of impact:

Gate analytics on click/keydown/touchstart — Best Practices 77 → 100. Resist the scroll-deferral instinct; audit tools auto-scroll.
Fix publicDir to deploy robots.txt — SEO 92 → 100. One file in the right directory.
Override the Prism light theme for WCAG AA — 76 of 78 contrast violations cleared with a single CSS block.
Label the range input, fix heading order, clean up button aria labels — Accessibility 90 → 100.
Add /llms.txt — Agentic Browsing 33 → 100.
Lazy-load Giscus + fetchpriority on LCP image — zero third-party scripts on initial load, LCP stays green.
Zoneless change detection + trim the eager bundle (drop zone.js, inline-SVG icons instead of Angular Material, lazy Sentry) — mobile Performance 66 → 77, eager JS 658 → 583 KB, TBT ~20 ms.
WebP cover images via <picture> — covers 92% smaller (12.2 MB → 0.9 MB), mobile 77 → 78, LCP 5.8 s → ~4.8 s.
Inline the global CSS + preload the LCP image — FCP 2.1 s → 0.9 s, LCP 4.8 s → 1.2 s, mobile Performance 78 → 100.

The audit fixes were small and surgical; going zoneless was the biggest single TBT win; but the real surprise was the last step — mobile 78 → 100 from inlining a 9 KB stylesheet and preloading one image. The recurring lesson: measure before reaching for the heavy tool. Scroll-gated deferral failed because audit tools auto-scroll; critical-CSS extraction was unnecessary because the CSS was tiny. The page that started at 90/77/92/33 desktop now sits at a perfect 100/100/100/100 on both desktop and mobile.

Run Coding Agents on Local AI — Zero Cloud, Full Control

Dale Nguyen — Sun, 07 Jun 2026 00:42:42 +0000

Coding agents — Codex CLI, Claude Code, Cursor, and Pi — are productivity multipliers. But they all assume you are happy sending your code to someone else's servers. For many of us that is a deal-breaker: proprietary codebases, client NDAs, compliance requirements, or just the principle of owning your own compute.

This guide shows how to swap out every cloud API with a local Ollama server running qwen3-coder:30b. Same tools, same workflows, no data leaving your network.

Why Run AI Locally?

The case is simple:

Zero data exfiltration. Your code never leaves your machine or LAN.
No per-token cost. Run 10,000 completions or 10 — the electricity bill does not care.
Works offline. Airplane mode, restricted network, flaky VPN — irrelevant.
No rate limits. No 429s at 2 am when you are in flow.

The honest tradeoff: frontier models (Claude Opus 4, GPT-5) still outperform local models on complex multi-step reasoning and very large context tasks. For the 80% of day-to-day coding work — autocomplete, refactors, test generation, documentation — a well-chosen local model is more than good enough.

Hardware Requirements

I run this on an Apple M4 Pro with 48 GB unified memory. Apple Silicon's unified memory architecture is exceptionally well-suited to LLM inference: the GPU and CPU share the same memory pool, so a 22 GB model fits comfortably alongside a full development environment.

Minimum viable setup:

RAM	What fits
16 GB	7–8B parameter models (qwen3:8b, llama3.2:8b)
32 GB	14–20B models (qwen3:14b, gpt-oss:20b)
48 GB	30–35B models (qwen3-coder:30b, qwen3.6:35b)
64 GB+	70B models (deepseek-r1:70b, llama3.3:70b)

On Intel/AMD systems with discrete GPUs the math is different: VRAM is the bottleneck, and models that don't fit entirely in VRAM fall back to slow CPU offloading.

Choosing a Model

For 48 GB unified memory, these are the models worth knowing about:

Model	Size on disk	Active params	Strengths
qwen3-coder:30b	~22 GB	3.3B (MoE)	Coding, 256K context, HumanEval SOTA
qwen3.6:35b	~24 GB	Full dense	General reasoning + vision
gpt-oss:20b	~14 GB	Full dense	Function calling, tool use
gemma4:27b	~18 GB	Full dense	Math, structured output
deepseek-r1:70b	~45 GB	Full dense	Chain-of-thought, complex reasoning

qwen3-coder:30b is the default recommendation for coding tasks. It uses a Mixture-of-Experts architecture — only 3.3B parameters are active per token — so inference is fast despite the large parameter count. The 256K context window handles entire codebases without chunking. It beats GPT-4o on HumanEval benchmarks.

Pull it with Ollama:

ollama pull qwen3-coder:30b

Setting Up Ollama as a Network Server

By default Ollama listens on localhost only. To reach it from other machines on your LAN (or to let coding tools that open their own network connections reach it), bind to all interfaces:

OLLAMA_HOST=0.0.0.0 ollama serve

To make this permanent on macOS, edit the Ollama launch agent or set the environment variable in your shell profile before starting Ollama. The server will then be reachable at:

http://192.168.2.200:11434

Replace 192.168.2.200 with your machine's LAN IP. Verify it is working:

curl http://192.168.2.200:11434/api/tags | jq '.models[].name'

Ollama exposes an OpenAI-compatible /v1 endpoint, which is what all the tools below use.

Codex CLI

Codex CLI is OpenAI's terminal-based coding agent. It supports custom model providers through its TOML configuration.

Install

npm install -g @openai/codex

Configuration

Create ~/.codex/config.toml:

model = "qwen3-coder:30b"
model_provider = "ollama_remote"
model_context_window = 262144
model_catalog_json = "/Users/me/.codex/model_catalog.json"

[model_providers.ollama_remote]
name = "Ollama Remote"
base_url = "http://192.168.2.200:11434/v1"
env_key = "OLLAMA_API_KEY"

A few gotchas discovered the hard way:

Provider ID must use underscores, not hyphens. ollama-remote fails with a parse error; ollama_remote works.
name is required in [model_providers.*]. Omitting it throws provider name must not be empty.
ollama, openai, and lmstudio are reserved built-in provider IDs and cannot be overridden. Always use a custom name like ollama_remote.
model_context_window must be set explicitly — Codex won't infer it for unknown models.

Set the API key environment variable (Ollama doesn't require auth, but Codex won't start without it):

export OLLAMA_API_KEY=ollama

Model Catalog

Without a model catalog, Codex prints Model metadata for qwen3-coder:30b not found and falls back to broken defaults. The catalog format requires every field from Codex's bundled schema — a simplified JSON with just a few keys will fail with missing field errors.

The cleanest approach: generate the catalog from Codex's own bundled metadata and patch in your model:

codex debug models --bundled | python3 -c "
import json, sys
d = json.load(sys.stdin)
m = d['models'][0].copy()
m['slug'] = 'qwen3-coder:30b'
m['display_name'] = 'Qwen3-Coder 30B'
m['description'] = 'Coding-specialized MoE model with 256K context.'
m['context_window'] = 262144
m['max_context_window'] = 262144
m['availability_nux'] = None
m['upgrade'] = None
m['supported_reasoning_levels'] = []
m['default_reasoning_level'] = 'low'
m['supports_reasoning_summaries'] = False
m['default_reasoning_summary'] = 'none'
print(json.dumps({'models': [m]}, indent=2))
" > ~/.codex/model_catalog.json

The two critical fields are supported_reasoning_levels: [] and supports_reasoning_summaries: false. Without them, Codex sends a thinking parameter that Ollama rejects with does not support thinking. Note that qwen3-coder:30b does support chain-of-thought reasoning — Qwen3 models reason internally via <think> tags. Disabling this API parameter just stops Codex from requesting it in an OpenAI-specific format that Ollama doesn't accept.

Verify the catalog loaded correctly:

OLLAMA_API_KEY=ollama codex debug models | python3 -c "
import json, sys
d = json.load(sys.stdin)
m = [x for x in d['models'] if 'qwen3-coder' in x['slug']][0]
print('slug:', m['slug'], '| context_window:', m['context_window'])
print('reasoning_levels:', m['supported_reasoning_levels'])
"
# slug: qwen3-coder:30b | context_window: 262144
# reasoning_levels: []

Running Codex

OLLAMA_API_KEY=ollama codex

Or add it permanently to ~/.zshrc:

export OLLAMA_API_KEY=ollama

Then just run codex from any project directory.

Claude Code

Claude Code is Anthropic's official CLI agent. It is hardwired to the Anthropic API but accepts a base URL override — which means you can point it at any OpenAI-compatible endpoint, including Ollama.

Configuration

Set two environment variables before launching Claude Code:

export ANTHROPIC_BASE_URL=http://192.168.2.200:11434
export ANTHROPIC_API_KEY=ollama

Start Claude Code:

claude

At the login prompt, select "Anthropic Console" as the login method. Claude Code will use the base URL you provided instead of api.anthropic.com.

To make this permanent, add the exports to your shell profile (~/.zshrc, ~/.bashrc):

# Local AI backend for Claude Code
export ANTHROPIC_BASE_URL=http://192.168.2.200:11434
export ANTHROPIC_API_KEY=ollama

Then reload:

source ~/.zshrc

One practical note: Claude Code's system prompts are written for Claude models and include Anthropic-specific formatting expectations. qwen3-coder:30b handles them well, but you may see occasional formatting quirks in responses. They do not affect functionality.

Cursor

Cursor has a similar configuration path. In Settings → Models → OpenAI API Key, switch to a custom base URL:

Open Cursor Settings (Cmd+,).
Navigate to Models.
Enable "Override OpenAI Base URL".
Enter http://192.168.2.200:11434/v1.
Enter ollama as the API key.
Select or type qwen3-coder:30b as the model.

Pi (pi.dev)

Pi is a minimal agent harness built for extensibility — "adapt Pi to your workflows, not the other way around." It supports 15+ providers and custom local endpoints via a models.json file that hot-reloads between sessions.

Install

npm install -g @pi-ag/coding-agent

Configuration

Add your local Ollama server to ~/.pi/agent/models.json:

{
  "providers": {
    "ollama_remote": {
      "baseUrl": "http://192.168.2.200:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        {
          "id": "qwen3-coder:30b",
          "contextWindow": 262144,
          "compat": {
            "supportsDeveloperRole": false,
            "supportsReasoningEffort": false
          }
        }
      ]
    }
  }
}

The compat block is important: Ollama doesn't understand the developer role or reasoning_effort parameter that Pi sends to reasoning-capable models by default. Setting both to false routes around those errors.

Running Pi

pi

Select the model with /model inside the session — it lists all providers including your custom ollama_remote entry. The models.json file reloads each time you open /model, so you can add or swap models without restarting.

Tradeoffs to Know Before Committing

Being honest about the limitations matters more than selling this as a perfect replacement.

Where qwen3-coder:30b matches or beats cloud models:

Single-file refactors and rewrites
Test generation from existing code
Documentation and comment generation
Code review and explaining what code does
SQL queries, shell scripts, configuration files
Repetitive boilerplate

Where frontier models still have an edge:

Multi-file architecture changes requiring deep cross-file reasoning
Tasks requiring the model to hold very large context simultaneously (though 256K helps significantly)
Novel algorithm design in complex domains
Catching subtle security vulnerabilities in unfamiliar code patterns

Operational considerations:

First inference after loading the model takes a few seconds. Subsequent calls are fast.
If your Mac goes to sleep, Ollama suspends. You may want to keep "Prevent computer from sleeping" on while working.
The Ollama server is unauthenticated by default. Keep it on your LAN only — do not expose port 11434 to the internet.

A Note on Model Selection for Other Tasks

If qwen3-coder:30b is not the right fit for a specific task, here is when to switch:

Vision tasks (screenshots, diagrams in context): use qwen3.6:35b — it has multimodal support.
Heavy function/tool calling in agentic workflows: gpt-oss:20b has more reliable structured output.
Math or formal reasoning: gemma4:27b has strong performance on reasoning benchmarks.
Chain-of-thought problems where you want to see the reasoning trace: deepseek-r1:70b (needs 45+ GB free RAM).

Switching models in Ollama is instant — just pull the model and update the model field in your config.

Conclusion

Replacing cloud APIs with a local Ollama server is a one-afternoon project that delivers permanent benefits: no cost, no data exposure, no rate limits. The setup is three configuration files and two environment variables.

qwen3-coder:30b is capable enough that you will not miss the cloud for most coding work. When you do need frontier-level reasoning, the cloud is still there — but now it is opt-in, not the default.

The key insight is that your hardware, your code, and your workflow should stay under your control. The tools were always willing to connect to any compatible endpoint. Now you know how to give them one that you own.

Local-first: a Model on Your Own Machine, Zero Cloud

Dale Nguyen — Sat, 30 May 2026 18:27:54 +0000

This is the concrete, runnable walkthrough for Post 1 of the Portway series. The goal: stand up a single model behind an OpenAI-compatible endpoint on hardware you already own, call it from the official OpenAI SDK, and internalize the stateless contract. Everything here runs locally for $0.

What this post covers

A demo.py script with two blocks:
1. Round-trip — one chat call via the OpenAI SDK, printing the content and the usage object.
2. Stateless proof — the same final question sent as a 1-turn message and as the last turn of a 5-turn fabricated history; both prompt_tokens values are printed alongside an explanation of the delta.

Engine choice on this machine

Apple Silicon Mac, 48 GB unified memory, Ollama already installed. The demo uses Ollama's OpenAI-compatible endpoint at http://localhost:11434/v1 and the gpt-oss:20b model (~14 GB).

The wider Portway series uses llama.cpp on Mac (Ollama is called out as problematic for Qwen3.5 in Post 2). For Post 1 — one model, prove the contract — Ollama is fine and already on the box.

Model options by available RAM

The demo script works with any Ollama-served model — just substitute the model name in demo.py. The table below covers machines from 9 GB unified memory upward.

Model	Pull command	Approx size	Min RAM	Notes
`llama3.2:3b`	`ollama pull llama3.2:3b`	~2 GB	8 GB	Fastest; good for testing the contract
`gemma3:4b`	`ollama pull gemma3:4b`	~3 GB	8 GB	Google; solid instruction-following
`mistral:7b`	`ollama pull mistral:7b`	~4.1 GB	8 GB	Classic 7B baseline
`llama3.1:8b`	`ollama pull llama3.1:8b`	~4.7 GB	9 GB	Best quality under 10 GB
`qwen2.5:7b`	`ollama pull qwen2.5:7b`	~4.4 GB	9 GB	Strong at instruction + reasoning
`gpt-oss:20b`	`ollama pull gpt-oss:20b`	~14 GB	24 GB	Used in this post's sample output

On a 9 GB machine, replace gpt-oss:20b in demo.py with llama3.1:8b or qwen2.5:7b — the contract demonstration is identical.

Prerequisites

Ollama running locally (curl -s http://localhost:11434/api/tags should return JSON)
uv installed (uv --version)
The model pulled. This post uses gpt-oss:20b (requires ~24 GB RAM); see Model options by available RAM for lighter alternatives on 9 GB+ machines.

ollama pull llama3.2:3b

Run it

From the repo root:

uv sync                                  # creates .venv at root, installs deps
uv run --project 1-local-first python 1-local-first/demo.py

Sample output

A real run on this machine (M4-class Mac, 48 GB, gpt-oss:20b via Ollama). Numbers will differ with smaller models — prompt_tokens for the same input stays deterministic regardless of model:

============================================================
Block 1 — round-trip via OpenAI SDK against localhost
============================================================
content: Toronto, Vancouver, Montreal.
usage:   CompletionUsage(completion_tokens=43, prompt_tokens=72, total_tokens=115, ...)

============================================================
Block 2 — same final question, 1-turn vs 5-turn history
============================================================
1-turn response: The capital of Canada is **Ottawa**.
5-turn response: The capital of Canada is **Ottawa**, located in the province of Ontario.

1-turn prompt_tokens: 75
5-turn prompt_tokens: 139
delta:                64

Why the delta exists: the server holds NO conversation state between
requests. The 5-turn call's prompt_tokens is higher only because the
client re-sent the full history in the request body. Each call is
evaluated from scratch — history is the client's responsibility.

completion_tokens and the response text will vary run-to-run (sampling is non-deterministic at default temperature). prompt_tokens for the same input is deterministic — 75 and 139 should reproduce.

Notice how the 5-turn response picks up the road-trip context ("located in the province of Ontario") while the 1-turn answer riffs on the bare "Driving." in its prompt — same model, different framing in the client-supplied messages.

The stateless contract, explained

This is the most important concept in the series. Every request to an LLM API — local or cloud — is evaluated from scratch. The server has no memory of previous turns. When you send a multi-turn conversation, you are the one re-sending the full history in the request body. The model sees it all at once.

The server's only "memory" between requests is the prefix cache (a compute optimisation that avoids re-evaluating tokens it has seen before), never conversation state. The cache is invisible to you — from the API contract's perspective, each call is stateless.

Understanding this is the foundation for everything that follows in the series:

Why conversation management belongs in the client, not the server
Why context windows matter for cost and latency
Why streaming usage requires an explicit opt-in (stream_options.include_usage)

Definition of done

OpenAI SDK round-trips against localhost — Block 1 prints a real content and a usage object.
Can explain why 5 turns vs 1 turn changes prompt_tokens while the server remembers nothing — Block 2 prints both numbers and the one-paragraph explanation.

Things worth noting now

Context size eats RAM/VRAM. Ollama's default context window is conservative for most models; raising it (e.g. ollama run llama3.2:3b → /set parameter num_ctx 32768) costs unified memory. It was not changed for this post.

gpt-oss emits a reasoning channel (Harmony format). The engine applies the template; you still get a normal message.content. The reasoning channel will be segregated at the gateway in Post 3.

No streaming yet. Post 5 covers the streaming usage trap — you must opt in via stream_options.include_usage, otherwise usage is null in streamed responses.

What's next

Post 2 moves from a single model to running multiple models simultaneously and routing requests between them — the first step toward a real local gateway.

The full series and all demo code live in the Portway repository.

Free Model Providers to Use with Hermes Agent

Dale Nguyen — Fri, 29 May 2026 04:12:02 +0000

Hermes Agent by NousResearch is an open-source AI agent that does something most frameworks do not: it learns from experience. It creates skills during use, refines its memory across sessions, and supports parallel subagents and a built-in cron scheduler. You can run it on a $5 VPS, a GPU cluster, or serverless infrastructure.

The part that makes it immediately approachable is its provider flexibility. Hermes supports 200+ models across a growing list of providers, and you switch between them with a single command — no code changes. That means you can get started entirely on free tiers before committing to a paid API.

This post covers which free providers work with Hermes Agent and how to get up and running with each.

Installing Hermes Agent

curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
source ~/.bashrc
hermes

That drops you into the interactive CLI. Run hermes setup to walk through the full configuration wizard, or hermes setup --portal to connect via Nous Portal (their own hosted service with OAuth, which consolidates all your API keys into one subscription).

Switching providers

Once installed, switching providers is a single command:

hermes model

Or inline during a conversation: /model [provider:model]

No config file edits, no restarts. This is what makes free-tier exploration practical — you can try a model, hit a rate limit, and switch to another in seconds.

Free providers supported by Hermes Agent

Nous Portal

Nous Portal is Hermes's native provider. It consolidates model access, web search, image generation, and TTS under one subscription. If you want the tightest integration with Hermes and the Nous ecosystem, this is the starting point. Check their current free tier on signup.

OpenRouter

OpenRouter is the most flexible free option. It gives you access to 200+ models — including Llama 4, DeepSeek, Gemma, and Mistral — through a single API key. The free tier covers 27+ models (marked with a :free suffix) at 200 requests per day and 20 requests per minute.

Because Hermes supports OpenRouter natively, you can point it at any of those free models immediately. This is the best option if you want to compare how Hermes behaves across different underlying models without managing multiple accounts.

NVIDIA NIM

NVIDIA NIM gives you free API credits on signup with no credit card required. The catalog includes 80+ models — Llama 4, Qwen, Mistral, and NousResearch's own Hermes 3 model — at roughly 40 requests per minute. Credits do not expire.

NIM is the best free option if you specifically want to run a NousResearch model underneath Hermes Agent, since it hosts the Hermes 3 fine-tune directly.

Hugging Face

Hugging Face Inference Providers gives every account monthly free credits routed across a network of inference providers (Groq, Together, SambaNova, and others). Hermes Agent supports HuggingFace natively.

The free tier is best for trying niche or smaller models. Cold starts can be slow on the free tier, so it is less suitable for long interactive sessions but fine for batch tasks or evaluation.

NovitaAI

NovitaAI is a cost-efficient inference provider with a generous free tier for new accounts. Hermes supports it natively. It is worth trying if OpenRouter and NVIDIA NIM rate limits become a bottleneck during heavy agent development.

Kimi / Moonshot

Kimi (by Moonshot AI) offers a free tier with long-context models. Hermes Agent lists it as a supported provider. Particularly useful if your agent tasks involve processing large documents or long conversation histories that would exhaust the context windows of other free-tier models.

Provider comparison at a glance

Provider	Free Tier	Best for with Hermes
Nous Portal	Check on signup	Tightest Hermes integration
OpenRouter	200 req/day, 27+ free models	Model variety, easy switching
NVIDIA NIM	Credits on signup, no expiry	Running NousResearch models
Hugging Face	Monthly credits	Niche models, batch tasks
NovitaAI	Free credits on signup	Alternative when hitting rate limits
Kimi / Moonshot	Free tier	Long-context tasks

Tips for running Hermes on free tiers

Rate limits compound in agentic loops. Hermes's learning loop and subagent features can make many model calls per task. At 20 req/min (OpenRouter free), a multi-step task hits the ceiling quickly. Use hermes model to switch to a provider with a higher limit mid-session, or configure a fallback provider.

Use NVIDIA NIM for NousResearch models specifically. If you want the model that Hermes was designed around — the Hermes 3 fine-tune — NIM hosts it directly at nousresearch/hermes-3-llama-3.1-70b. The free credits cover meaningful development without hitting a daily request cap.

OpenRouter first for exploration. Its unified free tier across 27+ models is the fastest way to understand how Hermes Agent behaves with different underlying models. No separate accounts, one key, switch with /model.

Run hermes doctor when something breaks. The built-in diagnostics command checks your provider configuration and surfaces misconfigurations before you spend time debugging the wrong thing.

Summary

Hermes Agent is one of the few open-source agents with a genuine learning loop built in, and its multi-provider support means you are not locked into any single API. To get started for free: install Hermes, run hermes setup, and connect OpenRouter or NVIDIA NIM — both work out of the box with no credit card. Switch models with hermes model as you explore what works best for your use case.

Building a Personal Assistant in Zo Computer and Adding Hermes as a Second Assistant

Dale Nguyen — Sun, 24 May 2026 14:03:23 +0000

Zo Computer works well as a personal operating system for life admin, creative work, learning, and day-to-day coordination. If you set it up intentionally, it can feel less like a chat app and more like a real assistant that lives with your files, routines, and preferences.

The good news is that getting started costs nothing. Zo has a free tier that is a perfect place to test a personal assistant setup before committing to anything. On the Hermes side, NVIDIA offers free API credits through their developer platform, so you can run Hermes against a capable model without paying for API usage upfront. That means you can build out the full two-assistant setup described in this post for free.

A strong pattern is to use Zo as your main personal assistant and Hermes as a second assistant that handles a different lane of work. Zo stays close to your files, your rules, your automations, and your personal workflows. Hermes becomes the extra pair of hands: a separate agent for focused tasks, parallel research, and longer-running jobs.

Why a personal assistant in Zo is useful

Zo is especially good for personal use because it can sit near the center of your life instead of scattered across separate apps. That means you can keep the things an assistant needs in one place:

notes and documents
recurring routines
calendar-aware tasks
saved preferences and rules
files you want the assistant to work from
automations that run without you micromanaging them

The real advantage is continuity. A personal assistant becomes much more useful when it can remember your style, keep track of ongoing projects, and work from the same environment every day.

What "personal assistant" should mean in Zo

A good assistant in Zo should do more than answer questions. It should help you actually move things forward.

That usually means:

drafting messages, notes, and documents
organizing ideas into something usable
turning vague thoughts into a plan
reminding you about routines and deadlines
summarizing information from files or the web
handling repeatable life admin tasks
keeping track of preferences so you do not repeat yourself

If you treat Zo like a small command center, it stops being a generic chatbot and starts becoming a trusted helper.

The cleanest setup: one main assistant, one second assistant

The best setup is usually not "one AI that does everything." It is:

Zo as your primary assistant
Hermes as your second assistant

That split is useful because the assistants can have different jobs.

Zo as the primary assistant

Use Zo for:

personal life management
writing and editing
working directly from your files
scheduling and recurring routines
decisions that need your preferences
things you want tightly integrated with your Zo workspace

Hermes as the second assistant

Use Hermes for:

parallel research
long-running work
independent task handling
a separate memory stream
a different working style
conversations or jobs you want isolated from your main assistant

This is the important idea: the second assistant should not be a clone of the first. It should be a specialist.

How Hermes fits in

Hermes is designed as an autonomous agent with a built-in learning loop. According to its documentation, it can keep improving through use, remember across sessions, and work through messaging platforms like Telegram. It is meant to live somewhere persistent, such as a local machine, a VPS, or another hosted environment.

That makes Hermes a good second assistant when you want another agent running in parallel with Zo instead of replacing Zo.

Basic Hermes installation path

The easiest way to get started is to simply ask Zo to do it for you. Open a Zo chat and type something like:

"Install Hermes on my machine"

Zo will handle the setup steps, walk you through any configuration it needs, and get Hermes running without you having to touch the terminal. This is the fastest path if you are already inside Zo.

If you prefer to install manually, the official Hermes docs show a straightforward install flow for Linux, macOS, and WSL2:

curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash

For native Windows, the docs also provide a PowerShell installer. After installation, Hermes can be configured to run in the environment you want and connected to messaging tools like Telegram.

The practical setup pattern is:

install Hermes
choose where it will run
connect its messaging or command interface
give it a clear role
keep Zo as the main personal assistant

How to divide work between Zo and Hermes

If you want the setup to feel sane, give each assistant a clear job.

Zo handles the personal layer

Zo should own:

your personal notes
your tasks and routines
files you keep locally
preferences and recurring habits
writing that needs your voice
decisions that depend on your context

Hermes handles the parallel layer

Hermes should own:

research you want to run in the background
tasks that can be broken into substeps
experiments
alternate drafts or viewpoints
work that benefits from isolation

This division prevents the assistants from stepping on each other.

A simple working model

Here is a practical way to think about it:

Zo is the assistant that knows you.
Hermes is the assistant that helps you scale.

Zo keeps your life organized. Hermes helps you get more done.

That combination is especially useful when you are juggling creative work, learning, and admin at the same time. One assistant can stay close to your personal system while the other handles the overflow.

Tips for making the setup actually useful

1. Give each assistant a role

Do not let both assistants do the same job. That creates noise. Be explicit:

Zo = personal coordinator
Hermes = second operator

2. Keep preferences in one place

The more you repeat yourself, the less useful the system feels. Store your standards, defaults, and routines where the assistant can reuse them.

3. Use the assistant for real workflows

Do not use it only for chat. Make it draft, organize, summarize, and schedule. Real utility comes from repeatable workflows.

4. Keep the assistant close to your files

Your files are the memory of the system. The assistant gets more useful when it can work from documents, notes, and living records instead of starting from scratch each time.

5. Treat the second assistant like an aide, not a boss

Hermes should support your system, not replace your judgment. The best assistant setup is still human-led.

Example use case

A good two-assistant workflow might look like this:

Zo manages your personal planning, drafts, and routines.
Hermes runs a background research task, collects notes, and returns a concise summary.
You review both outputs and decide what to do next.

That is the sweet spot: one assistant keeps your life coherent, and the other adds throughput.

Bottom line

If you want a personal assistant inside Zo Computer, start by making Zo the center of your daily system: files, rules, routines, drafts, and reminders. Then add Hermes as a second assistant for parallel work, deeper automation, and independent tasks.

The result is not just "two AIs." It is a better division of labor:

Zo for your personal operating system
Hermes for your extra capacity

That is the setup that actually feels like having help.

If you want to try Zo Computer, you can sign up using my invite link (affiliate link).

Securing Your MCP Server with Firebase Auth: A Production Walkthrough

Dale Nguyen — Fri, 15 May 2026 16:10:31 +0000

Model Context Protocol (MCP) servers let AI assistants interact with real user data. That means auth isn't optional — it's the difference between a useful tool and a data breach. This post walks through exactly how Can Tax Pro secures its Python MCP server with Firebase Authentication, supporting both Firebase ID tokens (for direct access) and a custom OAuth 2.0 flow (for third-party clients like Claude.ai).

Architecture Overview

The system has three moving parts:

Browser / Claude.ai Client
       │
       │  Authorization: Bearer <token>
       ▼
  MCP Server (Python/FastMCP on Cloud Run)
       │
       │  Firebase Admin SDK
       ▼
  Firestore (data isolated by userId)

The MCP server accepts two token types:

Firebase ID tokens — issued by Firebase Authentication, verified cryptographically
Custom OAuth tokens (ctpo_*) — issued by the web app's OAuth server, stored as hashes in Firestore

The web app itself acts as the OAuth authorization server for third-party integrations.

Step 1: Initialize Firebase Admin SDK

The server initializes Firebase Admin once at startup, with environment-aware credential resolution:

# main.py
import firebase_admin
from firebase_admin import credentials, auth as firebase_auth, firestore

if not firebase_admin._apps:
    sa_json = os.environ.get("FIREBASE_SERVICE_ACCOUNT")
    project_id = os.environ.get("FIREBASE_PROJECT_ID")
    if sa_json:
        cred = credentials.Certificate(json.loads(sa_json))
        firebase_admin.initialize_app(cred, {"projectId": project_id})
    else:
        firebase_admin.initialize_app(options={"projectId": project_id})

Locally: set FIREBASE_SERVICE_ACCOUNT to your service account JSON.
On Cloud Run: omit it entirely — the SDK picks up Application Default Credentials (ADC) automatically via Workload Identity.

This means no secrets in production. Your Cloud Run service account just needs the Firebase Admin SDK Administrator Service Agent IAM role.

Step 2: Token Resolution

Two resolver functions handle each token type:

OAuth Tokens (`ctpo_*`)

Custom tokens are never stored in plaintext. The server hashes them with SHA-256 and looks up the hash in Firestore:

def resolve_oauth_token(bearer_token: str) -> str:
    token_hash = hashlib.sha256(bearer_token.encode()).hexdigest()
    doc = db.collection("oauthTokens").document(token_hash).get()
    if not doc.exists:
        raise ValueError("Invalid or revoked OAuth token")
    data = doc.to_dict()
    expires_at = data.get("expiresAt")
    if expires_at and expires_at < datetime.now(timezone.utc):
        raise ValueError("OAuth token expired")
    return data["userId"]

The Firestore document stores userId, expiresAt, clientId, and a refreshTokenHash. Revocation is instant — delete the document and the token stops working on the next request.

Firebase ID Tokens

Firebase handles the hard part:

def resolve_id_token(id_token: str) -> str:
    decoded = firebase_auth.verify_id_token(id_token)
    return decoded["uid"]

verify_id_token checks the signature against Google's public keys and validates claims (expiry, issuer, audience). It caches the public keys locally so it doesn't make a network call on every request.

Step 3: The Auth Middleware

A Starlette BaseHTTPMiddleware wraps every request. It tries the OAuth path first, then falls back to Firebase ID tokens:

class AuthMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next):
        # Skip auth for public endpoints
        if request.url.path in _PUBLIC_PATHS:
            return await call_next(request)

        auth_header = request.headers.get("Authorization", "")
        if not auth_header.startswith("Bearer "):
            return Response(
                status_code=401,
                headers={"WWW-Authenticate": 'Bearer realm="tax-mcp"'},
            )

        token = auth_header.removeprefix("Bearer ").strip()
        token_set = _user_id_var.set("")

        try:
            # Try OAuth token first
            if token.startswith("ctpo_"):
                user_id = resolve_oauth_token(token)
            else:
                user_id = resolve_id_token(token)
            _user_id_var.set(user_id)
            return await call_next(request)
        except Exception as e:
            return Response(
                status_code=401,
                content=str(e),
                headers={"WWW-Authenticate": 'Bearer realm="tax-mcp"'},
            )
        finally:
            _user_id_var.reset(token_set)

Public paths (/health, /.well-known/oauth-protected-resource) bypass auth — necessary for health checks and OAuth discovery.

Step 4: Thread-Safe User Context

Tools shouldn't take a user_id parameter — that would pollute every signature and make testing awkward. Instead, use a ContextVar to propagate identity through the async call stack:

# _context.py
from contextvars import ContextVar

_user_id_var: ContextVar[str] = ContextVar("user_id")

def get_user_id() -> str:
    try:
        return _user_id_var.get()
    except LookupError:
        raise RuntimeError("User context not set — request did not pass through auth middleware.")

Every tool calls get_user_id() to scope its Firestore queries:

# tools/income.py
from .._context import get_user_id

@mcp.tool()
def list_income(tax_year_id: str) -> list[dict]:
    docs = (
        db.collection("users")
          .document(get_user_id())
          .collection("taxYears")
          .document(tax_year_id)
          .collection("incomeEntries")
          .stream()
    )
    return [{"id": d.id, **d.to_dict()} for d in docs]

ContextVar is async-safe — each concurrent request gets its own context, so there's no cross-contamination between users even under high concurrency.

The middleware resets the var in a finally block, which is critical: without cleanup, the context leaks to the next request on a reused coroutine.

Step 5: The OAuth 2.0 Server (Web App Side)

For third-party clients (Claude.ai, etc.), the web app implements an OAuth 2.0 authorization server. Here's the flow:

1. Client → GET /oauth/authorize?client_id=...&code_challenge=...
2. User logs in (Firebase Auth)
3. Server → redirect with authorization_code
4. Client → POST /oauth/token with code + code_verifier
5. Server → { access_token: "ctpo_...", refresh_token: "ctpr_..." }

Token generation (TypeScript, server-side):

// server/routes/oauth/token.post.ts
import { createHash, randomBytes } from "crypto";

const accessToken = "ctpo_" + randomBytes(32).toString("hex");
const tokenHash = createHash("sha256").update(accessToken).digest("hex");

await db.collection("oauthTokens").doc(tokenHash).set({
  userId: session.userId,
  clientId: session.clientId,
  expiresAt: new Date(Date.now() + 60 * 60 * 1000), // 1 hour
  createdAt: new Date(),
});

return { access_token: accessToken, token_type: "bearer", expires_in: 3600 };

The ctpo_ prefix lets the MCP server route to the right resolver without trying Firebase first. Only the hash ever touches the database.

PKCE (Proof Key for Code Exchange) protects the authorization code flow. The client sends a code_challenge (SHA-256 of a random code_verifier) in step 1, then proves ownership by sending the raw code_verifier in step 4. The server hashes it and compares:

const verifierHash = createHash("sha256")
  .update(body.code_verifier)
  .digest("base64url");

if (verifierHash !== session.codeChallenge) {
  throw createError({ statusCode: 400, message: "Invalid code_verifier" });
}

Step 6: Client-Side (Browser)

The Angular/Analog web app attaches Firebase ID tokens to outbound API requests via an HTTP interceptor:

// auth.interceptor.ts
export const authInterceptor: HttpInterceptorFn = (req, next) => {
  const auth = inject(Auth);
  return from(auth.currentUser?.getIdToken() ?? Promise.resolve(null)).pipe(
    switchMap((token) => {
      if (!token) return next(req);
      return next(req.clone({
        setHeaders: { Authorization: `Bearer ${token}` },
      }));
    }),
  );
};

getIdToken() auto-refreshes the token when it's close to expiry, so you never send a stale JWT.

Step 7: Firestore Security Rules

The MCP server uses the Admin SDK, which bypasses all Firestore security rules. The get_user_id() context is your authorization layer. But rules provide defense-in-depth for direct client SDK access:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /users/{userId}/{document=**} {
      allow read, write: if request.auth != null && request.auth.uid == userId;
    }
    match /oauthTokens/{tokenId} {
      allow read, write: if false; // Server-only
    }
  }
}

OAuth token documents are locked to server-only access. Users can never read or write them directly.

Step 8: OAuth Discovery Endpoint

Well-behaved OAuth clients (including Claude.ai) auto-discover server capabilities. Expose the RFC 9728 metadata endpoint:

@app.get("/.well-known/oauth-protected-resource")
async def oauth_protected_resource():
    return JSONResponse({
        "resource": "https://your-mcp-server.run.app",
        "authorization_servers": ["https://your-web-app.com"],
        "bearer_methods_supported": ["header"],
    })

This tells clients where to find the authorization server and how to present tokens. It's a public endpoint (no auth required) — make sure it's in _PUBLIC_PATHS.

Step 9: Local Testing

Don't rely on a real Firebase project for unit tests. Seed a test token directly:

# test_auth.py
import hashlib
from firebase_admin import firestore
from datetime import datetime, timezone, timedelta

TEST_TOKEN = "ctpo_localtest_" + "a" * 48
TEST_TOKEN_HASH = hashlib.sha256(TEST_TOKEN.encode()).hexdigest()

def seed_test_token(db, user_id: str):
    db.collection("oauthTokens").document(TEST_TOKEN_HASH).set({
        "userId": user_id,
        "clientId": "test-client",
        "expiresAt": datetime.now(timezone.utc) + timedelta(hours=1),
    })

def cleanup(db):
    db.collection("oauthTokens").document(TEST_TOKEN_HASH).delete()

Then test the three critical paths: no token → 401, invalid token → 401, valid token → 200.

Security Properties

Property	How it's achieved
Token confidentiality	Only SHA-256 hashes stored; raw tokens never persist
Revocation	Delete Firestore doc; effective immediately
Expiry	Enforced server-side on every request
Multi-user isolation	`ContextVar` scopes every Firestore query to `userId`
No key management in prod	Cloud Run Workload Identity + ADC
Code injection protection	PKCE on OAuth code exchange
Defense in depth	Firestore rules lock down client SDK access

Summary

Securing an MCP server with Firebase Auth comes down to four things:

Middleware that validates tokens before any tool runs
ContextVar to propagate user identity without polluting tool signatures
Hash-only storage for custom OAuth tokens
Workload Identity to eliminate service account key management in production

The dual-token design (Firebase ID tokens for direct use, custom OAuth tokens for third-party clients) keeps the server flexible while maintaining a single, auditable auth path. Every request either has a valid, unexpired token mapping to a real user, or it gets a 401.

Building MCP Apps with Angular

Dale Nguyen — Sat, 25 Apr 2026 03:43:44 +0000

If you've been building MCP servers, you know the drill: your tool returns JSON, the host renders it as text, and the user squints at a timestamp string. MCP Apps change that — they let your server ship an interactive UI that the host renders in an iframe, right inside the conversation.

MCP Apps are built on the Model Context Protocol — an open standard. They're not tied to Claude or any specific AI provider. Any host that implements the MCP Apps specification (Claude Desktop, custom chat clients, or other AI assistants that adopt MCP) can render your UI. You build it once, and it works everywhere MCP is supported.

This post walks through building MCP Apps with Angular. We'll start with a single tool, add a second tool with its own UI, and then show how to share code between them without bloating either bundle.

How MCP Apps Work (Quick Recap)

View (Angular App) <--PostMessageTransport--> Host (AppBridge) <--MCP Client--> MCP Server

Server registers tools and resources. Each tool can point to a resource URI containing the UI.
Host (the chat client) fetches that resource and renders it in a sandboxed iframe.
View is your Angular app running inside that iframe. It uses the App class from @modelcontextprotocol/ext-apps to communicate with the host.

The key insight: your UI is bundled into a single self-contained HTML file using Vite and vite-plugin-singlefile. The host doesn't need to know it's Angular — it just loads HTML.

Project Structure

basic-server-angular/
├── mcp-app.html              # HTML entry point for UI #1
├── greeting-app.html         # HTML entry point for UI #2
├── src/
│   ├── main.ts               # Angular bootstrap for UI #1
│   ├── app.component.ts      # Get Time component
│   ├── greeting-main.ts      # Angular bootstrap for UI #2
│   ├── greeting.component.ts # Greeting component
│   ├── shared/
│   │   └── mcp-app-setup.ts  # Shared App + theming setup
│   └── global.css            # Host-aware CSS variables
├── server.ts                 # MCP server (registers tools + resources)
├── main.ts                   # Server entry point (HTTP + stdio)
└── vite.config.ts            # Builds each HTML into a single file

Step 1: The Server

Every MCP App starts on the server side. You register a tool (what the LLM calls) and a resource (the HTML that gets rendered). They're linked by a resource URI.

// server.ts
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import type { CallToolResult, ReadResourceResult } from "@modelcontextprotocol/sdk/types.js";
import fs from "node:fs/promises";
import path from "node:path";
import {
  registerAppTool,
  registerAppResource,
  RESOURCE_MIME_TYPE,
} from "@modelcontextprotocol/ext-apps/server";

const DIST_DIR = import.meta.filename.endsWith(".ts")
  ? path.join(import.meta.dirname, "dist")
  : import.meta.dirname;

export function createServer(): McpServer {
  const server = new McpServer({
    name: "Basic MCP App Server (Angular)",
    version: "1.0.0",
  });

  const resourceUri = "ui://get-time/mcp-app.html";

  // Register the tool — this is what the LLM calls
  registerAppTool(server, "get-time", {
    title: "Get Time",
    description: "Returns the current server time as an ISO 8601 string.",
    inputSchema: {},
    _meta: { ui: { resourceUri } }, // Links this tool to its UI
  }, async (): Promise<CallToolResult> => {
    const time = new Date().toISOString();
    return { content: [{ type: "text", text: time }] };
  });

  // Register the resource — the bundled HTML for this tool's UI
  registerAppResource(server, resourceUri, resourceUri, {
    mimeType: RESOURCE_MIME_TYPE,
  }, async (): Promise<ReadResourceResult> => {
    const html = await fs.readFile(
      path.join(DIST_DIR, "mcp-app.html"),
      "utf-8",
    );
    return {
      contents: [{ uri: resourceUri, mimeType: RESOURCE_MIME_TYPE, text: html }],
    };
  });

  return server;
}

The _meta.ui.resourceUri is the glue. When the host calls this tool, it reads that field to know which resource to fetch and render.

Step 2: The HTML Entry Point

Each UI needs an HTML file at the project root. This is the Vite entry point that gets bundled into a single self-contained file.

<!-- mcp-app.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta name="color-scheme" content="light dark">
  <title>Get Time App</title>
  <link rel="stylesheet" href="/src/global.css">
</head>
<body>
  <app-root></app-root>
  <script type="module" src="/src/main.ts"></script>
</body>
</html>

Step 3: The Angular App

The bootstrap is minimal — Angular 19+ with zoneless change detection:

// src/main.ts
import "@angular/compiler";
import { bootstrapApplication } from "@angular/platform-browser";
import { provideZonelessChangeDetection } from "@angular/core";
import { AppComponent } from "./app.component";
import "./global.css";

bootstrapApplication(AppComponent, {
  providers: [provideZonelessChangeDetection()],
}).catch((err) => console.error(err));

Now the component itself. The App class from @modelcontextprotocol/ext-apps is the bridge between your Angular code and the host:

// src/app.component.ts
import { Component, type OnInit, signal } from "@angular/core";
import {
  App,
  applyDocumentTheme,
  applyHostStyleVariables,
  applyHostFonts,
  type McpUiHostContext,
} from "@modelcontextprotocol/ext-apps";
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";

function extractText(result: CallToolResult): string {
  return result.content?.find((c) => c.type === "text")!.text;
}

@Component({
  selector: "app-root",
  template: `
    <main
      [style.padding-top.px]="hostContext()?.safeAreaInsets?.top"
      [style.padding-right.px]="hostContext()?.safeAreaInsets?.right"
      [style.padding-bottom.px]="hostContext()?.safeAreaInsets?.bottom"
      [style.padding-left.px]="hostContext()?.safeAreaInsets?.left"
    >
      <p><strong>Server Time:</strong> <code>{{ serverTime() }}</code></p>
      <button (click)="handleGetTime()">Get Server Time</button>
    </main>
  `,
})
export class AppComponent implements OnInit {
  private app: App | null = null;
  hostContext = signal<McpUiHostContext | undefined>(undefined);
  serverTime = signal("Loading...");

  async ngOnInit() {
    const instance = new App({ name: "Get Time App", version: "1.0.0" });

    instance.ontoolresult = (result) => {
      this.serverTime.set(extractText(result));
    };

    instance.onhostcontextchanged = (params) => {
      const ctx = { ...this.hostContext(), ...params };
      this.hostContext.set(ctx);
      if (ctx.theme) applyDocumentTheme(ctx.theme);
      if (ctx.styles?.variables) applyHostStyleVariables(ctx.styles.variables);
      if (ctx.styles?.css?.fonts) applyHostFonts(ctx.styles.css.fonts);
    };

    await instance.connect();
    this.app = instance;

    const ctx = instance.getHostContext();
    this.hostContext.set(ctx);
    if (ctx?.theme) applyDocumentTheme(ctx.theme);
    if (ctx?.styles?.variables) applyHostStyleVariables(ctx.styles.variables);
    if (ctx?.styles?.css?.fonts) applyHostFonts(ctx.styles.css.fonts);
  }

  async handleGetTime() {
    if (!this.app) return;
    try {
      const result = await this.app.callServerTool({
        name: "get-time",
        arguments: {},
      });
      this.serverTime.set(extractText(result));
    } catch {
      this.serverTime.set("[ERROR]");
    }
  }
}

A few things to note:

App class — this is the SDK's main entry point. You create one, wire up callbacks, and call connect(). That's it.
ontoolresult — fires when the host sends a tool result. This is how data flows from the server to your UI.
callServerTool() — lets the UI call tools on the server. The host proxies this through the MCP client.
onhostcontextchanged — the host pushes theme and style updates. The helper functions (applyDocumentTheme, etc.) apply them as CSS variables on document, so your component styles just work.
safeAreaInsets — the host tells you how much padding to leave for its chrome. Use it on your root container.

Step 4: Adding a Second UI

Here's where it gets interesting. Say you want a "Greet" tool with its own UI. Each tool gets its own HTML entry point, its own Angular app, and its own resource registration.

Server: Register Both Tools

// server.ts — inside createServer()

// Tool #1: Get Time
const timeResourceUri = "ui://get-time/mcp-app.html";
registerAppTool(server, "get-time", {
  title: "Get Time",
  description: "Returns the current server time.",
  inputSchema: {},
  _meta: { ui: { resourceUri: timeResourceUri } },
}, async (): Promise<CallToolResult> => {
  return { content: [{ type: "text", text: new Date().toISOString() }] };
});
registerAppResource(server, timeResourceUri, timeResourceUri, {
  mimeType: RESOURCE_MIME_TYPE,
}, async (): Promise<ReadResourceResult> => {
  const html = await fs.readFile(path.join(DIST_DIR, "mcp-app.html"), "utf-8");
  return { contents: [{ uri: timeResourceUri, mimeType: RESOURCE_MIME_TYPE, text: html }] };
});

// Tool #2: Greet
const greetResourceUri = "ui://greet/greeting-app.html";
registerAppTool(server, "greet", {
  title: "Greet",
  description: "Returns a personalised greeting.",
  inputSchema: {
    name: z.string().optional().default("World").describe("Name to greet"),
  },
  _meta: { ui: { resourceUri: greetResourceUri } },
}, async ({ name }: { name?: string }): Promise<CallToolResult> => {
  const greeting = `Hello, ${name || "World"}! Welcome to the MCP Apps SDK.`;
  return { content: [{ type: "text", text: greeting }] };
});
registerAppResource(server, greetResourceUri, greetResourceUri, {
  mimeType: RESOURCE_MIME_TYPE,
}, async (): Promise<ReadResourceResult> => {
  const html = await fs.readFile(path.join(DIST_DIR, "greeting-app.html"), "utf-8");
  return { contents: [{ uri: greetResourceUri, mimeType: RESOURCE_MIME_TYPE, text: html }] };
});

Greeting Component

// src/greeting.component.ts
import { Component, type OnInit, signal } from "@angular/core";
import { FormsModule } from "@angular/forms";
import {
  App,
  applyDocumentTheme,
  applyHostStyleVariables,
  applyHostFonts,
  type McpUiHostContext,
} from "@modelcontextprotocol/ext-apps";
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";

@Component({
  selector: "greeting-root",
  imports: [FormsModule],
  template: `
    <main
      [style.padding-top.px]="hostContext()?.safeAreaInsets?.top"
      [style.padding-right.px]="hostContext()?.safeAreaInsets?.right"
      [style.padding-bottom.px]="hostContext()?.safeAreaInsets?.bottom"
      [style.padding-left.px]="hostContext()?.safeAreaInsets?.left"
    >
      <div>
        <label><strong>Your name:</strong></label>
        <input type="text" [(ngModel)]="nameText" placeholder="Enter your name">
        <button (click)="handleGreet()">Get Greeting</button>
      </div>
      @if (greeting()) {
        <div class="greeting-display">{{ greeting() }}</div>
      }
    </main>
  `,
})
export class GreetingComponent implements OnInit {
  private app: App | null = null;
  hostContext = signal<McpUiHostContext | undefined>(undefined);
  greeting = signal("");
  nameText = "";

  async ngOnInit() {
    const instance = new App({ name: "Greeting App", version: "1.0.0" });
    instance.ontoolresult = (result) => {
      this.greeting.set(result.content?.find((c) => c.type === "text")!.text);
    };
    instance.onhostcontextchanged = (params) => {
      const ctx = { ...this.hostContext(), ...params };
      this.hostContext.set(ctx);
      if (ctx.theme) applyDocumentTheme(ctx.theme);
      if (ctx.styles?.variables) applyHostStyleVariables(ctx.styles.variables);
      if (ctx.styles?.css?.fonts) applyHostFonts(ctx.styles.css.fonts);
    };
    await instance.connect();
    this.app = instance;
    const ctx = instance.getHostContext();
    this.hostContext.set(ctx);
    if (ctx?.theme) applyDocumentTheme(ctx.theme);
    if (ctx?.styles?.variables) applyHostStyleVariables(ctx.styles.variables);
    if (ctx?.styles?.css?.fonts) applyHostFonts(ctx.styles.css.fonts);
  }

  async handleGreet() {
    if (!this.app) return;
    try {
      const name = this.nameText.trim() || "World";
      const result = await this.app.callServerTool({ name: "greet", arguments: { name } });
      this.greeting.set(result.content?.find((c) => c.type === "text")!.text);
    } catch {
      this.greeting.set("[ERROR]");
    }
  }
}

Step 5: Sharing Code Between UIs

You probably noticed that both components have identical App setup and theming boilerplate. That's a great candidate for extraction — and since each HTML is a separate Vite entry point, Vite's tree-shaking ensures each bundle only includes what it actually imports.

// src/shared/mcp-app-setup.ts
import {
  App,
  applyDocumentTheme,
  applyHostStyleVariables,
  applyHostFonts,
  type McpUiHostContext,
} from "@modelcontextprotocol/ext-apps";
import type { WritableSignal } from "@angular/core";
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";

export function extractText(result: CallToolResult): string {
  return result.content?.find((c) => c.type === "text")!.text;
}

function applyContext(ctx: McpUiHostContext): void {
  if (ctx.theme) applyDocumentTheme(ctx.theme);
  if (ctx.styles?.variables) applyHostStyleVariables(ctx.styles.variables);
  if (ctx.styles?.css?.fonts) applyHostFonts(ctx.styles.css.fonts);
}

export async function createMcpApp(
  name: string,
  hostContext: WritableSignal<McpUiHostContext | undefined>,
  onToolResult?: (result: CallToolResult) => void,
): Promise<App> {
  const app = new App({ name, version: "1.0.0" });
  if (onToolResult) app.ontoolresult = onToolResult;
  app.onhostcontextchanged = (params) => {
    const ctx = { ...hostContext(), ...params } as McpUiHostContext;
    hostContext.set(ctx);
    applyContext(ctx);
  };
  await app.connect();
  const ctx = app.getHostContext();
  hostContext.set(ctx);
  if (ctx) applyContext(ctx);
  return app;
}

Both components import createMcpApp and extractText from the shared module. Since they're in separate Vite builds, tree-shaking still applies — each bundle only pulls in what it calls.

Step 6: The Build

// vite.config.ts
import { defineConfig } from "vite";
import { viteSingleFile } from "vite-plugin-singlefile";

const INPUT = process.env.INPUT;
if (!INPUT) throw new Error("INPUT environment variable is not set");

export default defineConfig({
  plugins: [viteSingleFile()],
  build: {
    rollupOptions: { input: INPUT },
    outDir: "dist",
    emptyOutDir: false, // Key: don't wipe previous builds
  },
});

{
  "scripts": {
    "build": "tsc --noEmit && cross-env INPUT=mcp-app.html vite build && cross-env INPUT=greeting-app.html vite build && tsc -p tsconfig.server.json && bun build server.ts --outdir dist --target node"
  }
}

Each HTML file produces a fully self-contained output (all JS, CSS, and Angular runtime inlined). The two bundles are completely independent.

Theming: Looking Native in Any Host

:root {
  color-scheme: light dark;
  --color-text-primary: light-dark(#1f2937, #f3f4f6);
  --color-background-primary: light-dark(#ffffff, #1a1a1a);
  --color-accent: #2563eb;
  --border-radius-md: 6px;
  --spacing-unit: var(--font-text-md-size);
}

When the host sends style updates via onhostcontextchanged, the helper functions overwrite these variables on the document root. Your Angular component styles reference the variables (var(--color-accent), var(--spacing-md)), so they adapt automatically — no theme prop drilling needed.

Recap

Server: register a tool + resource pair per UI, linked by a resource URI
HTML: one entry point per UI, each bootstrapping its own Angular app
Component: create an App instance, wire up callbacks, call connect()
Shared code: extract common setup into a shared module — Vite tree-shakes per entry point
Build: run Vite once per HTML file into the same dist/ directory
Theming: use host CSS variables with fallbacks, apply updates via onhostcontextchanged

The full source is available in the ext-apps examples.

How I Almost Got Pwned - A Tale of Supply Chain Attacks and GitHub Actions Gone Wrong

Dale Nguyen — Fri, 12 Sep 2025 02:35:15 +0000

Or: "That time someone tried to turn my innocent Node.js repo into a credential-harvesting machine"

So there I was, minding my own business with my trusty old Node.js REST API project, when I noticed something weird in the package-lock.json file from one of a PR from a random contributor - and another contributor actually approved the PR right after that. What started as a routine dependency check turned into a rabbit hole that led me straight into the middle of a supply chain attack campaign that was actively targeting developers like us.

Buckle up, because this is a story about how modern software development can go sideways fast, and why that "helpful" pull request might not be so helpful after all.

What Even Is a Supply Chain Attack?

It’s when hackers sneak bad stuff (malware, credential stealers, you name it) into your dependencies, build scripts, or CI/CD pipelines—any link where code, secrets, and environments touch. Think: “If I can poison upstream, why bother attacking thousands of repos individually?” Welcome to modern risk!

It's insidious because these are the tools we trust implicitly. When lodash releases an update, we don't usually audit every line of code, right? We just npm update and move on with our lives.

The Smoking Gun: The Lockfile Had Been Tampered With

It all started when I was reviewing the package-lock.json from the pull request. I noticed something that made my developer spidey-senses tingle - together with the recent NPM hack news on Sep 2025:

// BEFORE (Normal, secure)
"@types/body-parser": {
  "version": "1.17.0",
  "requires": {
    "@types/connect": "3.4.32",
    "@types/node": "10.12.18"
  }
}

// AFTER (Sus as hell)
"@types/body-parser": {
  "version": "1.17.0",
  "dependencies": {
    "@types/connect": "*",
    "@types/node": "*"
  }
}

Those little asterisks? Those are wildcards that basically tell npm "give me whatever version you feel like." It's like ordering food and telling the waiter "surprise me" – except the surprise might be malware.

This change completely bypasses version pinning, which is one of our main defenses against supply chain attacks. With wildcards, if any of those packages gets compromised later, you'll automatically pull the malicious version on your next install.

Enter the "Helpful" Contributor

Then I found the smoking gun: a pull request from September 5, 2025, submitted by a user called harshitcodez19. The PR looked innocent enough – just adding some CI/CD with GitHub Actions. How thoughtful!

name: Build on PR
# Heello this is a ymk file made up for cICD
on:
  push:
    branches:
      - main # ← Wait, my repo uses 'master'
  pull_request:
    branches:
      - main # ← This is wrong too...

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'

      - name: Install Dependencies
        run: npm install # ← Should be 'npm ci' for security

      - name: Run Build
        run: npm run build

At first glance, this looks like a newbie trying to be helpful. The typos in the comment ("Heello", "ymk", "cICD") make it seem like someone who's just learning. The branch configuration is wrong, so it wouldn't even run.

But here's the thing – this is exactly how the GhostAction campaign operates.

The GhostAction Campaign: When GitHub Actions Turn Evil

The timing of this PR wasn't coincidental. September 5, 2025 was when security researchers first detected the GhostAction campaign – a sophisticated supply chain attack targeting GitHub repositories.

Here's how GhostAction works:

Reconnaissance: Attackers identify repositories with outdated dependencies
Social Engineering: Submit "helpful" PRs adding CI/CD workflows
Intentional Sabotage: Include obvious mistakes to avoid immediate execution
Patience: Wait for maintainers to "fix" the obvious issues
Exploitation: When the workflow runs, it harvests secrets and credentials

The genius is in the plausible deniability. The PR looks like a well-meaning but inexperienced contributor. The "mistakes" make it seem harmless. But once those workflows run with the right permissions, game over.

The Attack Vector Breakdown

Let's dissect what would have happened if I'd merged this PR:

# The branch misconfiguration prevents immediate execution
on:
  push:
    branches:
      - main # My repo uses 'master', so this won't trigger

This gives the attacker time. If I had "fixed" the obvious branch issue and merged it, the workflow would then:

- name: Install Dependencies
  run: npm install # Pulls wildcarded dependencies

With those wildcards in my lockfile, npm install would fetch the latest versions of @types/node and @types/connect. If those packages had been compromised (which is exactly what the broader npm hack was doing), my CI environment would install malware with full access to:

GitHub secrets
AWS credentials
Database connection strings
API keys
Deploy tokens

Basically, everything you need to completely own my infrastructure.

The Broader Context: The Great NPM Hack of 2025

This attack was part of a much larger campaign according to OX Security's research.

The attackers were:

Compromising legitimate package maintainer accounts
Injecting malicious code into trusted packages
Specifically targeting CI/CD environments
Using GitHub Actions to exfiltrate credentials

My old lockfile, with packages from 2018-2019, was the perfect target. Outdated dependencies are like old locks – they work until someone with the right tools shows up.

Red Flags I Should Have Caught Earlier

Looking back, there were several warning signs:

1. The Contributor Profile

harshitcodez19 had minimal contribution history
Random username pattern typical of throwaway accounts
No established relationship with my project

2. The "Mistakes" Were Too Convenient

Wrong branch names that prevent immediate execution
Typos that create plausible deniability
Using npm install instead of the more secure npm ci

3. The Timing

Submitted on September 5, 2025 (exact GhostAction detection date)
My lockfile wildcards appeared around the same timeframe
Coincided with broader npm supply chain compromises

What Could Have Gone Wrong

If this attack had succeeded, here's what the attacker could have done:

# In my compromised CI environment:
echo $AWS_SECRET_ACCESS_KEY  # Exfiltrate cloud credentials
echo $DATABASE_URL          # Steal database access
echo $DEPLOY_TOKEN          # Compromise deployment pipeline

# Then pivot to:
# - Deploy backdoors to production
# - Access customer data
# - Crypto mining on my infrastructure
# - Lateral movement to other systems

The blast radius from a successful CI compromise is enormous because CI environments typically have elevated permissions to deploy, test, and integrate with other services.

Best Practices: How to Not Get Pwned

Here's what I learned and what you should do to protect yourself:

1. Lockfile Hygiene

# Always use npm ci in production/CI
npm ci  # Uses exact lockfile versions

# Never use npm install in CI
npm install  # Can update versions unexpectedly

// Keep your lockfile pinned
"dependencies": {
  "@types/node": "10.12.18",  // ✅ Exact version
  "@types/connect": "*"       // ❌ Wildcard = danger
}

2. CI/CD Security

# Secure GitHub Actions workflow
name: Secure Build
on:
  pull_request:
    branches: [master] # Correct branch names

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4 # Latest version

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm' # Cache for performance

      # Use npm ci for reproducible builds
      - name: Install dependencies
        run: npm ci

      # Audit before building
      - name: Security audit
        run: npm audit --audit-level=critical

      - name: Build
        run: npm run build

3. Dependency Management

# Regular maintenance
npm outdated        # Check for updates
npm audit          # Security vulnerabilities
npm audit fix      # Auto-fix where possible

# Keep dependencies current
npm update         # Update within semver ranges

4. PR Review Guidelines

When reviewing PRs that touch CI/CD or dependencies:

Who is the contributor? Do they have a history with your project?
Why are they making this change? Unsolicited CI/CD PRs are suspicious
What permissions would this grant? Review secrets and environment access
Are there obvious "mistakes"? Sometimes bugs are features for attackers

5. Repository Protection

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: 'npm'
    directory: '/'
    schedule:
      interval: 'weekly'
    reviewers:
      - 'your-username'

Enable branch protection rules:

Require PR reviews for CI/CD changes
Require status checks to pass
Restrict who can modify workflows

The Human Element

The scariest part of supply chain attacks isn't the technical sophistication – it's how they exploit our human nature. We want to be helpful to contributors. We want to trust the tools we use daily. We want to merge that PR that "fixes" an obvious mistake.

But in the age of automated attacks and compromised packages, a little paranoia goes a long way. That friendly contributor might be a bot. That helpful PR might be reconnaissance. That simple typo fix might be setting up a backdoor.

Lessons Learned

Trust but verify: Even helpful PRs need scrutiny
Keep dependencies updated: Old packages are attractive targets
Use lockfiles properly: Pin versions and use npm ci
Secure your CI/CD: It's the crown jewel attackers want
Stay informed: Follow security researchers and vulnerability databases

The Silver Lining

While getting targeted by a supply chain attack was unsettling, it was also educational. It reminded me that security isn't just about writing secure code – it's about securing the entire development pipeline.

The npm ecosystem is incredibly powerful, but that power comes with responsibility. Every npm install is a vote of trust in hundreds of package maintainers. Every GitHub Action workflow is a potential attack vector.

But don't let this scare you away from open source. The community is what makes JavaScript development amazing. Just be smart about it. Audit your dependencies. Review your workflows. Trust but verify.

And maybe, just maybe, be a little suspicious when harshitcodez19 offers to help with your CI/CD pipeline.

Stay safe out there, fellow developers. The supply chain attackers are getting creative, but so are we.

References

Mastering Email Rate Limits — A Deep Dive into Resend API and Cloud Run Debugging

Dale Nguyen — Sun, 07 Sep 2025 14:11:09 +0000

How we solved persistent rate limiting issues and implemented robust batch email functionality for our AI-powered learning platform

The Challenge: When Emails Start Failing

Picture this: You've built a beautiful lesson scheduling system that sends personalized daily lessons to your users. Everything works perfectly in development, but in production, you start seeing mysterious email failures:

This was exactly the situation we faced with DailyMastery, our AI-powered learning platform. Despite having a solid architecture built with Nx monorepo, Fastify microservices, and deployed on Google Cloud Run, our email delivery system was hitting Resend's rate limits consistently.

Understanding the Problem

The Initial Architecture

Our system had a straightforward approach:

Lesson scheduler triggered by Cloud Scheduler (3x daily)
Individual API calls to Resend for each lesson email
No rate limiting protection
Basic error handling

// The problematic approach
for (const lesson of lessons) {
  const emailResult = await this.emailService.sendEmail({
    to: user.email,
    subject: lesson.title,
    html: lessonContent,
  })
  // Rate limits hit here when processing multiple users rapidly
}

The Root Cause

Resend enforces a 2 requests per second rate limit across all endpoints. When our scheduler processed multiple users with multiple lessons each, we quickly exceeded this limit, causing cascading failures.

The Investigation: Debugging Cloud Run Like a Pro

Step 1: Enhanced Logging Strategy

The first step was implementing comprehensive logging to understand exactly what was happening:

async sendEmail(options: EmailOptions): Promise<{ success: boolean; error?: string; messageId?: string }> {
  try {
    console.log('Attempting to send email via Resend:', {
      to: options.to,
      subject: options.subject,
      htmlLength: options.html?.length || 0
    });

    const result = await this.resend.emails.send(emailData);
    console.log('Resend API response:', result);

    // Critical: Check if Resend returned an error (they don't throw exceptions!)
    if (result.error) {
      console.error('Resend API returned error:', result.error);
      return {
        success: false,
        error: result.error.message || JSON.stringify(result.error),
      };
    }

    return {
      success: true,
      messageId: result.data?.id
    };
  } catch (error) {
    console.error('Failed to send email via Resend:', error);
    return {
      success: false,
      error: error instanceof Error ? error.message : 'Unknown error',
    };
  }
}

Step 2: Cloud Run Log Analysis Techniques

You can view logs in the Google Cloud console, but if you like the terminal approach, then here are the essential Cloud Run debugging commands we used:

# Real-time log monitoring during issues
gcloud logging read "resource.type=cloud_run_revision AND resource.labels.service_name=lesson-scheduler" \
  --limit=50 --format="value(timestamp,textPayload)" | head -20

# Filter for specific errors
gcloud logging read "resource.type=cloud_run_revision AND resource.labels.service_name=lesson-scheduler" \
  --limit=100 | grep -E "(rate_limit|429|Failed)"

# Check logs from specific time range
gcloud logging read "resource.type=cloud_run_revision AND resource.labels.service_name=lesson-scheduler AND timestamp>=\"2025-09-03T16:00:00Z\"" \
  --limit=50

The Solution: Multi-Layer Rate Limiting Strategy

Layer 1: Proper Error Detection

The first critical insight: Resend doesn't throw exceptions for rate limits. They return error objects:

// Before: Missing rate limit detection
const result = await this.resend.emails.send(emailData)
return { success: true, messageId: result.data?.id } // Wrong!

// After: Comprehensive error checking
const result = await this.resend.emails.send(emailData)

if (result.error) {
  console.error('Resend API returned error:', result.error)
  return {
    success: false,
    error: result.error.message || JSON.stringify(result.error),
  }
}

if (!result.data) {
  return {
    success: false,
    error: 'No data returned from Resend API',
  }
}

return {
  success: true,
  messageId: result.data.id,
}

Layer 2: Intelligent Batch Email Implementation with Auto-Chunking

Resend offers a batch API that can send up to 100 emails in a single request. Here's our implementation with smart fallback and automatic chunking for larger batches:

async sendBatchLessonEmails(
  emails: BatchEmailOptions[]
): Promise<{ success: boolean; error?: string; messageIds?: string[]; sentCount?: number; failedCount?: number }> {
  try {
    if (emails.length === 0) {
      return { success: true, messageIds: [], sentCount: 0, failedCount: 0 };
    }

    console.log(`Sending batch of ${emails.length} lesson emails`);

    // Automatic chunking: Split into batches of 100 (Resend limit)
    const batchSize = 100;
    const batches: BatchEmailOptions[][] = [];
    for (let i = 0; i < emails.length; i += batchSize) {
      batches.push(emails.slice(i, i + batchSize));
    }

    let totalSent = 0;
    let totalFailed = 0;
    const allMessageIds: string[] = [];
    const errors: string[] = [];

    // Process each chunk sequentially with rate limiting
    for (let i = 0; i < batches.length; i++) {
      const batch = batches[i];
      console.log(`Sending batch ${i + 1}/${batches.length} with ${batch.length} emails`);

      const result = await this.emailService.sendBatchEmails(batch);

      if (result.success) {
        totalSent += batch.length;
        if (result.messageIds) {
          allMessageIds.push(...result.messageIds);
        }
        console.log(`✅ Batch ${i + 1} sent successfully (${batch.length} emails)`);
      } else {
        totalFailed += batch.length;
        errors.push(`Batch ${i + 1} failed: ${result.error || 'Unknown error'}`);
        console.error(`❌ Batch ${i + 1} failed: ${result.error}`);
      }

      // Add delay between batches to avoid rate limiting
      if (i < batches.length - 1) {
        console.log('⏳ Waiting 1 second between batches to avoid rate limiting...');
        await new Promise(resolve => setTimeout(resolve, 1000));
      }
    }

    const overallSuccess = totalFailed === 0;

    return {
      success: overallSuccess,
      messageIds: allMessageIds,
      sentCount: totalSent,
      failedCount: totalFailed,
      error: errors.length > 0 ? errors.join('; ') : undefined,
    };
  } catch (error) {
    console.error('Error sending batch lesson emails:', error);
    return {
      success: false,
      error: error instanceof Error ? error.message : 'Unknown error',
      sentCount: 0,
      failedCount: emails.length,
    };
  }
}

Layer 3: Architectural Refactoring

We completely restructured our email sending flow:

// Old approach: Send emails immediately during processing
for (const user of users) {
  for (const lesson of lessons) {
    await sendEmail(user, lesson) // Rate limit hit here
  }
}

// New approach: Prepare first, then batch send
const allPreparedEmails: PreparedEmail[] = []

// Phase 1: Prepare all emails
for (const user of users) {
  for (const lesson of lessons) {
    const preparedEmail = await this.prepareLessonEmail(user, lesson)
    if (preparedEmail) {
      allPreparedEmails.push({ email: preparedEmail, user, lesson })
    }
  }
}

// Phase 2: Batch send with intelligent chunking and accurate tracking
if (allPreparedEmails.length > 0) {
  console.log(`🚀 Sending ${allPreparedEmails.length} emails in optimized batches`)

  const emailsToSend = allPreparedEmails.map((pe) => pe.email)

  // The EmailCoordinator handles chunking internally for 100+ email limits
  const batchResult = await this.emailCoordinator.sendBatchLessonEmails(emailsToSend)

  if (batchResult.success) {
    // Only mark lessons as sent based on actual successful send count
    const successfulEmails = allPreparedEmails.slice(0, batchResult.sentCount)
    await this.markLessonsAsSent(successfulEmails)
  }
}

Layer 4: Smart Retry Logic

For critical emails like admin reports, we implemented retry logic:

async sendSummaryReport(result: SchedulerResult): Promise<void> {
  // Add delay to avoid rate limiting after lesson emails
  console.log('⏳ Waiting 3 seconds to avoid rate limiting...');
  await new Promise(resolve => setTimeout(resolve, 3000));

  const emailResult = await this.emailService.sendEmail({
    to: this.reportingEmail,
    subject: `📊 Lesson Scheduler Report - ${result.timeSlot}`,
    html: htmlContent,
  });

  if (!emailResult.success) {
    // Retry logic for rate limits
    if (emailResult.error?.includes('rate_limit_exceeded') ||
        emailResult.error?.includes('Too many requests')) {
      console.log('⏳ Rate limited - waiting 10 seconds and retrying...');
      await new Promise(resolve => setTimeout(resolve, 10000));

      const retryResult = await this.emailService.sendEmail({
        to: this.reportingEmail,
        subject: `📊 Lesson Scheduler Report - ${result.timeSlot} (Retry)`,
        html: htmlContent,
      });

      if (retryResult.success) {
        console.log('✅ Summary report sent successfully on retry');
      }
    }
  }
}

Key Debugging Techniques for Cloud Run

1. Structured Logging

Use consistent emoji prefixes and structured data:

// Good logging practices
console.log('🚀 BATCH EMAIL MODE: Sending emails', { count: emails.length })
console.log('✅ Email sent successfully', { messageId, recipient })
console.log('❌ Email failed', { error: error.message, recipient })
console.log('⏳ Rate limiting delay', { delayMs: 600 })

2. Revision Tracking

Always verify your code is actually deployed:

# Check current revision
gcloud run services describe lesson-scheduler \
  --region=us-central1 \
  --format="value(status.latestReadyRevisionName)"

# List recent revisions with images
gcloud run revisions list --service=lesson-scheduler \
  --format="table(metadata.name,status.conditions[0].status,spec.containers[0].image)"

3. Real-time Monitoring

Set up log streaming during debugging:

# Stream logs in real-time
gcloud beta logging tail "resource.type=cloud_run_revision AND resource.labels.service_name=lesson-scheduler"

# Filter for errors only
gcloud beta logging tail "resource.type=cloud_run_revision AND resource.labels.service_name=lesson-scheduler" \
  --filter="severity>=ERROR"

Results and Performance Impact

Before Optimization

❌ Cascading failures due to rate limits
❌ Poor user experience with missed lessons
❌ Manual intervention required daily

After Optimization

✅ 0% email failure rate
✅ Automatic handling of up to 100 emails per batch
✅ Zero manual intervention required

Performance Metrics

Final Test Results:
- totalUsers: 1
- emailsSent: 3
- emailsFailed: 0
- errorCount: 0
- processingTime: 7.048s
- batchEmailsUsed: true

Best Practices Learned

1. Email Service Design Patterns

interface EmailService {
  // Always return structured results
  sendEmail(options: EmailOptions): Promise<{
    success: boolean
    error?: string
    messageId?: string
  }>

  // Batch operations with fallback
  sendBatchEmails(emails: BatchEmailOptions[]): Promise<{
    success: boolean
    error?: string
    messageIds?: string[]
    sentCount?: number
    failedCount?: number
  }>
}

2. Rate Limiting Strategies

Batch First: Use batch APIs when available
Automatic Chunking: Split large batches into 100-email chunks automatically
Intelligent Delays: 600ms between individual requests (< 2 req/sec) and 1s between batch chunks
Accurate Tracking: Only mark operations as successful based on actual send counts
Exponential Backoff: Longer delays for retries
Circuit Breaker: Fail fast on repeated rate limits

Conclusion

Building a robust email delivery system requires understanding both your email provider's limitations and your cloud platform's deployment mechanics. Our journey from 30% failure rates to 0% taught us that:

Rate limiting is real - Plan for it from day one
Batch APIs are game-changers - Use them when available
Docker caching can bite you - Structure your builds carefully
Comprehensive logging saves time - Invest in good observability
Fallback strategies are essential - Always have a Plan B

The combination of Resend's batch API, intelligent fallback logic, proper rate limiting, and robust Cloud Run debugging techniques transformed our email delivery from a daily headache into a reliable, scalable system.

Remember: In production systems, it's not just about making it work—it's about making it work reliably, even when things go wrong.

This blog post is based on our real-world experience building DailyMastery, an AI-powered learning platform. The techniques described here are battle-tested in production and have been processing thousands of lesson emails daily without issues.

SupaWP Storage Filter Hooks - Seamless Supabase Storage Integration for WordPress

Dale Nguyen — Sat, 06 Sep 2025 04:05:16 +0000

Building modern WordPress applications often requires robust file storage solutions that scale beyond traditional media library limitations. Today, we're excited to introduce SupaWP Storage Filter Hooks – a powerful new feature that brings seamless Supabase Storage integration directly to your WordPress plugins and themes.

What Are SupaWP Storage Filter Hooks?

SupaWP Storage Filter Hooks are a set of WordPress filter hooks that provide a standardized, developer-friendly way to integrate Supabase Storage into any WordPress application. Instead of writing custom storage code for each project, you can now leverage these hooks to upload, delete, and manage files in Supabase Storage with just a few lines of code.

Prerequisites

Before diving into the implementation, ensure you have:

WordPress site with SupaWP plugin v1.3.4 or higher installed and configured
Supabase project with Storage enabled
Basic knowledge of WordPress filters and PHP
Supabase Storage bucket configured

The Three Essential Hooks

1. `supawp_upload_image_to_supabase`

Upload images directly to Supabase Storage with automatic validation and security:

// Upload a user profile image
$image_url = apply_filters('supawp_upload_image_to_supabase', $file, $fileName, 'user-images');

if (!empty($image_url)) {
    // Success! Save the URL to your database
    update_user_meta($user_id, 'profile_image_url', $image_url);
    return array('success' => true, 'url' => $image_url);
}

2. `supawp_delete_image_from_supabase`

Remove files from Supabase Storage with proper cleanup:

// Delete a user's profile image
$success = apply_filters('supawp_delete_image_from_supabase', $image_url, 'user-images');

if ($success) {
    // File deleted successfully
    delete_user_meta($user_id, 'profile_image_url');
}

3. `supawp_get_storage_config`

Access Supabase Storage configuration and verify setup:

// Check if storage is properly configured
$config = apply_filters('supawp_get_storage_config');

if (!empty($config)) {
    echo "Storage URL: " . $config['url'];
    echo "Authentication: Configured ✓";
}

Simple File Upload Implementation

Here's a basic implementation using the SupaWP Storage Filter Hooks:

<?php
/**
 * Simple File Upload with SupaWP Storage
 */
class Simple_SupaWP_Upload {

    public static function init() {
        add_shortcode('simple_file_upload', array('Simple_SupaWP_Upload', 'upload_shortcode'));
        add_action('wp_ajax_upload_to_supabase', array('Simple_SupaWP_Upload', 'handle_upload'));
    }

    public static function upload_shortcode() {
        $current_user = wp_get_current_user();
        if (!$current_user->ID) {
            return '<p>Please log in to upload files.</p>';
        }

        ob_start();
        ?>
        <form id="upload-form" enctype="multipart/form-data">
            <input type="file" name="user_file" accept=".jpg,.jpeg,.png,.webp" required>
            <button type="submit">Upload File</button>
        </form>

        <div id="upload-result"></div>

        <script>
            document.getElementById('upload-form').addEventListener('submit', function(e) {
                e.preventDefault();

                const formData = new FormData(this);
                formData.append('action', 'upload_to_supabase');
                formData.append('nonce', '<?php echo wp_create_nonce('upload_nonce'); ?>');

                fetch('<?php echo admin_url('admin-ajax.php'); ?>', {
                    method: 'POST',
                    body: formData
                })
                .then(response => response.json())
                .then(data => {
                    document.getElementById('upload-result').innerHTML =
                        data.success ? 'Upload successful!' : 'Upload failed: ' + data.data;
                });
            });
        </script>
        <?php
        return ob_get_clean();
    }

    public static function handle_upload() {
        check_ajax_referer('upload_nonce', 'nonce');

        if (!isset($_FILES['user_file'])) {
            wp_send_json_error('No file provided');
        }

        $file = $_FILES['user_file'];
        $user_id = get_current_user_id();

        // Generate file path
        $extension = pathinfo($file['name'], PATHINFO_EXTENSION);
        $fileName = "users/{$user_id}/" . time() . ".{$extension}";

        // Upload using SupaWP hook
        $url = apply_filters('supawp_upload_image_to_supabase', $file, $fileName, 'user-content');

        if ($url) {
            wp_send_json_success(array('url' => $url));
        } else {
            wp_send_json_error('Upload failed');
        }
    }
}

add_action('supawp_init', array('Simple_SupaWP_Upload', 'init'));

Advanced Implementation Examples

E-commerce Product Gallery

function upload_product_images($product_id, $files) {
    $uploaded_urls = array();

    foreach ($files as $file) {
        $fileName = "products/{$product_id}/" . time() . ".jpg";
        $url = apply_filters('supawp_upload_image_to_supabase', $file, $fileName, 'products');

        if ($url) {
            $uploaded_urls[] = $url;
        }
    }

    update_post_meta($product_id, '_gallery_urls', $uploaded_urls);
    return $uploaded_urls;
}

User Document Management

function upload_user_document($user_id, $file) {
    if (!current_user_can('upload_files')) {
        return false;
    }

    $fileName = "documents/{$user_id}/" . time() . ".pdf";
    $url = apply_filters('supawp_upload_image_to_supabase', $file, $fileName, 'documents');

    if ($url) {
        update_user_meta($user_id, 'document_url', $url);
        return $url;
    }

    return false;
}

Integration with Popular Plugins

Contact Form 7 Integration

function handle_contact_form_upload($contact_form) {
    $submission = WPCF7_Submission::get_instance();
    $files = $submission->uploaded_files();

    if (isset($files['resume'])) {
        $fileName = "resumes/" . time() . "_resume.pdf";
        $url = apply_filters('supawp_upload_image_to_supabase', $files['resume'], $fileName, 'resumes');

        if ($url) {
            update_option('latest_resume', $url);
        }
    }
}
add_action('wpcf7_mail_sent', 'handle_contact_form_upload');

WooCommerce Integration

function sync_product_to_supabase($product_id) {
    $product = wc_get_product($product_id);
    $image_id = $product->get_image_id();

    if ($image_id) {
        $image_path = get_attached_file($image_id);
        $file = array(
            'tmp_name' => $image_path,
            'name' => basename($image_path),
            'size' => filesize($image_path)
        );

        $fileName = "products/{$product_id}.jpg";
        $url = apply_filters('supawp_upload_image_to_supabase', $file, $fileName, 'products');

        if ($url) {
            update_post_meta($product_id, '_supabase_image', $url);
        }
    }
}
add_action('woocommerce_process_product_meta', 'sync_product_to_supabase');

Security Best Practices

Row Level Security (RLS) Policies

Set up proper RLS policies in your Supabase dashboard:

-- Users can only upload to their own folders
CREATE POLICY "Users upload own files" ON storage.objects
FOR INSERT WITH CHECK (
    bucket_id = 'user-content' AND
    (storage.foldername(name))[1] = auth.uid()::text
);

-- Users can only view their own files
CREATE POLICY "Users view own files" ON storage.objects
FOR SELECT USING (
    bucket_id = 'user-content' AND
    (storage.foldername(name))[1] = auth.uid()::text
);

-- Users can only delete their own files
CREATE POLICY "Users delete own files" ON storage.objects
FOR DELETE USING (
    bucket_id = 'user-content' AND
    (storage.foldername(name))[1] = auth.uid()::text
);

Simple File Validation

function validate_file($file) {
    // Check file size (5MB limit)
    if ($file['size'] > 5242880) {
        return false;
    }

    // Check file type
    $allowed_types = array('jpg', 'png', 'pdf');
    $extension = pathinfo($file['name'], PATHINFO_EXTENSION);

    return in_array($extension, $allowed_types);
}

Performance Optimization

Batch Upload

function batch_upload($files, $user_id) {
    $results = array();

    foreach ($files as $file) {
        $fileName = "users/{$user_id}/" . time() . ".jpg";
        $url = apply_filters('supawp_upload_image_to_supabase', $file, $fileName, 'user-content');

        $results[] = array(
            'name' => $file['name'],
            'url' => $url,
            'success' => !empty($url)
        );
    }

    return $results;
}

Using Your File Upload System

Once your code is in place:

Add the shortcode to any page or post: [simple_file_upload]
Ensure users are logged in for security
Configure your Supabase Storage with appropriate buckets

Example usage:

[simple_file_upload]

Common Supabase Storage Query Filters

When working with uploaded files, you can query them using various filters:

// Get user's files from database
$user_files = apply_filters('supawp_get_data_from_supabase', 'files', array(
    'user_id' => 'eq.' . $user_id,
    'order' => 'created_at.desc'
));

// Filter by file size
$large_files = apply_filters('supawp_get_data_from_supabase', 'files', array(
    'file_size' => 'gt.1000000'
));

Troubleshooting Common Issues

Troubleshooting

// Check if storage is configured
$config = apply_filters('supawp_get_storage_config');
if (empty($config)) {
    echo "Storage not configured";
}

Common Issues

Storage not configured - Check SupaWP settings
Upload fails - Verify file size and type limits
Permission denied - Check Supabase RLS policies
Files not showing - Ensure bucket is public

Conclusion

SupaWP Storage Filter Hooks revolutionize file management in WordPress by providing seamless integration with Supabase's powerful storage infrastructure. With enterprise-grade security, global CDN distribution, and a developer-friendly API, you can build sophisticated file management systems with minimal code.

Key benefits of this integration:

Zero Configuration Complexity - SupaWP handles authentication and API communication
Enterprise Security - Automatic RLS integration and secure file handling
Global Performance - Leverage Supabase's worldwide CDN network
WordPress Native - Uses familiar WordPress filter patterns
Scalable Architecture - No server storage limitations

Whether you're building a simple profile system, complex document management, or e-commerce product galleries, SupaWP Storage Filter Hooks provide the foundation for modern, scalable WordPress applications.

Resources

Ready to transform your WordPress file management? Download SupaWP v1.3.4+ and start building with Supabase Storage today!

DEV Community: Dale Nguyen

Building a Document-RAG Agent on GCP's Agent Development Kit (ADK)

The stack, end to end

Why ADK

The agent service, top to bottom

The RunnerProtocol seam

Retrieval as a pre-model step, not an ADK tool

The retrieval layer

The doc-processor — where the chunks come from

The prompt: grounded chunks with [doc:<id>] markers

Cloud Run + service-to-service ID tokens

The fallback I kept

What I'd do differently

The numbers

TL;DR for someone building this on ADK today

The Background Task That Froze: A Serverless CPU-Throttling Mystery

The setup

The symptom, and why it's invisible

Confirming it without a debugger

The fix: move it onto the request path

When inline genuinely isn't an option

Testing the difference

Lessons learned

Pitfalls and false signals

Quick checklist

One-line summary

All-Green Lighthouse: How I Fixed Every Audit on My AnalogJS Blog

Measurement method

PageSpeed Insights: before and after

Results

Fix 1 — Gate third-party loading on click/keydown/touchstart, not scroll

Fix 2 — publicDir override silently orphans public/

Fix 3 — The default Prism light theme fails WCAG AA

Fix 4 — Small accessibility papercuts

Performance fixes — Core Web Vitals

Summary

Run Coding Agents on Local AI — Zero Cloud, Full Control

Why Run AI Locally?

Hardware Requirements

Choosing a Model

Setting Up Ollama as a Network Server

Codex CLI

Install

Configuration

Model Catalog

Running Codex

Claude Code

Configuration

Cursor

Pi (pi.dev)

Install

Configuration

Running Pi

Tradeoffs to Know Before Committing

A Note on Model Selection for Other Tasks

Conclusion

Local-first: a Model on Your Own Machine, Zero Cloud

What this post covers

Engine choice on this machine

Model options by available RAM

Prerequisites

Run it

Sample output

The stateless contract, explained

Definition of done

Things worth noting now

What's next

Free Model Providers to Use with Hermes Agent

Installing Hermes Agent

Switching providers

Free providers supported by Hermes Agent

Nous Portal

OpenRouter

NVIDIA NIM

Hugging Face

NovitaAI

Kimi / Moonshot

Provider comparison at a glance

Tips for running Hermes on free tiers

Summary

The `RunnerProtocol` seam

The prompt: grounded chunks with `[doc:<id>]` markers

Fix 1 — Gate third-party loading on `click`/`keydown`/`touchstart`, not scroll

Fix 2 — `publicDir` override silently orphans `public/`

OAuth Tokens (`ctpo_*`)