DEV Community: Nerav Doshi

Containerized Ollama and Found the Real Memory Overhead

Nerav Doshi — Wed, 09 Sep 2026 00:47:46 +0000

Context: On Linux, containers run natively — they share the host's kernel directly. On macOS, they can't (containers need a Linux kernel underneath), so tools like Podman and Docker Desktop quietly run a small Linux virtual machine in the background and put your containers inside that. Every container on a Mac is really sharing a fixed slice of memory carved out for that VM, not your Mac's full RAM — which matters a lot once something inside the container needs more memory than the VM itself was given.

Ran: Started Ollama as a container with Podman, exposed on a different port (11435) than the native app already running:

podman run -d --name ollama-container -p 11435:11434 ollama/ollama
podman exec -it ollama-container ollama pull llama3.2:1b
podman exec -it ollama-container ollama run llama3.2:1b

The pull succeeded (after one transient network retry on the first attempt). Loading the model didn't:

Error: 500 Internal Server Error: model requires more system memory (1.3 GiB) than is available (620.8 MiB)

podman machine list showed why — the VM backing every Podman container on this machine was set to just 2GiB total, for the OS, runtime, and every container combined. After host overhead, only ~620MB was actually free — well under the ~1.3GB this model needs.

Fixed by resizing the VM itself (has to be stopped first):

podman machine stop
podman machine set --memory 4096
podman machine start

Restarting the VM also stopped the container, which then wouldn't exec into ("container state improper") until explicitly restarted:

podman start ollama-container
podman exec -it ollama-container ollama run llama3.2:1b

That worked — asked it "what is kubernetes?" to force a real response, then checked memory with Podman's equivalent of docker stats:

podman stats ollama-container --no-stream

Result:

	Entry 01 (bare metal, native Ollama app)	Containerized (Podman)
Process memory	~1.24 GB RSS	1.663 GB
Processor	100% GPU (Metal)	CPU only (no Metal passthrough in a container)

The container used about 34% more memory than the same model running natively — 1.663GB vs. 1.24GB. Some of that gap is Podman/Ollama server overhead inside the container; some is likely the lack of GPU acceleration forcing more CPU-side memory use, though that's an inference from the numbers rather than something directly measured here — worth a more controlled comparison in a future entry rather than treating it as confirmed. Either way, "same model, same memory" turned out to be false: containerizing an inference workload isn't memory-neutral.

Takeaway: The actual resource request to set for a containerized version of this model should be closer to 1.663GB than the 1.24GB bare-metal number from Entry 01 — the container overhead is real, not negligible, and sizing a Kubernetes pod off bare-metal numbers alone would under-provision it. Also worth remembering for local dev: Podman's VM has its own fixed memory ceiling independent of your Mac's total RAM, and that's the first thing to check before assuming a model itself is too large to run.

Wired the Local MCP Server into Claude Code

Nerav Doshi — Wed, 02 Sep 2026 19:58:27 +0000

Context: Every prior entry in this arc — building the Chroma index, chunking, querying — was done by hand, typing Python commands one at a time. An MCP server is what turns that into a real tool: a small program that exposes specific capabilities (like "search my notes") to an AI client such as Claude Code, so the client can call it directly instead of a person running queries manually. The point of this entry was proving that connection actually works, not just that the server starts.

Ran: After the earlier Connection closed error (Claude Code launching a bare python3, which resolved to system Python outside the venv, missing every package the server needs), re-registered the server pointing directly at the venv's Python:

claude mcp remove today-i-ran-notes
claude mcp add today-i-ran-notes -- ~/mcp-env/bin/python3 mcp_search_server.py
claude mcp list

That returned ✔ Connected. Then, in a real Claude Code session, asked it directly:

Use the today-i-ran-notes server to search for how to check pod status with oc

Result: The tool got called — twice. First a broad search, then a follow-up call pulling more detail from the closest match. Claude Code's own response is worth quoting closely, because it's more interesting than a clean success would have been:

The search returned results, but none of them directly cover checking pod status with oc. The closest match is a note about building an "oc CLI Mentor" system prompt... but doesn't appear to include specific pod-status commands.

That's the right answer. Entry 02 is about building a constrained system prompt, not a list of pod-status commands — the tool didn't hallucinate a match, it accurately reported the gap and said so plainly, then fell back to its own general knowledge to actually answer the question, clearly separating "your notes don't cover this" from "here's the answer anyway."

One more thing worth flagging clearly: when asked to save the new commands for future reference, Claude Code saved them to its own built-in memory system — not to the Chroma index built across Entries 05–07. Those are two separate mechanisms that are easy to conflate: one is the RAG pipeline this series has been building by hand, the other is Claude Code's own persistent memory feature. The note that got saved lives in the latter, not in chroma_db.

Takeaway: The full loop works — a local embeddings pipeline, exposed as a real MCP tool, correctly invoked by a real client, with honest reporting when the answer isn't in the index rather than a confident wrong guess. That honesty is the most important result here: a search tool that admits "not in here" is far more useful than one that always returns something and lets the caller assume it's relevant.

What is DevOps? A Plain English Guide

Nerav Doshi — Wed, 02 Sep 2026 15:51:45 +0000

Ever Wondered How Netflix Never Seems to Go Down?

Think about this for a second. Netflix has over 260 million subscribers worldwide. People are watching shows in Tokyo, London, Lagos, and New York — all at the same time. And yet, when was the last time Netflix crashed on you?

Now think about your favourite food delivery app. You open it, order food, track your driver in real time, and get a notification the moment your burger arrives. All of that happens in seconds.

Behind all of this is a way of working called DevOps. And by the end of this article, you'll understand exactly what it is — no jargon, no complicated diagrams, just plain English.

The Old Way (And Why It Was a Nightmare)

To understand DevOps, we first need to understand the problem it solved.

Imagine a software company in the early 2000s. They had two completely separate teams:

The Developers — the people who wrote the code and built new features

The Operations team — the people who managed the servers and kept everything running

These two teams barely talked to each other. Developers would spend months building new features, then hand over a massive pile of code to the operations team and say "here you go, make it work."

The operations team would panic. They hadn't been involved in building it, had no idea what it did, and now they had to deploy it to millions of users without breaking anything.

The result? Deployments took weeks. Bugs slipped through. Systems crashed. Customers complained. And the two teams blamed each other.

Sound stressful? It was.

So What is DevOps?

DevOps is simply the practice of bringing developers and operations teams together to build, test, and release software faster and more reliably.

The name itself is a combination of Dev (Development) and Ops (Operations). Instead of two teams working in silos, they work as one team with shared goals, shared tools, and shared responsibility.

Think of it like a restaurant kitchen.

In a badly run kitchen, the chefs cook the food and just slide it through a hatch to the waiters. The waiters don't know what's in the dish, the chefs don't know what the customers are saying, and when something goes wrong, everyone points fingers.

In a well run kitchen — like the ones you see at a great restaurant — the chefs and waiters communicate constantly. They know the menu inside out, they get feedback from customers quickly, and they work as one team to give people a great experience.

DevOps is that well run kitchen, but for software.

A Real World Example: Amazon

Amazon deploys new code to its website thousands of times per day.

That means engineers are constantly making small improvements — fixing a bug here, improving the checkout experience there, tweaking a recommendation — and those changes go live almost instantly.

How? Because Amazon uses DevOps practices. Small changes are automatically tested, automatically checked for problems, and automatically deployed without anyone having to manually press a button.

In the old way of working, those same changes might have taken weeks to go live, gone through five teams, and required a late night deployment session that everyone dreaded.

The Three Big Ideas Behind DevOps

You don't need to memorise these, but it helps to know the thinking behind DevOps.

1. Work in Small Steps

Instead of building for six months and releasing everything at once (terrifying), DevOps teams release small changes frequently. If something breaks, it's easy to find and fix because the change was tiny.

Uber does this constantly. Every few weeks, the Uber app gets tiny updates — a new button here, a faster map there. You barely notice, but the team is constantly improving without disrupting your experience.

2. Automate the Boring Stuff

Testing code manually, deploying to servers manually, checking for errors manually — all of this is slow and humans make mistakes. DevOps teams automate these tasks so they happen instantly and consistently every single time.

Think of it like a car factory. Cars aren't built by hand anymore — robots do the repetitive work faster and with fewer errors. DevOps applies the same thinking to software.

3. Get Feedback Fast

When something breaks, DevOps teams know about it within seconds, not days. Monitoring tools watch the system constantly and send alerts the moment something looks wrong.

Netflix actually has a famous practice where they intentionally break parts of their own system during working hours to make sure their team can fix things quickly. They call it Chaos Engineering. It sounds mad, but it means they're never caught off guard.

What Does a DevOps Engineer Actually Do?

A DevOps engineer is the person who builds and maintains the systems that help developers work faster and more safely. They work on things like:

Setting up automated testing so bugs are caught before they reach users
Building pipelines that automatically deploy code (we'll cover this in a future article)
Managing cloud infrastructure on platforms like AWS or Azure
Monitoring systems and making sure everything is running smoothly
Writing scripts to automate repetitive tasks

It's one of the most in-demand roles in tech right now, and the skills involved are exactly what this blog is here to help you build.

Why Should You Care About DevOps?

Whether you're a developer, a system admin, a project manager, or someone just getting into tech — DevOps matters because it's how modern software is built.

Every major tech company in the world uses DevOps practices. Banks use it to deploy new banking features. Airlines use it to update booking systems. Hospitals use it to improve patient management software. It's not just for Silicon Valley startups — it's everywhere.

Learning DevOps opens doors. And the best part is, you don't need to know everything at once. We'll take it one byte at a time.

Quick Recap

Here's everything we covered in plain English:

DevOps = Developers and Operations working together instead of in separate silos
It solves the old problem of slow, painful, risky software releases
The core ideas are: small changes, automation, and fast feedback
Companies like Amazon, Netflix, and Uber use DevOps to deploy changes thousands of times a day
A DevOps engineer builds the tools and systems that make all of this possible

What's Next?

In the next article we're going to look at Linux — The Operating System That Runs the Internet — the OS that powers most of the internet and why every DevOps engineer needs to know the basics.

It's going to be short, practical, and you'll be typing your first Linux commands before the end of the article. See you there.

Direct Connect and ExpressRoute: Fixing Asymmetric BGP Routing

Nerav Doshi — Tue, 01 Sep 2026 01:57:01 +0000

The Story

Back in Article 1, I said we'd get back to this: once you've decided what goes in the cloud and who manages it, there's a third question that decides whether any of it actually works — how does your data get there?

A telecom customer I worked with was pushing sustained real-time Kafka streams past 850 Mbps between on-prem and the cloud, with big unpredictable spikes on top, over an AWS Site-to-Site VPN. A single tunnel is rated up to 1.25 Gbps. On paper they had headroom.

They still hit a wall. It wasn't AWS's fault.

The Problem

The wall was the on-prem VPN appliance's CPU — specifically, it couldn't keep up with IPsec encryption for every packet, made worse by two tunnels that weren't splitting traffic evenly. I confirmed this from CloudWatch tunnel metrics and CLI inspection on the appliance itself, not from guessing based on symptoms.

This is the trap for any platform team running high-throughput streaming over a public-internet VPN: the bottleneck almost never shows up where the bandwidth numbers say it should. Think of a VPN over the public internet like a public road — cheap, open to everyone, fine most days. But there's a single-lane toll booth at the entrance where every car gets checked. On a normal day, minor delay. On a heavy day, that booth is the road, no matter how wide the highway gets afterward. The toll booth here was IPsec — every packet individually encrypted and decrypted, with real CPU cost.

Why the Obvious Fixes Didn't Work

Two fixes made sense on paper. Both failed, for different reasons.

Scaling up meant an emergency maintenance window to go from 4 vCPUs to 8, plus more RAM. No help — IPsec/IKE encryption for a given tunnel is bound to a single worker thread, and that thread doesn't spread across cores just because more exist. The new cores sat idle. The one thread doing the work stayed pegged at 98%.

Scaling out meant adding two more VPN tunnels, on the theory that more tunnels meant more hash buckets and distributed crypto load. Also failed. ECMP hashes traffic by flow — source/destination IP, source/destination port, protocol — and this Kafka stream was one sustained flow. It didn't matter how many tunnels existed; that flow kept landing on the same path. More lanes feeding the same toll booth. Retries got worse, not better.

Direct Connect and ExpressRoute are the private road built just for you — no public traffic, no toll booth. They cost more and take time to build, but they behave predictably once they're in place. Most teams start on the public road because it's what's available on day one. This is what happens once your traffic outgrows it.

The Architecture

The diagram lays out the "two private roads, no shared map" problem: the on-prem VPN appliance with its two IPsec tunnels to AWS; an HAProxy layer using 16 secondary IPs to feed ECMP; Direct Connect (AWS) and ExpressRoute (Azure) running as parallel dedicated circuits, each with dual-location redundancy; independent BGP autonomous systems per cloud with zero shared visibility into each other's routing; and the asymmetric-path failure itself — a request leaving via Direct Connect, its response coming back over ExpressRoute, hitting a stateful firewall that drops it as an unmatched session.

Direct Connect took six weeks to provision. A single-provider option was ruled out almost immediately — this customer was already deliberately multi-cloud for capacity, cost, and reliability reasons that had nothing to do with this problem. The design became Direct Connect into AWS and ExpressRoute into Azure, each with redundant physical locations, driven by a real contractual requirement: 99.99% uptime.

Here's where this specific engagement took an unexpected turn. In week two of running both circuits live, a new failure mode showed up: an asymmetric BGP routing loop that broke stateful firewalls. AWS and Azure each run independent BGP autonomous systems with no visibility into each other's routing decisions. Each cloud picked its own "best" path back to the same on-prem address block — unaware the other had picked differently. A request could leave over Direct Connect while the response came back over ExpressRoute, and to a stateful firewall expecting a matched pair, that looks like a session that was never opened. Dropped.

This is exactly why the failure was subtle: the data plane — the actual Kafka traffic — looked completely healthy right up until the control plane's routing asymmetry collided with a stateful security device.

Fixing It

Prerequisites: two cloud providers with existing dedicated-circuit relationships (Direct Connect for AWS, ExpressRoute for Azure), BGP peering already established on both circuits, a stateful firewall in the on-prem path, and TLS termination capability at the compute tier rather than just at a central gateway.

Step 1 — buy time while Direct Connect provisions

Pulled non-critical workloads off the IPsec tunnels to free capacity for the critical stream
Deployed local HAProxy bound to 16 separate secondary IPs on the on-prem appliance
Why it worked: ECMP hashes partly on source/destination IP, so splitting one flow across 16 source IPs made it look like sixteen distinct flows — forcing real distribution across both tunnels instead of one path absorbing everything
Not off-the-shelf load balancing — HAProxy's only job here was manufacturing enough distinct source IPs to break ECMP's per-flow hash
Rollback: trivial. Pull HAProxy out of the path, nothing left to unwind

Step 2 — fix the asymmetric routing

Rejected the easy fix: loosening firewall statefulness to just allow asymmetric traffic — would have weakened a required security posture
Real fix: BGP community tagging, MED tuning, and AS-path prepending on the Azure side, plus subnet-specific routing, to force one deterministic, symmetric path per address block
Rollback: reversible by withdrawing the community tags and MED values — but only cleanly if you documented the pre-change route table first. Untangling which of several manual tweaks caused a new asymmetry after the fact is much harder than reverting to a known-good baseline

Step 3 — get encryption out of the choke point

Rather than reintroduce a central encryption bottleneck — recreating the exact problem that started this story — we moved TLS to the application layer, handled independently by each service instance. Spreading crypto across the compute tier kept continuous pod-to-pod encryption without a single-threaded chokepoint.

Security Considerations

A private circuit isn't an encryption exemption. This was telecom data under SOC 2, which requires encryption in transit regardless of whether the path is public or private — "it's on a dedicated circuit" doesn't satisfy that requirement by itself.

And the obvious fix for the BGP asymmetry — loosening firewall statefulness — was a security regression we explicitly rejected. It would have widened the firewall's tolerance for traffic patterns that look identical to spoofing and session-hijacking attempts. We took the slower, harder BGP-policy route instead of the fast one that degraded posture.

Tradeoffs

Dedicated circuits gave us predictable throughput and latency, and a real path to 99.99% uptime through dual-cloud, dual-location redundancy. In exchange: six weeks of lead time per circuit, and an entirely new failure mode — BGP asymmetry across independent cloud ASNs — that a single-provider VPN never has to deal with.

Moving TLS to the application layer removed the single-threaded bottleneck entirely, but it cost us centralized visibility. One gateway doing crypto means one place to audit; per-service TLS means per-service certificate management and rotation discipline — more operational surface area, not less.

What I'd Do Differently

Design application traffic to use multiple distinguishable connections from the start — one giant flow is always a point of contention, VPN or dedicated circuit alike. And put BGP routing policy templates in place before the first circuit goes live, not after an outage forces the question. Both failures in this story were, in hindsight, predictable the moment more than one path existed.

Has any of this actually been tested? Partially. Automatic BGP failover from the primary Direct Connect circuit to the secondary has been proven for real — a production fiber cut triggered it, and it worked. Full failover down to the standby VPN has only run in a controlled drill, never a real outage. Worth naming rather than assuming away.

And this design has a ceiling. It holds for two clouds and a handful of regions, but it doesn't scale by just repeating the pattern — add enough regions and individually-advertised address blocks and you hit a hard limit on how many routes a provider's edge router will accept. Past that limit it doesn't degrade gracefully; it silently drops excess routes or drops the BGP session entirely, taking every route with it. That blackholes active streams — the exact failure this architecture was built to prevent. The fix at that scale isn't more hand-tuned BGP policy per circuit. It's a managed transit gateway: a centralized hub that handles routing between clouds and regions so no circuit is negotiating its own path in isolation.

What's Next

Next in Cloud Without the Chaos: Article 4 — Data Sovereignty in Practice: What "It's in an EU Data Centre" Actually Covers (and Doesn't) — going deeper on the compliance angle this article touched on with SOC 2.

Chunked a Document and Found a New Retrieval Problem

Nerav Doshi — Thu, 27 Aug 2026 12:47:43 +0000

Context: Chunking means splitting a long document into smaller overlapping pieces before embedding each one separately, instead of embedding the whole thing (or truncating it, as Entry 05 did). This fixes content loss — nothing gets silently dropped — but it introduces a structural question that's easy to miss: a document split into 17 chunks now has 17 separate entries competing for a spot in search results, while a short document still only has one. More chunks means more chances to appear in a top-N result, independent of whether that chunk is actually the most relevant thing stored.

Ran: Chunked Managed vs Self-Hosted (managed-vs-self-hosted-handing-over-keys.md, split into 17 pieces at 1000 characters with 200-character overlap) and embedded each chunk into the same Chroma collection from Entries 05/06. Two real snags on the way:

First, reconnecting to the collection returned an empty database with only the new chunks in it — no sign of the original 3 entries from Entries 05/06. Turned out PersistentClient(path="./chroma_db") uses a path relative to wherever Python was launched from, and this session started in a different folder than the earlier ones. A find across the filesystem turned up three separate chroma_db folders — the "empty" one was actually a brand-new database created by accident, not data loss. Fixed by returning to the original working directory before reconnecting.

Second, after fixing that, an early query attempt using collection.query(query_texts=[...]) failed with InvalidArgumentError: Collection expecting embedding with dimension of 768, got 384 — passing raw text instead of a pre-computed embedding makes Chroma fall back to its own default embedding model, which produces a different vector size than nomic-embed-text. Same lesson as Entry 06: always embed the query with the same model used for the documents.

With that sorted, chunked the new document and embedded each piece into the correct, 20-entry collection:

import ollama

text = open("managed-vs-self-hosted-handing-over-keys.md").read()

chunk_size = 1000
overlap = 200
chunks = [text[i:i+chunk_size] for i in range(0, len(text), chunk_size - overlap)]

for i, chunk in enumerate(chunks):
    resp = ollama.embeddings(model="nomic-embed-text", prompt=chunk)
    collection.upsert(
        ids=[f"managed-vs-self-hosted-handing-over-keys_chunk{i}"],
        embeddings=[resp["embedding"]],
        documents=[chunk],
    )

collection.count()   # → 20 (3 original entries + 17 new chunks)

Then ran three queries against it — the same "oc pod status" and "pizza topping" questions from Entry 06, plus a real question about the new document's actual topic. Each question has to be embedded with the same model used for the documents before querying — passing raw text via query_texts instead triggers Chroma's own default embedding model, which produces a different vector size and fails outright:

q1_embed = ollama.embeddings(model="nomic-embed-text", prompt="how do I check pod status with oc")
q1 = collection.query(query_embeddings=[q1_embed["embedding"]], n_results=3)

q2_embed = ollama.embeddings(model="nomic-embed-text", prompt="what's the best pizza topping")
q2 = collection.query(query_embeddings=[q2_embed["embedding"]], n_results=3)

q3_embed = ollama.embeddings(model="nomic-embed-text", prompt="What are my options for kubernetes, should I use managed or self-hosted Kubernetes")
q3 = collection.query(query_embeddings=[q3_embed["embedding"]], n_results=3)

print("Query 1:", q1["ids"], q1["distances"])
print("Query 2:", q2["ids"], q2["distances"])
print("Query 3:", q3["ids"], q3["distances"])

Result:

Query	Top 3 matches	Distances
"how do I check pod status with oc"	`02-oc-cli-mentor...` (correct), then 2 unrelated chunks	437.72, 450.32, 453.22
"what's the best pizza topping"	3 unrelated chunks (all from the new doc)	519.80, 531.01, 531.51
"managed or self-hosted Kubernetes"	3 correct chunks from the new doc	290.34, 314.06, 316.37

Two things stand out. The on-topic Kubernetes question is the tightest, cleanest match of the whole series so far — every one of the top 3 results came from the right document, at meaningfully lower distances than anything seen in Entries 05 or 06. Chunking clearly works for making a long document's actual content findable.

But the oc question shows the tradeoff directly: in Entry 06, its #2 result was the genuinely-related URL entry at distance 499.63. Here, that same document got pushed entirely out of the top 3, replaced by two irrelevant chunks from the 17-chunk document at 450.32 and 453.22 — lower distances not because they're more relevant, but because a 17-chunk document simply has more entries competing for the middle-ranked spots.

Takeaway: Chunking is a real fix for the content-loss problem from Entry 05, and the on-topic result here is the strongest retrieval this series has produced. But it's not a free upgrade — a document with many chunks crowds out equally-relevant single-entry documents just by having more shots at ranking. Production RAG systems typically handle this with per-document result caps or a re-ranking step after initial retrieval; that's the natural next thing to test, rather than assuming more chunks always means better search.

Queried the Local Embeddings Store for the First Time

Nerav Doshi — Mon, 24 Aug 2026 14:49:55 +0000

Context: Building the vector store in Entry 05 was only half the job — the actual point of embeddings is being able to ask a question and get back the right stored document, not just any document. Chroma does this by comparing distances: turn the question into a vector the same way you turned each document into one, then measure how numerically close the question's vector is to each stored vector. Lower distance means more similar in meaning. The real test isn't whether it returns something — it always will — it's whether the distance numbers actually mean anything, i.e. whether relevant matches consistently score lower than irrelevant ones.

Ran: Reconnected to the collection built in Entry 05 (collection.count() confirmed all 3 entries were still there), then ran two queries through the same embed-then-search pattern:

>>> q = ollama.embeddings(model="nomic-embed-text", prompt="how do I check pod status with oc")
>>> results = collection.query(query_embeddings=[q["embedding"]], n_results=2)
>>> results["ids"]
[['02-oc-cli-mentor-system-prompt.md', 'posts_03-1b-vs-3b-memory-comparison']]
>>> results["distances"]
[[437.72, 499.63]]

>>> bad_q = ollama.embeddings(model="nomic-embed-text", prompt="what's the best pizza topping")
>>> bad_results = collection.query(query_embeddings=[bad_q["embedding"]], n_results=2)
>>> bad_results["distances"]
[[542.33, 585.48]]

Result:

Query	Top match	Best distance	Worst distance
"how do I check pod status with oc"	`02-oc-cli-mentor-system-prompt.md`	437.72	499.63
"what's the best pizza topping"	(same 2 docs, wrong topic)	542.33	585.48

Two things worth calling out. First, the on-topic question correctly surfaced the oc-mentor file as the closest match — the system found the right document, not just a document. Second, and more convincing: every distance for the real question was lower than every distance for the pizza question — even the worst on-topic match (499.63) beat the best off-topic match (542.33). That's a real, if small, sample showing the distance metric isn't just noise; it's actually tracking relevance.

One loose thread: the third stored document — the long draft article that got truncated by 57% in Entry 05 — never showed up in either top-2 result, including for queries that might plausibly relate to it. Can't tell yet whether that's because it's genuinely less relevant to these two test questions, or because truncating away more than half its content damaged what got embedded. Worth testing directly once chunking replaces truncation.

Takeaway: The retrieval loop works — real semantic discrimination, not just returning whatever's closest by default. But this was a 3-document store and 2 test questions; that's a proof of concept, not a rigorous eval. The next real fix is the chunking work in Entry 07 — and specifically checking whether it changes how the long truncated document performs in queries like this one.

Managed Services vs Self-Hosted: When to Hand Over the Keys

Nerav Doshi — Fri, 21 Aug 2026 20:16:36 +0000

Managed Services vs Self-Hosted: When to Hand Over the Keys

☁ Cloud Without the Chaos — Article 02

You've placed the workload in the cloud. That decision is done. Now comes the one nobody gives you a clean framework for: do you run the infrastructure yourself, or do you hand the keys to a managed service?

I worked with a customer who answered this the hard way — twice. Once on their own platform, and once after they'd already made the switch. Both incidents involved the same underlying failure mode. Neither one turned out the way anyone in the room expected.

The warehouse analogy

In supply chain, this decision has a name: make vs. buy. Do you run your own warehouse, or use a third-party logistics provider (3PL)?

Running your own warehouse means you control everything — layout, staffing, hours, how fast you respond when a shipment needs to move at 2am. It also means you own every forklift breakdown and every understaffed shift.

A 3PL means someone else owns the building, the labor, the equipment. You focus on what you sell instead of how it's stored and moved. But you're also dependent on their systems, their SLAs, and their priorities when your shipment competes with someone else's for dock space.

Cloud infrastructure is the same trade in different clothes. Self-hosted is your own warehouse. Managed platforms are the 3PL. What follows is what that trade actually looked like for one team, in production, with real numbers attached.

The renewal that forced the question

The customer had a lean platform team self-managing Kubernetes — patching, logging, monitoring, upgrades, all of it in-house. Their platform renewal was coming up, expensive, and stacked against a team that was already stretched. The maintenance burden that made the renewal conversation painful had a specific shape: every major Kubernetes upgrade meant validating deprecated and removed APIs across their entire application estate, inside a fixed window, before the old control plane version went out of support.

That window was 30 days. The estate was roughly 1,400 microservices across 120+ namespaces. And on one upgrade cycle, something got missed.

The incident that made the case for managed

During a post-upgrade hotfix rollout, a Helm chart still referencing a removed Ingress API version hit the CD pipeline. The deployment failed. So the team did what you're trained to do — they rolled back.

The rollback failed too, and it failed for a specific, mechanical reason worth understanding rather than waving past. Helm doesn't recompute a prior release from source when you roll back — it retrieves the immutable historical manifest stored in a cluster Secret and re-applies it, which means that manifest still has to pass live API discovery validation on the way back in. Kubernetes 1.22 had removed the v1beta1 Ingress API group entirely. The API server had no way to accept a manifest written against an API version that no longer existed on the cluster. Helm halted before applying anything.

With the rollback path closed, the Ingress controller hit a state-sync failure. Payments traffic started returning 502s. Engineers resolved it by hand — removing orphaned resources, rewriting the manifests to the current API schema, and re-running the deployment.

Forty-five minutes of active, customer-facing outage on payments, before service was restored.

The team built a deprecation-scanning guardrail afterward. It was the right fix for the immediate problem. But it didn't touch the actual bottleneck, which wasn't technical — it was a small team with too many competing priorities trying to own an entire platform lifecycle by hand. That's what pushed the renewal conversation toward evaluating ROSA HCP and EKS as replacements, rather than just renewing what they had.

Why ROSA HCP over EKS

The deciding factor wasn't a feature checklist. It was a question of who owns which lifecycle. EKS would have meant the customer still assembling and operating most of the surrounding components themselves. ROSA HCP's integrated stack meant Red Hat owned Day 2 operations on the platform — the control plane, the surrounding components, the patching cadence — while the customer kept ownership of the application layer. For a team whose whole problem was "we can't keep operating all of this ourselves," that division of labor was the actual point — the same operational-ownership calculus that plays out anywhere a team decides what to stop managing directly.

The surprise: managed doesn't mean hands-off

Here's where the assumption cracked. The team expected that ROSA's managed Day 2 operations meant node draining, cordoning, and worker upgrades would happen the way a serverless abstraction behaves — invisibly, without touching running workloads.

That's not how it works. On ROSA HCP, control plane upgrades are decoupled and handled automatically by Red Hat. Worker node pool upgrades are not automatic in the same way — they require customer initiation, and once initiated, they're still governed by whatever workload configuration is sitting on top of those nodes.

A critical CVE forced the question. Red Hat's SREs patched the managed control plane within 24 hours of disclosure — the response time this engagement actually observed, not a contractual SLA figure. When the customer then initiated the worker pool upgrade, ROSA's automated drain process ran into a misconfigured PodDisruptionBudget (PDB) on a legacy service. The PDB was blocking pod eviction, so the node drain stalled.

The contrast that matters

This is the moment worth sitting with, because it's the clearest evidence for how this trade actually plays out.

On the self-hosted side, the team discovered their outage the way most self-hosted teams discover outages: customers saw 502s first. Forty-five minutes of active, visible damage before anyone had it under control.

On ROSA HCP, Red Hat's SRE automation caught the stalled drain and alerted the customer's team directly — before any workload was affected. The Machine Config Operator's drain process retried against the exponential backoff pattern it's designed to use rather than failing outright, so once the customer's application team fixed their own PDB configuration, the rollout resumed and completed on its own. No support ticket. No manual cluster-operator intervention from Red Hat.

That episode took 2 to 2.5 hours from alert to resolution — longer than the payments outage. But customer-facing impact was zero, because the misconfigured PDB had done its one useful job: it prevented running pods from being prematurely terminated while the drain was stuck.

Managed didn't mean faster. It meant the failure mode moved from "customer-facing outage you discover yourself" to "silent stall you get alerted to before anyone outside notices."

The exposure window nobody puts in the vendor deck

The CVE episode had a second layer worth naming plainly, because it's the part every "just move to managed" pitch tends to skip.

Red Hat's SREs patched the control plane in 24 hours. But because worker upgrades required customer initiation, and the customer held off starting that upgrade until their standard maintenance window — partly out of exactly the fear that then materialized as the PDB stall — the worker nodes running actual application workloads stayed unpatched for roughly four days. The 2.5-hour PDB delay was a small piece of that gap. The bigger piece was hesitation.

Managed infrastructure narrowed the attack surface. It didn't eliminate the scheduling risk sitting on the customer's side of the line.

What actually closes the gap

The fix wasn't anything Red Hat needed to change. It was proactive PodDisruptionBudget hygiene at the manifest level, applied consistently, so that node pool upgrades stop being a feared, manually-delayed event and become a routine background operation the team can trust. Once that trust exists, there's no reason to hold a CVE-driven worker upgrade for a scheduled maintenance window. The team's target shifted from a hesitant four-day window down to an automated 24-hour response following any control-plane security release — closing the exposure gap by removing the fear that created it.

Where this breaks at scale

At 120+ namespaces and 1,400+ microservices, PDB hygiene enforced only in CI/CD isn't enough. Pipeline linting has a blind spot: it can't see PodDisruptionBudgets generated at runtime by Operators, bundled inside third-party Helm charts, or applied directly via kubectl outside any pipeline at all. A policy that only lives in CI will pass clean while the cluster quietly accumulates exactly the kind of misconfiguration that stalled the worker upgrade in the first place. State drift on managed platforms follows the same pattern — the enforcement point that looked sufficient in isolation turns out to have blind spots once the estate gets large enough.

The customer's answer was to move enforcement to the cluster's front door — deploying Kyverno as a ValidatingWebhookConfiguration at the Kubernetes API server itself, so no PDB, regardless of where it came from, gets accepted onto the cluster without passing policy. It's deployed today. It hasn't yet been tested against a live attempted violation in production, so I'd call it the right architecture rather than a proven one — worth watching, not yet worth treating as settled.

The takeaway that doesn't show up in vendor documentation

The lesson here isn't "avoid managed services," and it isn't "always choose managed services" either. It's this: handing off infrastructure management doesn't reduce your governance burden. It relocates it, and intensifies it, at the API layer.

Once a vendor owns everything below the API — control plane, patching, node lifecycle — your job stops being "keep the cluster running." It becomes enforcing strict, API-level tenant policy so that the vendor's automation doesn't hit an invisible wall it has no way to see past. The vendor owns the warehouse. You still own what's on the shelves, and if what's on the shelves is stacked wrong, the automated forklift stops moving no matter how good the warehouse is.

A framework, not a rule

Three questions do most of the real work in this decision:

1. Is the scarce resource time, or is it control?
If the team is small and stretched, and the system isn't your competitive differentiator, lean managed. If it's core to what makes your product actually work, the operational knowledge from running it yourself has value beyond uptime.

2. Can you operate it today, under pressure — not eventually, today?
If not, that's not an argument for self-hosting. It's a sign you need training or headcount before this decision is even yours to make cleanly.

3. When it breaks, whose policy is actually holding it together?
Managed services don't remove your governance obligations. They just move the point of failure from "the platform" to "whatever policy your team enforces on top of it." Know which one you're actually testing before you sign the renewal.

None of these have a universal answer. They have an answer for your team, this quarter, with the staffing and policy discipline you actually have — not the version you're hoping to build eventually.

GitHub Repo

No companion repository for this article — this is a decision-framework piece about managed vs. self-hosted infrastructure, not an implementation walkthrough. Article 3 in this series returns to concrete implementation, where API-level governance becomes its own architecture decision.

What's Next

Cloud Without the Chaos, Article 03 picks up where this one leaves off: once you've handed the keys to a managed platform and enforced policy at the API layer, connectivity becomes the next architecture decision — VPNs, Direct Connect, ExpressRoute, and what actually breaks when two environments have to talk to each other reliably.

Built My First Local Embeddings Pipeline (Files + a Live URL)

Nerav Doshi — Fri, 21 Aug 2026 14:48:51 +0000

Context: An embedding model doesn't generate text — it converts text into a list of numbers (a vector) that represents the meaning of that text. Two pieces of writing about similar topics end up with similar vectors, even if they don't share any exact words, which is what makes semantic search possible: you can find "things that mean something like this," not just "things that contain this exact word." One detail that surprised me going in: the vector is always the same fixed length, no matter how long the original text is — a one-sentence note and a 10-page article both come out as the same-sized list of numbers, like a fingerprint that summarizes something much bigger into a fixed format. RAG (retrieval-augmented generation) is built on this: embed a bunch of documents, embed a question the same way, and find whichever documents are numerically closest to the question's meaning.

Ran: Built this up one command at a time instead of writing the full script upfront — easier to see what each piece actually does before combining them.

Step 1 — Pull the embedding model

ollama pull nomic-embed-text

Step 2 — Test the embedding API directly, before writing any Python

Ollama exposes embeddings over a local HTTP API, so you can sanity-check it works with a plain curl before involving any code:

curl http://localhost:11434/api/embeddings -d '{"model": "nomic-embed-text", "prompt": "hello world"}'

This returns a JSON object with a single embedding field — a list of 768 numbers. That's the whole idea of an embedding in one command: text in, fixed-length list of numbers out.

Step 3 — Install the Python pieces

pip3 install ollama chromadb requests beautifulsoup4 --break-system-packages

Step 4 — Embed one local file, interactively

Rather than run a full script blind, this is small enough to do a few lines at a time in a Python shell (python3):

import ollama
text = open("02-oc-cli-mentor-system-prompt.md").read()
response = ollama.embeddings(model="nomic-embed-text", prompt=text)
len(response["embedding"])   # → 768

Hit ModuleNotFoundError: No module named 'ollama' on the first attempt here — Step 3 hadn't been run yet. A reminder that even a five-step walkthrough has room to skip a step by accident.

Step 5 — Store it in Chroma

import chromadb
client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_or_create_collection(name="today_i_ran_notes")
collection.upsert(
    ids=["02-oc-cli-mentor-system-prompt.md"],
    embeddings=[response["embedding"]],
    documents=[text],
)
collection.count()   # → 1

Step 6 — Add a longer file, and hit a real limit

Repeating Step 4 against a longer draft article failed differently:

ollama._types.ResponseError: the input length exceeds the context length

nomic-embed-text has a 2048-token context window, and the longer draft (13,985 characters) blew past it. Quick fix for now — truncate to a safe length before embedding:

text = text[:6000]

Not the right long-term fix (more on that below), but enough to keep moving.

Step 7 — Add a live URL as a source

Pulling in a web page instead of a local file needs one extra step first: fetching the page and stripping out the HTML noise (nav bars, scripts, footers) so only the article text gets embedded.

import requests
from bs4 import BeautifulSoup

response = requests.get("https://pipelineandprompts.com/posts/03-1b-vs-3b-memory-comparison/",
                         headers={"User-Agent": "Mozilla/5.0"})
soup = BeautifulSoup(response.text, "html.parser")
for tag in soup(["script", "style", "nav", "footer", "header", "aside"]):
    tag.decompose()
article_text = soup.get_text(separator="\n", strip=True)
len(article_text)   # → 3359

That gets fed into the same ollama.embeddings() call from Step 4.

Result: Three sources ended up embedded across the two files plus the URL from the steps above:

Source	Type	Characters sent	Vector dimensions
`cloud-without-chaos-01.md`	local file	6,000 (truncated from 13,985)	768
`02-oc-cli-mentor-system-prompt.md`	local file	2,941	768
Entry 03 (live URL)	web article	3,359	768

Two things stood out. First, every vector came back at exactly 768 dimensions regardless of source length — confirms the "fixed-size fingerprint" idea from the context section wasn't just theory. Second, and more important: the truncation fix meant 57% of the longest draft's content (7,985 of 13,985 characters) never made it into its embedding at all. That's not a rounding error — over half the article is invisible to any future search against that vector. This is exactly why real RAG pipelines chunk long documents into smaller overlapping pieces instead of truncating: chunking keeps everything searchable, truncation just quietly throws away whatever didn't fit.

The URL fetch also worked cleanly — 3,359 characters extracted from the live page is close to the article's actual body length, suggesting the nav/footer/script stripping in the script did its job without pulling in much boilerplate.

Takeaway: The pipeline works end-to-end — files and live URLs, embedded into a queryable local store — but truncation is a real data-loss bug, not just a technical footnote, once you're working with anything longer than a short note. Entry 06 queries this store for the first time, and Entry 07 replaces truncation with real chunking. For the production-grade version of this — FastAPI, proper chunking, and a real API — see Build a RAG Pipeline for Internal Runbooks in the AI in the Stack series.

Compared Quantization Levels: Q4 vs Q8 vs FP16 on llama3.2:1b

Nerav Doshi — Wed, 19 Aug 2026 21:44:52 +0000

Context: A model's weights — the numbers it uses to reason — are normally stored at high precision, like FP16 (16-bit floating point). Quantization compresses those numbers down to fewer bits (Q8, Q4, etc.) to shrink the file size and memory footprint, trading some numerical precision for big space savings. It's a similar idea to compressing a high-resolution photo into a smaller JPEG: the file shrinks a lot, some detail is technically lost, but for most practical purposes it's hard to tell the difference.

Ran: Before pulling anything new, ran ollama show llama3.2:1b to check what Entry 01's baseline model actually was — and it turned out to already be Q8_0, not an unspecified "default" as I'd assumed back in Entry 01. Worth a correction: Entry 01 and Entry 03's 1b numbers were Q8, not full precision. Pulled two more variants for comparison: llama3.2:1b-instruct-q4_K_M (an explicit 4-bit quantization) and llama3.2:1b-instruct-fp16 (full, uncompressed precision). Loaded each in turn, sent the same test question, and captured ollama ps / ps aux for all three quantization levels.

Result:

	Q4_K_M	Q8_0 (Entry 01 baseline)	FP16
Download size	807 MB	1.3 GB	2.5 GB
`ollama ps` loaded size	997 MB	1.5 GB	2.7 GB
Process RSS	~0.94 GB	~1.24 GB	~2.57 GB

Memory scales cleanly with bit-width across all three: roughly 0.94 GB → 1.24 GB → 2.57 GB as precision doubles from 4-bit to 8-bit to 16-bit. That's a tidier relationship than I expected going in — quantization level is a solid, predictable lever for memory sizing.

For quality, I asked all three quantization levels the same real question: "what are good free AI tools that can create simple Google Slides using instructions?" None gave a clean answer. All three produced long lists mixing legitimate tools (Canva) with tools that have nothing to do with making slides — Q4 suggested Midjourney and DALL-E, FP16 added Artbreeder, Deep Dream Generator, and Prisma to the same mix. Full precision didn't fix this: FP16's list was just as padded with irrelevant suggestions as Q4's, and Q4 wasn't noticeably worse.

Takeaway: Memory scaled cleanly and predictably with quantization level — roughly doubling at each step from Q4 to Q8 to FP16, matching the bit-width math almost exactly. Answer quality told a different story: going all the way to full precision didn't reduce hallucinated suggestions on this open-ended recommendation question, and Q4 wasn't visibly worse than FP16. That's not proof quantization is "safe" in general — this was one soft question, not a rigorous eval — but it's a reminder that the real quality gap probably shows up on harder tasks (math, code, precise instruction-following), not casual recommendation queries. Also worth remembering going forward: check ollama show <model> before assuming what quantization a "default" pull actually gives you. This corrects assumptions from both Entry 01 and Entry 03.

$ ollama show llama3.2:1b
  quantization        Q8_0

$ ollama show llama3.2:1b-instruct-q4_K_M
  quantization        Q4_K_M

$ ollama ps
NAME                         ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.2:1b-instruct-fp16    2887c3d03e74    2.7 GB    100% GPU     4096       4 minutes from now

$ ps aux | grep ollama
flyers  10763  0.3  15.7  438101456  2630128  ??  S  llama-server --model ... -c 4096
flyers   2256  0.0   0.3  436826160    56448  ??  S  ollama serve

$ ollama ps
NAME                           ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.2:1b-instruct-q4_K_M    22bc6b92eb01    997 MB    100% GPU     4096       4 minutes from now

$ ps aux | grep ollama
flyers  10786  0.2  5.7  436484688  963808  ??  S  llama-server --model ... -c 4096
flyers   2256  0.0  0.4  436826160   60400  ??  S  ollama serve

Agentic AI Infrastructure: What It Takes to Do It Safely

Nerav Doshi — Wed, 19 Aug 2026 03:00:06 +0000

⚡ Byte Size Summary

See why we shipped an OpenShift diagnostic MCP server as read-only by design, and the RBAC wall that made write access harder than it looks

Walk through a real failed remediation test where an agent recommended a correct-looking fix built on stale, deprecated config — and what that failure mode actually is

Get the maturity-gated approval architecture we designed for write access — and why it's still sitting on paper, not in production

The Story

In Article 06 we wired an n8n workflow to MCP and RAG for automated incident triage. That article ended with a question: what happens when the agent gets a longer leash?

We built mcp-sre-tools — an MCP server that exposes OpenShift and Kubernetes diagnostics to an LLM, wired into Claude Desktop and n8n, covering ARO, ROSA HCP, OSD-GCP, and generic clusters. Nine diagnostic tools: get_cluster_health, diagnose_crashloop, get_failing_pods, and others in that family. READ_ONLY_MODE is on by default, and there are no write tools in the codebase at all. That part shipped clean.

The friction started when we scoped what came next: a remediation mode, where the agent wouldn't just diagnose a broken deployment — it would patch it.

That's where the story stopped being a build story and became an organizational one.

The Problem

Platform engineers and developers landed on opposite sides of the same question almost immediately, and for reasons that turned out to be more substantial than the usual risk-aversion reflex.

Developers were comfortable trusting agent-proposed changes roughly the way they'd trust a colleague's pull request — read the diff, sanity-check it, merge it. Platform engineers pushed back hard, and their objection wasn't reflexive. It was specific: a PR from a colleague comes with inspectable reasoning. You can ask them why. An LLM's proposed patch doesn't carry that same trail — the "why" is buried in a forward pass, not a code review comment.

Business stakeholders, meanwhile, were worried about something simpler and more immediate: an autonomous agent breaking a critical application in production.

Three legitimate concerns, three different vocabularies for the same underlying question — how much do we trust a system whose reasoning we can't fully inspect, applied to infrastructure we can't afford to break?

Why RBAC alone doesn't solve it

The instinct is to reach for RBAC and call it solved. Scope the agent's service account to a namespace, give it patch permissions on Deployments and nothing else, and let it operate inside a fence.

That fence has a hole in it. Meaningful remediation almost always eventually needs to touch Secrets or environment variables — a misconfigured database connection string, an expired credential reference, a missing env var causing a crash loop. The moment your remediation scope includes Secrets, "namespace-scoped RBAC" stops being clean sandboxing and starts being a much bigger trust surface than the phrase implies.

We didn't have a way around that with RBAC alone. So we fell back to a narrower, honest justification for read-only: even without write access, a diagnostic agent cuts human mean-time-to-resolution. It's a smaller value proposition than full self-healing, but it's a real one — and it's the one we could actually defend without hand-waving.

The Architecture

Diagram 1 — as built: the shipped, read-only MCP architecture.

The controls we designed the server to work with — note the repo intentionally ships without a default rbac.yaml, to stay adaptable across cluster types and org policies. Deployment teams are expected to write their own scoped ClusterRole/RoleBinding tailored to their access model; the sample below shows the shape we recommend, not a default that ships:

Namespace-scoped RBAC (recommended, not shipped) — bind the MCP server's service account to Role/RoleBinding resources scoped per-namespace, not a cluster-wide ClusterRole
Service-account-based access — no static kubeconfig or personal credentials in the agent's execution path
NetworkPolicy egress/ingress restriction — the MCP server's pod network is fenced to only the cluster API and the LLM endpoint it needs to reach
Logging and observability as non-functional requirements — every tool call is logged, not bolted on after the fact

# Recommended shape, not a shipped default — deployment teams write their own
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: mcp-sre-tools-reader
  namespace: <target-namespace>
rules:
  - apiGroups: [""]
    resources: ["pods", "events"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments", "replicasets"]
    verbs: ["get", "list", "watch"]

Diagram 2 — proposed, never implemented: the maturity-gated write-access flow.

⚠️ This is a design artifact, not a shipped system. The kill switch and rollback automation shown below were never built.

  ┌─────────────────────-┐
  │   Anomaly detected   │
  └──────────┬───────────┘
             │
             ▼
  ┌─────────────────────────────-┐
  │  Agent proposes RBAC-scoped  │
  │      remediation action      │
  └──────────┬───────────────────┘
             │
             ▼
  ┌─────────────────────────────-┐
  │      Risk-based routing      │
  │  low risk        high risk   │
  └──────┬───────────────┬───────┘
         │               │
         ▼               ▼
  ┌─────────────┐ ┌─────────────────────-┐
  │ Light review│ │  Human approval gate │
  └──────┬──────┘ └──────────┬───────────┘
         │                   │
         └─────────┬─────────┘
                   |
                   ▼
        ┌───────────────────────-┐
        │   Kill-switch check    │  ◄── [NEVER BUILT]
        └───────────┬────────────┘
                    │  pass
                    ▼
        ┌───────────────────────-┐
        │  Execute + full audit  │
        │        record          │
        └───────────┬────────────┘
                    │
                    ▼
        ┌───────────────────────-┐
        │ Pre-captured rollback  │  ◄── [NEVER BUILT]
        │   plan (ready before   │
        │      execution)        │
        └───────────┬────────────┘
                    │
                    ▼
        ┌───────────────────────┐
        │    Outcome logged     │
        └───────────────────────┘

The flow we designed, on paper, for a future write-capable mode:

Anomaly detected
Agent proposes an RBAC-scoped remediation action
Risk-based routing — low-risk actions get lighter review, high-risk actions route to human approval
Kill-switch check before execution
Execution, with a full audit record
Pre-captured rollback plan available before the action runs, not written after
Outcome logged

Two components in that list were never built: the kill switch and the rollback automation. They exist as a design, not as shipped capability. Worth saying plainly, because it's the honest state of the project, not a gap we're hiding.

How It Works: Step by Step

For the part that is live — the read-only diagnostic path:

An alert or a manual query triggers the agent via Claude Desktop or an n8n workflow
The agent calls one or more of the nine MCP tools (e.g., diagnose_crashloop) against the target cluster
The MCP server's service account, scoped by namespace RBAC, executes the read-only API calls
Results return to the LLM, which synthesizes a diagnosis
A human reads the diagnosis and decides what to do next — the agent stops there

For the part that stayed on paper — the proposed remediation path — see the seven-step flow above. It never advanced past step 2 in production; steps 3 through 7 are design artifacts.

Security and Operational Considerations

RBAC and least privilege. The repo ships without a default rbac.yaml by design, to stay adaptable across cluster types — deployment teams write their own scoped ClusterRole/RoleBinding for their environment. That's also where the write-mode proposal broke down: remediation that needs to touch Secrets can't stay inside a tidy read-only-style RBAC boundary no matter who authors it.

Secrets exposure. The live tool set never reads or writes Secret contents. The unbuilt remediation mode is exactly where that boundary would have been tested, and wasn't.

Blast radius. Contained by design in the shipped version — nine read-only tools can't mutate cluster state, full stop.

Rollback strategy. For the live tools: not applicable, nothing is mutated. For the proposed write mode: rollback was designed as a pre-captured plan, generated before execution rather than reconstructed after a failure — but this was never implemented, and the one real recovery event we had (below) had to be handled manually.

Auditability. Every diagnostic tool call is logged. The proposed write-mode flow adds a full audit record as one of its seven stages — again, unbuilt.

What Breaks at Scale

We ran a private test — outside the public repo — with write/patch/upgrade-capable variants of the agent, specifically to see whether the self-healing story held up. It didn't, and the way it failed is the most important finding in this whole project.

The agent correctly diagnosed a real problem. Then it recommended a fix built on fluentd-era OpenShift Logging configuration — the collector layer — against a cluster already running the current Vector-and-Loki stack. The recommendation looked plausible. It targeted a collector and CRD shape that the cluster had already moved past.

At close to the same time, in the same test, the agent made a second, unrelated bad call: a recommendation to change the cluster's node-level hardware settings via the worker MachineSet. That recommendation, applied, provisioned the wrong instance type.

Two independent, unrelated bad recommendations landing at the same time — one on the logging stack, one on node hardware — made it look, in the moment, like a single cascading failure. It wasn't. Untangling that during recovery took longer than fixing either problem alone would have, precisely because the two failures got conflated. Recovery meant two separate fixes, not one clean GitOps revert: an updated ClusterLogging CR to throttle ingress at the collector layer and redirect output pipelines directly to the Loki backend, and a reconciliation of the worker MachineSet back to its approved providerSpec baseline — scaling down the misconfigured nodes and letting the Machine Operator provision replacements.

The label that matters here isn't "hallucination" in the dramatic sense people usually mean. The logging recommendation wasn't nonsense — it was training-data staleness masquerading as competence: correct-sounding reasoning built on a collector and config shape the cluster had already moved past. That's a more insidious failure mode than the black-box framing usually implies, because the output looks exactly as confident whether the underlying knowledge is current or stale.

The instance-type recommendation was a different kind of mistake — not stale knowledge, just a bad call. What made the incident harder to diagnose wasn't either failure alone, but the two landing together and looking, briefly, like one problem instead of two.

Scaling this up doesn't just mean more permissions — though it does mean that, and the platform team's original objection was concrete and correct on that point alone. It also means the bottleneck moves. Once you gate every write action behind human approval, the approval step itself becomes the constraint, and approval fatigue doesn't scale linearly with the number of things an agent proposes.

GitHub Repo

Full implementation: mcp-sre-tools

What's Next?

The open question this leaves for the series: what does a safe path to write access actually look like, given that the blocker wasn't RBAC mechanics but the reliability of the reasoning behind each proposed action? That's worth its own piece.

Browse the full AI in the Stack series to see how the components — from evaluation framework through RAG, MCP, prompt governance, provider abstraction, and n8n orchestration — connect.

Compared llama3.2:1b vs llama3.2:3b Memory Footprint

Nerav Doshi — Fri, 14 Aug 2026 17:57:01 +0000

Context: The number in a model name like 1b or 3b refers to parameters — roughly, the tunable values inside the model that encode what it's learned. More parameters generally means better reasoning and more nuanced answers, at the cost of more memory and slower responses. One thing that trips people up early with Ollama: typing ollama run <model> drops you into an interactive chat session (marked by the >>> prompt), which is a different context from your regular shell. Commands like ollama ps only work back in a normal terminal prompt — typed inside the chat session, they get sent to the model as a question instead of running as a command.

Ran: Pulled llama3.2:3b (the 3-billion-parameter sibling of Entry 01's 1b model), loaded it into memory, and captured the same ollama ps / ps aux | grep ollama numbers for a direct comparison. Along the way, typed ollama ps inside the chat session by mistake — got a confused response from the model instead of the process table, a good real-world example of the shell-vs-chat distinction above. Then asked the same test question from Entry 02 ("how do I check the status of pods in my namespace?") — this time against the plain llama3.2:3b model, not the constrained oc-mentor build from Entry 02.

Result:

	`1b` (Entry 01)	`3b`
`ollama ps` size	1.5 GB	2.5 GB
Process RSS	~1.24 GB	~2.47 GB
Parameters	1B	3B

Tripling the parameter count roughly doubled the memory footprint — not a 1:1 scaling, which is worth remembering when estimating resource requests for larger models.

On the question test: since this run used the plain 3b model rather than the oc-mentor Modelfile from Entry 02, the answer came back as a verbose, multi-option explanation using kubectl — not oc, and not the single-command format Entry 02 enforced. That's not a knock on the bigger model; it's a reminder that the constrained, single-command behavior from Entry 02 came from the system prompt, not from model size. A bigger base model without that constraint just reverts to its default training bias (which, unsurprisingly, leans kubectl over oc).

Takeaway: More parameters bought roughly 2x memory for 3x the parameter count — a useful data point for future sizing — but it didn't buy domain-specific behavior on its own. Getting oc-only, single-command answers still requires the system prompt from Entry 02, regardless of model size.

$ ollama ps
NAME           ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.2:3b    a80c4f17acd5    2.5 GB    100% GPU     4096       4 minutes from now

$ ps aux | grep ollama
flyers  94626  0.2  15.1  438021552  2531232  ??  S  llama-server --model ... -c 4096
flyers   1905  0.0   0.4  436904528    60864  ??  S  ollama serve

Correction (Aug 19, 2026): the 1b baseline referenced from Entry 01 is Q8_0 quantization, not an unspecified default — see Entry 04 for the full breakdown across quantization levels.

The Hybrid Cloud Isn't a Destination. It's a Set of Tradeoffs.

Nerav Doshi — Wed, 12 Aug 2026 19:09:17 +0000

☁ Cloud Without the Chaos — Article 01 · ⏱ 10 min read

☁ Byte Size Summary

Hybrid cloud isn't a strategy — it's the outcome of real architectural decisions

Most teams default to cloud or on-prem for the wrong reasons (inertia, not intent)

Five dimensions drive every defensible placement decision: latency, data sovereignty, cost profile, operational ownership, and blast radius

A reusable decision framework for evaluating any workload — run it before the migration starts, not after

Network design thinking from logistics applies directly to hybrid cloud architecture — and most platform engineers have never seen it framed this way

I spent a few years working in supply chain and logistics before I ended up in cloud infrastructure. One of the first things you learn in that world is that the network is never fully yours.

You've got your own warehouses. You've got third-party fulfilment centres you use when demand spikes. You've got last-mile carriers you don't control at all. The goods flow across all of it depending on cost, speed, capacity, and — critically — what you're willing to risk having disrupted.

Nobody in logistics calls this a "hybrid warehouse strategy." They just call it running a network. The goal isn't ideological purity about who owns the infrastructure. The goal is getting the right things where they need to be, reliably, at a cost that makes sense.

When I moved into cloud and infrastructure, this framing was almost entirely absent. Teams were either "cloud first" or "on-prem forever" — both as fixed identities, not deliberate decisions. When hybrid cloud entered the picture, it was less of a strategy and more of a situation that accumulated over time.

This series is about bringing some rigour back to those decisions. Not the marketing version of hybrid cloud — the practical one.

What "Hybrid Cloud" Actually Means

Strip the vendor positioning out first.

Hybrid cloud means running workloads across at least two environments — typically some combination of on-premises infrastructure and one or more public cloud providers — with some degree of orchestration or connectivity between them. No requirement for a unified control plane, a specific vendor stack, or a particular ratio of workloads on each side.

In practice, most enterprise organisations are already hybrid whether they planned it or not. Legacy systems that can't be migrated sit in the data centre. New workloads spin up in AWS or Azure. An OpenShift cluster runs on-prem for regulated workloads. A Kubernetes service runs in Google Kubernetes Engine (GKE) for the developer-facing platform. The network connects them, imperfectly, and the operations team holds it all together with documentation, duct tape, and institutional memory.

The question isn't whether you're hybrid. It's whether you're hybrid on purpose.

This article focuses on the strategic decision layer — where workloads should live and why. The technical implementation of connectivity (VPNs, Direct Connect, ExpressRoute) comes later in the series.

Why Most Hybrid Architectures Are Accidents

Most hybrid architectures evolve the same way.

A team starts running everything on-prem because that's what they know and what the business already owns. At some point there's a project that needs to move fast, or a new team that doesn't want to wait for physical provisioning, and something lands in a public cloud. Then another thing. Then three more things during a peak traffic event that nobody had capacity for on-prem. Now you're hybrid.

Or the reverse: a cloud migration programme moves a significant portion of the estate to public cloud. But some workloads — the database with the compliance requirements, the legacy system that can't be containerised, the one application that can't tolerate a 100ms latency spike — stay behind. Now you're hybrid.

Neither is necessarily wrong. But both share a problem: the placement decisions were made reactively, and nobody wrote down the reasoning. Two years later, a new engineer reverse-engineers why a workload is where it is, and the answer is "history."

That's a real cost. It makes it harder to have intelligent conversations about what should move, what should stay, and what the actual cost model is.

The Five Dimensions That Actually Matter

After working across on-prem and cloud environments — and spending years in supply chain thinking about network design — five dimensions consistently drive the right placement decision. Answer these clearly and you'll reach a defensible architectural position.

1. Latency tolerance

How sensitive is this workload to network round-trip time? A real-time manufacturing control system talking to floor sensors has hard latency requirements that cloud can't reliably satisfy. A reporting pipeline that runs overnight doesn't. This is easy to underestimate — especially for workloads that look like batch jobs but have real-time dependencies buried inside them.

2. Data sovereignty and compliance

Where does the data have to live? Regulated data in healthcare, financial services, or government frequently has hard requirements about physical location, access controls, and audit trails. Public cloud providers have made significant progress with regional isolation and compliance certifications — but not every jurisdiction accepts "it's in an EU data centre" as sufficient. Know the actual requirements before you architect.

3. Cost profile and demand shape

Cloud is cheaper at the margins — burst capacity, unpredictable demand, short-lived environments. On-prem is cheaper at steady state — predictable, high-utilisation workloads where you'd be running equivalent cloud compute continuously. Applying cloud economics to steady-state workloads, or on-prem economics to spiky ones, is the most common cost mistake in hybrid architecture. The shape of demand matters as much as the raw compute cost.

4. Operational ownership appetite

Cloud shifts operational responsibility to the provider for the infrastructure layer, but it doesn't eliminate operational work — it changes it. You're still responsible for your application, your data, your network configuration, your identity model. A managed Kubernetes service in the cloud may be cheaper in operational time than a self-managed OpenShift cluster, even if the compute costs are higher. Or it may not be, depending on your team. Be honest about this.

5. Blast radius and failure domain

If this workload fails, or the environment it runs in fails, what's the impact? Placing a critical system on a single public cloud introduces a blast radius tied to that provider's regional availability. Placing it on-prem introduces a different blast radius — your data centre's power, cooling, and network uplinks. Neither is inherently safer. The question is which failure mode you have more control over, more visibility into, and more organisational capacity to recover from.

A Decision Framework You Can Actually Use

Dimension	Question	Signal
LATENCY	Sub-5ms requirements, or direct integration with on-prem systems?	→ On-prem or edge. Everything else is negotiable.
SOVEREIGNTY	Hard data residency requirement your cloud provider can't satisfy with certifications?	→ On-prem or private cloud. If they can satisfy it, document that explicitly.
DEMAND SHAPE	Consistent, predictable utilisation above ~60%?	→ On-prem likely wins on cost. Spiky, seasonal, or unpredictable? → Cloud wins.
OPERATIONS	Does your team have the skills and bandwidth to manage the infrastructure layer?	→ On-prem or self-managed. Would a managed service free up meaningful capacity? → Cloud.
BLAST RADIUS	Which failure mode is more recoverable — cloud regional outage, or on-prem infrastructure failure?	→ Architect toward the one you can recover from faster.

Run any workload through these five questions. If the answers point consistently in one direction, the decision is clear. If they conflict — and they often do — you're having an explicit architectural conversation rather than an implicit one. That's already progress.

The Placement Decision Table

Workload type	Typical best fit	Reason
Real-time control systems	On-prem / edge	Latency and reliability requirements
Regulated data stores (PII, PCI, PHI)	On-prem or certified cloud	Sovereignty — verify per jurisdiction
Dev/test environments	Cloud	Short-lived, variable, low blast radius
Burst compute (ML training, batch jobs)	Cloud	Elastic capacity on demand
Internal developer platforms	On-prem or hybrid	Integration with internal systems, security posture
Customer-facing APIs	Cloud	Global distribution, managed scaling, CDN integration
Legacy monoliths (can't containerise)	On-prem	Migration cost exceeds benefit — for now
Disaster recovery / backup	Cloud	Offsite resilience, pay-per-use when idle

What the Supply Chain World Understood First

In logistics, there's a discipline called network design — deciding where to place inventory, distribution centres, and transport capacity to optimise for cost, speed, and resilience across a network. It's not about owning everything. It's not about outsourcing everything. It's about understanding the specific characteristics of each node and flow, and making placement decisions accordingly.

The most important lesson from that world: over-centralising creates fragility, and over-distributing creates complexity that erodes the resilience you were trying to build. The answer is never all-in on one model. It's a considered network, with explicit reasoning behind each decision.

Hybrid cloud is the same problem. The data centre is your warehouse. The cloud is your third-party fulfilment network. Some workloads belong in your warehouse because you need control, compliance, or latency. Others belong in the fulfilment network because you need elasticity and don't want to own the capacity. The failure mode is treating this as a binary choice rather than a network design problem.

💡 From the field: Before any migration or placement decision, create a simple workload inventory: what it does, what it talks to, what its latency and compliance requirements are, and what it currently costs to run. Most teams don't have this and make significant architectural decisions without it. A rough spreadsheet surfaces conversations that would otherwise never happen.

The Honest Tradeoffs

There's no version of hybrid cloud that's free of complexity. Running across two environments means two sets of networking concerns, two operational models, two security boundaries, and often two teams with different toolsets. That's real cost, and it's worth naming before you commit.

The question isn't whether hybrid cloud creates complexity — it does. The question is whether the complexity is worth it relative to the alternative. For most enterprise organisations with a heterogeneous workload mix and real compliance requirements, the answer is yes. But "yes" should come from an analysis, not from a vendor briefing or a conference keynote.

The teams that get this right treat placement as a first-class architectural concern — documented, revisited periodically, and tied to actual cost and operational data. Not a one-time migration project. Not a fixed philosophical stance. A living decision.

Key Takeaways

Hybrid cloud isn't a destination — it's a network of deliberate placement decisions
Most hybrid architectures are accidents — the goal is to make yours intentional
Five dimensions drive placement — latency, sovereignty, cost profile, operational ownership, blast radius
Treat workload placement like network design — optimise for the whole, not the individual node
Complexity is unavoidable — the question is whether it's justified by the requirements

What's Next in Cloud Without the Chaos

Article 02 — Managed Services vs Self-Hosted: When to Hand Over the Keys

You've placed the workload in the cloud. Now: do you manage the infrastructure yourself, or hand it to a managed service? The next article works through the real tradeoffs — including the moment a managed service saves your team 20 hours a week, and the moment it quietly doubles your incident response time.

Read the Cloud Without the Chaos series index for the full learning path.