DEV Community: kirandeepjassal-crypto

MCP Deep Dive, Part 1: Why Model Context Protocol Kills Integration Glue Code for Good

kirandeepjassal-crypto — Sat, 04 Jul 2026 11:18:13 +0000

Your AI roadmap does not die from a bad model. It dies from integration glue code — the hand-written adapter that wires agent number four to backend number nine, times every agent and every backend you will ever build. Model Context Protocol (MCP) is the thing that stops that multiplication.

This is Part 1 of a 15-part deep dive. Every part uses the same running example: Mattrx, our multi-tenant marketing-analytics SaaS (.NET 9 / Azure), and every metric here is from that real system.

TL;DR

Dimension	Before (bespoke glue)	After (MCP)
Integration model	N agents × M backends	N agents + M servers
Mattrx integrations	14 point-to-point clients	3 MCP servers
Adding a capability	New adapter on both sides	Declare one MCP tool
Tool discovery	Hardcoded per agent	Discovered at runtime
Auth & audit	Reinvented per integration	One OAuth/Entra boundary
External AI access	Unsafe / not possible	Scoped, governed, audited

14 bespoke integrations collapsed to 3 MCP servers.
~9,000 lines of glue code deleted — roughly a 40% cut.
New-capability onboarding dropped from ~3 days to ~2 hours.
Agent tool-call error rate fell from 6% to 0.8%.
~85,000 MCP tool calls/day, all governed by the same boundary.
~40 tool-abuse / injection attempts per week blocked at the MCP boundary.

The one mental shift: stop building integrations and start publishing capabilities. An integration teaches one agent how to call one backend. A capability is a tool any agent can discover and call from its schema alone. MCP makes capabilities additive instead of multiplicative.

The N×M problem

With N agents and M backends, you write up to N×M integrations, and each one re-implements auth, retries, error mapping, and logging in its own slightly-wrong way.

BEFORE — N agents x M backends = up to N*M bespoke integrations

Insights ---+--> Campaigns API (custom client)
            +--> Events API   (custom client)
            +--> KPI API      (custom client)
            +--> Reporting API (custom client)

Help -------+--> Campaigns API (a DIFFERENT custom client)
            +--> KPI API       (a DIFFERENT custom client)

External AI ....> (no safe path at all)


AFTER — N agents + M servers = N+M, one protocol

Insights ---+
Help -------+           +--> mattrx-analytics (campaigns, events, kpis)
External AI +--- MCP ---+--> mattrx-reports  (create_report, status)
(approved) -+           +--> mattrx-admin    (flags, exports; locked)

1. The integration explosion

Before, every agent embedded a bespoke client for every backend:

// BEFORE: the agent is welded to four hand-written clients.
public sealed class InsightsAgent(
    CampaignsApiClient campaigns,   // bespoke HTTP client #1
    EventsApiClient events,         // bespoke HTTP client #2
    KpiApiClient kpis,              // bespoke HTTP client #3
    ReportingApiClient reporting)   // bespoke HTTP client #4
{
    // Each client has its own auth, retry policy, and error model.
    // The next agent we build re-implements a slice of all four.
}

After, each capability is declared once as an MCP tool:

// AFTER: a capability declared once on the mattrx-analytics MCP server.
[McpServerToolType]
public sealed class AnalyticsTools(ICampaignQueries campaigns, AiPrincipal principal)
{
    [McpServerTool(Name = "get_campaign_kpis")]
    [Description("Return the KPI time-series for a campaign in the caller's tenant.")]
    public async Task<CampaignKpis> GetCampaignKpis(
        [Description("Campaign id within the caller's tenant")] string campaignId,
        [Description("ISO-8601 range, e.g. 2026-06-01/2026-06-28")] string range,
        CancellationToken ct)
    {
        // Tenant comes from the authenticated principal — never from the arguments.
        return await campaigns.GetKpisAsync(principal.TenantId, campaignId, range, ct);
    }
}

The agent side collapses to one client that speaks MCP to every server:

var result = await mcp.CallToolAsync(
    "get_campaign_kpis",
    new { campaignId = "4821", range = "2026-06-01/2026-06-28" },
    ct);

Result: 14 integrations → 3 servers, ~9,000 lines of glue deleted, almost all deletions.

2. Capability discovery

Before, the toolset was a constant the agent was compiled with — the list and reality drift. After, the server advertises its tools and the client discovers them at runtime:

// AFTER: the agent asks the server what it can do, every session.
var tools = await mcp.ListToolsAsync(ct);
// each tool: name, description, JSON Schema for args — enough for an LLM to
// decide when and how to call it, with zero hardcoding.

Discovery is the quiet superpower: ship a new tool on the server, and every agent can use it next session. Onboarding a capability went from ~3 days to ~2 hours.

3. One auth and audit boundary

Before, every bespoke client reinvented auth (one static key, one OAuth scope, one that trusted a tenant id passed as an argument — the bug we shipped). After, every tool call enters through one MCP boundary:

// AFTER: one boundary enforces auth, scope, tenant binding, and audit for ALL tools.
public sealed class GovernedToolFilter(AiPrincipal principal, IAuthorizationService authz, IAiAuditLog audit)
{
    public async Task<ToolResult> InvokeAsync(McpToolCall call, Func<Task<ToolResult>> next, CancellationToken ct)
    {
        var decision = await authz.AuthorizeAsync(principal, call.RequiredScope, ct);
        if (!decision.Allowed) return ToolResult.Denied(call.RequiredScope);

        var result = await next();                 // tenant already bound from the token
        await audit.RecordAsync(principal, call, result, ct);
        return result;
    }
}

One OAuth 2.1 / Entra ID boundary replaced N bespoke auth flows. Tool-call error rate fell from 6% to 0.8% — most of those errors were auth and contract mismatches that simply stopped existing.

4. A safe door for external AI

Before, a partner wanting their AI assistant to pull your KPIs meant "build them yet another client" — so the answer was "no." After, an approved external assistant authenticates via Entra ID, gets a token scoped to its tenant and to campaigns:read, and calls the exact same tools our internal agents do — discovered, scoped, and audited identically. A capability that simply did not exist under bespoke integration.

What an MCP call actually looks like

The protocol is small — three message types do almost all the work: initialize, tools/list, tools/call, all JSON-RPC over the transport (Streamable HTTP + SSE in production, stdio in local dev). That small surface is the point: it's small enough that any client and any server can implement it, which is exactly what makes capabilities additive.

When NOT to adopt MCP

One agent, one backend (1×1). A direct method call is simpler and faster.
A stable internal toolset with no external consumers. The additive win is theoretical.
Ultra-low-latency hot paths. MCP adds a hop and JSON-RPC framing.
Auth is still a mess. MCP's value compounds with one identity provider.
You haven't shipped a v1. Build the naive integration first; adopt MCP when N or M actually grows. We did it at integration fourteen, not two.

The model to carry forward

Integrations scale as N×M. Protocols scale as N+M. Every bespoke client you write is a multiplication you'll pay for again with the next agent. Every capability you publish as an MCP tool is an addition every future agent gets for free.

Publish capabilities, not endpoints. Design each tool as a contract an unfamiliar agent can call from its schema alone.
Put one identity boundary in front of every tool. One OAuth/Entra door, scoped per tool, tenant bound in code.
Treat tool schemas as your public API. Version them, document them, break them carefully.

Originally published on PrepStack. Adopting MCP and want a second pair of eyes on where to draw your server boundaries? Reach me at randhir.jassal[at]gmail.com.

AI Code Review That Engineers Actually Trust: The Pipeline We Run on Every Pull Request

kirandeepjassal-crypto — Fri, 03 Jul 2026 18:37:46 +0000

Bolting an LLM onto your pull requests is a weekend project. Building AI code review that your engineers don't disable within two weeks is the actual problem. The failure mode isn't missing bugs — it's crying wolf. Post twenty nitpicks and three hallucinations on someone's PR and they'll mute the bot forever. This is the pipeline we built on Mattrx to earn — and keep — that trust.

Mattrx is our multi-tenant marketing-analytics SaaS: ~95k lines of C#, 11 engineers, and enough pull requests that senior-reviewer time was the bottleneck. We tried the naive thing first — pipe the changed file into a model, post the output — and watched the team stop reading it in nine days.

TL;DR

Dimension	Human-only / naive AI (before)	AI review pipeline (after)
Coverage	selective / whole-file dump	every PR, diff-focused
First-review latency	~6 hours (wait for a human)	~3 minutes (AI first pass)
Context	none / a naked file	diff + call sites + conventions
Reviewers	one mega-prompt	specialized dimensions, in parallel
False positives	~35% (so it gets ignored)	~6% (adversarially verified)
Merge control	human, or nothing	severity gate; human always decides
Governance	none	gateway: audit, cost, secret redaction

~90 PRs/week across 11 engineers; the pipeline reviews 100%.
First-pass review latency 6h → 3 min.
False-positive rate ~35% → ~6% — the single number that decides whether the bot lives or dies.
Escaped defects to production down ~40%; senior-reviewer time down ~30%.
~$0.05 per PR (cheap model for style, frontier only for correctness).

The one mental shift: AI code review is not about finding issues — models find plenty. It's about not crying wolf. The product is trust, and trust is a false-positive-rate problem. Verify before you comment; let the AI propose and the human dispose.

The naive approach — and why it collapses

// BEFORE: dump the whole changed file into one prompt, post whatever comes back.
foreach (var file in pr.ChangedFiles)
{
    var text = await File.ReadAllTextAsync(file.Path, ct);
    var review = await model.CompleteAsync($"Review this code and list problems:\n{text}", ct);
    await github.PostCommentAsync(pr, review); // a wall of unstructured, often-wrong text
}

It reviews the whole file, not the change. It has no project context, so it flags your conventions as bugs. No severity — a missing null-check and a stylistic preference arrive with equal weight. And no verification, so every hallucination goes straight to the developer. The result is a ~35% false-positive rate and a team that learns, correctly, to ignore the bot.

1. Context assembly — review the change, not the file

Build a review context: the diff (only what changed), the call sites of the symbols the change touches, and the project conventions for those files.

public async Task<ReviewContext> BuildAsync(PullRequest pr, CancellationToken ct)
{
    var diff = await git.GetDiffAsync(pr.BaseSha, pr.HeadSha, ct); // the change, nothing else
    var ctx = new ReviewContext { Diff = diff };
    foreach (var file in diff.ChangedFiles)
    {
        ctx.AddCallSites(await symbols.FindReferencesAsync(file.TouchedSymbols, ct)); // bugs hide at call sites
        ctx.AddConventions(conventions.ForPath(file.Path));                            // your rules
    }
    return ctx; // diff + call sites + conventions — never a naked file
}

Most false positives are the model not knowing the rules of your codebase. Feed it the conventions and the call sites and it stops flagging your patterns and starts catching the bug two callers away.

2. Multi-dimensional reviewers, not one mega-prompt

Specialized reviewers — correctness, security, performance, tests — each with a narrow remit, run in parallel and return typed, structured findings:

public sealed record ReviewFinding(
    string Dimension,      // "correctness" | "security" | "performance" | "tests"
    string File, int Line,
    Severity Severity,     // Blocker | High | Medium | Low | Nit
    string Summary,        // one sentence
    string Rationale,      // why it's a defect, grounded in the diff
    string? SuggestedFix);

A "security reviewer" told to hunt injection and secret leakage outperforms a generalist told to "find problems," and its output is a typed record you can gate on — not a paragraph you have to parse.

3. Adversarial verification — the feature that earns trust

Before any finding is posted, a separate model is prompted to refute it. Default to "not real" when uncertain.

public async Task<bool> IsRealAsync(ReviewFinding f, ReviewContext ctx, CancellationToken ct)
{
    var verdict = await gateway.EvaluateAsync(new EvalRequest
    {
        Feature = "code-review-verify",
        Prompt =
            $"A reviewer claims: \"{f.Summary}\". Using the diff and the call sites, decide " +
            "whether this is a REAL defect that would bite in production. Actively try to " +
            "refute it. If it depends on facts not present in the context, treat it as NOT real.",
        Context = ctx.ForFinding(f),
    }, ct);

    return verdict.IsReal && verdict.Confidence >= 0.90; // post only if a skeptic couldn't refute it
}

This asymmetry is the whole game. Precision matters far more than recall for an AI reviewer, because the cost of a false positive is the tool itself getting muted. A skeptical second pass is the cheapest precision you'll ever buy — it's what took us to ~6% FP and kept the bot alive.

4. Severity gating — a human on the button

The AI proposes; the human disposes. Only blocker/high findings request changes; everything else is a non-blocking comment, and a human can always override.

public MergeAdvice Gate(IReadOnlyList<ReviewFinding> findings)
{
    var blocking = findings.Where(f => f.Severity is Severity.Blocker or Severity.High).ToList();
    return blocking.Count == 0
        ? MergeAdvice.Comment(findings)                        // post comments, do not block
        : MergeAdvice.RequestChanges(blocking, findings);      // request changes; human may override
}

An AI that can unilaterally block merges will, the first time it's confidently wrong, get switched off — taking its real value with it. Advisory-by-default with human override is what makes it safe to leave on.

5. Governance — run it through the gateway

Every review call goes through the same governed AI gateway: per-repo token budgets, model routing (cheap model for style, frontier for correctness), secret redaction before code leaves the boundary, and an append-only audit. Code is one of your most sensitive assets — if your AI reviewer isn't redacting secrets, capping spend, and logging what it saw, you've traded a review bottleneck for a data-governance incident.

6. The feedback loop

Developers thumbs-up/down every comment; dimensions with poor precision get stricter verification thresholds, and conventions that keep getting mis-flagged get added to the context. That loop is why precision stays high after launch instead of drifting.

The honest stuff: when NOT to build this

Small team / low PR volume. If a human reviews everything within the hour, the overhead isn't worth it.
You haven't measured false positives. Ship a noisy bot and you train your team to ignore it permanently. Pilot, measure FP, roll out under ~10%.
You'd let the AI block merges alone. Don't. AI proposes, humans dispose.
Proprietary/regulated code that can't leave your boundary. Self-host or redact aggressively.
You think it replaces reviewers. It's an assistant — architecture and design stay human.
You're using it for style. A linter does style deterministically, instantly, and free. Aim the AI at logic and security.

The model to carry forward

An AI reviewer's job is to delete the noise so humans review what matters. The models can find issues all day; the engineering is in not crying wolf. Optimize for precision over recall, verify before you comment, and keep the human on the merge button. Get the false-positive rate low enough and the tool becomes something your team relies on; get it wrong and they'll mute it in nine days — we timed it.

Originally published on PrepStack. Rolling out AI code review and fighting the false-positive problem? Reach me at randhir.jassal[at]gmail.com.

How We Deliver 15 Million Webhooks a Day Without Losing a Single Event

kirandeepjassal-crypto — Wed, 01 Jul 2026 17:52:43 +0000

A webhook looks like the easiest feature you'll ever build: something happens, you POST it to the customer's URL. Then you ship it, and reality arrives — the endpoint is down, or slow, or returns 500, or times out, or your own process restarts mid-send. Multiply that by 15 million events a day across thousands of endpoints you don't control, and "just POST it" becomes one of the hardest reliability problems in your system.

This is the design we run on Mattrx, our multi-tenant marketing-analytics SaaS, to deliver ~15 million webhook events per day to customer-configured endpoints. The first version was a synchronous POST inside the request handler. It lost events on every deploy and turned one slow customer into an outage for everyone. This post is everything we changed, and why.

TL;DR

Aspect	Naive sync POST (before)	Outbox + queue + workers (after)
Durability	events lost on crash/deploy	persisted before delivery, never lost
API latency	blocked on the customer's endpoint	decoupled; API p95 unaffected
Retries	none	exponential backoff + jitter, 8 attempts / ~24h
Isolation	one slow customer stalls everyone	per-tenant partitioning + concurrency caps
Giving up	fails the API call	dead-letter queue + circuit breaker
Security	ad hoc	HMAC-SHA256 signed, HTTPS, timestamped
Duplicates	unhandled	stable event id; customers de-dupe

~15M events/day ≈ 175/sec average, peaks 5–10× (~1,500–1,730/sec).
Outbox pattern → zero events dropped after a committed change (we used to lose thousands per deploy).
Decoupling kept API p95 at 120 ms.
First-attempt delivery ~96%; ~99.98% eventual after retries.
~0.02% permanently fail → dead-letter queue → per-customer status + alert.

The one mental shift: you don't control the endpoints, so you cannot prevent failure — you can only make failure survivable. Persist before you deliver, retry with discipline, isolate the slow from the fast, and make giving up a first-class, observable outcome.

The naive approach — and why it collapses

The first version delivered the webhook inside the request that caused the event:

// BEFORE: fire the webhook synchronously, in the request path.
[HttpPost("campaigns/{id}/complete")]
public async Task<IActionResult> Complete(string id, CancellationToken ct)
{
    await campaigns.CompleteAsync(id, ct);

    var endpoint = await webhooks.GetEndpointAsync(TenantId, "campaign.completed", ct);
    using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(5) };
    await http.PostAsJsonAsync(endpoint.Url, new { type = "campaign.completed", id }, ct); // blocks

    return Ok();
}

If the process restarts between CompleteAsync and the POST, the event is gone forever. The API thread waits on the customer's endpoint, so slow customers exhaust the thread pool. A customer's 500 becomes your 500. And a transient blip is a permanently missed event.

Fix 1: the Outbox Pattern — persist before you deliver

Write the event into an outbox table in the same transaction as the domain change. If it commits, the event will be delivered — later, by a separate relay. If it rolls back, the event never existed.

// AFTER: the outbox row commits atomically with the state change.
public async Task CompleteCampaignAsync(string tenantId, string campaignId, CancellationToken ct)
{
    await using var tx = await db.BeginTransactionAsync(ct);

    await campaigns.MarkCompletedAsync(campaignId, ct);

    await db.Outbox.InsertAsync(new OutboxEvent
    {
        Id = Guid.NewGuid(),          // stable event id == idempotency key
        TenantId = tenantId,
        Type = "campaign.completed",
        Payload = JsonSerializer.Serialize(new { campaignId }),
        Status = OutboxStatus.Pending,
        CreatedAt = clock.UtcNow,
    }, ct);

    await tx.CommitAsync(ct);          // state change + event, all or nothing
}

A background relay claims rows with FOR UPDATE SKIP LOCKED so multiple relay instances never grab the same row, and publishes to the queue. Result: events dropped per deploy went from thousands to zero.

Fix 2: queue + dispatcher + workers — parallelism without noisy neighbours

Publish to a queue partitioned by tenantId, and run a pool of competing-consumer workers with per-tenant concurrency caps. Partitioning gives per-tenant ordering; the caps give isolation — a broken tenant can waste at most its own slots.

var slot = await limits.AcquireAsync(msg.TenantId, maxPerTenant: 20, ct); // noisy-neighbour guard
_ = DeliverAndRelease(msg, slot, ct);                                      // don't block the receive loop

Peak concurrency sits around ~1,400 in-flight deliveries — exactly what Little's law predicts (1,730/s × ~0.8s).

Fix 3: retries, exponential backoff + jitter, dead-letter queue

On a retryable failure, re-queue with a delay that grows exponentially and is jittered to avoid synchronized retry storms. After a fixed number of attempts, dead-letter it.

// Exponential backoff with FULL jitter, capped. 8 attempts span ~24h.
private static TimeSpan NextDelay(int attempt)
{
    var baseSeconds = Math.Min(BaseDelaySeconds * Math.Pow(2, attempt), MaxDelaySeconds); // cap at 6h
    var jittered = baseSeconds * (0.5 + Random.Shared.NextDouble() * 0.5);                // full jitter
    return TimeSpan.FromSeconds(jittered);
}

Not every failure is retryable: a 410 Gone or 400 Bad Request means stop — dead-letter immediately. A 503, 429, timeout, or connection reset is transient — retry. Retries lift delivery from ~96% first-attempt to ~99.98% eventual; the remaining ~0.02% land in a visible, queryable dead-letter queue.

Fixes 4–7: idempotency, HMAC signing, circuit breaker, observability

Idempotency: every event carries a stable id (X-Mattrx-Event-Id) that never changes across retries. You can't make delivery exactly-once, but you make processing effectively-once by shipping a stable id and telling customers to key on it.
Security: HMAC-SHA256 over timestamp + id + body with a per-tenant secret, HTTPS only. Sign and send the raw serialized body so a proxy reformatting JSON can't break verification.
Circuit breaker: track consecutive failures per endpoint; after 20, auto-disable and email the owner. This reclaims retry capacity otherwise burned forever by endpoints that will never succeed (~40/day for us).
Observability: every attempt writes a delivery record (event id, endpoint, attempt, status, latency, outcome) powering a per-customer dashboard. At 15M/day, aggregate "99.98% delivered" hides the one tenant at 40%.

The model to carry forward

At-least-once plus idempotency — never exactly-once. You cannot stop the endpoints from failing, so the whole design is about surviving their failure: persist before you deliver, isolate the slow from the fast, and make giving up a loud, observable, recoverable outcome instead of a silent drop.

A webhook really is just a POST. Delivering fifteen million of them a day, to endpoints you don't control, without losing one — that's a distributed system, and it deserves to be designed like one.

Originally published on PrepStack. If you're designing an event/webhook delivery system and want a second pair of eyes on the failure paths, reach me at randhir.jassal@gmail.com.

Stop Chunking Documents: The Open Knowledge Format (OKF) for Enterprise AI

kirandeepjassal-crypto — Tue, 30 Jun 2026 18:37:28 +0000

Originally published on PrepStack.

Everyone's first RAG pipeline is the same four boxes: documents, chunk, vector DB, LLM. It demos in an afternoon and then quietly betrays you in production — stale answers, no relationships, no governance, and a model guessing from fragments. The fix is not a bigger vector index. It is to stop storing documents and start storing knowledge. That is Open Knowledge Format (OKF).

To be clear up front, because the title is deliberately provocative: OKF does not kill embeddings. Vectors still do the recall. What OKF kills is blind chunking — slicing opaque documents into context-free fragments and hoping cosine similarity reassembles meaning. On Mattrx, a multi-tenant marketing-analytics SaaS (.NET 9 + Azure SQL + a Python FastAPI AI service), replacing blind chunking with OKF + a Context Engine took the assistant's hallucination rate from 18% to 3% and stale-answer rate from 11% to 1.5%.

TL;DR

Dimension	Documents → chunk → vector DB (before)	OKF + Context Engine (after)
Unit of knowledge	Opaque chunk of text	Typed, governed knowledge unit
Structure	None — chunks are islands	Metadata + relationships + schemas
Freshness	Snapshot, rots silently	`valid_until` + live API refs
Rules	Buried in prose, ignorable	First-class data the engine enforces
Retrieval	Top-k cosine	Hybrid + vector + graph
Multi-hop questions	Unanswerable	Answered via relationships

Results after the rebuild:

Knowledge base restructured into ~11,000 OKF units (Markdown + metadata + relationships + APIs + schemas + business rules).
Hallucination 18% -> 3%; faithfulness 0.96; answer-relevance 0.91.
Context tokens/call 14k -> 3.5k — structure lets the engine attach the right thing, not everything.
Outdated-answer rate 11% -> 1.5% (valid_until + metadata freshness).
Multi-hop questions unanswerable -> answered via graph retrieval.
Deprecated-plan recommendations recurring -> 0 (business rules enforced as data).

The one mental shift: a chunk is a fragment of text with no identity, no owner, and no expiry. An OKF unit is a governed, typed, related piece of knowledge your context engine can reason about. Stop indexing text. Start indexing knowledge.

Part 1 — The OKF Knowledge Base

1. From documents to knowledge units. A chunk has no identity — you can't say who owns it, when it expires, or what it relates to. The OKF atomic unit is a Markdown body plus structured frontmatter: id, type, owner, version, valid_from / valid_until, visibility, plus relationships, apis, schemas, and business rules. The body is still embedded; the metadata is what the engine filters and ranks on. The moment a unit has valid_until, the engine can refuse to ground an answer in expired knowledge — which dropped outdated answers 11% -> 1.5%.

2. Relationships and schemas: the knowledge graph. Chunks are islands; vector similarity finds text that sounds alike, not text that's connected. OKF makes edges first-class (relates_to, supersedes, governed_by, depends_on) and defines schemas for structured units — together a knowledge graph over the corpus. Graph retrieval starts from semantic seeds and expands along those edges, which is how "if a Growth customer downgrades mid-cycle, how is it prorated?" (an answer spanning three documents) became answerable. Vectors find the door; the graph walks the building.

3. APIs and business rules: live data and governance as data. A snapshot like "Our plans are Starter, Growth, Scale…" is wrong the day the catalog changes, and a rule written in a paragraph is a suggestion the model can ignore. OKF units reference APIs (resolved at query time, through a governed tool layer) so answers reflect the current catalog, and link to business-rule units (enforcement: hard) that the engine injects as constraints. "Recommended a deprecated plan" went from a recurring complaint to 0.

Part 2 — The Context Engine

OKF is how knowledge is stored; the Context Engine is how it becomes a prompt.

4. Retrieval: hybrid + vector + graph. Top-k cosine is one signal — it misses exact-term matches (a campaign id, an error code) and related-but-not-similar units. The engine runs hybrid search (BM25 + vector) for recall, vector retrieval for nuance, and graph retrieval for connected units — all tenant-scoped and filtered to currently-valid units (onlyValid: true, metadata doing security and quality work for free). This combination is the core of the 18% -> 3% hallucination drop and 0.96 faithfulness.

5. Memory, tool calls, prompt assembly. Memory supplies what's already known; tool calls fetch the live data OKF units reference; prompt assembly packs everything to a hard token budget in priority order — constraints first (never trimmed), then memory, then ranked knowledge, then live data. Ordering is a design decision: a budget squeeze can never silently drop the rule that keeps an answer compliant. This is how context holds at 3.5k tokens (down from 14k) while accuracy improves.

When NOT to adopt OKF

It's an investment in structure — pay it where structure exists and matters. Skip it for a small/static corpus, genuinely unstructured knowledge, when no one can own authoring (rotten metadata is worse than none), or before you've nailed plain hybrid RAG + a cross-encoder rerank (OKF is the next layer, not the first). And don't big-bang it — convert high-value, high-churn domains first (we did billing, then product, then runbooks) and leave the long tail as plain chunks.

And the honest framing of the title: OKF does not replace embeddings. Vector retrieval is still inside the Context Engine doing recall. OKF replaces blind chunking and adds the structure, governance, and graph embeddings alone cannot provide. If someone sells you "vectors are dead," walk away.

The model to carry forward

Documents are what you have; knowledge is what the model needs. Give every unit an identity, an owner, and an expiry; model relationships explicitly (the graph answers the multi-hop questions vectors structurally can't); encode rules as data, not prose.

👉 The full article — the complete OKF format, the C# records and graph/retrieval/assembly code, the end-to-end query walkthrough, the migration checklist, and the full "when not to do this" — is on PrepStack:
Stop Chunking Documents: The Open Knowledge Format (OKF)

AI-Native Architecture: The 9-Layer Blueprint Every Enterprise Will Adopt by 2027

kirandeepjassal-crypto — Mon, 29 Jun 2026 13:58:21 +0000

Originally published on PrepStack.

Every enterprise has now shipped "an AI feature." Almost none have shipped an AI-native architecture. The difference decides whether your AI roadmap survives 2027 — or quietly gets ripped out after the third incident.

We learned this the expensive way on Mattrx, a multi-tenant marketing-analytics SaaS (Angular 19 + .NET 9 / ASP.NET Core + Azure SQL, with a Python FastAPI AI service; 110k MAU, ~3,200 req/sec peak). The first "Mattrx Help" was a single MVC action calling a frontier model inline. It demoed beautifully. In production it leaked context across tenants, hallucinated 18% of the time, cost $0.021/query, and one tenant's runaway retry loop billed the entire fleet for an afternoon. The fix was not a better prompt — it was an architecture.

TL;DR

Dimension	Bolt-on AI (before)	AI-native (after)
Model access	SDK called inline in controllers	Single governed AI gateway
Tenant isolation	"Please don't leak" in the prompt	Filters pushed into the data layer
Context size	~14,000 tokens, stuff everything	~3,500 tokens, assembled + ranked
Memory	Stateless, cold every turn	Short-term buffer + semantic long-term
Retrieval	Naive top-k cosine	Hybrid recall + cross-encoder rerank
Reasoning	One mega-prompt	Orchestrator + specialists + eval gate
Model choice	frontier everywhere	Routed by task complexity, with fallback
Actions	Agent had raw DB access	Typed, authorized tool contracts

Production results after the rebuild:

Hallucination rate 18% -> 3% (hybrid retrieval + rerank).
Faithfulness 0.96, answer-relevance 0.91 (offline eval set).
Context tokens/call 14k -> 3.5k — same answers, a quarter of the spend.
Cost/query $0.021 -> $0.008 (mostly model routing).
Agentic p95 latency 4.2s -> 1.8s.
~40 prompt-injection attempts blocked/week at gateway + identity.
Eval gate threshold 0.90 — answers below it never reach a user.
Zero cross-tenant leaks in the six months since.
Help now deflects ~520 support tickets/month.

The one mental shift: stop treating the model as a feature you call, and start treating it as a tier you operate — with its own gateway, identity, memory, and governance, exactly like your database.

The nine layers (each added after a real failure)

1 → 2. The API gateway becomes an AI gateway. The first version put the model call inside a controller — no per-tenant rate limit, no token accounting, no PII redaction, no audit. Now every model call (Help, Insights, internal classification) passes through one gateway that reserves a per-tenant token budget, redacts PII before anything leaves the boundary, routes to a model, and writes an audit row. Runaway loops became a 429 for one tenant instead of a fleet-wide bill — and ~40 injection attempts/week get blocked here.

3. Identity stops meaning "who logged in." "Tell the model not to leak" is not a security control. Identity became a first-class AiPrincipal (tenant, user, scopes) that propagates into retrieval and every tool call, with the tenant filter pushed down into the vector store — data-layer enforcement, not a polite request. Zero cross-tenant leaks since.

4. The context layer (the one most teams skip). We used to concatenate everything and hope the window was big enough (~14k tokens). But bigger context isn't better — past a point recall drops and the model misses the middle. A context assembler now treats the prompt like a budget to pack: select, rank, compress to a hard token ceiling. Result: 14k -> 3.5k tokens, faithfulness 0.96.

5. Memory. Two tiers: a short-term conversation buffer (TTL-bounded) and a long-term semantic memory that persists only salient turns (decisions, preferences, corrections). It stops re-interrogating returning users — a big part of deflecting ~520 tickets/month.

6. The knowledge base — RAG done properly. Top-k cosine returns chunks that look relevant and are often subtly wrong — the worst failure mode because it sounds confident. We moved retrieval to Python: hybrid recall (BM25 + vector), then a cross-encoder rerank (the step most RAG skips), with the rule that returning nothing beats returning something wrong. This single change drove hallucination 18% -> 3%.

7. Agents. One mega-prompt that planned, queried, forecasted, and wrote up in a single shot drifted as a whole when any step drifted. Now an orchestrator plans, dispatches to specialist agents (sql / forecast / summarize), and nothing reaches the user until a 0.90 eval gate scores it. Agentic p95 4.2s -> 1.8s.

8. The LLM layer is a router, not a model. We used the frontier model for everything — including "is this about billing or analytics?" Instrumented, ~70% of calls were trivial. A router now picks the cheapest model that can do the job, with fallback. Cost/query $0.021 -> $0.008.

9. Business APIs become governed tools. The agent used to hold a database connection — functionally an unaudited admin user. Now every action is a typed, scope-checked tool: the model proposes, the tool layer decides, and the tenant is bound in code, never from model-supplied arguments. A model can hallucinate an action; it cannot hallucinate a scope check.

When NOT to build this

It earns its complexity at scale and under real risk — and is overkill otherwise: a single AI feature with no multi-tenancy, low traffic, no regulated/cross-tenant data, no v1 shipped yet, or no labeled eval set (the gate is theater without one). Smaller teams should adopt the gateway and the rerank first and stop there. We didn't build all nine at once either — each layer was a response to a specific failure, which is exactly how you should adopt it.

The model to carry forward

Treat AI as a tier, not a feature. Your database has a gateway, identity, isolation, and an audit trail because direct access doesn't stay safe at scale. The model is no different. Three habits: measure before you add a layer (no bad number, no layer); make governance structural, not textual (if a guarantee lives in a prompt, it isn't one); default to the cheapest path that passes eval.

👉 The full article — all nine layers with the complete C# (.NET 9) and Python code, the end-to-end request flow, the adoption checklist, and the full "when not to do this" — is on PrepStack:
AI-Native Architecture: The 9-Layer Blueprint

Context Engineering for Enterprise AI, Part 6: AI & Data Governance — The Foundation Everything Grows From

kirandeepjassal-crypto — Sat, 27 Jun 2026 08:57:14 +0000

Originally published on PrepStack. This is **Part 6* (the finale) of* Context Engineering for Enterprise AI.

Parts 1–5 built capability for Mattrx, a multi-tenant marketing-analytics SaaS (110k MAU, ~9,000 tenants, ~3,200 req/sec peak, ASP.NET Core / .NET 9 on Azure SQL plus a Python FastAPI AI service): a budgeted context window, a memory layer, multi-agent orchestration, an enterprise design spine, and multi-tenant isolation. This part is the layer underneath all of them — the control plane that decides what data is allowed to enter or leave the context at all, for whom, and for what purpose. Capability without governance is a breach waiting for a date.

TL;DR

Governance is a control plane, not a checklist. Five controls decide what may enter or leave the context, each enforced before the model sees data, never after a breach:

Control	The question it answers	Where it's enforced
Classification	How sensitive is this data?	A classify gate at ingestion; class + purpose tags to the Data Catalog
Entitlement-aware retrieval	May this user* (not just this tenant) see it?*	Principal ACL/group filter stacked on the Part 5 tenant filter
Consent & purpose	Are we allowed* to use it for AI at all?*	Purpose tags + a consent registry checked at use time
Lineage & provenance	What exactly did the model see, and why allowed?	An append-only record per generation
Policy-as-code (PDP)	Who decides, and can we prove the rule?	One deny-by-default Policy Decision Point every path calls

Mattrx production results (3-week build):

Confidential/Restricted documents reaching the shared embedding store ungoverned: ~3,100 -> 0.
Intra-tenant cross-principal leaks (a support agent retrieving a finance doc inside their own tenant), red-team: reproducible -> 0.
"What did the model see about subject X, and why allowed?": 0% -> 100% of generations; DSAR fulfillment ~2 days -> under 3 minutes (one query).
Customer data used for eval/fine-tuning without a consenting purpose: unbounded -> 0.
Governance enforcement: ~14 scattered sites -> 1 PDP + N versioned policies; a policy change ships without touching service code.
Governance overhead on the hot path: +4 ms p95 (PDP decision ~3 ms cached, deny-by-default). Retrieval p95 31 ms -> 35 ms, recall@5 held 0.94; cost/query $0.008 (unchanged).

The one mental shift

Governance is a control plane that runs before retrieval, not a report you generate after an incident. Make every data access a question — can(principal, data, purpose, context)? — answered deny-by-default by one engine, and recorded — so the answer to "could this leak?" is a query, not a prayer.

1. Classification at ingestion: you cannot govern what you never labeled

Before: everything a tenant connected was embedded the moment it arrived — no sensitivity label, no purpose. A "knowledge base" sync could vectorize a spreadsheet of customer emails and a file of API tokens, now one cosine hop from any prompt.

After: a classify gate runs at ingestion. Every asset gets a sensitivity class (Public / Internal / Confidential / Restricted) and purpose tags, is registered in a Data Catalog, and then routed — Public/Internal to the pool index, Confidential to a restricted index, and Restricted is catalogued and quarantined, never embedded at all. Cheap deterministic detectors (regex/Presidio for secrets and PII) run first; an LLM classifier handles only the ambiguous remainder. Classifying at the front door means a missing filter downstream can't expose what was never indexed. The gate moved ~3,100 sensitive docs out of the shared store (0.4% over-quarantine rate, human-reviewed).

2. Entitlement-aware retrieval: authorize the principal, not just the tenant

Part 5 made retrieval tenant-scoped. But inside a tenant, retrieval returned anything that tenant owned, to anyone in it — a support agent could pull the revenue runbook. The fix stacks a principal predicate (the user's groups/clearance) on top of the tenant predicate; both apply at once. Mattrx runs a hybrid: a fast ACL filter in the index, then a live revalidation of the surviving top-k against the entitlements service, so a just-revoked permission can't leak through a stale index. Intra-tenant cross-role leaks went reproducible -> 0, recall@5 held at 0.94, +4 ms p95.

3. Consent and purpose limitation: allowed to have it isn't allowed to use it

If Mattrx stored data, every feature treated it as fair game — retrieval, agent analysis, eval, fine-tuning. But "we hold this to run the customer's campaigns" is not consent to "use it to improve our product." Every asset carries purpose tags; a consent registry records what each tenant agreed to; each AI use names its purpose (serve / eval / train) and the PDP refuses data whose purposes don't include it. Opt-out tenants are structurally excluded from eval/train while still fully served their own features. Customer data used without a consenting purpose went -> 0.

4. Lineage and provenance: prove exactly what the model saw

A DSAR ("what does your AI know about me, and where did it come from?") used to be a multi-day dig through stateless logs. Now every generation writes one append-only lineage record: output hash, the exact source assets (id, version, class) that entered the prompt, the principal, the declared purpose, and the PDP decisions that allowed each source. It stores references and versions, not raw content (so the audit trail isn't a second copy of the sensitive data), the table grants no UPDATE/DELETE, and the write is off the hot path. Provenance coverage 0% -> 100%; DSAR ~2 days -> under 3 minutes.

5. Policy-as-code: one decision point everything asks

Each rule above — and in Parts 2/4/5 — lived as its own if statements in its own service: ~14 enforcement sites, four dialects of "allowed," nothing to read, test, or change centrally. Drift was inevitable. The fix: a single Policy Decision Point answering can(principal, data, purpose, context)? deny-by-default, with rules as versioned, unit-tested policy-as-code (Mattrx runs OPA/Rego as a sidecar). Ingest, retrieval, output, and agent tools all ask the same PDP; every decision is cached (~3 ms) and recorded into lineage. The PDP fails closed — an outage blocks access rather than risking an open one, the correct failure mode for governance (and a real availability coupling you design for).

Honest stuff

Governance can become theater — a catalog nobody trusts and policies nobody tests manufacture false confidence. The PDP earns trust only because its policies are unit-tested in CI and its decisions are recorded.
Classification is probabilistic — make the failure bounded and reviewable (every asset has a catalog row), and put humans on the quarantine queue.
The PDP is an availability coupling — deny-by-default makes it a tier-1 dependency (sidecar, health checks, cached decisions).
Don't classify what you can delete — minimization beats governance. The cheapest data to govern is the data you never kept.

The closing mental model

Govern at the front door (classify and decide before data is embedded). Ask one engine, deny by default (every path asks the same PDP; if a new path doesn't ask, it's a hole). Record so you can prove it (append-only lineage turns "trust us" into "here's the query"). If you can't reconstruct what the model saw and why it was allowed, you don't have governance — you have hope.

👉 The full article — with all the C# (.NET 9), Python, OPA/Rego, and SQL code, the control-plane diagram, the pre-ship checklist, and the full "honest stuff" — is on PrepStack:
Context Engineering for Enterprise AI, Part 6

Context Engineering for Enterprise AI, Part 5: Multi-Tenant Patterns That Don't Leak, Starve, or Overspend

kirandeepjassal-crypto — Thu, 25 Jun 2026 14:47:19 +0000

Originally published on PrepStack. This is **Part 5* of* Context Engineering for Enterprise AI.

Parts 1–4 built the context layer for Mattrx, a multi-tenant marketing-analytics SaaS (110k MAU, ~9,000 tenants, ~3,200 req/sec peak, ASP.NET Core / .NET 9 + a separate Python FastAPI AI service). Every one of those parts quietly leaned on one primitive — the tenant boundary. This part makes it the design surface instead of an afterthought.

TL;DR

Multi-tenancy is not a WHERE tenant_id = ? you sprinkle on queries. It is a single boundary that has to hold across every surface of the context pipeline — identity, retrieval, memory, prompt, cache, rate limit, model routing, cost, and residency — each with its own failure mode.

Mattrx results after making tenant scope a first-class context primitive (2-week build):

Cross-tenant leak incidents (docs + cache + prompt + logs, load + red-team): 0.
Noisy-neighbor: other tenants' p95 during the whale's nightly batch: 6.4s -> 1.9s.
Prompt-cache hit on the shared system preamble: 0% -> 71% (~50% cheaper prefill).
Per-tenant cost attribution: 0% (one blended bill) -> 100% (per-call ledger).
Runaway-tenant spend in one hour: ~$140 -> $5 (budget cap trips; clean 429).
New-tenant isolation onboarding: ~2 days manual -> < 5 min automated.
Retrieval p95 on the pool index with a tenant filter at ~9,000 tenants: 31 ms (held); whale on a dedicated index: 22 ms, and pool recall@5 recovered 0.88 -> 0.94.
Cost per AI query: $0.008 (unchanged) — now attributed, capped, and routed per plan.

The one mental shift

Stop treating the tenant as a filter you remember to add. Treat tenant scope as part of the context itself — resolved once at the edge, carried as an unforgeable token through retrieval, prompt, cache, model, and ledger. If isolation depends on anyone remembering to add a filter, it will leak the day someone forgets.

1. Tenant identity: resolve once, never trust the body

The first multi-tenant version read tenant_id from wherever was convenient — a query string, the JSON body — and passed it to the AI service. Any authenticated user could read another tenant by changing one field.

The fix: resolve the tenant once, at the edge, from the signed workspace_id claim in the JWT. Everything downstream receives a TenantScope it cannot forge or widen, and every method signature gains a required TenantScope — so the type system refuses to compile a query that forgot the tenant. Moving identity from the body to the token closed the entire "change one field, read another tenant" class: 0 successful cross-tenant reads in red-team testing.

2. Knowledge isolation: pool by default, silo on a threshold

Three models, and why the hybrid wins:

Model	What it is	Isolation	Best for
Pool	One index, every doc tagged `tenant_id`, hard filter on read	Logical (a missing filter = leak)	The long tail of small tenants
Silo	One index per tenant	Physical (nothing to forget)	Whales, Enterprise, EU-residency
Bridge	Pool by default; promote to silo on a threshold	Logical for most, physical for the few	Mattrx's choice

Pure silo is impossible at 9,000 tenants (Azure AI Search caps indexes per service). Pure pool can't give residency and lets one tenant's corpus degrade everyone's recall. Bridge keeps the cheap, instant pool for the 99% and spends physical isolation only where size, SLA, or regulation forces it. Crucially, the tenant_id filter is applied by the router, never by the call site — so no caller can forget it. Promoting the whale to its own index dropped its p95 to 22 ms and recovered the long tail's recall@5 from 0.88 to 0.94.

3. Prompt and policy variation: behavior as data, not branches

Per-tenant behavior used to accrete as if (tenantId == ...) branches, and large tenants got bespoke prompts rebuilt every call (a DateTime.UtcNow in the system block alone meant a 0% cache hit). The fix: tenant behavior is data (a TenantConfig row), and the prompt is assembled as a byte-stable shared preamble first (identical bytes for every tenant, so prompt caching reuses it) followed by a small tenant delta. Result: system-block cache hit 0% -> 71%, and every tenant inherits the same injection/redaction rules — no per-tenant safety drift.

4. Cache, quota, and routing: stop one tenant starving (or impersonating) another

Two shared resources had no tenant in them. The answer cache was keyed only by the question hash — so A's cached answer was served to B. And one global rate limiter let the whale's nightly batch consume the whole budget.

Fixes: the cache key carries residency + tenant (residency:tenant:hash), so answers can't cross the boundary; rate limiting is partitioned by tenant with per-plan token buckets (QueueLimit = 0 → a clean 429 with Retry-After, not an unbounded queue); model routing respects plan and remaining budget (over budget downgrades the model, never 500s). Partitioned fairness held small tenants' p95 at 1.9s during the whale's batch, down from 6.4s.

5. Cost attribution, residency, and the RLS backstop

Every model call books cost to a per-tenant ledger, which feeds dashboards, the budget cap from Section 4, and a usage-based billing export. Residency is enforced by region-pinned indexes and Azure SQL Row-Level Security, so even a buggy query that forgets the tenant filter returns nothing instead of leaking. The ledger took attribution 0% -> 100% and capped a runaway tenant's hour at $5 instead of ~$140; RLS + region pinning kept EU data in-region with 0 cross-region reads in audit.

The subtle part: RLS can mask a missing app filter by silently returning no rows. So Mattrx alerts when RLS filters a row the app should have scoped — the backstop catching something is a signal, not a success.

The closing mental model

Multi-tenancy is one boundary, enforced in many places, resolved once and never re-derived. Three habits:

Resolve at the edge, carry as a token. Derive TenantScope from auth once; make every layer take it as a required input; let the type system reject any tenant-less call.
Default to pool, promote on a threshold. Keep the cheap shared path for the 99%; spend physical isolation only where size, SLA, or regulation demands it — and automate the promotion.
Enforce isolation twice. App filter for speed, RLS for the day the filter is missing. If a single forgotten WHERE can leak, you have not isolated anything — you've documented an intention.

👉 The full article — with all the C# (.NET 9) and Python code, the end-to-end boundary diagram, the pool/silo/bridge comparison, the pre-ship checklist, and the "honest stuff" caveats — is on PrepStack:
Context Engineering for Enterprise AI, Part 5

Context Engineering for Enterprise AI, Part 4: Enterprise AI Design — Governance, Cost & Safety

kirandeepjassal-crypto — Tue, 23 Jun 2026 15:10:19 +0000

Originally published on PrepStack. This is **Part 4 of 6* of* Context Engineering for Enterprise AI.

Parts 1–3 gave us a context pipeline, a memory layer, and a multi-agent architecture. All real, all measurable — and all still a demo until you wrap them in what this part covers: governance, security, evaluation, observability, cost control, and reliability. That is the enterprise design that lets you ship AI to 110k paying users without losing sleep, money, or a compliance audit.

TL;DR

A context pipeline without governance is a liability, not a feature. The hard part of enterprise AI is not the model — it's the boundary around it.

Production metrics after the full enterprise design is in place:

Wrong-answer / hallucination rate: 18% (naive RAG) → 3%.
Faithfulness (groundedness) eval score: 0.96; answer-relevance: 0.91.
Eval gate threshold: any change dropping faithfulness below 0.90 is blocked in CI.
Prompt-injection attempts blocked at the boundary: ~40/week.
Cost per AI query: $0.021 → $0.008 (caching + model routing + context compression).
Context tokens per request: ~14,000 → ~3,500.
Agentic query p95: 4.2s → 1.8s.
The C# app API p95 stays 120 ms — the AI work never bled into the product API.
Every AI response carries a trace id + an immutable audit row (prompt hash, tokens, cost, citations).

The one mental shift

Stop treating the model as the system. The model is one untrusted, non-deterministic dependency inside a system you do govern. Everything around it — eval gates, the security boundary, cost routing, tracing, audit — is the part you actually own, test, and are accountable for. Engineer that, and the model becomes swappable.

Evaluation gates: stop shipping prompts on vibes

A prompt change is a code change with a non-deterministic compiler. You'd never merge a refactor without tests; don't merge a system-prompt edit without an eval.

A golden set of ~200 curated (question, ideal-answer, must-cite-source) tuples lives in version control. Every prompt or model change runs the offline harness in CI, scoring faithfulness and answer-relevance. A change that drops faithfulness below 0.90 fails the build. We sit at 0.96 faithfulness, 0.91 relevance.

Offline catches regressions; online catches drift. We sample ~2% of live traffic and run the same groundedness judge asynchronously (never on the hot path), alerting if rolling faithfulness dips.

Security: the boundary that says no

Every AI request passes through AiGovernanceMiddleware before it can reach the Python service. It enforces RBAC, stamps the authenticated tenant_id (never trusting a client-supplied one), redacts PII, and runs an injection classifier. Only a sanitized, scoped request crosses the HTTP boundary.

The injection classifier is cheap on the Python side — a small, fast model plus a deny-pattern check, kept off the expensive model entirely. PII redaction happens in C# at both ingress (before the model sees it) and egress (before we log or store the answer).

Result: the boundary blocks ~40 prompt-injection attempts per week, and zero cross-tenant retrievals have occurred since tenant_id enforcement moved from "in the query" to "in the token."

Cost and reliability: budgets, routing, and graceful failure

At 3,200 req/sec, a 2-cent query versus a 0.8-cent query is a $30k/month argument. And the Python service will go down; the only question is whether the user sees a 500 or a graceful degrade.

The C# client wraps the call in a Polly resilience pipeline (timeout + circuit breaker + fallback), and a budget gate refuses queries from a tenant that has blown its monthly AI spend. The Python service routes cheap tasks to a small model. Combined with caching and context compression, cost per query dropped from $0.021 to $0.008.

Observability and audit: trace every prompt, token, and citation

OpenTelemetry spans flow from the C# request through the HTTP boundary into the Python service and back, carrying the same trace id. Every AI response writes an immutable audit row: prompt hash, model, token counts, cost, and the exact citations. Mean time to answer "what did the AI cite for this response" went from "we can't" to under 30 seconds.

The closing mental model

The model is the cheapest, most replaceable part of an enterprise AI system. The eval gate, the security boundary, the cost router, and the audit trail are the product — and they're the parts you can actually be held accountable for.

No prompt or model change merges without passing the eval gate.
The boundary is the only door. Every AI request goes through governance or it doesn't go at all.
If you can't trace it and audit it, it didn't happen.

👉 The full article — with all the C# (.NET 9) and Python code, the architecture diagram, the pre-ship checklist, and the "honest stuff" caveats — is on PrepStack:
Context Engineering for Enterprise AI, Part 4

Context Engineering for Enterprise AI, Part 3: Multi-Agent Architecture That Survives Production

kirandeepjassal-crypto — Sun, 21 Jun 2026 17:03:03 +0000

Originally published on PrepStack.

Most "AI agents" in production are one giant agent with every tool and a 10,000-token prompt. It loops, stalls, and ships confident nonsense. This is Part 3 of my Context Engineering series.

The reframe

A single agent with every tool isn't "one smart system" — it's a state machine with no states, no guards, and no exits. You don't fix that with a better prompt. You fix it with structure: the model decides; the graph governs.

The architecture

ASP.NET Core owns orchestration, budgets, and governance; a Python LangGraph service runs the agent graph:

Supervisor → specialized workers — a cheap supervisor (gpt-4o-mini) routes to a retriever, analyst, writer, and critic
Model routing — only the analyst uses the expensive model; the rest run on mini
Typed hand-offs — workers pass structured data (Pydantic + C# records), not prose
Bounded loops + a critic gate — a hard step budget, and it can't finish until a critic verifies the answer is grounded
Parallelism with failure isolation — concurrent retrieval with per-fetch timeouts; one slow tool degrades to best-effort instead of killing the run
The C# boundary owns the global wall-clock budget, cost cap, and prompt-injection screen

The results

Metric	Before	After
p95 latency	4.2s	1.8s
Cost per query	$0.021	$0.008
Context tokens / agentic request	~12,000	~3,800
Runaway loops (>12 calls)	6%	0%
Endpoint 500 rate	1.4%	0.2%

Three habits: no worker without a budget, no hand-off without a type, no done without a verdict.

Read the full breakdown — with all the C# and Python code — on PrepStack:
https://prepstack.co.in/blog/context-engineering-enterprise-genai-part-3-multi-agent-architecture

Context Engineering for Enterprise AI, Part 2: The Memory Layer That Makes Agents Useful

kirandeepjassal-crypto — Sat, 20 Jun 2026 19:19:26 +0000

published on PrepStack.*

Your AI agent forgets everything the moment a request ends. That's not a model limitation — it's a missing memory layer. This is Part 2 of my Context Engineering series.

The reframe

The model is a stateless function. Memory is an enterprise database with embeddings bolted on — owned, scoped, audited, and forgettable. Not a chat transcript you keep pasting back into the prompt.

The architecture

Built across ASP.NET Core (system-of-record + governance on Azure SQL) and a Python FastAPI service (embeddings + semantic recall on Azure AI Search):

Tiered memory — working → short-term (Redis + SQL) → long-term episodic + semantic (vector index)
A salience-gated write policy — store what matters, not every turn (long-term writes cut ~85%)
Retrieval blends similarity + recency, packed into the Part 1 token budget
Tenant + user as hard query filters — never prompt instructions
Right-to-be-forgotten that fans out to SQL + vectors + cache in under 5 minutes (GDPR Art. 17)

The results

Metric	Outcome
Context tokens at turn 30	~3,500 (vs ~14,000 history-stuffing)
Cost per query	$0.021 → $0.008
Cross-tenant leaks (red-team)	0
Memory retrieval p95	74 ms
Cross-session continuity	resumes the prior thread, no re-explaining

The model is stateless. Memory is infrastructure you own — scoped, audited, and forgettable.

Read the full breakdown — with all the C# and Python code — on PrepStack:
https://prepstack.co.in/blog/context-engineering-enterprise-genai-part-2-memory-layer

Context Engineering for Enterprise AI: Cutting RAG Hallucination from 18% to 3% (C# + Python)

kirandeepjassal-crypto — Wed, 17 Jun 2026 19:18:42 +0000

Originally published on PrepStack.

We took an enterprise RAG assistant from an 18% wrong-answer rate to 3% — without changing the model. The lever wasn't the prompt. It was the context we assembled and fed the model.

The mental shift

The model isn't your product; the context you assemble is. Prompt engineering tweaks the wording. Context engineering controls what data enters the window. Treat the context window like a CPU cache — a scarce, governed resource — not a junk drawer.

The pipeline

Naive top-k RAG dumped 8 fuzzy chunks into a ~14,000-token prompt and hoped. We replaced it with a real pipeline, split across ASP.NET Core (orchestration) and a Python FastAPI service (retrieval + ranking):

Rewrite the vague user question into a self-contained query
Hybrid retrieval — BM25 keyword + vector, not vector-only
Cross-encoder re-rank a wide candidate pool down to the best 6
Budget the window (~3,500 tokens, every byte allocated)
Compress chunks to only the sentences that matter
Ground + cite every claim — or refuse and route to a human

The results

Metric	Before	After
Hallucination rate	18%	3%
Context tokens/request	~14,000	~3,500
Cost per query	$0.021	$0.008
Retrieval recall@5	0.71	0.94

The context window is a budget you spend on relevance, not a bucket you fill with hope.

Read the full breakdown — with all the C# and Python code — on PrepStack:
https://prepstack.co.in/blog/context-engineering-enterprise-genai-part-1-context-management

We Replaced REST with Kafka and Cut API Failures 90% — .NET 9 Event-Driven Architecture

kirandeepjassal-crypto — Sun, 14 Jun 2026 13:36:35 +0000

TL;DR — We swapped Mattrx's inter-service REST calls for Kafka topics (multi-tenant marketing analytics SaaS, .NET 9 / ASP.NET Core, Azure SQL, ~3,200 req/sec peak). Over 8 weeks: end-to-end ingestion failures 1.9% → 0.18% (−90%), ingestion p95 180 ms → 8 ms (−96%), events lost on downstream outage tens of thousands → 0, cascading-failure incidents ~3/month → 0, time to add a new consumer ~1 sprint → ~1 day.

🎥 3-minute video walkthrough: https://youtu.be/O21rbuQdM1Y
👉 Full deep-dive (architecture, code, pre-adoption checklist, when NOT to reach for Kafka): https://prepstack.co.in/blog/replaced-rest-with-kafka-cut-failures-90-percent

The one mental shift

The reflex in a .NET shop is: service A needs something to happen in service B, so A calls B's REST endpoint and waits. That's correct for a query ("give me this tenant's plan") — A genuinely needs the answer. It's wrong for an event ("this campaign event occurred") — A doesn't need anything back, yet it's now blocked on B, C, and D being healthy.

Distinguish commands/queries (need an answer → REST) from events (fire-and-forget → log). A synchronous call makes the caller's uptime a function of the callee's uptime. An event published to a durable log decouples them in time — the producer succeeds the instant the event is durably written, consumers process whenever they can.

Once you stop treating "something happened" as a function call and start treating it as a fact appended to a log, cascading failures, retry storms, and burst overload mostly disappear — because nothing downstream is on the request's critical path anymore.

Before — synchronous REST chain (the chain that breaks)

customer site ──► Collector ──► Enrichment ──► Analytics ──► Persister
                  (waits)       (waits)       (waits)      (waits)

// BEFORE — the collector waits on the whole chain. Any failure loses the event.
public async Task<IResult> Collect(CampaignEvent ev, CancellationToken ct)
{
    var enriched = await _enrichClient.EnrichAsync(ev, ct);  // can fail
    await _analyticsClient.RollupAsync(enriched, ct);        // can fail
    await _persistClient.SaveAsync(enriched, ct);            // can fail
    return Results.Ok();                                      // only if ALL three succeed
}

Availability multiplies. Four 99.9%s in series = ~99.6% uptime. Analytics slow at month-end → Collector times out → customer's event is gone → customer retries → MORE load on the struggling service → retry storm.

After — Kafka log decouples the pipeline

customer site ──► Collector ──► [Kafka: events.raw]
                  returns 202        │
                  in ~8ms            ├──► [analytics group]   (independent)
                                     ├──► [persister group]   (independent)
                                     └──► [enrichment group]  (independent)

// AFTER — produce ONE event to Kafka and return. No downstream on the critical path.
public async Task<IResult> Collect(CampaignEvent ev, CancellationToken ct)
{
    var msg = new Message<string, byte[]> {
        Key   = ev.TenantId.ToString(),                  // per-tenant ordering
        Value = JsonSerializer.SerializeToUtf8Bytes(ev),
    };
    await producer.ProduceAsync("events.raw", msg, ct);  // durable, ~ms
    return Results.Accepted();                            // 202 — collector p95: 8 ms
}

The collector succeeds the instant the event is in the log. The customer NEVER waits on a consumer. Analytics down? Its consumer group lags and catches up later. Zero loss, zero customer impact.

The consumer — groups, manual commit, retry, DLQ

Three things matter: a consumer group (so partitions parallelize), manual offset commit (commit only after successful processing — at-least-once delivery), and a retry + dead-letter path (so a poison message can't block the partition forever).

protected override async Task ExecuteAsync(CancellationToken ct)
{
    consumer.Subscribe("events.raw");
    while (!ct.IsCancellationRequested)
    {
        var result = consumer.Consume(ct);
        var ev = JsonSerializer.Deserialize<CampaignEvent>(result.Message.Value)!;
        try {
            await rollup.ApplyAsync(ev, ct);                              // do the work
            consumer.Commit(result);                                       // ✅ commit AFTER success
        }
        catch (TransientException) when (Attempts(result) < 5) {
            await producer.ProduceAsync("events.retry", result.Message, ct);
            consumer.Commit(result);
        }
        catch (Exception ex) {
            log.LogError(ex, "Poison → DLQ at offset {Offset}", result.Offset);
            await producer.ProduceAsync("events.dlq", result.Message, ct);
            consumer.Commit(result);                                       // don't block the partition
        }
    }
}

Consumer lag is the one health metric. If lag climbs, the consumer can't keep up. Scale it. The producer is untouched.

The outbox — atomic produce with a DB write

There's one trap. If the same handler writes to Azure SQL and produces to Kafka, those are two systems. A crash between them loses or duplicates the event. For anything tied to a committed DB change, use the outbox pattern: write the event to an outbox table in the same SQL transaction as the business data; a background relay reads unsent rows and publishes them.

// Atomic: the campaign row AND the event commit in ONE SQL transaction.
await using var tx = await db.Database.BeginTransactionAsync(ct);
db.Campaigns.Add(campaign);
db.Outbox.Add(new OutboxMessage(
    Topic: "events.raw",
    Payload: JsonSerializer.Serialize(ev)));
await db.SaveChangesAsync(ct);
await tx.CommitAsync(ct);
// Background relay reads unsent outbox rows, produces to Kafka, marks them sent.

Dual-write inconsistencies (the "DB says published, no event fired" bug class) → 0.

Idempotent consumers — because delivery is at-least-once

Kafka gives you at-least-once delivery. A consumer can crash after processing but before committing and reprocess on restart. So consumers must be idempotent. The simplest tool is a dedup key with a unique constraint.

public async Task ApplyAsync(CampaignEvent ev, CancellationToken ct)
{
    var firstTime = await _db.ProcessedEvents
        .ExecuteInsertIfAbsentAsync(ev.Id, ct);                 // unique index on EventId
    if (!firstTime) return;                                     // duplicate → no-op, safe

    await _db.Aggregates.IncrementAsync(ev.CampaignId, ev.Type, ct);
}

At-least-once delivery + idempotent processing = effectively-once, and it's far simpler than chasing true exactly-once.

The two superpowers REST never gave us

Replay. A bug corrupted yesterday's analytics rollup? Reset that one consumer group's offset and reprocess. The other groups are untouched. The raw events are still in the log — the log is the source of truth.

kafka-consumer-groups --bootstrap-server $BROKERS --group analytics \
  --reset-offsets --to-datetime 2026-06-10T00:00:00.000 --topic events.raw --execute

Adding a consumer for free. When we shipped fraud scoring, we created a new consumer group on the same events.raw topic. The producer didn't change. ~1 day instead of ~1 sprint.

Aggregate metrics

Metric	Before (REST)	After (Kafka)	Delta
Ingestion failures (incident windows)	1.9%	0.18%	−90%
Events lost on downstream outage	tens of thousands	0	eliminated
Ingestion p95	180 ms	8 ms	−96%
Cascading-failure incidents / mo	~3	0	eliminated
Retry-storm load during incidents	severe	none (202 + walk away)	eliminated
Time to add a new consumer	~1 sprint	~1 day	−80%+
Replay a bad day of processing	impossible	one command	new
Dual-write inconsistencies (outbox)	recurring	0	eliminated

Most of the win isn't throughput — it's failure isolation. The producer succeeds against a log that's almost always up, and every downstream problem became "a consumer is lagging" instead of "the pipeline is down and we're losing data."

When NOT to reach for Kafka (honest section)

Don't replace request/response with Kafka. If the caller needs the answer, that's a query — REST/gRPC is correct.
Kafka is operational weight. Brokers, partitions, schema registry, lag monitoring. For modest volume, Azure Service Bus or a DB-backed queue gives most of the decoupling with far less to operate. We used managed Kafka (Confluent Cloud) precisely because a 5-person team doesn't run brokers.
"Exactly-once" is a trap. Plan for at-least-once + idempotent consumers.
Ordering is per-partition, not global. Design your key around the ordering you actually need.
You traded synchronous errors for asynchronous lag. Watch lag, or you've hidden the problem.
On Azure, weigh Service Bus / Event Hubs first. Kafka was a deliberate choice (replay + ecosystem), not a default.

The mental model in one line

A synchronous call couples you to the callee being up; an event in a log couples you to the log being up. For anything that's "this happened" rather than "tell me this," publishing to a durable log decouples producers from consumers in time — and most of resilience, burst absorption, and extensibility falls out of that one change.

3-minute video walkthrough on YouTube: https://youtu.be/O21rbuQdM1Y

Full deep-dive with architecture diagrams, the complete outbox teardown, idempotency patterns, replay commands, and the honest list of when NOT to reach for Kafka:

👉 https://prepstack.co.in/blog/replaced-rest-with-kafka-cut-failures-90-percent

If this saved you from a "the pipeline went down at 3 AM" page, a ❤️ or 🦄 helps it reach more backend engineers.

What's the worst cascading-failure incident you've inherited because someone Kafka-ed a query — or REST-ed an event?