DEV Community: Marc

You asked how I know what the model did. Here are the answers, with numbers.

Marc — Thu, 06 Aug 2026 15:44:56 +0000

I wrote about our AI pipeline costs a while back. The comments were better than the post.

Valentin Monteiro made the point that cache alerts should be keyed per prompt version rather than on a global ratio, because a global number moves for boring reasons. Tae Kim pointed at how prompt caching fails silently, where a breakpoint on anything per-request gives you a miss on every call that logs exactly like a hit. Both need the same thing underneath, and I said I'd get cache metrics into our provenance records.

That turned into a bigger job than I expected. Here's what came out of it, what it costs, and the two parts I still haven't worked out.

Why store more than the output?

Because the output can't tell you whether something was always broken or broke last Tuesday, whether it's one record or ten thousand, or whether you changed something or the vendor did.

A total_amount column has the value. Nothing about where it came from.

Where do you record it?

At the provider, not in the feature.

I did it in the feature first. It worked fine. But then every new AI feature has to remember to do the same thing, and eventually one won't. So it moved down to where providers get built:

function withProvenance(provider, meta) {
  return async (request) => {
    const startedAt = performance.now()
    try {
      const response = await provider(request)
      await recordAiCall(buildAiCallPayload({ meta, request, response, startedAt }))
      return response
    } catch (error) {
      await recordAiCall(buildAiCallPayload({ meta, request, error, startedAt }))
      throw error
    }
  }
}

Nothing opts in because nothing gets asked.

One detail: this write goes outside the handler's transaction. If the business transaction rolls back, the call still happened and you still paid for it.

What's in the payload?

{
  "providerId": "anthropic",
  "handlerName": "invoice-extract",
  "requestedModel": "claude-sonnet-5",
  "respondedModel": "claude-sonnet-5-20260514",
  "promptVersion": "3f9a1c0e77b2",
  "inputHash": "9c4e1ab77f30d552",
  "latencyMs": 1180,
  "usage": {
    "inputTokens": 2140,
    "outputTokens": 318,
    "cacheCreationInputTokens": 0,
    "cacheReadInputTokens": 1890
  },
  "reportedCostUsd": 0.0042,
  "stopReason": "end_turn"
}

The two cache fields are there because of Valentin's comment. cacheReadInputTokens against inputTokens is your hit ratio, and now it's per call rather than a monthly average, so you can key an alert on it per prompt version like he suggested instead of watching one global number.

Requested and responded model are separate fields. Looks pedantic until a vendor routes you somewhere else and they don't match.

No prompt text, no output. Only hashes. The log lives forever and prompts are full of customer invoices.

Failures get a row too, with error kind and HTTP status. Most setups only log the successes.

How do you version a prompt?

I planned to put a version in the prompt file and bump it by hand. Then I didn't, because hand-maintained versions rot.

It hashes the stable, model-visible part of the request at runtime instead: system instructions, corpus, and the tool schemas. Twelve hex characters, and nobody bumps anything. Messages stay out on purpose, because if they were in it every call would get its own version and the thing would group nothing.

The "and the tool schemas" part is newer than this post. It used to hash the cacheable prefix only. Our extraction handler builds its tool from the caller's output schema, so you could change that schema, send the model a demonstrably different request, and promptVersion wouldn't move. inputHash saw it, but that one is unique per call, so it identifies without grouping. Which is the missed bump the runtime hash was supposed to make impossible. Found it writing this, wrote an issue, fixed it, merged it before the post went out. toolChoice went in at the same time, the field Tae flagged in the last thread ;)

What's left after that is key order, and it's a real one. JSON.stringify hashes the serialisation, so reordering properties in a tool schema flips the version without changing anything semantically.

The obvious move is to sort keys and hash a canonical form. We decided against it, and the reason is the interesting bit: the model sees the prompt as serialised text, and property order in a JSON schema affects the order an LLM generates fields in. Two differently sorted schemas are genuinely two different prompts. Sorting them into one bucket would rebuild exactly the missed bump we just removed, only invisibly. So the trade isn't false splits against churn. It's a visible false split against an invisible missed bump, and visible wins.

The other thing is on purpose. The hash is one-way, so you get 3f9a1c0e77b2 and no route back to the prompt. You can tell which calls are affected but not what changed in them. Reconstructing means git plus rehashing against the corpus from that day. The hand-bumped file version would have handled that with git blame. I didn't think about it until after I'd shipped the hash.

We have a prompt store with proper revision history sitting in the same codebase, connected to none of this. That's probably the answer and I haven't wired it up.

What does it cost?

Postgres 16, synthetic events, shape checked against real ones from the provider path:

513 bytes per payload at the median, 517 at p95
about 1,036 bytes per call with row overhead and the event store's three indexes
at 10k calls a day that's roughly 296 MB a month, 3.5 GB a year

Same per-call number at 10k, 100k and 1M rows. Storage isn't where the money goes.

Can you query it?

This is the part where measuring changed my mind.

"Every call with prompt version X" is what you run when something is wrong. An event store indexes tenant, aggregate type and time. Not the inside of a JSONB payload.

Scoped to one tenant and a time window: 162 ms over a million events. The index picks ~67,000 candidate rows and the payload filter narrows those to 8,490, so you pay for hauling 67,000 rows out of the heap.

Globally it's a seq scan, so you add an expression index on the payload field. At a million rows:

distinct prompt versions	no index	with index	plan
8	115 ms	281 ms	ignored it, seq scan
50	107 ms	61 ms	bitmap heap scan
500	107 ms	1.02 ms	bitmap heap scan

Index costs 6.9 MB in all three.

With eight versions one version is 12.5% of the table, so Postgres scans and is right to. The index just sits there.

The problem is that day one is when you benchmark this, see nothing, and decide the index isn't worth it. Six months in you've edited prompts a few hundred times, one version is 0.2% of the table, and that same index is a hundred times faster. Add it once you're past single digits.

Do I need event sourcing?

No. An append-only ai_calls table gets you nearly all of it. We already had the event log so it came out of that for free.

The bit worth copying either way is where you put the recording.

What would you change before this goes live?

Asking for real. It's built and merged but not out to a live tenant yet, so this is still a good moment to hear that it's wrong.

Which is also why there are no cost or latency numbers from real traffic in here. We don't have them yet. In two months I will.

If you've run something like this for a while: what's in your call log that isn't in mine? And where does this fall apart at a volume I haven't hit?

The benchmark SQL is in the repo if you'd rather measure your own database than trust mine. It's self-contained, so psql -v rows=1000000 -v prompt_versions=500 -f ai-call-provenance-benchmark.sql against a throwaway database reproduces the table above.

I build Kumiko, an event sourced framework for multi-tenant B2B systems in TypeScript.

Multi-Tenant AI Chat: From Hardcoded Config to BYOK in 4 Steps

Marc — Mon, 03 Aug 2026 08:35:50 +0000

Two tenants, two AI providers, two prompts. Sounds simple, and on day one it is. That's the trap. It stays simple right up until customer number two sends their first "quick question," and eighteen months later you're running a small distributed system to answer it. Here's the honest version of that slide, four stages, each one caused by a real human typing a real request into Slack.

Day 1: it just works (he says, foolishly) 😅

Tenant A wants OpenAI. Tenant B wants Claude. Both want their own system prompt. The obvious first version: one config object, one row per tenant. What could possibly go wrong. (Everything. Everything could go wrong. But not yet.)

┌─────────────┐     ┌──────────────────┐     ┌─────────────┐
│  Request    │ →   │  TENANT_CONFIG   │ →   │  Provider   │
│  (tenantId) │     │  (hardcoded obj) │     │  SDK call   │
└─────────────┘     └──────────────────┘     └─────────────┘

const TENANT_AI_CONFIG = {
  tenantA: { provider: 'openai', model: 'gpt-5', prompt: 'You are terse and technical.' },
  tenantB: { provider: 'anthropic', model: 'claude-sonnet-5', prompt: 'You are friendly. Antworte auf Deutsch.' },
} as const

async function handleChat(tenantId: string, userMessage: string) {
  const config = TENANT_AI_CONFIG[tenantId]
  const client = config.provider === 'openai' ? openai : anthropic
  return client.chat(config.model, config.prompt, userMessage)
}

Ships in an afternoon. Two tenants, two rows, demo goes great, everyone claps 👏. Put this moment in a frame, it's the calmest the codebase will ever be.

Evolution 1: Tenant B wants their own twist

A week in (a week, we didn't even get a full sprint), tenant B messages: "can we change the prompt ourselves, without waiting for a deploy?" Fair ask, they know their users, we don't, and also nobody wants to be the on-call engineer who gets paged to edit a string literal. A hardcoded object can't answer that, it needs a rebuild to change a comma.

┌─────────────┐     ┌──────────────────┐     ┌─────────────┐
│  Request    │ →   │  tenant_settings │ →   │  Provider   │
│  (tenantId) │     │  (DB row, admin  │     │  SDK call   │
│             │     │   editable)      │     │             │
└─────────────┘     └──────────────────┘     └─────────────┘

Config moves from a code constant to a table the tenant's own admin UI can write to.

async function getAiConfig(tenantId: string) {
  const row = await ctx.db.tenantSettings.findOne({ tenantId })
  return row.aiConfig // { provider, model, prompt }
}

Same shape as before, different source. handleChat doesn't change at all, it has no idea any of this happened, which is exactly the point of putting the lookup behind one function.

Evolution 2: "wait, who changed the prompt?" 🕵️

Self-service is great until it isn't: tenant B's prompt quietly changed last Tuesday, their bot started answering in pirate-speak for reasons nobody can reconstruct, support gets a ticket, and the honest answer is "we have no idea, the database doesn't remember either." A plain DB row just gets overwritten, the past has no representation, it's Ctrl+Z with no undo history.

┌──────────────┐     ┌────────────────┐     ┌──────────────────┐
│  Admin edits │ →   │  ConfigChanged │ →   │  current config  │
│  the prompt  │     │  event (who,   │     │  = fold(events)  │
│              │     │  when, diff)   │     │                  │
└──────────────┘     └────────────────┘     └──────────────────┘

The config becomes event-sourced instead of a mutable row: every change is an event, the current value is a projection over them.

async function updateAiConfig(tenantId: string, patch: Partial<AiConfig>, actor: string) {
  await ctx.emit('AiConfigChanged', { tenantId, patch, actor, at: ctx.now() })
}

async function getAiConfig(tenantId: string): Promise<AiConfig> {
  const events = await ctx.db.events.find({ tenantId, type: 'AiConfigChanged' })
  return events.reduce((cfg, e) => ({ ...cfg, ...e.patch }), DEFAULT_AI_CONFIG)
}

Now "who changed it and when" is a query, not a seance 🔮. handleChat still hasn't changed, it just calls getAiConfig, blissfully unaware it's now talking to an event log instead of a table.

Evolution 3: BYOK and a usage cap 💸

A bigger tenant shows up, the kind that gets its own Slack channel, with two demands: they want to use their own OpenAI key (cost control, their own rate limits, their own finance team breathing down their neck), and they want a hard cap on monthly spend so an over-caffeinated intern's script can't turn into a five-figure invoice.

┌──────────────┐     ┌───────────────────────┐     ┌──────────────┐
│  Request     │ →   │  config.apiKey?       │ →   │  usage < cap?│
│              │     │  (BYOK, encrypted)    │     │  → call      │
│              │     │  else our shared key  │     │  → else 429  │
└──────────────┘     └───────────────────────┘     └──────────────┘

async function callProvider(tenantId: string, userMessage: string) {
  const config = await getAiConfig(tenantId)
  const usage = await getMonthlyUsage(tenantId)
  if (config.usageCap && usage >= config.usageCap) {
    throw new UsageCapExceeded(tenantId)
  }

  const apiKey = config.byokApiKey ?? process.env.SHARED_API_KEY // BYOK overrides shared key
  const client = getClient(config.provider, apiKey)
  const reply = await client.chat(config.model, config.prompt, userMessage)

  await recordUsage(tenantId, reply.usage.totalTokens)
  return reply
}

Two additive fields on the same config, byokApiKey and usageCap, and one counter check before the call. No new architecture, no rewrite of the first three stages, no "sorry, we need a full quarter to redesign this."

The pattern behind the pattern 🧵

Every stage kept getAiConfig(tenantId) → { provider, model, prompt, ... } as the seam. Storage changed underneath it four times (constant, DB row, event-sourced projection, projection with encrypted secrets) and the call site never noticed, never cared, never even asked. That's the actual lesson: don't design the multi-tenant AI system upfront, design one seam that can absorb whatever the next Slack message throws at it.

Skipped on purpose: provider fallback, streaming, per-model cost tables. Add those when a tenant actually asks, same as everything above. If a tenant asks for a fifth provider before you've read this sentence, that's not a counterexample, that's Tuesday. 🙃

We Cut Our AI Pipeline Costs 25% Without Losing Accuracy (and the fix wasn't a cheaper model)

Marc — Sat, 01 Aug 2026 18:46:54 +0000

Our AI pipeline runs three step kinds (ai.generate, ai.extract, ai.classify), each independently resolving its own provider, model, and prompt revision at run time. The default model is Sonnet, not Opus — and for a while that felt like a compromise, because Opus was the expensive-but-reliable option and Sonnet needed babysitting to hit the same pass rate.

The fix that closed the gap wasn't a smarter prompt. It was tool_choice plus a tightened output schema, forcing the model to commit to an answer shape instead of spending tokens hedging its way there. That alone got Sonnet to pass-parity with Opus's prior output at roughly a quarter of the cost, in our eval. A second, separate lever: Anthropic's own recommended effort: "xhigh" for agentic tasks produced roughly twice the thinking tokens of "high" for the same pass rate — thinking tokens bill like output tokens, so that's a straight 2x for zero accuracy gain, and it only matters when Opus is used via an explicit override (Sonnet doesn't get adaptive thinking at all). A third, independent lever: max_tokens defaulted to 16000 out of caution; dropping it to 4000 (still comfortably above what any real step needed) cut effective output cost again, because Anthropic bills against the cap as an upper bound in some failure paths, not only against what the model actually emits.

None of these three touched the prompt content or the provider. All three came from looking at per-step token usage, which only exists because every step writes an immutable provenance record on completion: prompt revision, provider, model, token usage. Same idea as event sourcing, applied to LLM calls instead of domain writes. You don't trust "the pipeline probably used prompt v3, on whatever model was configured," you have a row that says so — and you can audit six months of "which prompt touched this tenant" without ever loading the actual LLM payloads (also the DSGVO-friendly shape: provenance rows carry no user content, only call metadata).

The other real trap, found the hard way: adaptive thinking and forced tool-use don't mix. Anthropic 400s with "Thinking may not be enabled when tool_choice forces tool use" the moment both are set, which sounds obvious in hindsight but not when you're setting both because both individually sound like "make the model try harder." The fix has to be automatic — detect a forced tool_choice and disable adaptive thinking before the request goes out — because leaving it to every caller to remember means every eval run using toolChoice: { type: "any" } against an Opus override breaks the same way, all at once.

Separately: prompt caching only pays off if the cache_control breakpoint sits on the last static block, with tools and system prompt as one cacheable prefix. Past roughly 16k tokens without it, requests start hitting the SDK's HTTP timeout before the response streams back far enough to matter. Put the breakpoint on something that changes per request and you get a silent cache-miss that looks identical to a hit in the logs while you keep paying full price.

Full writeup with the pipeline architecture (with diagrams), the provider-resolution code, and the provenance shape: docs.kumiko.rocks/en/guides/ai-pipeline-provenance.

We Shipped a Mortgage Calculator Bug Where Every Test Was Green and the Answer Was Still Wrong

Marc — Sat, 01 Aug 2026 11:48:07 +0000

A review pass on our credit calculator flagged something that should have been impossible: a
finding tagged HIGH, on a panel with a full test suite, where every individual test passed and
the answer was still wrong. Not "wrong in an edge case" wrong. Wrong for every single user who had
ever configured a Sondertilgung (a lump-sum extra payment on their mortgage) and then looked at
the "what if I invest the difference instead" comparison next to it.

The bug: the comparison panel always ran the loan projection with an empty additionals list. The
headline result above it, the number the user actually configured and trusted, included their
real Sondertilgungen. So if you had a 20k lump-sum payment scheduled for month 6, the headline
said "your loan is basically gone by year 8" and the panel right next to it, presented as
directly comparable, was quietly running a different loan. Not a rounding difference. A different
payoff date, a different remaining balance, a different verdict on whether investing instead
would have won.

Every test was green because every test checked one function against one input. Nothing checked
that the two numbers on the same screen were talking about the same loan.

Why "pay down debt faster vs. invest the difference" is rigged by default

That specific bug is a symptom of a more general trap. Almost every calculator online that lets
you compare "pay down debt faster" against "invest the difference instead" gets the comparison
itself wrong, independent of any coding bug — because it quietly changes the monthly budget
between the two scenarios and presents the result as a fair fight.

Say you have a 300k loan at 3.5%, paying 3% annual amortization. Someone suggests: "drop to 1%
amortization, invest the rest." The naive calculator compares:

Scenario A: 3% amortization, no investing, ~1200/month total outflow
Scenario B: 1% amortization, invest a fixed 200/month, ~950/month total outflow

Of course B looks better over ten years. You're comparing spending 1200/month against spending
950/month and marveling that the cheaper option built more wealth. It didn't win on insight. It
won because you fed it a smaller number.

The fix is a structural invariant, not a footnote:

credit installment (this month) + ETF contribution (this month) = fixed monthly budget

Every month, for both scenarios, and once a loan is paid off in one scenario, its full budget
rolls into the ETF contribution from that point on. You don't get to just stop spending because
the mortgage happened to end early in the model. Once that holds, "total interest paid" stops
being a meaningful metric (it no longer means the same thing across scenarios that pay
differently), and the only honest number left is:

net worth(t) = ETF portfolio(t) - remaining loan balance(t)

at a fixed horizon, for both scenarios, with identical cash outflow throughout.

The invariant was never the bug. Something else was.

Here's the part that made this finding interesting: our budget invariant was correctly
implemented from day one. Every month's ETF contribution is computed as monthlyBudget - loanRate, not hardcoded, not assumed. If you'd audited that one property in isolation, you'd have
signed off.

The actual failure mode was a level up from that: getting the fair-comparison math right and then
silently comparing two different loans anyway. The panel and the headline agreed on the rule
("budget minus installment goes to the ETF") but disagreed on the input ("what installment,
exactly, on what loan"). A structurally correct comparison of the wrong pair of scenarios is still
wrong, and none of the unit tests around the fair-comparison logic could have caught it, because
that logic was never the part that broke.

The fix threads the same additionals the headline uses through the comparison panel, so both
numbers on the screen are now guaranteed to describe the same loan.

The actual lesson, and something to go check right now

Budget-neutrality gets cited (correctly) as the thing naive pay-down-vs-invest calculators get
wrong. It's necessary. It is not sufficient. You also have to pin every other input, identically,
across both scenarios you're comparing, or the comparison quietly starts answering a different
question than the one on the label, and a green test suite will not tell you.

If you've got a mortgage calculator, a rent-vs-buy tool, or a pay-down-vs-invest comparison
bookmarked somewhere: go feed it a scenario with something extra configured (a lump-sum payment,
an irregular income month, whatever the tool supports), and check whether the comparison panel
still uses it. Most won't tell you either way. Check anyway.

This is the Tilgung-vs-ETF panel in cashcolt, a free
no-signup mortgage calculator hosted in Germany. No tracking, no login, poke at it with your own
numbers.

Your event store is already your audit log

Marc — Wed, 01 Jul 2026 07:51:25 +0000

Your event store is already your audit log

Almost every SaaS I've worked on ends up with an audit_log table. Someone files a compliance ticket — "we need to know who changed what and when" — and a new table appears next to the domain tables. Then the real work starts: writing to it on every mutating endpoint, keeping it in sync, and quietly discovering six months later that three endpoints forgot to log.

That table is a second source of truth. And second sources of truth drift.

What an audit log actually needs

Strip the compliance language away and an audit entry is five fields:

who did it
when they did it
what they touched (which entity)
which action it was
the delta — what actually changed

Plus, for a multi-tenant app: whose data it was, so tenant A can never read tenant B's history.

Now look at what an event in an event-sourced system carries. Every state change is an appended event with createdBy, createdAt, tenantId, aggregateType + aggregateId, type, and a payload holding the delta.

That's the same five fields. The event log already is the audit trail — append-only, ordered, and impossible to forget to write, because writing the event is how state changes in the first place. There's no code path that mutates data without producing an event.

So don't build the table. Query the log.

If the audit trail is already there, the whole "audit feature" collapses into one privileged read over the events table:

export const listQuery = defineQueryHandler({
  name: "list",
  schema: z.object({
    aggregateType: z.string().optional(),
    aggregateId: z.uuid().optional(),
    eventType: z.string().optional(),
    userId: z.string().optional(),
    from: z.iso.datetime().optional(),
    to: z.iso.datetime().optional(),
    limit: z.number().int().min(1).max(100).default(50),
    before: z.string().optional(), // cursor
  }),
  access: { roles: ["Admin", "SystemAdmin"] },
  handler: async (query, ctx) => {
    const p = query.payload;
    const where = { tenantId: query.user.tenantId }; // tenant-isolated at the WHERE
    if (p.aggregateType) where.aggregateType = p.aggregateType;
    if (p.aggregateId)   where.aggregateId   = p.aggregateId;
    if (p.eventType)     where.type          = p.eventType;
    if (p.userId)        where.createdBy      = p.userId;
    // ...time range + cursor omitted for brevity

    return selectMany(ctx.db, eventsTable, where, {
      orderBy: { col: "id", direction: "desc" },
      limit: p.limit,
    });
  },
});

No table, no projection, no write path, no sync job. The filter surface an audit UI wants — by entity, by actor, by action, by time — is just WHERE clauses over columns the events already have.

The two things you still owe

Reusing the event log doesn't come completely free. Two concerns are real:

1. Access control. The event log is the most sensitive read in the system — it's literally everything that ever happened. Gate it hard (Admin / SystemAdmin above) and pin tenant isolation into the WHERE clause itself, not into application logic that a future refactor can bypass. Cross-tenant peeking should be structurally impossible, not politely discouraged.

2. PII. If you dump raw event payloads into an audit view, you'll surface fields you didn't mean to. The clean fix is to strip sensitive values at append time — mark them in the entity definition and never let them into the stored event. Then the audit read physically cannot leak them, because they were never written. Doing it at read time is a filter you'll eventually forget on some new field; doing it at write time is a guarantee.

When this doesn't apply

Honesty: this only works if you're actually event-sourced. If your system does in-place UPDATEs, there's no historical record to query — you genuinely need to start capturing one, and a dedicated table (or CDC/logical decoding off the WAL) is the pragmatic path. This isn't an argument to adopt event sourcing for audit; it's an argument that if you already have it, the second table is redundant.

One caveat even when it fits: event schemas evolve, so your audit reader sees heterogeneous historical payloads. For an audit log that's a feature — you want the exact shape as it was written — but don't mistake it for a clean queryable projection.

The point

An audit log isn't a thing you build. It's a view onto history you're already keeping. If you're appending events, you've been sitting on a complete, tamper-evident audit trail the whole time — the only missing piece was a gated query with the right WHERE clauses.

This is exactly how the audit feature works in Kumiko, a Bun/TypeScript framework where multi-tenancy, GDPR, and audit are bundled features rather than boilerplate you rewrite per project — one ~40-line query handler, no separate table.

How do you prevent AI-generated code from drifting away from your conventions over time?

Marc — Sun, 28 Jun 2026 10:52:37 +0000

We've been generating production features with AI for a while now — auth flows, billing hooks, notification handlers. And we've hit a pattern we don't have a good answer to yet.

The first feature the AI generates looks great. It reads the codebase, picks up the patterns, and the output looks like something a senior dev wrote.

The tenth feature? Less so. Small inconsistencies creep in:

A handler that doesn't follow the error-handling convention
A schema field with a different naming pattern
A test that checks existence instead of behavior

None of it is wrong. All of it is subtly inconsistent.

Fixes we've tried

AGENTS.md / CLAUDE.md — helps, but gets stale and doesn't scale with the codebase

Code review — catches it, but defeats some of the speed advantage

Linting + formatting — catches easy stuff, misses semantic drift

What we haven't solved: giving the AI a "living" representation of your conventions that stays current as the codebase evolves.

We're building Kumiko — an opinionated SaaS framework — partly as an answer to this. If the framework constrains what's possible, drift has less surface area. But I'm not convinced that fully solves it either.

Curious what's actually working for others

Do you just review AI output carefully and accept some drift?
Custom guards / linters that encode your conventions?
Something that auto-generates AGENTS.md from the codebase?

What's your approach?

Multi-Tenancy in Bun/Hono Without Boilerplate

Marc — Wed, 24 Jun 2026 19:11:12 +0000

Every multi-tenant SaaS has the same problem: you need to make sure every query only returns data for the right tenant. Forget a WHERE tenant_id = ? once, and you have a data leak.

The obvious solution — a separate database per tenant — doesn't scale. Connections are expensive, migration overhead multiplies, and you lose cross-tenant reporting.

For Kumiko we went a different route: a single DB pool, but every query automatically gets the tenantId injected — without handler code ever having to do it manually.

The Idea: TenantDb Instead of a Raw DbRunner

Instead of passing a raw DB connection around, we create a TenantDb wrapper per request:

const db = createTenantDb(rawDb, tenantId)

From that point, db behaves like a normal database — but with automatic isolation baked in:

// Handler code — no tenantId needed
const users = await ctx.db.selectMany(usersTable)
// → SELECT * FROM users WHERE tenant_id IN ('tenant-123', 'system')

await ctx.db.insertOne(usersTable, { name: 'Max' })
// → INSERT INTO users (name, tenant_id) VALUES ('Max', 'tenant-123')

await ctx.db.updateMany(usersTable, { name: 'Moritz' }, { id: userId })
// → UPDATE users SET name='Moritz' WHERE id=? AND tenant_id='tenant-123'

Handlers write plain CRUD code. Isolation happens underneath — invisible, but enforced.

How the Injection Works

Reads: own rows + reference data

Read queries always see two tenants: the current one and SYSTEM_TENANT_ID. This allows reference data (e.g. global config) to be visible to all tenants without duplicating it:

// tenantId filter === [currentTenantId, SYSTEM_TENANT_ID]

If a handler passes its own tenantId in the WHERE clause, it can only narrow the scope, never widen it. A where: { tenantId: 'other-tenant' } is silently dropped.

Writes: own rows only

Inserts get tenantId forced in — and the value cannot be overridden by the caller:

// mode === "tenant": tenantId on INSERT is enforced last
return { ...data, tenantId }  // overwrites whatever the handler passed

Updates and deletes without a WHERE clause throw an error instead of hitting all rows:

// Prevents accidental mass-updates
"TenantDb.updateMany without where would mass-update all tenant rows."

System mode for operators

For admin screens there's r.systemScope() — queries run unfiltered across all tenants. Explicit opt-in only, never the default.

Tenant Resolution in Hono

TenantDb needs a tenantId. It comes from middleware that resolves it from the request — either from the hostname (for custom domains) or from the JWT:

// Middleware (simplified)
app.use('*', async (c, next) => {
  const tenant = await resolveTenant(c.req)
  c.set('db', createTenantDb(rawDb, tenant.id))
  await next()
})

Every handler then gets proper isolation through ctx.db — without a single line of tenant logic in actual feature code.

What This Means in Practice

Across three years of production use and several apps (CashColt, publicstatus, kumiko-studio) we've had zero data leak bugs from forgotten tenant filters. Not because we were particularly careful, but because it's structurally impossible to forget.

The overhead: nearly zero. A few extra conditions per query, no extra DB connection pool, no migration overhead multiplied per tenant.

If you want to try the framework: kumiko.rocks — open source under BUSL-1.1.