DEV Community: rinat kozin

redb.Route.Llm 3.1.1 — per-message audit fields for LLM compliance / replay

rinat kozin — Thu, 11 Jun 2026 22:48:57 +0000

When an auditor asks "reproduce this exact answer" six months later, you need more than (messages, model). You need the sampling parameters that were actually applied, the prompt template version, the tool set the model saw, and ideally the provider's backend fingerprint — because closed-source providers silently re-release weights.

Just shipped this in redb.Route.Llm 3.1.1: 7 nullable audit fields on every persisted message.

Set on assistant rows:

Temperature, MaxTokens, TopP — effective values (request override → factory default)
ToolSetHash — SHA-256 of canonical {name, description, InputSchema} set, sorted by name. Tool added/renamed/schema-changed → hash changes
ProviderSystemFingerprint — system_fingerprint from the response (OpenAI / xAI / Together echo it; Anthropic / Gemini-compat / Ollama leave it null)
Set on every row of the run:

PromptTemplateName, PromptTemplateVersion — when the caller used a managed template
Honest limitation: bit-exact replay is only possible with self-hosted (Ollama / vLLM / llama.cpp). Anthropic doesn't expose a fingerprint, OpenAI rotates them silently. For closed providers we record (model_id, params, tool_set_hash, prompt_template) and label it best-effort.

Zero migration. REDB stores props schemalessly — added 7 fields to the C# class, redeployed, done. No ALTER TABLE, no version bump on the storage layer.

Code & details: CHANGELOG.md → 3.1.1 — section per-message audit fields on MessageProps.

Triggered by feedback from a compliance auditor on Habr — turns out "we log conversations" doesn't cut it when the courtroom asks which exact prompt drove this.

Enterprise-grade AI integration: embedding LLMs into the business processes of large companies — redb.Route.Llm 3.1.1

rinat kozin — Wed, 10 Jun 2026 21:05:20 +0000

Series: redb ecosystem (LLM, part 2)

A few days ago I announced redb.Route.Llm — the 24th transport in redb.Route, where calling an LLM is .To("llm://claude") and an agent tool is just a route with .AsLlmTool("shell") on it. The whole pitch was: stop bolting an "AI framework" onto an integration framework that already has retry, throttle, breaker, audit, observability. Plug the LLM in as one more endpoint, get those primitives for free.

I closed that announcement with a deliberately uncomfortable section called the "honest skip-list" — features I hadn't shipped yet: streaming end-to-end, tool cache, RAG knowledge store, async batch + callback consumer, eval-run store, sliding-window memory, sandboxed tool execution. I'd rather under-promise in writing than over-promise in screenshots.

Most of that list is shipped. But that's not what this post is about. This post is about the thing I didn't quite see at announcement time: shipping the skip-list turned the whole thing into something different from a chat library. It turns out that once you put an LLM into an ESB and start adding the boring enterprise plumbing, the result isn't "a chat framework with extras". It's a runway from a 6-line demo to a multi-tenant, audited, budgeted, human-in-the-loop agent platform — with no rewrite in the middle.

That's the story I want to tell. Three angles: deep dive on the technically-tricky bits (streaming, persistence, RAG), eight enterprise patterns that collapse into one DSL line each, and a project-to-platform timeline I've watched live more than once.

Coming in cold? Earlier in this series:

redb.Route 3.1.0 — LLM as just another connector (announcement)

redb.Route — Apache Camel for .NET

Enterprise Integration Patterns in .NET, part 1

A quick word on EIP. Throughout this post I lean on Enterprise Integration Patterns — Multicast, Aggregator, Scatter-Gather, Wire-Tap, Choice, Aggregate-by-window — without unpacking each of them in detail. That's deliberate: the "EIP in .NET via redb.Route" series with per-pattern deep-dives (diagrams, code, side-by-sides) hasn't shipped yet, I'm writing those in parallel. So I'm getting a little ahead of myself here.

If you've worked with Apache Camel or WSO2 Micro Integrator, you'll spot the patterns on the fly — the names and semantics are identical. I've personally shipped and maintained integration projects on WSO2 MI for years (and on its ESB ancestor), but architecturally redb.Route sits closer to Camel — a compiled DSL over a typed Exchange, not an XML config. Side-by-side comparisons "Camel ↔ redb.Route" and "WSO2 MI ↔ redb.Route" are their own posts, also in flight. If anything EIP-flavoured in the code below trips you up, drop a question in the comments — I'll answer on the spot, and the answer goes straight into those upcoming pieces.

Why this isn't "yet another agent framework"

The agent-framework shelf is crowded. LangChain, LangChain4j, Semantic Kernel, AutoGen, LlamaIndex — each one solves the same single problem: how does my model call functions and remember a conversation. Each one leaves you to solve the other eighteen problems yourself: retry, idempotency, audit, multi-tenant, budget, observability, governance, approval, batch, scheduling, replay, dead-letter, throttling, circuit-breaking, correlation, timeouts, cost attribution, metric exporters.

redb.Route.Llm's entire reason for existing is to make those eighteen problems be already solved at the framework layer. Not "we have an integration library, plus an LLM library, glue them yourself" — same DSL, same runtime, same governance hooks. If you already have a redb.Route policy interceptor that writes every message to Kafka, it automatically writes every tool call from your agent to Kafka too — because they're the same Exchange.

This isn't theoretical elegance. It cuts through the single biggest production headache with agents: what happens the day your model decides to drop a production table? In the LangChain-plus-hooks shape, the answer is "wire up our approval callbacks and don't forget". In the LLM-as-transport shape, the answer is "intercept the Exchange before .To("exec://...") with the same .Process(...) you use for any other potentially-destructive step in any other route". Not a new pattern for AI. The same pattern as for FTP, Kafka, JMS.

Same people, same tools, same review gates. That's the actual win.

What got shipped from the skip-list

I said I would, and I'm saying I did.

Skip-list item	Status in 3.1.1
Streaming end-to-end (HTTP SSE + WS per-frame)	Shipped. `IAsyncEnumerable<string>` lives in `Out.Body`, the HTTP consumer flips into SSE, the WS consumer dispatches one frame per message.
ToolCacheStore on REDB	Shipped. `ToolCacheProps`, opt-in via `ToolCachingPolicy.Memoize`.
KnowledgeStore — RAG chunks in REDB	Shipped (partial). `KnowledgeChunkProps` with metadata + tenant + ACL; embeddings land in a follow-up release.
BatchStore + LlmCallbackProcessor	Shipped. Anthropic Message Batches and OpenAI Batch, async webhook consumer, idempotent dispatch, retry-aware.
EvalRunStore	Shipped. Eval runs are first-class objects with trace ids and prompt-version pinning.
PromptTemplateStore	Shipped. Versioned prompt registry, referenced from DSL with `#name`.
Sliding-window memory	Not shipped. Different bet placed instead: tree-branching conversations via REDB-tree (more on this below).
Sandboxed tools (per-call container)	Not shipped in container form. Shipped `redb.Route.Exec` with allowlist + working-dir + timeout + byte-cap — the practical enterprise minimum.

Bonus, not in the skip-list:

Per-exchange ?redb=<name> hint — pick a named REDB instance per Exchange, so one route serves N tenants without N route registrations or N IServiceScopes.
DSL/tool package split — redb.Route.Llm.Abstractions (contracts only) and redb.Route.Llm.Tools (six utility tools), so 22 connectors can register .AsLlmTool() without bumping a minor on every change.
Six utility tools out of the box: HttpFetch, JsonPath, XPath, RegexExtract, MathEval, TavilyWebSearch — each as both a DSL extension and a standalone tool route.
Bug fixes that only show up in real traffic: orphan tool_use (model asks for a tool, provider 5xxs mid-turn, leaves the conversation in an invalid state) and OEM codepage in Process.StandardOutput (Windows cmd /c returns cp866 bytes, breaks the UTF-8 contract every tool implicitly assumes).

Eleven REDB schemas now sit behind all of this: ConversationProps, MessageProps, ApprovalProps, CostBudgetProps, ToolCacheProps, ToolAuditProps, KnowledgeChunkProps, PromptTemplateProps, EvalRunProps, LlmBatchProps, ToolIdempotencyProps. That's not a database for your chat history. That's the operational layer of an agent platform, opt-in via one line: AddRedbLlmStorage().

Ten enterprise patterns — each one a single DSL line

Here's where it gets useful. None of these are pseudocode. They all run today.

1. Hard budget per conversation as a circuit breaker

In a side project, a "token budget" is a log line. In production, it's a safety primitive — your model must not be allowed to burn $10k because of a bad prompt. CostBudgetProps isn't an observability artifact; it's a preventive rule. Every provider call, the agent engine sums spend by conversation id and adds the worst-case for the pending request (via max_tokens). If the total would breach the cap, LlmBudgetExceededException fires before the network call, not after.

From("kafka://support-tickets")
    .To(Llm.Factory("claude")
        .Conversation(e => "ticket-" + e.In.Headers["TicketId"])
        .CostBudget(usd: 0.50)            // hard ceiling per conversation id
        .CostBudgetExceeded(BudgetPolicy.FailFast)
        .AsUri())
    .To("kafka://support-replies");

The same primitive scales to "budget per tenant", "budget per prompt template", "budget per model" — they're all just different keys in CostBudgetProps. Same shape, different GROUP BY.

2. Approval gates with a human in the loop

The most under-appreciated production pattern in agent-land. If your model can refund payments or delete records, a human stands between the tool call and the actual side effect. Not a webhook duct-taped on later — a native runtime concept.

From("direct:tool-payments-refund")
    .AsLlmTool("issue_refund")
        .Description("Refund a payment by id and reason.")
        .Input(/* JSON Schema */)
        .SideEffect(ToolSideEffect.Mutating)        // governance hook reads this
        .Cost(ToolCostClass.Expensive)
    .Then()
    .Process<ApprovalGate>()                        // suspend exchange,
                                                     // write ApprovalProps,
                                                     // notify Slack,
                                                     // wait for HTTP callback
    .To("kafka://payments.refund.commands");

ApprovalProps records the exchange id, conversation id, input arguments, the human approver, the wait deadline, and the outcome. The Slack bot, the email handler, the web form — all of those plug in as ordinary HTTP routes in the same redb.Route. When the human clicks Approve, the webhook wakes the suspended Exchange and feeds the result back to the agent engine. On timeout, the agent receives a tool_result with status:"timeout" and decides what to do next.

The point: this is not AI logic. It's the EIP Aggregator + Correlation Identifier + Wire-Tap + Reply-To pattern, which already applies to a refund engine without an LLM. They're both Exchanges in the same runtime, so the pattern is reused.

3. Idempotent tool retries via `ToolIdempotencyProps`

Webhook consumers retry. Network timeouts retry. Anthropic's batch sometimes delivers a payload twice. If your tool call "issue refund $50" runs twice, that's a bad day at standup.

ToolIdempotencyProps keeps idempotency-key → tool-result with a TTL. In the DSL:

From("direct:tool-issue-invoice")
    .AsLlmTool("issue_invoice")
        .Caching(ToolCachingPolicy.Idempotent)
    .Then()
    .Process(BuildIdempotencyKey)                    // sha256(args + customer-id + day)
    .To("...");

When the agent engine sees Caching = Idempotent, it consults ToolIdempotencyProps before the route runs. Hit → returns the saved tool_result, route never executes. Miss → route runs, result is recorded. Framework-level, not "remember to wrap your handler".

4. Multi-tenant via `?redb=<name>`

One worker, fifty customers. Each customer is their own REDB instance — their conversations, their cost budgets, their approvals, their knowledge base. In 3.1.0 this would have meant fifty IServiceScopes or fifty route registrations. In 3.1.1 it's a per-exchange hint:

From("http:0.0.0.0:5088/api/llm/ask?inOut=true")
    .Process(e =>
        e.In.Headers[LlmHeaders.Redb] = e.In.Headers["X-Tenant"]?.ToString())
    .To("llm://claude?conversationFromHeader=true");

?redb=acme or a header — the engine pulls the named REDB instance from the registry, on the current Exchange, no factory, no route swap, no restart. Conversations, audit, approvals all land in the right tenant. Each tenant gets its own billing surface and governance lane, with no awareness of the others.

This was the single feature most worth the late nights. It's the difference between "we have a multi-tenant agent platform" and "we have a multi-tenant deployment story, here's a wiki page".

5. Audit trail without a separate integration

ToolAuditProps is a REDB object: tenant id, conversation id, exchange id, tool name, input args (or hash, if PII), output (or hash), duration, status. Every tool invocation is recorded automatically — because tools are routes, and redb.Route already has post-processors.

The query "show me every Claude tool call in tenant acme over the last 7 days, side-effect=mutating, cost=expensive, ordered by time" isn't a separate analytics pipeline. It's a value_string index plus one SQL — the same indexed-business-id pattern the rest of the REDB ecosystem uses.

6. Async batch + callback consumer

Anthropic Message Batches and OpenAI Batch are up to 50% cheaper, with a price tag of up to 24 hours of latency. For offline workloads — classifying a million tickets, extracting fields from a million PDFs — that's the right knob to turn.

LlmBatchProps stores the batch id, statuses, and the link back to the originating Exchange collection. LlmCallbackProcessor is just an HTTP route that the provider calls when the batch finishes. The route reads the batch id, fetches the results, dispatches each one back to its originating Exchange via correlation id — and those Exchanges resume their journey as if the synchronous call had just finished, twenty-four hours later.

On top of this: idempotency (same ToolIdempotencyProps), retry (the standard redb.Route circuit breaker), backpressure (Kafka or queue downstream). No batch dispatcher to write. No webhook handler to write. No "what if the callback arrives twice" to test separately.

7. Versioned prompt registry: `#`-refs

The single most depressing prod bug pattern: "the model is answering differently". You go check git blame, somebody touched the system prompt three weeks ago, the tests passed because the eval set is small. The prompt is code, and it deserves a registry with versions.

// where prompts get registered:
promptRegistry.Register("triage-system", version: "v3", body: """
    You are a support triage agent. Classify into [billing, tech, sales, abuse]...
    """);

// in the route:
.To(Llm.Factory("claude").SystemPromptRef("#triage-system@v3").AsUri())

PromptTemplateProps is a REDB object: name, version, body, metadata (author, date, experiment id). When the engine resolves #name, it pins exactly that version into MessageProps for the call. Six months later you can say with certainty "this conversation ran on triage-system v3", not "probably v3, that's what we were doing then".

The juicy part: EvalRunProps records eval runs bound to a prompt version. "v4 gives +12% accuracy on the golden set" stops being a spreadsheet and becomes a query.

8. Tree-branching conversations for A/B and counterfactuals

ConversationProps is stored as a REDB tree via the native parent_id. That means a conversation isn't a flat list of messages — it's a tree. So you can:

branch from any message and run an alternative continuation with a different model or temperature;
keep a user branch and an experiment branch in parallel;
compute metrics across pairs of branches ("with-tools vs without-tools on the exact same context").

Sliding-window memory truncates the past. Tree-branching memory writes alternative pasts and compares them. For production agents, the second one is an order of magnitude more useful, because improving prompts in production is exactly that loop: take a real conversation, replay with a new prompt, score it, ship the winner.

And nothing here was bolted on for LLMs. Tree via parent_id has been a first-class REDB feature from day one — for product hierarchies, org charts, file systems. The LLM stack just wrote its own entity on the same primitive.

9. A jury of cheap models with a senior model as arbiter — Scatter-Gather + Aggregator

One of the most underrated production tricks in agent-land: don't trust a single model. Send the same task to several cheap models in parallel (Haiku, GPT-4o-mini, Mistral-Small, Gemini-Flash, Llama-3.1-70b on Groq), gather their answers, and hand the original prompt plus all five candidates to a senior model (Sonnet, Opus, GPT-4o) acting as arbiter — pick the best, synthesise a new one, or say "nobody nailed it, ask the human". The literature calls this mixture-of-agents or ensemble voting, and on hard tasks it routinely buys you +10 to +20% accuracy at a lower bill than running everything through Opus.

In a vanilla LangChain shape, this becomes a hundred-line orchestrator with try/catch, per-model timeouts, retry knobs, and a hand-rolled aggregator. In an ESB shape, it's the bog-standard EIP Scatter-Gather + Aggregator, with twenty-five years of hardening in integration buses. The LLM is just one more endpoint type the Scatter-Gather fans out to:

From("kafka://contract-clauses-to-classify")
    .RouteId("contract-jury")
    .Multicast()                                          // Scatter
        .Parallel()
        .StopOnException(false)                           // one provider down? carry on
        .Timeout(TimeSpan.FromSeconds(30))                // per branch
        .To(Llm.Factory("haiku")
            .SystemPromptRef("#contract-classify@v3")
            .Temperature(0.0).MaxTokens(200).AsUri())
        .To(Llm.Factory("gpt-4o-mini")
            .SystemPromptRef("#contract-classify@v3")
            .Temperature(0.0).MaxTokens(200).AsUri())
        .To(Llm.Factory("groq-llama-70b")
            .SystemPromptRef("#contract-classify@v3")
            .Temperature(0.0).MaxTokens(200).AsUri())
        .To(Llm.Factory("mistral-small")
            .SystemPromptRef("#contract-classify@v3")
            .Temperature(0.0).MaxTokens(200).AsUri())
    .End()                                                // Gather: results land in Exchange.Properties["multicast.results"]
    .Process<JuryAggregator>()                            // splice candidates into one arbiter prompt
    .To(Llm.Factory("sonnet")                             // Arbiter
        .SystemPromptRef("#jury-arbiter@v2")
        .Temperature(0.1).MaxTokens(500)
        .CostBudget(usd: 0.05)
        .AsUri())
    .To("kafka://contract-clauses-classified");

JuryAggregator is a tiny processor: takes the four answers and the original prompt, builds one arbiter message of the shape "here is the task; here are candidates A/B/C/D; return the final classification or say unclear". The arbiter answers with structured JSON. Headers llm.tokens.in/out are recorded per branch plus once for the arbiter — cost is transparent at the route level.

Why ESB-shape makes this so cheap to express:

Parallelism for free. Multicast EIP already knows how to fan out across N branches, await all-or-N-of-M, handle timeouts and partial failures. No Task.WhenAll with hand-written failure handling.
Idempotency for free. A retry on a failed branch hits the same ToolIdempotencyProps (or provider-side prompt-hash cache) and dedups. No double billing.
Budget for free. .CostBudget(usd: 0.05) on the arbiter is its own circuit-breaker; per-branch .CostBudget(usd: 0.01) caps the cheap models. A combined ceiling is a different CostBudgetProps key. Breach it and the whole jury fails fast — failover to a rule-based classifier (which sits as another branch on the route).
Audit for free. Each of the five LLM calls is captured in ToolAuditProps, all stitched to one exchange id. A month later you can answer "model X agrees with the arbiter 73% of the time — let's drop it and save".
Eval for free. EvalRunProps records the run with five branches plus the arbiter — replay against a golden set to find the optimal jury composition (swap GPT-4o-mini for Gemini-Flash, see if accuracy holds).

In production this pattern wins on two axes. Accuracy: on tasks like "is this contract clause a risk", where a single model wobbles, four cheap models plus a Sonnet arbiter routinely outperforms a lone Opus, at 2.5× lower cost. Robustness: when Anthropic is having a bad afternoon, the Anthropic branch fails, the other four return, the arbiter receives four candidates instead of five and ships a verdict. Graceful degradation, not a hard outage.

One subtle production gotcha: never tell the arbiter who wrote which answer. If the arbiter's prompt says "candidate from Claude Haiku, candidate from GPT-4o-mini", the arbiter develops favourites (so do humans). So JuryAggregator anonymises the candidates as A/B/C/D, shuffles their order (Latin-squared on exchange-id for reproducibility), and only after the arbiter responds do we map back to "A was the Haiku answer". Clean signal for the post-hoc analysis "which model agrees with the arbiter most often".

10. Sub-agents — an agent as another agent's tool

A direct consequence of the "tool = route" architecture: if a route can contain .To("llm://..."), then a tool can itself be an agent. The parent agent doesn't "know" there's another LLM hiding behind that tool. To it, research_topic is just another tool, like web_search or math_eval. Inside the tool, though, lives a fully-fledged second-tier agent with its own model, prompt, toolset, budget, iteration cap, retry policies, RAG sources.

In code:

// Sub-agent: a research specialist with its own toolset
From("direct:research-subagent")
    .AsLlmTool("research_topic")
        .Description("Deep research on a topic. Takes {topic, depth}. " +
                     "Returns a structured summary with sources.")
        .Input("""{"type":"object","properties":{
                    "topic":{"type":"string"},
                    "depth":{"type":"string","enum":["short","deep"]}},
                  "required":["topic"]}""")
        .SideEffect(ToolSideEffect.ReadOnly)
        .Cost(ToolCostClass.Expensive)              // parent sees the call is pricey
    .Then()
    .Knowledge("research-corpus", k: 12)            // sub-agent has its own RAG corpus
    .To(Llm.Factory("sonnet")                       // mid-tier model — research specialist
        .SystemPromptRef("#research-specialist@v3")
        .Tools("tavily_web_search", "http_fetch", "regex_extract")
        .MaxIterations(8)
        .CostBudget(usd: 0.30)                      // sub-agent has its own budget
        .Temperature(0.1).AsUri())
    .Process<ExtractResearchSummary>();

// Parent agent uses the sub-agent as just another tool
From("kafka://complex-business-questions")
    .To(Llm.Factory("opus")                         // planner — top-tier model
        .SystemPromptRef("#senior-analyst@v1")
        .Tools("research_topic",                    // ← our sub-agent
               "sql_query",                         // ← plain data tool
               "math_eval",                         // ← computation
               "draft_report")                      // ← another sub-agent (report drafter)
        .MaxIterations(15)
        .CostBudget(usd: 2.00)                      // top-level budget
        .AsUri())
    .To("kafka://business-answers");

What actually happens under the hood: when Opus decides to call research_topic, the agent engine builds the JSON input and routes it through RouteToolBridge to direct:research-subagent. That route runs as a fresh Exchange, inheriting transaction scope, principal, headers, and DI scope from the parent. Inside it, Sonnet runs its own tool-use loop (web search → fetch → extract), potentially with eight iterations and three tools. It returns a structured summary in Out.Body, which the engine repackages as a tool_result and hands back to Opus. Opus sees the result, keeps planning, calls more tools as needed.

In other words: the architecture is recursive without a single line of dedicated code. A sub-agent inside a sub-agent works the same way. At depth three, audit, budget, idempotency, prompt versioning, RAG, governance — all of it still works, because none of those are tied to "agent level"; they're tied to the Exchange, and the Exchange is the same primitive at any depth.

This unlocks three derivative patterns that are awkward to assemble in the flat "one agent, many tools" shape:

(a) Hierarchical agents — planner and workers. A top-tier model (Opus, GPT-4o) plays planner: decomposes the task and dispatches chunks to specialists. Each specialist is a sub-agent with a narrow prompt and a narrow toolset. The planner may have no direct data access at all — only through sub-agents. That gives you hard separation of authority: the planner can't accidentally call delete_records because that tool isn't in its set; only data-cleanup-subagent has it, and the planner has to hand off explicitly.

(b) Specialist sub-agents with their own memory. A sub-agent can have its own conversationId (a different branch of the REDB tree), its own Knowledge(...) sources, its own prompt registry. A legal-review-subagent lives inside the corporate legal corpus, sees only that, answers strictly in the legal register. The senior agent never gets direct access to those documents. ACL is enforced at the route level, not at the "we hope the model doesn't quote it" level.

(c) Cost-shaped escalation. A cheap agent (Haiku) takes the first swing. If it returns unclear or confidence is below threshold, it calls escalate_to_senior itself as a tool — and behind that tool sits a route to Opus with full context. Most queries land on Haiku for pennies; only the hard ones reach Opus. On high-volume workloads the economics change by an order of magnitude.

How this relates to the static jury in pattern #9: the jury is a statically-wired sub-agent pattern. The route knows up front there are N candidates and one arbiter; the DAG is hard-coded in the DSL. Sub-agents-as-tools is the dynamic version: the parent agent decides for itself whom to call and how often. Jury wins on predictable classification/forecasting pipelines. Sub-agents win on open-ended research where the number of steps and the toolset can't be known in advance.

The obvious failure mode is cycles. If sub-agent A can call B and B can call A, you've got an unbounded recursion in theory. In practice three guards keep it bounded: MaxIterations at every level (no sub-agent loops forever), CostBudgetProps (each nested call burns parent budget), and an optional depth limit in headers (LlmHeaders.SubAgentDepth increments per level; the route rejects calls past a configured limit). Real tasks rarely benefit beyond depth = 3-4.

And one more piece of architectural elegance: a sub-agent is just a route, which means it has a stock URI. Different parent agents can share one research-subagent, which has its own ?redb=acme for tenant isolation, its own rate limiter (?throttle=5/sec), its own circuit breaker. The sub-agent behaves like an internal LLM-tier microservice that you reuse across routes without duplicating prompts or proliferating clients.

Streaming: what actually changes when a token leaves the provider

In 3.1.0, providers streamed but the client never saw it — we accumulated tokens into a string and returned the whole thing. In 3.1.1, the chain closes:

the provider yields IAsyncEnumerable<string> — frames as they arrive;
the agent engine sticks that into Out.Body as IAsyncEnumerable<string>, without materialising;
the HTTP consumer sees IAsyncEnumerable<string> and switches to SSE — one data: ...\n\n per frame;
the WS consumer sends one WebSocket message per frame;
non-streaming consumers (Kafka, RabbitMQ, ActiveMQ) materialise the way they used to.

The architectural payoff: streaming stops being a separate mode and becomes a payload type. Same Exchange can fan out to SSE and Kafka simultaneously (multicast EIP) — Kafka waits for materialisation, SSE sees frames live. No "streaming endpoint" vs "regular endpoint" duplication.

Underneath, this is the well-known pipe pattern of "iterator instead of collection". The only LLM-specific bit is making sure async iteration runs inside the end-to-end Exchange tracking so traces and metrics see the whole journey, not "received an Exchange and lost the rest".

RAG: chunks as first-class REDB objects

In 3.1.1, KnowledgeChunkProps is a REDB object with:

the text;
the source (source-uri, tenant-id, doc-id, chunk-index);
metadata (language, date, tags, ACL — who's allowed to see it);
a placeholder for embeddings (vector store ships in a follow-up; the MVP is keyword search via value_string indexes plus FTS).

What this means: a RAG source is a route, not a separate vector service. From("file://docs?include=*.md").To("knowledge://acme") indexes documents. From("kafka://support-tickets").Knowledge("acme", k: 5) injects the top-5 chunks into the system prompt before .To("llm://claude"). ACL and tenant filtering happen as value_*-indexed SQL before chunks reach the prompt.

When the vector store lands, it sits next to keyword search behind the same IKnowledgeStore interface, no route changes. That's the architectural goal: tomorrow's features don't break today's routes.

Three real-world enterprise scenarios — reports, forecasts, alerts

The patterns above are atomic bricks. Now three end-to-end scenarios where those bricks combine into routes you'd actually run on Tuesday morning. No AI hype, no "cognitive automation", no pretending to "transform the industry". Just the dull stuff a human currently does by hand every day, taken over by an agent that lives in the same bus as the data.

Case 1 — A daily financial snapshot for the CFO, in their inbox by 7:00 AM

Every business morning a finance analyst pulls together yesterday's revenue, expenses by category, plan-vs-actual variances, top-5 largest transactions, account balances, FX rates. Then they write a paragraph or two — "revenue +3.2% to plan, expenses +1.7%, the variance is X". An hour and a half, gone. Worth automating? Obvious yes. Worth standing up a separate AI service for it? Hard no.

The whole route:

// Every weekday at 07:00 local
From("cron://daily-cfo-report?schedule=0 0 7 ? * MON-FRI")
    .RouteId("daily-cfo-report")
    .Process<LoadYesterdayMetrics>()                       // pulls from ERP/bank API/data warehouse
                                                            // → e.In.Body = {revenue, expenses, accounts, fx, top5}
    .Process<LoadQuarterContext>()                         // same ETL: plan, prior quarter, MTD/QTD
    .ConvertBody<FinancialDailySnapshot>()                 // strongly-typed payload
    .Multicast().Parallel()
        .To(Llm.Factory("haiku")                           // branch A: short-form for the CFO
            .SystemPromptRef("#cfo-daily-summary@v7")
            .Temperature(0.1).MaxTokens(800)
            .CostBudget(usd: 0.02).AsUri())
        .To(Llm.Factory("haiku")                           // branch B: bulletised for the board chat
            .SystemPromptRef("#cfo-daily-bullets@v7")
            .Temperature(0.1).MaxTokens(400)
            .CostBudget(usd: 0.02).AsUri())
    .End()
    .Process<RenderHtmlReport>()                           // mustache template: numbers as a table,
                                                            // model summaries as the lede
    .To("smtp://mail.acme.com?to=cfo@acme.com,board@acme.com" +
        "&subject=Daily%20FY%20snapshot%20${date:yyyy-MM-dd}")
    .To("teams://board-channel?card=adaptive")             // same HTML body → adaptive card in Teams
    .Wiretap("kafka://reports.cfo-daily.archive");         // copy to archive for audit and training

Seven steps, and those seven steps cover the whole job:

Schedule — cron lives in the URI; no separate scheduler service. redb.Route already does this.
Data pickup — LoadYesterdayMetrics is just a processor that hits your existing integrations through the same redb.Route (Kafka, REST, JDBC). Whatever retries and circuit breakers you already wired around those integrations apply.
Two-tone output — multicast to two branches of cheap Haiku with different prompt templates. Pennies, hard-budgeted.
Versioned prompt — #cfo-daily-summary@v7. Six months from now the CFO complains "the report has gotten worse" — open EvalRunProps, see v7 regressed against the golden set, roll back to v6 with no redeploy.
Delivery — two transports (SMTP + Teams), both stock redb.Route connectors. Tomorrow they want it in Slack too: .To("slack://..."), one line.
Audit — Wiretap copies the whole exchange to a Kafka archive. Twelve months later a regulator asks "what did we send on 2026-04-15?" — pulled from the archive with metadata that says "model X, prompt version Y, source data Z" (because MessageProps records both input and output).
Budget — .CostBudget(usd: 0.02) per branch. 250 trading days × 2 branches × $0.02 ≈ $10/year on token spend. The analyst's hour a day at fully-loaded cost is $50. Pays itself back in week one of month one.

What pushes this from "cute demo" to "actual enterprise asset": when the CFO asks about "that strange number in last Tuesday's report", the auditor pulls one trace id and in 30 seconds has: the snapshot at input, the prompt version that ran, the exact output the model produced, the file that hit SMTP, who opened it in Teams. Not "let me get back to you tomorrow". Right now — because it's all REDB objects pinned to one exchange id.

Case 2 — A weekly cash-flow forecast with a jury and an Opus arbiter

Cash-flow forecasting is the textbook case where one model is worse than zero models: a confident-sounding model error leads to a decision that costs more than the analyst's salary for the month. Pattern #9 (jury + arbiter) earns its keep here.

From("cron://weekly-cashflow-forecast?schedule=0 0 9 ? * MON")
    .RouteId("weekly-cashflow-forecast")
    .Process<BuildCashflowFeatures>()                      // bank balances, AR/AP, scheduled payments,
                                                            // seasonality, FX exposure
    .Knowledge("acme-finance", k: 8)                       // RAG: forecasting playbook, prior reports,
                                                            // methodology, known seasonality notes
    .Multicast().Parallel().Timeout(TimeSpan.FromMinutes(2))
        .To(Llm.Factory("sonnet")                          // four different models — cheap insurance
            .SystemPromptRef("#cashflow-forecast@v4")      // against correlated errors
            .Temperature(0.2).MaxTokens(2000).AsUri())
        .To(Llm.Factory("gpt-4o")
            .SystemPromptRef("#cashflow-forecast@v4")
            .Temperature(0.2).MaxTokens(2000).AsUri())
        .To(Llm.Factory("gemini-pro")
            .SystemPromptRef("#cashflow-forecast@v4")
            .Temperature(0.2).MaxTokens(2000).AsUri())
        .To(Llm.Factory("mistral-large")
            .SystemPromptRef("#cashflow-forecast@v4")
            .Temperature(0.2).MaxTokens(2000).AsUri())
    .End()
    .Process<JuryAggregator>()                             // anonymise A/B/C/D + shuffle
    .To(Llm.Factory("opus")                                // arbiter — strongest model in the rotation
        .SystemPromptRef("#cashflow-arbiter@v2")
        .Temperature(0.1).MaxTokens(3000)
        .CostBudget(usd: 0.50).AsUri())
    .Process<ExtractStructuredForecast>()                  // parse JSON: 30 days × {low, mid, high}
    .Choice()
        .When(e => e.In.Body<Forecast>().ConfidenceLow < 0.6)
            .Process<ApprovalGate>()                       // low confidence → wait for CFO sign-off
            .To("smtp://...?to=cfo@acme.com&priority=high")
        .Otherwise()
            .Process<RenderForecastReport>()
            .To("smtp://...?to=treasury@acme.com")
    .End()
    .Wiretap("kafka://reports.cashflow.archive");

This route runs about $2-3 per execution, once a week — call it $150/year. The analyst would spend a day producing the same artefact. The savings aren't really the point. The point is robustness: four independent forecasts from four different vendors (Anthropic, OpenAI, Google, Mistral) give you consensus as a confidence signal. Four models agree → the arbiter just codifies. Four models diverge → the arbiter writes "model A sees risk X, the rest don't, recommend human review", and the Choice branch routes the report to an approval gate without anyone wiring anything custom.

A year in, the treasury team queries EvalRunProps: "Gemini-Pro disagrees with consensus 18% of the time, and in 70% of those cases Gemini was right". Excellent — it gets upweighted in the jury, not dropped. That's not a guess; it's a SQL query.

Case 3 — A predictive plant alert: XGBoost does the math, the LLM does the sentence

In manufacturing — the real kind, with conveyor belts and SCADA tags, not the metaphorical kind — the bread-and-butter task is "from the metrics, predict that this line will throw a fault in 4-6 hours, and ping the on-shift engineer before it does". The classical play here is a feature store and a gradient-boosted model. That play is still correct, and the LLM is not there to replace it. What the LLM does well is the last-mile translation from "anomaly score + top features + line context" into a sentence a human reads at 3 AM and understands.

// Tap the SCADA bus (MQTT/Kafka), 5-minute aggregation window
From("kafka://scada.metrics?groupId=plant-anomaly-watcher")
    .RouteId("plant-anomaly-watcher")
    .Aggregate(by: e => e.In.Headers["LineId"],
               window: TimeSpan.FromMinutes(5))            // window per production line
    .Process<RunAnomalyModel>()                            // classical ML — XGBoost, nothing fancy;
                                                            // emits {score, top-features, line-context}
    .Choice()
        .When(e => e.In.Body<AnomalyReport>().Score > 0.8)
            .Knowledge("plant-runbooks", k: 3)             // RAG: this line's runbooks,
                                                            // history of similar incidents
            .To(Llm.Factory("haiku")
                .SystemPromptRef("#plant-incident-explainer@v9")
                .Temperature(0.0).MaxTokens(600)
                .Tools("metrics-history", "fetch-shift-log")  // agent fetches more if it needs to
                .MaxIterations(4)
                .CostBudget(usd: 0.10).AsUri())
            .Process<EnrichWithOnDutyEngineer>()           // who's on shift on this line, right now
            .Multicast()
                .To("teams://engineering-shift-{LineId}?card=adaptive")
                .To("sms://twilio?to={engineer.phone}")
                .To("kafka://incidents.predicted.archive")
            .End()
        .Otherwise()
            .To("kafka://scada.metrics.normal")            // archive the boring days too
    .End();

What's actually happening, and why this works:

The ML model isn't getting kicked out. Anomaly scoring is still XGBoost, which has been good at exactly this for a decade. The LLM stands after it.
The LLM explains, not predicts. The system prompt is "anomaly on Line 7, top-3 features look like X, runbook says Y, prior incidents Z. In one page: what likely happened, what the engineer should do in the next hour, and the three parameters to check first". This is synthesis — the place LLMs are objectively strong.
The agent's tools are the plant's data. metrics-history is a direct:metrics-history route that queries Influx/Prometheus. fetch-shift-log reads the shift log from REDB. The agent decides whether to invoke them, how many times. MaxIterations(4) caps it from looping.
Multi-channel delivery. Teams adaptive card with an "I've got it" button (which, by the way, fires an ApprovalGate-style webhook), SMS to the on-shift number via Twilio (already a redb.Route transport), copy to a Kafka archive.
Aggregate by LineId, window 5min is the EIP Aggregator — bucket metrics by group. Nothing redb.Route.Llm-specific; this is plain integration code that's been working in your bus for years.

The business outcome: time from "anomaly appears" to "the engineer on the line knows what to look for" drops from ~40 minutes (operator notices, calls, describes, engineer arrives, engineer starts hunting) to ~4-5 minutes. On expensive lines, an hour of unplanned downtime is tens of thousands of dollars; this route pays for itself the first time a near-miss is caught.

Notice the LLM's role is narrow in all three cases. Not "AI runs the factory". Not "AI runs the books". The LLM does one specific job in each route — summarise, synthesise, pick from candidates, explain. Around it sits all the boring enterprise plumbing: schedules, ETL, RAG, multi-channel delivery, audit, governance, budgets. That plumbing is redb.Route. The LLM is the last brick that previously demanded its own service.

Storytelling: how a chat demo becomes a platform

Here's the timeline I've now watched several times in different teams.

Day 1. Somebody shows the team that Claude can answer a support ticket. LlmHttpRoutes.cs — six lines, From("http://...").To("llm://haiku"), demo works. This is a project, not a platform.

From("http:0.0.0.0:5088/api/llm/ask?inOut=true")
    .ConvertBody<string>()
    .To(Llm.Factory("haiku").MaxIterations(1).AsUri());

Day 7. Coworkers say "it doesn't remember what I just told it". A Process(...) step adds an X-Chat-Id header, ConversationFromHeader() is flipped on. That's it — AddRedbLlmStorage() is already in Program.cs, MessageProps are already being written.

.Process(e => e.In.Headers[LlmHeaders.ConversationId] =
    e.In.Headers["X-Chat-Id"]?.ToString() ?? "default")

Day 14. "Can it run a command on the server?" A tool-shell route appears — a separate route with .AsLlmTool("shell"), backed by redb.Route.Exec, allowlist [cmd, pwsh], working dir in temp, timeout 5 sec, 8 KB stdout cap. Safety lives in the DSL, not in advice in a system prompt.

From("direct:tool-shell")
    .AsLlmTool("shell").SideEffect(ToolSideEffect.ReadOnly).Cost(ToolCostClass.Cheap)
    .Then()
    .To(ExecDsl.Run().AllowedCommands("pwsh", "cmd").TimeoutMs(5000).MaxStdoutBytes(8192));

Day 21. "Finance says the token bill is 4x plan." Add .CostBudget(usd: 0.50) per conversation. No new services, no new dashboards — CostBudgetProps is an existing REDB object, the tsak.web dashboard already knows how to render it.

Day 30. "We need every agent action to be auditable." Already done: ToolAuditProps started writing the day tools became routes. The auditor opens a SQL query — every call this quarter, filtered by tenant and side effect. No "let's set up an integration".

Day 45. "Legal wants human-in-loop on the refund tool." Add .Process<ApprovalGate>() before exec. ApprovalProps writes, the Slack bot is just another HTTP route. Done.

Day 60. "We're onboarding a second customer — separate database, separate billing." Add Process(e => e.In.Headers[LlmHeaders.Redb] = e.In.Headers["X-Tenant"]). One route, two tenants, zero changes to logic.

Day 90. "Run a million tickets through the classifier overnight." The route is already there. Change one knob: .Mode(LlmMode.Batch) — a synchronous call becomes an Anthropic Batch, LlmBatchProps records the id, LlmCallbackProcessor waits for the webhook, results land in the same Kafka the live route uses during the day. -50% on the bill.

Day 120. "Which prompt versions regressed?" EvalRunProps already stores runs. SQL across PromptTemplateRef, accuracy diff, v4 is bad, roll back to v3. No redeploy.

There's no "and then we rewrote it on a Kubernetes operator" beat in this story. There's no "and then we added an AI platform" beat. The six lines that worked as a chat demo on Day 1 are literally the same file that on Day 120 is running a multi-tenant, jury-arbitrated, batch-classifying, audit-grade platform. No migration, no rewrite, no architecture v2 — just lines accreting into the route as the requirements showed up. The chat demo doesn't grow into a platform — it was standing on a platform from day one, just opting into more of its features over time.

This is the pitch I couldn't quite articulate at announcement time. Now I can. The deliverable here isn't an agent framework. It's a runway. You walk along the runway and you're already platform-shaped.

Demo routes — see it live

The patterns above aren't concept art. The repo redbase-app/redb-route ships two demo files:

LlmDemoRoutes.cs — three shapes of the LLM call: inline step (.Llm("demo-stub")), endpoint (.To("llm://demo-stub")), tool (.AsLlmTool("echo_tool")). All on the stub provider, no API keys.
LlmHttpRoutes.cs — two HTTP endpoints (/api/llm/ask no tools, /api/llm/shell with the shell tool through redb.Route.Exec), both passing X-Chat-Id for conversation memory, both running real Claude Haiku.

dotnet run from redb.Route.Demo, port 5088 opens, and curl works:

curl -d "what time is it on this host?" -H "X-Chat-Id: test1" \
     http://localhost:5088/api/llm/shell

Repo: github.com/redbase-app/redb-route (Apache 2.0).

What's not done, and why I keep saying so

Sliding-window memory isn't done. A real vector store isn't done. There's no AI-graph editor in tsak.web — runtime conversation inspection shows up as plain REDB objects in the existing UI, no special pane. There's no built-in eval-service integration — EvalRunStore records the runs, but "click compare against prod" is manual.

That's fine. The skip-list is a technique, not an apology. An open-source project that lies in its README about scope is one you don't come back to. I'd rather say "not yet" and ship it next minor than say "shipped" and field issues for two months.

Zooming out: why the ESB shape matters

The phrase "AI-native architecture" is fashionable. What it usually means is "we built everything around the LLM" — and what usually backs that up is a new dev stack running parallel to the old one. That's not an architecture decision. That's a policy duplication problem.

If you have retry in two stacks, audit in two stacks, governance in two stacks, tenant isolation in two stacks — those two implementations drift. A year later, the AI stack adds bounded-context auditing, the integration stack doesn't. Legal asks for one report — nobody can produce it. The duplication wasn't visible on day one; it's load-bearing on day three hundred.

The ESB shape is a deliberate choice of one control point for I/O of all kinds. The LLM is a kind of I/O — asynchronous, with tools, with context, but I/O. Putting it inside the ESB isn't a philosophical pose; it's engineering economy: one governance policy covers everything.

This is not new. Garland and Ripley wrote it up in the SOA literature of the 2000s. What's new is that in 2026 the argument finally applies to LLMs, because LLMs grew up enough to expose standard interfaces: tool use, streaming, batch APIs, embeddings. Before that, "LLM in the ESB" meant "wrap a REST call and pray". Now it means "use existing EIP patterns with minor adaptations".

That economy is the entire pitch of redb.Route.Llm. Not "we have the best agent engine." Microsoft and LangChain have better agent engines. We have a perfectly average agent engine in the right place in the architecture. I'll take that trade.

Roadmap

3.1.2:

sliding-window memory as a built-in policy;
a vector-store interface behind IKnowledgeStore;
pgvector and Qdrant as the first back-ends;
an EvalCompare DSL for side-by-side prompt-version runs.

3.2:

a ConversationProps tree UI in tsak.web;
streaming-aware aggregator EIP (buffer partial frames to semantic boundaries);
distributed batch — multiple workers behind one LlmCallbackProcessor.

Later:

multi-modal (image input/output as a payload type);
voice agents as another transport (voice://...);
routing by cost / latency / accuracy SLA per message.

Links

Part 1 (3.1.0 announcement): redb.Route 3.1.0 — LLM as just another connector
redb.Route — Apache Camel for .NET
EIP series part 1 — channels and Exchange
redb.Route 3.0.1 patch notes
GitHub: github.com/redbase-app/redb-route (Apache 2.0)
Demo routes: redb.Route/demos/redb.Route.Demo/Routes/LlmDemoRoutes.cs, LlmHttpRoutes.cs
Habr companion (Russian, not a translation): link added after publication

TL;DR. Three weeks ago I shipped 3.1.0 with an honest skip-list. Three weeks later, most of it is shipped: streaming end-to-end, tool cache, RAG knowledge store, async batch + callback consumer, eval-run store, versioned prompt registry, multi-tenant ?redb=<name>, idempotent tool retries, human-in-loop approval gates, full audit. On top, two patterns that ship the architecture's real punchline: a jury of cheap models with a senior model as arbiter, dropping straight onto stock Scatter-Gather + Aggregator EIP — no custom orchestrator; and sub-agents as tools — recursive agent composition where one agent's tool is itself an agent with its own model, prompt, budget, RAG corpus, all working out of the box because RouteToolBridge → direct: → llm:// is a fixed point. Plus three honest enterprise scenarios with code: a daily CFO snapshot to inbox, a weekly cash-flow forecast with jury arbitration, and a predictive plant alert that hands XGBoost the prediction and the LLM the sentence. The point isn't the feature checklist. The point is that enterprise-grade properties land on the day you actually need them, with no rewrite, because the LLM lives inside the ESB along with every other I/O. Those six lines you typed for the Day-1 chat demo are literally the same file that's running the Day-90 multi-tenant audit-grade platform. There's no "now we rewrite" moment in this story.

redb.Route 3.1.0 — LLM(AI) as just another connector: `.To("llm://claude")` and tools-as-routes

rinat kozin — Tue, 09 Jun 2026 14:19:01 +0000

Series: redb ecosystem (announcement; deep-dive separate)

redb.Route 3.1.0 just shipped two new transports: redb.Route.Llm (the 24th) and redb.Route.Exec (the 25th). The LLM is now an addressable endpoint the same way Kafka, RabbitMQ and HTTP are: calling a model is .To("llm://claude"), an agent tool is a route with .AsLlmTool("shell") on it, a scheduled agent is From("llm://factory?schedule=5m"). Exec is a process spawner with allowlist, working-directory and timeout — both the backend that powers shell tools for agents and a first-class scheduled consumer in its own right (cron-less health probes, backups, deploy glue). No parallel "AI framework" sitting next to the integration framework — same DSL, same retry/throttle/circuit-breaker/audit, same OpenTelemetry traces, same dashboard endpoint counters.

This post is the announcement. A deep-dive on the agent loop, the governance hooks and the REDB storage internals will follow as a separate piece. Here: what landed, what it looks like in code, and what's honestly not done yet.

First time reading about redb.Route? Short context from earlier posts in the series:

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL — why it exists, and what "Apache Camel for .NET" actually means

RedBase / redb.Route / redb.Tsak 3.0.0 shipped — the 3.0.0 release that this LLM connector slots into

Enterprise Integration Patterns in .NET — Part 1: the four in-memory channels (and the Exchange they carry) — how the runtime is built, if you want internals

redb.Route 3.0.1 — flat DSL navigation, CRTP refactor, and a silent null fix — the previous patch right before 3.1.0

The shortest possible explanation

From("kafka://orders")
    .To(Llm.Factory("claude").Temperature(0.2).MaxTokens(1024).AsUri())
    .To("kafka://orders.translated");

One line, one full LLM call:

the inbound message body becomes the user prompt;
the agent runs (provider call → optional tool_use → next provider call → …) until EndTurn or MaxIterations;
the assistant text lands in exchange.Out.Body;
token usage, model id, stop reason, iteration count are pushed into headers;
OpenTelemetry traces and metrics light up automatically;
the endpoint shows up in the tsak.web dashboard with msg/sec, average duration, error rate and last-error info — like every other connector.

That's the whole reason "connector, not library" matters. If you already have an Apache Camel-class ESB with retry, breaker, idempotent consumer and audit, the only honest move is to turn the LLM into one more endpoint inside it. No re-implementing retries. No separate dashboard. No "AI infra" running parallel to the integration infra.

One adapter, 14 OpenAI-compatible APIs

Two production providers ship today:

OpenAiProvider — one universal transport for 14 OpenAI-compatible APIs: openai, anthropic/claude (via Anthropic's official OpenAI-compat endpoint), groq, cerebras, openrouter, gemini (OpenAI-compat surface), github-models, mistral, together, huggingface, deepseek, ollama, lmstudio, plus a generic custom for any self-hosted gateway.
StubProvider — deterministic echo for unit tests and CI runs with no keys.

A native AnthropicProvider (Messages API) is on deck for features the OpenAI-compat surface doesn't expose.

Switching provider is one DI registration:

services.AddLlmConnectionFactory("groq", f =>
{
    f.Provider = "groq";
    f.ModelId  = "llama-3.3-70b-versatile";
    f.ApiKey   = Environment.GetEnvironmentVariable("REDB_LLM_GROQ_KEY");
});

Change provider to "anthropic" and modelId to "claude-haiku-4-5" and the same .To("llm://...") is calling Claude. The DSL surface doesn't move a character.

Tools are routes

The central architectural call: a tool is an ordinary RouteBuilder route with one extra DSL aspect, .AsLlmTool("name"). Four properties fall out of that and would otherwise have to be paid for separately.

From("direct:tool-shell")
    .AsLlmTool("shell")
        .Description("Run a small shell command on the host. Input: {\"command\":\"<name>\",\"args\":[\"...\"]}.")
        .Input("""
            {
              "type":"object",
              "properties":{
                "command":{"type":"string"},
                "args":{"type":"array","items":{"type":"string"}}
              },
              "required":["command"]
            }
            """)
        .SideEffect(ToolSideEffect.ReadOnly)
        .Cost(ToolCostClass.Cheap)
    .Then()
    .To(ExecDsl.Run()
        .AllowedCommands("cmd", "pwsh")
        .WorkingDirectory(scratchDir)
        .TimeoutMs(5_000)
        .MaxStdoutBytes(8_192)
        .MaxStderrBytes(8_192));

What you get for free:

Tools-as-routes. The 30+ EIP processors are usable inside a tool — Splitter, Aggregator, CircuitBreaker, Throttle, Filter, TryCatch. A tool that talks to a fragile API gets wrapped in a breaker and a throttle without a line of runtime glue.
Tools from existing routes. Any From("http://...") or From("sql://...") you already have becomes a tool by adding .AsLlmTool("name"). Its auth, its audit trail, its metrics are already in place — because it's already a route.
Inventory-as-data. IToolDescriptorRegistry knows every tool by name with its JSON schema. Filter with ?tools=* or ?tools=lookup,shell on the URI; build "an agent that has every read-only tool" in one line.
Zero bumps on sibling connectors. The AsLlmTool() aspect lives in redb.Route.Llm.Abstractions. The other 22 connectors can be used as tools today without a NuGet bump on any of them.

A real example — a standalone "curl → Claude → shell → reply" demo

This is a complete program. Two files — Llm.HttpShell.csproj and Program.cs, top-level statements, no host abstractions on top. Drop your Anthropic key into one place and dotnet run brings up an HTTP endpoint on port 5088 where Claude Haiku can call a shell on the host and answer in the context of the previous turns.

csproj — five NuGet packages

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="redb.Route" Version="3.1.0" />
    <PackageReference Include="redb.Route.Http" Version="3.1.0" />
    <PackageReference Include="redb.Route.Llm" Version="3.1.0" />
    <PackageReference Include="redb.Route.Llm.Abstractions" Version="3.1.0" />
    <PackageReference Include="redb.Route.Exec" Version="3.1.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="9.0.0" />
  </ItemGroup>
</Project>

redb.Route is the ESB core, redb.Route.Http is the HTTP transport, redb.Route.Llm[.Abstractions] is the LLM connector and the .AsLlmTool() DSL aspect, redb.Route.Exec is the process spawner with allowlist and timeout. No redb.Core/redb.Postgres — the demo is self-contained and keeps conversation memory in process.

1. The key

const string ApiKey = "<your-key>";

The one you get from https://console.anthropic.com/. Switching to Groq is Provider = "groq", ModelId = "llama-3.3-70b-versatile" and a key from console.groq.com; the DSL below doesn't move.

2. DI and `RouteContext`

RouteContext ctx = null!;

var services = new ServiceCollection();
services.AddLogging(b => b
    .AddSimpleConsole(o => { o.SingleLine = true; o.TimestampFormat = "HH:mm:ss "; })
    .SetMinimumLevel(LogLevel.Information));
services.AddSingleton<IRouteContext>(_ => ctx);
services.AddSingleton<ILogger>(sp => sp.GetRequiredService<ILoggerFactory>().CreateLogger("redb.Route"));

var sp = services.BuildServiceProvider();
ctx = new RouteContext(sp, contextId: "llm-http-shell");
ctx.AddService(typeof(ILoggerFactory), sp.GetRequiredService<ILoggerFactory>());

RouteContext is the runtime container — routes, components, services. The _ => ctx closure registers IRouteContext as a DI service before the context itself is constructed (the context in turn needs the IServiceProvider). The last line hands ILoggerFactory to the context explicitly; without it .Log(...) steps in routes silently turn into no-ops (internals belong to the deep-dive piece).

3. Three components

ctx.AddComponent(new HttpComponent { ServerManager = new SharedHttpServerManager() });
ctx.AddComponent(new LlmComponent());
ctx.AddComponent(new ExecComponent());

A component in redb.Route is a transport plugin owning a URI scheme: http://, llm://, exec://. Pure registration, no business logic.

4. Connection factory for Claude Haiku

ctx.AddToRegistry("haiku", new LlmConnectionFactory
{
    Name        = "haiku",
    Provider    = "anthropic",
    ModelId     = "claude-haiku-4-5",
    ApiKey      = ApiKey,
    Temperature = 0.0,
    MaxTokens   = 512
});

"haiku" is the label the route will reference as Llm.Factory("haiku"). Same trick redb.Route uses framework-wide for named connection factories.

5. Agent engine + tool registry

var toolRegistry = new ToolDescriptorRegistry();
ctx.AddService(typeof(IToolDescriptorRegistry), toolRegistry);

var producerTemplate = new ProducerTemplate(ctx);
ctx.AddService(typeof(IProducerTemplate), producerTemplate);

var engine = new AgentEngine(
    logger:           null,
    producerTemplate: producerTemplate,
    observer:         new NoopAgentObserver(),
    budget:           new NoopBudgetEnforcer(),
    approval:         new AutoApproveGate(),
    redaction:        new NoopRedactionFilter(),
    shadow:           new NoopShadowRunner(),
    conversation:     new InMemoryConversationStore(),
    idempotency:      null,
    approvalStore:    null);
ctx.AddService(typeof(IAgentEngine), engine);

Every agent state surface — observability, budget, approval, redaction, shadow, conversation, idempotency — lives behind an interface. The demo wires Noop/InMemory implementations everywhere: the loop runs but nothing is persisted. Production swap is AddRedbLlmStorage() (covered below): replaces InMemoryConversationStore with RedbConversationStore, attaches RedbAuditObserver, etc.

6a. The HTTP route

var isWindows  = OperatingSystem.IsWindows();
var allowed    = isWindows ? new[] { "cmd", "pwsh", "powershell" } : new[] { "sh", "bash" };
var scratchDir = Path.Combine(Path.GetTempPath(), "redb-llm-shell");
Directory.CreateDirectory(scratchDir);

var systemPrompt =
    "You can run small shell commands through the 'shell' tool. " +
    $"The host is {(isWindows ? "Windows (use cmd /c)" : "Linux (use sh -c)")}. " +
    "Use the tool when the user asks about the system, files, or commands; " +
    "then summarise what you learned in one short sentence.";

ctx.AddRoutes(r =>
{
    r.From("http:0.0.0.0:5088/api/llm/shell?inOut=true")
        .RouteId("llm-http-shell")
        .ConvertBody<string>()
        .Process(e =>
        {
            e.In.Headers[LlmHeaders.SystemPrompt] = systemPrompt;
            e.In.Headers[LlmHeaders.ConversationId] =
                e.In.Headers.TryGetValue("X-Chat-Id", out var v) && v is string s && s.Length > 0
                    ? s : "default";
        })
        .Log("[LLM-SHELL] ▶ chat=${header.llm.conversation.id} prompt=${body}")
        .To(LlmDsl.Factory("haiku")
            .Tools("shell")
            .ConversationFromHeader()
            .MaxIterations(10)
            .Temperature(0.0)
            .AsUri())
        .Log("[LLM-SHELL] ◀ iters=${header.llm.tool.iterations} stop=${header.llm.stop_reason} " +
             "tokensIn=${header.llm.tokens.in} tokensOut=${header.llm.tokens.out}")
        .Log("[LLM-SHELL] ◀ reply=${body}");

The POST body is converted to a string and becomes the user prompt. Process puts the system prompt into the standard LlmHeaders.SystemPrompt header and resolves the conversation id from the client's X-Chat-Id (without it — default). .Tools("shell") is the only LLM-specific knob: "give the agent the tool registered under this name." MaxIterations(10) caps the tool loop (otherwise it's an unbounded ping between model and tool).

6b. The tool itself is a route

    r.From("direct:tool-shell")
        .AsLlmTool("shell")
            .Description(
                "Run a small shell command on the host. Input: " +
                "{\"command\":\"<name>\",\"args\":[\"...\"]}. Output: " +
                "{\"stdout\":\"...\",\"stderr\":\"...\",\"exitCode\":N}. " +
                $"Allowed commands: {string.Join(", ", allowed)}. " +
                $"Working directory is pinned to '{scratchDir}'.")
            .Input("""
                {
                  "type": "object",
                  "properties": {
                    "command": { "type": "string" },
                    "args":    { "type": "array", "items": { "type": "string" } }
                  },
                  "required": ["command"]
                }
                """)
            .SideEffect(ToolSideEffect.ReadOnly)
            .Cost(ToolCostClass.Cheap)
        .Then()
        .Log("[SHELL-TOOL] ▶ in=${body}")
        .To(ExecDsl.Run()
            .AllowedCommands(allowed)
            .WorkingDirectory(scratchDir)
            .TimeoutMs(5_000)
            .MaxStdoutBytes(8_192)
            .MaxStderrBytes(8_192))
        .Log("[SHELL-TOOL] ◀ exit=${header.redbExec.ExitCode} body=${body}");
});

Description and the JSON schema are what Claude sees as "this function is available, here's its signature." SideEffect.ReadOnly and Cost.Cheap are guidance for the governance hooks (budgets, approval gating). After .Then() it's an ordinary route: Log → ExecDsl.Run() with allowlist, working dir, 5-second timeout and a stdout/stderr cap → Log. No LLM-specific code below .Then(). It's still just a route — which means you can wrap it in a CircuitBreaker, a Throttle, a Transaction, a WireTap shadow, anything from the 30+ EIP processors redb.Route already ships.

7. Start and block

await ctx.Start();
producerTemplate.Start();

var stop = new ManualResetEventSlim();
Console.CancelKeyPress += (_, e) => { e.Cancel = true; stop.Set(); };
stop.Wait();

await ctx.DisposeAsync();

Run it

dotnet run

curl -d "how much free disk space?" http://localhost:5088/api/llm/shell
curl -d "what did you just say?" -H "X-Chat-Id: my-chat" http://localhost:5088/api/llm/shell

First request: Claude sees the system prompt ("you have a shell tool"), decides to check free space, calls the tool with something like {"command":"cmd","args":["/c","fsutil","volume","diskfree","C:"]}, gets the stdout back, summarises it for the human. Second request, with X-Chat-Id: my-chat, finds the previous turn in InMemoryConversationStore and answers in context.

The logs at that moment show the full path: [LLM-SHELL] ▶ prompt=... → [SHELL-TOOL] ▶ in={"command":"cmd",...} → [SHELL-TOOL] ◀ exit=0 body={"stdout":"...","exitCode":0} → [LLM-SHELL] ◀ iters=2 stop=end_turn tokensIn=... tokensOut=... reply=.... All of it is just the redb.Route .Log() step, not a separate LLM-tracing pipeline.

The command allowlist (cmd, pwsh, sh, bash) is the security envelope: anything outside it is rejected by redb.Route.Exec before a process starts.

`redb.Route.Exec` — process spawning as the 25th transport

In the demo above ExecDsl.Run() showed up as "the backend of the shell tool", but it's a connector in its own right, shipped in 3.1.0 alongside redb.Route.Llm. It closes a boring but ubiquitous gap: the framework already spoke 22+ transports (HTTP, Kafka, SQL, …), but reaching out to the operating system itself was missing.

Producer — .To(ExecDsl.Run(...)) — synchronous spawn. The command is resolved in this order: JSON body → headers redbExec.Command/redbExec.Args → URI options ?command=.... The Out body is JSON shaped for the LLM tool ABI:

{ "stdout": "...", "stderr": "...", "exitCode": 0, "timedOut": false }

This is exactly why the shell tool in the demo is .To(ExecDsl.Run().AllowedCommands(...)) with zero glue code in between. The model produces {"command":"cmd","args":[...]}, the producer parses it, runs it, returns structured JSON. Route-level features (CircuitBreaker, Throttle, OnException, audit) apply automatically — it's an endpoint like every other one.

Consumer — From(ExecDsl.Run(...).Schedule("5m")) — same spawner, but as a first-class source endpoint with a built-in scheduler. No cron, no separate worker:

routes.From(ExecDsl.Run("./scripts/health-check.sh")
                   .Schedule("5m")
                   .TimeoutMs(30_000))
      .Choice()
        .When(e => e.In.Headers["redbExec.ExitCode"]?.ToString() != "0")
          .To("http://alerts.internal/oncall")
        .Otherwise()
          .To("kafka://metrics.healthy");

Every 5 minutes the script runs, the route branches on exit code — one way to an HTTP webhook for the on-call, the other to a Kafka topic. ?schedule= accepts simple intervals (500ms, 30s, 5m, 1h). For cron expressions: From("quartz://<cron>").To(ExecDsl.Run(...)) — Quartz is already a scheduler, no point re-implementing it inside Exec.

The security envelope — what gives shell-as-a-tool the right to exist in the first place:

AllowedCommands — case-insensitive on the file name, so /usr/bin/git and git.exe both match git. Anything off the list is rejected with UnauthorizedAccessException before the process starts.
WorkingDirectory — pins the cwd; the spawned process can't escape it on its own.
EnvironmentOverrides + ScrubEnvironment — start from an empty environment and apply only the KEY=VALUE pairs you set.
TimeoutMs — wall-clock kill-switch that takes the whole process tree.
MaxStdoutBytes / MaxStderrBytes — caps that protect the host from a runaway process.

Why a separate package, not part of redb.Route.Llm — three reasons: (1) Exec is useful with no LLM in sight (scheduled health probes, log rotation, deploy glue, backups); (2) redb.Route.Llm shouldn't drag a dependency on process spawning into projects whose agents only call HTTP or SQL tools; (3) the same allowlist/timeout/cap mechanism will be reused by future connectors that share the same security profile.

`From("llm://...")` — the scheduled agent consumer

This is the bit that has no equivalent in Camel's langchain4j-* family (where LLM is producer-only): the LLM endpoint is a first-class source, with the scheduler baked in.

From("llm://groq?schedule=5m" +
     "&systemPromptRef=#watchdog-system" +
     "&initialBodyRef=#daily-brief" +
     "&tools=*")
    .To("rabbitmq://alerts");

Every 5 minutes: a fresh agent run with the system prompt resolved from registry key #watchdog-system and the user prompt from #daily-brief. The reply goes to RabbitMQ. ?schedule= takes simple intervals (500ms, 30s, 5m, 1h); for cron use From("quartz://...").To("llm://...") — Quartz is already a scheduler, no point duplicating it inside the LLM consumer.

Use cases that fit this shape: watchdog agents, scheduled report generation, self-improving agents with conversation memory (same history across runs).

`#`-prompts — the dynamic registry

A # prefix on a URI parameter turns the value into a registry lookup. Some other route can rewrite the prompt by name; the next agent run picks it up without a redeploy:

host.Context.AddToRegistry("style.terse", "Reply in fewer than 5 words.");

r.From("direct:chat")
    .To("llm://scripted?systemPromptRef=#style.terse")
    .To("mock:done");

// ... later, from another route ...
host.Context.AddToRegistry("style.terse", "Reply in French only.");
// the next call sees the new value

Resolution: first IPromptTemplateRegistry (versioned, what eval replay needs), then the generic context registry (a plain string). Without # the value is literal and arrives at the provider unchanged.

This is the same #name registry convention redb.Route uses framework-wide for connection factories. Same convention, not a separate prompt-ref DSL.

Agent memory — `AddRedbLlmStorage()`

By default every state surface is in-memory (conversation transcripts, tool idempotency, approvals, cost ledgers, audit). That keeps AddRedbRouteLlm() zero-dependency — fine for tests and stateless agents, lost on restart.

One line swaps the defaults for REDB-backed stores:

services.AddRedbRoute(route =>
{
    route.Services.AddRedbRouteLlm();
    route.Services.AddRedbIdempotentRepository();   // required for idempotency
    route.Services.AddRedbLlmStorage();             // ← REDB stores on every surface
});

What AddRedbLlmStorage() replaces:

Interface	Default	REDB store
`IConversationStore`	`InMemoryConversationStore`	`RedbConversationStore` — tree-backed, parent linkage via native `parent_id`, message ids on indexed `value_string`
`IToolIdempotencyStore`	`InMemoryToolIdempotencyStore`	`RedbToolIdempotencyStore` (on top of `IIdempotentRepository`)
`IApprovalStore`	`InMemoryApprovalStore`	`RedbApprovalStore`
`ICostBudgetStore`	`InMemoryCostBudgetStore`	`RedbCostBudgetStore` — running totals per tenant/window
`IAgentObserver`	`NoopAgentObserver`	`RedbAuditObserver` — one row per tool call

Two non-obvious design choices:

Per-row business identifiers live on _objects.value_string (an indexed column), not inside the props JSON. Lookups are O(log n) on one column, not a full scan with JSON decode.
Transcript integrity is the tree's native parent_id (IRedbService.CreateChildAsync), not a soft FK in props. The tree primitive enforces it.

Stores lazy-sync their scheme on first use. No migration step.

What the engine carries for free

This is the central argument for "connector, not library." Everything redb.Route already does applies to LLM calls without extra code:

Concern	DSL primitive
Retry / backoff	`RedeliveryPolicy`, `OnException`
Rate limiting	`Throttle`
Resilience	`CircuitBreaker`
Idempotency	`IdempotentConsumer`
Compensation	`Saga`
Audit / shadow	`WireTap`, `Multicast`
Tracing & metrics	`RouteActivitySource`, `RouteMetrics`
Persistence	`redb` schemes (typed object engine)

A tool that calls an expensive API gets wrapped in a CircuitBreaker and a Throttle. Want a shadow run of a new system prompt alongside the old one? Multicast + WireTap. These aren't "features of the LLM connector" — they're the engine, and the LLM is attached to it as an endpoint.

What's live-tested, and what isn't

The honest status of the provider matrix (this is in the README, repeated here):

Groq + Llama 3.3 70B — the most reliable free tier in scope. Carries the strict assertion in BasicChatTests and ToolRouteTests (the model has to produce a literal token).
Mistral small-latest — stable for short-form replies; tool-use on the free tier is flaky, so tool tests assert philosophically.
Gemini 2.0 Flash — 15 RPM on free; works on a quiet repo.
Anthropic Claude (Haiku 4.5 / Sonnet 4.6) — via the OpenAI-compat endpoint, live-tested in ClaudeChatTests and the redb.Route.Demo host.
OpenRouter / Cerebras — scaffolding works; individual free models rate-limit; tests gated on [EnvFact] and skipped without a key.

Honest skip-list, things not in this release:

Embeddings and vector stores — Phase 2 (embed://, vector:// schemes are planned).
RAG primitives, document loaders, web search — not first-class yet (a Tavily search tool is already shipped in redb.Route.Llm.Tools and plugs in as any other .AsLlmTool("web_search")).
Sliding-window memory shapes (window-by-N-messages, window-by-K-tokens) — not first-class. Persistent transcripts via AddRedbLlmStorage() are shipped; a windowed shape on top is realizable today via Process + the conversation store, just not as a one-line option.
Native AnthropicProvider — OpenAI-compat surface covers most scenarios; the native Messages API is for the features the compat surface doesn't expose.

Links

GitHub leads NuGet. Fresh bug fixes (e.g. the tool_use/tool_result
pairing recovery on conversation reload, or the Windows OEM-codepage
decoding fix in redb.Route.Exec) are already cut on main under version
3.1.1 — see CHANGELOG.md
in the public repo. NuGet packages are batched into the next release, so
if you hit a production bug — check main and the latest pre-release tag
first, then wait for the NuGet bump. This is normal practice, not a
process bug.


redb.Route on GitHub	github.com/redbase-app/redb-route
redb.Route.Llm on NuGet	nuget.org/packages/redb.Route.Llm
redb.Route.Exec on NuGet	nuget.org/packages/redb.Route.Exec
Full package README	redb.Route.Llm/README.md
USER-GUIDE (full walkthrough — DSL, governance, conversation)	redb.Route.Llm/doc/USER-GUIDE.md
STORAGE — REDB schemas for conversation / approval / audit / idempotency	redb.Route.Llm/doc/STORAGE.md
Exec connector README	redb.Route.Exec/README.md
Standalone `Llm.HttpShell` demo (curl → Claude → shell → reply)	demo/Llm.HttpShell/Program.cs
Full demo project (22+ routes)	demo/redb.Route.Demo/Routes/LlmHttpRoutes.cs
Discussions	github.com/redbase-app/redb-route/discussions

All Apache 2.0. A separate deep-dive will cover the agent tool-loop, the governance hook interfaces, and a tour of the REDB storage schemas. Questions in the comments are exactly what writes the next post — especially anything along the lines of "does the connector do X if I do Y?"

REDB inside, part 1.1 — why the same 13 tables stay fast no matter how many classes you throw at them

rinat kozin — Mon, 08 Jun 2026 21:57:41 +0000

A couple of weeks ago I published REDB inside, part 1 — the 13 tables the whole engine runs on. It walked through _objects, _values, _structures, why this isn't classical EAV, and what the _scheme_metadata_cache does. If you haven't read it, start there — the rest of this post assumes you know the layout.

This is part 1.1, not part 2. Same physical-storage conversation, different angle. Part 2 is going to be about code-first schemes (SyncSchemeAsync<T>), and the deep C# dive — LINQ translator, CRUD internals, trees — that's parts 3 through 5 of the series. This post stays in the database layer.

The reason for a 1.1 instead of jumping to 2 is simple: every time I publish part 1, the same question comes back in the comments and in DMs:

"Fine, typed columns beat string EAV. But you've still got 9.7 million rows in _values. Any real query — WHERE Salary > 80000, WHERE OrderDate >= '2026-06-01' — has to scan that table. Without one index per field per schema you're living in Seq Scan."

Fair question. Let's actually answer it, with the DDL, the prod numbers, and a side note about how to remove indexes once your app stabilizes.

Numbers in this post come from TSUM — a logistics system handling truck movements and orders through distribution centers. Real production, no benchmark setup.

TL;DR: In a classical EF schema, indexes multiply with tables. In REDB, the index set is designed once in DDL and serves any business schema you put on top — adding the 33rd class to your domain doesn't add a single line to redbPostgre.sql. The active surface for any business class is just 2 tables (_objects + _values), plus 2 more for lookups and 3 for RTTI metadata; the rest is infrastructure that doesn't grow with your class count. On TSUM prod this gives 999 orders / 991 ms on 2 cores with default Postgres settings and zero framework-level cache: one SELECT pulls all 999 existing routes (139 ms), bulk-save writes 32 changed objects through the COPY protocol (154 ms), and 967 unchanged rows never reach the database thanks to an app-level ComputeHash() short-circuit.

The "9.7 million rows" objection, defused in two sentences

Before we open any DDL, here's the structural answer to the objection above.

Classical EAV has rows like (object_id, attribute_name TEXT, value TEXT). An index on value is useless there because values are heterogeneous, casts happen at runtime, and selectivity on attribute_name is low. REDB is not that.

Two things change the picture:

Values are already split into typed columns. _Long, _String, _Numeric, _DateTimeOffset, _Boolean, _Guid, _ListItem, _Object. No casts at query time. An index on _Long is a normal bigint btree, not a polymorphic guess.
Every row in _values carries _id_structure — a bigint pointing at the schema field it belongs to. With 401 fields across the TSUM domain, WHERE _id_structure = X discards roughly 99.75% of the table before the optimizer even looks at the value column.

So a facet query never scans 9.7M rows. It scans ~_values_count / structure_count rows, then does an Index Only Scan on a typed column. We'll see this in the EXPLAIN-equivalent reasoning below.

That's the architecture. Now the question that actually matters: how exactly are the indexes laid out so that this runs in milliseconds?

The architectural payoff: indexes don't grow with your domain

In a classical EF layout, the table count grows linearly with business entities. Each table averages 3–5 indexes — primary key, FK indexes, business filters. So the total number of indexes in the database grows multiplicatively with project size.

Take TSUM. Here's what's in tsum.Domain/Entities:

TransportationRoute (the actual route)
TransportationPoint (route stops)
Driver, Vehicle, ShippingPoint, YardPlace
SliceSettings, SliceSnapshot, TransportSnapshot, TransportNorm
SpecialRcSettings, TonnageGroupSettings, GarageState
TsumAdUserRef, UserFilterPreference

Fifteen "root" entities. But that's just the start. Inside TransportationRoute alone you'll find RedbListItem references to: Drivers, Vehicles, CarMarks, ShippingPoints, BusinessTypes, YardPlaces (×2 — PlaceTo / PlaceFrom), LoadingZones, TransportStatuses, DeliveryStatuses, TripRisks. Every other entity has its own set: chassis types, readiness statuses, delay reasons, route classifications.

In a classical schema, every one of those lookups is its own table. Plus audit/history tables (TransportSnapshot, SliceSnapshot are clearly snapshot entities). Plus M2M tables for collection links. A realistic estimate for TSUM in a classical layout is 60–80 tables.

At 60–80 tables and 3–4 indexes per table, you're looking at 200–320 indexes to design, maintain, reindex, and drop when stale. Every release in an EF project ships an AddColumn + CreateIndex migration. By year two, somebody walks the schema with a flashlight removing duplicate indexes that piled up because three different developers added similar-but-not-identical indexes for slightly different queries.

In REDB the picture is different. The same 32 schemes / 401 properties in TSUM physically have 2 primary keys: pk__objects and pk__values. The full index list is hardcoded in redbPostgre.sql and runs around 50 indexes — for the entire system. Not per class. For the system.

Now here's the clarification I owe readers from part 1. I called REDB "13 tables" in that post, and a fair number of people walked away with the impression that all 13 are involved every time you query a class. They aren't. The actual breakdown is:

Data storage — 2 tables: _objects (the header) and _values (the field values). Every Query<T>(), every SaveAsync, every LINQ filter lands here.
Lookups — 2 tables: _lists and _list_items. These power RedbListItem fields (Driver, Vehicle, ShippingPoint, etc.). Lookups are shared across classes and don't grow when you add a new business entity.
RTTI / schema metadata — 3 tables: _types (primitives), _schemes (classes), _structures (class properties). The engine reads these through _scheme_metadata_cache; business queries never touch them.
Everything else — infrastructure: _users, _roles, _permissions, _links, _dependencies, _functions, _scheme_metadata_cache, soft-delete via @@__deleted. These don't grow with class count and aren't on the hot path.

So whatever class a developer writes in tsum.Domain, its data physically lives in 2 tables; reference fields hit 2 more for lookups; schema metadata sits in 3 RTTI tables. The six infrastructure tables look identical for a 5-class project and a 500-class one. That is what "indexes invariant of class count" really means — not "all 13 tables on every query," but "the active footprint for business data is two tables, and it doesn't change."

When a developer adds the 33rd class to tsum.Domain — say, WarehouseSlot — the database gets:

one row in _schemes,
N rows in _structures (one per property),
zero index migrations. None.

Because the entire query surface for the new class is served by the same indexes on _objects and _values that served the previous 32. WHERE _id_scheme = ID(WarehouseSlot) uses IX__objects__schemes. Sort by creation date — IX__objects__scheme_date_create. Find an object by some property value — IX__values__structure_object_lookup. Parent–child tree — IX__objects__scheme_parent.

This is the architectural payoff the post is built around. The index plan is fixed at engine design time, not smeared across business-app releases. The skeptic will counter: "so generic indexes on _values must be slower than per-field indexes in an EF schema." They aren't — and we'll see why below.

The same applies to MSSQL. Worth saying out loud: this post isn't Postgres-specific. redb.MSSql/sql/redbMSSQL.sql carries a mirror set of indexes with minor dialect differences (covered in a later section). The architectural trick — indexes on the physical schema, not on business tables — ports between engines without giving anything up.

A tour of `redbPostgre.sql`, grouped by what it's for

If you open the DDL and read top to bottom you'll see ~50 indexes interleaved. They're easier to understand grouped by use case rather than by table. Let's do it that way.

1. Storage uniqueness in `_values`

Three partial-unique indexes, one per shape:

-- Scalar fields: one (object, field) → one value
CREATE UNIQUE INDEX UIX__values__structure_object
ON _values (_id_structure, _id_object)
WHERE _array_index IS NULL AND _array_parent_id IS NULL;

-- The marker row for a nested class/array inside a parent
CREATE UNIQUE INDEX UIX__values__structure_object_parent
ON _values (_id_structure, _id_object, _array_parent_id)
WHERE _array_index IS NULL AND _array_parent_id IS NOT NULL;

-- Collection elements indexed by position/key
CREATE UNIQUE INDEX UIX__values__structure_object_array_index
ON _values (_id_structure, _id_object, _array_parent_id, _array_index)
WHERE _array_index IS NOT NULL;

Why three instead of one. Each shape has a different uniqueness key. Scalars are unique by (structure, object). Nested-structure markers add parent. Array elements add index. A single covering unique index can't express this (NULLs would break the semantics), but three partials map cleanly onto the physical model.

Bonus: each partial only stores its slice of rows, so the btree is smaller than one giant unique index would be. On 9.7M rows that's tens of percent saved in page cache.

2. Facet search — the hottest path

A facet query in REDB is "find objects whose field X compares to Y." In SQL terms:

SELECT v._id_object FROM _values v
WHERE v._id_structure = @structure_id
  AND v._Long > 80000;

That pattern is served by a single composite index:

CREATE INDEX IX__values__structure_object_lookup
ON _values (_id_structure, _id_object,
            _Long, _DateTimeOffset, _Boolean, _Double, _Guid, _Numeric, _ListItem, _Object);

Every typed column except _String is in the key. Index Only Scan: Postgres finds the matching rows and grabs the value directly from the index — no heap fetch.

In parallel, two covering indexes with INCLUDE:

CREATE INDEX IX__values__object_structure_lookup
ON _values (_id_object, _id_structure, _array_index)
INCLUDE (_Long, _Double, _DateTimeOffset, _Boolean, _Guid, _Numeric, _ListItem, _Object);

CREATE INDEX IX__values__object_array_null
ON _values (_id_object, _id_structure)
INCLUDE (_Long, _Double, _DateTimeOffset, _Boolean, _Guid, _Numeric, _ListItem, _Object)
WHERE _array_index IS NULL;

These cover the opposite direction: "load all values of an object in one query," for LoadAsync<T>. Same trick — Index Only Scan, zero heap reads.

Why _String is excluded. Postgres btree caps at ~2700 bytes per index row. If a JWT payload ends up in _String at 3 KB, the insert blows up. So strings get their own partial index with a length guard:

CREATE INDEX IX__values__String_not_null
ON _values (_id_structure, _id_object, _String)
WHERE _String IS NOT NULL AND length(_String) < 2000;

Long strings (CMS bodies, base64 payloads) aren't covered — and they aren't filtered with = either. For substring search there's a separate GIN:

CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE INDEX IX__values__String_pattern
ON _values USING gin (_String gin_trgm_ops)
WHERE _String IS NOT NULL;

This serves LIKE, ILIKE, $contains, $startsWith, $endsWith, regex — all through pg_trgm. The MSSQL counterpart is full-text search or a persisted computed column with a filtered index; the choice is in the dialect's DDL.

3. Collections (arrays and dictionaries)

Part 1 covered relational storage of collections — a marker row plus child rows linked by _array_parent_id and _array_index. The supporting indexes:

-- traverse collection elements by index
CREATE INDEX IX__values__array_parent_index
ON _values (_array_parent_id, _array_index);

-- look up by dictionary key (string keys)
CREATE INDEX IX__values__array_key
ON _values (_id_structure, _array_index)
WHERE _array_index IS NOT NULL;

-- partial: only rows that belong to collections
CREATE INDEX IX__values__parent_structure
ON _values (_array_parent_id, _id_structure)
WHERE _array_parent_id IS NOT NULL;

The third one feeds the pivot CTE that reads nested Class/Dictionary fields like AddressBook["home"].City.

4. Object trees (`_objects._id_parent`)

Trees are the backbone of REDB — sections, categories, org charts, all linked through _id_parent:

-- direct children: scheme + parent → id
CREATE INDEX IX__objects__scheme_parent
ON _objects (_id_scheme, _id_parent, _id);

-- covering index for subtree traversal (metadata in INCLUDE)
CREATE INDEX IX__objects__parent_id_descendant_lookup
ON _objects (_id_parent, _id_scheme)
INCLUDE (_id, _id_owner, _date_create, _date_modify)
WHERE _id_parent IS NOT NULL;

-- partial: roots (objects without a parent)
CREATE INDEX IX__objects__root_objects
ON _objects (_id_scheme, _id)
WHERE _id_parent IS NULL;

-- reverse path: child → parent → scheme
CREATE INDEX IX__objects__id_parent_scheme
ON _objects (_id, _id_parent, _id_scheme)
WHERE _id_parent IS NOT NULL;

WhereHasAncestor, WhereHasDescendant, LoadTreeAsync all run through this quartet.

5. Sorting and metadata

-- main "newest first" feed index
CREATE INDEX IX__objects__scheme_date_create
ON _objects (_id_scheme, _date_create DESC, _id);

-- sort by name within a scheme
CREATE INDEX IX__objects__scheme_name
ON _objects (_id_scheme, _name, _id);

-- global indexes for cross-scheme search
CREATE INDEX IX__objects__name ON _objects (_name);
CREATE INDEX IX__objects__hash ON _objects (_hash);

And the indexes for RedbPrimitive<T> (when an object is a primitive — counter, token, single value):

CREATE INDEX IX__objects__value_long ON _objects (_value_long) WHERE _value_long IS NOT NULL;
CREATE INDEX IX__objects__value_string ON _objects (_value_string) WHERE _value_string IS NOT NULL;
CREATE INDEX IX__objects__value_guid ON _objects (_value_guid) WHERE _value_guid IS NOT NULL;
CREATE INDEX IX__objects__value_datetime ON _objects (_value_datetime) WHERE _value_datetime IS NOT NULL;
CREATE INDEX IX__objects__value_numeric ON _objects (_value_numeric) WHERE _value_numeric IS NOT NULL;

All partial. Each one only indexes rows where its value column is set.

6. Schemes and structures (`_structures`, `_schemes`)

These are the cold tables — orders of magnitude smaller than _values, mostly read while building the metadata cache. They still need to be sub-millisecond:

-- covering index for ORDER BY field name
CREATE INDEX IX__structures__name
ON _structures (_name)
INCLUDE (_id, _id_type, _collection_type, _id_scheme);

-- covering lookup by structure ID
CREATE INDEX IX__structures__id_lookup
ON _structures (_id)
INCLUDE (_id_type, _name, _collection_type, _id_scheme);

-- partial: split collection vs non-collection
CREATE INDEX IX__structures__not_collection
ON _structures (_id, _name, _id_scheme)
WHERE _collection_type IS NULL;

CREATE INDEX IX__structures__collection
ON _structures (_id, _id_scheme, _collection_type)
WHERE _collection_type IS NOT NULL;

The DDL comments record the reason: these indexes kill Seq Scan on InitPlan and EXISTS subqueries inside build_hierarchical_properties_optimized, dropping query cost from 6.10 to 4.29 (-30%).

What got removed

redbPostgre.sql has a commented-out block worth quoting:

-- ============================================
-- REMOVED REDUNDANT INDEXES (migration_drop_redundant_indexes.sql)
-- Reason: Covered by composite index IX__values__structure_object_lookup
-- Facet search ALWAYS filters by (_id_structure, _id_object) BEFORE value
-- ============================================
-- CREATE INDEX "IX__values__String" ON _values (_String) ...;
-- CREATE INDEX "IX__values__Long" ON _values (_Long) ...;
-- CREATE INDEX "IX__values__Guid" ON _values (_Guid) ...;
-- ...

That's a cleanup history committed straight into the schema file. Early REDB shipped one index per typed column. Real workloads showed they never lit up: facet queries always start from _id_structure, never from a raw value column. The composite (_id_structure, _id_object, _Long, ...) swallowed them whole. The single-column ones got pulled.

Why this is fast on prod

Architecture is half the story. The other half is the actual numbers. Here's a typical TSUM order-processing tick from production logs:

[TSUM] orders=999 routes(+2 ~30 =967) drivers(+0 ~0) vehicles(+0 ~0)
       sync=482 query=139 save=154 total=991ms

What it means:

[TSUM] orders=999 routes(+2 ~30 =967) drivers(+0 ~0) vehicles(+0 ~0) sync=482 query=139 save=154 total=991ms

999 orders arrived in one XML batch from SAP S/4 (the stored procedure usp_TsUM_MonitoringReport_xml).
routes(+2 ~30 =967): of the 999 incoming, 2 were new routes, 30 changed, 967 weren't touched at all (content unchanged).
drivers / vehicles (+0 ~0): the driver and vehicle dictionaries had no changes in this batch.
sync = 482 ms — this is lookup-dictionary sync (DictionarySyncService.SyncFromOrdersAsync): walk all 999 orders, reconcile drivers / vehicles / shipping points / business types / delivery statuses against REDB lists, upsert any new list_items if needed. This is not schema sync.
query = 139 ms — one SELECT to REDB: WhereRedb(o => codes.Contains(o.ValueString)) — pulls all 999 existing routes by their codes in a single query.
save = 154 ms — SaveAsync for 32 objects (2 created + 30 updated). The 967 unchanged ones never get here.
Environment: 2 cores, default shared_buffers, no pg_prewarm, no framework-level cache.

Important caveat: schema sync (SyncSchemeAsync<T> — the bit that compares _structure_hash and refreshes _scheme_metadata_cache) is a separate step that runs once at process startup, before any orders flow. It's not in this log line at all. The "sync" number here is application-level dictionary reconciliation, not schema reconciliation.

Let's break down what's physically happening behind each number.

Query: 139 ms for 999 objects in one call

The most striking number in the log. This isn't 999 separate LoadAsync calls — it's one facet query:

var existing = await redb.Query<TransportationRoute>()
    .WhereRedb(o => codes.Contains(o.ValueString!))
    .ToListAsync();

It translates roughly to:

-- find all routes whose _value_string is in the list of 999 codes
SELECT o.* FROM _objects o
WHERE o._id_scheme = @routeSchemeId
  AND o._value_string = ANY(@codes);

After that, one batched query for _values of all matched objects via IX__values__object_array_null (covering INCLUDE) — Index Only Scan, no heap access. Output: 999 fully hydrated RedbObject<TransportationRoute> instances with all ~50 properties each, including ListItem references to dictionaries.

139 ms / 999 objects = 0.14 ms per object on paper, but that math is misleading — the real cost is two SQL round trips (one for _objects, one for _values) plus C# hydration.

No N+1. No Includes. ListItem references resolve through RefDataCache in memory, so Vehicle, Driver, ShippingPoint get filled without extra SQL.

Save: 154 ms for 32 changed objects

Two things matter here, and both are critical.

First — application-level change tracking. Before SaveAsync, every existing route runs through ComputeHash():

var hashBefore = obj.ComputeHash();
EnrichRouteFromOrder(obj.Props, routeProps);
if (obj.ComputeHash() != hashBefore)
{
    toSave.Add(obj);  // data really changed — queue for save
    updatedCount++;
}
else
{
    skippedCount++;   // byte-identical — don't touch the database
}

Of the 999 incoming orders, 967 are skipped right here. They never hit _objects/_values, never generate a single DML statement. This isn't a REDB-engine optimization — it's an application-code pattern, but REDB supports it cheaply via ComputeHash() over Props.

Second — bulk save in a single transaction. The remaining 32 objects go in one call:

await redb.SaveAsync(toSave);   // toSave: List<IRedbObject>, 32 items

Here's what happens inside. _objects gets normalized in a batch first — for all 32 objects, one INSERT/UPDATE pass (new rows go via COPY/UNNEST-INSERT, modified rows via batched UPDATE). Then _values: for each modified object the value rows are recreated, and again not row-by-row but in one bulk call through Postgres COPY protocol. All in a single transaction — _objects and _values commit atomically.

COPY matters. It's a streaming binary protocol that ingests N rows without a per-row round trip — the driver writes a continuous byte stream into the socket. For 32 objects × ~50 properties = ~1600 value rows, that's one COPY operation instead of 1600 INSERTs. Btree indexes get their batch and update amortized.

So 154 ms for 32 objects (~4.8 ms per object on full overwrite) isn't "the engine is very fast" — it's "the engine doesn't do unnecessary work." Bulk where it can bulk; early skip where data didn't change.

Sync: 482 ms — what it really is

482 ms is the heaviest line in the log, and it's also the most commonly misread one. It's not schema sync — schema sync already ran at Worker startup, before any orders showed up. The 482 ms is DictionarySyncService.SyncFromOrdersAsync:

Five parallel dictionary loads from REDB: Drivers, Vehicles, ShippingPoints, BusinessTypes, DeliveryStatuses — via redb.ListProvider.GetListByNameWithItemsAsync(...). Pulls _lists plus all their _list_items.
Walk over 999 orders: each DriverId, CarId, ShippingPoint, BusinessTypes is checked against existing list items. If new, an upsert is queued.
If dictionaries changed — refresh RefDataCache (a static in-memory cache so the Order → RouteProps mapping doesn't re-query the DB).

This batch had zero deltas on drivers and vehicles (+0 ~0), but 482 ms still went into proving they didn't change — that's 999 rows being checked against thousands of existing list items across multiple dictionaries.

This is application-level sync. It belongs to TSUM's logistics process, not REDB's engine. But it makes a useful point: even when "discovering nothing needs to change" costs more than the actual save (482 ms sync vs 154 ms save), the whole tick still fits in under a second for a thousand orders.

Schema sync is a separate story. Cold start of the Worker — tens to hundreds of milliseconds, one-off (32 schemes, hash comparison). Hot restart with no model changes — single-digit milliseconds. It happens before the first [TSUM] log line and isn't counted in the per-tick budget.

Coming back to the 9.7M-row objection

The original objection was "how can this be fast at that table size." The answer: the engine never scans _values whole. Any query starts from one of these four anchors:

PK (LoadAsync by ID) — O(log n) on the btree.
_id_scheme + _value_string for facets through a RedbPrimitive column, like the TSUM case above — Index Only Scan on IX__objects__value_string.
_id_structure (regular facet search) — discards ~99.75% of rows immediately. With 401 properties, the average _id_structure covers ~24K of the 9.7M total.
_id_object (LoadAsync — all values for an object) — discards even more aggressively, since an object has tens of values, not thousands.

Table size in rows only affects the index size in pages. Postgres btree fan-out is in the hundreds, so 9.7M rows is a 4–5 level deep tree. Search is logarithmic, not linear.

"You don't always need everything you've got": pruning indexes after stabilization

redbPostgre.sql carries the full spectrum of indexes for every REDB scenario: facet search, trees, collections, GIN full-text, partial indexes for NOT NULL, indexes for recursive permission CTEs. That's right for a library: you don't know up front which workload profile a given app will have.

But in your specific app, some of those indexes may never light up. That's fine. TSUM, for instance, never calls WhereHasDescendant because its domain hierarchies are flat — so IX__objects__parent_id_descendant_lookup sits in pages but no scan ever goes through it.

After the app's been in production for two to four weeks under stable load, pull the index stats and trim aggressively:

SELECT schemaname, relname, indexrelname,
       idx_scan,
       pg_size_pretty(pg_relation_size(indexrelid)) AS sz
FROM pg_stat_user_indexes
WHERE idx_scan = 0
  AND pg_relation_size(indexrelid) > 10*1024*1024  -- only indexes > 10 MB
ORDER BY pg_relation_size(indexrelid) DESC;

On TSUM, this query surfaces about 4423 MB of indexes with idx_scan = 0. That doesn't mean those indexes are bad. It means nobody's hitting those query paths in this workload profile.

Working with the result:

Don't drop everything immediately. First, walk the list: which indexes serve scenarios that "haven't fired yet" (quarterly reports, year-end reconciliations, big imports)? Leave those alone — re-creation later costs more than the disk you'd save now.
The rest are candidates for DROP INDEX CONCURRENTLY. It's safe — no table lock — and you can always restore the index with the same CREATE INDEX CONCURRENTLY, since the DDL knows what shape it should have.
After dropping, watch for two to four more weeks. If no unexpectedly slow queries show up in logs, the call was right. If something regresses, restore.

This is a standard PostgreSQL practice, not a REDB-specific trick. But it works cleaner in REDB because the index set is designed once and frozen in DDL. Pruning is one targeted decision in one place — not a hunt through three years of accumulated migrations across business tables.

The other direction works too. If you spot a single facet query that's hot enough to warrant an extra push, nothing stops you from adding a local index just for it:

-- e.g., "find routes for a specific driver in the last 7 days"
CREATE INDEX IX_my_app__route_by_driver
ON _values (_id_structure, _ListItem, _id_object)
WHERE _id_structure = <driver_field_id>;

That kind of index is application-side optimization. It lives in your app's migration, not in the REDB core. And the same pg_stat_user_indexes audit applies to it after a few weeks.

What's deliberately not in the DDL, and why

A second-level skeptic asks the next question: "Why no hash index on _String? Why no BRIN on _date_create? Why no expression index on LOWER(_String)?"

Hash indexes. Historically weaker than btree on most metrics in Postgres, and they don't support partial conditions. Almost every REDB index is partial — so hash is structurally a non-fit, not a coin-flip choice.

BRIN on _date_create. A reasonable candidate on big archival tables, but redundant for the current REDB profile. IX__objects__scheme_date_create (DESC, _id) already serves the exact scenario BRIN would compete with. BRIN would only win in a cold-storage case (>10M rows on a single scheme) — that's a separate "archival classes" phase, and the current core DDL doesn't address it yet.

Expression indexes (LOWER(_String), _DateTimeOffset::date, etc.). Deliberately omitted. This is an application choice: case-insensitive search on a specific text field, aggregation by day/month — these are options that belong in the app's migration, not the engine. GIN with pg_trgm already provides case-insensitive substring search via ILIKE, and that covers most needs.

Bloom, BRIN-multi, GIN on jsonb. All candidates for narrow scenarios. If you have one very specific query that's provably hot and provably not covered by the standard set — you add a local index for it in a separate migration. Without touching redbPostgre.sql.

The principle is general: the core DDL covers 95% of scenarios, the remaining 5% lives in the application layer. That's the opposite of the EF approach, where the DDL belongs to the application from day one and every index is part of its migration history.

MSSQL: nearly the same picture

While we're in DDL territory, worth saying clearly: this post isn't Postgres-specific. redb.MSSql/sql/redbMSSQL.sql carries a mirror set of indexes, and the architectural trick — "indexes don't grow with class count" — works identically on both engines.

Ports cleanly:

All three partial-unique indexes on _values (in MSSQL — filtered indexes with WHERE, different syntax, same semantics).
Composite indexes for facet search (IX__values__structure_object_lookup exists in both DDLs with an identical key shape).
INCLUDE covering indexes — natively supported in MSSQL since 2005, port one-to-one.
Object-tree indexes (IX__objects__scheme_parent, IX__objects__parent_id_descendant_lookup) — unchanged.
Partial / filtered indexes for RedbPrimitive<T> — unchanged.

Differs:

GIN + pg_trgm for substring search on _String. The MSSQL counterpart is full-text search or a filtered index over a computed column. redbMSSQL.sql takes a different path (LIKE over a regular btree with a length-bound partial).
NULLS NOT DISTINCT in Postgres 15+ — MSSQL spells it differently (UNIQUE + filtered WHERE NOT NULL).
ON DELETE CASCADE from _structures to _values works directly in Postgres but not in MSSQL because of multiple cascade paths — replaced with an INSTEAD OF DELETE trigger. The DELETE plan is slightly different in MSSQL but the indexes are the same.
Per-type filtered indexes are spelled out more explicitly in MSSQL (IX__values__Long_filter, IX__values__Guid_filter, ...) — in the Postgres DDL they're commented out as "covered by composite." That's a strategy difference: the MSSQL optimizer is a bit more partial to narrow filtered indexes per type.

But the main point — architectural invariance is preserved. Same 2+2+3+infrastructure layout, same facet schema, same fact that the index set doesn't grow when you add a class. To a developer writing C# models, the difference between Postgres and MSSQL is invisible.

TSUM as the proof case

Bringing the numbers from the top of the post into one place.

What REDB serves in TSUM:

32 classes in tsum.Domain (from TransportationRoute to UserFilterPreference),
401 properties across those classes,
227,896 objects at the time these metrics were captured,
9,773,174 rows in _values.

This whole schema lands not in 13 tables per entity, but in 2 data tables (_objects + _values) + 2 lookup tables (_lists + _list_items) + 3 RTTI tables (_types + _schemes + _structures). The remaining six are infrastructure — permissions, users, roles, links, dependencies, functions, soft-delete, metadata cache — invariant of business class count. The full index set is the same redbPostgre.sql you'd ship for a five-class project. In a classical EF layout, those 32 classes with their lookups and snapshots would expand to roughly 60–80 tables and 200+ indexes, with a migration history accumulating across the project's lifetime.

Production metrics for a typical order-processing tick:

999 orders in one XML batch from SAP, processed in 991 ms total:
- sync = 482 ms — reconciling lookup dictionaries (Drivers, Vehicles, ShippingPoints, BusinessTypes, DeliveryStatuses) against REDB lists,
- query = 139 ms — one SELECT loading all 999 existing routes by their codes,
- save = 154 ms — SaveAsync for 32 changed objects (2 created + 30 updated). The other 967 are short-circuited by ComputeHash() before they reach the database.
Infrastructure — 2 cores, stock PostgreSQL settings (no shared_buffers tuning, no pg_prewarm, no bumped work_mem).
No REDB-framework-level cache — every Query<T>() round-trips to the database.

This isn't a hand-picked benchmark — it's a real production tick where a developer writes ordinary C# models with a [RedbScheme] attribute and the logistics process just runs. Bulk on the read (one SELECT for 999 objects), bulk on the write (_objects and _values together in one transaction over the COPY protocol), early skip where data didn't change.

The technical takeaway: REDB hits these numbers not through magic, but because the engine doesn't do unnecessary work. The index plan is baked into the storage schema; bulk runs where bulk applies; application-level change tracking saves the most expensive thing — DML for changes that aren't there.

Wrap-up and what's coming next

Short summary. REDB ships one index set in DDL — about 50 indexes for the entire system. That set serves any business schema: 32 classes, 100 classes, 500 classes — same indexes. That's the inverse of the EF approach, where indexes multiply with tables and slowly become technical debt that needs periodic pruning.

Operational pattern — once your application stabilizes, run pg_stat_user_indexes, find the idx_scan = 0 candidates, and prune carefully (with DROP INDEX CONCURRENTLY for safe rollback). Add your own local indexes for hot application-specific queries — in a separate migration, leaving the engine DDL alone.

MSSQL is nearly the same picture. Minor differences in text indexing and per-type filtered indexes, but architectural invariance carries over.

Coming up in the series:

Part 2 — Code-first schemes: how SyncSchemeAsync<T> turns a C# class into rows in _schemes + _structures, what _structure_hash is, how automatic onboarding works when you add a new property.
Part 3 — CRUD internals: SaveAsync and LoadAsync from inside, change tracking via TreeDiff, COPY-protocol bulk insert, lazy loading.
Part 4 — LINQ to SQL: how Where(x => x.Salary > 80000) becomes CASE WHEN _id_structure = X THEN _Long END > 80000, how OrderBy and window functions work.
Part 5 — Object trees: LoadTreeAsync, GetDescendantsAsync, WhereHasAncestor, recursive CTEs.
Part 6 — Window functions: Win.RowNumber(), Win.Rank(), PartitionBy/OrderBy over REDB objects.

The deep C# dive is parts 3–5. In the storage layer, one topic remains — schema migrations — and that's where the next post picks up.

Earlier in the series (on dev.to)

An EF Core alternative for .NET apps with complex object graphs — full LINQ, no migrations, no DbContext — the wide-angle intro to RedBase: LINQ surface, production story, generated SQL.
REDB inside, part 1 — the 13 tables the whole engine runs on (with the actual SQL, and why it's not EAV) — the schema walkthrough this post builds on.
redb.Route 3.0.1 — flat DSL navigation, CRTP refactor, and a silent null fix — separate cycle on the ESB framework that sits on top of redb.

REDB inside, part 1 — the 13 tables the whole engine runs on (with the actual SQL, and why it's not EAV)

rinat kozin — Thu, 04 Jun 2026 18:49:10 +0000

A couple of weeks ago I published the redb.Core intro post — what RedBase is at the API level, why I wrote it, what production looks like, the LINQ surface, what generated SQL looks like for nested dictionary lookups. If you haven't read it, start there — it's the wide-angle shot.

This post starts a new series — "REDB inside" — that drills down into the engine. One article per layer:

Part 1 (this post) — the database schema. 13 tables, what each one does, why the design is what it is, and the SQL you'd run to dump any object flat.
Part 2 — code-first schemes. How SyncSchemeAsync<T> walks a C# class and turns it into rows in _schemes + _structures, the _structure_hash mechanism, automatic onboarding via InitializeAsync.
Part 3 — CRUD internals. SaveAsync, LoadAsync, the TreeDiff change-tracking algorithm, COPY-protocol bulk insert, lazy loading.
Part 4 — LINQ-to-SQL. How Where(x => x.Salary > 80000) becomes WHERE _id_structure = X AND _Long > 80000, the pivot CTE patterns, dialect differences (Postgres array_agg FILTER vs MSSQL MAX CASE WHEN).
Part 5 — trees. LoadTreeAsync, GetDescendantsAsync, WhereHasAncestor, closure-table vs recursive CTE.
Part 6 — window functions. Win.RowNumber(), Win.Rank(), PartitionBy/OrderBy over REDB objects.

Each one stands alone. You don't need to read them in order — but if you want to understand why anything in parts 2-6 works the way it does, you need this post. Everything else is built on the 13 tables.

The whole RedBase engine runs on 13 tables. No JSON blob hiding the schema, no NVARCHAR(MAX) catch-all column — every C# type lands in its own typed column. Let me show you how that works, and why this isn't classical EAV even though it kind of looks like it at first glance.

"Wait, isn't this just EAV?"

It's the most common reaction I get, and it deserves a real answer before we look at any DDL.

Classical EAV (Entity–Attribute–Value) looks like this:

object_id | attribute_name | value
----------|----------------|---------
42        | "FirstName"    | "Alice"
42        | "Age"          | "28"
42        | "Salary"       | "85000"

Everything in one table. Types erased. Attribute names are strings. Any non-trivial query becomes a self-join or a giant PIVOT. The filter "employees earning over $80k" turns into:

SELECT object_id
FROM   attributes
WHERE  attribute_name = 'Salary'
  AND  value::numeric > 80000;   -- runtime cast, no usable index

Add three more conditions and you're looking at three self-joins on the same table. Explain plans get embarrassing.

REDB doesn't do that. Here's what _values actually looks like:

CREATE TABLE _values (
    _id              bigint NOT NULL,
    _id_structure    bigint NOT NULL,   -- FK → _structures (which field this is)
    _id_object       bigint NOT NULL,   -- FK → _objects
    -- typed value columns, exactly one is non-NULL per row:
    _String          text NULL,
    _Long            bigint NULL,
    _Guid            uuid NULL,
    _Double          float NULL,
    _DateTimeOffset  timestamptz NULL,
    _Boolean         boolean NULL,
    _ByteArray       bytea NULL,
    _Numeric         numeric(38,18) NULL,
    _ListItem        bigint NULL,       -- FK → _list_items
    _Object          bigint NULL,       -- FK → _objects (cross-object reference)
    -- relational collection storage:
    _array_parent_id bigint NULL,       -- FK → _values (parent element)
    _array_index     text NULL          -- '0','1','2' for arrays, key for dictionaries
);

The field identity is a foreign key (_id_structure), not a string. The value lives in a typed column chosen at write time based on the field's declared C# type. Think of it as runtime type information (RTTI) persisted into the schema: the engine always knows what type each field is, because that's recorded once in _structures._id_type and reused for every value of that field.

Reading values back is one CASE expression dispatched on type, not a self-join:

CASE c.db_type
  WHEN 'String'  THEN v._String
  WHEN 'Long'    THEN v._Long::text
  WHEN 'Guid'    THEN v._Guid::text
  ...
END

The practical differences:

	Classical EAV	REDB `_values`
Field identity	string in `attribute_name`	FK → `_structures` → `_types`
Where the value lives	one `text`/`varchar(max)` column	typed column per C# type
Filter `Salary > 80000`	`WHERE attribute_name='Salary' AND value::numeric > 80000`	`WHERE _id_structure = $1 AND _Long > 80000`
Index on the value	string index + runtime cast	partial B-tree index on the typed column
Arrays/dictionaries	separate table or JSON blob	`_array_parent_id` + `_array_index` in the same row
Schema metadata	implicit in attribute names	first-class rows in `_schemes`/`_structures`/`_types`, denormalized into a metadata cache

So yes, the row shape rhymes with EAV — but the type system and the indexing story are completely different. That's why I've been pushing back on the EAV label.

The 13 tables, at a glance

_types          — type catalog (~37 system rows)
_schemes        — schemes (C# classes mapped to DB rows)
_structures     — fields of schemes (with nesting and collection metadata)
_objects        — objects (data rows, tree-shaped via self-FK)
_values         — field values (typed columns + relational collections)
_lists          — pick-list/dictionary catalog
_list_items     — pick-list entries
_users          — users (system IDs −1, 0, 1)
_roles          — roles
_users_roles    — M2M user ↔ role
_permissions    — permissions on objects (inherited along the tree)
_links          — M2M relations between objects
_functions      — stored expressions attached to schemes
_dependencies   — cross-scheme dependencies
─────────────────────────────────────────────
_scheme_metadata_cache   — denormalized cache of structures × types
_migrations              — history of props-schema migrations

The last two live in their own SQL files but matter just as much in practice. Let's walk through them layer by layer.

Layer 1 — the type catalog: `_types`

CREATE TABLE _types (
    _id      bigint NOT NULL PRIMARY KEY,
    _name    text NOT NULL UNIQUE,
    _db_type text NULL,   -- which _values column to use: 'Long', 'String', 'Guid', ...
    _type    text NULL    -- C# type name: 'long', 'string', 'Guid', ...
);

System type IDs are negative constants near long.MinValue. A small sample:

`_id`	`_name`	`_db_type`	C# type
-9223372036854775709	Boolean	Boolean	`bool`
-9223372036854775708	DateTime	DateTimeOffset	`DateTime`
-9223372036854775704	Long	Long	`long`
-9223372036854775700	String	String	`string`
-9223372036854775695	Decimal	Numeric	`decimal`
-9223372036854775675	Class	—	nested class (marker only)
-9223372036854775668	Array	—	`T[]` / `List<T>` (marker only)
-9223372036854775667	Dictionary	—	`Dictionary<K,V>` (marker only)

The scary-looking numbers are just constants picked far from anything a user-generated key could ever hit (user IDs start at 1_000_000 via a global_identity sequence). Class, Array, and Dictionary have no column of their own in _values — they're marker types; the actual leaves live in child rows.

There are ~37 built-in types total. Numeric ones (Int, Short, Byte, Float) physically store as Long/Double. Strings include semantic variants (Email, Url, Phone) that all use _String. Then DateOnly/TimeOnly/TimeSpan, geo (Latitude/Longitude), file metadata (FilePath/MimeType), Enum/EnumInt, and collection markers (Array/Dictionary/JsonDocument/XDocument).

Layer 2 — schemes and fields: `_schemes` + `_structures`

CREATE TABLE _schemes (
    _id             bigint NOT NULL,
    _id_parent      bigint NULL,          -- nesting (namespaces)
    _name           text NOT NULL UNIQUE, -- e.g. 'MyApp.Models.EmployeeProps'
    _alias          text NULL,
    _structure_hash uuid NULL,            -- field hash for fast change detection
    _type           bigint NOT NULL       -- FK → _types (Class by default)
);

CREATE TABLE _structures (
    _id              bigint NOT NULL,
    _id_parent       bigint NULL,     -- nested props class
    _id_scheme       bigint NOT NULL, -- FK → _schemes
    _id_type         bigint NOT NULL, -- FK → _types
    _id_list         bigint NULL,     -- FK → _lists (for ListItem fields)
    _name            text NOT NULL,   -- C# property name
    _alias           text NULL,
    _order           bigint NULL,
    _collection_type bigint NULL,     -- NULL=scalar, Array_ID or Dictionary_ID
    _key_type        bigint NULL,     -- key type for Dictionary<K,V>
    _readonly        boolean NULL,
    _allow_not_null  boolean NULL,
    _is_compress     boolean NULL,
    _store_null      boolean NULL,
    _default_value   bytea NULL
);

When you write:

[RedbScheme("Employee")]
public class EmployeeProps
{
    public string FirstName            { get; set; } = "";
    public int    Age                  { get; set; }
    public decimal Salary              { get; set; }
    public string[]? Skills            { get; set; }
    public Address? HomeAddress        { get; set; }   // nested class
}

…and await redb.SyncSchemeAsync<EmployeeProps>() fires for the first time, the engine:

Inserts a row into _schemes with _name = "MyApp.EmployeeProps".
Inserts one _structures row per public property.
For Skills: sets _collection_type = Array_ID.
For HomeAddress: sets _id_type = Class_ID and recursively creates child _structures rows whose _id_parent points back at the parent structure.

Then it computes a hash over all those structures and writes it to _schemes._structure_hash. Next time you call sync, comparing one UUID tells the engine whether anything actually changed — no row-by-row diff needed.

There's a DB-level trigger that validates field names: no system-reserved (_id, _name, _date_create), no C# keywords (class, int, string), no leading digits. If you accidentally try to name a property int, the INSERT throws before the bad row ever lands.

Layer 3 — objects: `_objects`

CREATE TABLE _objects (
    _id             bigint NOT NULL,
    _id_parent      bigint NULL,          -- tree parent (self-FK)
    _id_scheme      bigint NOT NULL,      -- FK → _schemes
    _id_owner       bigint NOT NULL,      -- FK → _users
    _id_who_change  bigint NOT NULL,      -- FK → _users
    _date_create    timestamptz NOT NULL,
    _date_modify    timestamptz NOT NULL,
    _date_begin     timestamptz NULL,
    _date_complete  timestamptz NULL,
    _key            bigint NULL,
    _name           text NULL,
    _note           text NULL,
    _hash           uuid NULL,
    -- value columns for RedbPrimitive<T>:
    _value_long     bigint NULL,
    _value_string   text NULL,
    _value_guid     uuid NULL,
    _value_bool     boolean NULL,
    _value_double   float NULL,
    _value_numeric  numeric(38,18) NULL,
    _value_datetime timestamptz NULL,
    _value_bytes    bytea NULL
);

Three things worth pointing out:

The tree via _id_parent is ON DELETE CASCADE. Drop a root, the whole subtree goes with it. Depth is unbounded. This is the primary organizational structure in REDB: sections, categories, folders, org charts, project trees — they're all just _objects rows pointing at a parent.

The _value_* columns are for RedbPrimitive<T>. When an object is conceptually a single primitive (e.g. RedbObject<long> for a counter, RedbObject<string> for a token), there's no need to spin up _values rows — the value rides in the object row itself. Eight columns, one per _db_type.

Soft delete is a scheme called @@__deleted (_id = -10). mark_for_deletion() walks the subtree via recursive CTE and atomically reparents everything under a trash container with _id_scheme = -10. Actual physical deletion is a separate, batched purge_trash(). This means you can offer "undelete" cheaply, and your data lake/CDC tools never see a destructive delete on the hot path.

Layer 4 — the values: `_values`

This is the table that earns its keep. Everything else exists to make this one fast and consistent.

The DDL was up top. The interesting part is how collections fit into a flat row layout without a side table.

Scalar field

Age = 28 produces exactly one row:

_id_structure=struct_Age   _id_object=42   _Long=28   _array_parent_id=NULL   _array_index=NULL

Array of primitives

Skills = ["C#", "SQL", "React"] produces a marker row plus one row per element:

-- marker: "the array property exists" (without it, the property is NULL, not [])
_id=100   _id_structure=struct_Skills   _id_object=42   _array_index=NULL   _array_parent_id=NULL

-- elements; _array_parent_id points at the marker; _array_index is the position
_id=101   _id_structure=struct_Skills   _id_object=42   _String="C#"     _array_index='0'   _array_parent_id=100
_id=102   _id_structure=struct_Skills   _id_object=42   _String="SQL"    _array_index='1'   _array_parent_id=100
_id=103   _id_structure=struct_Skills   _id_object=42   _String="React"  _array_index='2'   _array_parent_id=100

That marker row matters: it's how the engine tells null (no marker, no elements) apart from [] (marker present, zero children). The same shape works for empty dictionaries too.

Dictionary

PhoneDir = { "home": "+7 999…", "work": "+7 495…" }:

-- marker
_id=200   _id_structure=struct_PhoneDir   _id_object=42   _array_index=NULL   _array_parent_id=NULL

-- entries; _array_index holds the dictionary key (as text)
_id=201   _id_structure=struct_PhoneDir   _id_object=42   _String="+7 999..."   _array_index='home'   _array_parent_id=200
_id=202   _id_structure=struct_PhoneDir   _id_object=42   _String="+7 495..."   _array_index='work'   _array_parent_id=200

_array_index is text precisely so dictionaries with string keys work without a separate table. Numeric dictionaries store keys as their string representation.

Nested class

HomeAddress.City = "Moscow" works the same way. The _structures rows for Address.City, Address.Street, etc. carry an _id_parent pointing at the parent structure (HomeAddress). The _values rows for those leaves carry an _array_parent_id pointing at the marker row for HomeAddress on this particular object.

Three unique indexes keep all of this consistent

CREATE UNIQUE INDEX UIX__values__structure_object
    ON _values (_id_structure, _id_object)
    WHERE _array_index IS NULL AND _array_parent_id IS NULL;

CREATE UNIQUE INDEX UIX__values__structure_object_parent
    ON _values (_id_structure, _id_object, _array_parent_id)
    WHERE _array_index IS NULL AND _array_parent_id IS NOT NULL;

CREATE UNIQUE INDEX UIX__values__structure_object_array_index
    ON _values (_id_structure, _id_object, _array_parent_id, _array_index)
    WHERE _array_index IS NOT NULL;

These three together guarantee: (a) at most one scalar/marker row per (structure, object), (b) at most one nested-class marker per (structure, object, parent), and (c) at most one element per (structure, object, parent, index). Try to insert a duplicate array element and the DB rejects it before any logic bug can corrupt the shape.

Layer 5 — permissions: `_permissions`

CREATE TABLE _permissions (
    _id      bigint NOT NULL,
    _id_role bigint NULL,   -- XOR with _id_user (CHECK constraint)
    _id_user bigint NULL,
    _id_ref  bigint NOT NULL,  -- 0 = global, otherwise FK → _objects
    _select  boolean NULL,
    _insert  boolean NULL,
    _update  boolean NULL,
    _delete  boolean NULL
);

Permissions inherit along the object tree. A recursive CTE walks upwards from the target object up to 50 levels looking for the nearest ancestor that has an explicit permission row. _id_ref = 0 is the global fallback ("can this principal touch anything at all?"). Resolution priority is: user > role, specific object > global.

There's an automatic trigger that, when a child object is created without its own permission row, copies down the resolved permission from the nearest ancestor. The point isn't to materialize every permission — it's to keep the recursive CTE short. After a few months of activity the depth the resolver has to climb stays roughly constant.

`_scheme_metadata_cache` — why a cache table

Every query needs to know: "for an object with _id_scheme = X, which _structures rows exist, and what's the type of each?" That's a JOIN through _structures → _types that would otherwise fire on every single read.

So that JOIN is denormalized into a separate table that gets refreshed when the scheme changes:

CREATE TABLE _scheme_metadata_cache (
    _scheme_id           bigint NOT NULL,
    _structure_id        bigint NOT NULL,
    _parent_structure_id bigint,
    _name                text NOT NULL,
    type_name            text NOT NULL,   -- 'Long', 'String', 'Guid', ...
    db_type              text NOT NULL,   -- 'Long', 'String', 'Guid', ...
    type_semantic        text NOT NULL,   -- 'Object', '_RObject', 'Array', ...
    _collection_type     bigint,
    collection_type_name text,
    _key_type            bigint,
    key_type_name        text
    -- ... all the other _structures attributes inlined
);

A trigger on _schemes._structure_hash invalidates the cache for that scheme; on the next read, sync_metadata_cache_for_scheme(scheme_id) rebuilds it lazily. The big build_hierarchical_properties_optimized() function — the one that materializes an object's full property tree into JSON — never JOINs _structures or _types directly. It reads from this cache, and only from this cache.

Two SQL queries: dump any object flat

Here are two queries that show exactly what's in the box for a given object. The first uses raw JOINs (use this for ad-hoc debugging in DataGrip/SSMS). The second uses _scheme_metadata_cache — what the engine actually runs at scale.

Query 1 — flat pivot, no cache

-- PostgreSQL
SELECT
    s._name                                          AS scheme_name,
    st._name                                         AS field_name,
    t._name                                          AS type_name,
    CASE t._db_type
        WHEN 'String'         THEN v._String
        WHEN 'Long'           THEN v._Long::text
        WHEN 'Guid'           THEN v._Guid::text
        WHEN 'Double'         THEN v._Double::text
        WHEN 'Boolean'        THEN v._Boolean::text
        WHEN 'DateTimeOffset' THEN v._DateTimeOffset::text
        WHEN 'Numeric'        THEN v._Numeric::text
        WHEN 'ListItem'       THEN v._ListItem::text
        WHEN 'Object'         THEN v._Object::text
        WHEN 'ByteArray'      THEN encode(v._ByteArray, 'base64')
        ELSE                       NULL
    END                                              AS value_text,
    CASE
        WHEN v._array_parent_id IS NULL
         AND v._array_index     IS NULL
         AND t._name IN ('Array','Dictionary','Class')  THEN 'collection_marker'
        WHEN v._array_parent_id IS NULL
         AND v._array_index     IS NULL                 THEN 'scalar'
        WHEN v._array_parent_id IS NOT NULL             THEN 'element[' || v._array_index || ']'
        ELSE                                                 'scalar'
    END                                              AS slot,
    v._array_index,
    v._array_parent_id
FROM _values      v
JOIN _structures  st ON st._id = v._id_structure
JOIN _schemes     s  ON s._id  = st._id_scheme
JOIN _types       t  ON t._id  = st._id_type
WHERE v._id_object = :object_id
ORDER BY st._order NULLS LAST, v._array_index NULLS FIRST;

-- MS SQL Server
SELECT
    s._name                                          AS scheme_name,
    st._name                                         AS field_name,
    t._name                                          AS type_name,
    CASE t._db_type
        WHEN 'String'         THEN v._String
        WHEN 'Long'           THEN CAST(v._Long AS nvarchar(MAX))
        WHEN 'Guid'           THEN CAST(v._Guid AS nvarchar(MAX))
        WHEN 'Double'         THEN CAST(v._Double AS nvarchar(MAX))
        WHEN 'Boolean'        THEN CASE v._Boolean WHEN 1 THEN 'true' WHEN 0 THEN 'false' END
        WHEN 'DateTimeOffset' THEN CAST(v._DateTimeOffset AS nvarchar(MAX))
        WHEN 'Numeric'        THEN CAST(v._Numeric AS nvarchar(MAX))
        WHEN 'ListItem'       THEN CAST(v._ListItem AS nvarchar(MAX))
        WHEN 'Object'         THEN CAST(v._Object AS nvarchar(MAX))
        WHEN 'ByteArray'      THEN N'<binary, base64 in app code>'
        ELSE                       NULL
    END                                              AS value_text,
    CASE
        WHEN v._array_parent_id IS NULL AND v._array_index IS NULL THEN 'scalar'
        WHEN v._array_parent_id IS NOT NULL                        THEN 'element[' + ISNULL(v._array_index,'') + ']'
        ELSE                                                            'scalar'
    END                                              AS slot,
    v._array_index,
    v._array_parent_id
FROM [dbo].[_values]      v
JOIN [dbo].[_structures]  st ON st._id = v._id_structure
JOIN [dbo].[_schemes]     s  ON s._id  = st._id_scheme
JOIN [dbo].[_types]       t  ON t._id  = st._id_type
WHERE v._id_object = @object_id
ORDER BY st._order, v._array_index;

This is a diagnostic query — paste it into psql/DataGrip/SSMS, plug in any object ID, and you see exactly what's stored: which fields, which slot (scalar / collection marker / array element), which type. The marker rows for arrays show up too, which is exactly what you want when you're hunting down a null vs [] regression.

Query 2 — same result via `_scheme_metadata_cache`

-- PostgreSQL (via _scheme_metadata_cache)
SELECT
    c._scheme_id                                      AS scheme_id,
    c._name                                           AS field_name,
    c.type_name                                       AS type_name,
    c.db_type                                         AS db_type,
    c.collection_type_name                            AS collection_type,
    CASE c.db_type
        WHEN 'String'         THEN v._String
        WHEN 'Long'           THEN v._Long::text
        WHEN 'Guid'           THEN v._Guid::text
        WHEN 'Double'         THEN v._Double::text
        WHEN 'Boolean'        THEN v._Boolean::text
        WHEN 'DateTimeOffset' THEN v._DateTimeOffset::text
        WHEN 'Numeric'        THEN v._Numeric::text
        WHEN 'ListItem'       THEN v._ListItem::text
        WHEN 'Object'         THEN v._Object::text
        WHEN 'ByteArray'      THEN encode(v._ByteArray, 'base64')
        ELSE                       NULL
    END                                               AS value_text,
    v._array_index,
    v._array_parent_id,
    c._order
FROM _values                v
JOIN _scheme_metadata_cache c ON c._structure_id = v._id_structure
WHERE v._id_object = :object_id
ORDER BY c._order NULLS LAST, v._array_index NULLS FIRST;

-- MS SQL Server (via _scheme_metadata_cache)
SELECT
    c.[_scheme_id]                                    AS scheme_id,
    c.[_name]                                         AS field_name,
    c.[type_name]                                     AS type_name,
    c.[db_type]                                       AS db_type,
    c.[collection_type_name]                          AS collection_type,
    CASE c.[db_type]
        WHEN 'String'         THEN v._String
        WHEN 'Long'           THEN CAST(v._Long AS nvarchar(MAX))
        WHEN 'Guid'           THEN CAST(v._Guid AS nvarchar(MAX))
        WHEN 'Double'         THEN CAST(v._Double AS nvarchar(MAX))
        WHEN 'Boolean'        THEN CASE v._Boolean WHEN 1 THEN 'true' WHEN 0 THEN 'false' END
        WHEN 'DateTimeOffset' THEN CAST(v._DateTimeOffset AS nvarchar(MAX))
        WHEN 'Numeric'        THEN CAST(v._Numeric AS nvarchar(MAX))
        WHEN 'ListItem'       THEN CAST(v._ListItem AS nvarchar(MAX))
        WHEN 'Object'         THEN CAST(v._Object AS nvarchar(MAX))
        WHEN 'ByteArray'      THEN N'<binary>'
        ELSE                       NULL
    END                                               AS value_text,
    v._array_index,
    v._array_parent_id,
    c.[_order]
FROM [dbo].[_values]                v
JOIN [dbo].[_scheme_metadata_cache] c ON c.[_structure_id] = v.[_id_structure]
WHERE v.[_id_object] = @object_id
ORDER BY c.[_order], v.[_array_index];

Why the second one is the production query:

Two heavy JOINs (_structures, _types) gone. The cache row carries everything those joins would have produced.
The cache already has a B-tree on _structure_id and a stable _order for ordering — no extra sorts.
The cache is refreshed only when the scheme changes (_structure_hash flips), not on every read. Steady-state queries pay zero cost for it.
build_hierarchical_properties_optimized() goes one step further: it slurps all _values rows for one object into a _values[] array in a single SELECT, then walks the property tree purely in memory using unnest(). Recursion into nested classes and array elements never touches the table again. For deeply nested object graphs this is a big deal — you can materialize a 40-row, 8-level-deep object with two SELECTs total.

How this connects to the C# API

For context — what those tables look like from a SaveAsync/LoadAsync perspective. The full API tour is in the intro post; here's just the mapping back to the tables we just looked at:

[RedbScheme("Employee")]
public class EmployeeProps
{
    public string  FirstName { get; set; } = "";
    public string  LastName  { get; set; } = "";
    public int     Age       { get; set; }
    public decimal Salary    { get; set; }
    public string[]? Skills  { get; set; }
}

// InitializeAsync scans the assembly →
//   - inserts/updates rows in _schemes + _structures for each [RedbScheme]
//   - refreshes _scheme_metadata_cache on changed schemes
await redb.InitializeAsync(typeof(EmployeeProps).Assembly);

// SaveAsync →
//   - one row into _objects
//   - one row per scalar into _values (using the typed column for the C# type)
//   - for Skills: one marker row + one row per element, linked via _array_parent_id
var employee = new RedbObject<EmployeeProps>
{
    name  = "Alice Johnson",
    Props = new EmployeeProps
    {
        FirstName = "Alice",
        LastName  = "Johnson",
        Age       = 28,
        Salary    = 120_000m,
        Skills    = ["C#", "PostgreSQL", "Redis"]  // → 1 marker + 3 element rows
    }
};
long id = await redb.SaveAsync(employee);

// LoadAsync →
//   - SELECT _objects WHERE _id = id
//   - SELECT * FROM _values WHERE _id_object = id  (one shot, into a _values[])
//   - build_hierarchical_properties_optimized() materializes the C# graph
var loaded = await redb.LoadAsync<EmployeeProps>(id);

SaveAsync reads _scheme_metadata_cache to know which column to write each value to, mints IDs from the global_identity sequence, and writes _objects + _values. LoadAsync reads _objects first, then one SELECT pulls every _values row for that object into a memory array, and the recursive materializer never goes back to the database for that load.

A few design choices that aren't obvious

Negative constants for system IDs. User-generated keys come from a sequence starting at 1_000_000. System types, schemes, users live near long.MinValue. The two ranges can never collide. This means _types._id = -10 for the @@__deleted scheme isn't a special-case in any query — it's just a perfectly normal FK that happens to be negative.

_structure_hash on _schemes. Without it, every SyncSchemeAsync<T> call would have to re-read the schema structure and compare row-by-row. With it, comparing one UUID tells you whether anything changed. The cache-invalidation trigger fires only on real changes, so steady-state operation pays nothing.

Marker rows. The trickiest design choice in _values. There's no separate "collections" table — instead, a row with _array_index = NULL AND _array_parent_id = NULL and a collection-typed _id_structure is the marker, and child rows fan out from it via _array_parent_id. This is what makes null vs [] distinguishable, lets dictionaries with string keys work without a side table, and keeps nested-class hierarchies in the same physical structure as flat fields.

_Numeric NUMERIC(38, 18). Deliberate. double for money quietly loses pennies; nobody wants a 0.0000000001-off invoice total in production. The 38/18 precision/scale is more than enough for currency, percentage, weight, even small molar quantities. The cost is storage size (16 bytes vs 8), which on the kind of property volume REDB sees is noise.

Postgres vs MSSQL — the cascade-delete story. On Postgres, _values has ON DELETE CASCADE on its FK to _structures. On MSSQL the same constraint would create multiple cascade paths and SQL Server refuses to compile that. The workaround: an INSTEAD OF DELETE trigger on _structures that manually cascades into _values. Same observable behavior, different machinery. There are a handful of places in the codebase where the dialect abstraction exists specifically to paper over this kind of thing.

Series links

Intro post — An EF Core alternative for .NET apps with complex object graphs (published)
Part 1 (this post) — the 13 tables, RTTI vs EAV, _values, collection storage, _scheme_metadata_cache, two diagnostic queries
Part 2 — code-first schemes: SyncSchemeAsync<T>, _structure_hash, automatic onboarding
Part 3 — CRUD internals: SaveAsync/LoadAsync, TreeDiff change tracking, COPY-protocol bulk insert
Part 4 — LINQ → SQL: pivot CTEs, dialect splits, the OfficeLocations["HQ"].City walkthrough
Part 5 — trees: LoadTreeAsync, WhereHasAncestor, the closure-table vs recursive-CTE story
Part 6 — window functions: Win.RowNumber(), Win.Rank(), PartitionBy/OrderBy over REDB objects

Where to look

github.com/redbase-app — all repos in the ecosystem
github.com/redbase-app/redb — the redb.Core repo
Postgres schema (redbPostgre.sql) — all 13 tables + 40+ indexes + triggers + stored functions in one file
MSSQL schema (redbMSSQL.sql) — same shape, dialect-adjusted
redbase.app — docs and worked examples (EN)

Questions/critique very welcome in the comments — especially if you've built something similar and have war stories about indexing strategies on the values table, that's exactly the discussion I want to have.

redb.Route 3.0.1 — flat DSL navigation, CRTP refactor, and a silent null fix

rinat kozin — Wed, 03 Jun 2026 16:53:16 +0000

Series: redb ecosystem

Before 3.0.1, deep nested scopes required closing in exact reverse order — tedious and easy to get wrong. Three things changed.

Flat `End*()` navigation

A typed closer walks the Parent chain and exits to the right level in one call:

From("direct://demo-cascade-endchoice")
    .Choice()
        .When(e => true)
            .Split(e => new object?[] { 1, 2, 3 })
                .Process(e => { /* per-item work */ })
                .Log("item=${body}")
            .EndChoice()   // walks past Split → When → lands at route root

    .SetHeader("post-cascade", "ok")
    .Log("cascade done");

Universal .End() exits to the nearest scope without naming it:

.Choice()
    .When(e => true)
        .Split(e => new object?[] { "a", "b" })
            .Log(LogLevel.Information)
                .Message("inside")
            .End()   // closes RichLog → returns Split body
        .End()        // closes Split   → returns When body
    .EndChoice()      // closes Choice  → returns route root

Sibling branches open naturally after a closed sub-scope — .When() and .Otherwise() walk up to the enclosing ChoiceDefinition via the Parent chain, so this compiles as-is:

.Choice()
    .When(e => e.In.Body is IEnumerable<string> && e.In.Body is not string)
        .Split(...)
            .Process(...)
        .EndSplit()
        .Log("list branch done [${routeId}]")   // still on the When body, not on Split
    .When(e => e.In.Body is string s && s.Length > 0)   // ← sibling, works after EndSplit
        .Process(e => { /* ... */ })
    .Otherwise()
        .Process(e => { /* fallback */ })
.EndChoice()

Three logging styles, same pipeline step

The updated demo shows all three forms side by side — useful to see the tradeoffs:

// (A) Lambda — arbitrary C# at runtime
.Log(e => $"[lambda] item={e.In.Body} branch={e.In.Headers["branch"]}")

// (B) String template — compiled by the expression engine, zero alloc when level is off
.Log("[tmpl] item=${body} branch=${header.branch} [${routeId}]")

// (C) Rich-log scope — structured, multi-message, headers/properties as separate fields
.Log(LogLevel.Information)
    .Message("[rich-tmpl]   item=${body}")
    .Message(e => $"[rich-lambda] upper={((string)e.In.Body!).ToUpperInvariant()}")
    .Header("branch")
    .Property("item-index")
    .ShowRouteId(true)
.EndLog()

${body}, ${header.x}, ${property.y}, ${routeId}, ${exception.type}, ${exception.message} — all resolved by the compiled expression engine (Tokenizer → Parser → AST → IL). The template compiles once, runs as a cached delegate. .Message() in a rich-log scope accepts both forms simultaneously.

CRTP base — 27 duplicate class bodies removed

Every leaf method (To, Process, SetBody, Filter, Split, Transaction, ~40 more) now lives once in RouteDefinitionBase<TSelf>. Each returns TSelf — chaining stays on the concrete scope type throughout. Public API and route AST identical to 3.0.0.

Fix: `GetContext()` silently returned null inside nested scopes

IRouteDefinition.GetContext() was casting to RouteDefinition, which only matched the route root. Inside When, Loop, Traced, Catch — it returned null without throwing. Now walks Parent to the owning route. Matters if you have extension methods that read context at DSL build time.

Full demo: DeepDslShowcaseRoutes.cs · Changelog: 3.0.1 · Apache 2.0

Enterprise Integration Patterns in .NET, the deep-dive series — Part 1: the four in-memory channels (and the Exchange they carry)

rinat kozin — Mon, 01 Jun 2026 21:51:33 +0000

The earlier posts were the tour: what redb.Route is, an Apache Camel-style ESB for .NET — a fluent C# integration DSL with 22 connector projects (~30 URI schemes once you count the https/wss/es variants), ~30 Enterprise Integration Patterns implemented natively across 41 processors, 8 in-process components, a compiled expression engine, and a pluggable marshal/unmarshal layer. Enough touring.

This starts a deep-dive series — one piece at a time, the actual DSL, the actual semantics, the gotchas that bite you the first time. Not a feature list. A working manual. The series runs on four parallel tracks; you can follow whichever maps to the problem in front of you.

Track A — the in-process foundation (8 components)

The wires and the message model everything else stands on. Small surface, deep behavior.

The four channels + the Exchange — direct, direct-vm, seda, vm, and the object they carry. (this post — the foundation)
Scheduling & test components — timer, plus quartz/cron built on the same idea, and the log / mock / validator components you'll actually lean on in tests.

Track B — the Enterprise Integration Patterns (~30, across 41 processors)

Grouped the way Hohpe & Woolf's Enterprise Integration Patterns groups them, one article per cluster, each dissected against the shipped processor. (The codebase has 41 processors total; a handful — To, Log, Delegate, RoutePolicy — are plumbing rather than named patterns, so the honest count of distinct EIP is around thirty plus the 6 load-balancer strategies.)

Message Routing — Choice (Content-Based Router), Filter, Splitter (+ streaming splitter), Aggregator, Resequencer, Multicast, RecipientList, ScatterGather, DynamicRouter, LoadBalance (Round-Robin / Random / Weighted / Sticky / Failover).
Message Transformation — Marshal/Unmarshal and the data-format registry, ConvertBody, Enrich/PollEnrich (Content Enricher), ClaimCheck, StreamCaching, Normalize, and Transform/SetBody/SetHeader over the expression engine.
Messaging Endpoints — IdempotentConsumer, WireTap, competing consumers (seda/vm concurrentConsumers), Pipeline, request/reply (InOut), Bean<T>.
System Management & Reliability — OnException / TryCatch / DeadLetterChannel, CircuitBreaker, Throttle / KeyedThrottle, Delay / Debounce / Sampling, Timeout, Loop, Saga (+ compensation), RoutePolicy, Transaction/Transacted, and Validate (JSON Schema / XSD).

Track C — the expression & predicate engine (its own article, and it earns it)

The one piece every other track quietly depends on. I almost wrote it off as string interpolation in the roadmap — then I read the source. It's a genuine compiled little language: Tokenizer → Parser → an AST (BinaryOperationNode, UnaryOperationNode, FunctionCallNode, TernaryNode, PostfixOperationNode, property/index access), lowered to System.Linq.Expressions and .Compile()d to real IL — not tree-walked at runtime. There's a dedicated article because there's a dedicated language:

The ${...} Simple-style template isn't formatting — it's the entry point to the whole grammar: arithmetic (+ - * /), comparison, AND/OR/XOR/NOT, the ternary ?:, even postfix ++/--.
~22 built-in functions, mined from the AST's FunctionCallNode: string (concat, upper, lower, trim, substring, replace, length, contains, startswith, endswith), numeric (abs, round, min, max), aggregates (sum, avg, count), date math (now, dateformat, dateadd), and the two that bridge into the structured world: jpath(...) and xpath(...).
JSONPath and XPath as first-class expression types — typed (TypedJsonPathExpression, TypedXPathExpression) and precompiled (CompiledJPathExpression, CompiledXPathExpression) when the path itself is static.
The predicate builders — the fluent half, for .Filter/.Choice/.Validate: isEqualTo, isNotEqualTo, isGreaterThan/isLessThan (…OrEqualTo), isBetween, contains, startsWith, endsWith, regex, In(...), isNull/isNotNull, composed with and/or/not. Seventeen of them, each a real IPredicate, not a closure.
Four separate ConcurrentDictionary compile-and-cache pools (template / property-resolver / logical / value) so every one of the above compiles once per distinct expression string and runs as a cached delegate forever after.

That's why it's its own installment, and honestly why it might be two: the template grammar and the predicate DSL are different front-ends onto the same compiler.

Track D — the 22 connectors, one article each

One focused article per connector, every example lifted from real production routes, not toy snippets:

Brokers & messaging — kafka, rabbitmq, amqp, asb (Azure Service Bus), wmq (IBM MQ), mqtt.
RPC & web — http/https, grpc, signalr, ws/wss, tcp.
Data & storage — sql, redis, elasticsearch, s3.
Files & transfer — file, ftp, sftp.
Enterprise & misc — smtp/pop3/imap (mail), ldap, fcm/fstore/fbstorage (Firebase), qtimer/cron (Quartz).

We start with Track A on purpose. Every pattern in Track B, every expression in Track C, and every connector in Track D is built on two things: a channel that carries a message from one route segment to the next, and the Exchange that is the message. Get these two right and the rest of the series is just composition. Get them wrong and you'll spend an afternoon wondering why your transaction silently didn't roll back.

A pipeline in redb.Route is From → [processors] → To. Let's look at what actually flows through it.

The Exchange — the heart, and it isn't simple

Everything that moves through a route is an IExchange. Not a byte[], not your DTO — an Exchange, a small object with more going on inside it than the name suggests. Here's the shape that matters:

public interface IExchange : IAsyncDisposable
{
    IMessage  In  { get; set; }     // the primary message — always present
    IMessage? Out { get; set; }     // the reply — lazy, null until you need it
    ExchangePattern Pattern { get; set; }   // InOnly (default), InOut, or OutOnly

    IDictionary<string, object?> Properties { get; }   // route-level metadata
    Exception? Exception { get; set; }                 // error state, in-band
    bool ExceptionHandled { get; set; }
    string ExchangeId { get; }                         // identity, stable across clones

    IExchange Clone();                                 // deep-ish copy + NEW DI scope
    IServiceProvider? ServiceProvider { get; }         // per-exchange DI scope
}

And the message it carries:

public interface IMessage
{
    object? Body { get; set; }                        // your payload — any object, or null
    string? ContentType { get; set; }
    IDictionary<string, object?> Headers { get; }     // metadata that DOES travel to brokers
    T? GetHeader<T>(string key);
    IMessage Clone();
}

Five things about this object decide how every channel and every pattern behaves. None of them are obvious from the type signature.

1. `In` vs `Out`, and the `Pattern`

In is the message coming through. Out is the reply, and it's lazy — for the default InOnly pattern it stays null and is never allocated. It only appears when a processor explicitly sets it (request/reply, or .Respond()).

There are three patterns, not two — this is the Apache Camel 2.x model, ported wholesale:

public enum ExchangePattern
{
    InOnly  = 0,   // fire-and-forget. Producer result is written to In; Out stays null. (default)
    InOut   = 1,   // request/reply. Original preserved in In, response written to Out.
    OutOnly = 2,   // explicit response via .Respond(); the RPC reply is taken from Out.
}

Here's the part the type signature hides, and the one place people get it wrong: HasOut does not tell you where the answer is. Even on an InOut exchange, a processor isn't obligated to populate Out — if it just mutates In.Body, the result lives in In. So the framework never trusts HasOut to find a reply. It reads Out ?? In, every time:

// ProducerTemplate.RequestBody — the canonical reply-extraction rule
exchange.Pattern = ExchangePattern.InOut;
await producer.Process(exchange);
return exchange.Out?.Body ?? exchange.In.Body;   // Out if present, otherwise In

This is verbatim how Camel's ProducerTemplate extracts a result (getResultMessage: "has Out → use Out, else In"). Copy that rule into your own code — exchange.Out ?? exchange.In — and you'll never chase a reply that quietly stayed in In. HasOut is a fact about allocation, not about where the data is; don't use it to route on the answer.

One honesty note for the JVM crowd: the live Out message and the OutOnly pattern are Camel 2.x semantics. Camel 3+ deprecated getOut()/setOut() and collapsed the pattern set toward InOnly/InOut, precisely because a separate Out message copied headers and bred subtle bugs. redb.Route keeps the fuller 2.x model on purpose — but if you're coming from modern Camel, that's the difference you'll notice first.

2. `Properties` vs `Headers` — the distinction that leaks bugs

Both are IDictionary<string, object?>. They are not interchangeable:

In.Headers travel with the message to the broker. Put a correlationId here and Kafka/RabbitMQ carry it downstream.
exchange.Properties are route-level metadata — RouteId, transaction markers, your own scratch state. They do not leave the process (the interface XML doc says exactly that: "Does NOT travel to brokers — use In.Headers for that"). Stash a DbContext handle or a retry counter here.

Put a value in the wrong dictionary and it either fails to reach the consumer (you used Properties) or leaks internal state onto the wire (you used Headers). The compiler won't catch it; both are just string-keyed dictionaries. Knowing which is which is half of using the framework correctly.

Read either with the typed accessors instead of casting by hand:

var attempt = exchange.GetProperty<int>("retryCount");     // route-level, stays in-process
var corr    = exchange.In.GetHeader<string>("correlationId"); // travels to the broker

The well-known keys the framework itself writes

This is the part that makes the section concrete, and it's the answer to "what's actually in Properties?" The pipeline and the processors populate a set of reserved keys as your exchange flows. There is no single ExchangeProperties constants file — they live next to the processor that owns each one — but here is the real registry, mined from source.

exchange.Properties — route-level, never leave the process:

Key	Constant in code	Written by	Meaning
`TRANSACT_ACTION`	`TransactedProcessor.TransactActionPropertyKey`	`.Transacted()`	the transacted-action stack for synchronization
`TRANSACTION_SCOPE`	`BeginTransactionProcessor.ScopePropertyKey` (internal)	`.Transaction()`	the live `TransactionScope` for the block
`CamelDuplicateMessage`	`IdempotentConsumerProcessor.DuplicatePropertyKey`	Idempotent Consumer	`true` when the message was seen before
`ClaimCheck.Stack`	`ClaimCheckHeaders.StackPropertyKey` (internal)	Claim Check	the stack of stored payload keys
`ValidationErrors` / `ValidationResult`	`ValidateProcessor.*Property`	`.Validate()`	validation outcome for the current exchange
`CamelSplitSize`	— (streaming splitter)	streaming Splitter	running count of split parts
`__redb_scope:*`	— (prefix)	DI plumbing	named child DI scopes, freed by `ReleaseScopes()`

Plus RouteId is promoted to a first-class property on the exchange (exchange.RouteId), not just a dictionary entry — that's what the logger prints as [rId:…].

In.Headers — travel with the message:

The Camel-compatible message headers come from the same world Camel users expect. The Splitter, for example, stamps every part:

// SplitterProcessor — each split message carries its coordinates
splitMessage.Headers["CamelSplitIndex"]    = index;          // 0-based position
splitMessage.Headers["CamelSplitSize"]     = total;          // batch size
splitMessage.Headers["CamelSplitComplete"] = index == total - 1; // last one?

and every transport contributes its own namespaced header constants — KafkaHeaders (redbKafka.Topic, redbKafka.Partition, redbKafka.Offset, …), SqlHeaders (redbSql.rowCount, redbSql.generatedKeys, …), SignalRHeaders (redbSignalR.ConnectionId, …), TcpHeaders, WsHeaders, ElasticsearchHeaders. Each is a static class of public const string so you bind against KafkaHeaders.Offset, not a stringly-typed "redbKafka.Offset" you might misspell. The rule of thumb: anything prefixed Camel* or redb<Transport>.* is a header (on the wire); anything in Properties is yours and the process's alone.

3. The exception travels in-band

When a processor throws, the exception doesn't just unwind the stack — it's captured onto exchange.Exception, with an ExceptionHandled flag beside it. That's what makes a dead-letter route able to branch on why something failed (when e.Exception is TimeoutException → …) instead of just that it failed. The error becomes data you can route on. We lean on this hard in the error-handling installment.

4. `ExchangeId` is stable across clones

Each exchange gets a Guid-based ExchangeId at creation. The non-obvious part: Clone() preserves it. A split into 500 parts, or a seda hop that clones the exchange, keeps the same id — so your logs and traces stitch the whole flow back to one origin. Identity survives copying; that's deliberate.

5. The DI scope — and four ways to copy an exchange

This is the part that's genuinely not simple, and it's the reason the channels behave the way they do.

An Exchange can own a DI scope — a per-message IServiceScope. Processors resolve scoped services (DbContext, IRedbService, …) from exchange.ServiceProvider, and they get the same instances for the lifetime of that exchange. A TransactionScope lives in exactly that scope. So the question "are these two exchanges in the same transaction?" reduces to "do they share a DI scope?"

There are four ways an exchange gets copied, and they differ only in what they do with that scope:

Method	Body/Headers	DI scope	Owns scope?	Use
`Clone()`	copied	new scope	yes	hand-off to another thread (`seda`, `vm`)
`CloneLinked()`	copied	shares parent's	no	parallel fan-out inside the parent's transaction
`CreateChild(msg)`	new message	new scope	yes	a derived exchange, independent lifetime
`CreateLinkedChild(msg)`	new message	shares parent's	no	sequential children reusing the same connection

// from Exchange.Clone() — the scope-creating branch
if (_scopeFactory != null)
{
    clone._scopeFactory = _scopeFactory;
    clone._scope = _scopeFactory.CreateScope();   // <-- a brand-new scope
}

// from Exchange.CloneLinked() — the scope-sharing branch
_ownsScope = false,
_scope = _scope,            // <-- the SAME scope, and we won't dispose it
_scopeFactory = _scopeFactory

Hold onto that table. The entire transaction story of seda vs direct, and of Multicast vs a broker hop, is just which row got used. Clone() (new scope) means a new transaction; CloneLinked() (shared scope) means the same one.

There's also ReleaseScopes() — it disposes the DI scopes without touching the Body, so an aggregator can free database connections early while still holding the message data it's accumulating. And DisposeAsync() cleans up both the body (streams, stream caches) and the scopes. The object is IAsyncDisposable for a reason.

The gotcha: `Clone()` does NOT deep-copy the Body

Read Message.Clone() literally:

public IMessage Clone()
{
    var clone = new Message(Body) { ContentType = ContentType };  // Body reference copied
    foreach (var kvp in _headers)
        clone._headers[kvp.Key] = kvp.Value;
    return clone;
}

Headers get a fresh dictionary. Body is copied by reference. After a seda hop the producer's exchange and the worker's clone have independent headers, properties, and DI scopes — but they point at the same body object. If that body is a mutable List<T> or a POCO and both sides write to it, you have a data race the cloning looks like it prevented. The XML doc says "deep copy"; the honest truth is "deep copy of everything except the payload." Treat the body as immutable once it's in flight, or clone it yourself.

Two APIs on purpose

One last thing you'll notice in IntelliSense: every member has a C# idiomatic form (In, Body, GetHeader<T>) and a Java-style alias (getIn(), setBody(), getHeader<T>()). They're default interface methods over the same state, kept so the model reads the same as Apache Camel for anyone coming from the JVM. Use whichever; they're the same object.

Now — the wires that carry this thing.

The four channels — two axes

direct, direct-vm, seda, vm are how route segments talk to each other inside a process. Choosing between them is the single most common thing newcomers get wrong, and now you have the vocabulary for why: it comes down to threading and scope. They split on two axes — sync vs async, and one context vs across contexts:

Scheme	Sync/Async	Scope	Clones the exchange?	Same transaction?
`direct://`	synchronous	one context	no	yes — same thread, same scope
`direct-vm://`	synchronous	across contexts	no	yes
`seda://`	asynchronous	one context	yes (`Clone()`)	no — new scope
`vm://`	asynchronous	across contexts	yes	no

direct is the contrast. seda is where the real work — and the real footguns — live, so that's where we'll spend the page.

`direct://` — a method call wearing a URI

direct is not a queue. No thread, no buffer. A producer sending to a direct endpoint invokes the consumer's processor synchronously, on the same thread:

// the whole of DirectProducer.Process
var processor = _endpoint.ConsumerProcessor
    ?? throw new InvalidOperationException("No consumer registered for direct endpoint ...");
await processor.Process(exchange, ct);

The exchange is not cloned. Same object, same thread, same DI scope — straight to the consumer. Three consequences follow:

Exceptions propagate back to the caller. A throw in the direct consumer surfaces in the producer's route, where OnException/DoTry can catch it. (Remember §3 — it also lands on exchange.Exception.)
It's the same transaction. Same scope from the table above, so a direct hop inside a .Transaction() block commits and rolls back with everything around it.
The consumer must be started first, or the send throws. direct decouples your route definitions into named sub-routes — not your threads.

That's the entire personality of direct: a zero-cost, in-transaction call you can give a URI and reuse. It has no parameters because it has no machinery. Use it to break a big route into readable, reusable pieces.

`seda://` — the async queue, in detail

seda (Staged Event-Driven Architecture) is the opposite of direct in every cell of the table. It's a real in-memory queue built on System.Threading.Channels. The producer enqueues and returns immediately; one or more background workers drain the queue on their own threads.

// SedaProducer.Process — the whole thing
var copy = exchange.Clone();                       // §5: new scope. the trap lives here.
await _endpoint.Queue.Writer.WriteAsync(copy, ct);

Two facts are baked into those two lines, and everything else about seda follows from them: it clones (so the worker and producer never share scope — that Clone() is row 1 of the table, a new scope), and it returns before the work is done (so the producer's thread, and its transaction, move on).

The parameters

seda takes three, all on the URI: seda://name?concurrentConsumers=4&size=1000&timeout=30000.

Parameter	Default	What it does	When to change it
`concurrentConsumers`	`1`	Number of worker loops draining the queue in parallel	Raise when the downstream is slower than the inflow and order doesn't matter
`size`	`0` (unbounded)	Max queued exchanges. `0` = grow without limit; `>0` = bounded with `Wait` backpressure	Set a bound whenever the producer can outpace the consumer (almost always, in production)
`timeout`	`30000`	Declared as the enqueue wait for a bounded queue — see the honesty note below	—

`concurrentConsumers` — throughput, at the cost of order

One worker is the default and keeps strict FIFO. Raising it spins up N independent loops:

// SedaConsumer.RunAsync
_workers = new Task[_options.ConcurrentConsumers];
for (var i = 0; i < _options.ConcurrentConsumers; i++)
    _workers[i] = WorkerLoop(pollCt, processingCt);

// each worker
await foreach (var exchange in _endpoint.Queue.Reader.ReadAllAsync(pollCt))
{
    await ProcessWithTracking(exchange, processingCt);
    Interlocked.Increment(ref _processedCount);
}

Two consequences worth stating plainly:

You trade ordering for throughput. With concurrentConsumers=1 the channel runs in SingleReader mode (a real optimization in System.Threading.Channels) and messages come out in order. With N>1, N workers pull concurrently and strict FIFO is gone — message 2 can finish before message 1. Only raise it when out-of-order processing is acceptable.
It's per-endpoint backpressure relief: a slow downstream stops blocking the upstream producer, because the producer only ever writes to the queue and leaves.

`size` — bounded vs unbounded, and why you almost always want bounded

This is the parameter people skip and regret. The endpoint picks the channel implementation off size:

Queue = options.Size > 0
    ? Channel.CreateBounded<IExchange>(new BoundedChannelOptions(options.Size)
      {
          FullMode = BoundedChannelFullMode.Wait,    // producer awaits a free slot
          SingleReader = options.ConcurrentConsumers == 1,
          SingleWriter = false
      })
    : Channel.CreateUnbounded<IExchange>(/* ... */); // grows until you run out of memory

size=0 (default, unbounded): the queue grows as fast as producers write. If the consumer can't keep up, that's an unbounded memory leak with extra steps. Fine for bursty, bounded-volume work; dangerous for a firehose.
size>0 (bounded): FullMode = Wait means a full queue makes the producer await a free slot — backpressure that pushes the slowdown upstream instead of into your heap. This is what you want in production.

// bounded SEDA: 4 workers, at most 1000 queued, producer waits when full
From("seda://enrich?concurrentConsumers=4&size=1000")
    .Process(async (ex, ct) => await Enrich(ex, ct))
    .To("rabbitmq://enriched");

`timeout` — an honesty note

The options object documents timeout (default 30000 ms) as the enqueue wait for a bounded queue. Being straight with you: in the current code the producer enqueues with WriteAsync(copy, ct) and does not apply that timeout — a full bounded queue makes the producer wait on the channel until a slot frees or the CancellationToken fires, not until 30 seconds elapse. So today, plan around size and the cancellation token; treat timeout as declared-but-not-yet-wired and don't build a deadline assumption on it. (Flagging it here because guessing at framework behavior from a doc-comment is exactly how you ship a bug.)

Graceful shutdown — `seda` drains, it doesn't drop

When a route stops, seda doesn't throw away what's already queued:

// SedaConsumer.OnStopAccepting
_endpoint.Queue.Writer.TryComplete();   // stop accepting new; let readers finish

Completing the writer makes the workers' ReadAllAsync loop finish the remaining items and then exit cleanly (SedaConsumer is a DrainableConsumer). On a graceful stop, in-flight queued exchanges are processed, not lost.

The durability caveat

seda is in-memory and non-durable. A graceful stop drains; a crash or a hard kill does not — whatever was sitting in the channel is gone. seda is at-most-once across a process restart. When you need the queue to survive a restart, that's a broker (rabbitmq, kafka), not seda. seda is for decoupling within a process, not for durability.

The transaction trap, stated once and for all

Now §5 pays off. Because seda calls Clone() — row 1, a new DI scope on a different thread — anything past a seda:// hop is not in the caller's transaction.

From("kafka://orders")
    .Transaction()
        .To(Sql.Execute("INSERT …").Transacted())
        .To("seda://post-process")     // runs in a NEW scope, on another thread,
    .EndTransaction();                 //    OUTSIDE this transaction

If post-process throws, the INSERT above has already committed — the seda hop left the transaction the moment it cloned. This is the one mistake everyone makes exactly once. The fix is the table: if the hop must share the transaction, use direct (no clone, same scope); if you genuinely want to hand the work off and move on, seda is correct and you accept the new boundary. The DSL is the same; the scope is everything.

Mental model: direct = function call, seda = mailbox. One preserves your thread and your transaction; the other trades both for throughput and isolation.

`direct-vm://` and `vm://` — the same two, across module boundaries

In a multi-module host (this is how redb.Tsak runs several modules in one process) each module is its own RouteContext. Plain direct and seda are scoped to a single context — a producer in module A can't see a direct consumer in module B. The -vm variants lift exactly that wall by sharing the processor registry, and for vm the channel, through a SharedVmRegistry DI singleton:

direct-vm:// — synchronous, cross-context, no clone. A consumer in the billing module exposes direct-vm://charge; a producer in orders calls it like a local in-transaction method.
vm:// — asynchronous, cross-context, cloned-and-queued. The cross-module twin of seda, with the same concurrentConsumers and size parameters (and the same Clone(), so the same transaction boundary and the same shared-Body caveat).

The rule transfers cleanly: direct-vm for a synchronous cross-module call that shares the caller's transaction; vm for hand-it-off-and-move-on across modules. Same semantics as their in-context twins — just a wider blast radius.

Picking a channel

The whole decision in one table:

You want…	Channel
A reusable sub-route, same thread, inside my transaction	`direct://`
The above, but the consumer lives in another module	`direct-vm://`
To hand work off to a background worker and not wait	`seda://`
The above, across module boundaries	`vm://`
Survival across a process restart	not a channel — a broker (`rabbitmq`, `kafka`)

And the two facts that drive every row: does it clone (new scope = new transaction), and does it return before the work is done. Everything else is detail.

What's next

That's the foundation. You now know what flows through a route (Exchange — In/Out, Properties vs Headers, in-band exceptions, and the four scope-aware clone variants) and the four wires that carry it (direct/direct-vm sync-and-in-transaction, seda/vm async-and-isolated), down to the parameter that decides whether your queue applies backpressure or eats your heap.

Next in the series — Part 2: Splitter + Aggregator. We fan one message into many, process them with bounded parallelism, and re-assemble — and the Clone() vs CloneLinked() distinction from §5 turns out to be the whole story of whether the split shares the parent's transaction. Plus the aggregation-strategy contract where the first call hands you a null accumulator. Subscribe to the series if you want it when it lands.

If anything here fought you — especially the seda transaction boundary or the shared-Body clone — say so in the comments. That feedback is exactly what an early OSS release is for.

Links


redb.Route	github.com/redbase-app/redb-route
redb.Tsak (runtime / multi-module host)	github.com/redbase-app/redb-tsak
redb.Core (storage)	github.com/redbase-app/redb
Discussions	github.com/redbase-app/redb-route/discussions

All Apache 2.0. Questions in the comments are welcome — especially "does channel X do Y?", because that's how the docs get written.

RedBase / redb.Route / redb.Tsak 3.0.0 shipped

rinat kozin — Sun, 31 May 2026 14:26:56 +0000

redb hit 20k NuGet downloads

Three packages, one version bump.

RedBase 3.0.0 — Free tier now generates the same pivot CTE as Pro. array_agg FILTER on PostgreSQL, MAX CASE WHEN on MSSql. Same base-field pushdown, same index path. MSSql Free went from 0 to 145/145 parity tests. Auto-deployed SQL bundle on version mismatch — no DBA in the loop.

redb.Route 3.0.0 — the entire v1/v2 dual compiler stack is gone. One canonical RouteDefinition, AST built from IProcessorDefinition nodes. Dynamic endpoints (ToD, dynamic WireTap/Enrich), string-template DSL (${header.x}, ${body}), full OnException fluent parity with Apache Camel.

redb.Tsak 3.0.0 — runtime container for redb.Route pipelines. Rebuilt on Core 3.0.0 + Route 3.0.0. No breaking changes in Tsak's own API.

Upgrade: package bump, no migrations.

https://dev.to/rinat_kozin_d0a2ef43e7824/why-our-free-was-so-far-behind-pro-and-what-we-just-shipped-in-redbase-300-with-the-actual-sql-i7b

https://redbase.app/architecture

https://github.com/redbase-app/redb/blob/main/CHANGELOG.md
https://github.com/redbase-app/redb-route/blob/main/CHANGELOG.md
https://github.com/redbase-app/redb-tsak/blob/main/CHANGELOG.md

Why our Free was so far behind Pro — and what we just shipped in RedBase 3.0.0 (with the actual SQL)

rinat kozin — Fri, 29 May 2026 23:18:10 +0000

Since the launch posts I keep getting the same question, in DMs, issues, private chats:

"Why is the Free version so different from Pro? It feels like two different products."

It's a fair question, and the honest answer isn't a marketing answer — it's a "this is how the codebase grew over the years" answer. So let me just tell you, and then show you the SQL — both tiers, with the actual parameter values, so you can see exactly what hits the database.

This is one post. There won't be a second one. I'd rather over-explain once than dribble out a series.

30-second background

RedBase is a typed object store. You write a C# class, decorate it with [RedbScheme("name")], the engine syncs the scheme into PostgreSQL or MSSql, and you query with normal LINQ:

[RedbScheme("Employee")]
public class EmployeeProps
{
    public string Name { get; set; } = "";
    public int Age { get; set; }
    public string City { get; set; } = "";
    public Address HomeAddress { get; set; } = new();
    public Dictionary<string, Address> OfficeLocations { get; set; } = new();
    public RedbListItem? Status { get; set; }
    public DateTimeOffset DateCreate { get; set; }
}

var employees = await service.Query<EmployeeProps>()
    .Where(p => p.Age > 30 && p.City == "London")
    .OrderBy(p => p.Name)
    .ToListAsync();

Storage is six tables: _objects, _values, _structures, _schemes, _list_items, _users. One row per object lives in _objects. Each property of that object lives as one or more rows in _values, keyed by (_id_object, _id_structure, _array_index). So filtering by N properties means joining/pivoting N rows back into one logical row per object.

How you do that pivot defines the performance ceiling of the engine. That's the part 3.0.0 fixes for the Free tier.

How we got here — the two paths that existed before 3.0.0

Pro path (since ~2024) — compiled in C

Pro walks the LINQ expression tree in process via ExpressionToSqlCompiler + SqlExpressionVisitor, registers bind variables through SqlParameterCollector, and emits a single PVT (pivot) CTE per query. For the LINQ above, with _id_scheme=42, structure id 101=Age, 102=City:

-- Pro on PostgreSQL — exact text Npgsql sends to the server
WITH pvt_cte AS (
  SELECT v._id_object,
    (array_agg(v._Long)   FILTER (WHERE v._id_structure = $1 AND v._array_index IS NULL))[1] AS "Age",
    (array_agg(v._String) FILTER (WHERE v._id_structure = $2 AND v._array_index IS NULL))[1] AS "City"
  FROM _values v
  JOIN (SELECT _id FROM _objects WHERE _id_scheme = $3) o ON v._id_object = o._id
  WHERE v._id_structure = ANY($4)
  GROUP BY v._id_object
)
SELECT o.* FROM _objects o
JOIN pvt_cte pvt ON o._id = pvt._id_object
WHERE pvt."Age" > $5 AND pvt."City" = $6
ORDER BY (array_agg(v._String) FILTER (WHERE ...))[1];

-- bound parameters (NpgsqlParameter, never interpolated into text):
--   $1 = 101         (bigint, structure id of Age)
--   $2 = 102         (bigint, structure id of City)
--   $3 = 42          (bigint, scheme id)
--   $4 = {101, 102}  (bigint[])
--   $5 = 30          (bigint)
--   $6 = 'London'    (text)

What this gets you:

One index hit on (_id_structure, _id_object) — _id_structure = ANY($4) uses the structure-id covering index, narrowing to only the props the query touches.
One GROUP BY pass — FILTER (WHERE …) is PostgreSQL's way of splitting values into columns during aggregation. Each scalar pivot field costs one array_agg aggregator, all running in the same scan. The [1] subscript picks the single element because scalar fields have at most one _values row.
Outer WHERE on flat columns — pvt."Age" > $5 is a normal B-tree comparison on the CTE's projected columns. The optimizer reorders these freely.
Stable parameterized SQL text — same $1..$N placeholders every call, so PostgreSQL's prepared-statement plan cache works. Connection pools love this.

5 fields or 50 — same query shape, same cost class.

And here is the same query through Free 3.0.0 on PostgreSQL — byte-for-byte the same shape:

-- Free 3.0.0 on PostgreSQL — emitted by pvt_build_query_sql(scheme, facets_jsonb)
WITH pvt_cte AS (
  SELECT v._id_object,
    (array_agg(v._Long)   FILTER (WHERE v._id_structure = 101 AND v._array_index IS NULL))[1] AS "Age",
    (array_agg(v._String) FILTER (WHERE v._id_structure = 102 AND v._array_index IS NULL))[1] AS "City"
  FROM _values v
  JOIN (SELECT _id FROM _objects WHERE _id_scheme = 42) o ON v._id_object = o._id
  WHERE v._id_structure IN (101, 102)
  GROUP BY v._id_object
)
SELECT o.* FROM _objects o
JOIN pvt_cte pvt ON o._id = pvt._id_object
WHERE pvt."Age" > 30 AND pvt."City" = 'London';

-- The literals you see here are NOT string-concatenated user input.
-- They are emitted by format(..., %L) with explicit type casts inside
-- the plpgsql builder. The C# caller made two parameterized roundtrips:
--   1) SELECT pvt_build_query_sql($1, $2::jsonb, $3, $4)   -- returns SQL text
--        with ($1=schemeId, $2=facets, $3=limit, $4=offset) as Npgsql parameters
--   2) EXECUTE the returned text on the connection → collect _id values
--   3) SELECT * FROM get_objects_json($1::bigint[], $2)    -- hydrate by ids
--        with ($1={...matched ids...}, $2=maxDepth) as Npgsql parameters
-- See the SQL-injection section below for the full chain.

Same CTE, same array_agg FILTER, same outer flat WHERE. The only structural difference: the SQL text is built once per call by pvt_build_query_sql inside the database, so the outer EXECUTE runs a unique string each call — values come baked in as %L-quoted literals with type casts (30, 'London') rather than as $N placeholders. That has plan-cache implications versus Pro (the SQL text varies, so PG can't reuse a plan across calls with different values), but the index path and join shape the planner picks are identical. For the parameterized story, see how this gets called from any language a few sections down.

Free path (until 3.0.0) — interpreted on the database side

The Free path serialized the LINQ filter into a JSON facet structure, shipped it to a plpgsql function, and that function built dynamic SQL inside the database. For the same LINQ above:

// C# side just packs the facets and calls one function
var facets = new {
    @and = new object[] {
        new { Age  = new { gt = 30 } },
        new { City = new { eq = "London" } },
    }
};
await conn.QueryAsync("SELECT * FROM search_objects_with_facets($1, $2)",
                      schemeId, facets);

The function then emitted, for those exact arguments:

-- Old Free on PostgreSQL — N props in WHERE → N correlated EXISTS
SELECT o.*
FROM _objects o
WHERE o._id_scheme = 42
  AND EXISTS (
    SELECT 1 FROM _values v
    WHERE v._id_object = o._id
      AND v._id_structure = 101
      AND v._Long > 30
  )
  AND EXISTS (
    SELECT 1 FROM _values v
    WHERE v._id_object = o._id
      AND v._id_structure = 102
      AND v._String = 'London'
  );

PostgreSQL optimizes correlated EXISTS well when the right index is there, so this is fine at 1–3 props. By 10 props the planner is doing real work to choose join order. By 30 props you're seeing query times you don't want. (For curiosity: the legacy path stays in the database under the name search_objects_with_facets() for back-compat — old callers keep working.)

On MSSql Free the picture was uglier still — a wide-CASE inline pivot that hadn't received the same iterations as the PG side. Different SQL shape from PG, different code path inside the library, different bugs. The two Free implementations had drifted.

Why the gap existed for so long

I built the Pro engine because we needed it. We use Pro internally — that's what runs the production workloads. The Free path kept working for its users, but parity was always "next quarter." It's not a single feature, it's:

pivot CTE generation for both dialects;
base-field pushdown into the inner _objects subquery;
nested-dict accessor (Field[key].Child);
ListItem .Value / .Alias via a single JOIN;
GroupBy, HAVING, ArrayGroupBy, DistinctBy, Sql.Function whitelist, $expr trees;
null semantics, $exists / $notNull;
a filter-splitting optimizer that decides what's a Shape A (pure base), B (narrow) or C (wide pivot) query;
and all of it round-tripping through 200+ integration tests per backend.

Spread that across two databases and you understand why it kept slipping. The interest spike from the launch posts is what pushed me to drop the half-written stuff and ship this. It's not finished-finished — that's why it's an OSS release instead of a private polish round.

What 3.0.0 actually changes

The headline: Free and Pro now emit the same PVT CTE shape, on both PostgreSQL and MSSql.

1. MSSql Free hit full v2-pvt parity — 145/145

The old MSSql Free path is gone. The new one is a 27-file SQL module under redb.MSSql/sql/v2-pvt/, assembled by an MSBuild target into a single pvt_bundle.sql resource embedded in RedBase.MSSql.dll:

00_module_init.sql           -- version sentinel, drop-and-replace bootstrap
10_pvt_field_collection.sql  -- walks the facet JSON, harvests structure ids
13_pvt_condition.sql         -- per-field WHERE fragments
14_pvt_where.sql             -- recursive $and/$or/$expr walker
15_pvt_order.sql             -- ORDER BY building (incl. $expr ordering)
16_pvt_split.sql             -- Shape A/B/C classifier, pushdown engine
17_pvt_expr.sql              -- $expr classifier + scalar expression compiler
20_pvt_build_query_sql.sql   -- the entry point — returns the final SQL text
99_smoke_auto.sql            -- 195 PASS / 0 FAIL / 1 SKIP regression suite
... (and friends)

The MSSql dialect uses MAX(CASE WHEN …) instead of array_agg FILTER (SQL Server has no FILTER clause), but the rest of the shape is the same. Same LINQ as above, on MSSql Free, becomes:

-- Free MSSql 3.0.0 — emitted by the T-SQL builder under [dbo].[pvt_build_query_sql]
WITH pvt_cte AS (
  SELECT v.[_id_object],
    MAX(CASE WHEN v.[_id_structure] = 101 AND v.[_array_index] IS NULL THEN v.[_Long]   END) AS [Age],
    MAX(CASE WHEN v.[_id_structure] = 102 AND v.[_array_index] IS NULL THEN v.[_String] END) AS [City]
  FROM [_values] v
  JOIN (SELECT [_id] FROM [_objects] WHERE [_id_scheme] = 42) o
    ON v.[_id_object] = o.[_id]
  WHERE v.[_id_structure] IN (101, 102)
  GROUP BY v.[_id_object]
)
SELECT o.* FROM [_objects] o
JOIN pvt_cte pvt ON o.[_id] = pvt.[_id_object]
WHERE pvt.[Age] > 30 AND pvt.[City] = N'London';

-- Same idea as the PG Free block above: literals are inlined by the T-SQL
-- builder via QUOTENAME() for identifiers and quoted-literal emission for
-- values (numeric literals stay bare, strings become N'...' with embedded
-- quotes escaped). The C# caller made two parameterized roundtrips:
--   1) EXEC sp_executesql N'SELECT [dbo].[pvt_build_query_sql](@scheme, @facets, @limit, @offset)',
--        N'@scheme bigint, @facets nvarchar(max), @limit int, @offset int', ...   -- returns SQL text
--   2) EXEC sp_executesql @sql                                                    -- collect _id values
--   3) SELECT * FROM [dbo].[get_objects_json](@ids, @maxDepth)                    -- hydrate by ids
-- User input never touches a SQL parser unquoted. The SQL-injection section
-- below walks the chain.

MAX(CASE WHEN ...) is the SQL Server idiom that mirrors array_agg FILTER: NULLs are ignored by aggregates, so the CASE acts as a "pass through if matching, otherwise null" gate. Same one-pass GROUP BY, same outer WHERE on flat columns. (Unlike Pro — which sends a parameterized statement with @p1..@pN placeholders and benefits from SQL Server's plan cache — Free's text varies per call, so plan reuse depends on SQL Server's auto-parameterization. Shape and index path are identical to Pro either way.)

Everything PG Free had now works identically on MSSql Free: flat + tree queries, scalar / array / dict fields, nested-dict Field[key].Child, ListItem joins for .Id / .Value / .Alias, same-scheme nested POCO compound paths, OrderBy / Take / Skip / DistinctBy, full GroupBy with HAVING, ArrayGroupBy via OUTER APPLY, array aggregates, Sql.Function whitelist, $expr, null semantics.

2. Auto-deploy of the v2-pvt bundle

ISqlDialect got a new method:

public interface ISqlDialect
{
    // ... existing dialect surface ...
    string? Query_PvtRequiredVersion();   // semver the embedded bundle ships
}

RedbServiceBase.EnsurePvtModuleDeployedAsync runs on InitializeAsync():

var required = Dialect.Query_PvtRequiredVersion();
if (required is null) return;                 // dialect doesn't use v2-pvt

var deployed = await Context.ExecuteScalarAsync<string?>(
    Dialect.SelectPvtModuleVersionSql());     // calls pvt_module_version()

if (string.Equals(deployed, required, StringComparison.Ordinal))
    return;                                    // exact match — already deployed

Logger?.LogInformation("v2-pvt mismatch (deployed={D}, required={R}). Applying bundle.",
                       deployed, required);
await Context.ExecuteNonQueryAsync(ReadEmbeddedBundle());

The match is exact-string, not semver-numeric. Any change to any of the 27 source files bumps the bundle version, the MSBuild target regenerates pvt_bundle.sql on the next build, and on next service startup the new bundle gets applied. No DBA in the loop, no "did you forget to run the migration?" tickets.

(One MSBuild gotcha cost me half a day: declaring <EmbeddedResource Include="sql\v2-pvt\pvt_bundle.sql" /> silently renames the manifest resource from redb.MSSql.sql.v2-pvt.pvt_bundle.sql to redb.MSSql.sql.v2_pvt.pvt_bundle.sql — dashes become underscores. GetManifestResourceStream returns null and you stare at it for a while. Fix: pin <LogicalName>redb.MSSql.sql.v2-pvt.pvt_bundle.sql</LogicalName> explicitly. Free tip if you ever ship SQL through embedded resources.)

3. Base-field pushdown — the 894 ms → 11 ms one

This is the regression that originally motivated 2.0.1. When a query mixes a base-field predicate (on _objects columns) with a props predicate:

var children = await service.Query<EmployeeProps>()
    .WhereRedb(o => o._id_parent == 555)   // base field, manager id 555
    .Where(p => p.Age > 30)                 // props field
    .ToListAsync();

The naive layout puts the base predicate in the outer WHERE — that's what 2.0.0 did:

-- Bad: 894 ms on a scheme with millions of _values rows
WITH pvt_cte AS (
  SELECT v._id_object,
    (array_agg(v._Long) FILTER (WHERE v._id_structure = $1))[1] AS "Age"
  FROM _values v
  JOIN _objects o ON v._id_object = o._id
  WHERE o._id_scheme = $2 AND v._id_structure = ANY($3)
  GROUP BY v._id_object
)
SELECT o.* FROM _objects o
JOIN pvt_cte pvt ON o._id = pvt._id_object
WHERE o._id_parent = $4    -- outer: applies AFTER the CTE pivoted the entire scheme
  AND pvt."Age" > $5;

-- $1=101, $2=42, $3={101}, $4=555, $5=30

PostgreSQL aggregates _values for every employee in the scheme and only then narrows by _id_parent. Self-inflicted full pivot.

3.0.0 (both Free and Pro, both dialects) classifies the base predicate at compile time and injects it into the inner _objects subquery of the PVT CTE:

-- Good: 11 ms
WITH pvt_cte AS (
  SELECT v._id_object,
    (array_agg(v._Long) FILTER (WHERE v._id_structure = $1))[1] AS "Age"
  FROM _values v
  JOIN (SELECT _id FROM _objects
        WHERE _id_scheme = $2
          AND _id_parent = $3   -- pushed in
       ) o ON v._id_object = o._id
  WHERE v._id_structure = ANY($4)
  GROUP BY v._id_object
)
SELECT o.* FROM _objects o
JOIN pvt_cte pvt ON o._id = pvt._id_object
WHERE pvt."Age" > $5;

-- $1=101, $2=42, $3=555, $4={101}, $5=30

Free 3.0.0 emits the same shape — here it is on PostgreSQL, with the values baked in by pvt_split_filter:

-- Free 3.0.0 on PostgreSQL — pushdown identical to Pro
WITH pvt_cte AS (
  SELECT v._id_object,
    (array_agg(v._Long) FILTER (WHERE v._id_structure = 101))[1] AS "Age"
  FROM _values v
  JOIN (SELECT _id FROM _objects
        WHERE _id_scheme = 42
          AND _id_parent = 555           -- pushed in by pvt_split_filter
       ) o ON v._id_object = o._id
  WHERE v._id_structure IN (101)
  GROUP BY v._id_object
)
SELECT o.* FROM _objects o
JOIN pvt_cte pvt ON o._id = pvt._id_object
WHERE pvt."Age" > 30;

And on MSSql Free 3.0.0 — again, literals inlined by the T-SQL builder, not parameters:

-- Free MSSql 3.0.0 — pushdown via the same classifier, MAX/CASE pivot, values inlined
WITH pvt_cte AS (
  SELECT v.[_id_object],
    MAX(CASE WHEN v.[_id_structure] = 101 AND v.[_array_index] IS NULL THEN v.[_Long] END) AS [Age]
  FROM [_values] v
  JOIN (SELECT [_id] FROM [_objects]
        WHERE [_id_scheme] = 42
          AND [_id_parent] = 555         -- pushed in by the T-SQL split classifier
       ) o ON v.[_id_object] = o.[_id]
  WHERE v.[_id_structure] IN (101)
  GROUP BY v.[_id_object]
)
SELECT o.* FROM [_objects] o
JOIN pvt_cte pvt ON o.[_id] = pvt.[_id_object]
WHERE pvt.[Age] > 30;

The inner subquery resolves via the (_id_scheme, _id_parent) covering index, returns ~N matching _ids, and the pivot only sees _values rows for those N objects. Pro got this in 2.0.1; 3.0.0 brings the same to Free through pvt_split_filter and the pvt_expr_is_base_only classifier in 17_pvt_expr.sql.

4. Nested-dict accessor short-circuit

For Dictionary<string, T> fields, Where(o => o.OfficeLocations["HQ"].City == "New York") builds the dict-keyed pivot column in the CTE. There's no separate _dict_key column in _values — the existing _array_index slot (text) doubles as the dict key for dictionary fields (it stays an integer-as-text for arrays, holds the key for dicts, is NULL for scalars):

(array_agg(v._String) FILTER (
  WHERE v._id_structure = 117    -- structure id of City inside OfficeLocations
    AND v._array_index = 'HQ'    -- _array_index doubles as the dict key
))[1] AS "OfficeLocations[HQ].City"

The bug fixed in this cycle (MSSql 0.1.3, PG 0.6.1 earlier): the outer WHERE was also emitting a separate EXISTS over _values to re-check the same dict key, even though the CTE had already done that work. The six-line fix in 13_pvt_condition.sql makes the outer WHERE just reference the pivot column directly:

-- Before
WHERE EXISTS (
  SELECT 1 FROM _values v
  WHERE v._id_object = o._id
    AND v._id_structure = 117
    AND v._array_index = 'HQ'
    AND v._String = 'New York'
)

-- After
WHERE pvt."OfficeLocations[HQ].City" = $4    -- $4='New York'

One predicate, no subquery, the optimizer can reorder it freely with the rest of the WHERE.

5. What landed in Pro this cycle

While Free was catching up, Pro added its own things (the version is 3.0.0 for both):

GroupBy + HAVING in Pro. HavingAsync existed in Free since 1.2.x but had no Pro counterpart. Now:

var stats = await service.Query<EmployeeProps>()
    .GroupBy(p => p.Department.Value)
    .Where(g => g.Count() > 10 && g.Average(p => p.Age) < 40)   // HAVING
    .Select(g => new { Dept = g.Key, N = g.Count() })
    .ToListAsync();

→

WITH pvt_cte AS (
  SELECT v._id_object,
    (array_agg(v._listitem) FILTER (WHERE v._id_structure = $1))[1] AS "Department_id",
    (array_agg(v._Long)     FILTER (WHERE v._id_structure = $2))[1] AS "Age"
  FROM _values v
  JOIN (SELECT _id FROM _objects WHERE _id_scheme = $3) o ON v._id_object = o._id
  WHERE v._id_structure = ANY($4)
  GROUP BY v._id_object
)
SELECT li."_value" AS "Dept", COUNT(*) AS "N"
FROM _objects o
JOIN pvt_cte pvt ON o._id = pvt._id_object
LEFT JOIN _list_items li ON li._id = pvt."Department_id"
GROUP BY li."_value"
HAVING COUNT(*) > $5 AND AVG(pvt."Age") < $6;

-- $1=130, $2=101, $3=42, $4={130,101}, $5=10, $6=40

The Free side has had HavingAsync since 1.2.x — here's the same LINQ through Free 3.0.0 on PostgreSQL for parity, so you can see they really do meet in the middle now:

-- Free 3.0.0 PG — same HAVING shape, values inlined by pvt_build_query_sql
WITH pvt_cte AS (
  SELECT v._id_object,
    (array_agg(v._listitem) FILTER (WHERE v._id_structure = 130))[1] AS "Department_id",
    (array_agg(v._Long)     FILTER (WHERE v._id_structure = 101))[1] AS "Age"
  FROM _values v
  JOIN (SELECT _id FROM _objects WHERE _id_scheme = 42) o ON v._id_object = o._id
  WHERE v._id_structure IN (130, 101)
  GROUP BY v._id_object
)
SELECT li."_value" AS "Dept", COUNT(*) AS "N"
FROM _objects o
JOIN pvt_cte pvt ON o._id = pvt._id_object
LEFT JOIN _list_items li ON li._id = pvt."Department_id"
GROUP BY li."_value"
HAVING COUNT(*) > 10 AND AVG(pvt."Age") < 40;

And on MSSql Free 3.0.0 — same shape, literals inlined:

-- Free MSSql 3.0.0 — same HAVING shape, MAX/CASE pivot, values inlined by the T-SQL builder
WITH pvt_cte AS (
  SELECT v.[_id_object],
    MAX(CASE WHEN v.[_id_structure] = 130 THEN v.[_listitem] END) AS [Department_id],
    MAX(CASE WHEN v.[_id_structure] = 101 THEN v.[_Long]     END) AS [Age]
  FROM [_values] v
  JOIN (SELECT [_id] FROM [_objects] WHERE [_id_scheme] = 42) o
    ON v.[_id_object] = o.[_id]
  WHERE v.[_id_structure] IN (130, 101)
  GROUP BY v.[_id_object]
)
SELECT li.[_value] AS [Dept], COUNT(*) AS [N]
FROM [_objects] o
JOIN pvt_cte pvt ON o.[_id] = pvt.[_id_object]
LEFT JOIN [_list_items] li ON li.[_id] = pvt.[Department_id]
GROUP BY li.[_value]
HAVING COUNT(*) > 10 AND AVG(pvt.[Age]) < 40;

33/33 HAVING + 6/6 no-HAVING in GroupByHavingTestsBase, green on PG.Pro and MSSql.Pro — and identical row counts on the Free side.

ArrayGroupBy unified across all four tiers — GroupBy(items => items.SelectMany(o => o.Skills)) works identically on PG Free, PG.Pro, MSSql Free, MSSql.Pro.

MSSql.Pro AggregateBatch parity with PG.Pro — non-numeric MinAsync / MaxAsync (over string/DateTime/Guid) and a Where(...) filter inside a batch aggregation now produce the same plan shape on MSSql.Pro as PG.Pro.

Plus a stack of bug-fixes documented in CHANGELOG.md.

"Wait — Free builds SQL dynamically inside plpgsql. What about SQL injection?"

Right question. If you skimmed the Free SQL above and noticed it's EXECUTE-ing a dynamically-built string inside a database function — your instinct is correct, that would be a problem in a naively-written generator. Here's what actually guards against it.

Pro side (C#) — boring and safe

Pro builds parameterized SQL via SqlParameterCollector. Every value the user expression captured (30, "London", managerId) becomes an NpgsqlParameter / SqlParameter. The text Npgsql sends to PostgreSQL contains $1..$N placeholders only; values travel out-of-band through the wire protocol and never see a SQL parser. No template, no concatenation, no format(). There's literally no path for an injection because there's no string-to-SQL bridge for values.

Free side on PostgreSQL (plpgsql) — three layers, on purpose

Free has it harder than Pro because the SQL builder runs inside the database, so the values arrive baked into the JSON. The defenses, layer by layer:

1. Values pass through format(..., %L) with explicit type casts.

%L is PostgreSQL's literal-quoting format specifier — same semantics as quote_literal(). It single-quotes the value, escapes embedded quotes, and returns a SQL-safe literal. Example from 04_pvt_inner_condition.sql:

-- The leaf emitter for "Long $gt N" looks like this:
RETURN format('fv._Long %s %L::bigint', op_symbol, operator_value);

If a malicious payload arrives as operator_value = "1; DROP TABLE _objects --", %L produces:

fv._Long > '1; DROP TABLE _objects --'::bigint

The ::bigint cast then fails at parse time with a normal type error. The single quotes from %L make the injection inert; the type cast makes it a hard error rather than silent data corruption. Every value branch in 04_pvt_inner_condition.sql uses this pattern: %L::bigint, %L::numeric, %L::timestamptz, %L::uuid, %L::boolean, %L::interval. Strings use plain %L (which is '…'-quoting), and any further use as text is still inside single quotes.

2. Identifiers come from the database, not from the JSON.

The user's JSON facets refer to fields by name ("Age", "OfficeLocations[HQ].City"). The builder doesn't concatenate that string into the SQL. Instead, pvt_field_collection looks the name up in _structures for the queried scheme — by joining _structures._name = $jsonKey AND _structures._id_scheme = $scheme. If the name doesn't exist, the field is rejected. If it does exist, what gets emitted is the matched _id_structure (a bigint, always safe) and a pivot column alias built with quote_ident(structure_name_from_db) or %I. So:

The structure id is a number from the DB.
The pivot column name is quote_ident'd, so "weird name with quotes" becomes a double-quoted SQL identifier.
Reserved bases (0$:id, 0$:parent_id, etc.) are dispatched through a fixed CASE — unknown bases raise:

  RAISE EXCEPTION 'Unknown RedbObject base field: "%" ...'

No path lets arbitrary text become an unquoted identifier.

3. Operator names are a fixed whitelist.

$eq, $gt, $lt, $ne, $in, $like, $ilike, $arrayContains, $expr, etc. are matched by a CASE/IF tree in 04_pvt_inner_condition.sql and friends. Unknown operators don't fall through to anything — they RAISE EXCEPTION:

RAISE EXCEPTION 'pvt_build_agg_expr: unsupported aggregate "%"', v_op;

Same for Sql.Function — only names in the whitelist (COALESCE, LOWER, UPPER, …) survive; everything else raises.

4. The final EXECUTE evaluates a string built only from the three pieces above.

So even though EXECUTE is involved, every byte of that string came from either:

a quote_literal'd (%L) value with a type cast, or
a quote_ident'd (%I) identifier sourced from _structures by id lookup, or
a hardcoded operator template chosen by a whitelist switch.

That's three independent gates, and any of them is sufficient to defeat injection. The plpgsql generator is paranoid by design — format/%L/%I are used uniformly, not as a one-off. The 99-test smoke suite (99_smoke_auto.sql) includes adversarial payloads ("; DROP TABLE …, embedded quotes, type-mismatched literals) and expects parse errors or rejection, not successful execution.

If you want to audit the PG side: every emitter is in 04_pvt_inner_condition.sql, 05_pvt_single_facet.sql, 13_pvt_condition.sql. Grep for format( — every value site has %L and a cast, every identifier site has %I or quote_ident.

Free side on MSSql (T-SQL) — the same three layers, different primitives

SQL Server has no format(..., %L/%I) and no quote_literal/quote_ident shorthand, so the MSSql builder under redb.MSSql/sql/v2-pvt/ implements each layer with the T-SQL primitives that are available. Same three gates, just spelled differently.

1. Values are quoted with REPLACE(..., '''', '''''') and/or validated with TRY_CAST before they get inlined.

For strings (including dict keys and $expr literals) the builder wraps the value in N'…' after doubling embedded single quotes — the SQL Server equivalent of quote_literal. From the actual pivot builder (pvt_bundle.sql, the dict-key branch):

-- Real emitter for dict-key match in the CTE
-- (the dict key goes into _values._array_index — same slot used for array indices)
SET @dict_key_literal =
    N'N''' + REPLACE(@dict_key, N'''', N'''''') + N'''';
-- Then concatenated into:
--   ... AND v.[_array_index] = N'HQ' THEN v.[_String] END) AS [...]

If @dict_key = "HQ'; DROP TABLE _values; --", the REPLACE doubles the embedded quote to '', and the emitted SQL becomes:

v.[_array_index] = N'HQ''; DROP TABLE _values; --'

The whole payload stays a single string literal. The tail of the payload never escapes the quotes, so it never gets executed.

For numerics the builder doesn't even concatenate raw user text — it first parses it with TRY_CAST and only inlines the parsed value if the parse succeeded. From 06_pvt_hierarchical.sql:

DECLARE @ha_id BIGINT = TRY_CAST(@ha_raw AS BIGINT);
IF @ha_id IS NOT NULL
    SET @result += N' AND dbo.pvt_is_descendant_of(' + @alias + N'.[_id], '
                 + CAST(@ha_id AS NVARCHAR(20)) + N') = 1';
-- If @ha_id is NULL the clause is silently dropped — bad input never
-- becomes SQL text. The only thing that ever gets inlined is a real BIGINT.

TRY_CAST returns NULL rather than raising on bad input, and the builder simply omits the clause in that case — so "1; DROP TABLE _objects --" never reaches the SQL text. The behaviour is fail-closed (no clause emitted) rather than fail-loud (no exception thrown); the predicate effectively becomes a no-op. Same pattern for WhereLevel (TRY_CAST AS INT), WhereHasDescendant, WhereContainsObject, and the other base-id predicates. For dates it's TRY_CONVERT(datetimeoffset, …), for GUIDs it's TRY_CONVERT(uniqueidentifier, …) — quote-then-typed-parse, drop-on-fail.

2. Identifiers come from _structures via QUOTENAME(), never from raw JSON.

Every pivot column alias, every projection alias, every table/column name in the dynamic SQL goes through QUOTENAME(). From the pivot CTE builder:

DECLARE @alias NVARCHAR(420) = QUOTENAME(@field_name);  -- @field_name comes from _structures
-- ...
RETURN N'o.' + QUOTENAME(@col) + N' AS ' + @alias;

QUOTENAME('weird]name') returns [weird]]name] — embedded ] is doubled, the result is always a single safely-bracketed identifier. As on the PG side, the source of @field_name is a _structures._name row matched by (_id_scheme = @scheme AND _name = @json_key) — if the JSON refers to a field the queried scheme doesn't have, the lookup returns nothing and the field is rejected with a RAISERROR before any SQL text is built.

For base-fields (0$:id, 0$:parent_id, 0$:scheme_id, …) there's a fixed CASE; unknown bases RAISERROR with the same "Unknown RedbObject base field" message as the PG side.

3. Operator names are a fixed CASE/IF whitelist with RAISERROR on miss.

Same logic as PG: $eq, $gt, $lt, $in, $like, $arrayContains, $expr, etc. are matched by a CASE tree. Anything not on the list raises:

RAISERROR(N'pvt: unsupported operator "%s"', 16, 1, @op);

Sql.Function calls go through the same whitelist — COALESCE, LOWER, UPPER, LEN, LEFT, RIGHT, ISNULL, etc. Unknown function names raise; the user can't smuggle xp_cmdshell through Sql.Function<int>("xp_cmdshell", "...") because that name isn't in the CASE.

4. The dynamic SQL is built only from the three pieces above and executed with sp_executesql.

Final execution is EXEC sp_executesql @sql (with parameterized @scheme_id, @limit, @offset etc. as sp_executesql arguments, not concatenated). Every byte of @sql came from either:

a REPLACE-quoted string literal in N'…', or
a TRY_CAST-validated typed value inlined as its parsed form, or
a QUOTENAME-bracketed identifier sourced from _structures by id lookup, or
a hardcoded operator template chosen by a whitelist CASE.

The 99_smoke_auto.sql regression suite on MSSql includes the same adversarial payloads as the PG suite — '; DROP TABLE …, embedded quotes, ] in field names, type-mismatched literals, unknown operators — and expects each one to be either rejected with RAISERROR (operators / unknown bases / unknown identifiers), silently dropped from the predicate (typed-value parse failures), or come out the other side as an inert quoted literal that does nothing (string values).

Summary

Layer	PG Free emits via	MSSql Free emits via
String values	`format('… %L', val)`	`N'' + REPLACE(val, '''', '''''') + N''`
Numeric / date / GUID values	`%L::bigint` / `%L::numeric` / `%L::timestamptz` / `%L::uuid` (parse-or-die at runtime)	`TRY_CAST(... AS BIGINT)` / `TRY_CONVERT(datetimeoffset, ...)` / `TRY_CONVERT(uniqueidentifier, ...)` (parse-or-drop-clause before inline)
Identifiers	`%I` / `quote_ident(name_from__structures)`	`QUOTENAME(name_from__structures)`
Operators / function names	CASE whitelist → `RAISE EXCEPTION`	CASE whitelist → `RAISERROR`
Final execution	`EXECUTE` of the assembled string	`EXEC sp_executesql` of the assembled string

Same three independent gates on both databases. Any one of them is sufficient to defeat injection; the suite tests all three.

If you find a leak in either dialect, file an issue with the JSON payload that triggers it. I'll treat it as a security bug, not a feature request.

The polyglot facet API — Free's one structural advantage

A side-effect of Free's SQL-side builder: it's accessible from any language. Pro compiles in C#, which is its strength but also its limitation — you need a .NET process. Free compiles in SQL, so the same builder C# Free clients hit is callable from any driver that can run SQL.

The shipped surface is two functions, not one:

Function	Input	Output
`pvt_build_query_sql(scheme_id, facets jsonb, limit, offset, order jsonb, max_depth, …)`	facets	the assembled `SELECT _id FROM …` as `text`
`get_objects_json(ids bigint[], max_depth integer DEFAULT 10)`	matched ids	table of `(Id bigint, JsonData text)` with fully hydrated objects

The honest flow is therefore two SQL roundtrips: build → EXECUTE the returned text → hydrate. The 10-line plpgsql wrapper below collapses that into a single call you can ship into your DB once and then call from any language:

-- one-time setup: paste into your DB as a wrapper of the two shipped functions
CREATE OR REPLACE FUNCTION redb_run_facets(
    p_scheme_id bigint,
    p_facets    jsonb   DEFAULT NULL,
    p_limit     integer DEFAULT NULL,
    p_offset    integer DEFAULT 0,
    p_order     jsonb   DEFAULT NULL,
    p_max_depth integer DEFAULT 10
) RETURNS TABLE("Id" bigint, "JsonData" text)
LANGUAGE plpgsql STABLE AS $$
DECLARE v_sql text; v_ids bigint[];
BEGIN
    v_sql := pvt_build_query_sql(p_scheme_id, p_facets, p_limit, p_offset, p_order, p_max_depth);
    EXECUTE 'SELECT array_agg(_id) FROM (' || v_sql || ') t' INTO v_ids;
    IF v_ids IS NULL THEN RETURN; END IF;
    RETURN QUERY SELECT g."Id", g."JsonData" FROM get_objects_json(v_ids, p_max_depth) g;
END $$;

With that wrapper in place:

# Python — psycopg
import psycopg, json

facets = {
    "$and": [
        {"Age":  {"$gt": 30}},
        {"City": {"$eq": "London"}},
    ]
}

with psycopg.connect(DSN) as conn, conn.cursor() as cur:
    cur.execute(
        'SELECT "JsonData" FROM redb_run_facets(%s, %s::jsonb, %s, %s)',
        (scheme_id, json.dumps(facets), 100, 0),    # limit, offset
    )
    employees = [row[0] for row in cur.fetchall()]
    # each row is JSON text — nested objects, arrays, dictionaries,
    # ListItem references — all fully reconstructed.

// Node — pg
const facets = { $and: [{ Age: { $gt: 30 } }, { City: { $eq: 'London' } }] };
const { rows } = await client.query(
  'SELECT "JsonData" FROM redb_run_facets($1, $2::jsonb, $3, $4)',
  [schemeId, JSON.stringify(facets), 100, 0],
);
const employees = rows.map(r => JSON.parse(r.JsonData));

// Go — pgx
facets := []byte(`{"$and":[{"Age":{"$gt":30}},{"City":{"$eq":"London"}}]}`)
rows, err := conn.Query(ctx,
    `SELECT "JsonData" FROM redb_run_facets($1, $2::jsonb, $3, $4)`,
    schemeID, facets, 100, 0,
)
// scan each row into a json.RawMessage

// Java — JDBC
String facets = "{\"$and\":[{\"Age\":{\"$gt\":30}},{\"City\":{\"$eq\":\"London\"}}]}";
try (PreparedStatement ps = conn.prepareStatement(
        "SELECT \"JsonData\" FROM redb_run_facets(?, ?::jsonb, ?, ?)")) {
    ps.setLong(1, schemeId);
    ps.setString(2, facets);
    ps.setInt(3, 100);
    ps.setInt(4, 0);
    try (ResultSet rs = ps.executeQuery()) {
        while (rs.next()) {
            String hydratedJson = rs.getString(1);
        }
    }
}

-- raw psql — same engine, no driver at all
SELECT "JsonData"
FROM redb_run_facets(
    42::bigint,
    '{"$and":[{"Age":{"$gt":30}},{"City":{"$eq":"London"}}]}'::jsonb,
    100, 0
);

If you'd rather skip the wrapper and call the shipped functions directly, the two-roundtrip pattern is:

-- 1. build the query text
SELECT pvt_build_query_sql(42, '{"$and":[…]}'::jsonb, 100, 0) AS sql;
-- 2. EXECUTE the returned text on the client → collect _id values
-- 3. hydrate
SELECT * FROM get_objects_json(ARRAY[101,102,103]::bigint[], 10);

The JSON facet schema is a stable contract — see FREE-OVER-PRO.md for the full operator list. Tuple-key dictionaries, nested RedbObject<T> references, RedbListItem references with .Id/.Value/.Alias — all reconstruct correctly in the JSON get_objects_json emits. No schema knowledge needed on the client side.

And the injection story above applies the same way here — your psycopg/pgx/JDBC driver binds the JSON as a jsonb parameter, so even your facet payload doesn't touch a SQL parser until it's already inside the parameterized builder.

This is Free-only by design. Pro's edge has always been the CLR-side compiled pipeline plus parallel materialization (ProLazyPropsLoader does 2 bulk _values SELECTs, indexes them into ILookup, and ProPropsMaterializer runs Parallel.ForEach to assemble 20+ value types, nested objects, arrays, dictionaries, ListItem refs into typed RedbObject<T> — all CPU-bound, zero further DB access). That stays where it is.

The honest matrix

Capability	Free 3.0.0	Pro 3.0.0
Query shape (PVT CTE, base pushdown, MAX/CASE ↔ array_agg)	Same	Same
`GroupBy + HAVING`	✓	✓
`ArrayGroupBy` (unnest / OUTER APPLY)	✓	✓
`DistinctBy`, `Sql.Function` (whitelist), `$expr` trees	✓	✓
Tree queries (`WhereHasAncestor`, `WhereLevel`, etc.)	✓	✓
Window functions (`Lag`/`Lead`/`Rank`/`DenseRank`)	✓	✓
Polyglot facet API (Python / Node / Go / Java / raw SQL)	✓	—
Compiled in C# (no JSON intermediate)	—	✓
Parallel CLR materialization (`Parallel.ForEach`)	—	✓
Bulk save via `COPY` protocol	—	✓
Hash-based `ChangeTracking` (diff only what changed)	—	✓
Combined `GroupBy` + window + `AggregateBatch`	✓	✓

Two things to be honest about:

Free has higher latency on full-object load than Pro — get_object_json builds JSON in plpgsql and the client deserializes. Pro's bulk-+-parallel materialization is a real win when you're loading thousands of complex objects with 20+ nested fields each.
Pro still has features Free won't get — parallel bulk materialization (ProLazyPropsLoader + ProPropsMaterializer) and hash-based ChangeTracking remain Pro-only. The query shape itself is now identical.

But the query shape — the part that determines whether the database uses your indexes properly, whether the plan caches, whether 30 props in a WHERE becomes 30 EXISTS or one pivot — that's the same in both tiers now.

Migration notes

If you're on 2.0.x Free, upgrading to 3.0.0:

Update packages — RedBase.Core, RedBase.Postgres / RedBase.MSSql to 3.0.0.
First startup applies the v2-pvt bundle automatically. The new module is additive at the C# API level — your existing LINQ-style queries just compile through the new path. Note: the legacy plpgsql function search_objects_with_facets() (used only by pre-2.0 raw-SQL callers) is no longer installed by redb_init.sql — it lives under redb.Postgres/sql/deprecated/ if you still need it; paste it manually before upgrading if you had direct SQL callers.
Pre-existing config flag is now honored. DefaultStrictDeleteExtra in built-in presets Development, HighPerformance, Migration was set to false but silently ignored before 3.0.0. It now actually works — meaning those presets will no longer auto-delete _structures rows for fields removed from your Props class on startup. If you were relying on the old (silently-broken) behavior, set c.DefaultStrictDeleteExtra = true explicitly.
MSSql Free users: the wide-CASE inline pivot is gone. If you had query-text-matching log scrapers, the new shape is WITH pvt_cte AS (... MAX(CASE WHEN _id_structure = @p1 AND _array_index IS NULL THEN _X END) ...) SELECT o.* FROM _objects o JOIN pvt_cte pvt ON ....

No source-level breaking API changes. No data migrations. If you've never touched the SQL surface directly, you can upgrade and not notice anything except your slow queries getting faster.

Known gaps I'd like help finding

Nested-field cross-scheme JOIN — Where(o => o.CurrentProject.Props.Name == "X") where CurrentProject is a RedbObject<OtherScheme> reference. Free PVT rejects the path; Pro SchemeFieldResolver throws. Needs new infrastructure on both sides. Tracked in FREE-OVER-PRO.md §2 #6b.
MSSql Free is fresh. Two weeks of intensive testing, not two years. EXPLAIN traces and failing repros are very welcome.
Polyglot facet API documentation is thinner than the C# API. The operator surface is stable (it's what the C# parser emits), but it's not yet a documented wire-protocol RFC.

There will be follow-up fixes after this post lands — that's expected. If you hit something, please open a thread in GitHub Discussions rather than sitting on it.

Try it

# Free
dotnet add package RedBase.Core      --version 3.0.0
dotnet add package RedBase.Postgres  --version 3.0.0     # or RedBase.MSSql

# Pro (commercial)
dotnet add package RedBase.Core.Pro       --version 3.0.0
dotnet add package RedBase.Postgres.Pro   --version 3.0.0   # or RedBase.MSSql.Pro

Repo: https://github.com/redbase-app/redb
Full 3.0.0 changelog: CHANGELOG.md
Architecture deep-dive (with the same LINQ→SQL examples): https://redbase.app/architecture
Issues / bug reports: https://github.com/redbase-app/redb/issues — please cite the SQL and the dialect.

If you read all the way to here, thank you. Tell me what's wrong with it.

From Closed Internal Stack to Open-Source Ecosystem: I Finally Shipped Three Years of .NET Infrastructure

rinat kozin — Sat, 23 May 2026 16:07:54 +0000

This is a submission for the
GitHub Finish-Up-A-Thon Challenge

Why this exists (the actual pain)

Before the architecture talk, the why. Three years ago
I watched my team — and every other .NET team I knew —
burn most of their hours not on business logic but on
plumbing the same four layers over and over:

EF Core + migrations. DbContext, fluent mappings, navigation properties, N+1 puzzles, lazy/eager loading, a migration per field, separate read/write models if you go CQRS. On a non-trivial object graph that's hundreds of hours just for the data layer, before any business logic is written.
A pile of integration connectors. Kafka here, RabbitMQ there, S3 for files, SFTP for the legacy partner, an HTTP webhook for the new one, each one hand-rolled with its own retry policy, dead-letter, idempotency, serialization, health check, telemetry. Ten integrations × 40-80 hours each.
Auth. OAuth flows, refresh tokens, scope checking, claims transformation, multi-tenant, M2M, DPoP if you want to be modern — built from IdentityServer / Auth0 / Keycloak plus 200-600 hours of glue per serious deployment.
Runtime. Hot-reload without dropping in-flight messages, graceful drain on redeploy, cluster coordination, leader election, watchdog, dashboard — usually missing entirely on .NET, or assembled from 5-7 unrelated NuGets that don't share conventions.

The worst part isn't any single layer — it's the seams
between them. Where the auth user becomes the EF entity
becomes the Kafka message becomes the S3 blob, every seam
is one more serialization, one more mapping, one more
versioning headache, one more place a 3 AM page comes from.

RedBase is built so that business code is the only thing
left to write. The class IS the schema (no EF, no
migrations). The 22 transports share one DSL (no per-connector
plumbing). Identity is a direct-vm:// Route (calling auth
is calling a function, not a network round-trip). Tsak gives
you hot-reload, cluster, dashboard, drain out of the box. The
seams collapse because everything lives on the same fabric.

On the one full business workflow where I had honest before/after
numbers — built the traditional way vs built on the RedBase
stack — the human effort was roughly ~3,000 hours vs ~128
hours. That ratio isn't magic on any single layer (where
each subsystem gives 3-5× at most); it comes from the seams
no longer existing. When I asked Claude Opus 4.7 to
sanity-check that estimate against typical .NET project
breakdowns (data layer + integrations + auth + runtime +
inter-seam testing), the order of magnitude held up.

One stack, one team, one architectural style — so the team
gets to write features instead of wiring infrastructure.
That is the project. Everything below is just how I got
there.

What I Built

RedBase — a four-pillar open-source ecosystem for .NET
that grew up inside one production system over three years
and finally got shipped to the world this spring. Three
pillars are now public on GitHub under Apache 2.0, and the
fourth is in pre-release polish:

1. redb — typed
object storage engine. Your C# class IS the schema. Two
physical tables (_objects + _values), full LINQ, zero
migrations, recursive-CTE tree queries, bulk COPY BINARY
saves on PostgreSQL and SqlBulkCopy on SQL Server. Free
core + Pro tier with tree-diff ChangeTracking saves.

2. redb-route
— Apache Camel for .NET. Fluent C# DSL for
From → Process → To pipelines, 22 transport packages
(Kafka, RabbitMQ, MQTT, S3, gRPC, SFTP, AMQP, Azure
Service Bus, IBM MQ, Elasticsearch, Redis, LDAP, FTP, HTTP,
WebSocket, SignalR, Firebase, TCP, Mail, SQL, Quartz,
generic File), 80+ EIP patterns (Splitter, Aggregator,
CBR, Circuit Breaker, Saga, ...), compiled expression
engine, transactional routes, OpenTelemetry built-in.

3. redb-tsak
— the .NET analogue of Apache Karaf / Camel K. Production
runtime container: drop a .tpkg bundle (ZIP with
manifest.json + entry-point DLLs + dependency DLLs +
per-module JSON config) — or just a bare .dll for the
simplest cases — into Libs/, get hot-reload without
dropping in-flight messages, REST + Blazor dashboard,
watchdog, Quartz scheduler, three deployment modes
(Standalone / Single-node+redb / Cluster with leader
election & auto-rebalance), per-module
AssemblyLoadContext isolation, API Key + HMAC-SHA256
security, 30-command CLI, typed C# client. 415+ tests.

4. redb.Identity (pre-release, repo opening soon) —
OAuth 2.1 / OIDC server, architecturally transport-agnostic:
every endpoint is a direct-vm:// Route. That gives two
ready usage modes today: (a) in-process — call the
auth server directly from another module in the same
run-time with zero network hop, zero serialization, zero
HTTP stack (this is what direct-vm:// is), and
(b) HTTP — a full OAuth 2.1 / OIDC HTTP facade is
shipped and working. The other RPC-capable facades —
gRPC, RabbitMQ RPC, AMQP request/reply, IBM MQ
request/reply, WebSocket, SignalR, TCP — are on the
roadmap and become near-trivial to add because the core
logic is already wire-format independent. (The
fire-and-forget transports in the Route set — Kafka
producer, File, S3, Mail, etc. — don't fit an auth
server, so they're correctly out of scope.) OpenIddict
under the hood, REDB-backed storage, DPoP / PAR / Dynamic
Client Registration / SCIM 2.0 / FIDO2 / WebAuthn /
RFC 8417 shared-signals support.
1751+ tests passing before public release.

Total scope: ~385k LOC (326k C# + 58k SQL) across
~2200 source files, Apache 2.0, NuGet-published, all
docs at redbase.app/why-redbase.

Demo

Live docs and positioning:

redbase.app/why-redbase — full "Sound familiar?" pain-point walkthrough with side-by-side Traditional-vs-RedBase code examples
The docs site itself is powered by RedBase — every page, example, and API reference is a REDB object in MSSQL. We eat our own cooking.

Published article series on dev.to
(author page) —
the ecosystem is too large for one post, so I'm publishing
a deep-dive series, one pillar at a time:

May 13 — We built an enterprise integration stack for .NET from scratch: EAV + DSL + runtime — overview of the whole stack
May 14 — I spent a year building Apache Camel for .NET. Here's the honest state of it. — discussion piece, what's done / what isn't
May 17 — redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL — integration framework deep dive
May 21 — An EF Core alternative for .NET apps with complex object graphs — full LINQ, no migrations, no DbContext — REDB storage engine deep dive (this is the one that already triggered the multi-version-deploy thread referenced below)

…and the series continues. Tsak (runtime container,
hot-reload, cluster, watchdog) gets its own post next,
then Identity (transport-agnostic OAuth 2.1 / OIDC) when
the repo opens, then a deep dive on the tree-diff
ChangeTracking path, then a multi-DB benchmark post.
A stack this size is a genuinely hard engineering
problem to explain — trying to cram it into one article
would lie about the depth. So I'm taking it one layer
at a time.

The 30-second flavor:

// 1. Define schema — no migration needed
[RedbScheme("order")]
public class OrderProps
{
    public Customer Customer { get; set; }
    public Address ShippingAddress { get; set; }
    public List<OrderLineProps> Lines { get; set; } = [];
    public Dictionary<string, RedbObject<CouponProps>> Coupons { get; set; }
}

// 2. Save entire object graph in one call
await redb.SaveAsync(order);

// 3. Query with full LINQ over nested props
var hot = await redb.Query<OrderProps>()
    .Where(o => o.Customer.Address.City == "London"
             && o.Lines.Any(l => l.Qty > 10))
    .OrderByDescending(o => o.DateModify)
    .Take(50)
    .ToListAsync();

// 4. Tree query with recursive CTE under the hood
var allProducts = await redb
    .TreeQuery<Product>(londonHQ.Id, maxDepth: 10)
    .Where(p => p.InStock)
    .ToListAsync();

// Same dev defines a routing context with redb.Route.
// Real production patterns: REDB isn't a transport URI —
// it's accessed inside processors via DI (.ProcessWithRedb).
public class OrdersRouteBuilder : RouteBuilderBase
{
    public override void Build()
    {
        From("kafka://orders-raw")
            .Unmarshal<OrderProps>()
            .Split(o => o.Lines, EipParallel.Yes)
            .ProcessWithRedb(async (redb, exchange, ct) =>
            {
                var line = (OrderLineProps)exchange.In.Body;
                await redb.SaveAsync(line);   // REDB store
            })
            .Aggregate(by: e => e.In.Headers["OrderId"],
                       timeout: 5.Seconds())
            .To("rabbitmq://orders-enriched");

        // Staged queue with backpressure — exactly how the
        // production garage-sync route does it:
        From("seda://orders?concurrentConsumers=4")
            .ProcessWithRedb(EnrichFromInventoryAsync)
            .To("http://partner-api/orders");
    }
}

// Pack as a .tpkg bundle (manifest.json + entry-point DLL
// + dependency DLLs + module config.json) — or for trivial
// modules, just drop the bare .dll — into Libs/ of a Tsak
// node. Tsak picks it up, runs it across the cluster,
// rebalances on node failure, drains old versions on hot
// redeploy.
//
// Need the same flow gated by an OAuth scope?
// Two ready modes today:
//   * in-process — the call goes through direct-vm://
//     straight into redb.Identity, no network, no HTTP;
//   * over HTTP — standard OAuth 2.1 / OIDC endpoints.
// Tomorrow, the same auth server can also be exposed over
// any other RPC-capable Route transport (gRPC, RabbitMQ
// RPC, AMQP request/reply, IBM MQ, WebSocket, SignalR,
// TCP, ...) as those facades land — no change to the
// auth server itself:
//
//     .Process(ctx => ctx.RequireScope("orders.write"))

That last paragraph — "drops a .tpkg (or just a .dll),
gets hot-reload + cluster + dashboard + REST API + drain on
redeploy + an auth server callable from the same routing
fabric" — is the part I'm proudest of, because it's exactly
the piece
that's been missing from the .NET integration story for a
decade. Java had Karaf, Camel K, and a mature OIDC story
glued together. .NET had nothing equivalent that wasn't
either vendor-locked or hand-rolled per project.

The Comeback Story

Before (early 2026): all four pillars existed and
worked. Years of work. Real production load — 3-node
cluster, ~550 internal users at a HoReCa distributor,
~500k business objects, ~15M _values rows, three months
running every operational integration in the company.
And yet:

0 public repos. Everything in a closed monorepo.
0 NuGet packages. Even our own internal services built it from source.
0 published docs. A 14-line README.md in each project. "TODO: document this properly" in two of them.
0 external users. Nobody outside the company even knew it existed.
0 stars, 0 issues, 0 community. The classic "we'll open-source it when it's ready" trap — except it had been "ready enough for production" for over a year and "ready to share" still felt one polish-pass away every quarter.

The honest version: I was scared. ~385k LOC of opinionated
architecture decisions, exposed to public criticism, with
my real name on it. Easier to keep shipping it inside a
contract than to put it on the internet.

After (May 2026):

3 public GitHub repos under github.com/redbase-app: redb, redb-route, redb-tsak. Apache 2.0. CI, badges, real READMEs. The fourth (redb-identity) is in final polish — 1751+ tests green, repo opens before the contest deadline.
43 NuGet packages published (redb.Core, redb.Postgres, redb.Route.Kafka, redb.Tsak.CLI, ...) with ~18k cumulative downloads, the bulk of that (~15k) added in the active-launch window over the last ~10 days, on a clean upward curve day over day. External developers can dotnet add package today. Early signal: redb.Route (the integration framework) has already overtaken redb.Core in download count — 1497 vs 1482 — meaning the "Apache Camel for .NET" positioning is landing, not just the storage story. Inside Route, redb.Route.Http is the fastest-growing transport — confirming the hypothesis that HTTP is the natural first entry-point for new users exploring the DSL.
Marketing/docs site live at redbase.app — quick start, architecture page, pricing, full API docs. The site itself runs on REDB to prove the engine handles real content load.
A 4-post dev.to series already out (May 13 → May 21), with more deep dives queued. The REDB-storage post is already getting sharp technical questions in comments — including one about multi-version rolling deploys that surfaced a genuinely unsafe default in SyncStructuresFromTypeAsync (strictDeleteExtra = true) that I'd never noticed in three years of production use because we always single-version-deploy in our internal workflow. That defect went straight onto the fix list — exactly the kind of feedback that justifies open-sourcing in the first place.
First reactions and shares from the broader .NET community. Validation that the niche is real.

The completion arc isn't "I wrote a thing." The arc is
"I had a thing that worked for years inside one company
and I finally let it leave the building." Different muscle
entirely. Documentation, positioning, choosing what to call
it (RedBase / REDB / redb — we picked one and renamed
everything), writing READMEs for repos that had no audience
for years, recording first impressions, drafting tweets,
answering the first technical comment in public.

That last one is the hardest part of shipping. The code
was done. Letting it be judged was the work.

My Experience with GitHub Copilot

Honest framing first: a large part of this codebase is my
own work, written before Copilot was a serious factor.
The architectural decisions, the storage model, the
two-table layout, the tree-diff save strategy, the
direct-vm:// transport-agnostic routing fabric that lets
redb.Identity be called in-process today with zero
network hop and exposed over HTTP today and grow
additional facades over the RPC-capable Route transports
(gRPC, RabbitMQ RPC, AMQP, IBM MQ, WebSocket, SignalR,
TCP, ...) tomorrow — all without touching the auth server
itself — all of that came from years of sitting, thinking,
prototyping, and throwing prototypes away. No AI invented
those.

What Copilot changed was the finishing arc. The codebase
had the right shape, but it also had:

features sketched-in but not closed out;
helper layers half-typed because "I'll do this properly later";
XML doc-comments missing on 80% of public surfaces;
READMEs with TODO: document this;
corner cases I knew existed and had been deferring;
six different naming conventions across four years of rewrites that nobody had unified.

Copilot is what let me finally close all of that out
without burning a year of evenings.

Where it was a real force multiplier:

Finishing deferred features. Pieces I had on a
mental backlog for months — round-trip serialization
edge cases, missing operators on the LINQ provider,
Tsak CLI commands I'd been meaning to add, missing
transport options in redb.Route — Copilot let me
knock them out in evenings instead of weekends because
it could read the surrounding code, infer the
convention, and propose an implementation that I then
reviewed and corrected. The decisions stayed mine.
The typing speed and the "what was I about to do here"
recovery time collapsed.
Structuring code I already had. A lot of the work
wasn't writing new logic — it was taking working but
messy code, splitting it into the right files,
extracting the right interfaces, renaming things
consistently across ~2200 files. Copilot is excellent
at that kind of mechanical-but-context-sensitive
refactor where you need to keep semantics intact while
moving things around.
Comments and XML documentation at scale. Adding
/// <summary> to thousands of public members with
accurate descriptions of what each method actually
does — by hand this is a multi-week slog and you'll
give up after two days. With Copilot reading the
implementation and proposing the comment, then me
correcting where it was wrong, it became a steady
background task that actually finished.
Documentation and READMEs. ~12 projects, each
needing a consistent voice, accurate API examples,
correct cross-references, and a quick-start that
actually runs. With Copilot it became "draft → review
→ correct → ship" instead of "stare at empty file →
procrastinate." Same for the redbase.app positioning
pages — the "Sound familiar? / WITH REDBASE" pattern
you see on the site got drafted in one afternoon and
then iterated.
Cross-file architectural navigation while debugging.
The redb.Core.Pro ChangeTracking save path crosses
ValueTreeBuilder, ValueTreeDiff,
ProObjectStorageProviderBase, and the SQL bulk-ops
layer. Holding all four files in working memory
simultaneously while chasing a deduplication bug in
array hash updates — Copilot did the "where is this
called from, what's the contract" lookup, my brain did
the fix.
Honest fact-checking before publishing. I once
asked it to draft a reply claiming "old readers ignore
unknown structures gracefully on multi-version deploys."
It pushed back, we read the actual code together, and
the answer turned out to be "reads are graceful, writes
are destructive, and the production playbook
compensates with runtime drain." That nuance got
published. The first version would have been a small
lie. Same loop surfaced the strictDeleteExtra = true
default that's been on the fix list ever since.
RFC-driven implementation (this one mattered a lot
for Identity). redb.Identity is built almost
entirely on top of public RFCs: OAuth 2.1 / 6749,
Token Revocation 7009, Introspection 7662, Dynamic
Client Registration 7591 / 7592, JWK Thumbprint 7638,
SCIM 7644, OAuth for Native Apps 8252, Device
Authorization 8628, Token Exchange 8693, DPoP 9449,
Shared Signals 8417 — and that's not the full list.
For a solo developer, staying strictly compliant with
that many specs is brutal: every endpoint has
"MUST / SHOULD / MAY" clauses scattered across multiple
§-sections of multiple documents, edge cases like
"what status code on revoke of unknown token" (7009
§2.1: still 200), "must DPoP-Nonce rotate on every
bearing response" (9449 §8: yes), "is dpop_jkt binding
enforced at the token endpoint" (9449 §10.1: yes) \u2014
miss one and you've shipped a non-compliant auth
server. Copilot was constantly the "wait, RFC 7591 §3.2.1
says the response MUST include a registration access
token, did you cover that?" voice in the room. The
test suite reflects that: 1751+ tests with explicit
"RFC XXXX §Y.Z: ..." assertion messages, written
in the loop of "I remember the spirit of this clause,
Copilot, pull up the exact wording and let's make a
test out of it." Without that, achieving real RFC
compliance solo would have been a multi-year sub-project
on its own.

The pattern that emerged: I came in with the design
and most of the code already in place. Copilot saw what
was there, understood the conventions, and helped me
finish. Polish XML docs. Close out deferred TODOs.
Write the documentation pages. Refactor the inconsistent
bits. Catch the contradictions before they hit a public
comment thread. That's a different relationship than
"AI wrote my project." It's closer to having a very
fast pair-programmer who actually read the codebase
before sitting down.

Honest summary: ~385k LOC of infrastructure in three
years of part-time work was the human heroic effort.
Finishing it — closing out the deferred features,
filling in all the missing comments and documentation,
unifying the naming, writing the READMEs, shipping the
NuGet packages, drafting the dev.to articles, answering
the first hard comment in public — that's where Copilot
collapsed months into weeks. The contest prompt is
"revive and finish a project you started but never
completed." My project wasn't unfinished in design.
It was unfinished in all the small, deferred, boring,
necessary things that turn a working codebase into a
shippable product. Copilot is what made that finishing
arc actually fit inside one spring.

What's next (already in flight, not on a wish-list)

Shipping the four pillars wasn't the finish line — it was
the beginning of the public phase. Three big bets are
already underway, in source, in this repo:

Free engine is overtaking Pro. The
docs/FreePvtQuery
initiative is rewriting the REDB query path on top of
PostgreSQL PVT functions, and the free tier is already
ahead of Pro on a long list of features:
$case / $coalesce / $cast / n-ary $concat in
projections, full regex predicates ($regex,
$iregex, $regexReplace), extended math
($power, $sqrt, $log, $sin/cos/tan, ...),
extended string ops ($substring, $replace,
$indexOf, $padLeft/Right), projection-level
DISTINCT ON, date-extract in Select, and as of
2026-05-23 — HAVING for both regular and array
GroupBy (33/33 integration tests green across PG
Free + PG Pro + MSSql Pro). Net effect: most of what
used to be Pro-only is becoming free, and the
free engine is gaining capabilities Pro doesn't have
at all.
REDB outside C#. The whole point of building the
query layer as language-neutral JSON AST evaluated by
database-side PVT functions is that the C# LINQ
provider is just one front-end. The same _objects +
_values storage, the same scheme registry, the same
query AST can be driven from Python, from Google Apps
Script, from any language that can speak Postgres or
MSSQL — without porting the engine. That unlocks REDB
as a shared object store across heterogeneous stacks,
not just .NET shops. Architecturally, the work is
already done: every query goes through JSON AST →
PVT functions, no C#-side filter compilation required
for the heavy path.
Route keeps growing. New transport connectors are
on the roadmap, deep-dive articles for the Tsak
runtime container and for redb.Identity are queued
up after the contest deadline, and the EIP catalogue
is being expanded incrementally on top of the
existing 80+ patterns. The fastest-growing transport
(redb.Route.Http) is also driving a focused docs
pass on the HTTP path specifically.

The honest version: the ecosystem has been moving for
years and barely anyone outside one production deployment
knew. The potential is genuinely large — language-neutral
object storage with a full EIP runtime and a compliant
identity server on top of the same fabric is not a thing
that currently exists on .NET. That's the bet I'm
finishing.

Team: solo submission. All commits, all docs, all
articles by one author with Copilot in the loop.

Live in production: 3-node cluster, ~550 internal
users, ~500k objects, ~15M _values rows, ~2500 hours
uptime, zero data-layer incidents in 3 months.

Stack: .NET 8 / 9, PostgreSQL 17, MS SQL 2022,
Blazor Server, Quartz.NET, Npgsql, OpenIddict,
OpenTelemetry.

License: Apache 2.0 across all four pillars.

An EF Core alternative for .NET apps with complex object graphs — full LINQ, no migrations, no DbContext

rinat kozin — Thu, 21 May 2026 17:36:11 +0000

Today I'd like to slow down a bit and talk about redb.Core — the data engine at the heart of the RedBase ecosystem. The other pieces (redb.Route for pipelines, redb.Tsak for cluster runtime) lean on it, but this post is just about the database part.

I've been working on this project for several years. It started as an attempt to get rid of migrations and turned into what it is now — a typed object store for .NET over PostgreSQL and MSSQL.

It's not a weekend prototype. The free packages on NuGet are at version 2.0, there are 43 packages across the ecosystem, the architecture went through three rewrites, and as of this week it's been running 3 months on production at a 30-year national food distributor (more on that below).

This post is a technical walkthrough of redb.Core — what it is, how it differs from EF Core, what the generated SQL actually looks like, what the production workload looks like, and what's shipping next.

What redb.Core actually is

github.com/redbase-app/redb — Apache 2.0

RedBase stores typed C# objects in two tables (_objects + _values) over PostgreSQL or Microsoft SQL Server. Not JSON blobs. Not JSONB. Real typed columns with FK constraints — NUMERIC(38,18) for money, timestamptz for dates, uuid for GUIDs. Real B-tree indexes. ACID transactions.

The schema is your C# class:

[RedbScheme("Employee")]
public class EmployeeProps
{
    public string FirstName   { get; set; } = "";
    public string LastName    { get; set; } = "";
    public int    Age         { get; set; }
    public decimal Salary     { get; set; }
    public DateTime HireDate  { get; set; }
    public string[]? Skills   { get; set; }
    public Address? HomeAddress { get; set; }
    public Dictionary<int, decimal>? BonusByYear { get; set; }
}

That attribute is the entire schema definition. Call SyncSchemeAsync<EmployeeProps>() once — done. Add a property next sprint — redeploy, call sync again. Old objects still work. No migration files. No DBA ticket. No 2am rollback story.

// Save — entire object graph, one call
await redb.SaveAsync(employee);

// Load — full graph, arrays, dicts, nested classes — all materialized
var e = await redb.LoadAsync<EmployeeProps>(id);

// Query — real LINQ, real SQL
var seniors = await redb.Query<EmployeeProps>()
    .Where(e => e.Salary > 100_000 && e.Age >= 35)
    .OrderByDescending(e => e.Salary)
    .Take(50)
    .ToListAsync();

No DbContext. No Include chains. No Add-Migration. No mapper layer.

Object graphs in one call

This is the part that surprises people coming from EF. Props can contain other Props — single references, arrays, dictionaries — and the entire graph saves and loads as one operation:

[RedbScheme("Order")]
public class OrderProps
{
    public Customer Customer { get; set; }                          // nested class
    public Address ShippingAddress { get; set; }                    // nested class
    public Product[] Products { get; set; }                         // array of classes
    public RedbObject<PaymentProps>[] Payments { get; set; }        // array of full objects with own IDs
    public Dictionary<string, RedbObject<CouponProps>> Coupons { get; set; }  // dict of objects
    public Dictionary<(int Year, string Quarter), string> Reviews { get; set; } // tuple-key dict
}

await redb.SaveAsync(order);   // entire graph persisted, FK ordering handled
var loaded = await redb.LoadAsync<OrderProps>(id);
// loaded.Props.Customer.Address — ready
// loaded.Props.Payments[0].Props — ready (full RedbObject with own Id, DateCreate, etc.)
// loaded.Props.Coupons["SUMMER20"].Props — ready

In EF Core this would be 28 tables, ~40 Include/ThenInclude calls, manual junction tables for the many-to-many, and INSERT ordering that breaks every time someone adds a non-nullable FK without a default.

In RedBase: one SaveAsync, one LoadAsync. The nested RedbObject instances are real first-class objects — they have their own IDs, their own timestamps, they can be queried independently, they participate in tree structures. They are not denormalised JSON glued to the parent.

What the LINQ actually compiles to

Where(e => e.Salary > 100_000 && e.Age >= 35) doesn't get serialized to JSON and re-parsed (that's the Free engine's path — covered later). In Pro, the C# expression tree is walked node-by-node and emitted as parameterized SQL. Roughly:

WITH pvt AS (
  SELECT v._id_object,
         (array_agg(v._Numeric) FILTER (WHERE v._id_structure = $1))[1] AS "Salary",
         (array_agg(v._Long)    FILTER (WHERE v._id_structure = $2))[1] AS "Age"
    FROM _values v
   WHERE v._id_structure = ANY($3::bigint[])
     AND v._id_object IN (SELECT o._id FROM _objects o WHERE o._id_scheme = $4)
   GROUP BY v._id_object
)
SELECT o.*
  FROM _objects o
  JOIN pvt ON pvt._id_object = o._id
 WHERE pvt."Salary" > $5
   AND pvt."Age"    >= $6
 ORDER BY pvt."Salary" DESC
 LIMIT 50;

Parameterized. Plan-cached by PostgreSQL. One index scan on (_id_structure, _id_object), one aggregation pass, B-tree filter on flat columns. The number of filter fields doesn't change the shape of the query.

The C# → SQL compiler handles arithmetic (*, +, %), Math.*, String.Contains/StartsWith/Trim/ToLower, DateTime.Year/Month/..., nullable navigation (x.Address?.City) compiled to IS NOT NULL, the ternary operator compiled to CASE WHEN, StringComparison.OrdinalIgnoreCase compiled to native ILIKE, dictionary access dict["key"] compiled to pivot columns, and a few more edge cases that EF Core itself doesn't always handle.

You can preview the SQL of any query without executing it:

var sql = await query.ToSqlStringAsync();

Like IQueryable.ToQueryString() in EF, but works for trees, GroupBy, window functions too.

Why bulk save is so fast — two tables, two streams

Something to understand before the change-tracking section: the storage layout is two tables, so SaveAsync of a batch is two bulk operations, not N round-trips.

On PostgreSQL the provider uses Npgsql's BeginBinaryImportAsync — native COPY protocol, binary format:

// from redb.Postgres/Data/NpgsqlBulkOperations.cs
await using var writer = await conn.BeginBinaryImportAsync(
    "COPY _objects (_id, _id_parent, _id_scheme, _name, ...) FROM STDIN (FORMAT BINARY)");
foreach (var obj in objectsList) {
    await writer.StartRowAsync();
    await writer.WriteAsync(obj.Id,        NpgsqlDbType.Bigint);
    await writer.WriteAsync(obj.IdScheme,  NpgsqlDbType.Bigint);
    // ... typed writes
}
await writer.CompleteAsync();

For a batch save: one COPY stream writes the _objects rows, another writes the _values rows. Two streams, no per-row round-trips, no string-formatted INSERTs. MSSQL uses SqlBulkCopy for the same role.

This is why 1000 routes × ~40 fields = ~40,000 value rows save in tens of milliseconds inside the 200–300 ms budget. The bottleneck is the network round-trip and the COPY write, not the ORM machinery — there isn't any ORM machinery in the hot path.

Change tracking without DbContext (Pro)

DbContext keeps an in-memory snapshot of every entity you load — that's how it knows what changed. It's also why it isn't thread-safe and why the cache dies with the request.

RedBase Pro takes a different approach. With PropsSaveStrategy.ChangeTracking (Pro only; the free tier uses DeleteInsert), SaveAsync does this on a batch:

One bulk SELECT of existing values for all object IDs being saved.
Build two ValueTreeNode trees — one from your in-memory objects, one from the DB state.
Structural diff — subtrees with matching hashes are skipped entirely (no value comparison, no child traversal). Inserts, updates, and deletes are computed per node.
Three bulk operations — BulkInsertValuesAsync, BulkUpdateValuesAsync, BulkDeleteValuesAsync — each a single round-trip. Inserts go through COPY BINARY again.

Net effect: changing one field in a deeply nested object emits one UPDATE, not a full delete-and-reinsert of the entire props graph. And the comparison happens in C# on the application side — no DbContext lifetime to worry about, safe to run from a background Channel consumer or a Parallel.ForEach.

(In the production code from the previous section, the application also does its own obj.ComputeHash() check at the route level. That part is optional business-code — you could call SaveAsync on every route and the Pro tree-diff would still skip unchanged values internally. It's there as a coarse pre-filter so unchanged routes don't even enter the save pipeline at all, saving the diff work too.)

Production deployment — the numbers

The biggest deployment right now:

30-year national HoReCa food distributor
~150,000 orders/month, ~20,000 B2B customers, 600+ cities
3-node cluster, 4 cores / 8 GB RAM / 50 GB SSD per node
~550 daily internal users (operators, drivers, supervisors, dispatch, back-office)
10–15% CPU under full load
Integrations: SAP, Kafka, RabbitMQ, GPS feeds, Mercury / EGAIS / government APIs

Three months in production, no data-layer incidents. Two projects in the company use it, the second one came after the first one proved stable.

Real workload, real timings

The hottest pipeline is the SAP monitoring sync. Every 60 seconds a SQL polling consumer calls a stored procedure on SAP S/4 (usp_TsUM_MonitoringReport_xml), gets ~1000 transportation orders back as XML, syncs reference dictionaries (drivers, vehicles, list items), bulk-loads existing routes from RedBase, hash-compares each one, and saves only the changed ones.

The whole loop fits in ~200–300 ms for ~1000 routes. The actual production code:

From("direct://tsum")
    .RouteId("tsum-processing")
    .ProcessWithRedb(async (redb, exchange, ct) =>
    {
        var orders = (List<TransportationOrder>)exchange.In.Body;

        var sw = Stopwatch.StartNew();

        // 1. Sync dictionaries (Drivers, Vehicles, ListItems) — only new/changed
        var dicts = await DictionarySyncService.SyncFromOrdersAsync(redb, orders, ct);
        if (dicts.DriversNew + dicts.DriversChanged + dicts.VehiclesNew
            + dicts.VehiclesChanged + dicts.ListItemsRelinked > 0)
            await RefDataCache.RefreshAsync(redb);
        var syncMs = sw.ElapsedMilliseconds;

        // 2. Bulk load existing routes by Code — one query, ~1000 codes
        var codes = orders.Select(o => o.Code).ToList();
        var existing = await redb.Query<TransportationRoute>()
            .WhereRedb(o => codes.Contains(o.ValueString!))
            .ToListAsync();
        var existingByCode = existing.ToDictionary(o => o.ValueString!, o => o);

        // 3. Merge: update if hash changed, insert if new, skip if unchanged
        var toSave = new List<IRedbObject>();
        var updatedCount = 0;
        var skippedCount = 0;
        foreach (var order in orders)
        {
            var routeProps = MapOrderToRouteProps(order, dicts);
            if (existingByCode.TryGetValue(order.Code, out var obj))
            {
                var hashBefore = obj.ComputeHash();
                EnrichRouteFromOrder(obj.Props!, routeProps);
                if (obj.ComputeHash() != hashBefore) { toSave.Add(obj); updatedCount++; }
                else skippedCount++;
            }
            else
            {
                toSave.Add(new RedbObject<TransportationRoute>
                {
                    name = $"Route {order.Code}",
                    value_string = order.Code,
                    Props = routeProps
                });
            }
        }

        // 4. One batched save — mixed inserts + updates
        if (toSave.Count > 0)
            await redb.SaveAsync(toSave);

        Logger.LogInformation(
            "[TSUM] orders={Orders} routes(+{Created} ~{Updated} ={Skipped}) " +
            "drivers(+{DN} ~{DC}) vehicles(+{VN} ~{VC}) " +
            "sync={Sync} query={Query} save={Save} total={Total}ms",
            orders.Count, toSave.Count - updatedCount, updatedCount, skippedCount,
            dicts.DriversNew, dicts.DriversChanged,
            dicts.VehiclesNew, dicts.VehiclesChanged,
            syncMs, queryMs, saveMs, sw.ElapsedMilliseconds);
    });

TransportationRoute has ~40 fields and 12 RedbListItem references (Driver, Vehicle, CarMark, ShippingPoint, BusinessType, PlaceTo, PlaceFrom, LoadingZone, TransportStatus, Risk, DeliveryStatus, LoadStatus) plus 2 object references to AD users. Every single one is a foreign key in the database. None of them require a JOIN at query time — the materializer handles it.

Smaller queries (point-lookup of a single route, dictionary fetch, REST endpoints for the UI) run in 50–100 ms including HTTP overhead.

What's actually in the database

After running this in production, the storage looks like this:

~1000 transportation routes/day, plus delivery points, transport snapshots, garage states, slice settings, slice snapshots, drivers, vehicles, yard places, AD user refs — about a dozen [RedbScheme] classes in active use
~500,000 objects in _objects (routes, points, snapshots, dictionary items)
~15,000,000 rows in _values (every typed property of every object)
All of that lives in 2 tables — _objects and _values — plus the system tables for schemes, structures, lists, users, permissions

15 million typed value rows. Two tables. No 30-table schema. No 200 migration files. The query times above are on this dataset.

If you tried to model the same domain in EF Core flat tables, you'd end up with roughly: Routes, RoutePoints, TransportSnapshots, GarageStates, SliceSettings, SliceSnapshots, Drivers, Vehicles, CarMarks, ShippingPoints, BusinessTypes, YardPlaces, LoadingZones, TransportStatuses, DeliveryStatuses, Risks, TripRisks, AdUsers — plus junction tables and audit tables for each. 30+ tables, dozens of migrations, and every schema change is a deploy event.

In RedBase the schema change is git push. The next InitializeAsync() call adds the new structure rows. Done.

What would EF Core look like here?

I asked myself the same question before starting. Let's count what EF would need:

TransportationRoute entity → 1 table
12 ListItem references → 12 lookup tables + 12 nullable FKs
2 AD user references → 2 more FKs
The 1000-route batch update → 1000 entities tracked in DbContext for change detection
Hash-based skip of unchanged objects → doesn't exist in EF out of the box; you write it manually
The same logic across multiple processes/routes → DbContext is per-request, the change-tracker cache dies at the end of every batch

The realistic EF flow looks like this:

// Load existing — needs Include for every lookup or you get N+1
var existing = await db.Routes
    .Include(r => r.Driver)
    .Include(r => r.Vehicle)
    .Include(r => r.CarMark)
    .Include(r => r.ShippingPoint)
    .Include(r => r.BusinessType)
    .Include(r => r.PlaceTo)
    .Include(r => r.PlaceFrom)
    .Include(r => r.LoadingZone)
    .Include(r => r.TransportStatus)
    .Include(r => r.Risk)
    .Include(r => r.DeliveryStatus)
    .Include(r => r.LoadStatus)
    .Include(r => r.RiskSetBy)
    .Include(r => r.KcResponsible)
    .Where(r => codes.Contains(r.Code))
    .ToListAsync();

// Map, mutate, SaveChanges — 1000 tracked entities, full diff snapshot in RAM
// SaveChanges fires N UPDATE statements (one per row) in a transaction
await db.SaveChangesAsync();

In practice on a workload like this:

The Include chain on 14 lookups produces a query that PostgreSQL/MSSQL can't always optimise cleanly (cartesian explosion risk; you split-query and pay multiple round trips).
SaveChanges on 1000 modified entities emits 1000 individual UPDATEs unless you reach for EFCore.BulkExtensions or similar third-party libraries. RedBase ships COPY BINARY for inserts and a batched UPDATE path for updates in the box.
The DbContext snapshot of 1000 tracked entities, each with 14 included lookups, is real memory pressure. AsNoTracking() is faster but you lose change detection — you have to re-implement it.
The 200–300 ms budget on a 4-core container is not realistic in this scenario without significant manual optimisation and probably a separate bulk-update path.

I'm not saying EF can't do it. I'm saying the equivalent EF implementation is more code, more moving parts, and significantly slower without third-party packages. RedBase ships a single SaveAsync(toSave) and a hash-based skip primitive in the box.

The hash comparison itself (obj.ComputeHash() != hashBefore) is application-level business code — it's optional; the Pro tree-diff inside SaveAsync would skip unchanged values anyway. But used like this it lets the application skip the ~95% of routes that didn't change between SAP polls before they even enter the save pipeline. Combined with COPY BINARY into two tables and the Pro tree-diff for the routes that do change, the whole loop fits the 200–300 ms budget. No equivalent set of built-in primitives exists in EF Core; you either snapshot manually, re-save everything, or pull in third-party bulk packages.

What EF Core does with complex objects (vs what RedBase does)

The E000 benchmark uses EmployeeProps — a realistic model with nested classes, arrays, dictionaries, and RedbObject references:

What classic ORM needs	RedBase
~28 tables	2 tables
FK ordering on every INSERT	Single `SaveAsync`
40+ `Include`/`ThenInclude` calls	`LoadAsync<T>(id)` — one line
Migration file for every new field	Add property, call `SyncSchemeAsync`, done
~5,000 INSERTs for 100 employees	1 `BulkInsert` via COPY protocol

For 100 complex employees: EF Core ≈ 4,000–6,000 INSERTs across 28 tables in FK order. RedBase: ~3,000 typed value rows, one COPY command.

And the test results are published — 525 automated tests across all editions and both databases. All green.

The query engine: free vs Pro

This is where it gets technically interesting.

Free edition compiles your LINQ lambda to a JSON facet format, then calls a plpgsql stored procedure that parses that JSON and generates SQL dynamically. For simple 1–2 field filters, PostgreSQL optimizes correlated EXISTS subqueries well. It works.

Pro edition walks the C# expression tree directly in C# (ExpressionToSqlCompiler), emits native parameterized SQL. Same plan every call — PostgreSQL can cache it. 3–10× faster on complex multi-field filters.

But here's what's happening right now: I'm porting the Pro PVT CTE query engine into the free tier. The plan is full parity on PVT — same CTE shape, same expression coverage, same projection capabilities. Free and Pro will speak the same query dialect.

The remaining differences between Free and Pro will be elsewhere:

Query plan caching. Pro builds parameterized SQL in C# (ExpressionToSqlCompiler) — same shape every call, PostgreSQL caches the plan once and reuses it. The free tier generates SQL inside plpgsql with literal values inlined (properly escaped to prevent injection), so each call can produce a slightly different plan. Same correctness, different plan-cache behavior.
Materialization. Pro has a parallel materializer (multiple values streams hydrated concurrently into the object graph). Free uses a simpler sequential path.
Save strategy. PropsSaveStrategy.ChangeTracking (tree-diff with hash-based subtree skip) is Pro-only. Free uses DeleteInsert — still bulk via COPY, but no per-field diff.

So Pro stays faster on hot paths and large object graphs, but the query language itself — what you can write in .Where, .Select, projections, grouping, having, FTS, regex — will be the same on both sides.

Show me the SQL it generates

Here's a real example — a projection query on EmployeeProps with 16 computed columns:
full names, salary calculations, date extractions, seniority classification via CASE, coalesce for nullable fields, explicit type cast. This is the free tier:

WITH _pvt_cte AS (
    SELECT
        v._id_object,
        (array_agg(v._Long)    FILTER (WHERE v._id_structure = 1000017))[1] AS "Age",
        (array_agg(v._Numeric) FILTER (WHERE v._id_structure = 1000020))[1] AS "Salary",
        (array_agg(v._DateTimeOffset) FILTER (WHERE v._id_structure = 1000018))[1] AS "HireDate",
        (array_agg(v._String)  FILTER (WHERE v._id_structure = 1000016))[1] AS "LastName",
        (array_agg(v._String)  FILTER (WHERE v._id_structure = 1000015))[1] AS "FirstName"
    FROM _values v
    WHERE v._id_structure = ANY(ARRAY[1000015,1000016,1000017,1000018,1000020]::bigint[])
      AND v._id_object IN (SELECT o._id FROM _objects o WHERE o._id_scheme = 1000014)
    GROUP BY v._id_object
)
SELECT
    o._id                                                          AS id,
    ("FirstName" || ' ' || "LastName")                            AS full_name,
    UPPER("FirstName")                                             AS upper_name,
    LENGTH("FirstName")                                            AS name_len,
    ("Salary" * 12)                                                AS yearly_x12,
    (("Salary" * 0.15) - ("Salary" * 0.013))                      AS bonus_after_tax,
    ("Salary" / ("Age" + 1))                                       AS avg_per_year,
    ("Age" % 10)                                                    AS age_mod10,
    ABS(("Age" - 35))                                              AS abs_diff,
    FLOOR(("Salary" / 2))                                          AS floor_half_salary,
    EXTRACT(YEAR  FROM "HireDate")                                 AS hire_year,
    EXTRACT(MONTH FROM "HireDate")                                 AS hire_month,
    EXTRACT(YEAR  FROM AGE("HireDate", '2026-05-20'))              AS tenure_years,
    CASE
        WHEN ("Age" >= 60) THEN 'senior'
        WHEN ("Age" >= 35) THEN 'mid'
        ELSE 'junior'
    END                                                            AS seniority,
    COALESCE("FirstName", '<unknown>')                             AS display_name,
    ("Age")::text                                                  AS age_as_text
FROM _pvt_cte
JOIN _objects o ON o._id = _pvt_cte._id_object
WHERE "Salary" > '0'::numeric
ORDER BY "Salary" DESC
LIMIT 3

One _values scan. One GROUP BY. Sixteen computed output columns. Standard SQL — no magic, fully readable in pgAdmin, fully EXPLAIN ANALYZE-able.

This query was generated by pvt_build_projection_sql() — a plpgsql function that is shipping in the next free release.

When this fits and when it doesn't

Honest take.

Fits well: business apps with collections, nested structures, hierarchies, dictionaries, or schemas that change often. Apps where a business analyst can ask "add a field" and you want to ship the same day. Anything where you'd otherwise end up with 28 tables for one logical entity.

Doesn't fit: flat reporting databases with two or three fixed tables. Use Dapper there — it's honest and fast.

The common pushback is: "EF Core is good enough for 80% of projects." We'd flip that around: RedBase fits 80% of typical business projects — anything with collections, nested structures, lookup tables, hierarchies, dictionaries, or a schema that evolves alongside the product. The 20% where it's overkill is the genuinely flat case: two or three fixed tables, pure reporting, no evolving model. Use Dapper there; it's honest and fast. For everything else — which is most business software — the EF migration tax is real and it compounds with every sprint.

Three more axes that usually get left out of that comparison:

Extensibility. Adding a field is a property in C# and one InitializeAsync() call — no migration script, no review, no deploy window. The same change in an EF stack is at minimum: model edit + Add-Migration + reviewed SQL + coordinated deploy. Multiply by every iteration in a year.
Floor skill level. With RedBase, a junior developer who knows C# can add entities, fields, queries, and relationships on day one — the schema is just a class, the query is just LINQ. With EF Core on a complex model, you need to understand N+1, cartesian explosion from Include chains, DbContext lifetime, migration conflicts on shared branches, and when to reach for AsNoTracking(). That knowledge takes months to build and years to apply consistently. RedBase moves that complexity into the framework and out of the application code.
Cost of development. On the TsUM project the data-layer work — schemas, storage, queries, sync pipelines, change-tracking, bulk save — came in at roughly 128 person-hours. The estimate for the same scope on a classic EF + handwritten bulk-update + ASP.NET Identity + 30-table migration stack was on the order of 3000 person-hours. That's not a typo and it's not marketing — it's the gap between "add a property, redeploy" and "add a column, write migration, update DTOs, update mapper, update repository, write tests for all of it". The 80%/20% framing only holds if you count lines of code; once you count engineer-hours over a project lifetime it inverts.

What else is in the box

195+ working examples in the repo. The shortlist of things people usually don't expect to be built-in:

// Tree queries — recursive CTE underneath, no CTE to write yourself
var products = await redb.TreeQuery<Product>(londonHQ.Id, maxDepth: 10)
    .Where(p => p.InStock)
    .ToListAsync();

// Soft delete with atomic mark + background purge + progress in DB
await redb.SoftDeleteAsync(ids, user);

Also shipping in the core (this is the short list — the real list is much longer):

GroupBy + window functions in one query
Built-in users / roles / permissions (no ASP.NET Identity dependency, no separate auth schema)
Export / import via .redb files (PostgreSQL ↔ MSSQL portability)
Multi-database in one process with full domain isolation — separate connections, separate scheme caches, separate object caches per domain; one app can talk to several independent RedBase databases at the same time without their metadata leaking into each other
Scheme cache and object cache (both per-domain), invalidated on SyncSchemeAsync / SaveAsync, used by the materializer to skip repeated metadata lookups
Atomic soft-delete with background purge and progress visible in the DB
Tree queries with recursive CTE
Polymorphic trees — different C# types at each level of the same tree

There's a lot more that doesn't fit in one section. The architecture page has the full map.

What's coming next

The FreePvtQuery release is the nearest milestone. This is a complete rewrite of the free tier's query engine based on the PVT CTE architecture — bringing it to full parity with Pro on the query side:

Single-pass _values scan (vs correlated EXISTS per field)
Full expression system: arithmetic, string, date, math, regex, FTS, CASE, coalesce, cast
Projections with arbitrary computed columns (like the SQL above)
HAVING, DISTINCT ON, GROUP BY extensions

After that: MSSQL support for the new engine, C# LINQ-to-JSON bridge so you keep writing .Where(e => e.Salary > 100_000) without touching JSON. Pro will keep its edge on plan caching (parameterized SQL), parallel materialization, and tree-diff change tracking.

This is maybe 10% of what's in there

This post covered the basics: storage model, query engine, free PVT demo, production deployment. Topics I didn't touch:

Polymorphic tree hierarchies (different C# types at every level, same tree API)
Object graphs with RedbObject<T> references inside Props
Data migrations (Pro): fluent API, computed columns, in-place type changes
Export/import: .redb files (JSONL/ZIP), PostgreSQL ↔ MSSQL portability
Multi-database in one process (domain-isolated caches)
redb.CLI — schema management from command line
redb.PropsEditor — runtime props editing UI
Integration with redb.Route (22-transport pipeline engine) — separate article

If there's a direction you want covered next — migrations, polymorphic trees, benchmarks, internals of the diff-tree change tracking — drop it in the comments and that's what the next post will be.

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL

rinat kozin — Sun, 17 May 2026 10:26:13 +0000

Apache Camel has been solving enterprise integration on the JVM since 2007 — 22k stars, 300+ transports, hundreds of production deployments at banks, telcos, governments. The .NET ecosystem never got a real equivalent. MassTransit and Wolverine cover message-bus and saga scenarios well, but they aren't pipeline engines and they don't pretend to be.

redb.Route is the missing piece: a fluent C# DSL that wires Kafka, RabbitMQ, Redis, SQL, HTTP, gRPC, SFTP, MQTT, S3 and 14 more transports through From → Process → To pipelines, with 30+ Enterprise Integration Patterns and a compiled expression engine. Apache 2.0, .NET 8 / 9 / 10. This post is a technical walkthrough.

The shape of every pipeline

Every redb.Route pipeline is From → [processors] → To. Messages flow as IExchange instances carrying a body, headers, and properties.

From("kafka://orders?groupId=svc&brokers=localhost:9092")
    .Filter(Header("type").isEqualTo("new"))
    .To("rabbitmq://events?host=localhost");

For complex routing, group routes into a RouteBuilder:

public class OrderRoutes : RouteBuilder
{
    protected override void Configure()
    {
        From("kafka://orders?groupId=svc&brokers=localhost:9092")
            .RouteId("order-pipeline")
            .Choice()
                .When(Header("priority").isEqualTo("high"))
                    .Log("High priority order")
                    .To("direct://fast-lane")
                .When(Header("priority").isEqualTo("low"))
                    .To("seda://batch-queue")
                .Otherwise()
                    .To("direct://standard")
            .EndChoice();

        From("direct://fast-lane")
            .Retry(3)
            .SetHeader("processed-at", e => DateTimeOffset.UtcNow)
            .Process(async (exchange, ct) =>
            {
                var body = exchange.In.Body as string;
                exchange.In.Body = $"PROCESSED: {body}";
            })
            .To("rabbitmq://processed?host=localhost");
    }
}

Registration:

builder.Services.AddRedbRoute(route => route.AddRouteBuilder<OrderRoutes>());
builder.Services.AddRedbRouteKafka();
builder.Services.AddRedbRouteRabbitMQ();

22 transports as first-class URI schemes

kafka://          rabbitmq://       redis://          sql://
http://           grpc://           sftp://           ftp://
mqtt://           s3://             ibmmq://          amqp://
azuresb://        elasticsearch://  firebase://       ldap://
mail://           tcp://            websocket://      signalr://
cron://           file://

+ built-in: direct://   seda://   timer://   log://   mock://

Every transport uses the same IExchange contract. Swapping Kafka for RabbitMQ means changing the From URI — the pipeline logic is unchanged.

Type-safe fluent builders are available as an alternative to URI strings:

var source = Kafka.Topic("orders")
                  .Brokers("broker1:9092,broker2:9092")
                  .GroupId("order-svc")
                  .Acks("All");

From(source).To("seda://internal");

EIP patterns as DSL steps

Content-Based Router — route by message content:

From("kafka://events")
    .Choice()
        .When(Header("type").isEqualTo("order"))   .To("direct://orders")
        .When(Header("type").isEqualTo("invoice"))  .To("direct://invoices")
        .Otherwise()                                .To("seda://unclassified")
    .EndChoice();

Splitter + Aggregator — split a batch, process each item, re-aggregate by correlation key. Sequential by default; one method turns the split into a bounded-parallel pipeline:

From("seda://batch")
    .Split(Body())                                     // Body() helper — split the IEnumerable body
        .ParallelProcessing()                          // process items concurrently
        .MaxDegreeOfParallelism(8)                     // bounded fan-out
        .Process(async (ex, ct) => await EnrichOrder(ex))
        .To("direct://enriched")
    .EndSplit();

From("direct://enriched")
    .Aggregate(Header("batch-id"), new ListAggregationStrategy())
    .To("rabbitmq://processed-batches");

For true fan-out to multiple endpoints in parallel use Multicast:

From("kafka://orders")
    .Multicast("direct://pricing", "direct://inventory", "direct://fraud-check");

WireTap — copy every message to an audit sink without interrupting the main flow:

From("kafka://transactions")
    .WireTap("direct://audit-log")
    .Process(async (ex, ct) => await ProcessTransaction(ex))
    .To("rabbitmq://completed");

Idempotent Consumer — deduplicate across a cluster. Plug in the repository (in-memory, SQL, or redb.Core EAV) and the same message ID is rejected on every node:

From("kafka://payments")
    .IdempotentConsumer(
        e => e.In.GetHeader<string>("payment-id"),
        new RedbIdempotentRepository(redbService))   // cluster-wide, two-phase commit
    .Process(async (ex, ct) => await ProcessPayment(ex));

Saga — pipeline-level choreography with compensating steps. If ChargePayment throws, ReleaseInventory runs automatically. No external state-machine framework required:

From("direct://checkout")
    .Saga(s => s
        .Step(
            action:     async (ex, ct) => await inventory.Reserve(ex, ct),
            compensate: async (ex, ct) => await inventory.Release(ex, ct))
        .Step(
            action:     async (ex, ct) => await payments.Charge(ex, ct),
            compensate: async (ex, ct) => await payments.Refund(ex, ct))
        .Step(
            action:     async (ex, ct) => await shipments.Create(ex, ct),
            compensate: async (ex, ct) => await shipments.Cancel(ex, ct))
        .OnCompletion(async (ex, ct) => await PublishOrderCompleted(ex, ct)))
    .To("rabbitmq://order-confirmed");

Full catalogue: Filter, Choice, Splitter, Aggregator, Multicast, WireTap, Recipient List, Dynamic Router, Resequencer, Scatter-Gather, Claim Check, Idempotent Consumer, Saga, Circuit Breaker, Throttle, Retry, Dead Letter, Loop, Delay, Debounce, Enrich, Timeout, TryCatch, Transacted, Process, Validate. 30+ patterns, all first-class DSL.

Error handling — four composable layers

Most .NET libraries give you one mechanism for failures: a retry policy on the consumer. redb.Route exposes four, designed to compose: per-step Retry, scoped DoTry/DoCatch, route-local OnException, and global OnException declared at the RouteBuilder level. Plus DeadLetterChannel for messages that exhaust retries.

Dead Letter Channel + a DLQ sub-route that knows why it failed

The failing exception travels with the exchange. The DLQ sub-route can read it, branch on the type, log structured info, archive, retry later:

From("kafka://orders")
    .DeadLetterChannel("seda://orders-dlq")
    .Retry(3)
    .Process(async (ex, ct) => await ProcessOrder(ex, ct))
    .To("rabbitmq://processed");

// The DLQ is a real route — inspect, branch, react
From("seda://orders-dlq")
    .Log("DLQ: ${header.correlationId} — ${exception.message}")
    .Choice()
        .When(e => e.GetException() is TimeoutException)
            .Delay(TimeSpan.FromMinutes(5))
            .To("seda://retry-later")
        .When(e => e.GetException() is HttpRequestException)
            .To("sftp://archive/http-failures/")
        .Otherwise()
            .To("sftp://archive/poison/")
    .EndChoice();

OnException — per-exception redelivery with exponential backoff

Declared globally at RouteBuilder level (applies to every route in the builder) or scoped to a single route. Each block configures attempts, delay, backoff, and what to do when the handler succeeds:

public class OrderRoutes : RouteBuilder
{
    protected override void Configure()
    {
        // Global — applies to every From(...) below
        OnException<HttpRequestException>()
            .MaximumRedeliveries(5)
            .RedeliveryDelay(TimeSpan.FromSeconds(1))
            .UseExponentialBackOff()
            .BackOffMultiplier(2.0)
            .Handled()                          // mark as handled — exchange continues normally
            .To("seda://http-failures")
        .EndOnException();

        OnException<DbException>()
            .MaximumRedeliveries(2)
            .UseOriginalMessage()               // restore original body before sending to handler
            .OnWhen(e => !((DbException)e.GetException()).Message.Contains("deadlock"))
            .To("seda://db-failures")
        .EndOnException();

        // Multiple exception types in one block
        OnException(typeof(TimeoutException), typeof(SocketException))
            .MaximumRedeliveries(3)
            .RedeliveryDelay(TimeSpan.FromSeconds(2))
            .To("seda://network-failures")
        .EndOnException();

        From("kafka://orders")
            .To("http://payments-svc/charge");

        From("kafka://shipments")
            .To("http://logistics-svc/dispatch");
    }
}

Handled(), Continued(), OnWhen(predicate), RetryWhile(predicate), UseOriginalMessage() — all standard Camel error-handling primitives, and none of the .NET alternatives ship them as DSL.

TryCatch — scoped try/catch/finally inside a pipeline

When only one section of a route needs special handling:

.DoTry()
    .To("http://external-api/submit")
    .Process(async (e, ct) => await PostProcess(e, ct))
.DoCatch<HttpRequestException>()
    .Log("HTTP failure: ${exception.message}")
    .To("seda://retry-queue")
.DoCatch<TimeoutException>()
    .To("sftp://archive/timeouts/")
.DoFinally()
    .Log("Attempt complete")
.End()

Compiled expression engine

This is the one feature that distinguishes redb.Route from both Apache Camel and every .NET alternative. Inline expressions — string templates, arithmetic, comparisons, JSONPath, XPath — are translated to real Func<IExchange, T> delegates via System.Linq.Expressions at route-build time. No interpreter, no per-message parsing.

// String templates
.SetBody(Expr("${header.orderId}-${body}"))
.SetHeader("trace", Expr("${header.source}-${header.correlationId}"))

// Pre/post-increment
.SetHeader("attempt", Expr("${header.attempt++}"))   // returns old value
.SetHeader("attempt", Expr("${++header.attempt}"))   // returns new value

// Arithmetic
.SetHeader("total", Expr("${header.qty * header.price}"))
.SetHeader("net",   Expr("${header.gross - header.tax}"))

// Predicates — fluent on top of compiled expressions
.Filter(Expr("header.amount").isGreaterThan(1000))
.When(Header("status").isEqualTo("active"))

Apache Camel's Simple Language is interpreted at every message dispatch. MassTransit, Wolverine and NServiceBus have no expression engine at all — every conditional is a hand-written C# lambda. With redb.Route you get both: terse string DSL for configuration-driven rules and strongly typed lambdas where you want them, with the same zero-overhead delegate at the bottom.

The engine supports 9 value types and 17 predicates, and the result of every Expr(...) is cached per route. You pay for parsing once at startup.

Transactional pipelines

A pipeline can wrap several steps in a single transaction. .Transacted() opens a TransactionScope; transports that implement ITransactedAction enlist into it. That means the Kafka commit, the RabbitMQ publisher confirm, and the SQL UPDATE all succeed or fail together — no half-processed messages, no manual two-phase coordination:

From(Kafka.Topic("orders")
        .Brokers("broker:9092")
        .GroupId("order-svc")
        .IsolationLevel("ReadCommitted")
        .EnableAutoCommit(false))                      // commit driven by .Transacted()
    .Transacted()
    .Process(async (ex, ct) => await Validate(ex, ct))
    .To(Sql.Execute("INSERT INTO orders (...) VALUES (...)")
           .DataSource("main")
           .Transacted())                              // enlists into the same scope
    .To(Rabbit.Queue("order-events")
              .Confirms(true));                        // publisher confirm before commit

MassTransit, NServiceBus and Wolverine all solve this for their own bus, but only for the bus. redb.Route makes it work across any combination of transports that implement ITransactedAction — Kafka EOS, RabbitMQ tx channels, IBM MQ, AMQP 1.0, SQL.

Outbox without an outbox framework

The transactional outbox pattern is usually presented as a feature you opt into via a framework (MassTransit, NServiceBus, Wolverine all bundle one). In redb.Route it's just four lines composed from existing primitives — SQL polling, transactional sink, idempotent consumer:

From(Sql.Poll("SELECT id, payload FROM outbox WHERE processed = 0 LIMIT 100")
        .DataSource("main")
        .OnSuccess("UPDATE outbox SET processed = 1 WHERE id = ANY(@ids)")
        .Transacted())                                  // atomic claim + publish
    .Split(Body())
    .IdempotentConsumer(
        e => e.In.GetHeader<string>("eventId"),
        new RedbIdempotentRepository(redbService))      // dedup on republish
    .To(Kafka.Topic("events")
             .EnableTransactionalProducer(true)
             .Acks("All"));                             // exactly-once on the broker

No magic table conventions, no separate IOutbox interface to register, no required ORM. It's pipelines all the way down.

Request-Response — HTTP and gRPC as first-class endpoints

The same DSL that handles fire-and-forget Kafka also handles synchronous RPC. Mark the listener InOut(), set In.Body to the response anywhere in the pipeline — the HTTP transport sends it back to the caller:

public class OrderApi : RouteBuilder
{
    protected override void Configure()
    {
        From(Http.Listen("/api/orders").Port(8080).InOut())
            .Unmarshal(typeof(JsonMessageSerializer), typeof(CreateOrderRequest))
            .Validate(e => (e.In.Body as CreateOrderRequest)?.Amount > 0,
                      "Amount must be positive")
            .Process(async (e, ct) =>
            {
                var req  = (CreateOrderRequest)e.In.Body!;
                var resp = await orderService.CreateAsync(req, ct);
                e.In.Body = resp;                      // HTTP transport returns In.Body to the caller
            })
            .WireTap("kafka://order-created")          // audit — fire-and-forget, non-blocking
            .Marshal(typeof(JsonMessageSerializer));
    }
}

Replace Http.Listen with Grpc.Listen or Ws.Listen — same pipeline. RPC, validation, business logic, audit, and serialization in one declarative route. No separate controller layer, no separate consumer layer.

Testing without a broker — `mock://`

The mock: transport records every message it receives so unit tests can assert against an in-memory endpoint. No Kafka container, no RabbitMQ container, no Testcontainers — a plain Host and a few lines of xUnit:

[Fact]
public async Task Filter_only_forwards_new_orders()
{
    var host = Host.CreateDefaultBuilder()
        .ConfigureServices(s => s.AddRedbRoute(route => route.AddRoutes(r =>
        {
            r.From("direct://input")
                .Filter(Header("type").isEqualTo("new"))
                .To("mock://received");
        })))
        .Build();
    await host.StartAsync();

    var producer = host.Services.GetRequiredService<IRouteProducer>();
    var mock     = host.Services.GetRequiredService<MockComponent>().GetEndpoint("received")!;

    await producer.SendAsync("direct://input", "payload-1",
        new Dictionary<string, object> { ["type"] = "new" });
    await producer.SendAsync("direct://input", "payload-2",
        new Dictionary<string, object> { ["type"] = "old" });

    Assert.Equal(1, mock.ReceivedCount);
    Assert.Equal("payload-1", mock.ReceivedExchanges[0].In.Body as string);
}

For async routes use MockDsl.Endpoint("name").ExpectedMessageCount(n) — it awaits the expected count with a timeout. This is the Camel testing idiom and one of the reasons unit tests on Camel routes are pleasant. .NET integration libraries usually leave you to Testcontainers.

Telemetry — OpenTelemetry built in, on by default

Every step in every route emits an Activity and a Meter sample. No AddInstrumentation, no per-step manual spans, no decorator wrapping. EnableTelemetry and EnableMetrics are true out of the box — point your collector at the process and you immediately see per-route traces and per-step latency.

For named sections that should show up as their own span or counter in Grafana / Jaeger / Tempo, the DSL has Traced and Metered:

From("kafka://orders")
    .Traced("order-processing")                   // one Activity for the whole block
        .SetBody(JPath("$.order"))
        .Process(async (e, ct) => await Enrich(e, ct))
    .EndTraced()
    .Metered("order-throughput")                  // counter + histogram for the block
        .To("rabbitmq://processed")
    .EndMetered();

// Inline form when the named span wraps a single step
From("kafka://orders")
    .Traced("validate", async (e, ct) => await ValidateOrder(e, ct))
    .Metered("transform", e => { e.In.Body = Transform(e); })
    .To("rabbitmq://processed");

Standard metric names: redb.route.messages.processed, redb.route.messages.failed, redb.route.processing.duration — emitted per route and per named step. They drop straight into any OTel-compatible backend.

What a real production route looks like

The examples above are deliberately short. Real routes nest: HTTP listener → auth → permission check → method dispatch → business handler → audit tap. Indentation is the route hierarchy. The whole shape of the request is visible top-down:

From  http://0.0.0.0:5090/api/.../settings   (inOut, cors)
  Process            Auth.ProcessAsync
  ConvertBody<string>
  Choice on Header redbHttp.Method
  ├─ POST
  │    RequirePermission   EditSettingsTables
  │    ProcessWithRedb     HandlePost
  │    WireTap → direct://audit   (onPrepare + newBodyFactory)
  ├─ DELETE
  │    RequirePermission   EditSettingsTables
  │    ProcessWithRedb     HandleDelete
  │    WireTap → direct://audit
  └─ else
       ProcessWithRedb     HandleGet
  EndChoice
Respond → HTTP caller

This is one route from the production system mentioned at the bottom of the post — settings API for a logistics admin panel:

protected override void Configure()
{
    From("http:0.0.0.0:5090/api/tsum/special-rc-settings?inOut=true&cors=true&corsOrigins=*")
        .RouteId("tsum-api-special-rc-settings")
        .Process(Auth.ProcessAsync)                       // attach IPrincipal to exchange
        .ConvertBody<string>()
        .Choice()
            .When(Header("redbHttp.Method").isEqualTo("POST"))
                .Process(TsumAuthProcessor.RequirePermission(TsumPermission.EditSettingsTables))
                .ProcessWithRedb((redb, ex, ct) => HandlePost(redb, ex))
                .WireTap("direct://tsum-audit",
                    onPrepare:      e => TsumAuditHelper.SetHeaders(e, "DATA_CHANGE"),
                    newBodyFactory: TsumAuditHelper.BuildDetailsJson)
            .When(Header("redbHttp.Method").isEqualTo("DELETE"))
                .Process(TsumAuthProcessor.RequirePermission(TsumPermission.EditSettingsTables))
                .ProcessWithRedb((redb, ex, ct) => HandleDelete(redb, ex))
                .WireTap("direct://tsum-audit",
                    onPrepare:      e => TsumAuditHelper.SetHeaders(e, "DATA_CHANGE"),
                    newBodyFactory: TsumAuditHelper.BuildDetailsJson)
            .Otherwise()
                .ProcessWithRedb((redb, ex, ct) => HandleGet(redb, ex))
        .EndChoice();
}

private async Task HandleGet(IRedbService redb, IExchange exchange)
{
    var shippingPointId = ParseQueryLong(exchange, "shippingPointId")
                       ?? ParseQueryLong(exchange, "rcId");
    if (!shippingPointId.HasValue)
    {
        BadRequest(exchange, "shippingPointId required");
        return;
    }

    var items = await redb.Query<SpecialRcSettings>()
        .Where(s => s.ShippingPoint == shippingPointId)
        .ToListAsync();

    JsonRouteHelper.SetJsonBody(exchange, items
        .Select(MapToDto)
        .OrderBy(x => x.CustomName)
        .ToList());
}

Two features worth pointing out:

ProcessWithRedb((redb, ex, ct) => ...) — typed access to a named redb.Core service inside the pipeline. redb.Query<SpecialRcSettings>().Where(...).ToListAsync() is just LINQ over a typed EAV scheme. No DbContext, no migrations, no separate repository class.
WireTap("direct://tsum-audit", onPrepare: ..., newBodyFactory: ...) — the audit hop runs in parallel with the main response, gets its own headers and its own body built fresh from the exchange. The HTTP client doesn't wait for audit, but audit sees the exact state of the exchange at that step.

This is the actual shape of a real route. Method dispatch, auth, permission, business handler, audit — in one declarative block that reads top-down.

How it compares

	Apache Camel	MassTransit	NServiceBus	Wolverine	redb.Route
Language	Java/JVM	C#	C#	C#	C#
Transports	300+	5	7	4	22
EIP patterns (DSL)	80+	~5	~5	~5	30+
Expression engine	Interpreted	—	—	—	Compiled
Transactional pipelines across transports	Yes	Bus only	Bus only	Bus only	Yes
Runtime container	Karaf / Camel K	Worker Service	Worker Service	Worker Service	redb.Tsak
License	Apache 2.0	Apache 2.0	Commercial (>2 endpoints)	MIT	Apache 2.0

MassTransit, NServiceBus and Wolverine solve a different problem — they are message-bus frameworks with handler discovery, durable sagas and managed outbox. redb.Route is a pipeline and transport integration engine. They are not mutually exclusive: use redb.Route for cross-protocol routing and transformation, MassTransit for handler-style messaging and long-running saga state.

Deploying to production — redb.Tsak

Writing RouteBuilder classes is one thing. Running them in production across multiple nodes is another.

redb.Tsak is the runtime container built for redb.Route:

Drop a .dll or .tpkg (ZIP + manifest) into Libs/ — Tsak loads it without restart
Hot-reload — update the file while running, zero downtime for other routes
Cluster mode — leader election, automatic context redistribution across nodes. No ZooKeeper, no etcd. Coordination uses row locks in redb.Core.
REST API (32 endpoints), CLI (30 commands), Blazor dashboard with per-route metrics, logs, watchdog, cluster view

You do not change a single line of RouteBuilder code to go from dotnet run to a 3-node production cluster.

Full writeup on Tsak is coming in the next article. For now: github.com/redbase-app/redb-tsak.

Current state

Running at EWS — a 30-year-old national HoReCa food distributor:

3-node cluster (4 cores / 8 GB / 50 GB SSD per node)
~150k orders/month, ~3 months stable, 10–15% CPU under full load
Active transports: SAP, Kafka, RabbitMQ, GPS feeds, Mercury / EGAIS / Chestny Znak / FGIS Grain

27 NuGet packages (core engine + 22 transports + 5 support libraries). Apache 2.0.

It's not Apache Camel — not in transport count, not in maturity, not in ecosystem. But for the kind of integration work most .NET teams actually ship, it covers the ground a single library reasonably can. Honest feedback, missing transports, and breaking-case bug reports are all welcome.

Links


redb.Route	github.com/redbase-app/redb-route
redb.Tsak	github.com/redbase-app/redb-tsak
Architecture	redbase.app/architecture
Discussions	github.com/redbase-app/redb-route/discussions

Over to you

This is exactly the moment when honest outside input is worth more than another internal sprint. A few specific things I'd love to hear:

Transports. 22 cover most stacks I've seen, but obvious gaps remain. NATS / NATS JetStream? Pulsar? Service Bus topics with sessions? Google Pub/Sub? SQS+SNS as a pair? Salesforce Streaming API? OPC UA? Tell me what would make redb.Route a real fit for your stack.
EIP patterns. 30+ ship today. Anything from the Hohpe/Woolf catalogue you'd actually use and can't easily get elsewhere in .NET? Message Store with replay? Routing Slip? Process Manager? Normalizer? Format Indicator?
DSL ergonomics. Anything in the examples above that reads awkwardly in C#? Where would [Source]/[Sink] attributes, source generators, minimal-API-style MapRoute("/orders"), or top-level statement DSL feel better than RouteBuilder classes?
Observability. OpenTelemetry is built in and on by default; Grafana dashboards and the live metrics / logs UI live in redb.Tsak (full writeup in the next article). What's still missing from your point of view — opinionated dashboard JSON, a turnkey OTel collector recipe, exemplars wired to trace IDs?
Migration. If you have an existing Apache Camel route in production, would a side-by-side translation walkthrough (Camel Java → redb.Route C#) be useful? Which patterns hurt most?
What stops you from trying it. "License is good, but…" / "I like the DSL, but…" — the but is the most valuable feedback I can get right now.

Drop a comment, open an issue, or start a thread in Discussions. Critical responses get the same priority as kind ones — both move the project forward.

DEV Community: rinat kozin

redb.Route.Llm 3.1.1 — per-message audit fields for LLM compliance / replay

Enterprise-grade AI integration: embedding LLMs into the business processes of large companies — redb.Route.Llm 3.1.1

Why this isn't "yet another agent framework"

What got shipped from the skip-list

Ten enterprise patterns — each one a single DSL line

1. Hard budget per conversation as a circuit breaker

2. Approval gates with a human in the loop

3. Idempotent tool retries via ToolIdempotencyProps

4. Multi-tenant via ?redb=<name>

5. Audit trail without a separate integration

6. Async batch + callback consumer

7. Versioned prompt registry: #-refs

8. Tree-branching conversations for A/B and counterfactuals

9. A jury of cheap models with a senior model as arbiter — Scatter-Gather + Aggregator

10. Sub-agents — an agent as another agent's tool

Streaming: what actually changes when a token leaves the provider

RAG: chunks as first-class REDB objects

Three real-world enterprise scenarios — reports, forecasts, alerts

Case 1 — A daily financial snapshot for the CFO, in their inbox by 7:00 AM

Case 2 — A weekly cash-flow forecast with a jury and an Opus arbiter

Case 3 — A predictive plant alert: XGBoost does the math, the LLM does the sentence

Storytelling: how a chat demo becomes a platform

Demo routes — see it live

What's not done, and why I keep saying so

Zooming out: why the ESB shape matters

Roadmap

Links

redb.Route 3.1.0 — LLM(AI) as just another connector: `.To("llm://claude")` and tools-as-routes

The shortest possible explanation

One adapter, 14 OpenAI-compatible APIs

Tools are routes

A real example — a standalone "curl → Claude → shell → reply" demo

csproj — five NuGet packages

1. The key

2. DI and RouteContext

3. Three components

4. Connection factory for Claude Haiku

5. Agent engine + tool registry

6a. The HTTP route

6b. The tool itself is a route

7. Start and block

Run it

redb.Route.Exec — process spawning as the 25th transport

From("llm://...") — the scheduled agent consumer

#-prompts — the dynamic registry

Agent memory — AddRedbLlmStorage()

What the engine carries for free

What's live-tested, and what isn't

Links

REDB inside, part 1.1 — why the same 13 tables stay fast no matter how many classes you throw at them

The "9.7 million rows" objection, defused in two sentences

The architectural payoff: indexes don't grow with your domain

A tour of redbPostgre.sql, grouped by what it's for

1. Storage uniqueness in _values

2. Facet search — the hottest path

3. Collections (arrays and dictionaries)

4. Object trees (_objects._id_parent)

5. Sorting and metadata

6. Schemes and structures (_structures, _schemes)

What got removed

Why this is fast on prod

Query: 139 ms for 999 objects in one call

Save: 154 ms for 32 changed objects

Sync: 482 ms — what it really is

Coming back to the 9.7M-row objection

"You don't always need everything you've got": pruning indexes after stabilization

What's deliberately not in the DDL, and why

MSSQL: nearly the same picture

TSUM as the proof case

Wrap-up and what's coming next

Links

Earlier in the series (on dev.to)

REDB inside, part 1 — the 13 tables the whole engine runs on (with the actual SQL, and why it's not EAV)

"Wait, isn't this just EAV?"

The 13 tables, at a glance

Layer 1 — the type catalog: _types

Layer 2 — schemes and fields: _schemes + _structures

Layer 3 — objects: _objects

Layer 4 — the values: _values

3. Idempotent tool retries via `ToolIdempotencyProps`

4. Multi-tenant via `?redb=<name>`

7. Versioned prompt registry: `#`-refs

2. DI and `RouteContext`

`redb.Route.Exec` — process spawning as the 25th transport

`From("llm://...")` — the scheduled agent consumer

`#`-prompts — the dynamic registry

Agent memory — `AddRedbLlmStorage()`

A tour of `redbPostgre.sql`, grouped by what it's for

1. Storage uniqueness in `_values`

4. Object trees (`_objects._id_parent`)

6. Schemes and structures (`_structures`, `_schemes`)

Layer 1 — the type catalog: `_types`

Layer 2 — schemes and fields: `_schemes` + `_structures`

Layer 3 — objects: `_objects`

Layer 4 — the values: `_values`

Layer 5 — permissions: `_permissions`

`_scheme_metadata_cache` — why a cache table

Query 2 — same result via `_scheme_metadata_cache`

Flat `End*()` navigation

Fix: `GetContext()` silently returned null inside nested scopes

1. `In` vs `Out`, and the `Pattern`

2. `Properties` vs `Headers` — the distinction that leaks bugs

4. `ExchangeId` is stable across clones

The gotcha: `Clone()` does NOT deep-copy the Body

`direct://` — a method call wearing a URI

`seda://` — the async queue, in detail

`concurrentConsumers` — throughput, at the cost of order

`size` — bounded vs unbounded, and why you almost always want bounded

`timeout` — an honesty note

Graceful shutdown — `seda` drains, it doesn't drop

`direct-vm://` and `vm://` — the same two, across module boundaries