DEV Community: rinat kozin

Basic OP: 35 modules, 0 failures. Config OP: 0 failures. Found and fixed: PII leak in id_token, missing Cache-Control: no-store. report: https://github.com/redbase-app/redb-identity/blob/main/OPENID_CERTIFICATION.md #dotnet #oidc #oauth

rinat kozin — Wed, 22 Jul 2026 20:05:11 +0000

redb-identity/OPENID_CERTIFICATION.md at main · redbase-app/redb-identity · GitHub

Transport-agnostic OAuth 2.1 / OpenID Connect server for the redb ecosystem — OpenIddict on redb.Route, redb-backed stores, MFA, WebAuthn, federation, SCIM, DPoP. Green on PostgreSQL, MSSQL and SQL...

github.com

The night job that carves its own partitions: cron, a plpgsql function, and a Splitter

rinat kozin — Tue, 21 Jul 2026 17:17:47 +0000

Series: redb ecosystem

There's a table of GPS points. Vehicles push coordinates every few seconds, a day adds up to millions of rows, and the table is partitioned by time. Which means somebody has to create tomorrow's partition ahead of time and detach the ones older than ninety days. Otherwise, one fine night, an insert dies with no partition of relation "gps_points" found for row — at precisely the hour nobody is watching.

The problem is as old as the hills, and it's usually solved one of two ways: pg_cron inside the database, or an IHostedService with a PeriodicTimer in the app. The first has an observability problem — the job lives in the database, and your application logs know nothing about it. The second has the problem that a small private infrastructure grows around PeriodicTimer very quickly: retries, logging, "what if the previous run is still going", "what if there are three nodes".

I'll show a third way — a route. What follows is a walkthrough of the redb.Route.Quartz connector, calling a PostgreSQL function through the SQL connector, a splitter with error isolation, and an honest conversation about clustering, because that's where it actually gets interesting.

All the code in this post uses string URIs. redb.Route has fluent builders too, but a URI reads without knowing the API, and you can paste it into a config file.

redb ecosystem series. This is a continuation — newest posts first:

The Amazon SQS + SNS connector in redb.Route, at-least-once and pub/sub via SNS→SQS. Still leaving MassTransit

The RabbitMQ connector in redb.Route: the whole broker in one URI — RPC, competing consumers, dead-lettering

redb 3.3.0: an enterprise .NET stack you actually own — typed store, a homegrown Apache Camel, and a runtime with a dashboard (all free)

Two routes in an evening: from a debug worker to an enterprise runtime with redb.Route + Tsak

Leaving MassTransit for a Camel state of mind: the Kafka connector, Scatter-Gather, and what really happens under a transaction

Full list on the profile. Sources: github.com/redbase-app/redb-route. About the database itself: redbase.app.

The SQL first

There's no magic at this level, so let's start from the most honest place — the function in the database. It takes a table name and a retention window, creates tomorrow's partition if it isn't there yet, and detaches everything past the window:

CREATE OR REPLACE FUNCTION maintain_partitions(tbl text, keep_days int)
RETURNS text AS $$
DECLARE
    next_day   date := (now() + interval '1 day')::date;
    part_name  text := format('%s_%s', tbl, to_char(next_day, 'YYYYMMDD'));
    cutoff     date := (now() - make_interval(days => keep_days))::date;
    dropped    int  := 0;
    old_part   text;
BEGIN
    IF to_regclass(part_name) IS NULL THEN
        EXECUTE format(
            'CREATE TABLE %I PARTITION OF %I FOR VALUES FROM (%L) TO (%L)',
            part_name, tbl, next_day, next_day + 1);
    END IF;

    FOR old_part IN
        SELECT c.relname
        FROM pg_class c
        JOIN pg_inherits i ON i.inhrelid = c.oid
        JOIN pg_class p ON p.oid = i.inhparent
        WHERE p.relname = tbl
          AND c.relname < format('%s_%s', tbl, to_char(cutoff, 'YYYYMMDD'))
    LOOP
        EXECUTE format('DROP TABLE %I', old_part);
        dropped := dropped + 1;
    END LOOP;

    RETURN format('%s: +1 partition, -%s dropped', tbl, dropped);
END;
$$ LANGUAGE plpgsql;

The function returns a string so it shows up in the log. That "so it shows up in the log" is the single concession to convenience here; everything else is ordinary plpgsql you'd have written anyway.

The tick: `cron://`

The redb.Route.Quartz connector gives you two schemes. The first is cron:, a regular Quartz trigger with a cron expression:

cron://[group/]jobName?schedule=<cron-expression>&<options>

The second is qtimer:, a plain periodic trigger for when you don't need cron:

qtimer://[group/]jobName?period=5000&delay=1000&fixedRate=true

There is deliberately no quartz: scheme. That's not an oversight: scheduler setup — where jobs are stored, whether it's a cluster or a single node, what the thread pool looks like — is a property of the host, not of the route. Only the schedule belongs in a route URI. If a quartz: scheme existed, job store settings would immediately start growing into it, and the route would stop being portable.

The cron expression is the Quartz one, six fields, with seconds. Let's put partition maintenance at 02:30:

From("cron://maintenance/gps-partitions?schedule=0 30 2 * * ?")
    .RouteId("gps-partitions")

The expression is validated when the endpoint is created, not when it first fires. A typo in the schedule is an ArgumentException at application startup, not silence until three in the morning.

Registering the component is one line at the module entry point:

context.AddComponent(new CronComponent());

Calling the function: `sql:`

The SQL connector has a single scheme, sql:, and the mode is picked with the mode parameter. To call a PostgreSQL function there's mode=Procedure with the asFunction=true flag — the connector then assembles SELECT maintain_partitions(@p1, @p2), executes it as a scalar, and puts the result in the message body:

sql:maintain_partitions
  ?mode=Procedure
  &dataSource=#pg
  &procedureName=maintain_partitions
  &asFunction=true
  &procedureParams=IN:tbl:String,IN:keep_days:Int32
  &param.keep_days=90

procedureParams declares the parameters in direction:name:type form, and the declaration order is the argument order in the call. There are three directions — IN, OUT, INOUT — and OUT values come back into the message headers under their own names after execution.

A parameter's value is resolved along a chain: first an explicit param.name from the URI, then a message header with the same name, then the body if it's a dictionary. Here keep_days is a constant right in the URI, and tbl will arrive from a header set by the splitter.

For simple cases there's a shorter path — mode=Execute (the default) with @name placeholders directly in the query text:

sql:SELECT maintain_partitions(@tbl, @keep_days)
  ?dataSource=#pg
  &outputType=Scalar
  &param.tbl=${header.tbl}
  &param.keep_days=90

Note ${header.tbl} — that's an expression, resolved at runtime from the message header. Placeholders in SQL are @name only, the colon form isn't supported, and an unsubstituted parameter silently becomes NULL, so it pays not to mistype the names.

Both variants work. For the rest of the post I use mode=Procedure because it's the more illustrative one.

The splitter: three tables, and one failure doesn't take down the rest

There's rarely just one time-partitioned table. We have three: points, tracks, and events. The naive move would be a loop inside a processor, but then you're deciding by hand what happens when the second table fails — abort everything or carry on, and how do you find out afterwards what didn't run.

This is exactly the EIP Splitter pattern. A message carrying a list is split into one message per element, each goes down its own branch, and the branches can be processed in parallel:

From("cron://maintenance/gps-partitions?schedule=0 30 2 * * ?")
    .RouteId("gps-partitions")
    .Process(e => e.In.Body = new[] { "gps_points", "gps_tracks", "gps_events" })
    .Split(Body())
        .ParallelProcessing()
        .MaxParallelism(2)
        .SetHeader("tbl", Body())
        .DoTry()
            .To("sql:maintain_partitions"
                + "?mode=Procedure"
                + "&dataSource=#pg"
                + "&procedureName=maintain_partitions"
                + "&asFunction=true"
                + "&procedureParams=IN:tbl:String,IN:keep_days:Int32"
                + "&param.keep_days=90")
            .Log("[PART] ${body}")
        .DoCatch<Exception>()
            .Log("[PART] ${header.tbl}: ${exception.Message}", LogLevel.Error)
        .End()
    .EndSplit()
    .Process(Summary);

Twelve lines, and they already contain everything you'd normally bolt on a week later. MaxParallelism(2) means two tables get serviced at once and the third waits for a free slot; DROP TABLE takes an ACCESS EXCLUSIVE lock, and there's no point burying the database under parallel locks. DoTry/DoCatch sit inside the split, so the exception is isolated to its own branch: a failing gps_tracks won't undo the gps_points that already succeeded, and won't stop gps_events. After EndSplit, control lands in Summary, where you can count how many branches made it and decide whether to page somebody.

This isn't a trick invented for the article. Exactly this shape runs in production — a job that syncs shipping points from SAP splits the list of points and processes three at a time, because one unreachable point shouldn't take down the whole tick:

From("timer://tsum-points?period=180000&delay=60000")
    .RouteId("tsum-points-timer")
    .ProcessWithRedb(PreloadContextAsync)
    .Split(Body())
        .ParallelProcessing()
        .MaxParallelism(3)
        .SetHeader("ShippingPoint", Body())
        .DoTry()
            .To(sqlTo)
            .Process(DeserializeXml)
            .ProcessWithRedb(ProcessPointsAsync)
        .DoCatch<Exception>()
            .Process(AddPointSyncError)
            .Log(LogLevel.Error)
                .Message("[TSUM-PT] SP=${header.ShippingPoint} failed, skipping: ${exception.Message}")
            .EndLog()
        .End()
    .EndSplit()
    .Process(BuildPointSyncSummary);

The splitter isn't the only EIP that docks onto a scheduler. A schedule pairs naturally with a Content-Based Router (one thing on weekdays, another on weekends), a Throttler (don't hammer an external API more than N times a second), an Aggregator (collect branch results into a single report), an Idempotent Consumer (more on that below) and a Dead Letter Channel. redb.Route implements nearly the whole Hohpe & Woolf catalogue — Splitter, Aggregator, Resequencer, Multicast, Recipient List, Dynamic Router, Wire Tap, Content Enricher, Claim Check, Saga, Scatter-Gather, Circuit Breaker, Load Balancer, Transactional Client and the rest. The scheduler here is just a source, not a separate world with its own rules.

What happens when the server was down at 02:30

This is where the reason you'd want Quartz rather than a PeriodicTimer starts.

The job didn't fire, because the node was down or a deploy ran long. What should happen when the scheduler comes back at 02:47? The answer depends on the job, and it isn't a philosophical question: for partition maintenance, a skip is a catastrophe — tomorrow's partition has to be created, at 02:47 or at 06:00. For a "send the morning report" job, running at noon is worse than not running at all.

Quartz calls this misfire, and the connector passes the policy straight through the URI:

cron://maintenance/gps-partitions?schedule=0 30 2 * * ?&misfireInstruction=CronFireOnceNow

CronFireOnceNow — catch up, run once, return to the schedule. CronDoNothing — skip it, wait for the next scheduled time. Simple triggers (qtimer:) have more policies — five of them — differing in what to do with the accumulated repeat count. But the choice always collapses to one question: does a missed run need catching up, or has it already gone stale?

The connector's default for qtimer: is picked by common sense: with fixedRate=true, catch up (you asked for a fixed rate); otherwise, reschedule from the next tick.

What happens when the previous run is still going

Partitions have piled up, DROP TABLE is waiting on a lock, the job is hanging. The next fire time arrives. Now what?

The standard Quartz answer is the [DisallowConcurrentExecution] attribute on the job class. The connector went a different way: concurrency is governed by the consumer's own semaphore, and if every thread is busy the fire is silently skipped:

// QuartzConsumerBase.cs
if (!await _semaphore.WaitAsync(0).ConfigureAwait(false))
    return; // all threads busy, skip this fire

The semaphore size is set in the URI via threads (default 1). Why not the attribute: [DisallowConcurrentExecution] is either one run or no control at all, with nothing in between. A semaphore lets you say "up to three concurrent runs of this job" while staying friendly to a cluster, where the limit lives at the job store level rather than in a class attribute.

If you do want the Quartz semantics, there's a stateful=true flag that switches the job to a class carrying [DisallowConcurrentExecution] and [PersistJobDataAfterExecution].

One more detail about shutdown: when a route is stopped, the connector unschedules the trigger and waits for runs already in flight — up to thirty seconds. A job that happens to be dropping a partition right then won't be cut off halfway. And if a job does fire when its route is already gone (the module was unloaded, say), the job notices on startup that its consumer is dead and deletes itself from the scheduler, leaving no litter behind.

Three nodes

The most common question about cron in a distributed application: if there are three nodes, does the job fire three times?

It does — if every node keeps its own scheduler in memory. That's exactly how the connector's fallback works: no IScheduler found in the context, so it creates its own, RAM-backed, unique to that context. Fine for local development, not fine for three nodes.

The right answer is one scheduler per cluster — more precisely, one shared job store. Quartz does this out of the box via AdoJobStore, and the connector deliberately stays out of the way: it does not create its own scheduler if a ready one is already sitting in the route context. The host drops a clustered one in there, and the route needs no changes at all — the URI stays the same.

And here's where all the plumbing that usually gets hand-waved away comes into view. AdoJobStore is a set of QRTZ_* tables in your database. QRTZ_TRIGGERS holds the next fire time, QRTZ_FIRED_TRIGGERS holds who is executing what right now, QRTZ_LOCKS holds row-mutexes. The "only one node runs the job" mechanism isn't clever consensus — it's SELECT ... FOR UPDATE on a row in QRTZ_LOCKS: whoever takes the lock first takes the trigger. Each node checks in periodically in QRTZ_SCHEDULER_STATE, and if a node stops checking in, another one picks up its unfinished jobs — but only those marked recoverableJob=true. The flag is off by default: restarting a job when you don't know whether it's idempotent is a bad idea.

The table schema is created by the host at startup; the connection string and dialect come from the application's database configuration. So you don't write the DDL, but the tables are perfectly ordinary, they sit next to yours, they're visible in any client, and when something goes wrong you go in with a plain SELECT and see which trigger is stuck and on which node.

And since we brought up recoverableJob: recovery after a node dies means the job may run twice. For our partition maintenance function that's safe — it's written idempotently (IF to_regclass(...) IS NULL), and that's a requirement, not an accident. If the job weren't idempotent — say, it credited bonuses — you'd put an Idempotent Consumer in front of it, and redb.Route has one backed by SQL with a unique index. A unique index, not a "smart cache": in a cluster, that's the only thing that saves you from double execution.

What lands in the message

A scheduler is a source with no message body. The body is null, the pattern is InOnly, and everything about the firing lives in the properties:

Property	What's in it
`CamelQuartzFireTime`	when the job actually fired
`CamelQuartzScheduledFireTime`	when it should have fired per the schedule
`CamelQuartzNextFireTime`	when it fires next
`CamelQuartzPreviousFireTime`	when it fired last
`CamelCronSchedule` / `CamelCronName` / `CamelCronGroup`	the expression, name and group of the job

The gap between FireTime and ScheduledFireTime is that misfire, in numbers. If they're seventeen minutes apart, the job was catching up.

The Camel-style names aren't nostalgia. redb.Route deliberately keeps Apache Camel's nomenclature wherever the semantics match, so someone arriving from Java integrations can read the headers without a dictionary.

Two jobs from production

So this doesn't read like a piece about a spherical cron in a vacuum — here are two routes that run every night in a transport management system.

Database backup at three:

From("cron://tsum-backup?schedule=0 0 3 * * ?")
    .RouteId("tsum-backup-cron")
    .ProcessWithRedb(RunBackupAsync);

Dead-route cleanup at four:

From("cron://tsum-cleanup?schedule=0 0 4 * * ?")
    .RouteId("tsum-cleanup-cron")
    .ProcessWithRedb(CleanupDeadRoutesAsync);

Nothing spectacular, and that's the point: the schedule is in the URI, the logic is in the processor, retries and logging come from the framework. Note that the schedule is hardcoded in the URI rather than pulled from config — that's allowed, and it's how things live for a while. When you need to change a schedule without a rebuild, you assemble the URI from config with ordinary concatenation, because it's just a string.

The full option list

So this doesn't turn into half a post of reference material — everything cron: understands:

schedule (required), timeZone (IANA name), threads, misfireInstruction, stateful, recoverableJob, durableJob, deleteJob, pauseJob, startAt, endAt, customCalendar, triggerStartDelay, prefixJobNameWithEndpointId.

qtimer: swaps schedule for period, delay, fixedRate, repeatCount; everything else is the same, minus the time zone (a simple trigger doesn't need one).

customCalendar is worth calling out: it's the Quartz exclusion calendar, registered in the context by name. That's how you get "except holidays" and "business days only" — things a cron expression cannot express at all.

Wrapping up

The scheduler in redb.Route is a message source, not a subsystem with its own rules. A job is a route, which means everything available to any route is available to it: the splitter, exception handling, transactions, retries, metrics. The trigger is described by a single URI string, and there's nothing about infrastructure in that string — only the schedule.

The infrastructure isn't hidden, either. The cluster runs on QRTZ_* tables and a row lock in the database, idempotency in a cluster comes from a unique index, and partition maintenance is an ordinary plpgsql function that you wrote and can read. The framework here spares you the glue code, not the understanding of what's happening in your database.

Code: github.com/redbase-app/redb-route · Docs: redbase.app

If this was useful — a ⭐ on GitHub helps others find it.

The Amazon SQS + SNS connector in redb.Route, at-least-once and pub/sub via SNS SQS. Still leaving MassTransit

rinat kozin — Thu, 16 Jul 2026 13:24:18 +0000

series: "redb.Route"

We've done Kafka and RabbitMQ earlier in this series. Next up: Amazon SQS — and riding along in the same package, SNS. Two transports, because in AWS-land they travel together. The redb.Route.Sqs connector sits on the native AWS SDK for .NET v4 (AWSSDK.SQS, AWSSDK.SimpleNotificationService), but you don't write a "client" — you write routes, and the whole queue collapses into a single URI:

From("sqs://orders?waitTimeSeconds=20&concurrentConsumers=4")
    .Log("Order in: ${body}")
    .To("direct://process");

Read it once and the whole story's there: long-poll the orders queue in 20-second pulls, four consumers wide, log it, pass it on. No "new up a client," no ReceiveMessage, no "remember to DeleteMessage after you've handled it" — the connector does all of that.

About "leaving MassTransit," same disclaimer as last time: it's not that MassTransit can't do this. It's had SQS/SNS for years — MassTransit.AmazonSQS — and it wires up the same SNS→SQS pairing. The difference is the model. MassTransit is a bus — message contracts, consumers, bus config, the whole worldview bought at once. redb.Route is explicit routes in the Apache Camel spirit: an endpoint is a URI, the integration patterns are steps in a route, the transport is an abstraction underneath. This post is about what SQS and SNS look like in that second model.

And the thing everyone's actually asking about right now — worth stating precisely rather than gleefully. MassTransit went commercial with v9: the line now ships under Massient, Inc., and MassTransit.AmazonSQS 9.0.0 on NuGet is already from there. But no spin: v8 stays Apache 2.0, with security patches and critical fixes through at least the end of 2026; v9 isn't a closed binary — it's source-available; and organizations under $1M USD annual revenue (plus non-profits under $1M in expenses) get it at a 100% discount — free, minus commercial support. So "MassTransit is paid now" is false for a small team and true for a large one, where it turns into a real decision: pay, ride v8 to the end of its support, or look around. redb.Route is Apache 2.0, this SQS connector included — but pick it because the routing model fits your head, not because of a license line. A license is a reason to look around; it isn't an argument about architecture.

All the code is in English; the examples aren't invented — they run against LocalStack (a Docker container on http://localhost:4566), the same way the connector's integration tests do.

Part of the redb / redb.Route series — recent posts first:

The RabbitMQ connector — the whole broker in one URI: RPC, competing consumers, dead-lettering (previous in the series)

Leaving MassTransit for a Camel state of mind: the Kafka connector, Scatter-Gather, and transactions

Apache Camel for .NET, dissected: the HTTP connector with no ASP.NET MVC + the Content-Based Router

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL

Sources: github.com/redbase-app. About the database itself: redbase.app.

One package, two transports

redb.Route.Sqs registers two schemes, and they don't play the same role:

sqs:// — a queue. Works as both a consumer (in From(...)) and a producer (in .To(...)).
sns:// — a topic. Publisher-only. SNS has no pull consumer — delivery is push (to SQS / HTTP / email / SMS). Try to From("sns://...") and it throws NotSupportedException on purpose: to "read" a topic, you subscribe an SQS queue to it and read that with sqs://.

Why they share a package: at AWS they're a pair. SQS is a durable point-to-point queue (one message, one consumer). SNS is a publish-subscribe topic (one message, every subscriber). The canonical AWS fan-out is an SNS topic with several SQS queues hanging off it — so one connector covers both: sqs:// and sns://.

Wiring it into DI:

services.AddRedbRoute(route =>
{
    route.Services.AddRedbRouteSqs();   // registers both sqs:// and sns://
    route.AddRouteBuilder<MyRoutes>();
});

Anatomy of the URI

An endpoint is a string:

sqs://queue-name?region=us-east-1&waitTimeSeconds=20&concurrentConsumers=4
sns://topic-name?region=us-east-1&autoCreateTopic=true

Scheme, the queue or topic name in the path, then query parameters (names are in the tables below). The fluent builder gives you the same thing: Sqs.Queue("orders").WaitTimeSeconds(20)... and Sns.Topic("events").Region("us-east-1")... build that exact string, URL-encoding included.

Three things are AWS-specific and worth a pause:

Don't repeat your credentials. Twenty queues on one account? Register an AwsConnectionFactory in the registry and point at it with connectionFactory=name. One factory builds both the SQS and the SNS client — they share credentials, region and service URL:

context.AddToRegistry("prod-aws", new AwsConnectionFactory
{
    Region = "eu-west-1",
    UseDefaultCredentialsProvider = true,   // IAM role on the instance / SSO / env vars
});

From("sqs://orders?connectionFactory=prod-aws&concurrentConsumers=4");
To("sns://events?connectionFactory=prod-aws");

LocalStack / ElasticMQ via serviceUrl=. An explicit URL overrides the regional endpoint (the region sticks around only to sign the request). That's your local-dev and test mode:

sqs://orders?serviceUrl=http://localhost:4566&region=us-east-1&accessKey=test&secretKey=test

Queues and topics get created on the spot. autoCreateQueue=true (or autoCreateTopic=true) creates the resource on startup if it's missing. A name ending in .fifo comes up as a FIFO resource automatically, with ContentBasedDeduplication=true.

Every parameter

The reason to bookmark this post. Names are exactly as they appear in the URI; defaults are what you get out of the box.

Connection and credentials (shared by `sqs://` and `sns://`)

Parameter	Default	What it's for
`region`	`us-east-1`	AWS region
`serviceUrl`	(empty)	Explicit endpoint (LocalStack/ElasticMQ); overrides the region
`accessKey` / `secretKey`	(empty)	Static keys
`sessionToken`	(empty)	Temporary (STS) credentials
`profileName`	(empty)	A named profile from the shared credentials file
`useDefaultCredentialsProvider`	`false`	The default AWS chain (env / IAM role / SSO)
`connectionFactory`	(empty)	A factory in the registry; overrides URI credentials
`retryCount`	`3`	SDK retries on transient errors
`connectionTimeout`	`30000`	HTTP client timeout, ms
`proxyHost` / `proxyPort`	—	HTTP proxy

Resolution order: the default chain → a profile → a session token → static keys. Validate() insists on at least one path — accessKey+secretKey, profileName, useDefaultCredentialsProvider=true, or a registered connectionFactory — otherwise it throws at startup.

SQS — consumer (`From("sqs://...")`)

Parameter	Default	What it's for
`waitTimeSeconds`	`20`	Long-poll (0–20). Fewer empty receives, fewer API calls
`maxNumberOfMessages`	`10`	How many to pull per receive (1–10)
`visibilityTimeout`	`0` (queue default)	How long a message stays hidden while you work it
`concurrentConsumers`	`1`	Number of competing receive loops (see the concurrency recipe)
`extendMessageVisibility`	`false`	Keep extending visibility while the handler runs (a heartbeat); needs `visibilityTimeout > 0`
`deleteAfterRead`	`true`	Delete the message after a clean pass (this is the ack)
`resetVisibilityOnFailure`	`false`	On error, reset visibility to 0 → immediate redelivery
`transacted`	`false`	Defer the delete into the route transaction (`.Transacted()`)
`attributeNames`	`All`	Which system attributes to request
`messageAttributeNames`	`All`	Which message attributes to request
`delay`	`0`	Pause (ms) after an empty receive before polling again
`initialDelay`	`0`	Delay (ms) before the first poll

SQS — producer (`.To("sqs://...")`)

Parameter	Default	What it's for
`delaySeconds`	`0`	Delivery delay (0–900). FIFO queues reject a per-message delay
`messageGroupId`	—	FIFO group id. A constant or a `${...}` expression (per message)
`messageDeduplicationId`	—	FIFO dedup id. Skip it if the queue has content-based dedup
`enableBatch`	`false`	Send an `IEnumerable` body as one `SendMessageBatch`
`batchMaxMessages`	`10`	Batch size (SQS's hard cap is 10)
`autoCreateQueue`	`false`	Create the queue on startup (FIFO if the name ends in `.fifo`)
`queueUrl`	(empty)	An explicit queue URL instead of resolving by name

SNS — publisher (`.To("sns://...")`)

Parameter	Default	What it's for
`autoCreateTopic`	`false`	Create the topic on startup (FIFO on `.fifo`)
`topicArn`	(empty)	An explicit topic ARN instead of resolving by name
`subject`	(empty)	Subject (for email delivery). Supports `${...}`
`messageStructure`	(empty)	`json` — per-protocol payloads
`messageGroupId` / `messageDeduplicationId`	—	For FIFO topics
`subscribeSnsToSqs` + `subscribeQueueArn`	`false`	On startup, subscribe an SQS queue (by ARN) to this topic
`rawMessageDelivery`	`false`	On that subscription, deliver the bare payload (see the pub/sub recipe)

Headers and tracing through the broker

The sqs:// consumer stamps message metadata onto the headers (prefixed redbSqs.), and incoming message attributes land under redbSqs.attr.:

Header	What's in it
`redbSqs.queue`	The queue name
`redbSqs.messageId`	Message id
`redbSqs.receiptHandle`	Receipt handle (needed to delete or change visibility)
`redbSqs.approximateReceiveCount`	How many times this message has been delivered ("is this a retry?")
`redbSqs.messageGroupId` / `redbSqs.sequenceNumber`	FIFO group + sequence number
`redbSqs.sentTimestamp`	When it was sent (epoch millis)
`redbSqs.attr.<name>`	An incoming user message attribute

Going the other way, the header → message-attribute mapping on the producer goes like this: user headers ride out as attributes as-is; an incoming redbSqs.attr.<name> is forwarded under the bare <name> (the prefix is stripped, so sqs→sqs and sqs→sns bridges keep their attributes); the internal redbSqs.* / redbSns.* keys are dropped, so one hop's metadata doesn't leak into the next.

Tracing comes for free: before sending, the producer injects the W3C traceparent/tracestate as message attributes via the standard DistributedContextPropagator, and the consumer on the far end picks them up and continues the trace. Over plain SQS (sqs:// → sqs://) the chain is unbroken with zero setup. (There's a wrinkle when SNS is in the middle — see the pub/sub recipe.)

Straight talk on delivery: at-least-once, visibility, transacted

This is where the marketing likes to say "exactly-once." Let's read the code instead.

The SQS consumer is at-least-once. A message is deleted (DeleteMessage) only after it clears the route. The settle logic:

clean pass + deleteAfterRead=true → DeleteMessage (the ack);
failed + resetVisibilityOnFailure=true → ChangeMessageVisibility(0) → it comes right back (fast retry);
failed without that flag → it simply times out on visibility and gets redelivered.

visibilityTimeout is how long the message is hidden while you work it. Long handler? There's extendMessageVisibility=true: a background heartbeat bumps the visibility every visibilityTimeout / 2, so slow work never triggers a redelivery out from under you.

transacted=true rolls the ack into the route transaction. The delete doesn't happen immediately — a deferred SqsAckAction is registered, whose Commit deletes the message and whose Rollback resets visibility to 0 (immediate redelivery). Commit and rollback ride along with your redb work at the .Transacted() boundary:

From("sqs://orders?transacted=true&visibilityTimeout=60")
    .Transacted()
        .ProcessWithRedb((redb, ex) => redb.SaveAsync(Parse(ex)))   // write to the DB
    .EndTransaction();     // DB committed → the message is deleted; DB failed → it comes back

What's not here is cross-queue "read from A, write to B" atomicity at the broker level — SQS doesn't have it to give. Crash in the gap between "work done" and "delete," and after a restart you get a repeat — at-least-once, not exactly-once. So make your handler idempotent; redbSqs.approximateReceiveCount is there to help. The honest label is "at-least-once with a transactional ack at the route level" — no exactly-once fairy dust.

Ordering holds only on a FIFO queue (.fifo) and only with concurrentConsumers=1.

Recipe 1: producer, consumer, FIFO, batch

The bare minimum:

// Consumer — long-poll, delete after success
From(Sqs.Queue("orders").WaitTimeSeconds(20))
    .Process(HandleOrder);

// Producer — send the body to the queue
To(Sqs.Queue("orders").Region("eu-west-1"));

FIFO — the name ends in .fifo, and a group id is required (a constant or a per-message expression):

To(Sqs.Queue("orders.fifo").MessageGroupId("${header.customerId}"));

Batch — an IEnumerable body goes out as a single SendMessageBatch (in chunks of batchMaxMessages, SQS's cap being 10):

From("timer://tick?period=5000")
    .SetBody(_ => new[] { "a", "b", "c", "d", "e" })
    .To(Sqs.Queue("orders").EnableBatch());   // one call instead of five

Recipe 2: competing consumers (concurrency)

One parameter — concurrentConsumers:

From(Sqs.Queue("orders").ConcurrentConsumers(8).MaxNumberOfMessages(1))
    .Process(HandleOrder);   // up to 8 messages handled at once

This is the SQS-native concurrency model: the connector spins up N independent receive loops (one task each), and each pulls and handles its own message. Setting maxNumberOfMessages=1 alongside it makes each of the N workers hold exactly one message at a time, so the pool saturates honestly instead of one loop grabbing a batch of 10. The default is 1 (strictly serial). Go N > 1 and per-queue ordering is off the table, and your handler had better be thread-safe.

How much this actually parallelizes shows up in the SqsRpcDemo (more on it below): 12 requests, roughly 300 ms of work each, a pool of 4 — the whole batch clears in about 1.2 s instead of the ~3.6 s a serial consumer would spend, and the measured peak concurrency pins at exactly 4.

And if what you want to parallelize is the processing inside the route rather than the intake (a serial source, heavy work), there's an orthogonal EIP step, .Threads(N). concurrentConsumers scales reading off the queue; .Threads(N) scales the work; the CONCURRENCY.md guide walks through the difference. They compose freely.

Recipe 3: RPC (request/reply) — and why SQS won't hand it to you

Here's the honest contrast with the last post. On RabbitMQ, RPC flipped on with one flag — replyTo=true on the client and zero config on the server, because the broker does reply queues itself. SQS has none of that — no protocol-level reply-to, no built-in waiting for an answer. RPC over SQS is something you build, using the classic correlation pattern: a reply queue, a correlationId, and matching the answer back. That's exactly what the SqsRpcDemo shows.

The client tags each request with two message attributes — correlationId and replyTo (the queue to answer on) — and drops it on the request queue. The worker computes a result and sends it to the queue named in replyTo; the correlationId rides back on its own (the incoming redbSqs.attr.correlationId is forwarded straight back out as an attribute). The worker is a plain consumer plus a dynamic .ToD(...):

// Worker: N competing consumers on the request queue, reply to the queue named in replyTo
From(Sqs.Queue("rpc-requests").ConcurrentConsumers(4).MaxNumberOfMessages(1))
    .Process(async (ex, ct) =>
    {
        var n = int.Parse(ex.In.Body!.ToString()!);
        await Task.Delay(300, ct);              // simulated work
        ex.In.Body = (n * (long)n).ToString();  // the reply body
    })
    // dynamic destination — the reply queue comes from the request's replyTo attribute
    .ToD(ex => Sqs.Queue(ReplyTo(ex)).Build());

On the client side a separate route reads the reply queue and matches the correlationId back to the pending call:

From(Sqs.Queue("rpc-replies").MaxNumberOfMessages(10))
    .Process((ex, ct) =>
    {
        var id = Attr(ex, "correlationId");
        if (id is not null && Pending.TryRemove(id, out var tcs))
            tcs.TrySetResult(ex.In.Body?.ToString() ?? "");
        return Task.CompletedTask;
    });

The takeaway: RPC on SQS works, but it's assembled from primitives (attributes + .ToD + correlation), not toggled with a flag. When you want RPC out of the box, you reach for a broker with a native reply path (RabbitMQ). SQS is honest about what it is: durable queues and redelivery, and request/reply is a thing you wire up on top. The full, runnable version lives in redb.Route/demos/SqsRpcDemo (it runs against LocalStack).

Recipe 4 (EIP): Publish-Subscribe via SNS→SQS fan-out

Here's the headline pattern, the whole reason SQS keeps SNS company. Publish-Subscribe Channel, straight out of Hohpe & Woolf: the publisher sends one message and every subscriber gets it — independently, into its own queue, with its own retries. At AWS you don't do this in code, you do it with topology: an SNS topic with several SQS queues subscribed to it.

publish "order" ──▶ SNS topic ──┬──▶ SQS "orders-billing"  ──▶ billing route
                                 └──▶ SQS "orders-shipping" ──▶ shipping route

The publisher just publishes to the topic, and every subscribed queue gets a copy — each with its own independent consumer:

// Publisher — publish an order event; SNS fans it out to every subscribed queue:
From("timer://orders?period=5000")
    .SetBody(_ => BuildOrder())
    .To(Sns.Topic("orders-events"));

// Two independent subscribers — each its own SQS queue, each its own consumer:
From(Sqs.Queue("orders-billing")).Process(Charge);
From(Sqs.Queue("orders-shipping")).Process(Ship);

The subscriptions themselves the connector can set up from the URI — subscribeSnsToSqs + the queue ARN; the subscribe runs when the sns:// producer starts. Each sns:// endpoint subscribes one queue (a single ARN), so a two-queue fan-out is two subscriptions on the one topic:

// Subscribe each queue to the topic with raw delivery (bare payload, no envelope):
Sns.Topic("orders-events").SubscribeSnsToSqs(billingArn).RawMessageDelivery();
Sns.Topic("orders-events").SubscribeSnsToSqs(shippingArn).RawMessageDelivery();

For the exact fan-out wiring (creating the queues, reading their ARNs, subscribing, publishing), see redb.Route/demos/SqsPubSubDemo.

⚠️ The catch: the envelope vs. raw

By default SNS does not drop your payload into the queue as-is — it wraps it in a JSON notification envelope:

{ "Type": "Notification", "MessageId": "...", "TopicArn": "...",
  "Message": "<your payload, as a string>", "MessageAttributes": { ... } }

So the subscribing queue receives the envelope, not your message, and your SNS attributes sit inside it rather than as SQS attributes. That's standard AWS SNS behavior (the subscription's Raw Message Delivery = OFF), and it earns its keep in exactly one case: when a single queue listens to several topics and you need the envelope's TopicArn to tell them apart. For an ordinary fan-out it's just in the way — you end up unwrapping the envelope, and the SNS→SQS trace breaks (the traceparent goes inside the envelope, while the SQS consumer looks for it among the SQS attributes).

The fix is the subscription's Raw Message Delivery = true: the queue gets the bare payload, and the SNS attributes become SQS attributes (and the trace is whole again). In the connector that's .RawMessageDelivery() — it sets RawMessageDelivery=true on that subscribeSnsToSqs subscription. The SqsPubSubDemo prints the first delivered body: with .RawMessageDelivery() it's {"orderId":1,"amount":100}, not an envelope.

On rawMessageDelivery. This option lands in the next NuGet release of redb.Route.Sqs. You can already grab it from source right now — the redb.Route.Sqs connector on GitHub (the connector folder — SnsEndpointOptions / SnsProducer / the RawMessageDelivery fluent method). Without it, SNS→SQS still works; you just get the JSON envelope and unwrap it yourself (envelope.Message).

Recipe 5 (a cousin): Multicast — the same fan-out, but inside the route

Publish-Subscribe copies the message on the broker (SNS hands a copy to each subscribed queue). There's a close cousin that copies inside the route — Multicast: your step sends a copy of the exchange to several destinations itself.

From("sqs://orders")
    .Multicast()
        .To("sqs://orders-billing")
        .To("sqs://orders-shipping")
    .End();

Looks similar, but the difference is fundamental — where the copying happens, and who knows about whom:

	SNS→SQS (pub/sub)	Multicast
Where it copies	on the broker (SNS)	in the route (your process)
Who knows the recipients	SNS (the subscriptions)	the route (the `.To` list)
Adding a recipient	subscribe another queue, leave the publisher alone	edit the route's code
Coupling	fully decoupled	recipients are baked into the route
Parallelism / aggregating replies	none (fire-and-forget per subscription)	yes: `.Parallel().MaxParallelism(N)`, merge the branches

The rule of thumb: need to decouple the publisher from the subscribers (anyone can subscribe later, the publisher never knows) — SNS→SQS. Need to fan out to a fixed list right here and maybe collect the answers — Multicast (or its sibling Scatter-Gather from the Kafka post). One is broker topology, the other is a route step; you pick by where the fan-out logic should live.

Wrap-up

SQS and SNS in redb.Route are routes where the whole queue and the whole topic are one URI: From("sqs://…") / .To("sqs://…") / .To("sns://…"), with all the AWS SDK plumbing, the receive/delete dance, visibility, subscriptions and correlation tucked behind the string's parameters. Long-poll, FIFO, batching, competing consumers on a single parameter, at-least-once with a transactional ack, SNS→SQS fan-out with a choice of "envelope or bare payload" — the set is complete, and what you're left with is a short, legible string.

Straight talk on the trade-offs: there's no exactly-once (SQS itself doesn't offer it) — you get at-least-once plus idempotency; RPC isn't out of the box, it's assembled from attributes and .ToD; and an SNS subscription wraps your payload in an envelope by default. All of that is AWS being AWS, not the connector cutting corners — and the connector is upfront about every bit of it.

Two runnable examples — redb.Route/demos/SqsRpcDemo (RPC + concurrency) and redb.Route/demos/SqsPubSubDemo (SNS→SQS fan-out + raw delivery); both come up on LocalStack (http://localhost:4566, test/test) and run as a self-test. If something bites in your scenario, say so in the comments.

Sources and releases: github.com/redbase-app. About the redb database: redbase.app.

If this was useful — a ⭐ on GitHub helps others find it.

redb.Identity, the full tour: a transport-agnostic OAuth 2.1 / OpenID Connect server for .NET — protocol transport, and no migrations

rinat kozin — Tue, 14 Jul 2026 15:35:16 +0000

series redb.Route

Heads up — this is the long one. The most complete write-up of redb.Identity: from the inside, with code, no shortcuts. Budget around an 30min — it really is that long. It covers the full standards catalogue, audited line by line, the production-grade internals (live JWKS rotation with no restart, cluster-safe primitives, the password-hashing pipeline), the audit pipeline, logical-level backup, host-agnostic deployment, and the addon architecture that lets your own service sit next to Identity in the same database and transaction. Want the overview instead? There's a shorter version. Otherwise — grab a coffee.

There are three usual ways to do OAuth/OIDC in .NET, and every one of them makes you give something up.

The first is the ASP.NET-bound camp — Duende IdentityServer, ASP.NET Identity, the OpenIddict samples. Powerful, but every endpoint is an HTTP middleware. Want to call token from a worker service or a bus consumer? Spin up an HTTP listener and loop back to yourself. Want to test the issuance pipeline in isolation? Reach for WebApplicationFactory.

The second is the full IAM platforms — Keycloak, Auth0, Okta. Feature-rich, but each one is a standalone service with its own runtime, its own admin UI, its own database, its own config model, its own deployment story. Multi-tenant, sure. Embeddable, no.

The third is roll your own — and reimplement Code+PKCE, refresh-token rotation, consent storage, session revocation, MFA replay protection, JWKS rotation, key-ring sharing across replicas, and RFC 8417 back-channel logout, for the third time this decade.

redb.Identity is the missing fourth option, and it comes down to one word: transport-agnostic. The protocol is decoupled from the transport.

It is, first and foremost, a full OAuth 2.1 / OIDC server — stand it up over HTTP, expose it to external clients, discovery, JWKS, the works. It's just that internally every endpoint is a redb.Route route, not an HTTP middleware. And direct-vm:// is redb.Route's internal, cross-context transport — same family as http://, rabbitmq://, kafka://, except the network hop is swapped for a call between RouteContexts in-process.

Because of that, the same token can be reached over HTTP, over gRPC, over a queue — or, if the caller lives in the same worker, with no network at all, over that same internal transport. That last one — "no network hop, just a call from the next context" — is what gives you the embedded mode. But it's a consequence of being transport-agnostic, not the whole story: most of the time Identity runs as a normal HTTP server, and the in-process call is a nice bonus where you want it. The standards conformance and storage model are a full IS server's; the only thing that changes is which transport you reach it over.

It's open source under Apache 2.0 as of today: github.com/redbase-app/redb-identity, packages on nuget.org (redb.Identity.*, version 1.2.0). What follows is the substance — code, no marketing.

Part of the redb / redb.Route series. Identity sits on top of the whole stack, so if a layer
below is unfamiliar, these are the ones worth having open in a tab:

The runtime it ships in — redb 3.3.0: an enterprise .NET stack you actually own, and Two routes in an evening: from a debug worker to an enterprise runtime

The router every endpoint here is a route on — Apache Camel for .NET, dissected: the HTTP connector with no ASP.NET MVC, and the RabbitMQ connector / Kafka connector

The store that makes "no migrations" true — REDB inside, part 1: the 13 tables the whole engine runs on and part 1.1: why they stay fast

Full list on the profile. Sources: github.com/redbase-app. About the database itself: redb.ru.

Skipping ahead: the receipts

A post like this is easy to write and hard to trust. "Full OAuth 2.1 / OIDC server" is a sentence anyone can type. So before the architecture, before a single line of code — here is the part you'd otherwise have to take on faith.

This is the official OpenID Foundation conformance suite — the same suite the OIDF uses to certify providers — run against a live redb.Identity over native HTTPS. Not our tests. Theirs.

Basic OP: 35 modules, zero failures. The 4 REVIEW modules pass — the suite just wants a human to upload a screenshot as certification evidence. The 1 SKIPPED is request objects, which we honestly don't advertise. The 1 WARNING is two extra claims in the id_token that we put there on purpose, and I explain exactly why further down rather than hiding it.

And it was no rubber stamp. The suite found real defects in our server — including PII leaking into the id_token — and they're all fixed. That story is in the conformance section, unvarnished.

One thing I won't claim: we do not carry the OpenID Certified™ mark. That's a trademark, awarded through a formal (and paid) submission to the OIDF. What's true is exactly what's on the screen — the server is run against the official suite, and the results are in the repo.

Now, the architecture.

The layers underneath, in one breath

redb.Identity doesn't stand on nothing — it stands on our ecosystem. Very briefly:

redb — a typed store for .NET. You write a plain POCO, tag it with [RedbScheme], and work with it through full LINQ, server-side, with no migrations and no Include. Providers: PostgreSQL, MSSQL, SQLite.
redb.Route — an integration engine in the Apache Camel spirit: From(...)….To(...) routes, 30+ connectors, enterprise integration patterns, transactions, telemetry.
redb.Tsak — a runtime that turns those routes into a production service: dashboard, hot-reloadable modules (.tpkg), a cluster with a coordinator.

redb.Identity is the fourth layer on top. It doesn't own a database schema and doesn't make its own HTTP stack mandatory. It takes the redb.Route engine, drops the OpenIddict pipeline onto it, and hands storage to redb. What falls out of that is the rest of this post.

All the code samples are in English (they always were); everything's in the repo, so you can reproduce it.

Killer feature #1: protocol ≠ transport

Every public endpoint in Identity is registered on a direct-vm://identity-* route. direct-vm is redb.Route's in-process, cross-context, synchronous zero-copy transport. Three things fall out of that for free.

1. Another module in the same worker calls Identity with no network at all

Picture a single Tsak worker running Identity next to your business module. The business module needs a service-account token to reach a downstream API. In the ASP.NET world that's an HTTP call to yourself over loopback. Here it's a plain in-process call:

// Inside another .tpkg module loaded into the same Tsak worker
public class CheckoutRoutes : RouteBuilder
{
    private IProducerTemplate _identity = null!;   // request-reply into Identity, in-process

    protected override void Configure()
    {
        _identity = new ProducerTemplate(Context!);
        _identity.Start();

        From("rabbitmq:checkout.orders")
            // Need a service-account access token to call a downstream API?
            // Just hit Identity's token endpoint right here — same process, no socket.
            .Process(async (e, ct) =>
            {
                var token = await _identity.RequestBody<TokenResponse>(
                    IdentityEndpoints.Token,                // "direct-vm://identity-token"
                    new { grant_type = "client_credentials", client_id = "checkout-svc", /* ... */ });
                e.In.SetHeader("Authorization", $"Bearer {token!.AccessToken}");
            })
            .To("http://pricing-api/quote");
    }
}

What's happening here. The message travelling down the route is the order off the queue — that's the body. Halfway through we make a side call: ask Identity for a service-account token, take access_token out of the reply, and hang it on the current message as a header. The order itself is untouched. At the end the order — now carrying Authorization — goes off to somebody else's API.

Why the token is fetched inside .Process(...) rather than with .To(IdentityEndpoints.Token) in the DSL is a fair question, and the answer matters. Because .To() is a pipeline step: the reply becomes the new body. Write this:

From("rabbitmq:checkout.orders")
    .To(IdentityEndpoints.Token)       // ← the body is no longer the order, it's a TokenResponse
    .To("http://pricing-api/quote");   // ← pricing-api receives the TOKEN instead of the order

— and the order is gone: the downstream API gets a JSON blob with an access_token where the cart items should be.

What you want instead is to step aside, take one field out of the reply, attach it to the current message and leave the body alone. That's the Content Enricher EIP, and RequestBody(...) inside a processor is the canonical way to express it (in Apache Camel it's exactly ProducerTemplate.requestBody). redb.Route does have a declarative .Enrich(uri, merge) — but it forwards the current exchange to the resource, so the order would land on the token endpoint, and Identity expects grant_type=client_credentials. Enrich is for "enrich this message"; ours is a side call with a different payload.

And yes, that last line really is http:// — as it should be. pricing-api is somebody else's service; you reach it over the network and no architecture changes that. The point is that there is now one network call instead of two:

	ASP.NET-bound IS	here
get a token	HTTP to yourself (loopback)	`direct-vm://` — a method call, no network
call pricing-api	HTTP	HTTP (unavoidable)

We removed the first one — the one that should never have existed. Making an HTTP request to your own process, serializing JSON and shaking TLS hands with yourself, is a tax you pay purely because token happens to be an HTTP middleware.

No HTTP listener. No TLS handshake. No JSON over the loopback. The exchange flows straight from your route into the Identity processor and back — same thread, same IExchange instance. Total cost: a method call plus the OpenIddict work you'd be doing anyway.

This is exactly the thing ASP.NET-bound IS servers can't do: for them token is an HTTP middleware, and calling it "from the inside" without a socket isn't a thing.

2. Facades are pure transport adapters — take the ones you need

HTTP is the first facade, not the only one and not a requirement. A facade is a thin bridge with no business logic in it:

// HTTP facade — ships in the box
From(Http.From("0.0.0.0:5000"))
    .RedbController<TokenController>();     // POST /connect/token
    // ↓ The controller's entire job:
    //   exchange.To(IdentityEndpoints.Token)   // direct-vm://identity-token

// gRPC facade (planned) — same pattern
From(Grpc.Server("0.0.0.0:5001/IdentityService"))
    .Filter(e => e.In.GetHeader("grpc.method") == "Token")
    .To(IdentityEndpoints.Token);

// RabbitMQ RPC facade (planned)
From("rabbitmq:identity.rpc.token")
    .InOut()
    .To(IdentityEndpoints.Token);

Adding a transport is dropping another .tpkg that points at direct-vm://. Core never gets touched. Removing a transport is rm facade.tpkg. There's nothing in a facade to break — no logic lives there.

Under the hood the shipped HTTP facade (redb.Identity.Http) is exactly that thin: parse, forward, serialize.

// HttpFacadeRouteBuilder.cs
From($"http:POST:0.0.0.0:{port}/connect/token?inOut=true{ClientCorsParams()}")
    .RouteId("http-token")
    .Process(HttpIdentityProcessors.PropagateCorrelationId)
    .Process(ClientAuthHttpProcessors.ExtractClientCredentials)
    .To(IdentityEndpoints.Token)                  // ← forward into direct-vm
    .Process(HttpIdentityProcessors.SerializeJsonResponse);

Want to swap HTTP for RabbitMQ entirely? In an ASP.NET stack that's a rewrite. Here it's drop the HTTP facade, keep Core.

3. A browser shows up exactly where the spec demands one

The only OAuth interaction that fundamentally needs a browser is the authorization-code redirect (and its sibling, the device-code verification URL). Everything else — token, refresh_token, introspect, revoke, userinfo, the device_code request, the management API, MFA verify, SCIM — is transport-neutral and works over gRPC, AMQP, MQ, or a direct call.

Flow / endpoint	Needs HTTP + browser	Works on any transport
`client_credentials` (M2M)	—	✅
`authorization_code` — redirect to `/authorize`	✅	—
`authorization_code` — `code → token` exchange	—	✅
`refresh_token`, `revoke`, `introspect`, `userinfo`	—	✅
`device_code` — initial request	—	✅
`device_code` — user verification URL	✅	—
Management API, SCIM 2.0, audit query	—	✅

The same TokenEndpointProcessor runs whether the call came in over HTTP, gRPC, RabbitMQ, or a direct direct-vm call from the next module. One pipeline, one set of tests, one audit trail. In practice that also buys you cheap integration tests (most of the 1768-test suite drives direct-vm and never spins up Kestrel) and multi-port isolation — public OIDC endpoints on one port, the management API and SCIM on another, firewalled independently, same pipeline behind both.

Killer feature #2: no migrations

This one comes up from redb, but it lands especially well for an identity server. redb.Identity doesn't own a schema. It sits on redb, where a "table" is a plain C# class with an attribute. Here's a user:

// src/redb.Identity.Core/Models/UserProps.cs
[RedbScheme("identity.user")]
public class UserProps
{
    // Standard OIDC profile claims
    public string? GivenName    { get; set; }
    public string? FamilyName   { get; set; }
    public string? Picture      { get; set; }
    public bool    EmailVerified { get; set; }

    // Structured OIDC address (§5.1.1) — a nested redb object, not a JSON blob
    public AddressClaim? Address { get; set; }

    // Arbitrary tenant claims — each pair becomes its own props row,
    // queryable and indexable without an ALTER TABLE.
    public Dictionary<string, string>? CustomClaims { get; set; }

    // Multi-provider federation links — native props rows, hot reverse lookup
    // via RedbObject.value_string = "{providerId}:{sub}".
    public Dictionary<string, ExternalIdentity>? ExternalIdentities { get; set; }

    public string? ScimExternalId { get; set; } // RFC 7643 §3.1
}

Need to add a field to your user — a LoyaltyTier, a ManagerSubject, a DepartmentCode? You literally add it to the class. No migration, no ticket to a DBA, no downtime. redb reads and writes it the next instant; once it's in production data, queries can filter and project it.

A word on CustomClaims. That's not an nvarchar(max) blob or a jsonb column you later hand-index with GIN. It's a Dictionary<string,string> on props, and every key is its own queryable, indexable row. ContainsKey, the indexer, nested access — all of it runs as native LINQ server-side. A typical EF-Core identity server keeps custom claims either as a blob or normalized into eight satellite tables with joins.

There's a hot/cold split too: login, password hash, and status live in the relational _users table (hot keys, narrow indexes), while the cold OIDC profile lives in props rows linked by RedbObject.key = _users._id. No over-indexed wide rows, no JSON-blob lookups on the hot path.

The whole thing is composed from 24 typed redb schemes (identity.application, identity.scope, identity.token, identity.session, identity.user, identity.group, identity.mfa, identity.webauthn_*, identity.federation_provider, identity.claim_mapper, identity.dpop_consumed_jti, identity.signing_key, …). All plain C# classes. InitializeAsync() syncs the schema. No migration files.

RTTI at the database level — what the storage model actually is

Most identity servers spread storage across half a dozen moving parts: EF Core + migrations for users/apps/claims, Redis for session cache and MFA OTP, the filesystem or Azure Key Vault for the DataProtection key ring, a separate table for signing keys, Elasticsearch for the audit log, and something else again for rate-limit counters. Each part is its own client, its own init, its own health check, its own backup plan, its own migration story.

Here a single store — REDB — handles all of it, because it keeps typed data schemes declared in the database itself — runtime type information at the DB level:

Language concept	In REDB
`class Foo { int X; string Y; }` (type declaration)	a row in `_schemes` describing the fields
`new Foo { X = 42, Y = "bar" }` (instance)	a row in `_objects` + typed slots in `_values`
`typeof(Foo)` / `obj.GetType()` (RTTI)	`_objects._id_scheme → _schemes._id`
`where T : class, new()`	the C#-side `RedbObject<TProps>` proxy

The C# classes (UserProps, ApplicationProps, …) are just a type-safe view over the schemes. What that buys the identity server:

One query language for everything. Finding a user, a token by reference_id, an authorization by subject GUID — one IRedbService, one LINQ provider, one transactional context:

await _redb.Query<UserProps>()
    .Where(u => u.EmailVerified)
    .WhereRedb(o => o.Key == coreUserId)
    .FirstOrDefaultAsync();

await _redb.Query<TokenProps>()
    .WhereRedb(o => o.ValueString == referenceId)
    .FirstOrDefaultAsync();

Every OpenIddict store (RedbApplicationStore, RedbAuthorizationStore, RedbTokenStore, RedbScopeStore) plus ours (PropsSigningKeyStore, PropsServerSideOtpStore, PropsWebAuthnChallengeStore, PropsPasswordHistoryStore) sits on one interface.

One backup story. Back up Postgres and you've backed up the entire identity stack — keys (DataProtection-encrypted in the same DB), sessions, signing keys, audit. There's no "oh, and that one component is still in Redis, back that up separately."

Multi-tenant isolation via schemes. Want tenant A on one shape of UserProps and tenant B on a slightly different one? Per-tenant schemes — the split is physical, through _id_scheme.

Pro mode with PVT. The Pro build ships Precomputed Value Tables — materialized indexes for hot-path queries. A lookup on value_string == clientId goes through the PVT index instead of an unrolled join over _values. Details in the REDB indexing post.

Killer feature #3: the same server — also as an embedded library

This one falls straight out of the first two: once the protocol is decoupled from the transport and direct-vm:// is the internal cross-context transport, one of the available modes is no network facade at all. Not a replacement for "a normal HTTP server" — an extra option for when Identity lives in the same worker as whatever calls it.

"Stand up an identity server" usually means run a separate process: its own runtime, its own port, its own admin, its own DB, its own deploy. Even the "embeddable" ASP.NET libraries still need an HTTP pipeline to call them.

With redb.Identity you can take just the engine — redb.Identity.Core — and zero facades. No Kestrel, no HTTP. You reference the package, bring up the OpenIddict pipeline inside your process, and hit the endpoints over direct-vm:// from your own code:

// In your app: you need a client token for an internal call —
// no network hop, no Identity HTTP server standing up at all.
var producer = new ProducerTemplate(ctx);   // ctx — your IRouteContext
producer.Start();

var token = await producer.RequestBody<TokenResponse>(
    IdentityEndpoints.Token,                // "direct-vm://identity-token"
    new { grant_type = "client_credentials", client_id = "internal-svc", client_secret = secret });

That changes the embedding model. The identity server stops being "a service next door" and becomes a library inside your process — one that happens to have a full OAuth 2.1 / OIDC engine under it, but you talk to it with a method call, not a network request. When the day comes to expose those same endpoints to external clients, you add an HTTP facade as one .tpkg, and the exact same processors start answering over HTTP too. Core doesn't change.

Neither Duende, nor Keycloak, nor Auth0 does this: the first has HTTP-middleware endpoints, the rest are external services with a network hop.

Two contexts, one worker

Architecturally it's two RouteContexts inside one Tsak worker:

┌───────────────────────────────────────────────────────────────────────┐
│                             Tsak worker                                 │
│                                                                         │
│  ┌───────────────────────────────────────────────────────────────────┐ │
│  │  RouteContext "identity"   ← redb.Identity.Core.Module.tpkg        │ │
│  │  ─────────────────────────────────────────────────────────────    │ │
│  │  • ~50 direct-vm:// routes                                         │ │
│  │  • OpenIddict server pipeline (token, authorize, userinfo, ...)    │ │
│  │  • redb stores: Users, Apps, Scopes, Tokens, Sessions, Audit       │ │
│  │  • DataProtection key-ring (RedbXmlRepository)                     │ │
│  │  • Signing keys (RSA 2048, encrypted at rest)                     │ │
│  │  • MFA: TOTP / SMS-Email OTP / WebAuthn / recovery codes           │ │
│  │  • cleanup timers (.Cluster(true) → leader-only in a cluster)      │ │
│  └───────────────────────────────────────────────────────────────────┘ │
│                        ▲  direct-vm:// (synchronous, in-process)         │
│                        │                                                 │
│  ┌───────────────────────────────────────────────────────────────────┐ │
│  │  RouteContext "identity.http"  ← redb.Identity.Http.tpkg           │ │
│  │  • Kestrel → redb.Route.Http → RedbController dispatcher           │ │
│  │  • /connect/token, /authorize, /userinfo, /introspect, ...         │ │
│  │  • /api/v1/identity/* (management), /me/* (self-service)           │ │
│  │  • /scim/v2/Users, /Groups, /Bulk                                  │ │
│  └───────────────────────────────────────────────────────────────────┘ │
│                                                                         │
│  Your modules → call IdentityEndpoints.Token / .ManageUsers / ...       │
│  over direct-vm:// directly, no HTTP                                     │
└───────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
                 redb store (PostgreSQL / MSSQL / SQLite)

Worth calling out: project-reference isolation. The HTTP facade compiles without seeing a single type from redb.Identity.Core:

redb.Identity.Http  ─project-ref→  redb.Identity.Contracts       ✅
redb.Identity.Http  ─project-ref→  redb.Identity.DataProtection  ✅
redb.Identity.Http  ─project-ref→  redb.Identity.Core            ❌ FORBIDDEN

The facade only sees wire DTOs and endpoint constants (redb.Identity.Contracts) plus a couple of SPI interfaces. Core's internal types never leak across the ABI boundary — CI fails the build if a using redb.Identity.Core; ever shows up under redb.Identity.Http/. That's what makes the ".tpkg story" honest: the two packages build and version independently.

Anatomy of a token request

Since every endpoint is a route, it's worth seeing what one is made of. The token endpoint isn't a "controller method" — it's a declarative redb.Route pipeline:

// IdentityCoreRouteBuilder.tokenRoute (RouteId: identity-token)
WithRedbTx(...)                        // atomic write boundary (one tx per issuance)
  .Process(trustedProxy)               // sanitize X-Forwarded-For BEFORE rate-limit
  .Process(perIpThrottle)              // per-IP limit (optional)
  .OnException<InvalidOperationException>()   // RFC 6749 error → HTTP status mapping
  .Throttle(clientId, ...)             // per-client token bucket
  .Traced("identity.token-request")
    .Metered(...,
      new TokenEndpointProcessor(handler, timeProvider))   // ← OpenIddict pipeline
      //   ↑ inside: redb stores — RedbTokenStore, RedbApplicationStore, ...
  .EndTraced()
  .WireTap("direct-vm://identity-events");   // audit + multicast, fire-and-forget

Three things are visible here that ASP.NET usually smears across middleware, filters, and attributes: the transactional boundary (WithRedbTx — the token is issued and persisted whole, or it rolls back), rate limiting before the request reaches the expensive work, and audit as a branch (WireTap) that can't block issuance or take it down. Same TokenEndpointProcessor, whether the call arrived over HTTP, gRPC, or direct-vm. One pipeline, one audit trail, one test suite.

Storage: what it buys you over EF-Core-on-tables

Property	redb.Identity	Typical EF-Core identity server
Storage engines	PostgreSQL, MSSQL and SQLite from one codebase — swap by the provider package	one provider per build; switching means rewriting the EF model + migrations
Schema evolution	no migrations. Add a field to a `*Props` class — the scheme picks it up on next `InitializeAsync`	generate a migration, review the SQL, run `Update-Database`, hope rollback works
Custom claims / tenant extensions	`Dictionary<string,string>? CustomClaims` — each key is a queryable, indexable row	`jsonb`/`nvarchar(max)` blob; you write the GIN / computed-column indexes
Multi-provider federation links	`Dictionary<string, ExternalIdentity>` — native props rows, hot reverse lookup	one-to-many join table, scaffolding per provider
Hot/cold split	hot keys in relational `_users`; cold profile in props linked by `key = _users._id`	either one wide table or eight satellite tables
Multi-tenant data isolation	`GetRedbService("identity")` — the named instance can target a separate DB / schema / connection	single `DbContext`; isolation means separate ASP.NET apps

Providers: zero code change between rows

Engine	OSS package	Pro package
PostgreSQL 13+	`redb.Postgres`	`redb.Postgres.Pro`
Microsoft SQL Server 2019+	`redb.MSSql`	`redb.MSSql.Pro`
SQLite 3.44+	`redb.SQLite`	`redb.SQLite.Pro`

Identity references only the redb.Core OSS abstraction — the host worker picks the provider, Identity code never names one. And this isn't a claim: the same test suite (1768 tests) is green on all three — Passed: 1767, Skipped: 1, Failed: 0 on PostgreSQL, MSSQL and SQLite, flipped with a single REDB_PROVIDER env-var. (The one skip is a known host-teardown probe on PG, not a product gap.)

Standards — the full catalogue

No skimping here, because standards conformance is why you reach for an IS server in the first place. Close to 40 standards — not "mentioned in the code," but implemented: every assertion sits next to its spec reference, and every contract has a demo_*.ps1 probe against a live server.

Before publishing, we audited this catalogue line by line — against the code, against live discovery, against the demos. Several rows did not survive: some are now worded precisely, some were struck and moved to the "what's missing" section below, and three turned out to be cheaper to build than to delete — which is where the claims parameter, the SCIM Enterprise extension and loopback redirects came from. The limitations list at the end isn't a disclaimer; it's the output of the same audit. Anything we overclaimed here, you could disprove from our own /.well-known/openid-configuration in ten seconds.

OAuth 2.0 / 2.1 core

RFC / spec	What	Demo
RFC 6749 OAuth 2.0 Framework	grant types, scopes, error responses	`demo_client_credentials.ps1`, `demo_authcode_pkce.ps1`, `demo_refresh_rotation.ps1`, `demo_password_ropc.ps1`
RFC 6750 Bearer Token Usage	`Authorization: Bearer`, `WWW-Authenticate: Bearer` challenge	`demo_userinfo.ps1` — asserts WWW-Authenticate on missing/empty/garbage/tampered
RFC 6585 Additional HTTP Status Codes	429 Too Many Requests + Retry-After	`demo_throttle_rfc6585.ps1` — parallel burst + recovery + per-key isolation
RFC 7009 Token Revocation	`/connect/revoke`	`demo_introspect_revoke.ps1`
RFC 7521 Assertion Framework	assertions for client authentication (`client_assertion`). No assertion grant types (`jwt-bearer`, `saml2-bearer`)	covered via RFC 7523
RFC 7523 JWT Bearer Assertion (private_key_jwt)	client assertion on token / introspect / revoke / par / device	`demo_private_key_jwt.ps1` — DCR with inline JWKS + token + introspect + tamper-negative
RFC 7591 Dynamic Client Registration	`POST /connect/register`	`demo_dcr_lifecycle.ps1`
RFC 7592 DCR Management Protocol	`GET/PUT/DELETE /connect/register/{client_id}`	`demo_dcr_lifecycle.ps1` (same file)
RFC 7636 PKCE	code_verifier / code_challenge / S256 (`plain` rejected per OAuth 2.1)	`demo_authcode_pkce.ps1`
RFC 7662 Token Introspection	`/connect/introspect`	`demo_introspect_revoke.ps1`
RFC 8252 OAuth 2.0 for Native Apps	PKCE-required public clients + §7.3 loopback redirect: the port is not compared (without it no `az login`-style CLI can work — they get an ephemeral port from the OS at launch)	`demo_loopback_redirect.ps1` — 14 assertions, most of them negative
RFC 8414 Authorization Server Metadata	`/.well-known/oauth-authorization-server`	`demo_discovery_jwks.ps1`
RFC 8628 Device Authorization Grant	`/connect/deviceauthorization`, 5s polling	`demo_device_code.ps1` + non-interactive `demo_device_code_ci.ps1` for CI
RFC 8693 Token Exchange	impersonation + delegation chain (`act`). Opt-in; `subject_token_type` is `access_token` only	`demo_token_exchange.ps1` — positive + negative
RFC 8725 JWT BCP	algorithm allow-listing for DPoP proofs and client assertions; `none` is rejected	`demo_dpop.ps1` (DPoP proof), discovery (`*_auth_signing_alg_values_supported`)
RFC 9068 JWT Profile for Access Tokens	typed `at+jwt`. Access tokens are JWE-encrypted by default — a resource server either introspects, or you flip `DisableAccessTokenEncryption` for local validation	token header visible in `demo_jwt.ps1`
RFC 9126 Pushed Authorization Requests (PAR)	`/connect/par`, per-client `require_pushed_authorization_requests`	`demo_par.ps1` + `demo_par_per_client.ps1`
RFC 9207 Authorization Response `iss`	`iss` in the auth response	`demo_auth_extras.ps1`
RFC 9449 DPoP	proof-JWT validation, replay cache, `DPoP-Nonce`, resource-server validator	`demo_dpop.ps1`

OpenID Connect 1.0 family

Spec	What	Demo
OIDC Core 1.0	code flow (OAuth 2.1 — no implicit, no hybrid). id_token (RS256), sub/aud/exp/iat, nonce, prompt, max_age, acr_values, claim mappers	`demo_discovery_jwks.ps1`, `demo_claim_probes.ps1`, `demo_prompt_max_age.ps1`, `demo_acr_values.ps1`
OIDC Core §5.1 full `profile` set	all 14 claims; `updated_at` a JSON number, `_verified` JSON booleans*	`demo_claim_probes.ps1`
OIDC Core §5.4 claim delivery	scope-derived claims (profile/email/phone/address) are served from UserInfo, not baked into the id_token — the id_token stays a statement about the authentication event, not a PII container	`demo_claim_probes.ps1` — asserts the absence of PII in the id_token
OIDC Core §5.5 `claims` parameter	an RP names the exact claims it needs instead of pulling the whole `profile` scope to get one `name`, and picks the channel: `userinfo` or `id_token`. `essential` / `value` / `values` supported	`demo_claims_parameter.ps1` — 14 assertions, negatives included
OIDC Discovery 1.0	`/.well-known/openid-configuration` + JWKS	`demo_discovery_jwks.ps1` + manual RS256 verification
OIDC Dynamic Client Registration 1.0	RFC 7591 extensions for OIDC client_metadata	`demo_dcr_lifecycle.ps1`
RP-Initiated Logout 1.0	`/connect/logout` + `post_logout_redirect_uri`	`demo_logout_endsession.ps1`
Back-Channel Logout 1.0	`backchannel_logout_uri` + JWT logout token + pull-based revoked-SID feed for multi-replica RPs	`demo_backchannel_logout.ps1`

Validated by the official OpenID Foundation conformance suite — not just our own tests

A catalogue is a claim. The proof is running the very conformance suite the OpenID Foundation uses to certify providers — not our tests, not "works on my machine," the official suite against a live server. Most .NET OAuth/OIDC implementations have never seen it; we drive redb.Identity through it as a real OP.

Basic OP — 35 modules, 0 failures:

Result	Modules	What it means
PASSED	29	green
REVIEW	4	the suite wants a human-uploaded screenshot (login form, error page) — counted as passing
WARNING	1	two extra claims in the id_token — a deliberate extension of ours, discussed below
SKIPPED	1	request objects (RFC 9101) — we honestly don't advertise them, so the suite skips
FAILED	0	—

The Config OP profile is clean too (native HTTPS, no reverse proxy).

The suite is unforgiving: it checks exactly the RFC requirements hand-rolled servers trip on. And it was no rubber stamp — it found real defects. Here they are, unvarnished:

authorization errors are delivered only to the client's registered redirect_uri (RFC 6749 §4.1.2.1) — an error open-redirect, closed, not "redirect to whatever was sent";
a reused authorization code returns 400 invalid_grant, not 401 invalid_token (§5.2);
Cache-Control: no-store on token / introspection / revocation (§5.1);
email_verified / phone_number_verified are JSON booleans, not strings (OIDC §5.1);
prompt=login / max_age route to /login and complete on re-login; the re-auth marker is bound to the session id — no redirect loop, no same-second bypass;
PII was leaking into the id_token. Claims from the profile / email / phone / address scopes were baked straight into the id_token, when in the code flow they belong in UserInfo (OIDC §5.4). An id_token gets forwarded to third parties and logged as proof of the sign-in event, so the user's phone number travelled a great deal further than the RP ever intended. It doesn't any more;
UserInfo, conversely, was returning too much — the token's own plumbing: OpenIddict's oi_* internals, jti / exp / iat / at_hash. Those describe the token, not the user; UserInfo (§5.3) must return the user's claims and nothing else;
/connect/userinfo didn't accept the access token in the POST body (RFC 6750 §2.2);
the profile claim set was incomplete — the suite diffs UserInfo against exactly the §5.1 list and flags every claim you omit.

The suite also raised a WARNING on oidcc-claims-essential: "name not found in userinfo". We dug in: the test asks for name through the claims parameter (§5.5) — which we had not implemented at all, and our own discovery document admitted it ("claims_parameter_supported": false). So we built it.

Two warnings we kept on purpose

The very first module, oidcc-server, finishes with two warnings:

WARNING  EnsureIdTokenDoesNotContainNonRequestedClaims
         id_token contains non-requested claim 'oi_tkn_id'
WARNING  EnsureIdTokenDoesNotContainNonRequestedClaims
         id_token contains non-requested claim 'redb:user_id'

We're not going to bury those. Here's why they're there and why they stay.

First, the important bit: these are warnings, not failures. OIDC Core does not forbid additional claims in an id_token. The suite warns because an extra claim can mean user data is leaking — and its own message concedes the alternative: "…or that it implements an extension the conformance suite is not aware of." That's our case.

Neither claim is data about the user:

oi_tkn_id — the id of the token's entry in the store. It is what makes the id_token revocable, and what drives back-channel logout. Removing it means giving up id_token revocation. That trades a real capability for a clean warning list, and we declined the trade.
redb:user_id — our own private claim, namespaced per the RFC 7519 §4.3 convention. The public sub is a GUID (stable, correct across instances); the hot key in the relational _users table is a bigint. This claim lets a client decode the id_token and join the user to its own table by our internal id, without a round-trip back to us.

And here's the part that proves this isn't a rationalisation. In that same run we deleted a third such claim from the id_token. OpenIddict was also stamping oi_au_id in there — its internal link to the authorization entry. That one has no business in a token handed to a client: it means nothing to them, and there was no defending it. Gone (kept on the access_token, where introspection needs it).

So we didn't wave the warnings away. We went through each one and kept exactly the two we can answer for.

That's the line between "wrote OAuth over a weekend" and a server that clears the same checks industrial IdPs do. Full per-module breakdown and local setup: OPENID_CERTIFICATION.md in the repo.

To be scrupulous: we do not carry the OpenID Certified™ mark. That's a trademark, granted by the OIDF through a formal submission. What we claim is exactly what's true — the server is run against the official OIDF conformance suite, and the results are in the repo.

JOSE / JWT

RFC 7515 (JWS), 7517 (JWKS), 7518 (JWA), 7519 (JWT), 7638 (JWK thumbprint, used as the DPoP jkt), 7800 (cnf / proof-of-possession), 8176 (amr).

SCIM 2.0

RFC	What	Demo
RFC 7643 SCIM Schema	User, Group, Meta	`demo_scim.ps1`
RFC 7643 §4.3 Enterprise User extension	`department`, `manager`, `employeeNumber`, `costCenter`, `organization`, `division`. `manager` is a complex attribute; `$ref` and `displayName` are derived, not stored (a stored copy would rot the moment the manager is renamed)	`demo_scim_enterprise.ps1` — 24 assertions
RFC 7644 SCIM Protocol	CRUD + ETag concurrency (RFC 7232 → 412) + discovery. Filter is single-expression: `userName` / `externalId` / `displayName` / `emails.value`, no `and`/`or`. Bulk is opt-in (`Features.EnableScimBulk`, off by default — and the ServiceProviderConfig says so honestly)	`demo_scim.ps1` (CRUD) + `demo_scim_bulk.ps1` (bulk) + `demo_scim_etag.ps1` (concurrency)

The Enterprise extension is what corporate provisioning sends on its very first request: Okta, Entra ID and Workday all push department and manager on the initial sync. A provider that advertises only the core schema makes them drop that data on the floor.

The SCIM discovery endpoints (/scim/v2/ServiceProviderConfig, ResourceTypes, Schemas) are served unauthenticated and unconditional, so an RP can probe them before SCIM provisioning is switched on.

MFA / WebAuthn / OTP

Standard	What	Demo
RFC 6238 TOTP	time-based OTP, atomic + replay-protected per §5.2 (160-bit secret per RFC 4226 §4)	`demo_mfa_totp.ps1`
W3C WebAuthn Level 2 / FIDO2	passkey enrollment + assertion, `attestation=none/direct/indirect/enterprise`, `userVerification` variants	API ready, demo waits on a frontend companion
NIST SP 800-63B / OWASP ASVS 4.0.3 §2.1	password policy (min 12, upper+lower+digit, history)	`demo_password_change_negatives.ps1` 13/13
Argon2id (OWASP 2023 primary)	64 MiB / 3 iterations / parallelism 4	enforced, upgrade-on-login for legacy BCrypt

SMS/Email OTP and recovery codes (one-shot, consumed in the same transaction that creates the session) — demo_mfa_recovery_codes.ps1. Disable / replace a method — demo_mfa_disable_replace.ps1.

Federation, self-service, admin

External OIDC IdPs (Google, Microsoft, Keycloak) plus a dedicated GitHub OAuth2 path (no discovery, no id_token — profile and emails pulled from the REST /user and /user/emails, endpoints configurable for GitHub Enterprise / Gitea). Provision-on-first-login, link-on-replay (same sub → same user, no duplicate), self-service link/unlink via /me/federated-identities/*, email-conflict resolution that refuses a silent takeover. Demos: demo_federation.ps1, demo_federation_e2e.ps1 (full round-trip against navikt/mock-oauth2-server), demo_federation_github.ps1, demo_federation_link_unlink.ps1. LDAP / Active Directory is a separate redb.Identity.Ldap package with bind-on-login and UserAccountControl parsing.

Self-service and admin round it out: registration + email verify, profile and email change with dual-confirm, password change (13 negatives) and forgot-password round-trip, sessions list/revoke (own and admin, with dry-run), self-service account delete with cascade, OIDC client CRUD, a granular scope gate (identity:audit.read is distinct from identity:read is distinct from identity:users.manage), group hierarchy with per-member role labels, and the emergency-admin bootstrap endpoint. Each has its own demo.

Two that deserve a shout because they're rare in .NET: DPoP (RFC 9449) and private_key_jwt (7521/7523) — both implemented, both pinned by demo probes, not roadmap items.

Production-grade features — the "later" list, done now

The unglamorous things that decide whether an identity server survives production.

Live JWKS rotation — no process restart

The classic problem: you shipped an OIDC server, 60 days pass, you need to rotate the signing key (RP JWKS caches live up to a week, and then there are incidents). Most implementations — vanilla OpenIddict included — need a process restart to rotate, because OpenIddictServerOptions.SigningCredentials is built once via IPostConfigureOptions<> and cached.

Here:

An admin hits POST /signing-keys/rotate (requires scope identity:applications.manage).
PropsSigningKeyStore.RotateAsync persists the new key and marks the old one demoted — but it stays in the JWKS until retire, a grace window for in-flight tokens.
IOptionsMonitorCache<OpenIddictServerOptions> and …ValidationOptions are invalidated immediately.
The next OIDC handler re-evaluates the whole Configure → PostConfigure chain, and PropsSigningKeyStoreOpenIddictPostConfigure rebuilds SigningCredentials from the current store snapshot.
New tokens are signed under the just-rotated kid right away; in-flight tokens validate under the old key until it's retired.

demo_jwks_rotation.ps1 asserts the full grace-window flow: after rotate the JWKS serves K1 + K2 (old-in-grace + new-active); new id_tokens carry the just-rotated kid; after DELETE /signing-keys/{K1} the JWKS holds only K2 and old tokens stop validating; K1 stays in the admin audit list with inJwks=false for the compliance trail. All without a restart.

Cluster-safe primitives — built in

It's an identity server; it'll live in a cluster. Baked in:

Optimistic concurrency on every RedbObject<T> via a _hash field. UpdateAsync takes a row lock, checks the hash, and throws OpenIddictExceptions.ConcurrencyException on a mismatch. A test drives exactly that with two concurrent writes.
A cluster-wide lock for schema init. When several workers come up at once, IdentitySchemaInitListener takes LockForUpdate on a specific row — "leader under a cluster lock" — initializes the schemes, and releases. Followers see another node holds the lock, proceeding idempotently and wait.
Atomic claim for background tasks. Trash purge and orphaned-task recovery go through TryClaimOrphanedTaskAsync — an atomic UPDATE with a WHERE condition. Whichever worker flips pending → running first owns the work. No distributed locking needed.
Per-request cache invalidation via the DI scope. RedbApplicationStore._clientIdCache lives for exactly one HTTP request and is invalidated on CRUD — no stale data across requests.

Background deletion — the DB as a queue

Deleting an OIDC application with thousands of accumulated tokens and authorizations is an expensive cascade. Do it on the request thread and the user times out; do it through an in-memory channel and you lose it on a worker crash. Here:

SoftDeleteAsync synchronously re-parents the object under a trash scheme (scheme_id = -10). It instantly disappears from every Query<>(). The RP sees 204, the operation "succeeded."
A BackgroundDeletionService polls the DB for pending trash containers every 5 seconds. Cluster-safe (TryClaimOrphanedTaskAsync); on a worker crash the next poll cycle picks it up.
The purge runs in batches of 10 with a Task.Delay(50ms) between them, so live traffic can interleave.

No in-memory state — a crash mid-operation loses nothing; the DB stays the source of truth.

DataProtection key ring on the storage layer

The cookie session (SameSite=Strict), federation state, MFA setup tokens, the recovery-code pepper — all signed or encrypted via ASP.NET Core DataProtection. By default DataProtection keeps keys on the filesystem or in Azure Storage, which isn't cluster-safe out of the box and needs its own operational care. Here the key ring lives in the same REDB store via RedbXmlRepository: back up Postgres and the ring comes with it; every cluster node sees the same keys (rotation streams through REDB notifications too); the keys are encrypted at rest by a configurable master key (AES-GCM, certificate-based, or a custom KMS factory).

Rate limiting with a distributed backend

/connect/token, /login, /mfa/verify, /mfa/recovery are all throttled — per-IP and per-(IP+username) counters. The backend is an option: memory (single-node in-memory bucket) or redis (cluster-wide via StackExchange.Redis, one Redis for rate limits + the federation-state nonce store). 429 + Retry-After is exactly to spec; demo_throttle_rfc6585.ps1 runs a parallel burst, recovery after the window, and isolation between keys.

Password hashing — Argon2id and BCrypt side by side, with upgrade-on-login and a timing-attack fix

The details that separate a production hashing pipeline from a tutorial one. Two algorithms run in parallel, no deployment-level either/or.

Argon2id — the primary for new deployments. Defaults straight from the OWASP Password Storage Cheat Sheet 2023:

new Argon2idPasswordHasher(
    memoryKib: 65536,    // 64 MiB
    iterations: 3,
    parallelism: 4,
    saltBytes: 16,
    hashBytes: 32);

A memory-hard function: 64 MiB per verify makes GPU/ASIC attacks uneconomical compared to BCrypt (compute-hard but memory-cheap, so a mass FPGA attack works). Argon2id is the hybrid that won the Password Hashing Competition in 2015.

BCrypt — kept for legacy and compliance-driven deployments. It's been in production since 1999; an auditor sees a familiar algorithm with known failure modes, and the PCI-approved libraries allow it outright. And there's the migration story: if you're moving off IdentityServer / ASP.NET Identity, your DB already holds BCrypt hashes — we read them natively, no forced password reset on 100k users.

Upgrade-on-login — a transparent migration. Every successful login checks whether the stored hash matches the current OWASP parameters. If not, a fire-and-forget rehash into the fresh format:

// LoginService.cs (simplified)
if (_passwordHasher is not null && NeedsRehash(_passwordHasher, coreUser.Password))
{
    _ = Task.Run(async () =>
    {
        using var rehashScope = scopeFactory.CreateScope();
        var rehashRedb = rehashScope.ServiceProvider.GetRequiredService<IRedbService>();
        var freshUser = await rehashRedb.UserProvider.GetUserByIdAsync(userId);
        await rehashRedb.UserProvider.SetPasswordAsync(freshUser, capturedPassword, ...);
    });
}

Over N logins the whole base drifts onto Argon2id — no forced reset, no migration window.

Constant-time fake-verify — against user enumeration by timing. ValidateUserAsync returns null for three failure modes: user not found, user disabled, wrong password. Without mitigation (1) and (2) return in 5–10ms (the DB lookup misses immediately) while (3) takes ~250ms (BCrypt actually runs), and an attacker can tell "does this email exist" from the response time — that's CWE-204. A precomputed FakeBcryptHash (workFactor 12) runs on every negative path, so the wall-clock cost is identical across all three:

// LoginService.AuthenticateLocal — fake-verify to keep timing consistent
if (coreUser is null)
{
    try { _ = BCrypt.Net.BCrypt.Verify(password ?? "", FakeBcryptHash); } catch { }
    _securityLogger.LogWarning("Login denied: user '{Username}' — invalid credentials or not found", username);
    return LoginResult.Failed("Invalid credentials.");
}

Password policy — NIST SP 800-63B / OWASP ASVS 4.0.3 §2.1, enforced before hashing:

Setting	Default	Configurable
`MinLength`	12	✅
`RequireDigit` / `RequireUppercase` / `RequireLowercase`	true	✅
`RequireSpecial`	false	✅
`HistoryCount` (can't reuse the last N)	5	✅
`MaxAge` (force rotation)	90 days	✅
`BreachCheckEnabled` (haveibeenpwned probe)	false	✅

HistoryCount lives in PropsPasswordHistoryStore — SHA-256 hashed, peppered prior-password digests in REDB, which closes the "change the password, come back an hour later to the old one" gap. demo_password_change_negatives.ps1 runs all 13 negative paths.

Audit — "anywhere," proven by integration tests

Audit is first-class, not a bolt-on. 116 typed events across 9 categories, single source of truth in IdentityAuditEventIds. Each event lands in a relational identity_audit_log table (user_id BIGINT, indexed — a direct integer seek, not a scan) and, if configured, multicasts to external sinks. Plaintext secrets are never written — a rotation logs a ClientSecretRotated marker only.

Category	Count	Examples
`authentication`	4	`UserLoggedIn`, `LoginFailed`, `PasswordChanged`
`authorization`	14	`TokenIssued`/`Revoked`/`Introspected`, `ConsentGranted`, `DpopReplayDetected`
`admin`	22	`Client`, `Scope`, `User`, `Group`, `ClaimMapper*`
`federation`	9	`FederatedUserLoggedIn`, `FederationStateValidationFailed`, `FederatedEmailConflict`
`mfa`	11	`MfaEnrolled`, `MfaVerifyFailed`, `MfaWebAuthnSignCounterAnomaly`
`scim`	9	`ScimUserCreated`/`Replaced`/`Patched`, `ScimBulkProcessed`
`system`	9	`SessionRevoked`, `SidRevoked`, `TokensRevokedByUser`

First candidates for a SIEM: LoginFailed, MfaVerifyFailed, MfaWebAuthnSignCounterAnomaly, DpopReplayDetected, FederationStateValidationFailed, ClientSecretRotated, AllSessionsRevoked.

The mechanism: a WireTap on direct-vm

Every mutating action (login, token issue, scope grant, MFA enroll, federation link, user delete, …) goes through EventDispatchProcessor, which writes a typed IdentityEvent into exchange.Out.Body + headers. From there it's a standard redb.Route WireTap:

WithRedbTx(From(IdentityEndpoints.Token))
    .Process(...)
    .WireTap(IdentityEndpoints.Events);   // ← a copy branches off to the audit pipeline

The identity-events route fans out to any number of sinks at once — without touching identity code, it's route config.

What the tests actually prove

"Anywhere" isn't marketing. Nine different audit targets each have their own integration test in redb.Identity.Tests/Audit/, run against real containerized brokers in the dev compose stack:

Sink	Test	What it checks
SQL (Postgres / MS SQL table)	`AuditSqlIntegrationTests`	event lands with the right columns, parameter binding via the redb.Route SQL connector
PROPS store (typed storage in the same REDB)	`AuditEavPersistenceIntegrationTests`	`IdentityEvent` saved as `RedbObject<AuditEventProps>` next to identity data — queryable
Kafka	`AuditKafkaIntegrationTests`	publish to a topic, partition key from user_id / event_type, acks=all
RabbitMQ	`AuditRabbitMqIntegrationTests`	publish to an exchange, routing key from event-type, durable queue with manual ack
IBM MQ / WMQ	`AuditIbmMqIntegrationTests`	publish to an MQQueue, MQMD with correlation-id, transactional context
AMQP 1.0	`AuditAmqpIntegrationTests`	publish to a standard AMQP node, properties + application-properties + body
MQTT	`AuditMqttIntegrationTests`	publish to a topic with QoS 1+, retained flag (audit stream for IoT gateways)
Elasticsearch	`AuditElasticsearchIntegrationTests`	bulk indexing into `identity-events-*`, ready for Kibana
Redis	`AuditRedisIntegrationTests`	publish to pub/sub (live firehose) + optional append to a Stream (replayable history)

Plus AuditEventSinkProcessorTests (unit tests on the fan-out processor) and AuditCompletenessFullCycleTests in FullStack/ — a full-cycle run that performs N mutating actions over HTTP and then checks that every one landed in the configured sinks. So "audit goes anywhere" means CI runs 8 real sinks on every commit (Kafka, RabbitMQ, IBM MQ, AMQP, MQTT, Elasticsearch, Redis, SQL) and verifies events serialize and arrive. Not an architecture slide — a green build against real infrastructure.

Real-world fan-out

From(IdentityEndpoints.Events)
    .Marshal(...)
    .Multicast(
        To("postgres://?table=identity_audit"),                  // durable retention
        To("kafka://broker:9092/identity-events?acks=all"),      // real-time SIEM stream
        To("elasticsearch://logs/identity-events?bulk=true"),    // Kibana ops dashboard
        To("wmq://QM.PROD?queue=COMPLIANCE.IDENTITY.AUDIT"),      // a bank's compliance core
        To("file:///var/log/identity/audit-${date:yyyy-MM-dd}.jsonl?append=true")  // locked-down fallback
    );

One identity server, one event source (direct-vm://identity-events), five parallel sinks of different natures. No identity server on the market does this out of the box — it's either "pick one sink in config" or "a webhook to your HTTP endpoint, do the fan-out yourself." This is transport-agnosticism at the event level: IBM MQ for banks that can't leave WMQ on compliance grounds, RabbitMQ for general-purpose, Kafka for streaming analytics — Identity uses the same transport set as any other business route.

Observability

Metrics ride the standard OpenTelemetry meter RedbIdentity — plug it into any OTel pipeline:

builder.Services.AddOpenTelemetry()
    .WithMetrics(m => m.AddMeter("RedbIdentity"));

Login/failure counters (tagged reason), MFA verifications (method, result), tokens issued (grant_type, token_type), token errors, rate-limit rejections, and a password-verify duration histogram that catches CPU regressions on hashing. Plus a dedicated security log channel (RedbIdentity.Security) so a SIEM can subscribe to audit-grade events without sifting routine logs, and module health probes (db, signing-keys, data-protection) under Tsak's aggregated /api/health/{startup,live,ready}.

Logical-level backup via redb.Export — no `pg_dump`, no vendor lock

The section that usually gets left for later and then suddenly becomes a problem. Most identity servers have one of two backup stories: pg_dump/mssqldump (a binary snapshot — you can't restore a subset, it's pinned to the DB version, and Postgres → MSSQL migration is off the table) or an application-level export through an admin API (usually shallow, no FK order, doesn't cover custom schemes).

There's a separate package — redb.Export — that does a logical-level dump into a portable .redb format:

JSONL stream (newline-delimited JSON), optionally zipped.
Foreign-key-safe order — types → roles → users → user_roles → lists → list_items → schemes → structures → objects → permissions → values. Single-pass restore.
Scheme-subset filter — --schemes UserProps,ApplicationProps. You can restore a subset too.
Per-type counts — the header knows exactly how many rows of each type were exported; the footer asserts it.
Multi-provider — an IDataProvider for Postgres / MSSQL / SQLite. One .redb file is portable across engines; dialect differences are encapsulated in the provider.
dryRun — count what would be exported without writing to disk.

A production example — a daily cron backup through redb.Route

From a real project. TsumBackupRouteBuilder.cs:

public class TsumBackupRouteBuilder : RouteBuilder
{
    protected override void Configure()
    {
        var backupConfig = Context?.GetProperty<IDictionary<string, object?>>("Backup");
        var directory = backupConfig?.TryGetValue("Directory", out var dir) == true
            ? dir?.ToString() ?? "backups" : "backups";
        var retentionDays = backupConfig?.TryGetValue("RetentionDays", out var ret) == true
            && int.TryParse(ret?.ToString(), out var rd) ? rd : 7;

        Context!.SetProperty("_backup.directory", directory);
        Context.SetProperty("_backup.retentionDays", retentionDays);

        From("cron://tsum-backup?schedule=0 0 3 * * ?")   // ← Quartz cron, daily at 03:00
            .RouteId("tsum-backup-cron")
            .ProcessWithRedb(RunBackupAsync);              // ← a named-scope IRedbService is injected
    }

    private async Task RunBackupAsync(IRedbService redb, IExchange exchange, CancellationToken ct)
    {
        var pgConn = redb.Configuration.ConnectionString;
        var directory = Context!.GetProperty<string>("_backup.directory")!;
        var retentionDays = Context.GetProperty<int>("_backup.retentionDays");

        Directory.CreateDirectory(directory);
        var timestamp = DateTime.UtcNow.ToString("yyyy-MM-dd_HHmmss");
        var filePath = Path.Combine(directory, $"tsum_backup_{timestamp}.redb");

        var provider = ProviderFactory.Create("postgres");
        await provider.OpenAsync(pgConn, ct);
        var exportService = new ExportService(provider, verbose: false, batchSize: 10000);
        await exportService.ExportAsync(filePath, schemeIds: null, compress: true, dryRun: false, ct);
        await provider.DisposeAsync();

        RotateBackups(directory, retentionDays);
    }

    private void RotateBackups(string directory, int retentionDays)
    {
        var cutoff = DateTime.UtcNow.AddDays(-retentionDays);
        foreach (var file in Directory.GetFiles(directory, "tsum_backup_*.redb")
            .Select(f => new FileInfo(f))
            .Where(f => f.CreationTimeUtc < cutoff))
        {
            file.Delete();
        }
    }
}

No separate backup service, no Bash cron job, no separate deployment: one From("cron://...") (Quartz is already built into the route system), .ProcessWithRedb(...) injects a named-scope IRedbService, ExportService.ExportAsync(..., compress: true, ...) turns on compression by argument, and RotateBackups is a plain date-based retention sweep. The file drops into the project's domain with a single context.AddRoutes(new TsumBackupRouteBuilder()). Identity gets the same backup for free — because Identity's data lives in the same REDB.

Restore and cross-provider migration

var importService = new ImportService(provider, verbose: false, batchSize: 10000);
await importService.ImportAsync("tsum_backup_2026-06-20_030000.redb",
    schemeIds: null,           // null = whole file; a subset also works
    truncateBefore: false,     // or true for a clean restore
    ct);

One of the killer moves is migrating the whole identity stack between engines without touching code, in three CLI commands:

# 1. Export from the old DB
redb export -p postgres -c "Host=src;Database=redb;..." -o data.redb --compress -v
# 2. Bootstrap the schema on the new DB
redb init -p mssql -c "Server=dst;Database=redb;..." -v
# 3. Import
redb import -p mssql -c "Server=dst;Database=redb;..." -i data.redb --clean -v

The .redb format is typed and cross-provider; dialect differences are encapsulated in IDataProvider. After the migration Identity sees the exact same data — the same OpenIddictApplication ids, the same user subs, the same signing keys. That's a level of portability IdentityServer (bound to its EF provider and migrations), Keycloak (Java SPI ↔ Postgres/MariaDB), and Auth0 (no on-prem at all) don't offer.

redb.CLI — a global .NET tool for the BAU operations

The same pipeline is on the command line via redb.CLI (on nuget.org):

dotnet tool install --global redb.CLI

Four commands cover the identity operational cycle:

redb init — bootstrap the REDB tables/sequences/functions/views in a clean DB (-p postgres|mssql|sqlite). The one-time analog of "migrations init"; after that the schema lives through the scheme registry.
redb schema — emit the SQL script for DBA review (redb schema -p postgres -o redb_schema.sql, or pipe into psql). Handy for change management (a PR flow), a CI/CD DDL artifact, or a compliance review without standing up a DB.
redb export — backup to .redb (--compress, --schemes 100,200,300, --dry-run).
redb import — restore from .redb (--clean to truncate first, --dry-run to check compatibility).

For identity that gives you: compliance retention (encrypt the .redb at the FS/KMS level, retention for N years), DR testing (restore into staging for a regular drill without pg_restore choreography), a forensic snapshot at incident time (one cron trigger over direct-vm:// and you have it), a scheme-subset export for a regulator, and cross-version migration. The kind of thing cloud IDPs charge for as an Enterprise add-on; here it's first-class.

Host-agnostic — Tsak is an option, not a dependency

A common question: "you keep mentioning hot-reload, a cluster, a dashboard — is all of that mandatory?" No. The identity server is a set of redb.Route routes; what hosts them is up to you.

Option 1 — a plain .NET worker. Your own Program.cs on Host.CreateApplicationBuilder:

var builder = Host.CreateApplicationBuilder(args);
builder.Services.AddRedbCore(builder.Configuration);
builder.Services.AddRedbRoute();
builder.Services.AddRedbIdentity(builder.Configuration);

var host = builder.Build();
var ctx = host.Services.GetRequiredService<IRouteContext>();
ctx.AddRoutes(new IdentityCoreRouteBuilder(/*...*/));
ctx.AddRoutes(new HttpFacadeRouteBuilder(/*...*/));
await ctx.Start();
await host.RunAsync();

dotnet run and the OIDC server is up. No runtime container, no dashboard, no .tpkg. Deploy it as a plain Worker Service / systemd unit / Windows service / Docker container.

Option 2 — inside an ASP.NET Core monolith. Identity embeds in the same process and reuses the host's IServiceProvider (DataProtection key ring, IConfiguration, ILogger) — no second initialization.

Option 3 — a service-mesh sidecar. Identity as an in-process compliance capsule inside each microservice; service A mints tenant-local tokens over direct-vm://identity-token with no network hop. No Tsak — services.AddRedbIdentity(...) is enough.

Option 4 — redb.Tsak for an enterprise control plane. When Identity lives in Tsak you get a management layer you'd otherwise hand-write:

What Tsak adds	For Identity
Hot-reload of one `.tpkg` without dropping in-flight messages in the other contexts	Update `redb.Identity.Core.Module.tpkg` (a new redirect_uri check) — Kafka consumers / the audit pipeline / federation callbacks keep running
Multiple named contexts in one process	Identity next to your `orders` / `payments` / `analytics` — each isolated via its own `AssemblyLoadContext`
Blazor Server dashboard + REST API	An operator sees every identity route, per-route metrics, ring-buffer logs, and can stop one route (say `/connect/register` under attack) without touching the rest
A CLI (`tsak route stop`, `tsak context stop orders`, …)	CI/CD without a custom kubectl wrapper
REST endpoints (`/api/contexts`, `/api/modules`, `/api/routes`, `/api/cluster`, `/api/logs`, `/api/watchdog`, `/api/scheduler`)	Your own ops portal via the typed `ITsakApiClient`
Cluster mode with leader election + context redistribution	A replica dies — Tsak redirects its contexts to healthy nodes
A Quartz `IScheduler` in every context	Token cleanup, session expiry, JWKS auto-rotation, audit retention on a standard cron
A watchdog that spots hung routes	A federation callback stuck at 30s gets restarted — no manual ops page at 3 a.m.
Prometheus `/metrics` + OTLP/Jaeger traces	Per-token-issue p50/p95/p99 + per-route error rate in Grafana, traces in Jaeger
API key + HMAC-SHA256 + roles + expiry + revocation for the admin API	Identity admin endpoints (e.g. `/signing-keys/rotate`) sit behind the same auth layer as route management
Five-layer config hot-reload	Change `Identity.RateLimit.PerIpPerMinute` in `context.json` — picked up, no restart

Tsak earns its keep when you have dozens of independent pipelines and Identity is one of them; when an ops team wants a browser dashboard over kubectl exec; when Identity runs on 3+ nodes and you want automatic redistribution; when you need hot-reload without downtime windows; or when you want to drop in your own extensions (IModuleProvider, IRouteLifecycleListener, a watchdog strategy — plain interfaces in redb.Tsak.Core). It's overkill when it's one server you can restart, an ASP.NET monolith with Identity inside, or a single sidecar microservice.

The identity stack doesn't know Tsak exists — there's not one using redb.Tsak in redb.Identity.Core. The deployment spectrum:

┌── Option A ── A 30-line Program.cs (junior-friendly) ─────────────────────┐
│   Worker Service + AddRedbIdentity + AddRoutes. Deploy via systemd / Docker.│
├── Option B ── An ASP.NET Core monolith with Identity inside ──────────────┤
│   One process, Identity next to your controllers.                          │
├── Option C ── A sidecar in each microservice ─────────────────────────────┤
│   In-process embedding — direct-vm token issue, no network hop.            │
├── Option D ── Tsak with a single identity module ─────────────────────────┤
│   Hot-reload, dashboard, REST/CLI. No cluster mode, no dozens of modules.  │
├── Option E ── Tsak full enterprise: Identity + your 20 other modules ─────┤
│   Cluster + leader election + Prometheus + Jaeger + Grafana + k8s.         │
│   Your own custom Tsak extensions (IModuleProvider, watchdog, ...).        │
└────────────────────────────────────────────────────────────────────────────┘
                  ⬇️ Same identity server at every level ⬇️

Extending alongside Identity — your own scheme, your own routes

The identity server isn't a closed box — it lives in the same REDB and the same IRouteContext as any business route of yours. So you declare a typed props class, register the scheme next to the identity schemes (no ALTER TABLE), get full CRUD + LINQ through the same IRedbService, and WireTap your routes onto identity-events:

// Your module next to Identity — no permission to ask for
public sealed class OrgEnrollmentProps
{
    public long UserId { get; set; }            // FK to an identity user via _users._id
    public string OrganisationCode { get; set; } = "";
    public DateTimeOffset EnrolledAt { get; set; }
    public string Role { get; set; } = "";
    public Dictionary<string, string>? Tags { get; set; }
}

services.AddRedbScheme<OrgEnrollmentProps>();

var enrollments = await redb.Query<OrgEnrollmentProps>()
    .Where(e => e.OrganisationCode == "ACME")
    .WhereRedb(o => o.Key == userId)
    .ToListAsync();

From(IdentityEndpoints.Events)
    .Filter(e => e.GetHeader<string>("event-type") == "UserCreated")
    .Process((e, ct) => /* your onboarding flow */)
    .To("rabbitmq://org-enrollment-pipeline");

No extension points, no SPI, no plugin manifests, no rebuild of the identity server — it's ordinary C# in an ordinary assembly that co-exists with the identity stack in one process and one database. Drop it into Tsak as its own .tpkg (deploys independently) or compile it into the same worker.

An addon calls the identity server from the inside over direct-vm — no HTTP, one transaction

This is stronger than a WireTap. The identity routes are published as direct-vm://identity-token, direct-vm://identity-authorize, direct-vm://identity-users-delete, … — all the standard endpoint URIs (IdentityEndpoints.Token, .Authorize, …, the full list in IdentityEndpoints.cs). So an addon calls Identity like its own library — via message passing, but inside one process with no network hop.

Example: the addon enrolls a user in an organization and mints their onboarding token through the identity server, all in one transaction:

public sealed class OrgEnrollmentRoutes : RouteBuilder
{
    protected override void Configure()
    {
        WithRedbTx(From("direct-vm://org-enrollment-create-with-token"))
            .ProcessWithRedb(async (redb, e, ct) =>
            {
                // 1. Create the enrollment record in OUR own scheme
                var enrollment = new RedbObject<OrgEnrollmentProps>(new OrgEnrollmentProps
                {
                    UserId = e.GetHeader<long>("user_id"),
                    OrganisationCode = "ACME",
                    EnrolledAt = DateTimeOffset.UtcNow,
                    Role = "member",
                });
                await redb.SaveAsync(enrollment, ct);
            })
            // 2. ↓↓↓ CALL THE IDENTITY SERVER over direct-vm — NO HTTP ↓↓↓
            .Process((e, ct) =>
            {
                e.In.Headers["grant_type"] = "client_credentials";
                e.In.Headers["client_id"] = "org-onboarding-bot";
                e.In.Headers["scope"] = "openid org:onboarding";
                return Task.CompletedTask;
            })
            .To(IdentityEndpoints.Token)  // ← "direct-vm://identity-token" — Identity in the same process
            // ↑↑↑ access_token is already in e.Out.Body, no HTTP round-trip happened ↑↑↑
            .Process(async (e, ct) =>
            {
                var tokenResponse = e.Out!.Body as Dictionary<string, object?>;
                var accessToken = tokenResponse!["access_token"]?.ToString();
                await _emailSender.SendOnboardingAsync(/* userId */, accessToken!, ct);
            });
    }
}

What happened: one route does the enrollment in its own scheme + a token issue in the identity stack + an email notification, in sequence, in a single exchange chain; WithRedbTx(...) wraps all of it — both the save into OrgEnrollmentProps and Identity's token issue (which also writes to _objects via RedbTokenStore) — one redb tx, one atomic commit; no HttpClient, no JSON serialization, no TLS handshake; and the same correlation id flows through, so Identity logs the token issue under the same trace id as the enrollment. Any direct-vm endpoint is available to the addon as an in-process call — not "SDK wrappers," the endpoints themselves.

For contrast: to mint a token from Java next to Keycloak you have to write a wrapper (HttpClient → form-data → HTTP → JSON back → retry/circuit-breaker → SDK classes → keep the SDK in sync with the Keycloak version). Every wrapper is a failure point, a serialization point, a separate-transaction point (atomic semantics between "minted a token" and "wrote to my DB" are impossible), a version-drift point, and a separate-test point. Here it's one line, .To(IdentityEndpoints.Token), and an atomic commit via WithRedbTx. That gap is exactly why people write their own identity servers in the first place: the boxed ones won't let you use identity as a library next to your business logic.

Your own facade on your own protocol

The HTTP facade is just one connector over the direct-vm core. Nothing stops you from putting your facade in front of a corporate/proprietary protocol. The core doesn't care — it receives messages over direct-vm://, and how they got there is your business:

Scenario	Solution
Corporate binary RPC (custom framing over TCP, cert pinning)	`From("custom-rpc://0.0.0.0:9999")` → parse the frame → `To(IdentityEndpoints.Token)`
A gRPC facade	`From("grpc://.../oidc.IdentityService/IssueToken")` → map to headers → `To(IdentityEndpoints.Token)`
MQTT for IoT (devices that can't do HTTP/2)	`From("mqtt://broker:1883/device/+/token-request")` → forward into the device_code grant
An internal enterprise bus (WMQ as the only transport in a bank's perimeter)	`From("wmq://QM.PROD?queue=IDENTITY.TOKEN.REQ")` → forward → reply on `IDENTITY.TOKEN.RESP`
A crypto wrapper (token requests in a custom envelope)	`.Process(DecryptEnvelope)` → `To(IdentityEndpoints.Token)` → `.Process(EncryptEnvelope)`

Any redb.Route connector can front the identity server. Want tokens minted from a RabbitMQ queue? One RouteBuilder. A Slack bot as the identity admin API? One RouteBuilder.

A separate control facade for admin endpoints on a public perimeter

A scenario people rarely spell out: the identity server has to sit on a public IP (cloud edge, partner API, multi-region). You can't blanket it in a VPN/mTLS — real RP clients come in over HTTPS from browsers and mobile apps, an external federation IdP calls back to your /connect/federation/callback, and back-channel logout needs a publicly resolvable URL. So the standard OIDC endpoints (/connect/*) have to be public.

But the critical admin endpoints — /signing-keys/rotate, /users/{id}/force-revoke, /internal/bootstrap-admin, application mutations — don't have to be HTTP. You can expose them through a separate facade over your own proprietary TCP framing with custom magic bytes, a protocol version, non-standard markers — on a separate port, on a separate network:

// A control facade over a proprietary protocol — separate port, separate network
From("custom-tcp://0.0.0.0:7891")
    .RouteId("identity-admin-proprietary-facade")
    .Process(ValidateMagicBytes)       // ← a client that doesn't know the format never gets in
    .Process(ParseCustomFraming)       // ← binary format with CRC + version + opcode
    .Process(VerifyProprietaryAuthHmac)// ← your own HMAC scheme, not an RFC
    .Choice()
        .When(e => e.GetHeader<int>("opcode") == 0x42).To(IdentityEndpoints.SigningKeyRotate)
        .When(e => e.GetHeader<int>("opcode") == 0x43).To(IdentityEndpoints.UserForceRevoke)
        .When(e => e.GetHeader<int>("opcode") == 0x44).To(IdentityEndpoints.BootstrapAdmin)
    .EndChoice()
    .Process(SerializeCustomResponse);

This is not a substitute for crypto — auth + HMAC + a role check are on every opcode. It's an extra layer of defense-in-depth that works not on crypto but on reduced attack surface and information asymmetry: off-the-shelf toolkits (Burp, sqlmap, msf) and botnet scanners don't understand custom framing and don't scan custom TCP protocols at all; an attacker has to reverse-engineer the format before they can even form a syntactically valid request. Every day of that delay is a day for your security team to detect and respond. It's a pattern banks, defense, and the public sector have long used (their own RDBMS protocols, their own serialization on service-to-service channels); it used to require rewriting the stack around a proprietary protocol, and now it's one RouteBuilder in front of a standard OIDC core. Standard RPs keep talking /connect/* over HTTPS — nothing changes for them.

Extension points — everywhere, not just Identity

The freedom to extend is a property of the whole ecosystem:

Layer	Extension points
REDB storage	`IRedbObjectStorageProvider` (a different backend under `_objects`), `ISqlDialect`, `IPasswordHasher`, `IPropsSaveStrategy`
redb.Route	your own connector (`From("myprotocol://...")`), `IProcessor`, a custom EIP, `IDataFormatRegistry`, `IExpression`
redb.Identity	`IExternalUserProvider`, `IPasswordHasher`, `IPasswordBreachChecker`, `IRateLimitStore`, `IEmailNotificationChannel`, `IBackgroundDeletionService`, `IClaimMapper` — plus any custom RouteBuilder as a peer route
redb.Tsak	`IModuleProvider`, `IRouteLifecycleListener`, `IWatchdogStrategy`, `ITsakAuthProvider`
redb.Route.Controllers	`IControllerDispatcher`, `IActionResultMapper`

Same shape everywhere: a public interface with a clear contract + DI registration + your implementation. Register yours and Identity picks it up without noticing the swap. No SPI manifests, no vendor lock-in moments.

Combinatorics with the LLM connector

Put an LLM connector next to Identity (redb.Route.Llm — Anthropic / OpenAI / Azure OpenAI), a Kafka stream of identity events, and your own domain schemes. Passwords stay Argon2id-hashed in the DB the whole time — the LLM never sees plaintext (the prompt gets an IdentityEvent: event-type, user-id, client-id, ip-address, timestamps; sensitive material is filtered in EventDispatchProcessor before the WireTap). What you get is another .To() / .Filter() / .Process() in your RouteBuilder, not a separate product behind an extra license:

Scenario	Pipeline
AI anomaly detector on login patterns	`From(IdentityEvents).Filter(type=="LoginSuccess").To("llm://anthropic?prompt=...")` → risk score → `direct-vm://identity-sessions-revoke` past a threshold
Real-time fraud scoring on token issue	`From(IdentityEndpoints.Token).WireTap(To("llm://..."))` → score > threshold → revoke + flag-for-review
Conversational admin via Telegram/Slack	`From("telegram://admin-bot")` → an LLM agent with `.AsLlmTool()` wrappers over admin endpoints → "revoke all sessions for foo@acme.com" in one message
LLM classification of audit events	`From(IdentityEvents).To("llm://claude?prompt=Classify severity...")` → `To("kafka://security-incidents-prioritised")`
Auto compliance reports per quarter	`From("cron://quarterly-report")` → `To("postgres://identity_audit?query=...")` → LLM aggregation → `.To("email://compliance@org.com")`
LLM validation of a redirect_uri at DCR	a new OIDC client → `.To("llm://...?prompt=Is this redirect_uri benign?")` → human-in-the-loop for the suspicious ones

You already have Identity with direct-vm endpoints, the LLM connector with .AsLlmTool(), Kafka/RabbitMQ for streaming, Tsak for deployment, and your own RouteBuilders — wiring them into pipelines is a couple of hours, not a quarter-long epic.

Enterprise scale — thousands of microservices, hundreds of thousands of users

Those features (a transport-agnostic core, schemes as data, cluster-safe primitives, live JWKS refresh) exist because a modern enterprise looks like this: thousands of microservices talking client_credentials and token-exchange delegation chains; hundreds of thousands of live sessions (mobile + web + thick clients) with concurrent federation round-trips; several datacenters running identity replicas behind an LB; and regulatory rules that forbid identity data from leaving the perimeter.

Service-to-service auth — without the round-trip

The classic shape: service-A → POST /connect/token → identity (HTTP), then service-B → POST /connect/introspect → identity (HTTP). That's two network hops to Identity on every internal call — at 10K internal RPS the identity server becomes a bottleneck, a single point of failure, and p99 tax. Here:

Embedding mode. redb.Identity.Core links into each microservice that has to mint/validate tokens; client_credentials issue goes over direct-vm://identity-token — zero network hop.
JWT validation without introspect. Resource servers validate the bearer locally via redb.Identity.Resource.Dpop (or the vanilla JwtSecurityTokenHandler for non-DPoP cases). The JWKS is fetched through IConfigurationManager<OpenIdConnectConfiguration> with a long TTL — a network call once an hour, not per bearer.
Token exchange (RFC 8693) for delegation chains. Service A takes the user's token and calls service B on their behalf; service B sees the act claim chain (RFC 8693 §4.1) with the delegation history. demo_token_exchange.ps1 covers it.

Cluster stability — for many-thousand sessions

Identity is built for N replicas where leadership rotates with no external coordination:

Concern	Mechanism
Stateless replicas	all handlers are Scoped, no static state — any replica serves any request
DataProtection key ring across replicas	`RedbXmlRepository` — keys in redb; each replica refreshes its snapshot every `XmlRepositoryRefreshInterval` (60s). Not cluster-gated — every node must catch keys others rotated
OAuth signing keys across replicas	redb-backed RSA 2048 PEMs, DataProtection-encrypted, bootstrapped under a distributed lock (one replica mints the first key)
Federation nonce / rate-limit counters	a `Redis` backend, cluster-wide (RFC 6749 §10.12 / RFC 6585 §4)
Schema init	a cluster-wide lock via `LockForUpdate` — leader initializes, followers wait
Background trash purge	`TryClaimOrphanedTaskAsync` — any replica picks up an orphan after another crashes
Signing key rotation	`IOptionsMonitorCache.TryRemove` invalidates the cache on every replica
Back-channel logout across replicas	`/revoked-sids/add` writes the revocation; `/revoked-sids/since?cursor=` lets each RP replica pull deltas. Push-and-poll survives lost nodes and network splits

No ZooKeeper, no separate store for keys — it's all redb objects. In a three-node cluster around ~5K /token RPS the stack runs at ~50–80ms p99 (the Pro path with PVT indexes on value_string for the client_id lookup).

Data sovereignty — identity stays inside the perimeter

Self-hosted means the identity data physically lives on your hardware. No callback to a third-party processor, no shipping emails/phones to a cloud vendor. That matters for banks (KYC on a regulated network), health/medical (HIPAA / GDPR Article 9), the public sector (data residency), and B2B SaaS on-prem (the customer deploys into their own VPC). redb.Identity doesn't phone home — no telemetry, no pings, no update checks. One Postgres + one .NET worker + your network.

Multi-tenant through schemes

Classic OIDC multi-tenancy — /{tenant}/connect/token, separate DB schemas, separate key rings — needs tenant routing in every handler, tenant-aware migrations, and per-tenant key rotation. Here isolation is possible at the REDB scheme level: the split is physical, through _id_scheme, and a query from tenant A can't touch tenant B's data even through a bad filter. Today it's a capability, not a packaged feature: production deployments usually start single-tenant and layer multi-tenancy on via a tenantId claim — we'll prioritize it on inbound demand.

Live demos: 61 probes against a real server

Beyond the xUnit suite, demos/ holds 61 self-contained PowerShell probes that drive a live server over its real HTTP surface — one demo_*.ps1 per protocol contract. Each sets up its own client via DCR, runs the flow end-to-end, and asserts the wire-level result (status codes, headers, token/JWKS shape, replay rejection). Executable RFC docs and a black-box regression net in one.

cd redb.Identity/demos
pwsh -File .\run_all.ps1                 # every demo_*.ps1 in canonical order
pwsh -File .\run_all.ps1 -Only mfa       # only the MFA ones
pwsh -File .\run_all.ps1 -StopOnFail     # bail on the first failure

run_all.ps1 runs each probe in its own pwsh child process (a hard failure in one can't take the rest down), streams output, captures a transcript to demos/_logs/, and prints a pass/fail table. The map by category:

Grant types        demo_client_credentials / demo_authcode_pkce / demo_refresh_rotation
                   demo_device_code (+_ci) / demo_password_ropc / demo_token_exchange
Endpoints          demo_discovery_jwks / demo_discovery_shape / demo_introspect_revoke
                   demo_userinfo / demo_jwks_rotation
Authorize surface  demo_auth_extras (RFC 9207 iss, form_post) / demo_prompt_max_age
                   demo_acr_values / demo_claim_probes
PAR / DPoP         demo_par / demo_par_per_client / demo_dpop
Logout             demo_logout_endsession / demo_backchannel_logout
DCR                demo_dcr_lifecycle / demo_private_key_jwt
SCIM 2.0           demo_scim / demo_scim_bulk / demo_scim_etag
MFA                demo_mfa_totp / demo_mfa_recovery_codes / demo_mfa_disable_replace
                   demo_password_change_negatives
Self-service       demo_account_register_verify / demo_me_profile / demo_me_email_change
                   demo_me_sessions / demo_me_delete / demo_password_reset
Federation         demo_federation / demo_federation_e2e / demo_federation_github
                   demo_federation_link_unlink
Admin              demo_admin_scopes / demo_groups_roles_claims / demo_sessions_admin
                   demo_throttle_rfc6585 / demo_jwt

The only probe that needs a human is device-code consent — there's a non-interactive demo_device_code_ci.ps1 for CI.

Reference BFF + Blazor admin, included

redb.Identity.Web is a reference Blazor Server app that demonstrates the HARDLINE BFF pattern:

references only redb.Identity.Contracts + redb.Identity.Client — no Core, no Http;
every Identity call goes through the typed IIdentityClient — never a raw HttpClient;
OIDC Authorization Code + PKCE, SaveTokens=true, back-channel-logout sink + poll fallback for multi-replica RPs.

Pages cover both self-service (/Me/*: profile, password, MFA/TOTP, WebAuthn credentials, sessions, federated links, consents) and admin (/Admin/*: users, groups-as-tree, applications, scopes, claim mappers, roles, federation providers, sessions, tokens, audit with a JsonViewer, SCIM browser, settings). It ships as source in the repo, not a package — clone it and run it, or lift it as a starting point.

And a typed SDK for anyone calling Identity from the outside:

services.AddIdentityClient(o =>
{
    o.BaseUrl = "https://identity.local";
    o.AccessTokenProvider = new ClientCredentialsAccessTokenProvider(
        clientId: "my-svc", clientSecret: cfg["IdentitySvcSecret"]!, scopes: ["identity:manage"]);
});

public class UserSync(IIdentityClient identity)
{
    public async Task RunAsync(CancellationToken ct)
    {
        var page = await identity.Users.ListAsync(new UsersListRequest { Limit = 100 }, ct);
        foreach (var u in page.Items) await PersistAsync(u, ct);

        // back-channel revocations — pull deltas since the last cursor
        var since = await identity.RevokedSids.GetSinceAsync(_cursor, ct);
        foreach (var e in since.Entries) _cache.Apply(e);
        _cursor = since.NextCursor;
    }
}

Every HTTP error is normalized to RFC 7807 ProblemDetails and surfaces as ApiException.

A little honest comparison

	ASP.NET-bound IS (Duende, OpenIddict samples)	redb.Identity	Standalone IAM (Keycloak / Auth0)
Call `token` from another in-process module	Loopback HTTP	`To("direct-vm://identity-token")` — same exchange, zero-copy	Network hop
Add a transport (gRPC, MQ, SignalR)	Rewrite endpoints as gRPC services / consumers	Drop a facade `.tpkg`, point it at `direct-vm://`	Vendor adapter (if it exists)
Replace HTTP with RabbitMQ entirely	Major refactor	Drop the HTTP facade. Keep Core.	Not possible
Storage	Custom EF schema + migrations	redb `RedbObject<TProps>` — code-first, no migrations	Vendor schema, vendor migrations
Custom claims	`jsonb` blob + your own indexes	`Dictionary` on props — natively queryable	Vendor attribute model
Multi-replica key sharing	DIY DataProtection + JWKS	Built-in: redb key-ring + signing key store	Built-in (vendor-specific)
Embed in your worker as a library	No	Yes — one or two `.tpkg`, or call Core directly	No
DB providers	One per build	PG / MSSQL / SQLite from one codebase	Vendor
An addon as a peer in the same DB/transaction	Controllers alongside, but storage is separate (EF)	Own scheme + direct-vm calls + one redb tx	Only from outside, over REST
License	Mixed (Duende: commercial)	Apache 2.0	Mixed

Two honest caveats so this doesn't read as a hit piece. Keycloak/Auth0 are finished products with admin UX, multi-tenancy, and an ecosystem we don't reproduce one-for-one; if you want "install the box and stop thinking," they're still a valid pick. Duende IdentityServer is the most mature .NET stack out there with a huge community — the argument isn't "it's bad," it's "it's architecturally married to the HTTP pipeline, and that isn't always what you want." On social providers out of the box (Sign in with Apple, LinkedIn, X) Auth0/Okta are ahead — we ship generic OIDC + GitHub OAuth2, and the rest you write as federation providers (there's a mock provider in demo_federation_e2e.ps1 showing how). That's an honest limit, not a blocker. redb.Identity wins precisely where identity should be a built-in part of your .NET system rather than a service next door, and where you'd rather not drag along migrations and a second storage model.

What isn't done yet (and that's fine)

This list is honest because it's checkable: everything absent from it, our own /.well-known/openid-configuration will confirm or refute in a single curl.

Not implemented: RFC 8705 (mTLS + cert-bound tokens — tls_client_certificate_bound_access_tokens: false), 9101 (JAR — request objects aren't consumed; request= gets request_not_supported), 8707 (Resource Indicators), 9396 (RAR), 9470 (step-up), FAPI 2.0, CIBA, OIDC Federation 1.0. From OIDC Core: no pairwise sub (public only) and, per OAuth 2.1, no implicit and no hybrid flow.

Two things we deliberately chose not to build — and the reasoning matters more than the checkbox:

Front-Channel Logout 1.0. It signs RPs out through an iframe to each client, which means it rides on third-party cookies. Safari's ITP blocks them; Chrome is burying them. The mechanism is broken by design in a modern browser, and anyone ticking that box owes you a footnote saying "works sometimes." We ship Back-Channel Logout instead — server to server, signed logout token, plus a pull feed of revoked sids for multi-replica RPs. It is strictly better and doesn't depend on cookies at all. We'd rather have the mechanism that works than the checkbox.

HOTP (RFC 4226) as a standalone method. Counter-based OTP is effectively unused in 2026; the world runs on TOTP. RFC 4226 lives on as the foundation of TOTP — and in that role we do implement it (160-bit secret per §4). Building a separate counter with a resync window purely to add a row to a table is work for the table, not for the user.

Planned facades (same .tpkg pattern, no Core changes): redb.Identity.Grpc, .Rmq/.Amqp/.IbmMq, .SignalR, .Kafka (event-only sink). PRs welcome.

Quick start

Fastest path — Docker (SQLite + HTTPS, out of the box). The image is a working OpenID provider with zero configuration:

docker run -d -p 5002:5002 ghcr.io/redbase-app/redb-identity-backend:latest
curl -k https://localhost:5002/.well-known/openid-configuration

It bundles a self-signed dev cert (CN=localhost) — call it with curl -k for a local trial. Variants: backend (OIDC only), managed (+ Tsak dashboard), full (+ BFF login/admin on :8087). Compose stacks and DOCKER.md live in publish/docker/. Images are cosign-signed.

Not a Docker fan? The releases page has a self-contained archive (Worker + modules, also SQLite + HTTPS out of the box, cosign-signed): unpack and run start-full.ps1 / start-full.sh.

For embedding in your own project — packages are on nuget.org. Take what you need — the host picks the provider:

# Core OIDC / OAuth 2.1 engine (OpenIddict on redb.Route) + a storage provider
dotnet add package redb.Identity.Core
dotnet add package redb.Postgres.Pro      # or redb.MSSql.Pro / redb.SQLite.Pro

# HTTP facade (OIDC + management + SCIM)
dotnet add package redb.Identity.Http

# Typed client SDK (IIdentityClient) — for BFFs / services
dotnet add package redb.Identity.Client

Or build the .tpkg modules from source and drop them into a Tsak worker:

cd redb.Identity
.\scripts\pack-tpkg.ps1   # → redb.Identity.Core.Module.tpkg + redb.Identity.Http.tpkg

A full dev setup with real infrastructure:

# 0. (optional) redb.CLI globally — for DB work from the terminal
dotnet tool install --global redb.CLI

# 1. Dev compose (Postgres + GreenMail + mock IdP + Redis + LDAP)
docker compose -f redb.Identity/deploy/observability/docker-compose.yml up -d

# 1.1. Initialize the REDB schema
redb init -p postgres -c "Host=localhost;Database=redb;Username=postgres;Password=postgres" -v

# 2. Run the worker with the tpkg modules
cd redb.Tsak/src/redb.Tsak.Worker
dotnet run

# 3. In another window — run a demo
cd redb.Identity/demos
pwsh demo_discovery_jwks.ps1     # OIDC discovery + JWKS + manual verification
pwsh demo_authcode_pkce.ps1      # full PKCE round-trip
pwsh demo_jwks_rotation.ps1      # runtime signing-key rotation
pwsh run_all.ps1                 # every probe

Check a live server:

curl http://localhost:5000/.well-known/openid-configuration | jq
curl http://localhost:5000/.well-known/jwks | jq

And the one-shot first-admin bootstrap (self-seals via a sentinel flag; a second call returns 410 Gone):

curl -X POST http://localhost:5000/internal/bootstrap-admin \
  -H "Content-Type: application/json" \
  -d '{ "username": "admin", "password": "...", "email": "admin@local" }'

The full README — every endpoint table, the 79-event audit catalogue, the OpenTelemetry metrics map — is in the repo. Issues and feedback via GitHub Issues and Discussions.

Wrapping up

redb.Identity is the fourth option that was missing. Not an ASP.NET-bound engine where every endpoint is an HTTP middleware. Not a standalone IAM box with its own runtime and deploy. And not "roll your own" for the third time.

Three things you don't get elsewhere in one package:

Protocol ≠ transport. Every endpoint is a direct-vm:// route. HTTP is the first facade, but the next module calls token with no network, zero-copy. Want another transport — drop a .tpkg. Want HTTP gone — drop the facade, keep Core.
No migrations. Users, applications, scopes are C# classes. Add a field and it's in prod the next instant. Custom claims are natively queryable, no jsonb, no hand-rolled indexes.
Identity as part of your project. Take just the engine, zero facades, and call OAuth endpoints with a method call from inside your process — down to an addon project that mints a token in the same transaction as its own domain data. An identity server as a library, not a service next door.

Plus: PostgreSQL / MSSQL / SQLite from one codebase (1767 tests green on all three), RFC numbers cited right in the code, DPoP / PAR / DCR / SCIM / MFA / WebAuthn / back-channel logout, live JWKS rotation with no restart, a password-hashing pipeline with Argon2id + upgrade-on-login, audit "anywhere" proven by 8 integration tests against real brokers, logical-level backup + cross-provider migration, a cluster with no external coordination, host-agnostic deployment from a 30-line worker to a Tsak cluster, and a reference Blazor admin — all Apache 2.0.

If you try it, tell me in the comments which embedding shape fits you: service next door, in-process library, or a full server behind HTTP. Deeper dives are coming in the series — WebAuthn/passkeys as a Route processor, DPoP step by step, SCIM sync with AD, MFA without a god-class.

Sources and release: github.com/redbase-app/redb-identity. About the redb database: redbase.app. Past posts on the profile.

If this was useful — a ⭐ on GitHub helps others find it.

The RabbitMQ connector in redb.Route: the whole broker in one URI — RPC, competing consumers, dead-lettering. Still leaving MassTransit

rinat kozin — Mon, 13 Jul 2026 15:07:01 +0000

We did Kafka earlier in this series. Now it's RabbitMQ's turn, and this one leans hard on how you actually use it. The redb.Route.RabbitMQ connector sits on top of the official RabbitMQ.Client 7.x, but you don't write a "client" — you write routes, and the whole broker collapses into a single URI:

From("rabbitmq://orders?exchange=shop&exchangeType=topic&routingKey=order.*&concurrentConsumers=4")
    .Log("Order in: ${body}")
    .To("direct://process");

Read it once and you already know the whole story: consume from queue orders, bound to the topic exchange shop on order.*, four consumers wide, log it, pass it on. No "open a channel," no "declare a binding," no "set the QoS" — the connector handles every bit of that.

About "leaving MassTransit," let me be straight with you: it's not that MassTransit can't do this — it can, RPC included (request/response via IRequestClient<T>). The difference is the model. MassTransit is a bus: message contracts, consumers, bus configuration — you buy into its whole worldview at once. redb.Route is explicit routes in the Apache Camel spirit: an endpoint is a URI, integration patterns are steps in a route, the transport is an abstraction underneath. Fewer ceremonies around contracts and registrations, more plain "from here → to there → and here's what happens along the way." Which one fits you is a matter of taste and of the job in front of you; this post is about what RabbitMQ looks like in the second model.

Here's the map: the anatomy of the URI, every parameter as a reference table (what it does and why), the properties the consumer and producer move to and from headers, competing consumers (concurrency), broadcasting across cluster nodes (a real production example), RPC (RabbitMQ makes this one especially clean), and the two EIP patterns that fit a broker like a glove — Filter (on compiled predicates) and Dead Letter Channel.

All the code is in English; the examples aren't invented — they live in redb.Route/demos/redb.Route.Demo/Routes, the same demo project that runs in CI.

Part of the redb / redb.Route series — recent posts first:

Leaving MassTransit for a Camel state of mind: the Kafka connector, Scatter-Gather, and transactions

Apache Camel for .NET, dissected: the HTTP connector with no ASP.NET MVC + the Content-Based Router

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL

Sources: github.com/redbase-app. About the database itself: redb.ru.

Anatomy of the URI

An endpoint is a string:

rabbitmq://queue-name?host=localhost&exchange=my-exchange&exchangeType=topic&routingKey=order.*

Scheme rabbitmq://, the URI host is the queue name, then query parameters (names are in the table below). The same string works as a consumer (in From(...)) and as a producer (in .To(...)) — the role is decided by where the endpoint sits in the route. You don't have to repeat the connection: register a connection factory once and point at it with connectionFactory=name — host, credentials and SSL live in one place, and the connection gets reused.

One detail worth pausing on: parameter values are templates. routingKey=order.new is a constant, but routingKey=order.${header.type} is dynamic — the key is computed per message from its header (the same mechanism as Kafka's partition key). So "a string" here doesn't mean "static only" — a ${...} inside the value turns it into an expression.

On the fluent builder. The connector also ships a builder, Rabbit.Queue("orders")…, but it's expression-first: the string-ish methods take an expression so a producer value can be computed per message (.RoutingKey(Header("order.type")) — a dynamic key, Kafka-style). For a plain constant there's now a string overload too — .Host("localhost") and .RoutingKey("order.new") just work, and ${...} still interpolates. In this post I use the URI form — for static config it's shorter and easier to read at a glance; the dynamic-via-expressions side gets its own writeup.

Every parameter

The reason to bookmark this post. Names are exactly as they appear in the URI; defaults are what you get out of the box.

Connection

Parameter	Default	What it's for
`host`	`localhost`	Broker host
`port`	`5672`	AMQP port
`username` / `password`	`guest` / `guest`	Credentials
`virtualHost`	`/`	Virtual host — resource isolation inside the broker
`connectionFactory`	—	Name of a factory in the registry: settings in one place, connection reused
`clientName`	`redb.Route`	Connection name — shows up in the management UI

Exchange

Parameter	Default	What it's for
`exchange`	(empty)	Exchange name; empty = the default exchange (routed by queue name)
`exchangeType`	`direct`	`direct` / `topic` / `fanout` / `headers`
`exchangeDurable`	`true`	Survives a broker restart
`exchangeAutoDelete`	`false`	Delete once it's no longer in use
`declare`	`false`	Create the exchange + queue on startup if they don't exist

Queue

Parameter	Default	What it's for
`queue`	(from the URI host)	Name; empty = an auto-generated exclusive queue
`durable`	`true`	Survives a restart
`autoDelete`	`false`	Delete once the last consumer leaves
`exclusive`	`false`	Only one consumer allowed

Routing and message

Parameter	Default	What it's for
`routingKey`	(empty)	Key for publish and bind
`contentType`	`application/json`	Message content type
`messageTtl`	`0`	Message TTL, ms (`x-message-ttl`)
`expires`	`0`	Queue is deleted after N ms with no consumers (`x-expires`)

Consumer

Parameter	Default	What it's for
`concurrentConsumers`	`1`	The one and only concurrency knob — up to N messages handled at once
`prefetchCount`	`10`	Prefetch per consumer; keep it ≥ `concurrentConsumers`
`autoAck`	`false`	`true` = at-most-once (no requeue on error); `false` = at-least-once
`transacted`	`false`	Transactional AMQP channel (mutually exclusive with `autoAck`)
`mandatory`	(auto)	Return an undeliverable message; defaults to `true` for direct/headers

RPC

Parameter	Default	What it's for
`replyTo`	`false`	Turn on request/reply: publish the request and wait for the correlated response
`timeout`	`60`	How long to wait for the reply, seconds
`maxOutstandingConfirms`	`2048`	Cap on unconfirmed publishes — so you don't trip over the client's rate limiter under load

Queue limits and dead-lettering

Parameter	Default	What it's for
`maxLength` / `maxLengthBytes`	`0`	Queue cap in messages / bytes (`x-max-length[-bytes]`)
`overflow`	—	On overflow: `drop-head` / `reject-publish` / `reject-publish-dlx`
`deadLetterExchange`	—	DLX — where dead messages go (`x-dead-letter-exchange`)
`deadLetterRoutingKey`	—	Routing key used when dead-lettering
`queueType`	—	`classic` / `quorum` (a replicated queue for a cluster)
`maxPriority`	`0`	Max priority for priority queues

Resilience and TLS

Parameter	Default	What it's for
`automaticRecovery`	`true`	Auto-recover the connection
`topologyRecoveryEnabled`	`true`	Re-declare exchanges/queues/binds after recovery
`recoveryInterval`	`5`	Retry interval for recovery, seconds
`heartbeat`	`60`	Heartbeat interval, seconds — broker and client ping each other so a dead connection gets noticed in time
`connectionTimeout`	`60`	Don't hang on an unreachable node
`ssl`	`false`	Turn on TLS
`sslServerName` / `sslCertPath` / `sslCertPassphrase`	—	SNI and a client cert for mTLS

The startup validator won't let you shoot yourself in the foot: prefetch/concurrentConsumers/timeout must be > 0, and autoAck + transacted together is flat-out rejected.

AMQP properties ↔ headers: a full round-trip

The connector maps a RabbitMQ message's properties to exchange headers both ways — on the consumer and on the producer. When the consumer takes a delivery, it spreads BasicProperties across the headers; when the producer sends, it reads them back and puts them on the outgoing message. So a consume → produce hop carries every AMQP property straight through, and you can set or override any of them by just placing a header.

Naming comes in two tiers.

Delivery metadata — there's no standard AMQP name for these, so they get the redbRmq. prefix:

Header	What's in it
`redbRmq.Exchange`	the exchange the message arrived on
`redbRmq.RoutingKey`	the delivery's routing key
`redbRmq.Redelivered`	redelivery flag (after a nack/requeue or a consumer crash)
`redbRmq.DeliveryTag`	delivery tag (used for the ack)
`redbRmq.ConsumerTag`	consumer tag

Standard message properties — under their well-known names, no prefix (everyone knows them already, and the round-trip carries them as-is):

Header	AMQP property
`CorrelationId`	correlation-id
`ReplyTo`	reply-to — the address to reply to
`MessageId`	message-id
`Priority`	priority 0–9
`Expiration`	message TTL, ms
`Type` / `AppId` / `UserId`	type / app / user
`ContentType` / `ContentEncoding`	content-type / encoding
`Timestamp`	timestamp
`Persistent`	persistent / transient

redbRmq.Redelivered earns its keep: use it to handle "this is already the second attempt" separately — send the redelivery down a different route instead of looping forever on requeue.

The ReplyTo header ≠ the replyTo parameter. Similar names, different things. The URI parameter replyTo=true switches the client into RPC mode (below). The ReplyTo header is the AMQP reply-to property itself: set it on a normal send and you're pointing the message's reply address directly, and the producer will carry the header into the property. All of this works because the mapping is two-way — the consumer doesn't just read these; the producer honors them too.

Recipe 1: pub/sub

A fanout exchange fans a copy out to every subscriber; the key is ignored. The producer publishes to the exchange, each consumer declares its own queue and binds:

// publisher — publishes to a fanout exchange (a copy to every subscriber)
From("timer://tick?period=5000")
    .SetBody(_ => BuildEvent())
    .To("rabbitmq://events-pub?exchange=events&exchangeType=fanout");

// two independent subscribers — each with its own queue
From("rabbitmq://audit?exchange=events&exchangeType=fanout&declare=true").To("direct://audit");
From("rabbitmq://email?exchange=events&exchangeType=fanout&declare=true").To("direct://email");

Want smarter routing? Switch to exchangeType=topic and a routingKey with wildcards (order.*, #.error). Only the exchange type and the key change.

Recipe 2: competing consumers (concurrency)

One parameter — concurrentConsumers:

From("rabbitmq://orders?concurrentConsumers=8&prefetchCount=16")   // up to 8 in flight
    .Process(HandleOrder);

This is the single knob for consumer concurrency: it sets both the channel's AMQP dispatch concurrency and the app-level semaphore. The default is 1 (strictly serial, order preserved). Set N > 1 and per-queue ordering is no longer guaranteed, your handler has to be thread-safe, and you keep prefetchCount ≥ N.

A note for anyone who kicked the tires on an earlier build: before 3.2.2, concurrentConsumers quietly gave you no parallelism thanks to a trap in RabbitMQ.Client 7.x (the channel was pinned to serial dispatch). Since 3.2.2 it's the one knob and it actually works — if you set N > 1 and "it somehow didn't speed up," update.

Acknowledgements are a parameter too. The default is at-least-once: ack after a clean pass, nack + requeue on error. Want fire-and-forget? autoAck=true flips you to at-most-once (the broker settles the delivery on dispatch; an error doesn't bring the message back).

And if what you need to parallelize is the processing inside a route rather than the intake (a serial source but heavy work), there's an orthogonal EIP step, .Threads(N). concurrentConsumers scales reading from the queue; .Threads scales the work; they compose freely.

Recipe 3: broadcasting across cluster nodes (local-cache invalidation)

A pattern straight from production. A service runs as a cluster of several nodes, each with its own local cache. Write to the database on any node, and you want the cache to refresh on all of them. The classic move over RabbitMQ: an audit producer publishes a domain event to a durable topic exchange, and every node hangs its own temporary queue off it and listens.

Producer (wherever the DB write happens) — a route per event kind, all into one exchange lt.events, each with its own routing key. Stamp the headers up front so subscribers can route/filter without cracking open the body. The connection comes from a shared factory:

// shared stamp: domain headers for subscribers — routing without parsing the body
Action<IExchange> stamp = e =>
{
    var a = e.In.GetBody<AuditEvent>();
    e.In.Headers["CorrelationId"] = a.CorrelationId;
    e.In.Headers["EntityType"]    = a.EntityType;
    e.In.Headers["EventType"]     = a.EventType.ToString();
};

From("direct://audit.entity.created")
    .RouteId("audit-entity-created")
    .Process(stamp)
    .SetBody(e => JsonSerializer.Serialize(e.In.GetBody<AuditEvent>()))
    .To("rabbitmq://?connectionFactory=RabbitMQ&exchange=lt.events&exchangeType=topic"
        + "&routingKey=dal.events.created&exchangeDurable=true");

// updated / deleted / batch / error — same exchange, routingKey=dal.events.{updated|deleted|…}

A consumer on every node — here's the whole trick. queue= empty + autoDelete=true: each node gets its own server-named temporary queue bound to the shared exchange. A node disconnects, its queue is gone. So a copy of every event lands on every node:

// created and updated — two consumers into one shared handler
From("rabbitmq://?connectionFactory=RabbitMQ&exchange=lt.events&exchangeType=topic"
        + "&routingKey=dal.events.created&queue=&autoDelete=true&expires=300000"
        + "&concurrentConsumers=2&prefetchCount=10")
    .RouteId("cache-consumer-created")
    .To("direct://cache-update");

From("rabbitmq://?connectionFactory=RabbitMQ&exchange=lt.events&exchangeType=topic"
        + "&routingKey=dal.events.updated&queue=&autoDelete=true&expires=300000"
        + "&concurrentConsumers=2&prefetchCount=10")
    .RouteId("cache-consumer-updated")
    .To("direct://cache-update");

// one processor for both streams — refreshes this node's local cache
From("direct://cache-update")
    .RouteId("cache-update")
    .Process(new CacheUpdateProcessor(/* … */));

Here's the key bit — the contrast with Recipe 2. That was competing consumers: one shared durable queue, the work split across consumers (each message handled by exactly one of them). This is the opposite: every node has its own ephemeral queue on the shared exchange → a broadcast, every node sees every event. Same connector, opposite topology, and the whole difference is a single parameter (queue=name&durable=true versus queue=&autoDelete=true). A DB write on any node → one event to the exchange → the local cache refreshes on every node, with no polling and no single point of failure.

Recipe 4: RPC (request/reply)

This is where it gets genuinely nice. Classic RabbitMQ RPC means a temporary reply queue, a correlationId, and waiting for the answer. Here it's one parameter on the client and nothing on the server. Here's how the demo wires it (DemoEndpoints.cs):

// server — an ordinary consumer; declare creates the queue
public const string RabbitConsumer =
    "rabbitmq://demo-rpc-queue?host=localhost&port=5672&username=admin&password=admin&declare=true";

// client — replyTo=true turns on RPC, timeout=15 — wait no longer than 15s
public const string RabbitProducer =
    "rabbitmq://demo-rpc-queue?host=localhost&port=5672&username=admin&password=admin&replyTo=true&timeout=15";

The client is a plain .To(...), except it doesn't fire and forget — it waits for the reply and rolls on carrying it (MainPipelineRoutes.cs):

.Log("[3-RABBIT] → Sending to RabbitMQ RPC...")
.To(RabbitProducer)                                      // publish the request and WAIT for the reply
.Log("[3-RABBIT] ← stamp.rabbit=${header.stamp.rabbit}") // the reply is already on the exchange

The server is just a consumer — there's nothing to configure for RPC. The connector sees reply-to on the incoming message, makes the route InOut, and sends whatever's in Out back to the reply queue with the same correlationId. The entire "server" from the demo:

From(RabbitConsumer)
    .RouteId("demo-rabbit-worker")
    .Log("[RABBIT-W] ▶ Received: ${body}")
    .SetHeader("stamp.rabbit", e => $"ok:{DateTime.UtcNow:HH:mm:ss.fff}")
    .Log("[RABBIT-W] ◀ Stamped, replying");         // the reply goes out on its own

Not a word about reply queues or correlation — the client sent a request and got back the stamp.rabbit header the server stamped on. Clean request/reply over a broker, with none of the temporary-queue plumbing by hand.

This is exactly where the contrast with MassTransit shows. It has request/response too — but that's IRequestClient<TRequest>, typed TRequest/TResponse contracts, and a consumer registered on the bus. Powerful, but it's a separate abstraction you have to step into. Here it's replyTo=true in the URI, and on the server side, nothing at all: an ordinary consumer, and the connector saw reply-to and replied. Two takes on the same problem: contracts and a bus versus one flag on an endpoint.

Recipe 5 (EIP): Filter — let only what matters through

Message Filter is a plain, perpetually useful EIP: let through only the messages that satisfy a condition and quietly drop the rest. In redb.Route that's .Filter(...) … .EndFilter() (or the shorter .End() — same thing): the block between them runs only for messages that pass, and for everything else the route ends right there. And this is where the compiled predicate engine shows up — the same one we got into in the expressions piece.

The condition can be written three ways. As a predicate over a header (or the body):

From("rabbitmq://orders?concurrentConsumers=4")
    .Filter(Header("score").isGreaterThan(50))       // score > 50
        .To("rabbitmq://vip-orders")
    .EndFilter();

The predicate set is everything you'd expect: .isEqualTo / .isNotEqualTo / .isGreaterThan / .isLessThan / .isGreaterThanOrEqualTo / .isLessThanOrEqualTo / .isBetween(a, b) / .contains(...). On the left, Header("...") or Body():

.Filter(Header("email").contains("@example"))   .To("direct://internal") .EndFilter();
.Filter(Header("grade").isEqualTo("B+"))         .To("direct://grade-b")  .EndFilter();

As a string expression — the ${...} resolves and the result is coerced to a boolean. For a truthy check, just the value; for a comparison, wrap it in logical(...) (a bare ${header.score > 50} inside a template comes back as an empty string — you can't write it that way):

.Filter("${header.status}")                  // 'active' → truthy → passes
    .Log("active only")
.EndFilter();

.Filter("${logical(header.score > 50)}")     // comparison → "True"/"False"
    .To("rabbitmq://vip")
.EndFilter();

Or as a lambda, when you want arbitrary C#:

.Filter(ex => ex.In.Body is string s && s.Contains("urgent"))
    .To("rabbitmq://urgent")
.EndFilter();

The handy thing is filtering on the very headers the consumer already stamped — say .Filter(Header("redbRmq.RoutingKey").contains("order")), so only messages with the routing key you care about make it into the block.

How an expression becomes a `bool`

Under the hood, for each message Filter evaluates the expression to object? and coerces it to bool by the ConvertToBoolean rules:

bool   → as-is
string → "true"/"false" parse; otherwise a non-empty string = true, empty = false
null   → false
anything else (number/object, non-null) → true

Which explains how the three forms behave:

A predicate like Header("amount").isGreaterThan(1000) is an IPredicate with a ready Func<IExchange,bool>. No coercion needed: a real boolean straight from the comparison. It compiles to a delegate and gets cached — not reflection per message, but a fast call.
A template string like "${header.status}" — the ${...} interpolates to a value, then the truthy rules apply. status="active" → non-empty string → true (that's why the demo says "active is truthy"); and "${header.enabled}" with "false" → bool.TryParse → false. A comparison written straight into the template ("${header.score > 50}") doesn't count — it comes back empty (i.e. false); for a comparison you need logical(...), as above.
A lambda ex => … — whatever you return is the predicate.

So "a boolean expression" is either an honest bool from a predicate/comparison, or a truthy coercion of an interpolated value. An empty string, null and "false" cut it off; a non-empty string, a number, true let it through.

The DSL is open: your own step is an extension method

Worth noticing one thing here. The string and predicate forms Filter(string) / Filter(IPredicate) aren't on the IRouteDefinition interface — that has only the canonical Filter(Func<IExchange,bool>) and Filter(IExpression). The string form is an extension method, a thin façade over the canonical one:

// the whole string overload (redb.Route/…/RouteDefinitionCamelDslExtensions.cs):
public static FilterDefinition Filter(this IRouteDefinition self, string simpleTemplate)
    => self.Filter(new StringExpression(simpleTemplate));   // wrap the string in a template expression

Which carries an important corollary: the DSL is extensible, and a step of your own is added the exact same way — an extension method that delegates to the canonical one. Want a custom "verb" in your routes? Write:

// your own step: pass only "urgent" in the body
public static FilterDefinition OnlyUrgent(this IRouteDefinition self)
    => self.Filter(ex => ex.In.Body is string s && s.Contains("urgent"));

// and in a route it reads like a native step:
From("rabbitmq://orders")
    .OnlyUrgent()
        .To("rabbitmq://urgent")
    .EndFilter();

No magic, no engine edits — just an extension method over IRouteDefinition. That's how the framework itself does the Camel-compatible forms, the string templates, and the predicate overloads. So if a step is missing for you, you write it yourself — no forking the connector.

We've only brushed the edge of expressions here — Header(...), Body(), ${...}, logical(...). Expressions in redb.Route are really a small compiled language of their own: access to the body and headers, exchange properties, indexers (items[0]), string functions, JSONPath and XPath. All of it gets tokenized, parsed into an AST, and compiled to a delegate (System.Linq.Expressions, not reflection), with a cache. It's a big topic — the engine gets its own post; it's earned it.

Recipe 6 (EIP): Dead Letter Channel

Of the Hohpe & Woolf patterns, Dead Letter Channel fits RabbitMQ perfectly — because on the broker it's a built-in feature (x-dead-letter-exchange), not something emulated in code. The idea: a message that didn't process (or expired on TTL, or didn't fit the queue's limit) isn't lost and doesn't spin forever on requeue — it heads to the morgue, a separate dead-letter exchange you can pick apart at your leisure.

It's set right in the working queue's URI:

// working queue: on nack-without-requeue / expiry / overflow the message goes to the DLX
From("rabbitmq://orders?declare=true"
        + "&deadLetterExchange=orders.dlx"          // where
        + "&deadLetterRoutingKey=orders.failed"     // with which key
        + "&messageTtl=30000"                       // not processed in 30s → DLX
        + "&overflow=reject-publish-dlx&maxLength=100000"   // and overflow goes there too
        + "&concurrentConsumers=4")
    .Process(HandleOrder);

// the morgue: a separate route listens on the DLX — logs and stores into redb for triage
From("rabbitmq://orders.dead?declare=true&exchange=orders.dlx&exchangeType=direct&routingKey=orders.failed")
    .Log("[DLQ] dead-lettered: ${body}")
    .ProcessWithRedb((redb, ex) => redb.SaveAsync(BuildDeadLetter(ex)));

No hand-rolled retry loop, no "sleep and try again" — reliability is handed to the broker, and what's left on your side is a clean triage route that writes the problem messages into a database. And the queue doesn't balloon under a surge: the excess leaves for the DLX right away instead of piling up in the broker's memory.

Wrap-up

RabbitMQ in redb.Route is routes where the whole broker is one URI: From("rabbitmq://…") / .To("rabbitmq://…"), with all the channel juggling, QoS, binds, reply queues and dead-lettering tucked behind the string's parameters. Every exchange type, durable/quorum queues, priorities, TTL, DLX, auto-recovery for a cluster, TLS/mTLS — the set is complete, and what you end up with is a short, legible string. Competing consumers turn on with one parameter, RPC with one flag on the client and zero config on the server, Filter trims the noise with compiled predicates, and Dead Letter Channel is a couple of parameters over the native DLX.

The full demo is in redb.Route/demos/redb.Route.Demo — it comes up against a local RabbitMQ (admin/admin) and runs as a self-test. If something bites in your scenario, say so in the comments.

Sources and releases: github.com/redbase-app. About the redb database: redb.ru.

If this was useful — a ⭐ on GitHub helps others find it.

redb 3.3.0: an enterprise .NET stack you actually own — typed store, a homegrown Apache Camel, and a runtime with a dashboard (all free)

rinat kozin — Thu, 09 Jul 2026 16:45:47 +0000

Series: redb ecosystem

Say "enterprise .NET stack" out loud and most people picture a menagerie: a database from one vendor, a bus from another, an ORM with migrations, a separate orchestrator, something for observability — and a layer of glue between all of it that you write yourself and then debug yourself at 2 a.m.

We took the other road. For the past six months we've been building this as one coherent ecosystem: a typed store, redb, on top of Postgres/MSSQL/SQLite; an integration engine, redb.Route (our take on Apache Camel for .NET); and a runtime, redb.Tsak, with a dashboard, hot-reload, and clustering. Three layers, one codebase, one style.

Today we shipped 3.3.0 — a synchronized bump across the whole ecosystem. This isn't a "we added a couple of features" release. It's the one where we fixed the things that were quietly broken under load (that's the headline, honestly), added two new transports, closed the loop on RAG for LLMs, and made concurrency work on any source. Code below, no fluff.

And the part everyone asks about first: as of 3.3.0, every Pro package is free. No licenses, no keys, no sign-up — you install the package and use it. Change tracking, bulk ops, the advanced cache, analytics, the Tsak cluster with its coordinator and failover — all in the box, same in dev and in prod. dotnet add package redb.Postgres.Pro and it just works. No paywall, no license server, nothing to activate.

Part of the redb / redb.Route series — recent posts first:

Leaving MassTransit for a Camel state of mind: the Kafka connector, Scatter-Gather, and transactions

Apache Camel for .NET, dissected: the HTTP connector with no ASP.NET MVC + the Content-Based Router

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL

Sources: github.com/redbase-app. About the database itself: redb.ru.

The three layers, in one breath

So the rest of this makes sense, here's the cast:

redb — a typed store for .NET. You write a plain POCO, tag it with an attribute, and work with it through full LINQ, server-side, with no migrations and no Include. Providers: Postgres, MSSQL, SQLite. Free and Pro editions (change tracking, bulk, cache, analytics).
redb.Route — an integration engine in the Camel spirit: a route DSL (From(...).….To(...)), 30+ connectors (Kafka, RabbitMQ, HTTP, gRPC, S3, LLM, and so on), enterprise integration patterns, transactions, telemetry. The glue — except it's a real engine with tests, not something you hand-rolled over a weekend.
redb.Tsak — a runtime that takes redb.Route routes and turns them into a production service: a dashboard, hot-reloadable modules (.tpkg), live context management, metrics, a cluster with a coordinator and failover. Ships as NuGet packages, Docker images, and standalone archives.

All the code samples are in English (they always were); everything's in the repo, so you can reproduce it.

To make the bottom layer concrete rather than a claim — here's what redb looks like in practice. A class, an attribute, and then full LINQ that runs server-side, no migrations, no Include:

[RedbScheme]
public class Note
{
    public string Tag  { get; set; } = "";
    public string Text { get; set; } = "";
}

// write
await redb.SaveAsync(new RedbObject<Note> { Props = new() { Tag = "work", Text = "hello" } });

// read — server-side query, the param comes straight from your code, no hand-written JOINs
var notes = await redb.Query<Note>()
    .Where(n => n.Tag == "work")
    .ToListAsync();

The schema comes from SyncSchemeAsync<Note>() — generated off the class itself, no migration files. This same layer stores both the knowledge-base chunks from the RAG example later on and your own domain objects, exactly alike.

redb.Route: what's new

Starting with the engine, because that's where most of the meat is.

Two new transports: Amazon SQS/SNS and Telegram

`redb.Route.Sqs` — an AWS queue and topic in one package

One package, two schemes: sqs:// (queue: consumer + producer) and sns:// (topic: publisher + SNS→SQS fan-out). Under the hood it's the native AWS SDK for .NET v4 — not a reinvented client.

using redb.Route.Sqs.Fluent;

// Consumer: 5 competing workers, long-poll, at-least-once (delete on success — ack is automatic)
From(Sqs.Queue("orders").WaitTimeSeconds(20).ConcurrentConsumers(5))
    .Process(ex => Handle(ex.In.Body));

// Publish an event to an SNS topic (fans out to subscribed queues)
From("timer://tick?period=5000")
    .SetBody(_ => BuildEvent())
    .To(Sns.Topic("order-events"));

The consumer does long-polling, ConcurrentConsumers(N) (real competing loops), a visibility timeout you can extend while a handler runs, transactional ack via .Transacted(), and FIFO. The producer sends single and batched messages with FIFO group/dedup ids; the SNS publisher handles subject, message structure, FIFO, and an SNS→SQS auto-subscription. Point serviceUrl= at LocalStack or ElasticMQ and it all works locally; the full AWS credential chain is honored, and W3C trace context rides along across the hop. Details in redb.Route.Sqs/README.md.

`redb.Route.Telegram` — a bot as a route

The telegram:// scheme, built on Telegram.Bot. The consumer is a long-poller (receive, a single getUpdates stream per token); the producer covers send / document / photo / edit / delete / answer.

// Echo bot: take an update, reply into the same chat
From(Tg.Receive(token))
    .Process(ex => ex.Out.Body = $"You said: {ex.In.Body}")
    .To(Tg.Send(token).ChatId(Header(TelegramHeaders.ChatId)));

Out of the box: it honors the 429 retry_after contract (waits and retries instead of falling over), validates parseMode (throws on a typo rather than quietly shipping raw markup), supports inline/reply keyboards (WithInlineKeyboard / WithReplyKeyboard), a webhook-unpack pipeline (UnpackTelegramUpdate), and a fluent DSL (Tg.Receive/Send/Document/...). Delivery is at-most-once — Telegram advances the update offset the moment it hands you the update, and that's documented, not a surprise. Parallelism comes from .Threads(N) (more on that below), with consumer telemetry and producer spans.

Two transports people actually run: SQS for anything living on AWS, Telegram for ChatOps and bots. Both are tested against emulators.

RAG, end to end: `knowledge://`, `embed://` and `.Knowledge()`

redb.Route.Llm already had a universal OpenAI-compatible provider and a native Anthropic one. In 3.3.0 we closed the RAG loop — from ingesting documents to answering grounded on them, all inside routes.

What landed:

knowledge:// (an ingest scheme) — a producer that takes the message body as a document, chops it into chunks (deterministic character windows with overlap), embeds each one if an IEmbeddingProvider is registered, and upserts into the IKnowledgeStore. Loading documents becomes a route:

  From("file://docs?include=*.md").To("knowledge://handbook");

Chunk ids are {docId}#{index}, so re-ingesting the same document replaces its chunks in place instead of piling up duplicates.

embed:// (embeddings as a step) — the mirror image of llm://: To("embed://openai") turns the message body (a text, or a batch of texts with order preserved) into a vector on Out.Body. The URI host names a connection factory, so different routes pick different embedding models by name.
.Knowledge(collection, k) (a retrieval DSL step) — pulls the top-K chunks for the current message and injects them into the system prompt, so the next .To("llm://…") answers off them:

  From("kafka://questions")
      .Knowledge("handbook", k: 5)
      .To("llm://claude");

Semantic when an IEmbeddingProvider is around (embed the query → cosine SearchAsync), keyword otherwise (SearchTextAsync, a server-side LIKE over an indexed column). No store wired or nothing retrieved? The step is a no-op and never breaks the pipeline.

knowledge_search (a ready-made tool) — an .AsLlmTool route over the search, so the agent can hit the knowledge base itself: input {query, top_k?, collection?} → {results:[…]}. You can pin the collection, which makes the model's argument moot — that's how you fence an agent into a single tenant or document set.

The full loop: knowledge:// ingest → embeddings → keyword/semantic search → the knowledge_search tool or .Knowledge() injection. And no external vector store unless you want one — the chunks live in redb, and search is just a server-side query.

We also killed an annoying one along the way: several of the LLM connector's JSON serializers used the default encoder, which escapes every non-ASCII character as \uXXXX. On tool results that roughly 6×'d the tokens the model saw for Cyrillic/CJK content (and could leak literal \u… into the text); in the knowledge store it buried chunk text so the new keyword LIKE couldn't match a raw non-ASCII query. Switched to UnsafeRelaxedJsonEscaping everywhere it mattered. If your data isn't pure ASCII, you'll feel this in both the token bill and the output.

`.Threads(N)` — concurrency on any source

There's a Camel-style processing-concurrency stage now: From(...).Threads(N)…EndThreads() caps a section of the route at N. The point: even a strictly serial source (a poll consumer, MQTT, a single request thread) can now chew through up to N messages at once — no named seda:// endpoint required.

From("mqtt://sensors")          // serial by nature
    .Threads(8)                 // up to 8 in flight
        .Process(ex => HeavyWork(ex))
    .EndThreads();

The nice part is that it's adaptive to the exchange pattern. For InOnly (fire-and-forget) it's a hand-off: clone plus a worker pool, a transaction boundary just like .To("seda://"). For InOut it runs the body inline on the same exchange under a SemaphoreSlim gate, so the reply — on Out or In — survives intact, request/reply (RPC) works through it, and the ambient transaction flows into the inline body. You get .MaxQueueSize(n) and .EnqueueTimeout(TimeSpan). Ordering isn't preserved at N > 1 — that's the price of parallelism, and we say so plainly.

The big one: `ConcurrentConsumers(N)` finally does something

Now the important — and frankly embarrassing — part. Brace yourself.

ConcurrentConsumers(N) was silently giving you no parallelism on the brokers. The option existed, you passed a number, it even sized an internal SemaphoreSlim — and no actual concurrency came out. The cause differed per connector; the symptom was always the same: the consumer processed one message at a time, unacked climbed to the prefetch limit, and the route never kept up.

What it was, and what it is now:

RabbitMQ. The channel was built with the three-argument CreateChannelOptions(...) ctor, leaving the fourth parameter (consumerDispatchConcurrency) at its compile-time default — and in RabbitMQ.Client 7.2.1 that default is 1, not null. A non-null per-channel value overrides the connection-level setting, so every channel was pinned to serial dispatch and the value from the URI/factory was quietly dropped. Now ConcurrentConsumers is the single knob for consumer parallelism: the channel always opens with an explicit dispatch concurrency. (redb.Route.RabbitMQ 3.2.2.)
Kafka. Plus a new EnableAutoCommit option (default true) to bring offset-settle in line with RabbitMQ's post-process ack — and a fix for the transacted producer that was throwing Local: Erroneous state on the deferred send. (redb.Route.Kafka 3.2.1.)
AMQP 1.0 and IBM MQ. Same class of bug: a serial receive loop awaited Process inline before pulling the next message, so the SemaphoreSlim was a dead gate — grabbed and released by the one loop. Now ConcurrentConsumers(N) means N genuine competing consumers, each with its own session/connection and loop (AMQPNetLite sessions and the IBM MQ managed client aren't thread-safe, so the concurrency comes from N independent workers, never from sharing a link). For IBM MQ it's also a correctness fix: the syncpoint is connection-scoped and Backout() rolls back the whole connection, so each worker must own its own. IBM MQ topics get clamped to a single subscriber with a warning — N non-durable subscriptions would each get a copy of every message rather than share the load. (redb.Route.Amqp / redb.Route.IbmMq 3.2.1.)

All three hotfixes (RabbitMQ 3.2.2, Kafka 3.2.1, Amqp/IbmMq 3.2.1) are now folded into the single 3.3.0 bump. Each ships a live regression test against a real broker: publish a batch, set ConcurrentConsumers(5), assert observed max concurrency is above 1, and assert ConcurrentConsumers(1) stays strictly serial.

Heads-up when you upgrade. If a route of yours set ConcurrentConsumers(N > 1) and "worked," it was working serially. After 3.3.0 it actually parallelizes: per-queue ordering is no longer preserved on that route, and its handlers need to be thread-safe. Routes left at the default 1 are untouched — strictly serial, exactly as before.

Bundled in are a new RabbitMQ AutoAck (broker-side auto-ack / at-most-once, the Kafka EnableAutoCommit analogue) and a fix for a double BasicAck when a route-level .Transacted() wraps a non-transacted consumer.

Per-exchange connections: the end of the captive singleton

The second half of that same problem lives in the engine core. IRedbService wraps one non-thread-safe DB connection (think EF DbContext). The DSL steps ProcessWithRedb(...), SetBodyFromRedb(...), SetHeaderFromRedb(...), and BeginRedbTransaction() used to fall back to a single IRedbService captured from the root DI container. Under real concurrency (a Splitter with parallel processing, SEDA, that very ConcurrentConsumers(N), concurrent HTTP requests) two exchanges drove that one connection at once — and the driver threw "A command is already in progress" / "connection is busy".

Now each exchange gets its own DI scope → its own scoped IRedbService → its own pooled connection, cached on the exchange and disposed with it. The unscoped singleton is only used when there's no exchange at all (a single-threaded seed). There's also controller.Redb() for controllers: it resolves the per-request instance instead of the shared captive singleton.

While we were in there we closed the matching DI-scope/connection leaks on the exchange lifecycle — ThreadsProcessor, SedaProducer/VmProducer (leaked the clone on a failed hand-off), WireTapProcessor (leaked the tap clone when a user onPrepare/newBody callback threw), and the scheduled llm:// and exec:// consumers (never disposed their per-tick exchange). Everything now disposes in finally. And we made lazy producer start-up thread-safe: on a cold start under concurrency you could catch a NullReferenceException (a Redis producer whose _db was still null, say); both paths are single-flight now, and a producer only reads as started once ConnectAsync() is fully done.

Short version: 3.3.0 is the release after which you can actually push redb.Route hard and stop chasing flaky connection errors. That's the line between "works in the demo" and "works in prod."

Putting it together: a RAG bot on Telegram, in a handful of routes

So the new pieces don't read like a shopping list, here's how they snap together. Goal: a Telegram bot that answers from your knowledge base (a folder of markdown) instead of making things up. Three routes — ingest, receive-and-answer — and it runs on a concurrent source.

First, ingest. One route: read the files, chunk them, embed them, upsert. The store is redb; no separate vector engine.

// 1. Ingest: a markdown folder → chunks with embeddings in redb
From("file://docs?include=*.md")
    .To("knowledge://handbook?chunkChars=1000&overlap=100&embed=true");

Now the bot. Take an update from Telegram, pull the top-5 relevant chunks for the question text, inject them into the system prompt, ask the model. .Knowledge() decides on its own — semantic if an IEmbeddingProvider is wired, keyword otherwise.

// 2. Question from Telegram → RAG → answer back into the same chat
From(Tg.Receive(token))
    .Threads(4)                          // up to 4 conversations in parallel
        .Knowledge("handbook", k: 5)     // top-5 chunks into the system prompt
        .To("llm://claude")              // answer grounded on them
    .EndThreads()
    .To(Tg.Send(token).ChatId(Header(TelegramHeaders.ChatId)));

That's the whole thing. Notice what isn't there: no hand-rolling of vectors, no separate embeddings service, no bespoke glue between Telegram and the LLM, no connection pool to babysit. .Threads(4) gives you parallel conversations on a source that hands out updates one at a time, and each worker gets its own connection when it touches redb.

Want the model to decide when to hit the knowledge base rather than doing it on every turn? Swap .Knowledge() for the knowledge_search tool and pin the collection so it can't wander outside handbook:

// register the tool once; the collection is pinned, so the agent is fenced into it
context.AddRoutes(new KnowledgeSearchTool(new KnowledgeSearchOptions {
    Collection        = "handbook",
    EmbeddingProvider = embeddings      // set → semantic; unset → keyword
}));

From(Tg.Receive(token))
    .Threads(4)
        .To("llm://claude?tools=knowledge_search")
    .EndThreads()
    .To(Tg.Send(token).ChatId(Header(TelegramHeaders.ChatId)));

Same set of primitives — knowledge://, embed://, .Knowledge(), knowledge_search, telegram://, .Threads() — all of it new or matured in 3.3.0. Each piece is small on its own; together they're a production RAG bot with zero external dependencies beyond the model.

redb (the DB core): what made it in

Three fixes and one guard rail in the core, all in the same concurrency-and-connections theme.

A fail-fast guard on the provider connection (RedBase.Postgres, RedBase.MSSql, RedBase.SQLite + .Pro). If the same IRedbService is entered from two threads at once, the provider now throws a clear InvalidOperationException that names the cause — instead of the murky driver error ("A command is already in progress", "connection is busy", "another read operation is already in progress"). It's a cheap Interlocked check with zero cost on the normal single-threaded path. It backstops the per-exchange connections in redb.Route: if someone does share an instance, they hear about it immediately and in plain English.
The query parser choked on array.Contains(x) in WhereRedb on .NET 9 / C# 13. A string[] (or any array) .Contains(x) inside a predicate now binds to the ReadOnlySpan overload (MemoryExtensions.Contains) instead of Enumerable.Contains, which the filter parser was rejecting with NotSupportedException. The parser recognizes MemoryExtensions.Contains, unwraps the array→span conversion, and translates it to the same IN clause. Small, but it bit you out of nowhere on the new compiler.
ComputeHash() NRE on an object with Props == null. The generic ComputeFor<TProps> path dereferenced the object before the null check. Added the guard — the path now returns null (→ Guid.Empty) consistently instead of throwing.
A connection-pool leak on transaction/connection dispose (RedBase.Postgres, RedBase.MSSql, RedBase.SQLite + .Pro). A throw from the driver's transaction DisposeAsync() (possible mid error-storm on an already-broken connection) skipped disposing _connection, so the physical connection never went back to the pool. Since SaveAsync runs inside an explicit transaction, every write armed this path — under a burst of failures the leak fed itself and eventually drained the pool (symptom: a healthy pool suddenly climbs past MaxPoolSize with connection-timeout errors, cleared only by a restart). Dispose now runs through try/finally on both the connection and the transaction wrapper, so the connection always goes back, and the dispose fault is no longer swallowed — it propagates so it stays visible.

redb.Tsak (the runtime): what made it in

The runtime picks up everything above and adds its own.

The new connectors are bundled. redb.Route.Sqs (sqs://, sns://) and redb.Route.Telegram (telegram://) now ship in the distribution, so modules on Tsak can use them without wiring anything extra. Technically that's a change to the shared-assembly layer — and a small war story from behind the curtain: that's exactly where we caught that the image build script didn't know about the new connectors. We built the image locally, ran the container, watched the logs print Loaded shared assembly redb.Route.Sqs and redb.Route.Telegram, and only then published.
The dashboard scheduler shows cron jobs even without a Quartz config section. Previously, with no Quartz config Tsak registered no shared scheduler, a cron route spun up its own per-context in-memory one, and the management _system context couldn't see it — so the scheduler page was blank even though the route was running. Tsak now always hands out one shared IScheduler (falling back to an in-memory RAMJobStore when there's no config). No cluster and no database needed for a single node; AdoJobStore is still how you persist and share jobs across nodes.
The Users admin API uses a per-request scoped IRedbService. UsersController used to resolve the shared captive singleton, so concurrent admin requests could collide. Now it's a per-request instance via controller.Redb().
Folded in from 3.2.1: the Endpoints page no longer hides anonymous contexts it was still counting in the stats, and the standalone web archive now starts on the documented port 8080 instead of the default 5000.

It ships the usual three ways: NuGet packages, Docker images redb-tsak-{worker,web,stack} (.NET 9; tags :3.3.0-net9, :3.3.0, :latest) on ghcr.io/redbase-app, cosign-signed, and standalone archives redb-tsak-3.3.0-linux-x64.tar.gz / redb-tsak-3.3.0-win-x64.zip with checksums.txt and .bundle signatures on the GitHub release. All on .NET 9.

[SCREENSHOT: the Tsak dashboard on localhost:8080 — the context list with the system echo route]

About Pro being free — once more, with detail

I said it up top, but the "what's the catch" question always comes first, so: there's no catch. Pro stays proprietary (the source is closed), but across the whole 3.x line it's handed out for free and with no bookkeeping — no licenses, keys, sign-up, license server, or node/volume limits.

Concretely, what used to cost money and now doesn't: on the redb core — change tracking, bulk ops, the advanced cache, analytical queries (grouping, window functions); on Tsak — the cluster with its coordinator, the distributed lock, leader election, failover. Exactly the stuff other stacks charge for. Here it's dotnet add package and you're working.

Upgrading from 3.2.x: the short checklist

The release is API-compatible, but there's one behavioral thing worth a look — the parallelism, of course.

Walk your routes that set ConcurrentConsumers(N > 1). They ran serially before (the bug); now they genuinely parallelize. Make sure the handlers on those routes are thread-safe and you aren't leaning on message order. If order matters, keep ConcurrentConsumers(1) — that's the default and it hasn't changed.
Check shared state in your handlers. Anything that "happened to work" because the route was effectively single-threaded can now run from N threads.
IRedbService in routes — nothing to do. Per-exchange connections turned on by themselves; if anything, those old "A command is already in progress" errors under concurrency will disappear. If you were manually caching one IRedbService across exchanges to route around the DSL — stop; the core will now tell you fail-fast.
One version, across the board. Everything moves to 3.3.0; you don't need to mix redb.Route 3.3.0 with a 3.2.0 connector (older 3.2.x connectors are API-compatible — they just don't carry the fresh concurrency fixes).
Tsak: rebuild or pull a fresh image. If you build your own distribution, the new connectors (Sqs, Telegram) only land in the shared layer after a rebuild; the published :3.3.0 images already have them.

None of this is urgent or build-breaking — it's a "go see where parallelism actually turned on for you" pass.

Getting it

Core and providers, from NuGet:

dotnet add package redb.Core
dotnet add package redb.Postgres      # or redb.MSSql / redb.SQLite
dotnet add package redb.Postgres.Pro  # Pro — free, no key

The engine and the connectors you need:

dotnet add package redb.Route
dotnet add package redb.Route.Sqs
dotnet add package redb.Route.Telegram
dotnet add package redb.Route.Llm

The runtime, as an image or an archive:

docker pull ghcr.io/redbase-app/redb-tsak-stack:3.3.0
# or a standalone archive from the v3.3.0 GitHub release (linux-x64 / win-x64), cosign-signed

The images are cosign-signed; the public key ships in the release (cosign.pub):

cosign verify --key cosign.pub ghcr.io/redbase-app/redb-tsak-worker:3.3.0

Wrap-up

3.3.0 is about maturity. The new transports (SQS/SNS, Telegram) and the finished RAG loop are nice, but they aren't the point of this release. The point is that we found and fixed a systemic hole in concurrency: ConcurrentConsumers(N) gave you no parallelism on any broker, and a shared captive IRedbService fell apart under load with murky connection errors. That's exactly the class of bug you don't see in a demo and that goes off in prod under load. Now concurrency is honest — on the brokers, on any source via .Threads(N), and with per-exchange connections.

On top of that: the whole stack is on 3.3.0 in lockstep, all on .NET 9, and every Pro package is free with no key. Your own database, your own integration engine, your own runtime with a dashboard and a cluster — one ecosystem, not a menagerie.

If you take it for a spin, drop a comment with what shows up in your setup. Individual connectors and EIP patterns get their own deep-dives in the series.

Source and releases: github.com/redbase-app. About the redb database: redb.ru.

If this was useful — a ⭐ on GitHub helps others find it.

Two routes in an evening: from a debug worker to an enterprise runtime with redb.Route + Tsak

rinat kozin — Thu, 02 Jul 2026 22:31:42 +0000

Series: redb ecosystem

Integration code has a funny asymmetry to it.

Writing a couple of routes — "take an HTTP request, stash it in a database, hand something back" — is a half-hour job. But getting that same thing to run in production, come up on its own, show you metrics, let you stop and start individual pieces by hand, and redeploy without a rebuild? That's usually a completely different stack and a completely different afternoon.

This post is about how, with redb.Route + redb.Tsak, it's literally the same code. We'll:

write two tiny routes (POST to save, GET to fetch by a query param) on redb.Route;
build our own debug worker — a plain console app you can run under the debugger with breakpoints;
and then, without changing a single line of the routes, pack it into a module and drop it into Tsak — where the exact same project picks up a dashboard, hot-reload, live route management, and enterprise deployment (with Docker and without).

All the code is in the repo (redb.Route/demos/EchoWorkerDemo), and I'm pasting it here in full so you can follow along.

Part of the redb / redb.Route series — recent posts first:

Leaving MassTransit for a Camel state of mind: the Kafka connector, Scatter-Gather, and transactions

Apache Camel for .NET, dissected: the HTTP connector with no ASP.NET MVC + the Content-Based Router

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL

Sources: github.com/redbase-app. About the database itself: redb.ru.

This one is the gentle intro to workers and deployment. Clustering — the coordinator, leader election, failover — gets its own post.

What we're building

A tiny notes service. Two endpoints on a single HTTP port:

POST /api/notes with a body like {"tag":"work","text":"hello"} — save a note into a redb database (on SQLite);
GET /api/notes?tag=work — return notes with that tag; the lookup runs server-side via Where(...).ToListAsync(), with the parameter pulled straight from the query string.

Nothing fancy. The point of the post isn't the business logic — it's the lifecycle: how one and the same set of routes first lives in a debug console, and then in an enterprise runtime.

Project layout: two projects, one entry point

Here's the shape of it:

EchoWorkerDemo/
├─ EchoModule/            <- project 1: the module (class library -> EchoModule.tpkg)
│  ├─ InitRoute.cs        <- main(IRouteContext): two routes + the Note class
│  ├─ manifest.json       <- { Name, Version, EntryPoints: ["EchoModule.dll"] }
│  ├─ EchoModule.config.json  <- context name (ContextName) + AutoStart
│  └─ EchoModule.csproj   <- references to the connectors + the PackTpkg target
└─ EchoWorker/            <- project 2: the debug host (exe)
   ├─ Program.cs          <- redb on SQLite + a call to InitRoute.main + Start
   └─ EchoWorker.csproj

The whole thing hinges on one idea: there is exactly one entry point — InitRoute.main(IRouteContext). Both our debug worker and the real Tsak runtime call it. The route code lives in exactly one place — EchoModule. The debug EchoWorker is just the plumbing that stands up a database and calls that same main.

Why two projects and not one? You can do one — an exe still produces a DLL, and Tsak finds InitRoute.main by reflection in any assembly. But splitting the two roles makes them obvious: EchoModule is what you ship (it packs into a .tpkg), and EchoWorker is what you debug with. The module carries zero host code — which is exactly right, because in production Tsak hands it the database.

Project 1: the module with two routes

Here it is in full — EchoModule/InitRoute.cs:

using System.Text.Json;

using redb.Core;                          // IRedbService, Query, SaveAsync, SyncSchemeAsync
using redb.Core.Attributes;               // RedbScheme
using redb.Core.Models.Entities;          // RedbObject<T>

using redb.Route.Abstractions;            // IRouteContext, IExchange
using redb.Route.Core;                    // RouteContext
using redb.Route.Http;                    // HttpComponent, SharedHttpServerManager
using redb.Route.RedbCore.Extensions;     // ProcessWithRedb, GetRedbService

namespace EchoModule;

/// <summary>
/// Tsak module entry point.
/// The worker discovers it by convention — a public static class named InitRoute
/// with a public static main(IRouteContext) — and calls it once when the module
/// loads. The debug host (EchoWorker/Program.cs) calls the very same method, so the
/// route code below lives in exactly one place.
///
/// Two minimal endpoints on the shared HTTP server (port 5099), backed by redb/SQLite:
///   POST /api/notes   body {"tag":"work","text":"hello"}   -> save one note
///   GET  /api/notes?tag=work                               -> list notes with that tag
/// </summary>
public static class InitRoute
{
    private static readonly JsonSerializerOptions Json = new() { PropertyNameCaseInsensitive = true };

    public static IRouteContext main(IRouteContext context)
    {
        // redb schema for Note. Idempotent — safe to call every load. The worker
        // (or the debug host) has already brought redb + SQLite up by now.
        context.GetRedbService().SyncSchemeAsync<Note>().GetAwaiter().GetResult();

        // One shared HTTP server; both routes below bind to it.
        context.AddComponent(new HttpComponent { ServerManager = new SharedHttpServerManager() });

        ((RouteContext)context).AddRoutes(r =>
        {
            // --- POST /api/notes — save one note ---
            r.From("http:0.0.0.0:5099/api/notes?inOut=true&methods=POST")
                .RouteId("notes-post")
                .ConvertBody<string>()                       // HTTP body -> string (JSON)
                .ProcessWithRedb(async (db, ex, ct) =>
                {
                    var note = JsonSerializer.Deserialize<Note>(ex.In.Body?.ToString() ?? "{}", Json) ?? new Note();
                    var obj = new RedbObject<Note> { name = $"note:{note.Tag}", Props = note };
                    await db.SaveAsync(obj);                 // one insert into redb (SQLite)
                    Reply(ex, new { saved = true, id = obj.id });
                }).Log("Save ${body}");

            // --- GET /api/notes?tag=work — list by tag ---
            r.From("http:0.0.0.0:5099/api/notes?inOut=true&methods=GET")
                .RouteId("notes-get")
                .ProcessWithRedb(async (db, ex, ct) =>
                {
                    // ?tag=... arrives as the header redbHttp.QueryParam.tag
                    var tag = ex.In.Headers.TryGetValue("redbHttp.QueryParam.tag", out var t)
                        ? t?.ToString() ?? ""
                        : "";

                    // Server-side filter: the GET parameter goes straight into Where(...).
                    var found = await db.Query<Note>()
                        .Where(n => n.Tag == tag)
                        .ToListAsync();

                    Reply(ex, found.Select(o => new { o.Props.Tag, o.Props.Text }));
                }).Log("Load ${header.redbHttp.QueryParam.tag}");
        });

        return context;
    }

    // inOut=true -> whatever the body is at the end becomes the HTTP response.
    private static void Reply(IExchange ex, object body)
    {
        ex.In.ContentType = "application/json";
        ex.In.Body = JsonSerializer.Serialize(body);
    }
}

/// <summary>Persisted note. [RedbScheme] marks the class as a redb schema.</summary>
[RedbScheme]
public sealed class Note
{
    public string Tag { get; set; } = "";
    public string Text { get; set; } = "";
}

Let's walk through the parts that matter — nearly every line is pulling its weight here.

The `main` entry point

public static IRouteContext main(IRouteContext context)

This is the Tsak module contract. No attributes, no interface to implement — just a naming convention: a public static class InitRoute, a public static method main taking an IRouteContext. When Tsak loads your assembly, it scans it by reflection, finds this method, and calls it with a context. Whatever you hang on that context — components, routes, listeners — becomes part of the runtime.

The neat bit: this works in the debug worker too. There, we create the RouteContext and we call InitRoute.main(ctx). Same method, two hosts.

The schema

context.GetRedbService().SyncSchemeAsync<Note>().GetAwaiter().GetResult();

Note is tagged [RedbScheme], which marks it as a persistable redb class. SyncSchemeAsync<Note>() registers its schema (idempotent — call it on every load, no harm done). GetRedbService() pulls IRedbService off the context: in Tsak the database is already up before your module runs, and in the debug worker we'll stand it up ourselves before main. Either way, redb is there by the time we ask.

Notice what the module doesn't do: it never configures the database. It doesn't know or care whether that's SQLite, Postgres, or MSSQL — it just asks the context for an IRedbService. The provider is the host's problem.

One HTTP server, two routes

context.AddComponent(new HttpComponent { ServerManager = new SharedHttpServerManager() });

SharedHttpServerManager is a shared HTTP server — several routes can hang off one port. Both of ours sit on 5099.

r.From("http:0.0.0.0:5099/api/notes?inOut=true&methods=POST")
r.From("http:0.0.0.0:5099/api/notes?inOut=true&methods=GET")

Both listen on the same path /api/notes, but they split on method via ?methods=POST / ?methods=GET. The server matches an incoming request on the pair "path + method": a POST goes to the first route, a GET to the second, and a PUT /api/notes gets an honest 405 Method Not Allowed (path matched, method didn't). inOut=true means request/response: whatever ends up in the exchange body at the end of the route becomes the HTTP response.

POST: save

.ConvertBody<string>()
.ProcessWithRedb(async (db, ex, ct) =>
{
    var note = JsonSerializer.Deserialize<Note>(ex.In.Body?.ToString() ?? "{}", Json) ?? new Note();
    var obj = new RedbObject<Note> { name = $"note:{note.Tag}", Props = note };
    await db.SaveAsync(obj);
    Reply(ex, new { saved = true, id = obj.id });
}).Log("Save ${body}");

ConvertBody<string>() turns the raw HTTP body (bytes) into a string. ProcessWithRedb is a processing step into which redb.Route injects the IRedbService (db) for you: under the hood it grabs the per-exchange DI scope if there is one, otherwise the context singleton. The rest is plain C#: deserialize the JSON into a Note, wrap it in a RedbObject<Note>, save it with a single SaveAsync, hand back { saved, id }.

The trailing .Log("Save ${body}") is a log step with interpolation: ${body} substitutes the current exchange body. redb.Route's ${...} can pull body, headers (${header.X}), and more — handy for tracing right inside the DSL.

GET: fetch by param

var tag = ex.In.Headers.TryGetValue("redbHttp.QueryParam.tag", out var t)
    ? t?.ToString() ?? ""
    : "";

var found = await db.Query<Note>()
    .Where(n => n.Tag == tag)
    .ToListAsync();

Reply(ex, found.Select(o => new { o.Props.Tag, o.Props.Text }));

redb.Route lays out HTTP query params as headers prefixed with redbHttp.QueryParam., so ?tag=work shows up as the header redbHttp.QueryParam.tag. Grab it, drop it straight into a server-side query:

db.Query<Note>().Where(n => n.Tag == tag).ToListAsync()

This is not "load everything into memory, then filter" — it's a query on the database side: the lambda n => n.Tag == tag compiles to a condition on the Note.Tag field. The result is a collection of RedbObject<Note>, so we reach the fields through .Props.

And that's the whole module. Two routes, a data class, zero infrastructure. Now let's run and debug it.

Project 2: our own debug worker

To run the routes under a debugger without spinning up any Tsak at all, we'll build a tiny console app. It reproduces exactly what the Tsak worker does when it loads a module, just in the smallest possible way. Here's EchoWorker/Program.cs in full:

// ============================================================================
//  EchoWorker — a debug host for the EchoModule Tsak module.
//
//  It reproduces, in the smallest possible way, what the Tsak worker does:
//    1) stand redb up on SQLite (Free tier — the worker's default),
//    2) create the redb system tables once,
//    3) hand a RouteContext to EchoModule.InitRoute.main — the SAME entry point
//       the worker calls, so no route code is duplicated here.
//
//  Run it, then (PowerShell — JSON in single quotes; cmd.exe needs \" escaping instead):
//    POST: curl.exe -X POST http://localhost:5099/api/notes -H "Content-Type: application/json" -d '{"tag":"work","text":"hello"}'
//    GET:  curl.exe "http://localhost:5099/api/notes?tag=work"
// ============================================================================

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

using redb.Core;                          // IRedbService
using redb.Core.Extensions;               // AddRedb
using redb.Core.Models.Configuration;     // PropsSaveStrategy
using redb.SQLite.Pro.Extensions;         // UseSqlite (tier-agnostic: AddRedb -> Free)
using redb.SQLite.Data;                   // SqliteDataSource.NativeExtensionPath

using redb.Route.Core;                    // RouteContext

namespace EchoWorker;

public static class Program
{
    public static async Task Main(string[] args)
    {
        // Free SQLite needs the native redb extension. The packaged Tsak worker ships
        // it; running from source we point at the one built under redb.SQLite/native/build.
        SqliteDataSource.NativeExtensionPath ??= ResolveNativeExtension();

        // DI: console logging + redb on SQLite (single-file DB next to the exe).
        var services = new ServiceCollection();

        services.AddLogging(b => b
            .AddSimpleConsole(o => { o.SingleLine = true; o.TimestampFormat = "HH:mm:ss "; })
            .SetMinimumLevel(LogLevel.Information));

        services.AddRedb(o => o
            .UseSqlite("Data Source=echo_demo.db")
            .Configure(c => c.PropsSaveStrategy = PropsSaveStrategy.DeleteInsert));

        var sp = services.BuildServiceProvider();

        // Create the redb system tables once (the worker does this on boot).
        // ensureCreated: true builds the base tables on a fresh SQLite file.
        await sp.GetRequiredService<IRedbService>().InitializeAsync(ensureCreated: true);

        // Build a route context over that provider and call the module entry point.
        var ctx = new RouteContext(sp, contextId: "echo-worker");
        ctx.AddService(typeof(ILoggerFactory), sp.GetRequiredService<ILoggerFactory>());
        EchoModule.InitRoute.main(ctx);       // <- the exact method the Tsak worker calls

        await ctx.Start();

        Console.WriteLine();
        Console.WriteLine("EchoWorker running: http://localhost:5099/api/notes");
        Console.WriteLine("  POST  {\"tag\":\"work\",\"text\":\"hello\"}   -> save");
        Console.WriteLine("  GET   ?tag=work                          -> list by tag");
        Console.WriteLine("Ctrl+C to exit.");
        Console.WriteLine();

        var stop = new ManualResetEventSlim();
        Console.CancelKeyPress += (_, e) => { e.Cancel = true; stop.Set(); };
        stop.Wait();

        await ctx.DisposeAsync();
    }

    // Walk up from the app dir to the repo's built Free SQLite native extension.
    // Returns null when running from a packaged worker (it resolves the extension itself).
    private static string? ResolveNativeExtension()
    {
        var suffix = OperatingSystem.IsWindows() ? ".dll"
                   : OperatingSystem.IsMacOS()   ? ".dylib"
                   : ".so";
        for (var dir = new DirectoryInfo(AppContext.BaseDirectory); dir != null; dir = dir.Parent)
        {
            var candidate = Path.Combine(dir.FullName, "redb.SQLite", "native", "build", "redb" + suffix);
            if (File.Exists(candidate))
                return candidate;
        }
        return null;
    }
}

Step by step:

1. The SQLite native extension. On its Free tier, redb-on-SQLite uses a native loadable extension (redb.dll / .so / .dylib) — part of the query machinery lives in there. The packaged Tsak worker ships that extension in the box. When you run from source, you have to point at the built binary — ResolveNativeExtension() just walks up the directory tree looking for redb.SQLite/native/build/redb.dll. One honest detail worth not hiding: without that extension, the Free SQLite provider won't start.

2. DI and the database. A regular ServiceCollection: console logging plus AddRedb(...).UseSqlite("Data Source=echo_demo.db"). UseSqlite is tier-agnostic: AddRedb gives you Free, AddRedbPro gives you Pro — you flip tiers without touching the usings. And this is the exact same tier (Free/SQLite) the Tsak worker runs by default, so your debug environment matches production.

3. System tables. InitializeAsync(ensureCreated: true) creates redb's base tables on a fresh file. In Tsak the worker does this on boot; in the debug host, we do it.

4. The context and the module call. We build a RouteContext over our provider and call EchoModule.InitRoute.main(ctx). That's the exact entry point Tsak will call. No route code is duplicated — it all lives in the module.

5. Start and wait. ctx.Start() brings up the HTTP server on 5099; then we sit until Ctrl+C.

The project files

EchoModule/EchoModule.csproj — a library plus a .tpkg packaging target:

<Project Sdk="Microsoft.NET.Sdk">

  <!-- Project 1 of 2: the Tsak MODULE.
       A class library. Its DLL is what gets packed into EchoModule.tpkg and
       hot-loaded by the Tsak worker. It contains ONLY route code — no host,
       no DB provider: the worker supplies redb + SQLite at runtime. -->
  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <RootNamespace>EchoModule</RootNamespace>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
  </PropertyGroup>

  <!-- The connectors the worker already ships in its shared Libs. We compile
       against them but the .tpkg carries only EchoModule.dll (see PackTpkg). -->
  <ItemGroup>
    <ProjectReference Include="..\..\..\src\redb.Route\redb.Route.csproj" />
    <ProjectReference Include="..\..\..\src\redb.Route.Core\redb.Route.Core.csproj" />
    <ProjectReference Include="..\..\..\src\redb.Route.Http\redb.Route.Http.csproj" />
    <ProjectReference Include="..\..\..\..\redb.Core\redb.Core.csproj" />
  </ItemGroup>

  <!-- After build: zip manifest.json + the module DLL into output/EchoModule.tpkg. -->
  <PropertyGroup>
    <TsakModuleName>EchoModule</TsakModuleName>
  </PropertyGroup>
  <Target Name="PackTpkg" AfterTargets="Build">
    <PropertyGroup>
      <_TpkgStaging>$(IntermediateOutputPath)tpkg</_TpkgStaging>
      <_TpkgFile>$(MSBuildThisFileDirectory)output\$(TsakModuleName).tpkg</_TpkgFile>
    </PropertyGroup>
    <RemoveDir Directories="$(_TpkgStaging)" />
    <MakeDir Directories="$(_TpkgStaging)" />
    <MakeDir Directories="$(MSBuildThisFileDirectory)output" />
    <Copy SourceFiles="$(MSBuildThisFileDirectory)manifest.json" DestinationFolder="$(_TpkgStaging)" />
    <Copy SourceFiles="$(TargetPath)" DestinationFolder="$(_TpkgStaging)" />
    <!-- {ModuleName}.config.json gives the module a named context (ContextName). -->
    <Copy SourceFiles="$(MSBuildThisFileDirectory)$(TsakModuleName).config.json"
          DestinationFolder="$(_TpkgStaging)"
          Condition="Exists('$(MSBuildThisFileDirectory)$(TsakModuleName).config.json')" />
    <ZipDirectory SourceDirectory="$(_TpkgStaging)" DestinationFile="$(_TpkgFile)" Overwrite="true" />
    <Message Importance="high" Text="Packed $(TsakModuleName) -> $(_TpkgFile)" />
  </Target>

</Project>

EchoWorker/EchoWorker.csproj — a console exe that references the module:

<Project Sdk="Microsoft.NET.Sdk">

  <!-- Project 2 of 2: the DEBUG HOST.
       A tiny console app. It stands redb up on SQLite (the same Free tier the Tsak
       worker uses by default), then calls EchoModule.InitRoute.main(ctx) — the exact
       method the worker calls. Run/F5 this to debug the route without Tsak. -->
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <RootNamespace>EchoWorker</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\EchoModule\EchoModule.csproj" />
    <!-- UseSqlite is tier-agnostic: AddRedb -> Free (what Tsak uses by default). -->
    <ProjectReference Include="..\..\..\..\redb.SQLite.Pro\redb.SQLite.Pro.csproj" />
    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="10.0.3" />
  </ItemGroup>

</Project>

There's an important detail in the dependencies. The debug host references the SQLite provider (redb.SQLite.Pro) — because it stands up the database. The module itself does not reference any DB provider: in Tsak, the worker provides the database. So the .tpkg only ships EchoModule.dll (see the PackTpkg target — it puts exactly the target DLL, the manifest, and the config into the archive, not the dependencies). The redb.Route.* and redb.Core connectors are already in the worker, which is why the package stays tiny.

Run it and poke it

dotnet run --project EchoWorker

Then, in another window, hit the endpoints. One Windows wrinkle worth calling out: escaping JSON for curl differs between cmd and PowerShell. In PowerShell the \" form does not work — you'll get '\' is an invalid start of a property name. So here's both.

cmd.exe (JSON in double quotes, inner ones escaped with \"):

curl -X POST http://localhost:5099/api/notes -H "Content-Type: application/json" -d "{\"tag\":\"work\",\"text\":\"hello\"}"
curl "http://localhost:5099/api/notes?tag=work"

PowerShell (wrap the JSON in single quotes — no escaping of the inner double quotes; and it must be curl.exe, since curl is an alias for Invoke-WebRequest):

curl.exe -X POST http://localhost:5099/api/notes -H "Content-Type: application/json" -d '{"tag":"work","text":"hello"}'
curl.exe -X POST http://localhost:5099/api/notes -H "Content-Type: application/json" -d '{"tag":"home","text":"other"}'
curl.exe "http://localhost:5099/api/notes?tag=work"

PowerShell, no curl (native):

Invoke-RestMethod -Method Post -Uri http://localhost:5099/api/notes -ContentType 'application/json' -Body '{"tag":"work","text":"hello"}'
Invoke-RestMethod "http://localhost:5099/api/notes?tag=work"

The responses:

{"saved":true,"id":1000019}
{"saved":true,"id":1000022}
[{"Tag":"work","Text":"hello"}]

A GET with ?tag=home returns the other note, and PUT /api/notes gives you a 405. It all works, and it's all under the debugger: drop a breakpoint inside ProcessWithRedb, fire a curl, and you're sitting right in the handler with a live db and ex.

That's the debug zone: a plain console app, F5, breakpoints, a live SQLite file next to the exe, raw logs to stdout. This is where you write and debug your routes. Iteration loop: "tweak → run → poke with curl."

Here's the trick: the same code, now enterprise

Now for the good part. We don't touch the routes at all. We take the same EchoModule.dll, add two tiny descriptor files, pack it into a .tpkg, and hand it to Tsak. One step — and our "console app under F5" gains a dashboard, metrics, hot-reload, and live management. New code written: zero.

What a `.tpkg` is

A .tpkg is just a ZIP with three things:

manifest.json — the module's passport;
EchoModule.dll — the assembly with InitRoute.main;
EchoModule.config.json — the module's config (first and foremost, its context name).

manifest.json:

{
  "Name": "EchoModule",
  "Version": "1.0.0",
  "EntryPoints": ["EchoModule.dll"],
  "Dependencies": []
}

EchoModule.config.json:

{
  "ContextName": "echo",
  "AutoStart": true
}

A word on ContextName, because it's an easy trap. If you don't set it, Tsak loads the module into an anonymous context with an auto-generated name (something like EchoModule_dyn_<date>_<guid>). The module runs fine, the routes work — but the dashboard's Endpoints page hides anonymous contexts by default (while still counting them in the totals, so the numbers won't match what you see in the list). The takeaway is simple: name your context via EchoModule.config.json, and it shows up neatly as echo. Small thing, big peace of mind.

Pack it

The PackTpkg target in the .csproj does everything at build time:

dotnet build EchoModule -c Debug

Out comes EchoModule/output/EchoModule.tpkg. Peek inside:

Archive:  output/EchoModule.tpkg
  Length      Name
---------  ----
       49  EchoModule.config.json
    13312  EchoModule.dll
      108  manifest.json

Three files, ~7 KB. That's your deployment artifact.

How Tsak finds modules: folders and hot-reload

This is the part the whole thing was built for. Tsak doesn't "watch" the folder with filesystem events — internally there's a background HotReloadService that polls the directories listed in the Tsak:Modules:AssemblyPaths config on a timer.

The flow:

The worker config has a list of paths, Tsak:Modules:AssemblyPaths. The module folder is usually called modules.
Every Tsak:HotReload:ScanIntervalSeconds (default 10 seconds) the service walks each path and grabs every *.tpkg (and bare *.dll too).
"New / changed / removed" is decided by file modification time:
- unchanged mtime → skip;
- new file → unzip it, load the assembly into an isolated load context (its own AssemblyLoadContext per package), find InitRoute.main by reflection, bring the context up;
- changed file → reload the package atomically (drop the old version, load the new one, preserving state such as whether the context was running);
- file gone from disk → unload every module in that package.
Shared connectors (redb.Route.*, redb.Core) resolve from the worker's shared libraries (Libs/shared), which is why they're not inside the .tpkg.

The practical upshot:

Drop or update EchoModule.tpkg in modules/ and it gets picked up within ~10 seconds, no restart.
The trigger is the file's mtime. A plain copy bumps the mtime, so "re-copied" = "reloaded."
Each .tpkg lives in its own load context — module versions are isolated from each other.

In other words, deploying a new module (or a new version) is copying one file into a folder. Tsak takes it from there.

The dashboard: watch it, and run it

Stand Tsak up (two ways below), drop EchoModule.tpkg into the modules folder, wait a few seconds, and open the web dashboard. Our echo context is right there, with its two endpoints.

And here's the second big difference from the debug console. In the console we only had text logs. In Tsak it's the same information, but:

you see the echo context and its endpoints — type (HTTP), role (Consumer), in/out counters, errors, throughput, uptime, health;
you see the _SYSTEM context — that's the worker's own management API;
you can drill into each endpoint for details (messages, bytes, average processing time, last errors).

But the real point is that you can operate it, right from the browser, without touching configs or redeploying. On the node detail page for our echo context you get:

the context as a whole — Start / Stop / Restart, plus Reset route states (clear the saved route states);
individual routes — stop/start a specific notes-post or notes-get without touching its neighbor;
schedules (if the module has Quartz jobs) — pause and resume jobs on the scheduler tab.

And here's the important bit: state sticks. If you manually stop a route, then hot-reload will not bring it back up when the package updates — Tsak respects the operator's decision. A route you stopped in production won't suddenly come back to life just because you rolled out a new .tpkg version. Management is both "hot" (takes effect immediately) and "sticky" (survives a module reload).

The management actions live behind the admin role — so this is genuinely an operator's console, not just a metrics wall.

Feel the contrast: in the custom worker you debug (F5, breakpoints, a bare console); in Tsak you operate (observe, manage, deploy). Same code, two environments, two very different jobs.

Ports: defaults, and how to set them

Since we're running this across different environments, let's put the ports on the table — it's easy to get lost in what lives where.

The module (our routes) — 5099. That's the port we wrote into From("http:0.0.0.0:5099/..."). The module's own HTTP server. Want a different one? Change it in the route.
The worker (Tsak management API) — 9090. A separate server: health, metrics, context management, the cluster API. This is what the dashboard talks to. The port comes from the worker config.
The web dashboard — pay attention here. If neither ASPNETCORE_URLS nor a Kestrel binding is set in the web config, ASP.NET Core falls back to its default — http://localhost:5000. The Docker images and the distribution's startup scripts set ASPNETCORE_URLS=http://localhost:8080 (or http://+:8080 in a container), which is why the dashboard is on 8080 under Docker and in the docs. Run the web "bare" without that variable and don't be surprised to find the UI on 5000.

To set it explicitly:

# before launching the web
$env:ASPNETCORE_URLS = "http://localhost:8080"

or in the web's appsettings.json:

"Kestrel": { "Endpoints": { "Http": { "Url": "http://localhost:8080" } } }

Cheat sheet: 9090 — worker API, 5099 — our endpoints, 8080 — dashboard (when ASPNETCORE_URLS is set; otherwise the framework default 5000).

Enterprise deploy #1: Docker (the whole stack in one container)

The simplest way to bring it all up at once is the redb-tsak-stack image: worker + dashboard in a single container. The images live on the org's packages page — there are several: redb-tsak-worker, redb-tsak-web, and the combined redb-tsak-stack (plus per-TFM variants). For a quick start, stack is all we need. Here's the docker-compose.yml in full:

# Minimal redb.Tsak stack — Worker (API) + Blazor dashboard in one container.
# Drop a module as a .tpkg into ./modules and the worker hot-loads it.
#
# Start:
#   docker compose up -d
#
# Dashboard:   http://localhost:8080   (login admin / admin)
# Tsak API:    http://localhost:9090/api/health/live
# EchoModule endpoints:  http://localhost:5099/api/notes
#   POST (PowerShell): curl.exe -X POST http://localhost:5099/api/notes -H "Content-Type: application/json" -d '{"tag":"work","text":"hello"}'
#   POST (cmd.exe):    curl -X POST http://localhost:5099/api/notes -H "Content-Type: application/json" -d "{\"tag\":\"work\",\"text\":\"hello\"}"
#   GET:  curl.exe "http://localhost:5099/api/notes?tag=work"

services:
  tsak:
    image: ghcr.io/redbase-app/redb-tsak-stack:latest
    container_name: tsak
    restart: unless-stopped
    ports:
      - "8080:8080"   # Blazor dashboard
      - "9090:9090"   # Tsak management REST API (health, cluster, contexts...)
      - "5099:5099"   # the EchoModule's own HTTP server (/api/notes)
    environment:
      # redb store on the bundled SQLite — no external dependencies
      Tsak__Storage__Type: Redb
      # HMAC secret for API-key auth (any >=16 chars; change for real use)
      Tsak__Auth__Secret: "demo-secret-change-me-please-0123456789"
    volumes:
      # your .tpkg modules — hot-loaded by the worker
      - ./modules:/app/worker/modules

What matters here:

The image ghcr.io/redbase-app/redb-tsak-stack:latest — worker and dashboard in one container (under a supervisor). Storage defaults to the bundled SQLite, no external dependencies.
The ports — the same three: 8080 (dashboard), 9090 (worker API), 5099 (our endpoints).
The volume ./modules:/app/worker/modules — this is where the .tpkg goes. Inside the stack image the module folder is /app/worker/modules, which is exactly the path hot-reload watches.
Tsak__Auth__Secret — the HMAC secret for signing API keys; for a local run any value of 16+ chars will do.

Layout on the host:

tsak/
├─ docker-compose.yml
└─ modules/
   └─ EchoModule.tpkg

Bring it up:

docker compose up -d

Dashboard at http://localhost:8080 (admin / admin). Test the endpoints (PowerShell):

curl.exe -X POST http://localhost:5099/api/notes -H "Content-Type: application/json" -d '{"tag":"work","text":"hello"}'
curl.exe "http://localhost:5099/api/notes?tag=work"

That's it. The same module we debugged in a console now runs in a container with a dashboard. To update the module: rebuild the .tpkg, re-copy it into modules/, and it gets picked up in ~10 seconds.

Enterprise deploy #2: no container (standalone archive)

Don't want Docker? Fine. The releases page has self-contained archives per platform — for example, the v3.2.0 tag ships archives for win-x64, linux-x64, and so on (each is cosign-signed, with a .bundle and an SBOM alongside for integrity checks).

Inside an extracted archive (redb-tsak-<version>-<platform>), the layout is:

redb-tsak-3.2.0-win-x64/
├─ worker/     <- the runtime: redb.Tsak.Worker.exe (+ Libs/shared with connectors, + modules/)
├─ web/        <- the dashboard: redb.Tsak.Web.exe
├─ cli/        <- the tsak command-line tool
├─ scripts/    <- start-worker / start-web / start-stack (.bat / .ps1 / .sh)
├─ README.txt
├─ LICENSE, NOTICE

Launch from scripts/:

start-stack — bring up worker and dashboard together;
start-worker — just the runtime;
start-web — just the dashboard.

On Windows, for instance:

.\scripts\start-stack.ps1

Same ports: worker on 9090, dashboard on 8080 (the scripts set ASPNETCORE_URLS; run web completely bare, outside the script, and remember the 5000 default from the ports section).

Where the module goes. In the standalone layout, modules live next to the worker — in its modules/ folder (the path comes from Tsak:Modules:AssemblyPaths). Drop our EchoModule.tpkg there:

worker/
└─ modules/
   └─ EchoModule.tpkg

Leave Libs/shared alone — that's the distribution's shared connectors, the layer your lightweight package loads on top of.

Licensing: with no key, the runtime starts in OSS mode. Pro features (clustering included) turn on via the Tsak__Redb__License__0 environment variable with your JWT, or by editing worker/appsettings.json before launch.

So — two deployment paths, container or archive, but one model: the .tpkg artifact goes into the modules folder, and hot-reload takes it from there.

Two zones: build and operate

Let's put the whole picture together.

The build/debug zone — the EchoWorker project. A console app, F5, breakpoints, a local SQLite file next to the exe, raw logs to the console. This is where you write and debug routes. Loop: "tweak → run → poke with curl."
The operations zone — Tsak (Docker or archive). The same module, but with a dashboard, metrics, hot-reload, and live management. This is where you deploy and operate. Loop: "rebuild the .tpkg → copy into modules/ → picked up."

And the key thing: not a single line of the routes changes between those zones. InitRoute.main is the same. The only difference is the plumbing — in debug you write it (a dozen lines of Program.cs), in production Tsak provides it.

What's next: clustering (a teaser)

Everything above is single-node. But Tsak clusters too: multiple workers, one shared dashboard, modules and routes distributed across nodes, leader election, automatic takeover when a node dies — and the whole cluster topology lives in the redb database itself, with no separate membership infrastructure (no ZooKeeper/etcd/Consul). There's also active-passive for routes: the same route consumer runs on exactly one node, and if that node falls over, another one picks it up.

That's a topic for the next post, though. This one was the intro: how to knock out two routes in an evening, build yourself a debug worker, and then take the exact same code into an enterprise runtime. The coordinator, the leader, and failover — we'll get into those properly next time.

Wrap-up

We walked the whole path on one tiny example:

wrote two routes on redb.Route (POST to save into redb/SQLite, GET to fetch with a server-side Where(...).ToListAsync());
built our own debug worker — a console app under F5 that calls the same InitRoute.main;
packed the module into a .tpkg (DLL + manifest.json + a config.json with a context name);
watched Tsak pick it up on its own from the modules/ folder via hot-reload;
operated it from the dashboard (start/stop/restart of contexts and routes, with sticky state);
sorted out the ports (9090 — worker API, 5099 — our endpoints, 8080/5000 — dashboard);
deployed it two ways: with Docker (the whole docker-compose.yml) and without (an archive from the releases page).

One and the same codebase — comfortable to debug, and ready for an enterprise runtime. Clustering next.

The full project: redb.Route/demos/EchoWorkerDemo in the repo. Clone it, run it, break it.

If this was useful — a ⭐ on GitHub helps others find it.

Leaving MassTransit for a Camel state of mind: the Kafka connector, Scatter-Gather, and what really happens under a transaction

rinat kozin — Tue, 30 Jun 2026 20:37:22 +0000

Series: redb ecosystem

Part of the redb.Route series — the Apache Camel we built for .NET. If you're just tuning in:

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL — the origin story;

Enterprise Integration Patterns in .NET — Part 1: the four in-memory channels (and the Exchange they carry);

redb.Route 3.0.1 — flat DSL navigation, CRTP refactor, and a silent null fix;

Apache Camel for .NET, dissected: the HTTP connector with no ASP.NET MVC + the Content-Based Router pattern — the previous "EIP + connector" deep-dive.

This time we open with the Kafka connector — taken apart the same way we did HTTP — and then we mount two EIP patterns on top of it: Scatter-Gather and Aggregator. And the part tutorials skip: how the whole thing behaves once you wrap a transaction around it. Also, 3.2.0 shipped.

One spoiler up front

There's no "exactly-once out of the box" in Kafka here — and you'll see exactly why from the code below. What works and what doesn't — stated plainly.

next A real production example: one HTTP request → six parallel aggregations

Why leave MassTransit at all
What landed in 3.2.0
The Kafka connector: URI anatomy and the connection factory
The Kafka producer: what .To("kafka:...") actually does
The Kafka consumer: polling, batches, rebalance
Headers and tracing across the broker
Straight talk about transacted=true
EIP #1 — Scatter-Gather: one processor, both fan-out and join
EIP #2 — Aggregator: assembly over time
Transactions: two models, and that's the whole point
The parallel cousins: Splitter and Multicast
Saga — "a slightly different one"
Outbox — there isn't one, and that's correct
Bottom line: a trade, not "better/worse"

1. Why leave MassTransit at all

MassTransit is genuinely good, and I'm not here to bury it. But it's an opinionated framework: the durable state-machine saga, the transactional outbox, the retry policies — they're baked in, and you live by the framework's rules. When your scenario matches its worldview, it's a joy; everything "just works." When it doesn't, you're fighting an abstraction, spelunking through internals and bolting hacks onto someone else's decisions.

Apache Camel is a different school. The bedrock is EIP primitives + connectors + a DSL. Nobody gifts you an outbox as a feature — you get Splitter, Aggregator, Scatter-Gather, transactional transports, and you assemble exactly what your invariants need. Less magic, more explicit composition from bricks.

redb.Route sits in the second school. Hence the title: not "batteries included," but "here are the bricks, and they snap together predictably." The rest of the article is about what that trade costs you, and what you get back.

2. What landed in 3.2.0

Full notes are in the CHANGELOG. Two items matter here:

Parallel Splitter / Multicast branches now isolate the ambient transaction per branch. Each branch gets its own DependentTransaction.DependentClone(BlockCommitUntilComplete); the parent commit blocks until every branch signals done. Before this, parallel branches shared a single Transaction.Current, and System.Transactions forbids using one transaction concurrently across threads. We'll read this fix line by line and see why Scatter-Gather never needed it.
Throttle got an RFC 6585 mode — .RejectOnOverflow() (429 + Retry-After). Off-topic today, but handy if you front all of this with an HTTP edge.

All redb.Route.* packages are version-aligned at 3.2.0.

3. The Kafka connector: URI anatomy and the connection factory

We open with Kafka, as promised — and we open with the simplest thing there is: a string endpoint URI. Here's a minimal consumer and a minimal producer:

// Consumer: read topic 'orders', group 'order-workers', from the beginning
From("kafka:orders?brokers=kafka:9092&groupId=order-workers&autoOffsetReset=earliest")
    .Log("Got: ${body}")
    .To("direct:process");

// Producer: publish to 'notifications' with acks=all
From("direct:notify")
    .To("kafka:notifications?brokers=kafka:9092&acks=All");

The URI shape is kafka:<topic>?<query>. The topic comes from the path (KafkaEndpoint):

public KafkaEndpoint(EndpointUri uri, KafkaComponent component, KafkaEndpointOptions options)
    : base(uri, component, options)
{
    TopicName = uri.Path;                       // ← topic = URI path

    // If the URI says connectionFactory=<name>, resolve it from the registry
    if (!string.IsNullOrEmpty(options.ConnectionFactory) && component.Context is not null)
        ResolvedFactory = component.Context.GetFromRegistry<KafkaConnectionFactory>(options.ConnectionFactory);

    // No brokers in the URI but the factory has them — fill it in
    if (string.IsNullOrWhiteSpace(options.Brokers) && ResolvedFactory is not null)
        options.Brokers = ResolvedFactory.Brokers;
}

Note the last touch: on a consumer, groupId is mandatory, and forgetting it fails endpoint creation rather than blowing up later:

public override IConsumer CreateConsumer(IProcessor processor)
{
    if (string.IsNullOrWhiteSpace(Options.GroupId))
        throw new InvalidOperationException(
            $"The 'groupId' parameter is required for Kafka consumer on topic '{TopicName}'.");
    return new KafkaConsumer(this, processor, Options);
}

The connection factory — so you don't repeat yourself

When you've got twenty Kafka endpoints on one cluster, spelling out brokers=...&securityProtocol=...&saslMechanism=... in every URI is suicide. That's what KafkaConnectionFactory is for: register it in the registry under a name, then reference it via connectionFactory=name.

Its full parameter surface (all from source, nothing invented):

Group	Properties	Default
Brokers	`Brokers`	`localhost:9092`
Security	`SecurityProtocol`, `SaslMechanism`, `SaslUsername`, `SaslPassword`	`Plaintext`
SSL/TLS	`SslCaLocation`, `SslCertificateLocation`, `SslKeyLocation`, `SslKeyPassword`, `SslEndpointIdentificationAlgorithm`	—
Producer	`Acks`, `Retries`	`Leader`, `3`
Consumer	`GroupId`, `AutoOffsetReset`	—, `Latest`
Consumer tuning	`GroupInstanceId`, `SessionTimeoutMs`, `HeartbeatIntervalMs`, `MaxPollIntervalMs`, `PartitionAssignmentStrategy`, `IsolationLevel`	—
Producer tuning	`LingerMs`, `BatchSize`, `CompressionType`, `MessageTimeoutMs`	—
Reconnect	`ReconnectBackoffMs`, `ReconnectBackoffMaxMs`	—
Advanced	`ClientId`, `RequestTimeoutMs`, `MetadataMaxAgeMs`, `SocketTimeoutMs`, `MaxInFlight`, `AdditionalProperties`	`redb.Route`, 30000, 300000, 60000, 5

context.AddToRegistry("prod-cluster", new KafkaConnectionFactory
{
    Brokers          = "kafka1:9092,kafka2:9092,kafka3:9092",
    SecurityProtocol = "SaslSsl",
    SaslMechanism    = "ScramSha512",
    SaslUsername     = "svc-orders",
    SaslPassword     = secret,
    CompressionType  = "Zstd",
    MaxInFlight      = 5,
});

// Now the URI only carries what's endpoint-specific:
From("kafka:orders?connectionFactory=prod-cluster&groupId=order-workers");
To("kafka:notifications?connectionFactory=prod-cluster&acks=All");

The factory builds three configs — BuildConsumerConfig(), BuildProducerConfig(), BuildAdminConfig() (the last for topic-metadata queries). Endpoint-level options are applied on top of the factory (if (!string.IsNullOrWhiteSpace(Brokers)) config.BootstrapServers = Brokers; and so on), so the factory sets the baseline and the URI overrides it surgically.

The full endpoint option set

The complete list — KafkaEndpointOptions, grouped, with defaults:

Connection / security: brokers (required, or Validate() throws), securityProtocol (Plaintext/Ssl/SaslPlaintext/SaslSsl), saslMechanism/saslUsername/saslPassword, sslCaLocation/sslCertificateLocation/sslKeyLocation/sslKeyPassword, connectionFactory.

Consumer: groupId, autoOffsetReset (Latest/Earliest/Error, default Latest), enableAutoCommit (default true, framework-level — since 3.2.1), maxPollRecords (0 = single mode, >0 = batch), pollTimeoutMs (1000), breakOnFirstError, seekTo (beginning/end), topicIsPattern, groupInstanceId, sessionTimeoutMs, heartbeatIntervalMs, maxPollIntervalMs, partitionAssignmentStrategy (Range/RoundRobin/CooperativeSticky), isolationLevel (ReadUncommitted/ReadCommitted).

Producer: acks (None/Leader/All, default Leader), retries (3), recordMetadata, key, partitionNumber, transacted, transactionIdPrefix (redb-kafka).

Producer tuning: lingerMs, batchSize, compressionType (None/Gzip/Snappy/Lz4/Zstd), messageTimeoutMs.

Advanced: additionalProperties — arbitrary librdkafka properties, applied last, overriding everything typed.

Validate() checks exactly: brokers non-empty, maxPollRecords >= 0, pollTimeoutMs >= 0, retries >= 0. Everything else has a default or is optional.

One thing to get precise — about the offset commit, and it matters. At the librdkafka level, EnableAutoCommit is hard-wired to false always — the library never commits on a background timer (which would commit un-processed offsets). From BuildConsumerConfig:

var cfg = new ConsumerConfig
{
    BootstrapServers = Brokers,
    GroupId          = GroupId,
    AutoOffsetReset  = ParseAutoOffsetReset(),
    EnableAutoCommit = false,   // librdkafka level: always manual commit
};

But there is a commit — one level up, at the framework level. As of 3.2.1 the consumer has a typed EnableAutoCommit option (default true): after a successful Process it commits the offset inline — exactly like the RabbitMQ consumer acks after processing. Set it from the URI (?enableAutoCommit=false) or fluently (.EnableAutoCommit(false)). A transactional route takes precedence: if a .Transacted() already committed the offset, the inline path is skipped. The mechanics are in §5.

The same thing, fluent

String URIs are great for short snippets and config files. In code you usually build endpoints fluently — Kafka.Topic(...):

From(Kafka.Topic("orders")
        .Brokers("kafka1:9092,kafka2:9092")
        .GroupId("order-workers")
        .AutoOffsetReset("Earliest")
        .MaxPollRecords(500)
        .PartitionAssignmentStrategy("CooperativeSticky")
        .IsolationLevel("ReadCommitted")
        .SessionTimeout(30_000)
        .MaxPollInterval(300_000));

To(Kafka.Topic("notifications")
        .Brokers("kafka1:9092")
        .Acks("All")
        .Compression("zstd")
        .Linger(20)
        .BatchSize(64 * 1024)
        .Key("${header.orderId}"));   // partition key — an expression

The fluent builder just assembles the same query string (Kafka.Topic("orders").Brokers(...)...Build() → "kafka:orders?brokers=...&..."), URL-encoding the values. The setters take an IExpression, but options resolve at different times: the producer key is evaluated as a ${...} expression per message (in DetermineKey at send time — see below), whereas connection-level options (brokers, credentials) are resolved once on connect, where there's no exchange yet, so an expression there is meaningless. The per-message story is specifically the partition key.

4. The Kafka producer: what `.To("kafka:...")` actually does

Now let's read what KafkaProducer really does when a message hits .To("kafka:..."). It's a ConnectableProducer, so it requires Start() first (EnsureStarted() at the top of Process).

Connect

protected override Task ConnectAsync(CancellationToken ct)
{
    var config = _options.BuildProducerConfig(_endpoint.ResolvedFactory);

    _producer = new ProducerBuilder<string, byte[]>(config)
        .SetValueSerializer(Serializers.ByteArray)
        .SetErrorHandler((_, e) =>
            Logger?.LogError("Kafka producer error: {Reason} (Code: {Code})", e.Reason, e.Code))
        .Build();

    if (_options.Transacted)
    {
        _producer.InitTransactions(TimeSpan.FromSeconds(30));
        Logger?.LogInformation("Kafka transactional mode enabled: topic={Topic}", _endpoint.TopicName);
    }
    return Task.CompletedTask;
}

Key is string, value is byte[]. On disconnect it does Flush(30s) before Dispose so buffered messages aren't lost.

Preparing the message

PrepareMessage turns the exchange body into a Message<string, byte[]>:

byte[] valueBytes = body switch
{
    byte[] bytes => bytes,                                  // already bytes — as-is
    string str   => Encoding.UTF8.GetBytes(str),            // string → UTF-8
    null         => Array.Empty<byte>(),                    // null → empty
    _            => Encoding.UTF8.GetBytes(body.ToString() ?? string.Empty)  // else → ToString → UTF-8
};

Then headers. ContentType rides as a content-type header, and the exchange's user headers move into Kafka headers — except the internal redbKafka.* ones, which are filtered out:

foreach (var (key, value) in exchange.In.Headers)
{
    if (KafkaHeaders.IsRedbHeader(key))   // redbKafka.* are internal — don't propagate
        continue;
    msg.Headers.Add(key, Encoding.UTF8.GetBytes(value?.ToString() ?? string.Empty));
}

The partition key

DetermineKey — three branches, in priority order:

private string? DetermineKey(IExchange exchange)
{
    // 1. Explicit partition → no key (partitioner is bypassed)
    if (_options.PartitionNumber.HasValue)
        return null;

    // 2. Key from the option — supports ${...} expressions, resolved per message
    if (!string.IsNullOrWhiteSpace(_options.Key))
    {
        var resolved = _options.ResolveOption(_options.Key, exchange);
        if (!string.IsNullOrEmpty(resolved)) return resolved;
    }

    // 3. Otherwise — keyless (round-robin across partitions)
    return null;
}

So key=${header.orderId} means "partition by orderId" — all events for one order land in one partition and keep their order.

Send: immediate vs deferred

Here's the fork that matters for transactions:

if (_options.Transacted)
{
    // Deferred send — the real publish happens when the route transaction commits
    var action = new KafkaSendAction(_producer, _endpoint.TopicName, message,
        _options.PartitionNumber, _options.RecordMetadata, exchange, Logger);
    RegisterTransactedAction(exchange, $"kafka-send-{Guid.NewGuid():N}", action);
}
else
{
    // Immediate send
    var result = _options.PartitionNumber.HasValue
        ? await _producer.ProduceAsync(new TopicPartition(_endpoint.TopicName,
              new Partition(_options.PartitionNumber.Value)), message, ct)
        : await _producer.ProduceAsync(_endpoint.TopicName, message, ct);

    if (_options.RecordMetadata)
        AddDeliveryMetadata(exchange, result);   // redbKafka.Sent.Topic/Partition/Offset/Timestamp
}

In transactional mode the producer doesn't send right away — it parks a KafkaSendAction in the deferred-action bag under a unique key kafka-send-{guid}. The real ProduceAsync happens later, at the .Transacted() boundary. More on that below.

KafkaSendAction clones the message at construction (a deep header copy via Buffer.BlockCopy) so the deferred send doesn't depend on what happens to the exchange afterwards:

public async Task Commit(CancellationToken ct = default)
{
    var result = _partition.HasValue
        ? await _producer.ProduceAsync(new TopicPartition(_topicName, new Partition(_partition.Value)), _message, ct)
        : await _producer.ProduceAsync(_topicName, _message, ct);
    if (_recordMetadata) { /* write the Sent.* headers */ }
}

public Task Rollback(CancellationToken ct = default)
{
    // The message is simply discarded — nothing was published
    return Task.CompletedTask;
}

Tracing across the broker

Before sending, the producer injects W3C trace context (traceparent/tracestate) into Kafka headers via the standard DistributedContextPropagator:

var propagator = DistributedContextPropagator.Current;
propagator.Inject(activity, headers, static (carrier, key, value) =>
{
    if (carrier is Headers h && !string.IsNullOrEmpty(value))
    {
        h.Remove(key);
        h.Add(key, Encoding.UTF8.GetBytes(value));
    }
});

The consumer on the other end lifts it back out — and your trace shows an unbroken chain across the broker. Zero configuration on your side.

5. The Kafka consumer: polling, batches, rebalance

KafkaConsumer is a DrainableConsumer (it can gracefully drain in-flight messages on stop). Step by step.

Start: subscribe/assign, seek, metadata

protected override Task OnStarting(CancellationToken ct)
{
    var config = _options.BuildConsumerConfig(_endpoint.ResolvedFactory);

    _consumer = new ConsumerBuilder<string, byte[]>(config)
        .SetValueDeserializer(Deserializers.ByteArray)
        .SetErrorHandler((_, e) => { /* fatal → LogError, else LogWarning */ })
        .SetPartitionsAssignedHandler((c, partitions) => { /* log assigned */ })
        .SetPartitionsRevokedHandler((c, partitions) =>
        {
            // Commit current offsets before partitions are revoked
            try { c.Commit(partitions); }
            catch (KafkaException) { /* nothing to commit — fine */ }
        })
        .SetPartitionsLostHandler((_, partitions) => { /* log involuntary loss */ })
        .Build();

    if (_options.PartitionNumber.HasValue)
        _consumer.Assign(new[] { new TopicPartition(_endpoint.TopicName, new Partition(_options.PartitionNumber.Value)) });
    else
        _consumer.Subscribe(_endpoint.TopicName);

    HandleSeekTo();        // seekTo=beginning/end on first start
    LogTopicMetadata();    // dump: partitions, leaders, replicas, ISR
    return Task.CompletedTask;
}

Two membership modes: Subscribe(topic) (dynamic group assignment + rebalance) or an explicit Assign(partition) when partitionNumber is set (static assignment, no group rebalance).

LogTopicMetadata spins up an AdminClient at start and logs the topic topology — partition count, each partition's leader, replicas, ISR. Handy for "why did I only get assigned 2 of 6 partitions" debugging.

The poll loop

RunAsync kicks off a long-running task with the poll loop:

private async Task PollLoop(CancellationToken pollCt, CancellationToken processingCt)
{
    while (!pollCt.IsCancellationRequested)
    {
        try
        {
            if (_options.MaxPollRecords > 0)
                await ProcessBatch(pollCt, processingCt);     // batch mode
            else
                await ProcessSingleMessage(pollCt, processingCt);  // one at a time
        }
        catch (OperationCanceledException) { break; }
        catch (Exception ex)
        {
            Logger?.LogError(ex, "Error in Kafka poll loop for topic {Topic}", _endpoint.TopicName);
            try { await Task.Delay(1000, pollCt); } catch (OperationCanceledException) { break; }
        }
    }
}

Caught an exception in the loop? Log it, wait a second, carry on (one bad message doesn't kill the consumer).

Single mode

private async Task ProcessSingleMessage(CancellationToken pollCt, CancellationToken processingCt)
{
    var result = _consumer!.Consume(pollCt);
    if (result?.Message is null) return;

    var exchange = CreateExchange(result);
    IncrementInflight();
    try
    {
        // Register the offset commit as a deferred action — for the transactional-route case.
        var commitAction = new KafkaCommitAction(_consumer, result, Logger);
        RegisterTransactedAction(exchange, $"kafka-commit-{result.Offset.Value}", commitAction);

        await Processor.Process(exchange, processingCt);
        ProcessedCount++;

        // 3.2.1: auto-commit after success — unless a transaction already committed it (Committed flag).
        if (_options.EnableAutoCommit && !commitAction.Committed)
            await commitAction.Commit(processingCt);
    }
    finally
    {
        await exchange.DisposeAsync();
        DecrementInflight();
    }
}

The key detail: the consumer registers a KafkaCommitAction before processing. So the offset isn't committed immediately — it commits when the route transaction commits, alongside any deferred sends. If processing throws, the offset isn't committed and the message comes back.

KafkaCommitAction.Commit is a plain _consumer.Commit(result):

public Task Commit(CancellationToken ct = default)
{
    _consumer.Commit(_result);   // ← plain offset commit, NOT SendOffsetsToTransaction
    return Task.CompletedTask;
}
public Task Rollback(CancellationToken ct = default) => Task.CompletedTask;  // no commit → message is re-read

Batch mode

With maxPollRecords > 0, the consumer collects up to maxPollRecords records, or until the pollTimeoutMs deadline, and hands them off as one exchange whose body is a List<IMessage>:

while (batch.Count < _options.MaxPollRecords && DateTime.UtcNow < deadline && !pollCt.IsCancellationRequested)
{
    var result = _consumer!.Consume(TimeSpan.FromMilliseconds(100));
    if (result?.Message is not null) batch.Add(result);
}
if (batch.Count == 0) return;

var exchange = CreateBatchExchange(batch);   // Body = List<IMessage>, redbKafka.BatchSize = N
// Commit only the last offset of the batch
var last = batch[^1];
RegisterTransactedAction(exchange, $"kafka-batch-commit-{last.Offset.Value}", new KafkaCommitAction(_consumer!, last, Logger));
await Processor.Process(exchange, processingCt);

Subtlety: only the last offset is committed (Kafka offsets are monotonic per partition, so committing the last covers the whole batch). Downstream you can .Split(body => body) and handle each message individually — but the commit is still one per batch.

How the exchange gets its metadata

CreateExchange sets the body (byte[]), transfers Kafka headers (UTF-8 decoded), restores ContentType, and stamps metadata:

message.Headers[KafkaHeaders.Topic]     = result.Topic;
message.Headers[KafkaHeaders.Partition] = result.Partition.Value;
message.Headers[KafkaHeaders.Offset]    = result.Offset.Value;
if (result.Message.Timestamp.Type != TimestampType.NotAvailable)
    message.Headers[KafkaHeaders.Timestamp] = result.Message.Timestamp.UtcDateTime;
if (!string.IsNullOrEmpty(result.Message.Key))
    message.Headers[KafkaHeaders.Key] = result.Message.Key;

var exchange = Exchange.Create(message, _endpoint.ScopeFactory);
exchange.Pattern = ExchangePattern.InOnly;   // a consumer is InOnly

What the consumer does while a message travels the route

Let's walk ProcessSingleMessage step by step, because it also explains the commit story:

Consume(pollCt) — pulled one record (in single mode the poll is blocking: the next record isn't pulled until the current one finishes);
CreateExchange — built the exchange;
IncrementInflight() — marked the message in-flight (this is what DrainableConsumer needs for a graceful stop: on shutdown it drains all in-flight messages);
registered a KafkaCommitAction in TRANSACT_ACTION — not committed yet;
await Processor.Process(exchange) — ran the message through the whole route (synchronously awaited);
ProcessedCount++;
if EnableAutoCommit (default true) and the offset wasn't already committed by a transaction — commits it inline right here;
finally: exchange.DisposeAsync() + DecrementInflight().

The key point: by default (EnableAutoCommit=true, since 3.2.1) the consumer commits the offset inline after a successful Process — at-least-once, just like RabbitMQ. The KafkaCommitAction registered in TRANSACT_ACTION is for the other case — when a transaction should own the commit: then .Transacted() commits it at its boundary, and the action's Committed flag stops the consumer from committing it a second time inline.

Batch mode is the same, except Process gets a single exchange whose body is a List<IMessage>, and the last offset of the batch is committed (inline or by the transaction).

How the offset commits: default vs transactional mode

As of 3.2.1 the default is simple and matches RabbitMQ: process it, commit it.

// ✅ Default (EnableAutoCommit=true): the offset commits inline after a successful Process.
//    A plain consumer needs no configuration — at-least-once out of the box.
From("kafka:orders?brokers=kafka:9092&groupId=w")
    .To("direct:process");

Transactional mode is for when the offset must commit together with other work (deferred sends to Kafka/Redis, a redb write) — atomically at the route boundary. Turn auto-commit off and wrap it in a transaction; the offset commit becomes part of the TRANSACT_ACTION batch.

// ✅ Transactional: the offset + deferred sends commit together at the .Transacted() boundary.
//    enableAutoCommit=false → no inline commit; the transaction owns the offset.
From("kafka:orders?brokers=kafka:9092&groupId=w&enableAutoCommit=false")
    .Transacted()
        .To("kafka:orders.done?brokers=kafka:9092&transacted=true")
    .EndTransaction();

// ✅ Imperative — the same via an explicit commit.
From("kafka:orders?brokers=kafka:9092&groupId=w&enableAutoCommit=false")
    .BeginTransaction()
    .To("direct:process")
    .CommitTransaction();

If you leave enableAutoCommit=true (the default) AND wrap in .Transacted(), the transaction wins: it commits the offset at the boundary, and the Committed flag on KafkaCommitAction stops the consumer from duplicating the commit inline. So inside a transactional route the option is effectively ignored.

By the way, in the redb.Route.Demo/Routes demos Kafka is used only as a producer (.WireTap(KafkaWireTap) to the demo-audit topic in MainPipelineRoutes.cs); there's no consumer there — so the production consume-process-produce pattern is in the transactions section.

6. Headers and tracing across the broker

The full header reference — KafkaHeaders. They all share the redbKafka. prefix:

Header	Set by	Meaning
`redbKafka.Topic`	consumer	topic of the consumed record
`redbKafka.Partition`	consumer	partition
`redbKafka.Offset`	consumer	offset in the partition
`redbKafka.Timestamp`	consumer	record timestamp
`redbKafka.Key`	consumer	key (if present)
`redbKafka.BatchSize`	consumer	batch size (batch mode)
`redbKafka.Sent.Topic`	producer	topic sent to (with `recordMetadata`)
`redbKafka.Sent.Partition`	producer	destination partition
`redbKafka.Sent.Offset`	producer	assigned offset
`redbKafka.Sent.Timestamp`	producer	send time

In a route you reach them as ${header.redbKafka.Offset}:

From("kafka:orders?brokers=kafka:9092&groupId=w")
    .Log("offset=${header.redbKafka.Offset} partition=${header.redbKafka.Partition} key=${header.redbKafka.Key}");

On re-publish these redbKafka.* headers are not propagated into the outgoing Kafka message (the IsRedbHeader filter in the producer) — so one hop's metadata doesn't leak into the next.

7. Straight talk about `transacted=true`

This is where marketing wants to say "exactly-once." Let's read the code with no spin.

What transacted=true does (from BuildProducerConfig):

if (Transacted)
{
    config.EnableIdempotence = true;
    config.Acks              = Acks.All;   // required for the idempotent producer
}

That's it: no transactional.id, and InitTransactions / BeginTransaction / CommitTransaction / SendOffsetsToTransaction appear nowhere. None of librdkafka's transactional API is in play.

What that means in practice:

"transacted" here = an idempotent producer (EnableIdempotence=true, dedup of retries within the session) + a deferred send (via KafkaSendAction, real ProduceAsync at the route's transaction boundary);
this is not Kafka EOS over consume-process-produce: there's no atomic Kafka transaction via SendOffsetsToTransaction;
a crash between the send and the offset commit yields a duplicate on restart (at-least-once), not exactly-once.

So the honest phrasing is "idempotent + route-level deferred commit," not exactly-once. For most needs — "don't lose the message, don't commit the offset before the work is done" — that's enough; true read-process-write EOS (BeginTransaction / SendOffsetsToTransaction / CommitTransaction) is a separate story, and it isn't here.

Apache Camel, by the way, doesn't give you EOS via transacted() either: it's auto-commit by default + a manual-commit hook (KafkaManualCommit) + an idempotent repository for dedup — the same model. So this isn't "Kafka-lite" — it's the same honest trade a mature integration framework makes.

With Kafka taken apart, we can put EIP patterns on it.

8. EIP #1 — Scatter-Gather: one processor, both fan-out and join

Classic Scatter-Gather from Hohpe/Woolf is "broadcast a request to N recipients and gather their responses into one." In redb.Route it's a single processor, ScatterGatherProcessor, whose aggregator is mandatory and whose parallelism is on by default.

The most stringly-typed shape — Kafka in, fan out to three services, fold, publish the result:

From("kafka:orders.incoming?brokers=kafka:9092&groupId=enricher&autoOffsetReset=earliest")
    .RouteId("order-enrich")
    .ScatterGather(
        // (accumulated, current) → merged. Called pair-wise.
        aggregationStrategy: (acc, cur) =>
        {
            var merged = (acc.In.Headers.TryGetValue("merged", out var m) ? m as string : "") ?? "";
            acc.In.Headers["merged"] = merged + cur.In.Body;
            return acc;
        },
        "http://pricing:8080/quote",
        "http://inventory:8080/check",
        "http://fraud:8080/score")
    .To("kafka:orders.enriched?brokers=kafka:9092&acks=All");

ℹ️ No transaction here, and that's fine: by default (3.2.1) the inbound offset commits inline after successful processing. You only need a transaction for an atomic "commit-offset + publish-result" coupling — then enableAutoCommit=false + .Transacted() (see §5).

A realistic note about Kafka. Scatter-Gather gathers responses, so its natural fan-out targets are request/reply endpoints: HTTP, gRPC, SQL SELECT. A Kafka producer is fire-and-forget — there's nothing to "gather" beyond delivery metadata. So Kafka lives at the edges here: source (From("kafka:...")) and sink (To("kafka:...")), while the fan-out targets services. That's how it shakes out in production.

Under the hood: the parallel path

private async Task ProcessParallel(IExchange exchange, IReadOnlyList<string> recipients, CancellationToken ct, CancellationToken callerCt)
{
    var clones = new IExchange?[recipients.Count];
    var maxDop = _maxDegreeOfParallelism > 0 ? _maxDegreeOfParallelism : Environment.ProcessorCount;
    using var semaphore = new SemaphoreSlim(maxDop);

    async Task<IExchange> SendToRecipient(int index)
    {
        await semaphore.WaitAsync(ct);
        try
        {
            var clone = exchange.Clone();                 // ← Clone(): Properties copied shallowly
            clones[index] = clone;
            var producer = GetOrCreateProducer(recipients[index]);  // per-URI producer cache
            await producer.Process(clone, ct);
            return clone;
        }
        finally
        {
            if (clones[index] != null)
                await clones[index]!.ReleaseScopes();      // release DI scopes early; body/headers live on for aggregation
            semaphore.Release();
        }
    }

    var tasks = Enumerable.Range(0, recipients.Count).Select(SendToRecipient).ToList();
    var results = await Task.WhenAll(tasks);   // (stopOnException; best-effort wraps each branch in try/catch)

    // Aggregation — in deterministic index order, not arrival order
    IExchange? aggregated = null;
    foreach (var result in results)
    {
        if (result == null) continue;
        aggregated = aggregated is null ? result : _aggregationStrategy(aggregated, result);
    }
    ApplyAggregation(exchange, aggregated);
}

A few things here are critical:

exchange.Clone() — each recipient works on its own clone. Clone() copies Properties shallowly (more on this in the transactions section), which matters for a consistent commit.
SemaphoreSlim(maxDop) — parallelism is capped. MaxDegreeOfParallelism=0 means Environment.ProcessorCount.
Aggregation in index order. Even if fraud answers first, the fold runs pricing → inventory → fraud. Predictability you can lean on.
ReleaseScopes() early — the clone's DI scopes are released right after the send, while body and headers stay alive for aggregation.
Producer cache — GetOrCreateProducer holds a ConcurrentDictionary<string, Lazy<ToProcessor>>; DisposeAsync stops every producer it created.

The sequential path

With ParallelProcessing(false) — different code, and a different clone:

foreach (var uri in recipients)
{
    var clone = exchange.CloneLinked();        // ← CloneLinked(), not Clone()
    var producer = GetOrCreateProducer(uri);
    await producer.Process(clone, ct);
    aggregated = aggregated is null ? clone : _aggregationStrategy(aggregated, clone);
}

The Clone() vs CloneLinked() difference isn't cosmetic — covered under transactions.

Error handling: best-effort vs stop-on-exception

StopOnException(false) (default, best-effort): a failed branch gets its clone.Exception set and is still handed to the aggregator — your strategy decides what to do with a branch carrying an Exception:

catch (Exception ex)
{
    if (_stopOnException) throw;
    clone.Exception = ex;
    aggregated = aggregated is null ? clone : _aggregationStrategy(aggregated, clone);
}

Timeout in best-effort: the sequential path aggregates the partial set and bails (break); the parallel path drops the timed-out branches (null). StopOnException(true) propagates the first failure, and a timeout is wrapped into a TimeoutException:

catch (OperationCanceledException) when (_timeout > TimeSpan.Zero && !ct.IsCancellationRequested)
{
    throw new TimeoutException($"Scatter-gather timed out after {_timeout}.");
}

Every parameter

The full contract — IScatterGatherDefinition:

Parameter	Default	What it does
`Recipients(params string[])`	—	Static list of recipient URIs.
`Recipients(Func<IExchange, IEnumerable<string>>)`	—	Dynamic — computed from the live message.
`AggregationStrategy(...)`	required	Pair-wise `(acc, cur) → merged`. Missing it fails the build.
`ParallelProcessing(bool)`	`true`	Parallel or sequential.
`MaxDegreeOfParallelism(int)`	`0` → ProcessorCount	Concurrency cap.
`StopOnException(bool)`	`false`	best-effort or fail-fast.
`Timeout(TimeSpan)`	`Zero` (none)	Whole-operation deadline.

The fully-knobbed fluent form:

.ScatterGather(sg => sg
    .Recipients("http://pricing:8080/quote", "http://inventory:8080/check", "http://fraud:8080/score")
    .AggregationStrategy((acc, cur) => { /* ... */ return acc; })
    .ParallelProcessing(true)
    .MaxDegreeOfParallelism(3)
    .Timeout(TimeSpan.FromSeconds(2))
    .StopOnException(false))

Dynamic recipients

The recipient list can be computed from the message — say, a fan-out across shards whose numbers arrived in a header:

.ScatterGather(sg => sg
    .Recipients(e =>
    {
        var shards = (e.In.Headers["shards"] as string ?? "").Split(',');
        return shards.Select(s => $"http://shard-{s.Trim()}:8080/query");
    })
    .AggregationStrategy((acc, cur) => MergeJson(acc, cur)))

That's essentially a Dynamic Recipient List inside Scatter-Gather — exactly what EIP primitives exist for.

A real production example: one HTTP request → six parallel aggregations

Enough synthetic snippets — here's a real production route (a logistics monitoring dashboard). The POST /api/tsum/routes endpoint returns a page of routes plus five different aggregate blocks for the dashboard widgets. That used to be six sequential database round-trips; here it's a single Scatter-Gather:

From("http:0.0.0.0:5090/api/tsum/routes?inOut=true&cors=true")
    .RouteId("tsum-api-routes")
    .Process(Auth.ProcessAsync)                 // authentication
    .ConvertBody<string>()
    .Process(ParseAndStashFilter)               // parse the filter → into Properties
    .ScatterGather(sg => sg
        .Recipients(
            "direct://routes-page",             // the result page (pagination)
            "direct://routes-ownership",        // aggregate: owned/hired × load status
            "direct://routes-route-status",     // aggregate: by route status
            "direct://routes-point-status",     // aggregate: by point status
            "direct://routes-territory",        // aggregate: vehicles on territory
            "direct://routes-departure")        // aggregate: departure stats
        .AggregationStrategy(MergeRouteFragments)
        .ParallelProcessing(true)
        .MaxDegreeOfParallelism(4)
        .StopOnException(true)
        .Timeout(TimeSpan.FromSeconds(30)))
    .Process(ComposeRoutesResponse);            // assemble the final JSON from the fragments

Each branch is its own direct:// route with its own redb query that drops its slice of the response into a frag:* property (plus its own timing metric):

From("direct://routes-ownership")
    .ProcessWithRedb(async (redb, ex, ct) =>
    {
        var sw = Stopwatch.StartNew();
        var query = await BuildFilteredQuery(redb, Filter(ex));   // the same filter every branch uses
        var groups = await query
            .GroupBy(r => new { r.Own, r.LoadStatus })            // server-side aggregation in redb
            .SelectAsync(g => new { g.Key.Own, g.Key.LoadStatus, Count = Agg.Count(g) });

        ex.Properties["frag:ownership"]      = BuildOwnershipFragment(groups);
        ex.Properties["metric:ownershipMs"]  = sw.ElapsedMilliseconds;
    });

The aggregator just folds each branch clone's frag:* and metric:* into the accumulator:

private static IExchange MergeRouteFragments(IExchange aggregated, IExchange current)
{
    foreach (var kv in current.Properties)
        if (kv.Key.StartsWith("frag:") || kv.Key.StartsWith("metric:"))
            aggregated.Properties[kv.Key] = kv.Value;   // branch fragment → into the shared result
    return aggregated;
}

and ComposeRoutesResponse builds the final JSON out of the six frag:* pieces.

Why this is "very fast." The six branches run in parallel (cap of 4 at a time). The endpoint's latency is the slowest single branch, not the sum of six. If each aggregation is ~80–150 ms, sequentially you'd be at ~0.6–0.9 s; through Scatter-Gather it's ~150 ms. One request from the frontend → one HTTP hop → six server-side redb aggregations at once → one JSON.

And here everything we covered about transactions lands in practice: each branch's ProcessWithRedb spins up its own per-exchange redb scope → its own connection (see §10). Six parallel queries on six connections, not sharing one transaction — which is exactly why it doesn't fall over. No .Transacted() is needed here: the branches only read, and the join is frag:* in memory. StopOnException(true) means "no half-built widget" — if any branch fails, the whole response is an error, not partial data.

9. EIP #2 — Aggregator: assembly over time

Scatter-Gather joins responses right now (fan-out → join). The Aggregator is a different pattern: it collects independent messages over time by a correlation key and emits the merge once a completion predicate trips.

The AggregatorProcessor contract is four things: correlationKey, aggregationStrategy, completionPredicate, target. The core:

public async Task Process(IExchange exchange, CancellationToken ct = default)
{
    var key = _correlationKey(exchange);
    IExchange? completed = null;

    lock (_lock)
    {
        if (_aggregated.TryGetValue(key, out var existing))
        {
            var merged = _aggregationStrategy(existing, exchange);
            _aggregated[key] = merged;
            if (_completionPredicate(merged))     // is the group complete?
            {
                completed = merged;
                _aggregated.Remove(key);
            }
        }
        else
        {
            _aggregated[key] = exchange;          // first in the group
            if (_completionPredicate(exchange)) { completed = exchange; _aggregated.Remove(key); }
        }
    }

    if (completed != null)
        await _target.Process(completed, ct);     // only completed groups flow downstream
}

A live example from the EipRoutes.cs demo — collect 3 events sharing a batchId:

From("timer://agg-source?period=2000&repeatCount=9")
    .RouteId("demo-aggregator")
    .SetHeader("batchId", e => $"batch-{DateTime.UtcNow.Second % 3}")
    .SetBody(e => $"event-{DateTime.UtcNow:ss.fff}")
    .Aggregate(
        correlationKey:      e => GetHeader(e, "batchId") ?? "default",
        aggregationStrategy: (oldEx, newEx) =>
        {
            oldEx.In.Body = $"{oldEx.In.Body} + {newEx.In.Body}";
            var count = oldEx.Properties.TryGetValue("agg.count", out var c) ? (int)c! : 1;
            oldEx.Properties["agg.count"] = count + 1;
            return oldEx;
        },
        completionPredicate:  e => e.Properties.TryGetValue("agg.count", out var c) && (int)c! >= 3)
    .Log("[AGG] done — merged 3 events: ${body}");   // runs on the completed group

.Aggregate(...) opens a scope (AggregateDefinition) — the steps after it (.Log(...)) build the target pipeline that runs on completed groups. Pre-completion messages are consumed silently.

On Kafka it's the classic "collect N by key":

From("kafka:order-lines?brokers=kafka:9092&groupId=order-assembler")
    .Aggregate(
        correlationKey:      e => e.In.Headers["redbKafka.Key"]?.ToString() ?? "x",  // key = orderId
        aggregationStrategy: (acc, cur) => AppendLine(acc, cur),
        completionPredicate: e => IsOrderComplete(e))
    .To("kafka:orders.assembled?brokers=kafka:9092&acks=All");

ℹ️ By default the inbound offset commits after processing. For an atomic "offset + publish-assembled-result" coupling, use enableAutoCommit=false + .Transacted().

The hard limitation, said out loud

The group store is a plain Dictionary<string, IExchange> behind a lock:

private readonly Dictionary<string, IExchange> _aggregated = new(StringComparer.Ordinal);
private readonly object _lock = new();

From that follows:

it's lost on a process restart — all in-flight groups vanish;
there is no timeout / eviction — only the completion predicate. A group that never reaches its condition (you wait for 3 events, 2 arrived, the source died) sits in memory forever. A potential leak if you have many keys that don't always complete.

This is not the persistent, recovery-capable aggregator from "big" Camel (completion timeout, persistent repository, recovery). For "collect N by key and emit a batch, a deploy losing in-flight state is acceptable" — great. For "buffer for a day, survive a restart" — you need persistence that isn't here. Honestly. There's a PendingGroupCount for monitoring how many groups are pending — at least you can see the pile growing.

10. Transactions: two models, and that's the whole point

Here's the fork to internalize. redb.Route has two distinct transaction models, and the connectors split into two camps. Getting this fork straight is half of using the framework well.

Camp 1: deferred actions (`ITransactedAction`) — they do NOT touch `System.Transactions`

This is every broker plus redb: Kafka, Redis, RabbitMQ, AMQP, IBM MQ, Azure Service Bus, and redb itself. One mechanic for all. The transport doesn't do the "real" work immediately — it parks an ITransactedAction in the shared bag at exchange.Properties["TRANSACT_ACTION"] (a ConcurrentDictionary) under a per-message-unique key:

private static void RegisterTransactedAction(IExchange exchange, string key, ITransactedAction action)
{
    if (!exchange.Properties.TryGetValue("TRANSACT_ACTION", out var raw) ||
        raw is not ConcurrentDictionary<string, ITransactedAction> dict)
    {
        dict = new ConcurrentDictionary<string, ITransactedAction>(StringComparer.OrdinalIgnoreCase);
        exchange.Properties["TRANSACT_ACTION"] = dict;
    }
    dict[key] = action;
}

Keys are kafka-send-{guid}, redis-write-{guid}, redb:{name}, an offset, a deliveryTag. Uniqueness is critical: parallel fan-out branches write into one shared bag (Clone() copies Properties shallowly → all clones share the dictionary), and without unique keys they'd clobber each other.

At the .Transacted() boundary, TransactedProcessor takes over:

public async Task Process(IExchange exchange, CancellationToken ct = default)
{
    if (!exchange.Properties.ContainsKey(TransactActionPropertyKey))
        exchange.Properties[TransactActionPropertyKey] = new ConcurrentDictionary<string, ITransactedAction>(...);

    using var scope = _policy.CreateScope();   // System.Transactions.TransactionScope (AsyncFlow enabled)
    try
    {
        await _inner.Process(exchange, ct);
        await CommitActions(exchange, ct);      // commit ALL deferred actions
        scope.Complete();
    }
    catch (Exception ex)
    {
        await RollbackActions(exchange, ct);    // on error — roll them all back
        throw;
    }
}

CommitActions walks the bag sequentially and commits each action — which bites us in the trade-off below:

foreach (var kvp in actions)
    await kvp.Value.Commit(ct);

These transports never touch Transaction.Current. So parallel fan-out over them is safe: unique keys → no bag collisions → no concurrent enlistment in a single System.Transactions transaction.

redb joins this camp via RedbTransactedAction — it wraps a redb-native IRedbTransaction (its own BEGIN/COMMIT on its own connection) and drops it into the same bag under key redb:{name}:

public async Task Commit(CancellationToken ct = default)
{
    if (Interlocked.Exchange(ref _completed, 1) != 0) return;  // single-use
    try { if (_tx.IsActive) await _tx.CommitAsync(); }
    finally { await _tx.DisposeAsync(); }
}

Camp 2: SQL — it enlists in `System.Transactions`

The SQL connector behaves fundamentally differently (SqlProducer):

var connection = await factory.CreateConnectionAsync(readOnly: ..., ct);

// If an ambient TransactionScope exists (route-level .Transacted()), the connection auto-enlists —
// no local transaction needed. Otherwise always wrap in a local one (like EF SaveChanges).
var hasAmbientTx = Transaction.Current != null;
DbTransaction? tx = null;
if (!hasAmbientTx)
    tx = await connection.BeginTransactionAsync(ct);
// ... execute ...
if (tx != null) await tx.CommitAsync(ct);

So:

no ambient → each execution is wrapped in a local DbTransaction on its own connection (atomic, like EF's SaveChanges, with no ?transacted=true);
an ambient Transaction.Current present (opened by .Transacted() / .BeginTransaction()) → it skips the local one and the connection auto-enlists in the ambient TransactionScope.

SQL is the one in-box connector that genuinely participates in System.Transactions. That's both a feature (one .Transacted() wrapper around several SQL writes = one atomic transaction, perfect for an outbox write) and a thing whose edges you must understand.

Why parallel Scatter-Gather doesn't blow up under a transaction

The reasonable fear: parallel branches run on different threads, Transaction.Current flows into each via ExecutionContext — won't that be concurrent use of one transaction and a promotion to MSDTC (which on PG/Linux simply throws)?

No. Here's why — from the code:

1. Scopes are created per-exchange. A named redb instance resolves through GetRedbService(name, exchange) (RedbRouteExtensions), which caches a DI scope under exchange.Properties["__redb_scope:{name}"] and pulls the service from that scope's provider:

var cacheKey = ScopeCachePrefix + cleanName;          // "__redb_scope:orders-db"
if (exchange.Properties.TryGetValue(cacheKey, out var cached) && cached is IServiceScope cachedScope)
    return cachedScope.ServiceProvider.GetRequiredService<IRedbService>();

var scope = factory.CreateScope();                    // not cached — create our own
exchange.Properties[cacheKey] = scope;
return scope.ServiceProvider.GetRequiredService<IRedbService>();

2. Clone() deliberately does NOT copy scopes. From Exchange.cs:

foreach (var kvp in _properties)
{
    // Named redb scopes are per-exchange; the child creates its own on first access.
    if (kvp.Key.StartsWith("__redb_scope:", StringComparison.Ordinal))
        continue;
    clone._properties[kvp.Key] = kvp.Value;
}

So each parallel branch spins up its own scope → its own IRedbService → its own connection.

3. redb never enlists in System.Transactions — it uses its own IRedbTransaction, parked in TRANSACT_ACTION.

Net result: there is no single System.Transactions transaction being hit concurrently by multiple connections → no MSDTC promotion, no "transaction context in use by another thread," no crash. Different connections, not one transaction — that's the design, not a bug.

Same story for SQL. In a Scatter-Gather without a .Transacted() wrapper, each branch opens its own connection (CreateConnectionAsync per Process) and its own local DbTransaction → independent atomic writes → no crash. The only way to hit the multi-connection-enlistment-to-MSDTC crash is to deliberately wrap a parallel fan-out in .Transacted() (an ambient System.Transactions scope) and fan out to several SQL connections — on PG/Npgsql that escalates to a distributed transaction and throws. A narrow edge nobody walks into, because the deferred model exists for brokers/redb.

So what is the 3.2.0 `DependentTransactionBranch` for, then?

That 3.2.0 per-branch isolation fix is wired only into Multicast and Splitter (plus the helper itself) — nowhere else. Here's its core (DependentTransactionBranch):

internal static async Task RunAsync(Func<Task> branch)
{
    var ambient = Transaction.Current;
    if (ambient is null) { await branch(); return; }   // no transaction — zero overhead

    // Fork a dependent clone: this branch's enlistment is private to its thread.
    var dependent = ambient.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
    using (var scope = new TransactionScope(dependent, TransactionScopeAsyncFlowOption.Enabled))
    {
        await branch();
        scope.Complete();
    }
    dependent.Complete();   // release the parent commit (it waited on BlockCommitUntilComplete)
}

It matters only for resources that genuinely enlist in Transaction.Current — i.e. when parallel Multicast/Splitter branches do inline work on an enlisting connection. Scatter-Gather doesn't need it: per-exchange scopes + deferred ITransactedAction already give isolation without sharing a connection. Not a missing-feature asymmetry — two mechanisms for two situations.

The honest trade-off (a design choice, not a bug)

TransactedProcessor.CommitActions commits the deferred actions sequentially in a loop, and Scatter-Gather/Multicast branches are different connections, different transactions. So: if branch A commits and branch B's commit throws, you get a partial commit. No cross-branch DB atomicity here. That's the price of "no crash, no MSDTC promotion." If you need real cross-branch atomicity with enlistment, that's Multicast/Splitter under .Transacted() with the 3.2.0 fix — not Scatter-Gather.

Transaction policies

.Transacted() is backed by TransactionPolicy — four ready-made:

Policy	`ScopeOption`	Behavior
`Default`	`Required`	Join the ambient or create a new one. 30s timeout, `ReadCommitted`.
`RequiresNew`	`RequiresNew`	Always a new transaction; the ambient is suspended.
`Suppress`	`Suppress`	Run with no transaction (ambient suppressed).
`Mandatory`	(marker)	Requires an existing ambient, else `InvalidOperationException` at scope creation.

TransactionDefinition also gives Camel-parity hooks: .Retry(attempts, delay) (wraps the body in a RetryProcessor) and .DeadLetterChannel(uri) (on failure after rollback, send the exchange to the DLC). So a transactional block with retries and dead-letter is:

From("kafka:orders?brokers=kafka:9092&groupId=w")
    .Transacted()
        .Retry(3, TimeSpan.FromMilliseconds(200))
        .To("sql:INSERT INTO orders ...")
        .To("kafka:orders.done?brokers=kafka:9092&transacted=true")
    .EndTransaction();

11. The parallel cousins: Splitter and Multicast

Scatter-Gather isn't the only fan-out in the family. Next to it sit Splitter (split the body into parts, process each) and Multicast (send a copy to N processors). Both, like Scatter-Gather, have an optional aggregation, parallelism, and stopOnException. The difference is where the "branches" come from:

Scatter-Gather: branches = endpoints (producer URIs), aggregator mandatory.
Multicast: branches = processors / sub-pipelines, aggregator optional (MulticastProcessor).
Splitter: branches = body parts (SplitterProcessor), aggregator optional, with some Camel-compat nuances.

Splitter is actually the most feature-rich on aggregation: its strategy (IExchange?, IExchange) → IExchange is called even for the first part with oldExchange == null (Camel's seed/wrap contract), and there are flags parallelAggregate (aggregate inline under a lock from workers instead of a deterministic post-pass) and aggregateOnException (include failed parts in the aggregate). And it's Splitter/Multicast's parallel path that carries the DependentTransactionBranch.RunAsync:

// MulticastProcessor.ProcessParallel / SplitterProcessor.ProcessParallel
await DependentTransactionBranch.RunAsync(() => _targets[idx].Process(clone, ct));

Why they have it and Scatter-Gather doesn't — covered above: Splitter/Multicast run inline processors (which can open an enlisting connection right in the branch), whereas Scatter-Gather runs producers by URI with per-exchange scopes. Different risks, different mechanisms.

A simple Multicast from the demo:

From("direct://demo-multicast")
    .Multicast(new[] { "direct://mcast-a", "direct://mcast-b", "direct://mcast-c" }, parallelProcessing: true)
    .Log("[MCAST] ◀ All endpoints received the message");

12. Saga — "a slightly different one"

redb.Route has a saga, but it's not MassTransit's durable state machine. It's SagaProcessor — an in-process compensating saga within a single exchange:

public async Task Process(IExchange exchange, CancellationToken ct = default)
{
    var completedCount = 0;
    try
    {
        for (var i = 0; i < _steps.Length; i++)
        {
            ct.ThrowIfCancellationRequested();
            await _steps[i].Action(exchange, ct);
            completedCount++;
        }
    }
    catch (Exception ex) when (ex is not OperationCanceledException)
    {
        // Compensate completed steps in REVERSE order
        for (var i = completedCount - 1; i >= 0; i--)
        {
            if (_steps[i].Compensate is null) continue;
            try { await _steps[i].Compensate!(exchange, ct); }
            catch (Exception compEx) { _logger?.LogError(compEx, "Saga compensation for step {i} failed", i); }
        }
        throw;
    }
    if (_onCompletion is not null) await _onCompletion(exchange, ct);
}

Compensation failures are logged and don't abort the rest of the rollback. The DSL (SagaDefinition) comes in two styles, callback and fluent-scope:

From("direct:checkout")
    .Saga(s => s
        .Step(reserve, compensate: unreserve)
        .Step(charge,  compensate: refund)
        .Step(ship)                              // forward-only, no compensation
        .OnCompletion(e => Log("order placed")));

Bluntly: no state persistence, no correlation across messages/time, does not survive a process restart. It's Camel-style "routing slip with compensation," not a persisted state machine. Only the name overlaps. There are metrics, by the way: SagaCompleted / SagaCompensated / SagaFailed.

Again — not a shortcoming, a different school. Need a durable saga with persistence over Kafka? Build it from Aggregator + a SQL/Redis store + a correlation key. The bricks are there.

13. Outbox — there isn't one, and that's correct

Grep across src for Outbox — zero. No built-in outbox. That's not a gap, it's a stance: the transactional outbox is a pattern, not a framework button. Baking it into the core is over-engineering; anyone who wants one wires their own in five lines for their own invariants:

// 1. Write: business data + an outbox row under one transaction.
//    For SQL this is exactly where the System.Transactions enlistment works FOR you:
//    one .Transacted() wrapper → one ambient scope → atomic.
From("direct:place-order")
    .Transacted()
        .To("sql:INSERT INTO orders(id, payload) VALUES (@id, @payload)")
        .To("sql:INSERT INTO outbox(id, topic, payload, sent) VALUES (@id, 'orders', @payload, false)")
    .EndTransaction();

// 2. Deliver: a separate poller route reads the outbox and publishes.
From("sql:SELECT * FROM outbox WHERE sent = false ORDER BY id?outputType=SelectList&delay=1000")
    .Split(body => (IEnumerable<object?>)body)        // one message per row
        .To("kafka:orders?brokers=kafka:9092")
        .To("sql:UPDATE outbox SET sent = true WHERE id = @id");

Want the inbox pattern, dedup by messageId, a TTL on rows, a claim-check for big payloads? Add it. Nobody forces their outbox schema or retry semantics on you. That is "moving toward Camel": less baked-in magic, more explicit assembly.

14. Bottom line: a trade, not "better/worse"

	MassTransit	redb.Route (Camel school)
Saga	durable state machine, persisted, survives restarts	in-process compensation within an exchange
Outbox	a framework feature	your own route from SQL/Redis in five lines
Transactions	a framework abstraction	two explicit models: deferred `ITransactedAction` (brokers+redb) / `System.Transactions` (SQL)
Kafka "transactional"	EOS out of the box	idempotent producer + deferred commit (not EOS)
Philosophy	batteries included, do it their way	EIP + connectors + DSL, compose it yourself

This isn't "redb.Route beats MassTransit." It's a different trade: less baked-in magic, more explicit assembly from primitives. If your scenarios don't fit someone else's saga/outbox model, the Camel approach lets you assemble exactly yours. If they do fit, maybe you don't need to leave at all.

Scatter-Gather, for the record, has been rock-solid for us in production — one processor that fans out and joins, in parallel, with a deterministic merge order and transaction semantics you can actually reason about. Exactly the case where an EIP primitive solves the real problem without a single line of infrastructure code.

We run 3.2.0 in production — CHANGELOG here. War stories, things to re-verify (looking at you, transacted Kafka), which EIP to dissect next — drop them in the comments and we'll dig through the code together.

If this was useful — a ⭐ on GitHub helps others find it.

SQLite provider anatomy: moving away from EF Core — typed object storage for desktop, mobile, and Blazor WASM

rinat kozin — Mon, 29 Jun 2026 21:53:41 +0000

Series: redb — ecosystem, engineering teardown following the 3.2.0 SQLite announcement.

What this is

Full anatomy here, recommend a quick look at redbase.app first for context: https://redbase.app/

When we shipped the SQLite provider in redb 3.2.1, the pitch was two sentences: same LINQ, one line in DI. This post is the opposite of that pitch. Not "what shipped" — how it's built and where it leaked. Concretely: how redb's query engine moved into a native C extension on a database with no stored procedures; how we store DateTimeOffset in a database that has no date type; and three real bugs from this release, each with the filter JSON, the generated SQL, and the fix.

It's long and it has code. If you want the short version, the announcement is linked above. If you want what's under "one line in DI," pull up a chair.

For context, if redb is new to you (the rest assumes you've seen this):

Two tables, zero migrations: a typed object store for .NET over Postgres/MSSQL — full LINQ, no Include — the what and why.

redb inside, part 1: the 13 tables everything runs on — the storage model. Load-bearing for this post: the SQLite provider had to reproduce it one-for-one.

Why our Free was so far behind Pro — and what we shipped in 3.0.0 (with the actual SQL) — what "Free = server-side functions" means, which is exactly what we had to port to C here.

SQLite provider for RedBase is coming — the 3.2.1 teaser — the short version of what this post takes apart.

Two disclaimers before we start, so nothing trips us up later.

One — about a word. Yes, redb's model is "flexible": a class is spread across rows in a values table. No, this is not EAV in the dismissive sense people throw that acronym around. What lives in a redb database is RTTI — real type information: schemes, structures, field types, references. The DB knows that EmployeeProps.HireDate is a DateTime, that Contacts is an array of objects, that CurrentProject is a reference to another scheme. That's a runtime type system at the storage layer, not a blind bag of tuples. You'll see below why neither the materializer nor the query compiler could physically exist without it — at every step they need the type of the thing they're materializing or filtering.

Two — about fit, so nobody's disappointed later. redb is for complex business classes: object graphs, nesting, cross-scheme references, trees, dictionaries, arrays of objects. Pumping flat, high-volume streams into it — a coordinate feed, sensor telemetry, thousands of metrics a second — technically works, but it's an anti-pattern. That's what time-series and columnar stores exist for; redb pays for its flexibility and typing exactly where the data is rich and connected, not where you've got one (timestamp, value) table at a billion rows. Short version: redb shines when you have a real domain model.

Part 0. The constraint that shapes everything

SQLite has no procedural language. No PL/pgSQL, no T-SQL — nowhere to put server-side logic. And redb's "Free" tier on Postgres and MSSql is built exactly around having that: the heavy machinery (query compiler, materializer, soft-delete, permission views) lives inside the database as server-side functions. That's literally the Free/Pro line: where JSON gets materialized and who generates the SQL.

"No stored procedures" forks into exactly two paths, and we took both — one per tier:

Pro — pure C#. ProSqlBuilder generates the query SQL in code, props materialize in code, zero DB-function calls. Consequence: it runs anywhere Microsoft.Data.Sqlite runs — including Blazor WebAssembly and mobile (MAUI/iOS/Android), where you flat-out cannot load a native SQLite extension.

Free — a native C extension. Same shape as Postgres/MSSql: the engine lives in the database. On SQLite, "in the database" means a loadable extension written in C (redb.dll / .so / .dylib) on top of sqlite3ext.h. It runs wherever native code loads: desktop, server, CI — and it lets a non-.NET host (Python, the sqlite3 CLI) talk to a redb database directly if it wants to.

Both stories at full size below. Native first (it's the fun one), then dates (the important one), then the bugs, then the traps.

Part 0.5. What the provider reproduces: `_objects` and `_values`

Before diving into the native side — what it actually reads and writes. redb's model is ~13 tables (full teardown in the "13 tables" post), but two are load-bearing for the SQLite story, and the provider had to recreate both column-for-column.

_objects is the "header" of every object: identity, tree, ownership, dates, the reference to a scheme (i.e. to a type):

CREATE TABLE _objects(
    _id             INTEGER NOT NULL PRIMARY KEY,
    _id_parent      INTEGER NULL,            -- tree (parent)
    _id_scheme      INTEGER NOT NULL,        -- RTTI: which CLASS this object is
    _id_owner       INTEGER NOT NULL,
    _id_who_change  INTEGER NOT NULL,
    _date_create    REAL    NOT NULL DEFAULT (julianday('now')),  -- UTC Julian day (REAL)
    _date_modify    REAL    NOT NULL DEFAULT (julianday('now')),  -- UTC Julian day (REAL)
    _name           TEXT    NULL,
    _hash           TEXT    NULL,            -- hash of the props set (delta/cache)
    -- slots for RedbPrimitive<T> (when Props IS the primitive, no nested structure)
    _value_long     INTEGER NULL,
    _value_string   TEXT    NULL,
    _value_bool     INTEGER NULL,            -- bool = 0/1
    _value_double   REAL    NULL,
    _value_numeric  REAL    NULL,            -- NUMERIC(38,18): REAL by default
    ...
);

_values is the row-per-property store. One row per (object, structure, [array index]). The key idea is typed column slots: a value lives not in one "universal" text column, but in the column for its type:

CREATE TABLE _values(
    _id              INTEGER NOT NULL PRIMARY KEY,
    _id_structure    INTEGER NOT NULL,       -- RTTI: WHICH field this is
    _id_object       INTEGER NOT NULL,
    _String          TEXT    NULL,
    _Long            INTEGER NULL,
    _Guid            TEXT    NULL,
    _Double          REAL    NULL,
    _DateTimeOffset  REAL    NULL,           -- DateTime/DateTimeOffset/DateOnly as UTC Julian
    _Boolean         INTEGER NULL,           -- 0/1
    _ByteArray       BLOB    NULL,
    _Numeric         REAL    NULL,
    _ListItem        INTEGER NULL,
    _Object          INTEGER NULL,           -- reference to another object
    _array_parent_id INTEGER NULL,           -- arrays/dictionaries — relational
    _array_index     INTEGER NULL
);

Two consequences that everything else rests on:

Typed slots, not "everything as a string." _Boolean is 0/1, _DateTimeOffset is REAL Julian, _Long is an integer. So comparisons and sorts in SQL run on the column's native type (and use indexes) instead of a string cast. That's the "not EAV" part: _values is a typed props store, and _id_structure → _structures carries RTTI about which field this is and what type it is. The query compiler needs the field's type at every step — to pick the slot column for MAX(...) FILTER in the pivot; without RTTI it wouldn't know which column LastName comes out of.
Arrays and dictionaries are relational, via _array_parent_id/_array_index, not a JSON blob. So the materializer assembles them with GROUP BY over the index, and the compiler can filter through them (_array_index IS NULL in the pivot is exactly what separates a field's scalar row from its array elements).

From here, "the engine in the database" = functions that read/write exactly these two tables, checking the metadata in _schemes/_structures.

Part 1. The native extension: anatomy

Entry point

A SQLite loadable extension is a .so/.dll/.dylib with one exported init function. The default name is derived from the file basename: for redb.dll, that's sqlite3_redb_init. Inside, it registers all our SQL functions:

SQLITE_EXTENSION_INIT1

int sqlite3_redb_init(sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi){
  SQLITE_EXTENSION_INIT2(pApi);
  int rc;

  rc = sqlite3_create_function(db, "get_object_json", 1, SQLITE_UTF8, 0,
                               getObjectJsonFunc, 0, 0);
  if(rc != SQLITE_OK) return rc;
  rc = sqlite3_create_function(db, "get_object_json", 2, SQLITE_UTF8, 0,
                               getObjectJsonFunc, 0, 0);   // overload: (_id, max_depth)
  if(rc != SQLITE_OK) return rc;

  rc = sqlite3_create_function(db, "save_object_json", 1, SQLITE_UTF8, 0,
                               saveObjectJsonFunc, 0, 0);
  if(rc != SQLITE_OK) return rc;

  rc = redbRegisterPvt(db);   // the whole pvt_* compiler
  return rc;
}

SQLITE_EXTENSION_INIT1/INIT2 are macros from sqlite3ext.h that rewrite direct sqlite3_* calls into calls through the pApi pointer table. The gotcha that's easy to miss: after INIT2, every SQLite API call inside the extension goes through that table. Forget INIT2 and the extension compiles fine — then segfaults on the first sqlite3_* call through a garbage pointer.

Loading — on every connection

The defining trait of loadable extensions: they are not persistent. SQLite forgets registered functions on a new connection. And Microsoft.Data.Sqlite pools connections. So you must load the extension on every connection, after PRAGMAs. Our wrapper around SqliteConnection does it: open → set foreign_keys=ON and busy_timeout → load the extension → hand it to the pool.

The binary is located by:

An explicit SqliteDataSource.NativeExtensionPath (if set in code).
The REDB_SQLITE_EXTENSION environment variable.
Otherwise redb.{dll,so,dylib} from the NuGet runtimes/<rid>/native/ folder; in dev, by walking up the directory tree to redb.SQLite/native/build.

Pro never sets a path: it doesn't need the native code, and in WASM/mobile there wouldn't be any.

Shipping the binary: why it cross-compiles for free

The extension is a loadable module, and that has a packaging payoff people don't expect: it links against nothing. sqlite3ext.h hands you the API as a pointer table resolved from the host at load time (that's what SQLITE_EXTENSION_INIT2 wires up) — so there's no libsqlite3 to find, no import library, no target sysroot. Cross-compiling collapses to "point CMake at a cross compiler":

# linux-arm64 from an x64 box, in a throwaway container — no sysroot needed
docker run --rm -v "$PWD:/work" debian:bookworm bash -c '
  apt-get update && apt-get install -y cmake make gcc-aarch64-linux-gnu
  cd /work/redb.SQLite/native
  cmake -S . -B build-linux-arm64 -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc
  cmake --build build-linux-arm64'
# → build-linux-arm64/redb.so: ELF 64-bit LSB shared object, ARM aarch64

A native lib that linked libsqlite3 would need the arm64 build of that library to link against; this one doesn't, because it only ever calls SQLite through the host's pointer table. One cross-gcc, one valid arm64 binary.

Delivery has its own twist. SQLite loads the extension by an explicit path (conn.LoadExtension(...)), not like a P/Invoke native lib the host resolves from the NuGet cache — so the file must physically sit in the app output. A RID-targeted dotnet publish -r <rid> flattens runtimes/<rid>/native/ for you; a framework-dependent build (no RID) doesn't, so the package ships a buildTransitive .targets that copies the OS-matching binary into the consumer's output, Exists-gated per RID. That's why "ship one more platform" = "produce one more redb.{so,dylib} and drop it in native/build-<rid>/" — the csproj and .targets already enumerate all five RIDs.

War story #5: the `.targets` that broke every consumer (3.2.0 → 3.2.1)

That delivery .targets is also where this release drew blood — and since this is an honest teardown, here it is. We shipped 3.2.1 with a banner comment in redb.SQLite.targets that had a row of - characters inside . XML comments cannot contain --, so MSBuild refused to even load the file: every consumer that pulled the package got

error MSB4024: An XML comment cannot contain '--', and '-' cannot be the last character.

and couldn't build at all. And it rode transitively — redb.SQLite.Pro depends on redb.SQLite, and buildTransitive assets flow to dependents, so the Pro package was just as broken.

Why didn't the 200/200 suite catch it? Because buildTransitive only imports when the package is consumed as a NuGet package. Our own solution references the projects by ProjectReference, which skips the build/ import entirely — so the malformed file sat in every green build, invisible, until the first real dotnet add package redb.SQLite. We found it the moment we built a packaged consumer (a sample app in the public repo), not a second earlier.

The fix is a one-line deletion (and yes — I reproduced the identical bug in a csproj comment while writing the fix; -- is a persistent little landmine). Because you can't overwrite a published version, the corrected packages went out as redb.SQLite / redb.SQLite.Pro 3.2.1, and the broken 3.2.1 pair was unlisted. So, to be exact about versions: the engine, the date handling, the bugs above — all 3.2.1; the two SQLite NuGet packages are 3.2.1. The unglamorous lesson: a package's build/ and buildTransitive assets are shipped code too — validate them from a packaged consumer in CI, because ProjectReference will happily lie to you.

What's inside: a materializer and a compiler

Two big subsystems live in the extension.

get_object_json(_id [, max_depth]) — a recursive materializer. Takes an _id, reads _values, assembles the JSON object that System.Text.Json then deserializes into a typed RedbObject<TProps>. It assembles everything: base fields from _objects, scalars, arrays and dictionaries (relationally, via _array_index / _array_parent_id), nested Class fields, object references, ListItem (including a ListItem that itself carries an Object). This is exactly where RTTI is required: to assemble a nested object, the materializer must know its scheme, its field types, that this field is an array and that one is a reference. Without type metadata, it'd just be a pile of rows in _values.

The pvt_* compiler — ~9k lines of PL/pgSQL logic ported into C as a SQL-string generator: pvt_build_query_sql, pvt_build_aggregate_sql, pvt_build_groupby_sql, pvt_build_window_sql, pvt_build_projection_sql, pvt_build_array_groupby_sql. The pipeline:

LINQ expression
   → (C#) facet filter as JSON + a field list
   → (native) pvt_build_query_sql(scheme, filterJson, ...)
   → a finished SQL SELECT string
   → (C#) execute it, materialize via get_object_json

So the native side is a filter-JSON-to-SQL translator. C# doesn't build the SQL itself (Pro does that) — it builds a filter JSON and asks the database to assemble the SQL. It sounds inside-out, but it's exactly what buys parity: the same filter JSON, on Postgres and on SQLite Free, runs through the same engine — just emitting different dialects.

A bit deeper on `get_object_json`: recursion, arrays, references

"Assembles JSON" sounds simple until you remember it's assembling a graph. The materializer walks the scheme's structure and, for each field, decides what it is from the RTTI in _structures:

scalar — reads the value from the matching _values slot column (for a date, wraps it in strftime so an ISO string goes out);
array/dictionary — gathers all rows with the same _array_parent_id, ordered by _array_index, into a JSON array/object (this is why the pivot uses _array_index IS NULL — to separate a field's "scalar" row from its elements);
nested Class — recurses into the subtree assembly;
reference (_Object) — by max_depth, either expands the target object (recursion with decremented depth) or leaves an id reference;
ListItem — a lookup-list entry; and a ListItem can itself carry an Object, so that's one more level of expansion.

The max_depth parameter (the second overload of the function) is a guard against infinite recursion on cyclic references and, at the same time, a budget for how deep to pull related objects into one JSON. None of these transitions is possible without RTTI: the materializer must know that this field is an array and that one is a reference, or it's just rows in _values with no meaning.

The other direction: `save_object_json` — the write path

Reading has a symmetric twin — save_object_json(json). It takes an object's JSON and, by the scheme, lays it back out into _objects (base) and _values (props). Two SQLite-specific subtleties:

Dates on write. In the incoming JSON, dates are ISO strings (that's how System.Text.Json serializes a DateTimeOffset). The native write wraps them in julianday('<iso>') so _DateTimeOffset gets a REAL Julian. It's the mirror of the read-side strftime — and exactly what got fixed this release under "fix save_object_json too": before the fix, a date went in as a string and didn't agree with the REAL column.
The props-save strategy. On Free it's PropsSaveStrategy.DeleteInsert: a save = delete the object's existing _values rows and insert the new set (ChangeTracking — a delta off _hash — is a Pro feature). Simple, predictable, no tracking layer — at the cost of extra rewrites; fine for an embedded load.

Loading the right type: the CLR registry and polymorphism

get_object_json hands back JSON — but deserialize it into what? For LoadAsync<EmployeeProps>(id) the answer is in the generic. But for polymorphic loads (GetChildren on a tree where the children are objects of different schemes; LoadDynamicObject) the type is unknown at compile time: you have to map the object's _id_scheme to the *Props class to materialize into.

This got rebuilt this same cycle into a two-layer CLR registry (we were fixing a polymorphic LoadAsync that returned the wrong type out of nowhere):

a global scheme name ↔ Type index — self-healing: subscribed to AppDomain.AssemblyLoad, so loading an assembly with new *Props bumps a "generation" and the index rebuilds lazily (no "register all your types at startup");
a per-domain scheme_id → Type cache — where "domain" = SHA256(sanitized connection string) or an explicit CacheDomain. Per-domain partitioning matters because one process can hold several databases (including several SQLite files), and scheme_id=1000010 in one is not the same class as in another.

This ties straight to SQLite: the materializer hands back JSON + _id_scheme, and the registry turns _id_scheme into a Type to deserialize a polymorphic child into. Get it wrong and a tree of mixed-type children all materializes into one (wrong) type.

The Postgres → SQLite phrasebook

A port isn't "retype it in C," it's a dialect translation. The most frequent swaps (all real lines from SqliteDialect/the native side):

Postgres	SQLite	where
`array_agg(x) FILTER (WHERE …)`	`json_group_array(x) FILTER (WHERE …)`	pivoting arrays
`x = ANY($1)`	`x IN (SELECT value FROM json_each($1))`	IN-lists (SQLite can't bind array params)
`EXTRACT(year FROM x)`	`CAST(strftime('%Y', x) AS INTEGER)`	date parts
`col ILIKE $1`	`col LIKE $1 ESCAPE '\'`	case-insensitive LIKE
`DELETE … WHERE _id = ANY($1)`	`DELETE … WHERE _id IN (SELECT value FROM json_each($1))`	delete-by-id-list
`DISTINCT ON (col)`	`ROW_NUMBER() OVER (PARTITION BY col)` + `WHERE rn=1`	DistinctBy (its own section below)

That ESCAPE '\' deserves a sentence, because it's a classic landmine. Postgres uses \ as the default LIKE escape character. SQLite (and MSSQL) do not. Meanwhile UserProviderBase escapes user input with a backslash (50% → 50\%), assuming PG semantics. On SQLite, without an explicit ESCAPE '\', that escaped \% starts matching a literal backslash followed by anything — a silent search corruption. So the SQLite dialect always emits LIKE with an explicit ESCAPE '\'. Small thing, easy half-day to lose.

Minimum version, and why

SQLite 3.44.0+ (Nov 2023). We deliberately lean on FILTER (WHERE …), modern window functions, RETURNING, JSON1, and recursive CTEs. The goal is to keep the SQLite SQL structurally close to the Postgres SQL, not rewrite it from scratch. The closer the dialects, the fewer places they diverge in behavior (not just syntax) — and behavioral divergences are the ones you find in prod, not in the compiler.

Identifiers: AUTOINCREMENT instead of sequences

SQLite has no sequences. redb needs globally-unique ids that both the .NET key generator and the native side (for non-.NET hosts) can hand out. The solution: a native AUTOINCREMENT table with sqlite_sequence as a shared high-water mark. Both the C extension and the C# key generator advance the same counter and reserve id blocks from it. The result: ids are unique no matter who handed them out — .NET or a Python process writing to the same file.

War story #1: `%%` in `sqlite3_mprintf`

So that "porting PL/pgSQL to C" doesn't sound sterile — here's the kind of bug the port hands you for free.

The materializer has a column-list macro VCOLS that gets interpolated into an sqlite3_mprintf format string:

char *sql = sqlite3_mprintf("SELECT" VCOLS "FROM _values WHERE _id_structure=?1 "
                            "AND _id_object=?2 AND %s LIMIT 1", cond);

When I switched the datetime columns to emit via strftime('%Y-%m-%dT%H:%M:%fZ', _DateTimeOffset), I dutifully put that strftime right into VCOLS. Everything fell over silently: objects started loading with empty properties, while base dates were fine.

I spent half an hour looking in the wrong place. The reveal: VCOLS flows into the format string of sqlite3_mprintf, and %Y %m %d %H %M %f are format specifiers to mprintf. It started eating the cond argument at the first %Y, the formatting derailed, the SQL came out malformed, sqlite3_prepare_v2 silently returned an error code, and the props function returned empty. Base dates survived because their SELECT goes through prepare_v2 directly, no mprintf.

The fix is escaping — %% (mprintf collapses %%→% before SQLite ever sees the string):

// was: ... strftime('%Y-%m-%dT%H:%M:%fZ',_DateTimeOffset) ...   // breaks mprintf
// now:
#define VCOLS " _id,_String,_Long,_Guid,_Double, " \
  "strftime('%%Y-%%m-%%dT%%H:%%M:%%fZ',_DateTimeOffset), " \
  "_Boolean,_ByteArray,_Numeric,_ListItem,_Object,_array_parent_id,_array_index "

The lasting lesson: when you generate SQL through a printf-style formatter, any % in the data is a landmine. What makes it nasty is that the failure is silent: prepare doesn't throw, it returns an error code that's easy to skip, and it surfaces as "why are my props empty."

Part 2. SQLite has no date type. How we lived with that for a release

This is the central engineering story of the release, and it didn't exist yet at announcement time.

Three storage classes, zero date types

In SQLite, datetime is a convention over three storage classes, not a type:

TEXT — an ISO-8601 string ('2024-06-15 13:45:30').
REAL — a Julian day, a floating-point number (astronomical day count).
INTEGER — Unix epoch seconds.

A .NET DateTimeOffset doesn't drop into any of these on its own. You pick a representation and hold it everywhere: on write (the parameter binder), on read (the materializer + scalar converters), in filter comparisons, in aggregates. One miss in any of those points and dates "kind of work" but lie at the boundaries.

Why not TEXT (even though that's how it started)

The first version stored dates as ISO text. It broke exactly the way string-comparing dates breaks.

datetime('now') in SQLite returns a string with a space between date and time: 2024-06-15 13:45:30. But the literal the C# layer interpolates into a comparison arrives with a T: 2024-06-15T13:45:30. TEXT comparison in SQLite is lexicographic, byte-by-byte. Here's what happens at position 10:

'2024-06-15 13:45:30'   byte[10] = 0x20 (space)
'2024-06-15T13:45:30'   byte[10] = 0x54 ('T')
0x20 < 0x54  →  the space string is ALWAYS "less than" the 'T' string

So any "stored (with a space) ⟷ literal (with a T)" comparison goes one way always, regardless of the actual time. Range filters started lying silently. In a cluster, this once marked a live node dead: the heartbeat comparison last_seen < cutoff was "always true" because the stored last_seen (with a space) sorts lexicographically below the cutoff literal (with a T). Node's alive; monitoring thinks it's a corpse.

We could normalize the separator. But that treats the symptom: TEXT comparison stays lexicographic, and any other format drift (milliseconds, a timezone suffix, leading zeros) reopens the same hole. We had to leave strings.

The fix: REAL Julian day, everything in UTC

We moved to REAL Julian day, everything in UTC — exactly how Postgres keeps timestamptz in UTC. Three reasons this is a right choice, not "yet another convention":

1. SQLite's native functions eat the Julian number directly. julianday(), strftime(), datetime(), date() take a REAL Julian as-is, no wrapping:

sqlite> SELECT strftime('%Y-%m-%dT%H:%M:%fZ', 2460477.0732638887);
2024-06-15T13:45:30.000Z
sqlite> SELECT datetime(2460477.0732638887);
2024-06-15 13:45:30
sqlite> SELECT julianday('2024-06-15T13:45:30Z');
2460477.0732638887

So to emit a date into JSON, the materializer just wraps the column in strftime and gets ISO that System.Text.Json parses natively.

2. Comparison becomes numeric. col < X on a double is correct and unambiguous. No lexicographic surprise, because you're comparing numbers, not bytes.

3. And, crucially for prod — it's sargable (index-friendly). Here's the subtlety the whole thing was for. Wrap the column in a function — julianday(col) < X — and the index on col dies: the optimizer can't use an index over an expression of the column. But comparing the raw REAL column to a constant — col < julianday('2024-06-15') — is indexable: bare column on the left, constant on the right. So in SQL generation we put julianday(...) on the literal side, never on the column:

-- NOT this (kills the index on _date_create):
WHERE julianday(o._date_create) >= julianday('2023-01-01')
-- this (sargable):
WHERE o._date_create >= julianday('2023-01-01')

The column stores a Julian number → compare it to a Julian number computed from the literal, on the constant side.

Conversion: no magic, built-in .NET

Converting DateTime/DateTimeOffset ⟷ Julian is arithmetic on the built-in ToOADate/FromOADate. An OLE Automation date (epoch 1899-12-30) differs from a Julian day by exactly the constant 2415018.5:

internal static class SqliteJulian
{
    // Julian = OADate + 2415018.5. ToOADate/FromOADate are built-in and lossless
    // within double precision — the same precision SQLite's julianday() lives in.
    private const double OADateToJulianOffset = 2415018.5;

    // DateTimeOffset → UTC Julian. .UtcDateTime APPLIES the offset → the true UTC instant.
    public static double ToJulian(DateTimeOffset dto) => dto.UtcDateTime.ToOADate() + OADateToJulianOffset;

    // DateTime → UTC Julian. The clock value is treated as UTC per redb's contract
    // (NormalizeForStorage sets Kind=Utc without converting). ToOADate ignores Kind — they agree.
    public static double ToJulian(DateTime dt) => dt.ToOADate() + OADateToJulianOffset;

    // REAL Julian → DateTimeOffset(+00:00)
    public static DateTimeOffset FromJulian(double julian)
    {
        var utc = DateTime.SpecifyKind(DateTime.FromOADate(julian - OADateToJulianOffset), DateTimeKind.Utc);
        return new DateTimeOffset(utc, TimeSpan.Zero);
    }
}

Where it plugs in: four points

For dates not to lie, the REAL-Julian representation has to hold at every point where a value crosses the C# ⟷ SQLite boundary:

Write — the central parameter binder. Every write (base dates and props alike) goes through one place — CreateCommand in SqliteRedbConnection. There, DateTimeOffset/DateTime turn into double:

switch (param)
{
    case DateTimeOffset dto:
        // REAL Julian day (UTC) — SQLite's native date format.
        sqliteParam.Value = SqliteJulian.ToJulian(dto);
        break;
    case DateTime dt2:
        sqliteParam.Value = SqliteJulian.ToJulian(dt2);
        break;
    // ...
}

Read scalars — ConvertScalar. A value from SQLite arrives as double; for a temporal target we convert back:

if (targetType == typeof(DateTimeOffset))
    return (T)(object)(value is double jdo ? SqliteJulian.FromJulian(jdo)
        : value is DateTimeOffset d ? d
        : value is DateTime dt ? new DateTimeOffset(dt, TimeSpan.Zero)
        : DateTimeOffset.Parse(value.ToString()!, ...));

Read rows — MapRow. The same double → DateTimeOffset/DateTime/DateOnly when mapping columns to properties.

Native. get_object_json emits the date via strftime(ISO, col) (see the %% story), and pvt comparisons wrap the literal in julianday('<iso>').

Three CLR types on one column

The _values._DateTimeOffset column (REAL) serves three CLR types:

DateTimeOffset — directly.
DateTime — clock value as UTC (redb's contract).
DateOnly — UTC midnight.

(TimeOnly/TimeSpan go to _String.) The scheme's RTTI disambiguates them: the field's db-type tells the materializer what to unfold the double into.

Timezones: why `+4` resolves correctly

A recurring comment-section question: "if I write a DateTimeOffset with a +04:00 zone in LINQ, does it compare correctly against what's in the DB?" Yes, for two reasons:

C# side: ToJulian(DateTimeOffset) takes dto.UtcDateTime — and .UtcDateTime applies the offset and yields the true UTC instant. So +04:00 collapses to UTC before Julian.
Native side: even if an offset leaks into the literal, the comparison wraps it in julianday('<iso-with-offset>'), and julianday() parses the offset itself:

sqlite> SELECT julianday('2026-06-25T20:00:00+04:00') = julianday('2026-06-25T16:00:00Z');
1

Stored UTC → compared by UTC instant → it agrees with any incoming offset.

War story #2: analytics and a `FormatException` from nowhere

The trickiest part of the datetime story.

A normal object load goes through get_object_json — it emits the date as an ISO string (via strftime), and C# parses it normally. But analytics — MinRedbAsync/MaxRedbAsync, AggregateRedbAsync, windows, group-bys — bypass get_object_json. They pull the date column straight into the SELECT and hand the raw Julian number to the core converter. The converter expected a string or a DateTime. The result: a FormatException out of nowhere (and funnier still: elem.GetInt64() on a fractional 2460477.07 — because the code assumed an integer Unix timestamp).

The fix was a design fork. We could patch a SQLite hack right into the core — but redb.Core has no knowledge (and shouldn't) of Julian: that's a SQLite storage detail, and the core serves three dialects. PG/MSSql return dates fine, and dragging the word "Julian" into the shared converter would leak one provider's detail into all of them.

We did it through a neutral extension point. In the core, an optional "number → temporal type" hook, empty by default:

// redb.Core: the core does NOT know the word "Julian". Only: "if a NUMBER targets a
// date and a decoder is registered, ask it."
public static class TemporalDecoder
{
    public static Func<double, Type, object?>? NumericDecoder;

    public static bool IsTemporal(Type t) =>
        t == typeof(DateTime) || t == typeof(DateTimeOffset) || t == typeof(DateOnly);

    // Convert.ChangeType that first lets a number-to-date pass through the decoder.
    public static object ChangeType(object value, Type targetType) =>
        TryDecode(value, targetType, out var d) && d != null ? d
        : System.Convert.ChangeType(value, targetType);
}

And the SQLite provider itself registers the decoder — in SqliteDataSource's static constructor, which runs for both Free and Pro:

TemporalDecoder.NumericDecoder = static (julian, targetType) =>
{
    var dto = SqliteJulian.FromJulian(julian);
    if (targetType == typeof(DateTimeOffset)) return dto;
    if (targetType == typeof(DateOnly))       return DateOnly.FromDateTime(dto.UtcDateTime);
    return dto.UtcDateTime; // DateTime
};

Then two places in the core, through which all analytics materialization flows (for both Free and Pro, because Pro has no materializer of its own — it reuses the core converters):

JsonValueConverter — the Number branch → temporal type (group-by, window, projections).
A TemporalDecoder.ChangeType wrapper at scalar points (MinRedbAsync/MaxRedbAsync, AggregateResult.Get<T>).

PG/MSSql never return a number for a date — their NumericDecoder stays null, behavior unchanged to the byte. The constant 2415018.5 and the word "Julian" stay inside redb.SQLite, and the core stays storage-agnostic.

That, to my taste, is the shape of a right fix: the problem is local (SQLite stores dates as numbers), and the cure isn't smearing SQLite specifics across the core — it's one generic extension point that only the party who needs it ever pulls.

Part 3. The pvt compiler: how a filter becomes SQL

Since the engine is a filter-JSON-to-SQL translator, let's take apart a piece of that translator. This is the most "a database inside the database" part.

The filter JSON

C# folds a LINQ Where into a facet filter — JSON the native side understands. Say Where(e => e.LastName == "NullableTest") on a prop field gives:

{ "LastName": { "$eq": "NullableTest" } }

And WhereRedb(o => o.ParentId == null) on a base field (the 0$: marker means "this is base, not a prop"):

{ "0$:ParentId": null }

Combining the two conditions is an implicit $and over the object's keys:

{ "0$:ParentId": null, "LastName": { "$eq": "NullableTest" } }

The split: push vs residual

The key function is pvtSplitFilter. It divides the filter into two parts:

push — conditions on base fields and props that can be pushed inside the CTE (into the _objects/_values subquery).
residual — what's applied outside, on top of the assembled pivot.

That split decides which "shape" the query takes. There are three:

Shape A — pure-base flat: the filter is base-only, no props. No CTE at all: SELECT _id FROM _objects o WHERE o._id_scheme=? AND <push>.
narrow — props exist, the filter reduces to the pivot: build a _pvt_cte (pivoting the needed structures via MAX(...) FILTER (WHERE _id_structure=? AND _array_index IS NULL)), join _objects.
non-narrow — there are non-pushable checks (e.g. presence), an outer WHERE is needed.

The actual assembled SQL for Where(LastName) + WhereRedb(ParentId IS NULL) (narrow shape):

WITH _pvt_cte AS (
    SELECT v._id_object,
           MAX(v._String) FILTER (WHERE v._id_structure = 1000012 AND v._array_index IS NULL) AS "LastName"
    FROM _values v
    WHERE v._id_structure IN (1000012)
      AND v._id_object IN (
            SELECT o._id FROM _objects o
            WHERE o._id_scheme = 1000010 AND o._id_parent IS NULL   -- ← pushed base condition
          )
    GROUP BY v._id_object
)
SELECT o._id FROM _pvt_cte
JOIN _objects o ON o._id = _pvt_cte._id_object
WHERE "LastName" = 'NullableTest'                                    -- ← residual prop condition

Notice: the base condition o._id_parent IS NULL went inside the _objects subquery (push), and the prop condition on LastName stayed outside (residual). That's not incidental — it's exactly what pvtSplitFilter does, and exactly where we had a bug (next).

War story #3: a multi-key filter dropped `null`

WhereRedb(o => o.ParentId == null).Where(e => e.LastName == "X") on Free returned rows that have a parent. The IS NULL condition silently vanished — but only when combined with a prop filter. Base-only WhereRedb(o => o.ParentId == null) worked.

Diagnosis. Calling the native function directly on three filters:

-- 1) base IS NULL only — WORKS:
SELECT pvt_build_query_sql(1000010, '{"0$:ParentId":null}', ...);
→ ... WHERE o._id_scheme = 1000010 AND o._id_parent IS NULL

-- 2) base equality — WORKS:
SELECT pvt_build_query_sql(1000010, '{"0$:ParentId":5}', ...);
→ ... WHERE o._id_scheme = 1000010 AND o._id_parent = 5

-- 3) base IS NULL + prop — the base condition VANISHED:
SELECT pvt_build_query_sql(1000010, '{"0$:ParentId":null,"LastName":{"$eq":"X"}}', ...);
→ ... (only LastName in the CTE, no o._id_parent IS NULL)

So {"0$:ParentId":null} on its own is fine, but as part of multiple keys it's lost. The root is in pvtSplitFilter's multi-key branch. To split each key separately, it reassembles a single-key filter object per key via pvtSingleton:

static char *pvtSingleton(sqlite3 *db, const char *k, const char *v_json){
  sqlite3_stmt *st = 0; char *r = 0;
  sqlite3_prepare_v2(db, "SELECT json_object(?1, json(?2))", -1, &st, 0);
  // ...
}

And the loop took each key's value from json_each:

// WAS:
"SELECT key, value FROM json_each(?1)"

The trap: for a JSON null, the value column in json_each is SQL NULL. So v_json arrived as an empty string, json("") is a parse error, json_object(...) returned NULL, the singleton came out NULL → pvtSplitFilter on a NULL filter returned "nothing" → the condition silently disappeared. (The same would happen with a bare text value: json_each.value returns text without quotes, and json("NullableTest") is again a parse error.) The base-only path worked because it takes the value's type from a separate type column, where null is detected correctly.

The fix — reassemble the value into a valid JSON atom by type, in SQL, before pvtSingleton:

// NOW:
"SELECT key, CASE type "
"  WHEN 'text'  THEN json_quote(value) "   // "X" with quotes
"  WHEN 'null'  THEN 'null' "              // valid JSON null
"  WHEN 'true'  THEN 'true' "
"  WHEN 'false' THEN 'false' "
"  ELSE value END "                        // integer/real/object/array — already valid JSON
"FROM json_each(?1)"

After re-encoding, {"0$:ParentId":null} stays valid JSON, the singleton assembles, and the condition reaches push and attaches to the _objects subquery. Lesson: json_each.value is a lossy source — it drops type (null → SQL NULL, text → unquoted); if you reconstruct JSON from it, do it off the type column.

DistinctBy: emulating `DISTINCT ON` with `ROW_NUMBER()`

Postgres has DISTINCT ON (col) — "one row per value of col." SQLite has none. Pro already solved it in ProSqlBuilder via ROW_NUMBER(); on Free the native distinct_on was ignored (an explicit TODO), and DistinctBy(e => e.Department) returned duplicates.

Bringing it to parity. pvt_build_query_sql accepts a 12th argument distinct_on — but the wrapper function only read up to the 11th, so C# sent the param and the native side dropped it. The fix is threefold:

Read the 12th argument and thread it through.
Pull the distinct field into the pivot. If the field isn't mentioned in the filter/sort, it's not in the collected fields → not in the CTE → nothing to partition on. So the distinct field gets mixed into field collection (pvtCollectFields) by the same mechanism as ORDER BY.
Wrap the result in a _ranked CTE with ROW_NUMBER() and keep rn=1:

WITH _pvt_cte AS ( ... pivot Department ... ),
_ranked AS (
  SELECT o._id AS _id,
         ROW_NUMBER() OVER (PARTITION BY _pvt_cte."Department" ORDER BY o._id) AS _rn
  FROM _pvt_cte
  JOIN _objects o ON o._id = _pvt_cte._id_object
)
SELECT _id FROM _ranked WHERE _rn = 1

The partition expression resolves from field metadata: base → o.<column>, prop → _pvt_cte."<FieldName>" (the pivot column). The group representative is the row with the minimum o._id (same as Pro). All of this engages only when distinct_on is present; ordinary queries take the old path — zero regression risk for 99% of queries.

Part 4. Free vs Pro: where Pro accidentally reached into Free

Architecturally Free and Pro share the base providers (redb.Core) but diverge at materialization:

Free calls get_object_json (native) to assemble objects.
Pro materializes in C# (ProLazyPropsLoader, ProSqlBuilder) and must never call a native function.

And here a principled bug surfaced. DeleteSubtreeAsync (subtree delete) collects descendant ids via the base TreeProviderBase.CollectDescendantIds. Pro overrides the loading tree methods (GetChildren, GetPolymorphicChildren, LoadDynamicObject) with C# materialization — but it did not override CollectDescendantIds, and that one used a recipe with get_object_json:

-- Tree_SelectPolymorphicChildren — the recipe that called the native function:
SELECT o._id as ObjectId, o._id_scheme as SchemeId, get_object_json(o._id, 1) as JsonData
FROM _objects o WHERE o._id_parent = $1

On PG/MSSql this passes silently: get_object_json there is a server-side function present in every tier. On SQLite Pro the function isn't there (Pro doesn't load the native code) → a hard crash, no such function: get_object_json. And on PG/MSSql Pro it was silent waste: materializing each subtree node's full JSON only to throw the JSON away and take the _id.

The fix isn't a Pro override (that would leave the base calling get_object_json for a method that doesn't need JSON); it's removing JSON from the base method itself — it only needs an id list. We added an id-only recipe to all three dialects:

// ISqlDialect + PostgreSqlDialect / MsSqlDialect / SqliteDialect:
public string Tree_SelectChildrenIds() =>
    "SELECT o._id FROM _objects o WHERE o._id_parent = $1 ORDER BY o._name, o._id";

// CollectDescendantIds — was QueryAsync<ChildObjectInfo>(Tree_SelectPolymorphicChildren), now:
var childIds = await Context.QueryScalarListAsync<long>(Sql.Tree_SelectChildrenIds(), parentId);
foreach (var childId in childIds) { ids.Add(childId); await CollectDescendantIds(childId, ids, ...); }

Now the leak is closed at the source for every tier: Free doesn't materialize extra JSON for an id list, Pro doesn't reach into the native code, PG/MSSql Pro stop materializing for nothing. Pro source now has exactly zero get_object_json calls. And, incidentally, SQLite Pro turned out to be the perfect detector for these leaks: it crashes on any native call out of Pro — the thing PG/MSSql silently tolerate.

Also: `DeleteSubtree` and cascade

The same DeleteSubtree returned the wrong delete count. On SQLite the schema has FK _id_parent ... ON DELETE CASCADE — delete a parent and the children go by cascade. But changes() (rows-affected) does not count cascade-deleted rows. So DELETE WHERE _id IN (parent, children) could return 1 (only the parent deleted directly; children by cascade). We fixed the semantics: the method returns the size of the collected subtree (objectIds.Count), not the cascade-dependent rows-affected. On PG/MSSql (no cascade on _id_parent) it's the same number — no divergence.

Bool is an INTEGER

One more small one, surfaced in group-bys. SQLite has no boolean type — it stores 0/1 as INTEGER. In a pivot/projection, a bool value reaches the shared converter as a JSON number, and the bool branch in JsonValueConverter only caught true/string → the number 1 produced false. Group-by on a bool key collapsed (everything "false"). The fix — accept Number in the bool branch:

Type t when t == typeof(bool) => elem.ValueKind == JsonValueKind.True
    || (elem.ValueKind == JsonValueKind.Number && elem.TryGetDouble(out var bn) && bn != 0)
    || (elem.ValueKind == JsonValueKind.String && bool.TryParse(elem.GetString(), out var bl) && bl),

PG/MSSql send true/false — untouched.

Part 5. How to poke at it yourself

This isn't article pseudocode. The repo ships two tools you verify all of this with by hand.

redb.Examples — ~150 runnable examples that run on any provider, SQLite included:

dotnet run --project redb.Examples -- E021 E146 E148   # date filter, aggregates, windows

Flip AddRedb/AddRedbPro + UseSqlite and the same suite runs live on SQLite Free or Pro. The same code runs on Postgres/MSSql unchanged.

redb.CLI — a global .NET tool for schema and data management, supports sqlite across every command:

redb schema -p sqlite -o redb_sqlite.sql            # dump the full schema SQL (review/CI)
redb init   -p sqlite -c "Data Source=app.db"       # create the tables in an empty DB
redb export -p sqlite -c "Data Source=app.db" -o data.redb --compress
redb import -p sqlite -c "Data Source=app.db" -i data.redb --clean

And the part that matters — trust is verified by tests. SQLite Free and Pro each pass the integration suite 200/200 — the same suite that gates Postgres and MSSql. For a brand-new provider that's worth more than adjectives: same suite, same bar.

War story #4: the CLI that "supported" sqlite — but silently didn't

While preparing this post, I wanted to show redb schema -p sqlite — and walked straight into my own tooling landmine. redb.CLI's code did support sqlite: a full SqliteProvider, the ProviderFactory.Create("sqlite") factory, the redbSqlite.sql schema resource. The csproj did not:

<!-- redb.CLI.csproj — pulled the engine from an old NuGet version: -->
<PackageReference Include="redb.SQLite" Version="1.2.*" />

1.2.* is a version where the SQLite provider didn't exist at all (it's new, at 3.2.1). So typeof(redb.SQLite.RedbService).Assembly and the embedded redbSqlite.sql resource resolved against an assembly that doesn't have them, and any -p sqlite command silently drifted from the real code. The fix — point the references at the local projects (as redb.Examples already does):

<ProjectReference Include="..\redb.SQLite\redb.SQLite.csproj" />

Same moral as the %% story: "the code supports it" ≠ "the build sees it." A version pin is part of the contract too, and a stale pin breaks a feature as quietly as a typo in SQL. It's trivially checkable — by running the command itself: redb schema -p sqlite now dumps the real schema (REAL Julian, _DateTimeOffset REAL) instead of failing on an empty resource.

Part 6. Traps you will hit

This series is honest about "what's not done and what you'll trip over," so no glossing:

The .db path follows the working directory. A relative string (Data Source=app.db) creates the file relative to the process cwd, not the project folder. I lost a couple of hours to this myself: dotnet run from different directories wrote to different files, and tests "passed/failed" against different DBs. Use an absolute path or pin the cwd.
:memory: is per-connection. For a connection pool to see one in-memory DB you need Mode=Memory;Cache=Shared plus one held-open connection. That's SQLite's lifecycle, not redb's: close the last connection and the DB evaporates.
NUMERIC → REAL by default. Fast, but lossy past double range. An exact path via TEXT is a planned setting. A known SQLite weak spot.
SQLite is single-writer. One writer per file; redb sets busy_timeout for concurrent writes, but don't expect Postgres-grade parallelism. For embedded/local, that's normal.
Free's native binaries ship for Windows x64, Linux x64 and Linux arm64. All three are packed into runtimes/<rid>/native/ and delivered to framework-dependent builds by a buildTransitive .targets — the extension is loaded by an explicit path, so the file must physically land in your output, and NuGet doesn't flatten runtimes/ for a no-RID build. macOS (osx-x64/osx-arm64 .dylib) builds from the same CMake project but needs a macOS runner — the one remaining gap, slated for the CI matrix. Pro has no native dependency — it's already everywhere today, which is exactly what WASM/mobile need.
bool in raw form is 0/1. Remember it when debugging _Boolean/_value_bool: true is stored as 1.

None of these poke through ordinary code — but in a debug session each one saves an evening.

Part 7. Pro on mobile and in the browser — and yes, free

Back to what the whole SQLite effort was for.

Writing Blazor WebAssembly, MAUI, or a standalone client? You want SQLite Pro: pure C#, loads no native code, runs in the browser sandbox and on a phone. A typed LINQ store in a single file inside your app.

And the part that draws skepticism, so plainly: Pro for this is free.

Go to redbase.app, register, and email a key request — you get a free license key back.
No bank/payment details. No card asked. Registration is the key-issuing mechanism, not a sales funnel.
The key goes in .WithLicense(...); the wiring instructions are right there after you register.

The barrier to a client-side scenario is zero.

Wrap-up

The SQLite provider made us do two non-obvious things and catch three bugs that would've cost more in prod than any review.

The non-obvious things: move the whole query engine into a C extension where the database has no stored procedures (and where a % in your data breaks mprintf), and re-answer "how do you store a date" for a database that has no date type — REAL Julian in UTC, sargable comparisons, julianday() on the literal side, and a neutral core hook instead of leaking SQLite specifics.

The bugs: a silently-dropped IS NULL in a multi-key filter (json_each.value is lossy by type), DISTINCT ON via ROW_NUMBER() instead of an ignore, and Pro accidentally reaching into the Free-only get_object_json on the subtree-delete path.

These are exactly the spots where the "one LINQ for every database" abstraction either holds or leaks. Ours holds: Free and Pro are green at 200/200 on the same suite as the other dialects, and you can poke at it with redb.Examples and redb.CLI from the repo.

Repo, docs, packages — redbase.app. The stack is at 3.2.1; the two SQLite NuGet packages (redb.SQLite / redb.SQLite.Pro) are 3.2.1 — a buildTransitive .targets hotfix, see War story #5. Questions like "does it do X on SQLite," bug reports — bring them; the provider's new and the feedback channel is open.

If this was useful — a ⭐ on GitHub helps others find it.

SQLite provider for RedBase is coming. full LINQ, typed columns — same API as PostgreSQL and MSSQL. Free tier: native extension (.so / .dll / .dylib). Pro tier: pure C# — works in Blazor WASM. SQLite 3.44.0+. Follow if you want to know when it drops.

rinat kozin — Thu, 18 Jun 2026 19:00:50 +0000

.net

rinat kozin — Thu, 18 Jun 2026 16:38:21 +0000

Apache Camel for .NET, dissected: the HTTP connector with no ASP.NET MVC + the Content-Based Router pattern

rinat kozin — Thu, 18 Jun 2026 15:48:31 +0000

Series: redb ecosystem

This continues the redb.Route series. Earlier on dev.to:

I spent a year building Apache Camel for .NET. Here's the honest state of it.

redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL

Enterprise Integration Patterns in .NET, the deep-dive series — Part 1: the four in-memory channels

redb.Route 3.0.1 — flat DSL navigation, CRTP refactor, and a silent null fix

redb.Route 3.1.0 — LLM(AI) as just another connector

Enterprise-grade AI integration: embedding LLMs into the business processes — redb.Route.Llm 3.1.1

Full list on the author's dev.to profile.

In redb.Route — our Apache Camel-style ESB for .NET — a route always reads the same way: From(source) → [processors] → To(sink). This installment takes one simple integration pattern and one connector and dissects both all the way down.

The pattern: the Content-Based Router — the most basic of the routing patterns from Hohpe & Woolf: look inside a message and decide where it goes next. In the DSL it's .Choice().When(...).Otherwise().
The connector: redb.Route.Http — built-in HTTP/HTTPS. On one side it's a producer (an HttpClient-based caller); on the other, a consumer (an embedded Kestrel server). No controllers, no [ApiController], no ASP.NET middleware pipeline you wire up by hand.

This is a long, technical piece. You'll get the five-line "hello world", but then we go into how the connector works internally: how a single Kestrel is shared across routes, how request headers and route values flow into the Exchange and back, how CORS actually works on a shared server, what happens with streaming, and why there isn't a single app.UseCors() in the codebase.

Every snippet is verified against redb.Route/src/redb.Route.Http; every example is lifted from the real redb.Route.Demo.

Part 0. The scenario everything hangs on

Take a down-to-earth task: an HTTP gateway. A POST /api/demo comes in, and inside we:

take the body,
look at the mode header and pick a processing branch accordingly — that's the Content-Based Router;
reply synchronously over the same HTTP request (request/reply).

Here's the skeleton (full version at the end), from redb.Route.Demo/Routes/MainPipelineRoutes.cs:

From("http:0.0.0.0:5088/api/demo?inOut=true")
    .RouteId("demo-http-entry")
    .ConvertBody<string>()

    .Choice()
        .When(e => GetHeader(e, "mode") == "full")
            .SetHeader("stamp.dsl", "full-branch")
        .When(e => GetHeader(e, "mode") == "short")
            .SetHeader("stamp.dsl", "short-branch")
        .Otherwise()
            .SetHeader("mode", "default")
            .SetHeader("stamp.dsl", "default-branch")
    .EndChoice()

    .SetHeader("Content-Type", "application/json")
    .SetBody(e => BuildResponse(e));

One From brings up an HTTP server on port 5088, one .Choice() decides the message's fate, one .SetBody(...) builds the reply. Now let's see how it works.

Part 1. Content-Based Router — the simple pattern, honestly dissected

What it actually is

The Content-Based Router answers one question — "where next?" — by looking at the message itself, not at external configuration. The textbook example: orders with region=EU go to one handler, region=US to another, everything else to a default.

In redb.Route this is the ChoiceProcessor (redb.Route/src/redb.Route/Processors/ChoiceProcessor.cs), and in the DSL it's a .Choice() block:

.Choice()
    .When(<predicate>)   // branch 1
        ...processors...
    .When(<predicate>)   // branch 2
        ...processors...
    .Otherwise()         // default branch (optional)
        ...processors...
.EndChoice()

Semantics are exactly a switch: predicates are checked top to bottom, the first branch whose predicate returns true runs, the rest are skipped. If none match and there's an .Otherwise(), it runs; if there's no .Otherwise(), the message passes through untouched.

Two ways to express a predicate

1. A lambda — when the condition is easier to write as code:

.Choice()
    .When(e => GetHeader(e, "mode") == "full")  ...
    .When(e => GetHeader(e, "mode") == "short") ...
    .Otherwise() ...
.EndChoice()

2. Fluent predicates over the expression engine — when you want it declarative. From redb.Route.Demo/Routes/DataObservabilityRoutes.cs:

.Choice()
    .When(Header("amount").isGreaterThanOrEqualTo(1000).Matches, w => w
        .SetHeader("tier", "gold"))
    .When(Header("amount").isBetween(500, 999).Matches, w => w
        .SetHeader("tier", "silver"))
    .When(Header("amount").isLessThan(500).Matches, w => w
        .SetHeader("tier", "bronze"))
    .Otherwise(o => o
        .SetHeader("tier", "unknown"))

Header("amount").isBetween(500, 999) isn't a closure — it's a real IPredicate, compiled once and cached as a delegate forever after. Under the hood it's the series' compiled expression engine (Tokenizer → Parser → AST → System.Linq.Expressions → IL), but that's a whole article of its own.

Why pair it with HTTP

The Content-Based Router and an HTTP gateway are made for each other. On the way in, the HTTP consumer decomposes the request into Exchange headers (more below): method, path, query params, route params, every HTTP header. Any of them is ready material for .When(...):

.Choice()
    .When(e => GetHeader(e, "redbHttp.Method") == "DELETE")        ... // by method
    .When(e => GetHeader(e, "X-Tenant") == "acme")                 ... // by header
    .When(e => GetHeader(e, "redbHttp.QueryParam.debug") == "1")   ... // by query
.EndChoice()

The router never touches HTTP itself — it works on an already-decoded Exchange. That's the whole point of a connector: turn transport into a message so the integration patterns know nothing about transport.

Straight from production

Here's a production route from the TsUM system (delivery monitoring), verbatim. HTTP entry + Content-Based Router by method — GET and POST on one path fan out to different handlers:

From("http:0.0.0.0:5090/api/tsum/user-filters?inOut=true&cors=true&corsOrigins=*")
    .RouteId("tsum-api-user-filters")
    .Process(Auth.ProcessAsync)                 // JWT auth — just a processor
    .ConvertBody<string>()
    .Choice()
        .When(e => e.In.Headers.TryGetValue("redbHttp.Method", out var m) && m?.ToString() == "POST")
            .ProcessWithRedb((redb, exchange, ct) => HandlePost(redb, exchange))
        .Otherwise()
            .ProcessWithRedb((redb, exchange, ct) => HandleGet(redb, exchange))
    .EndChoice();

You can see everything we're about to cover in one shot: the HTTP consumer on port 5090 (inOut=true, cors=true&corsOrigins=*), the Content-Based Router over redbHttp.Method, and authentication as an ordinary processor in the chain — no [Authorize] attributes. Now let's dissect how each piece works inside.

Part 2. The HTTP connector from 10,000 feet

The same http/https scheme yields two fundamentally different roles depending on whether it sits in From(...) or To(...):

Role	Class	Built on	What it does
Consumer (`From`)	`HttpConsumer`	Kestrel	Brings up an embedded HTTP server and accepts inbound requests
Producer (`To`)	`HttpProducer`	`HttpClient`	Sends outbound HTTP requests to a remote address

The DSL entry points are the static Http and Https classes (redb.Route.Http/Fluent/HttpDsl.cs):

// Consumer — listen for inbound
.From(Http.Listen("/webhook").Port(8080).Cors("https://app.example.com").InOut())

// Producer — send outbound
.To(Http.Post("api.example.com/orders").BearerAuth().Timeout(5000))

Or the raw URI string (the builder compiles to exactly this):

.From("http:0.0.0.0:8080/webhook?cors=true&corsOrigins=https://app.example.com&inOut=true")
.To("http:api.example.com/orders?method=POST&timeout=5000")

HTTP methods: the consumer listens, the producer sends

Methods (GET/POST/PUT/…) are set differently for the two roles — and there's more than one way. First the producer (To) — which method to send:

// fluent — the method is chosen by the factory method:
.To(Http.Get("api.example.com/users"))           // GET
.To(Http.Post("api.example.com/users"))          // POST
.To(Http.Put("api.example.com/users/42"))        // PUT
.To(Http.Delete("api.example.com/users/42"))     // DELETE
.To(Http.Patch("api.example.com/users/42"))      // PATCH
.To(Http.Head("api.example.com/users/42"))       // HEAD

// the same as a URI string (singular parameter — method):
.To("http:api.example.com/users?method=POST")

// the method can be overridden per-message by a header — it wins over the option:
.SetHeader("redbHttp.Method", "PUT")
.To(Http.Post("api.example.com/users/42"))        // actually sends PUT

Now the consumer (From) — which methods to accept:

// by default ALL methods are accepted:
.From(Http.Listen("/webhook").Port(8080))

// restrict the allowed set (anything else → 405 Method Not Allowed):
.From(Http.Listen("/webhook").Port(8080).Methods("POST"))
.From(Http.Listen("/orders").Port(8080).Methods("POST,PUT"))

// the same as a URI string (plural parameter — methods):
.From("http:0.0.0.0:8080/webhook?methods=POST")

// shorthand: a method prefix right in the path (for the consumer):
.From("http:POST:0.0.0.0:8080/webhook")
.From("http:GET:/health")

The terminology difference is easy to trip over:

	Producer (`To`)	Consumer (`From`)
URI parameter	`method` (singular)	`methods` (plural, comma-separated)
Meaning	which method to send	which methods to accept (empty = all)
Default	`GET`	all methods
Per-message override	`redbHttp.Method` header	— (the filter is static)

The prefix shorthand (http:POST:/...) is a special case: it sets both values at once (method and methods), because the same string can serve as either a producer or a consumer.

And the typical move when different methods hit one path: accept several and fan them out with a Content-Based Router on redbHttp.Method (this is exactly the production example from Part 1):

.From(Http.Listen("/api/tsum/user-filters").Port(5090).Methods("GET,POST").Cors("*").InOut())
    .Choice()
        .When(e => GetHeader(e, "redbHttp.Method") == "POST")
            .ProcessWithRedb((redb, ex, ct) => HandlePost(redb, ex))   // write
        .Otherwise()
            .ProcessWithRedb((redb, ex, ct) => HandleGet(redb, ex))    // read
    .EndChoice();

We'll dissect both roles separately — but first, the big question.

Part 3. "Where's ASP.NET?" — there isn't any, on purpose

When a .NET developer hears "embedded HTTP server", they picture WebApplication, controllers, [HttpPost], filters, model binding, app.UseRouting(), app.UseCors(), a DI middleware pipeline. The redb.Route HTTP connector has none of that. There's Kestrel — bare, with no MVC layer on top.

Here's how the server comes up (SharedHttpServerManager.StartServer, abridged):

var builder = WebApplication.CreateSlimBuilder();   // slim — no MVC, no extra services

builder.WebHost.ConfigureKestrel(kestrel =>
{
    kestrel.Listen(IPAddress.Parse(entry.Host), entry.Port, listenOptions =>
    {
        listenOptions.Protocols = protocols;          // HTTP/1, /2, /3 — see below
        if (entry.Ssl) listenOptions.UseHttps(entry.SslCertPath, entry.SslCertPassword);
    });
    kestrel.Limits.MaxRequestBodySize = entry.MaxRequestBodySize > 0
        ? entry.MaxRequestBodySize : null;
});

builder.Logging.ClearProviders();
var app = builder.Build();

// ONE catch-all endpoint — we route from here ourselves
app.Map("/{**path}", (HttpContext ctx) => HandleCatchAll(entry, ctx));
app.MapGet("/",    (HttpContext ctx) => HandleCatchAll(entry, ctx));
app.MapPost("/",   (HttpContext ctx) => HandleCatchAll(entry, ctx));
// ... PUT/DELETE/PATCH/HEAD/OPTIONS on "/"
await app.StartAsync(ct);

Note the key decisions:

WebApplication.CreateSlimBuilder(), not CreateBuilder(). The slim builder doesn't drag in MVC, Razor, ASP.NET auth, or the rest of the scaffolding — only what Kestrel needs.
Exactly one catch-all route /{**path}. ASP.NET routing is used solely to intercept everything and hand it to our own dispatcher, HandleCatchAll. No controller matching.
builder.Logging.ClearProviders() — the server stays quiet on the host console; logs go through the route's logger.

Why? Because redb.Route is an integration engine, and HTTP is just transport to it — the same as Kafka or RabbitMQ. A route shouldn't know Kestrel is behind it: it gets an Exchange. ASP.NET controllers would impose their own model (attributes, model binding, ActionResult) that's redundant and alien inside a DSL route.

One Kestrel per process, not per route — and Tsak relies on it

A frequent question: "if my app already runs on ASP.NET/Kestrel (say, inside a Tsak worker), does the connector reuse the server or spawn new ones?"

First, what the connector does not do: it doesn't graft onto an external ASP.NET pipeline. The redb.Route.Http project has zero integration points with an external host (no IApplicationBuilder, no UseEndpoints, no IServer — none of it). It stands up its own Kestrel via WebApplication.CreateSlimBuilder().

But "stands up its own" ≠ "spawns instances." The key is that SharedHttpServerManager is registered in DI as a singleton (redb.Route.Http/Extensions/ServiceCollectionExtensions.cs):

public static IServiceCollection AddRedbRouteHttp(this IServiceCollection services, ...)
{
    services.AddSingleton<SharedHttpServerManager>();   // ← one server manager per process
    services.AddSingleton(sp => { var c = new HttpComponent();
        c.ServerManager = sp.GetRequiredService<SharedHttpServerManager>(); ... return c; });
    services.AddSingleton(sp => { var c = new HttpsComponent(); ... });
    ...
}

One manager per process means one Kestrel pool, shared across all route contexts. And Tsak leans on exactly this. Its worker is a plain Host.CreateDefaultBuilder (not a WebApplication) — it has no Kestrel of its own. Even Tsak's own REST admin API (the _system context, port 9090 by default) is not a separate web server — it's an ordinary redb.Route HTTP route brought up through that same singleton manager (redb.Tsak.Core/Services/SystemContextBuilder.cs):

// Tsak registers the HTTP connector itself...
services.AddRedbRouteHttp();   // redb.Tsak.Core/Extensions/ServiceCollectionExtensions.cs

// ...and brings up its admin API as a regular route on the shared manager:
var listenUri = $"http:{host}:{port}/{{**path}}?host={host}&port={port}&inOut=true";
routeContext.AddRoutes(r => r.From(listenUri).Process(/* bridge → auth → dispatch */));

So nothing is "forwarded from the host's Kestrel" — on the contrary, the host (Tsak) brings up Kestrel through the connector and reuses it. Any route targeting the same (host, port) joins the already-running server instead of starting a second one. Tsak even mounts its system-echo route on the admin port — and they don't collide: the specificity ordering from Part 4 separates the concrete /api/echo from the catch-all {**path} (called out in a comment in SystemContextBuilder).

Without Tsak — same thing, you just wire the manager yourself

Tsak isn't magic here: Kestrel is always brought up by SharedHttpServerManager; Tsak is merely a host that registers that manager and routes through it. In a standalone app (no Tsak) you wire the manager by hand. Here's a bare RouteContext from the Llm.HttpShell demo — no DI, no Tsak:

var ctx = new RouteContext(sp, contextId: "llm-http-shell");
ctx.AddComponent(new HttpComponent { ServerManager = new SharedHttpServerManager() });
// ...
ctx.AddRoutes(r => r.From("http:0.0.0.0:5088/api/llm/shell?inOut=true") ... );

Or the same through DI — AddRedbRouteHttp() registers that same singleton manager for you. Either way, the first From("http:host:port/...") that starts brings up a fresh Kestrel via CreateSlimBuilder() for that (host, port) pair, and the rest of the routes on the same (host, port) join it.

Subtlety: pooling works within a single SharedHttpServerManager instance. Create two separate managers and point both at one port and you get a socket-bind conflict, not sharing. The "one Kestrel per (host, port)" guarantee comes from a shared manager — a singleton out of the box under Tsak, and your responsibility standalone.

The practical takeaway: within a process (and one manager) there are exactly as many Kestrels as there are distinct (host, port) pairs. Mounting a route on a port another redb route already listens on (including Tsak's admin port) is fine — that's the whole "don't multiply" point. A socket-bind conflict only happens if a foreign, non-redb server already grabbed the port.

Why bus frameworks like MassTransit don't have this

It's worth contrasting with MassTransit — one of the most popular .NET messaging frameworks. It has no HTTP consumer, no embedded server, no "http as a transport." And that's not an omission; it follows from the architecture.

MassTransit is a message bus over brokers: RabbitMQ, Azure Service Bus, Amazon SQS, plus Kafka/Event Hubs as "riders." Its model is asynchronous broker-mediated delivery with guarantees, retries, and sagas; consumers are keyed to a message type (IConsumer<TMessage>), not a URI endpoint. HTTP doesn't fit that picture: synchronous request/reply contradicts the async/durable bus model. So MassTransit leaves HTTP ingress to ASP.NET — you stand up a controller or minimal API and Publish/Send to the bus from there. The "HTTP → message" boundary lives outside the framework, by hand, in your host code.

redb.Route (like Apache Camel, which it follows) is built differently: it's a mediation engine, and to it HTTP is just another transport, the same as Kafka or Rabbit. An HTTP request is normalized into the same Exchange a broker message becomes, and flows through the same EIP processors. That's why From("http:...") exists as a first-class route source, the connector owns Kestrel itself, and an "HTTP → Kafka → SQL → reply" bridge is one DSL chain without leaving the framework.

These are different tools for different jobs, not "better/worse": MassTransit shines at reliable broker delivery and sagas over queues; Camel-style engines shine at stitching together heterogeneous transports and routing by content. An embedded HTTP server is a natural part of the second approach and fundamentally alien to the first.

Part 3½. "But I love controllers" — `redb.Route.Controllers`

This is where an indignant voice usually pipes up: "Attributes, [HttpGet], model binding — I like that, I don't want to write a .Choice() per endpoint!" Fair. That's why there's a separate package: redb.Route.Controllers. It hands you back the familiar MVC-controller ergonomics, but it does not hand you back the ASP.NET hosting model. Here's the trick.

A controller that looks like ASP.NET — but isn't

Here's a working controller (from the redb.Route.Tests.Controllers tests):

[Route("modules")]
public class ModulesController : RedbController
{
    [HttpGet]
    public string[] GetAll() => ["module1", "module2"];

    [HttpGet("{id}")]
    public string GetById([FromRoute("id")] string id) => $"module-{id}";

    [HttpPost]
    public object Create([FromBody] CreateModuleRequest request) =>
        new { created = request.Name };

    [HttpPut("{id}")]
    public object Update([FromRoute("id")] string id, [FromBody] CreateModuleRequest request) =>
        new { updated = id, name = request.Name };

    [HttpDelete("{id}")]
    public void Delete([FromRoute("id")] string id) { }
}

Painfully familiar: [Route] on the class, [HttpGet]/[HttpPost]/[HttpPut]/[HttpDelete]/[HttpPatch] on methods (with an optional sub-template "{id}"), parameter binding via [FromBody], [FromRoute], [FromQuery], [FromHeader], [FromProperty]. Return an object and it goes out as JSON.

But two differences are fundamental:

The base class is RedbController, not ControllerBase. No HttpContext, no IActionResult, no [ApiController]. Instead, two properties: Context (the route context) and Exchange (the current message). The controller sees an Exchange, not HTTP.
The controller is transport-agnostic. It knows nothing about HTTP. That matters one paragraph from now.

How a controller enters a route

A controller isn't an endpoint — it's a processor inside a route. You mount it on an HTTP entry via .RedbHttpController<T>():

From("http:0.0.0.0:5088/api/{**path}?inOut=true")
    .RouteId("modules-api")
    .RedbHttpController<ModulesController>();

Or via a registry with several controllers (or an assembly scan):

var registry = new ControllerRegistry();
registry.RegisterController(typeof(ModulesController));
registry.RegisterController(typeof(ContextsController));
// or: registry.RegisterAssembly(typeof(ModulesController).Assembly);

From("http:0.0.0.0:5088/api/{**path}?inOut=true")
    .RedbHttpController(registry);

Note the {**path} — the HTTP consumer from Part 5 catches everything under /api, drops redbHttp.Method, redbHttp.Path, redbHttp.RouteParam.*, redbHttp.QueryParam.* into the Exchange, and the HttpControllerDispatcher parses those and finds the right action. No manual header translation:

// HttpControllerDispatcher.Process (abridged)
var method = exchange.In.GetHeader<string>("redbHttp.Method");
var path   = exchange.In.GetHeader<string>("redbHttp.Path");
var action = _registry.Resolve(method, normalizedPath, out var routeParams);
if (action is null) { WriteError(exchange, 404, "NotFound", ...); return; }

Routing and binding — its own, not ASP.NET's

ControllerRegistry.Resolve matches (method, path) to an action segment by segment, picking the most specific one (literals beat {param}): GET /me/sessions/current beats GET /me/sessions/{id}. Same specificity principle as the shared server in Part 4, just at the controller level.

Parameter binding (ResolveHttpParameter) is exactly what the attributes promise: [FromBody] is JSON-deserialized from the byte[], [FromRoute] comes from the template, [FromQuery] from redbHttp.QueryParam.*, [FromHeader] / [FromProperty] from the Exchange headers/properties. With no attribute, it tries a route param by name, otherwise a complex type is bound from the body.

The response is assembled like this: return an object → JSON (camelCase, with UnsafeRelaxedJsonEscaping so Cyrillic and emoji don't become А), status 200; return null/void → 204; throw → an error envelope and 500 (a route miss → 404). The dispatcher sets status.code and redbHttp.ResponseCode, and the HTTP consumer from Part 5 picks them up. The loop closes.

Why this way, not "the ASP.NET way"

Here's the whole point. The same ModulesController, without changing a line, can be invoked over something other than HTTP:

// same controller — over gRPC
From(grpcConsumer).RedbGrpcController<ModulesController>();

// same controller — over SignalR
From(signalRConsumer).RedbSignalRController<ModulesController>();

RedbController is transport-agnostic precisely because it works with an Exchange, not an HttpContext. An ASP.NET controller can't do that — it's welded to the HTTP pipeline. And since .RedbHttpController<T>() is just a processor, it composes with the rest of the DSL: put .Throttle() before it, .WireTap() after it, wrap it in OnException/TryCatch, combine it with the .Choice() from Part 1.

So: love controllers? Use controllers. Just know that under them is not ASP.NET MVC but the same Exchange and the same route pipeline. You get the ergonomics without inheriting the hosting model.

And this isn't theory — Tsak itself runs on it

The best proof the approach is battle-tested: Tsak's entire REST admin API is built exactly this way. ContextsController, RoutesController, ModulesController, AuthController, UsersController, SchedulerController, LogsController, and a dozen more (redb.Tsak.Core/Controllers) all inherit RedbController and carry the same [Route] / [HttpGet] / [HttpPost] / [FromRoute] / [FromQuery]. Here's a slice of the real ContextsController:

[Route("/api/contexts")]
public class ContextsController : RedbController
{
    [HttpGet("")]
    public object? ListContexts() => /* ... */;

    [HttpGet("/{name}")]
    public object? GetContext([FromRoute("name")] string name) => /* ... */;

    [HttpPost("/{name}/stop")]
    public async Task<object?> StopContext(
        [FromRoute("name")] string name,
        [FromQuery("timeoutSeconds")] int? timeoutSeconds = null) => /* ... */;
}

Tsak registers their assembly (ControllerRegistry.RegisterAssembly) and dispatches them via ControllerDispatcherProcessor in the _system context — on the same HTTP connector as everything else. So the controllers are as production-hardened as the HTTP consumer itself: the Tsak dashboard and CLI talk to exactly these.

A full treatment of redb.Route.Controllers (the registry, the IControllerActionFilter action filters, the gRPC/SignalR dispatchers, the error envelope) is its own article in the series. Here it's just to put the "where are my controllers" question to rest.

Part 4. One Kestrel per (host, port) — `SharedHttpServerManager`

The least obvious part of the consumer is that multiple routes can listen on one port. If you have three From(...) on 0.0.0.0:5088 with different paths (/api/demo, /api/echo, /api/llm/ask), you get one Kestrel, not three.

SharedHttpServerManager owns this. The key is the (host, port) pair:

private readonly ConcurrentDictionary<string, ServerEntry> _servers = new(...);
private static string BuildKey(string host, int port) => $"{host}:{port}";

Registering a route

When a consumer starts, it doesn't "create a server" — it registers a route on the server for its (host, port). The server is created lazily, on the first registration:

// HttpConsumer.Start (abridged)
_registration = _serverManager.RegisterRoute(
    host, port, path, _options.Methods, HandleRequest,
    ssl, _options.SslCertPath, _options.SslCertPassword,
    corsOptions, _options.MaxRequestBodySize, _options.Protocol);

await _serverManager.EnsureStarted(host, port, ct);
BaseUrl = _serverManager.GetBaseUrl(host, port);   // e.g. "http://localhost:5088"

RegisterRoute drops a RouteRegistration (path template + methods + handler + CORS) into the ServerEntry's route list. EnsureStarted brings up Kestrel if it isn't running; if it already is, it's a no-op.

Dispatch: a route table of its own

Every request lands in one HandleCatchAll, which finds the matching route itself:

private static async Task HandleCatchAll(ServerEntry entry, HttpContext ctx)
{
    var match = entry.MatchRoute(ctx.Request.Path, ctx.Request.Method);

    if (match.Registration is null)
    {
        // path matched but method didn't → 405; otherwise → 404
        ctx.Response.StatusCode = match.PathMatched
            ? StatusCodes.Status405MethodNotAllowed
            : StatusCodes.Status404NotFound;
        return;
    }

    if (match.RouteValues is not null)
        ctx.Items["__redbRouteValues"] = match.RouteValues;   // {id} etc. — into the Exchange next

    await match.Registration.Handler(ctx);
}

Two subtleties worth knowing:

1. 404 vs 405. If the path matched but the method isn't allowed, you get an honest 405 Method Not Allowed, not a 404. Small, but correct.

2. Path templates and specificity ordering. Paths are parsed via TemplateParser/TemplateMatcher from ASP.NET routing — {id} parameters and {**rest} catch-alls are supported. But the check order is not registration order; it's by descending specificity (ServerEntry.GetCompiled):

_compiled = built
    .OrderBy(x => x.spec.HasCatchAll ? 1 : 0)   // concrete paths first, catch-all last
    .ThenByDescending(x => x.spec.Literals)      // more literal segments = more specific
    .ThenBy(x => x.spec.Parameters)              // fewer parameters = more specific
    .ThenBy(x => x.order)                        // ties: stable, by registration order
    .Select(...).ToArray();

Why? So a concrete /api/echo always beats a catch-all /{**path} registered on the same port, even if the catch-all was registered first. Without this rule, a catch-all would swallow every later route. The behavior matches what you intuitively expect from ASP.NET routing.

Lifecycle: reference counting

A server lives exactly as long as it has at least one route. When a consumer stops:

// HttpConsumer.Stop
_serverManager.UnregisterRoute(_registration);
await _serverManager.StopIfEmpty(host, port, ct);

StopIfEmpty stops and unloads Kestrel only if no routes remain:

public async Task StopIfEmpty(string host, int port, ...)
{
    if (entry.Routes.Count > 0) return;   // someone's still listening — leave it
    _servers.TryRemove(key, out _);
    await StopServer(entry, ct);          // graceful stop with a 5s timeout
}

So if you stop one of three routes on port 5088, the server keeps running for the other two. Stop the last one, and Kestrel goes down. This lets you add and remove routes on the fly (e.g. from a dashboard: tsak route start demo-http-echo / stop) without bouncing the whole server.

Guard against mismatched schemes

You can't bind both HTTP and HTTPS on the same (host, port) — the manager throws on registration:

if (entry.Ssl != ssl)
    throw new InvalidOperationException(
        $"Server on {host}:{port} is already registered as {(entry.Ssl ? "HTTPS" : "HTTP")}...");

Part 5. Consumer: how an HTTP request becomes an `Exchange`

This is the heart of the inbound side — HttpConsumer.BuildExchange. Let's see exactly what reaches a route.

The request body

if (request.ContentLength > 0 || request.ContentType is not null)
{
    if (_options.StreamRequest)
        body = request.Body;                       // the stream as-is (passthrough)
    else
    {
        using var ms = new MemoryStream();
        await request.Body.CopyToAsync(ms, ...);
        body = ms.ToArray();                        // buffer into a byte[]
    }
}
var message = new Message(body);

By default the body is buffered into a byte[]. That's why you almost always see .ConvertBody<string>() right after From — to turn bytes into a string. With streamRequest=true the body stays a Stream (for large uploads), and an important detail from the code comment: that stream is owned by Kestrel, Exchange.DisposeAsync does not close it, but it stays valid until the response is written.

Headers: what the connector puts into the `Exchange`

This is probably the single most useful reference table in the article. The consumer decomposes the request into Exchange headers under the redbHttp. prefix (HttpHeaders.cs):

Exchange header	What it holds	Example
`redbHttp.Method`	HTTP method	`POST`
`redbHttp.Path`	request path	`/api/demo`
`redbHttp.Url`	full URL	`http://localhost:5088/api/demo?x=1`
`redbHttp.Port`	server port (int)	`5088`
`redbHttp.Query`	raw query string	`x=1&y=2`
`redbHttp.QueryParam.<name>`	one query parameter	`redbHttp.QueryParam.x` = `1`
`redbHttp.RouteParam.<name>`	one path-template value	`/users/{id}` → `redbHttp.RouteParam.id`
`redbHttp.RemoteAddress`	client IP	`127.0.0.1`
`<any HTTP header>`	as-is	`Content-Type`, `Authorization`, `X-Chat-Id`…

A few implementation details that matter:

Multi-value query/headers. If a parameter repeats (?tag=a&tag=b), the values are joined with a comma:

message.Headers[$"redbHttp.QueryParam.{qp.Key}"] = qp.Value.Count switch
{
    0 => string.Empty,
    1 => (object)qp.Value[0]!,
    _ => string.Join(",", qp.Value!)
};

Route values from the template. Remember ctx.Items["__redbRouteValues"] from the dispatcher? Here's where they're unpacked:

if (httpContext.Items.TryGetValue("__redbRouteValues", out var rvObj)
    && rvObj is RouteValueDictionary routeValues)
{
    foreach (var (key, value) in routeValues)
        if (value is not null)
            message.Headers[$"redbHttp.RouteParam.{key}"] = value;
}

So From("http:0.0.0.0:8080/users/{id}") gives you ${header.redbHttp.RouteParam.id} in the route.

HTTP/2 pseudo-headers are filtered out. Headers like :method, :path (HTTP/2) are skipped — if (header.Key.StartsWith(':')) continue;.

Remembering inbound header names. Subtle but important. The connector collects the names of all inbound headers into a set and stores it in the exchange properties:

exchange.Properties["redbHttp.RequestHeaderNames"] = requestHeaderNames;

Why — becomes clear on the response side: so request headers are not reflected back into the response. Without it, an inbound Host or User-Agent could accidentally ride back to the client in the reply.

Exchange pattern: InOnly vs InOut

exchange.Pattern = _options.InOut ? ExchangePattern.InOut : ExchangePattern.InOnly;

InOnly (default) — fire-and-forget. The server replies with an empty 200 OK immediately, and the route runs "in the background" relative to the response. This is a webhook receiver: "got it, thanks."
InOut (?inOut=true) — request/reply. The server waits for the route to finish and returns the result as the HTTP response. This is an API endpoint.

In practice: to return a body in the response you need inOut=true. Otherwise the body you assembled with .SetBody(...) goes nowhere (see WriteResponse — all body writing is under if (_options.InOut)).

The response: how an `Exchange` becomes HTTP again

WriteResponse assembles the HTTP response. Status code resolution order (by descending priority):

var statusCode = _options.ResponseCode;                       // 3. options default (200)
if (responseMsg.Headers.TryGetValue("redbHttp.ResponseCode", out var rc)) ...   // 1. explicit header
else if (responseMsg.Headers.TryGetValue("status.code", out var sc)) ...        // 2. transport-neutral fallback

So from a route you can return, say, 404 just by setting a header:

.SetHeader("redbHttp.ResponseCode", 404)

Response Content-Type follows a similar chain: redbHttp.ResponseContentType → Message.ContentType → options default (application/json).

Carrying headers into the response. This is where that RequestHeaderNames set earns its keep. A message header makes it into the HTTP response only if it clears several filters:

foreach (var (key, value) in responseMsg.Headers)
{
    if (value is null) continue;
    if (requestHeaderNames?.Contains(key) == true) continue;  // don't reflect request headers
    if (HttpHeaders.NonBridgedHeaders.Contains(key)) continue; // hop-by-hop and internal
    if (HttpHeaders.IsRedbHeader(key)) continue;               // redbHttp.* — internal
    if (IsInternalHeader(key)) continue;                       // redb*/Camel* — internal
    // ... set with multi-value support
}

And a special touch for Set-Cookie and other multi-value headers — StringValues is used so ASP.NET emits multiple header lines instead of one stringified array:

StringValues sv = value switch
{
    string s => s,
    string[] arr => arr,
    IEnumerable seq when value is not string =>
        seq.Cast<object?>().Select(o => o?.ToString() ?? "").ToArray(),
    _ => value.ToString()
};
if (ContainsInvalidHeaderValueCharacters(sv)) continue;  // Kestrel rejects control / non-ASCII
httpContext.Response.Headers.TryAdd(key, sv);

A gotcha from the repo's history. Response headers are copied always, even when the body is empty. Otherwise body-less responses (a 302 redirect, a 204 No Content, a Set-Cookie-only reply) would silently drop Location/Set-Cookie. It's called out in a code comment.

Part 6. CORS on a shared server — without `app.UseCors()`

CORS in ASP.NET means middleware and named policies. Here there's no ASP.NET CORS middleware at all. Instead there's one dispatch layer per server that selects the policy by the matched route. Why: a single (host, port) hosts different routes, and each can carry its own CORS policy. A classic UseCors with one policy per server can't do that.

The CORS parameters

At the endpoint level (HttpEndpointOptions):

URI parameter	Property	Meaning
`cors=true`	`Cors`	enable CORS for the route
`corsOrigins=...`	`CorsOrigins`	comma-separated origin whitelist, or `*`
`corsCredentials=true`	`CorsCredentials`	allow `Access-Control-Allow-Credentials`
— (code only)	`CorsOriginsResolver`	a `HttpRequest → string?` delegate for dynamic origin selection

In the fluent DSL:

.From(Http.Listen("/api").Port(8080).Cors("https://app.example.com").CorsCredentials())

Plus global defaults for the whole component, via DI:

services.AddRedbRouteHttp(cors =>
{
    cors.Enabled = true;
    cors.Origins = "https://example.com";
});

Endpoint parameters always override globals (HttpComponent.ApplyCorsDefaults).

No implicit `*`

A deliberate decision: if cors=true, you must supply either corsOrigins (including an explicit "*" for public endpoints) or a resolver. Otherwise it throws at startup (HttpEndpointOptions.Validate):

if (Cors && string.IsNullOrEmpty(CorsOrigins) && CorsOriginsResolver is null)
    throw new ArgumentException(
        "Cors=true requires CorsOrigins (use \"*\" for public endpoints) or CorsOriginsResolver to be set.");

Why: the old implicit * is a classic footgun. Combined with credentials, the browser silently rejects it, and the developer burns an afternoon wondering why "CORS doesn't work." Far better to fail fast at startup with a clear message.

How the dispatcher works (`CorsDispatchMiddleware`)

The layer is installed once per server, and only if at least one route declared CORS (entry.CorsEnabled). For each request:

var route = entry.MatchByPath(requestPath);   // NOTE: by path, method-agnostic
var cors = route?.Cors;
if (cors is null) { await next(); return; }    // a route without CORS — the layer is transparent

Matching by path, ignoring method is deliberate — so an OPTIONS preflight finds the route's policy even when OPTIONS isn't in the route's allowed-methods list.

Then origin resolution:

var resolved = ResolveOrigin(cors, ctx.Request, requestOrigin);

// wildcard+credentials footgun: the browser rejects it anyway → fail closed
if (resolved == "*" && cors.AllowCredentials)
    resolved = null;

if (resolved is not null)
{
    ctx.Response.Headers["Access-Control-Allow-Origin"] = resolved;
    AppendVary(ctx.Response.Headers, "Origin");   // mandatory, else caches mix up policies
    if (cors.AllowCredentials)
        ctx.Response.Headers["Access-Control-Allow-Credentials"] = "true";
    // ... preflight reflection below
}

ResolveOrigin behaves like this:

a resolver is set → its word is final (may return a specific origin, "*", or null);
static "*" → emit * verbatim;
static whitelist → reflect the request's Origin, only if it's in the list (browsers don't understand a CSV in Access-Control-Allow-Origin, so we single-select);
otherwise → null (origin not allowed, no CORS headers emitted).

Preflight (OPTIONS). On preflight, the layer reflects the method and headers the browser asked for:

if (HttpMethods.IsOptions(ctx.Request.Method))
{
    var requestedMethod = ctx.Request.Headers["Access-Control-Request-Method"].ToString();
    ctx.Response.Headers["Access-Control-Allow-Methods"] = !string.IsNullOrEmpty(requestedMethod)
        ? requestedMethod : (cors.AllowedMethods ?? route!.Methods ?? "*");

    var requestedHeaders = ctx.Request.Headers["Access-Control-Request-Headers"].ToString();
    ctx.Response.Headers["Access-Control-Allow-Headers"] = !string.IsNullOrEmpty(requestedHeaders)
        ? requestedHeaders : (cors.AllowCredentials ? "Content-Type, Authorization" : "*");

    ctx.Response.Headers["Access-Control-Max-Age"] = cors.MaxAgeSeconds.ToString();  // default 86400
}

// OPTIONS always short-circuits to 204 — even when the origin was rejected
if (HttpMethods.IsOptions(ctx.Request.Method))
{
    ctx.Response.StatusCode = StatusCodes.Status204NoContent;
    return;
}

So the whole of CORS is a careful hand-rolled implementation of the spec over Kestrel, with a correct Vary: Origin, proper preflight handling, and a guard against wildcard+credentials. No app.UseCors() anywhere.

Part 7. Streaming the response: SSE and chunked

An InOut consumer can return more than a finished body — it can return a stream, an IAsyncEnumerable<string>. This is used, for example, in the LLM connector to stream tokens. The logic lives in WriteResponse:

else if (body is IAsyncEnumerable<string> asyncStrings)
{
    var useSse = responseContentType?.Contains("text/event-stream", ...) == true;

    httpContext.Response.Headers["Cache-Control"]   = "no-cache, no-transform";
    httpContext.Response.Headers["X-Accel-Buffering"] = "no";   // don't buffer at nginx/LB

    httpContext.Features.Get<IHttpResponseBodyFeature>()?.DisableBuffering();  // turn off ASP.NET buffering

    if (useSse)
        await WriteSseStreamAsync(...);          // text/event-stream → SSE framing
    else
        await WriteChunkedTextStreamAsync(...);  // otherwise → chunked plain text
}

The framing choice is driven by the response Content-Type (the canonical end-to-end signal, Accept ↔ Content-Type), not a private header:

text/event-stream → Server-Sent Events: each yield → one data: frame; at the end, an event: done carrying late-bound summary headers (llm.tokens.in/out, llm.stop_reason, …) that the producer sets only after iteration finishes.
everything else → chunked plain text: each yield is one chunk, with a FlushAsync after each.

DisableBuffering() is critical: without it, ASP.NET would accumulate chunks and flush them in a batch, killing the whole point of streaming.

Part 8. Producer: an HTTP client with every knob

Now the outbound side — HttpProducer over HttpClient. The client is created once in ConnectAsync:

_handler = new HttpClientHandler
{
    AllowAutoRedirect = _options.FollowRedirects,        // followRedirects (default true)
    MaxAutomaticRedirections = _options.MaxRedirects      // maxRedirects (default 50)
};
_httpClient = new HttpClient(_handler)
{
    Timeout = _options.Timeout > 0
        ? TimeSpan.FromMilliseconds(_options.Timeout)     // timeout (default 30000)
        : Timeout.InfiniteTimeSpan
};
ConfigureAuthentication(_httpClient);

Building the URL — four layers

ResolveUrl assembles the target address layer by layer:

// 1. Base URL from the endpoint, with ${...} expressions resolved
var baseUrl = _options.ResolveOption(_endpoint.BuildProducerUrl(), exchange) ?? ...;

// 2. Substitute {name} parameters from .Param(...)
baseUrl = ResolveNamedParams(baseUrl, exchange);

// 3. Query string from the redbHttp.Query header, if present
if (exchange.In.Headers.TryGetValue("redbHttp.Query", out var query) && query is string qs && qs.Length > 0)
{
    var sep = baseUrl.Contains('?') ? "&" : "?";
    return $"{baseUrl}{sep}{qs}";
}

The base URL is built by HttpEndpoint.BuildProducerUrl() — simply scheme + host[:port]/path. And {name} parameters are substituted with URL escaping:

// ResolveNamedParams
var resolved = valueTemplate.Contains("${")
    ? (_options.ResolveOption(valueTemplate, exchange) ?? valueTemplate)
    : valueTemplate;
url = url.Replace($"{{{name}}}", Uri.EscapeDataString(resolved), ...);

So:

.To(Http.Get("api.example.com/users/{id}/orders").Param("id", Header("userId")))

turns at runtime into http://api.example.com/users/42/orders, where 42 comes from the userId header and is escaped.

The request method

The method comes from options (?method=POST) but can be overridden by the redbHttp.Method header:

if (exchange.In.Headers.TryGetValue("redbHttp.Method", out var hm) && hm is string methodStr)
    return new SysHttpMethod(methodStr.ToUpperInvariant());

A body is set only for POST/PUT/PATCH (HasBody). byte[] → ByteArrayContent, Stream → StreamContent, anything else → StringContent in UTF-8.

Header bridging

With bridgeHeaders=true (default), Exchange headers ride out into the HTTP request — except internal and hop-by-hop ones (NonBridgedHeaders):

foreach (var (key, value) in exchange.In.Headers)
{
    if (value is null) continue;
    if (HttpHeaders.NonBridgedHeaders.Contains(key)) continue;  // Connection, TE, redbHttp.*, Content-* etc.
    if (!request.Headers.TryAddWithoutValidation(key, strValue))
        request.Content?.Headers.TryAddWithoutValidation(key, strValue);  // else try as a content header
}

NonBridgedHeaders is the union of internal redbHttp.*, hop-by-hop (Connection, Keep-Alive, Transfer-Encoding, TE, Trailer, Upgrade, Proxy-*), and the content headers managed by HttpClient (Content-Type, Content-Length, Content-Encoding…).

Authentication

Three schemes (HttpAuthScheme):

Basic — username/password, the Authorization: Basic ... header is set once on the client.
Bearer, static — a constant token, also set once on the client.
Bearer, dynamic — a token expression (DynamicValue<string>.IsDynamic), resolved per request from the Exchange:

// ConfigurePerRequestAuth
if (_options.AuthScheme == HttpAuthScheme.Bearer && _options.AuthToken.Value.IsDynamic)
{
    var token = _options.AuthToken.Value.Resolve(exchange);
    if (!string.IsNullOrEmpty(token))
        request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
}

So .BearerAuth().AuthToken(Header("jwt")) injects a fresh JWT from the header on every outbound call.

Response → `Exchange`

MapResponse drops the response body into Out and sets the standard headers:

outMessage.Headers["redbHttp.StatusCode"] = (int)response.StatusCode;
outMessage.Headers["redbHttp.StatusText"] = response.ReasonPhrase ?? "";
outMessage.Headers["redbHttp.Url"]        = response.RequestMessage?.RequestUri?.ToString() ?? "";

With copyResponseHeaders=true (default) it copies the response headers plus Content-Type/Content-Length. With streamResponse=true the body becomes a Stream, not a byte[] — and then the HttpResponseMessage is deliberately not disposed right away: Exchange.DisposeAsync closes the stream.

`throwOnError`

By default throwOnError=true — on 4xx/5xx the producer throws an HttpRequestException (which your OnException/TryCatch will catch). Turn it off with .NoThrowOnError() if you'd rather handle statuses by hand via ${header.redbHttp.StatusCode}.

Part 9. Putting it all together — a real route

Back to MainPipelineRoutes.cs, here's the full HTTP entry with the Content-Based Router and request/reply (redb.Route.Demo):

From("http:0.0.0.0:5088/api/demo?inOut=true")
    .RouteId("demo-http-entry")

    .ConvertBody<string>()              // byte[] → string
    .Throttle(10)                       // no more than 10 req/sec
    .Log("[1-HTTP] ▶ body=${body}, contentType=${contentType}")

    .SetHeader("traceId", e => Guid.NewGuid().ToString("N")[..12])
    .Log("[1-HTTP]   traceId=${header.traceId}, mode=${header.mode}")

    .ValidateJsonSchema(MessageSchema)  // body must be JSON with a "message" field
    .IdempotentConsumer(e => GetHeader(e, "traceId") ?? "", IdempotentRepo)

    // ── Content-Based Router: route by the mode header ──
    .Choice()
        .When(e => GetHeader(e, "mode") == "full")
            .SetHeader("fastTrack", e => GetHeader(e, "priority") == "high" ? "true" : "false")
            .SetHeader("stamp.dsl", "full-branch")
        .When(e => GetHeader(e, "mode") == "short")
            .SetHeader("fastTrack", "false")
            .SetHeader("stamp.dsl", "short-branch")
        .Otherwise()
            .SetHeader("mode", "default")
            .SetHeader("stamp.dsl", "default-branch")
    .EndChoice()

    // ... in the demo: a cross-transport pipeline, SQL, WireTaps ...

    .SetHeader("Content-Type", "application/json")
    .SetBody(e => BuildResponse(e));     // inOut=true → this is the HTTP response

Hit it with three curls — three different branches:

# full-branch
curl -X POST http://localhost:5088/api/demo \
  -H "Content-Type: application/json" -H "mode: full" -H "priority: high" \
  -d '{"message":"hello"}'

# short-branch
curl -X POST http://localhost:5088/api/demo \
  -H "Content-Type: application/json" -H "mode: short" \
  -d '{"message":"hi"}'

# default-branch (no mode)
curl -X POST http://localhost:5088/api/demo \
  -H "Content-Type: application/json" \
  -d '{"message":"yo"}'

And right next to it, on the same port 5088, lives an echo route (EchoRoutes.cs) — a separate From, the same shared Kestrel:

From("http:0.0.0.0:5088/api/echo?inOut=true")
    .RouteId("demo-http-echo")
    .AutoStart(false)                    // dormant until you start it by hand
    .ConvertBody<string>()
    .SetHeader("Content-Type", e => /* mirror the request's Content-Type */ ...)
    .Log("[ECHO] ◀ Echoing back: ${body}");

.AutoStart(false) is a perfect illustration of the lifecycle from Part 4: the route is registered but doesn't start with the module. Start it by hand (tsak route start demo-http-echo) and it registers on the already-running Kestrel for port 5088. Stop it, and the server stays up — because /api/demo is still listening.

Gotcha cheat sheet

No body in the response? Check for inOut=true. Without it the consumer returns an empty 200 OK, no matter what you do with .SetBody(...).
CORS "doesn't work"? With cors=true you must supply corsOrigins (or a resolver) — otherwise it throws at startup. Wildcard * + credentials is rejected by browsers, and the connector honestly returns the response with no CORS headers (fail closed).
Port already in use? Within a process Kestrel is shared by (host, port) through the singleton SharedHttpServerManager — routes on one port share one server (under Tsak you can even sit on the admin API's port). A bind conflict only happens if a foreign, non-redb server already took the port.
Request headers leaking into the response? They shouldn't — the connector tracks them (RequestHeaderNames) and won't reflect them. But if you set a header with the same name yourself, it will go out.
Stream arrives as one batch? SSE needs Content-Type: text/event-stream on the response; otherwise it's chunked plain text. Buffering is disabled automatically.
{id} didn't substitute on the producer? The parameter is supplied via .Param("id", ...); without .Param, the {id} placeholder stays in the URL as-is.
405 instead of 404? That's a feature: the path matched, the method didn't. Narrow methods= or check which method you're calling with.

Wrap-up

redb.Route.Http isn't a "wrapper over a controller" — it's a transport in its own right:

The consumer brings up its own Kestrel (CreateSlimBuilder, no MVC), shares it across routes by (host, port) through the SharedHttpServerManager singleton, keeps its own route table with proper specificity ordering, ref-counts for lifecycle, and implements CORS by hand — one policy per route.
The producer is HttpClient with every knob exposed: layered URL building, {name} parameters, header bridging, Basic/Bearer (including a dynamic per-request token), response streaming, and a controllable throwOnError.
HTTP ↔ Exchange is a two-way bridge through redbHttp.* headers, with honest handling of multi-values, route parameters, status codes, and body-less responses.

And the Content-Based Router (.Choice().When().Otherwise()) shows what it's all for: a route makes decisions on the content of an already-decoded message and knows nothing about whether Kestrel, Kafka, or RabbitMQ sits beneath it.

Next in the series we take the next connector and the next EIP cluster. Want a specific one? Say so in the comments.

Every snippet is verified against the sources in redb.Route/src/redb.Route.Http (HttpConsumer, HttpProducer, HttpComponent, SharedHttpServerManager, HttpEndpointOptions, HttpHeaders, Fluent/HttpDsl) and redb.Route.Controllers. Examples come from the redb.Route.Demo/Routes demo project (MainPipelineRoutes, EchoRoutes, DataObservabilityRoutes) and from the production TsUM system (tsum.Api/Routes, tsum.Api/Auth) — the same routes run in production on port 5090. The controllers section is production-backed: Tsak's entire REST admin API (redb.Tsak.Core/Controllers — Contexts, Routes, Modules, Auth, Users, … — ~14 controllers) is built on RedbController and runs in production via ControllerDispatcherProcessor. The application-level TsUM API prefers .Process/.Choice — both approaches coexist and both are battle-tested.

DEV Community: rinat kozin

Basic OP: 35 modules, 0 failures. Config OP: 0 failures. Found and fixed: PII leak in id_token, missing Cache-Control: no-store. report: https://github.com/redbase-app/redb-identity/blob/main/OPENID_CERTIFICATION.md #dotnet #oidc #oauth

redb-identity/OPENID_CERTIFICATION.md at main · redbase-app/redb-identity · GitHub

The night job that carves its own partitions: cron, a plpgsql function, and a Splitter

The SQL first

The tick: cron://

Calling the function: sql:

The splitter: three tables, and one failure doesn't take down the rest

What happens when the server was down at 02:30

What happens when the previous run is still going

Three nodes

What lands in the message

Two jobs from production

The full option list

Wrapping up

The Amazon SQS + SNS connector in redb.Route, at-least-once and pub/sub via SNS SQS. Still leaving MassTransit

One package, two transports

Anatomy of the URI

Every parameter

Connection and credentials (shared by sqs:// and sns://)

SQS — consumer (From("sqs://..."))

SQS — producer (.To("sqs://..."))

SNS — publisher (.To("sns://..."))

Headers and tracing through the broker

Straight talk on delivery: at-least-once, visibility, transacted

Recipe 1: producer, consumer, FIFO, batch

Recipe 2: competing consumers (concurrency)

Recipe 3: RPC (request/reply) — and why SQS won't hand it to you

Recipe 4 (EIP): Publish-Subscribe via SNS→SQS fan-out

⚠️ The catch: the envelope vs. raw

Recipe 5 (a cousin): Multicast — the same fan-out, but inside the route

Wrap-up

redb.Identity, the full tour: a transport-agnostic OAuth 2.1 / OpenID Connect server for .NET — protocol transport, and no migrations

Skipping ahead: the receipts

The layers underneath, in one breath

Killer feature #1: protocol ≠ transport

1. Another module in the same worker calls Identity with no network at all

2. Facades are pure transport adapters — take the ones you need

3. A browser shows up exactly where the spec demands one

Killer feature #2: no migrations

RTTI at the database level — what the storage model actually is

Killer feature #3: the same server — also as an embedded library

Two contexts, one worker

Anatomy of a token request

Storage: what it buys you over EF-Core-on-tables

Providers: zero code change between rows

Standards — the full catalogue

OAuth 2.0 / 2.1 core

OpenID Connect 1.0 family

Validated by the official OpenID Foundation conformance suite — not just our own tests

Two warnings we kept on purpose

JOSE / JWT

SCIM 2.0

MFA / WebAuthn / OTP

Federation, self-service, admin

Production-grade features — the "later" list, done now

Live JWKS rotation — no process restart

Cluster-safe primitives — built in

Background deletion — the DB as a queue

DataProtection key ring on the storage layer

Rate limiting with a distributed backend

Password hashing — Argon2id and BCrypt side by side, with upgrade-on-login and a timing-attack fix

Audit — "anywhere," proven by integration tests

The mechanism: a WireTap on direct-vm

What the tests actually prove

Real-world fan-out

Observability

Logical-level backup via redb.Export — no pg_dump, no vendor lock

A production example — a daily cron backup through redb.Route

Restore and cross-provider migration

redb.CLI — a global .NET tool for the BAU operations

Host-agnostic — Tsak is an option, not a dependency

Extending alongside Identity — your own scheme, your own routes

An addon calls the identity server from the inside over direct-vm — no HTTP, one transaction

Your own facade on your own protocol

A separate control facade for admin endpoints on a public perimeter

Extension points — everywhere, not just Identity

Combinatorics with the LLM connector

Enterprise scale — thousands of microservices, hundreds of thousands of users

Service-to-service auth — without the round-trip

The tick: `cron://`

Calling the function: `sql:`

Connection and credentials (shared by `sqs://` and `sns://`)

SQS — consumer (`From("sqs://...")`)

SQS — producer (`.To("sqs://...")`)

SNS — publisher (`.To("sns://...")`)

Logical-level backup via redb.Export — no `pg_dump`, no vendor lock

How an expression becomes a `bool`

`redb.Route.Sqs` — an AWS queue and topic in one package

`redb.Route.Telegram` — a bot as a route

RAG, end to end: `knowledge://`, `embed://` and `.Knowledge()`

`.Threads(N)` — concurrency on any source

The big one: `ConcurrentConsumers(N)` finally does something

The `main` entry point

What a `.tpkg` is