DEV Community: rinat kozin

Redis in .NET routes: cache, Pub/Sub, Streams with consumer groups and Claim Check in one connector

rinat kozin — Wed, 23 Sep 2026 19:06:00 +0000

The Redis connector for redb.Route: every data structure as a route step, Pub/Sub, Streams with consumer groups, a list as a queue, and Claim Check on Redis.

In an integration project Redis is rarely just one thing. It caches answers from external systems, holds counters and locks, carries events over Pub/Sub, keeps a log of messages in Streams, and works as a job queue on a list. Usually each of those roles lives in its own piece of code, with its own connection, its own retries and its own logging.

In redb.Route they all live in one connector, redb.Route.Redis. Any Redis operation becomes a step of a route, and a subscription, a stream or a list becomes its input. The connection, the telemetry and the graceful shutdown are the ones every other connector uses. Underneath it is StackExchange.Redis 2.11.8.

The consumers are described here as of version 4.1.0: pattern subscriptions, a stream without a group keeping a position of its own, claiming back stream entries that got stuck, and a reliable list queue all arrived in it.

Installing the package, and the address

dotnet add package redb.Route.Redis

services.AddRedbRoute(route =>
{
    route.Services.AddRedbRouteRedis();
    route.AddRouteBuilder<OrderRoutes>();
});

An endpoint address reads redis:OPERATION:resource?parameters. The first path segment is the Redis operation (case does not matter), everything after the first colon is the key, the channel or the stream name. Colons inside a key are kept, so redis:SET:session:42?ttl=300 writes the key session:42 with a 300 second time to live.

The fluent builder produces the same address, and the rest of this post uses it:

.To(Redis.Set("session:42").Ttl(300))   // redis:SET:session:42?ttl=300

Redis has ready factories for keys, Pub/Sub, streams and lists (Set, Get, Incr, Publish, Subscribe, XAdd, XRead, LPush and the rest). Any other operation comes from Redis.Command(operation, key), for example Redis.Command("HSET", "customer:42").Field("email").

Every data structure as a route step

The producer covers the operations of all the main Redis structures:

Structure	Operations
Strings and keys	`SET`, `GET`, `DEL`, `EXISTS`, `EXPIRE`, `INCR`, `DECR`, `SETNX`
Lists	`LPUSH`, `RPUSH`, `LPOP`, `RPOP`, `LLEN`, `LRANGE`
Hashes	`HSET`, `HGET`, `HMSET`, `HMGET`, `HGETALL`, `HDEL`, `HLEN`
Sets	`SADD`, `SREM`, `SMEMBERS`, `SCARD`, `SISMEMBER`
Sorted sets	`ZADD`, `ZREM`, `ZRANGE`, `ZCARD`, `ZSCORE`, `ZRANGEBYSCORE`
Geo	`GEOADD`, `GEODIST`, `GEORADIUS`
HyperLogLog	`PFADD`, `PFCOUNT`, `PFMERGE`
Bitmaps	`SETBIT`, `GETBIT`, `BITCOUNT`
Messaging	`PUBLISH`, `XADD`
Any command	`COMMAND`, with the command name in `CustomCommand(...)` and its arguments in a header

Where an operation takes its data from is the same rule everywhere:

the value or the element is the message body. In lists, hashes, sets, sorted sets and streams a byte[] body is written as it is, without being turned into a string;
scalar parameters live in the address: field for hashes, score, minScore and maxScore for sorted sets, start and stop for ranges, longitude, latitude, member1, member2 and geoUnit for geo, offset and bit for bitmaps;
sets of values come in headers: the field map for HMSET (redbRedis.HashFields), the field names for HMGET (redbRedis.FieldNames), the source keys for PFMERGE (redbRedis.SourceKeys), the centre and the radius for GEORADIUS, the arguments for COMMAND.

The key, the field, the channel and the stream name may carry ${...} templates, which are resolved per message.

The result of an operation becomes the body of the next step. GET returns a string or null, INCR the new value of the counter, LRANGE and SMEMBERS an array of strings, HGETALL a dictionary, GEORADIUS the members it found:

From("direct://order-created")
    .To(Redis.Incr("stats:orders:today"))
    .Log("orders today: ${body}");

SETNX writes the value only when the key is not there yet and returns "OK" or null. Together with Ttl(...) that is a lock for the duration of the work, or a way to drop repeats.

Pub/Sub: events for whoever is listening right now

From("direct://publish-event")
    .To(Redis.Publish("orders.events"));

From(Redis.Subscribe("orders.events"))
    .To("direct://notify");

After a publish the number of recipients is in the body and in the redbRedis.Publish.Recipients header. On the subscriber side the body is the message text, and the headers carry the channel (redbRedis.Channel), the type (PubSub) and the time it arrived.

A pattern subscription is PSUBSCRIBE: From(Redis.PSubscribe("orders.*")) receives messages from every channel matching the pattern, and redbRedis.Channel holds the channel the message actually came from. The usePattern option turns a plain SUBSCRIBE into a pattern subscription too.

Redis Pub/Sub delivers a message only to those connected at the moment of the publish, and stores nothing. That is the right fit for notifications and cache invalidation. When you need delivery you can rely on, use Streams.

Streams: a log of messages with consumer groups

Writing to a stream:

From("direct://order-created")
    .To(Redis.XAdd("orders").StreamMaxLength(100_000));

The body goes into a data field, alongside a timestamp field in milliseconds. For a field map of your own, pass a dictionary in the redbRedis.StreamFields header. StreamMaxLength trims the stream by length, approximately by default (MAXLEN ~); StreamApproximate(false) makes the trimming exact. The id of the new entry comes back in the body and in the redbRedis.Stream.MessageId header.

Reading with a consumer group:

From(Redis.XRead("orders")
        .ConsumerGroup("billing")
        .ConsumerName("node-1")
        .StreamReadCount(50)
        .StreamClaimMinIdle(60_000))
    .To("direct://bill");

What happens underneath:

the group is created for you at start, together with the stream if it isn't there yet. By default it starts at the end of the stream and gets the entries added after it was created; StreamStartPosition("0") starts at the first entry. If the group already exists, the consumer simply joins it;
entries are read in batches of streamReadCount. With nothing new to read the consumer waits streamBlockTimeMs (1000 ms by default) and polls again;
the body is the dictionary of the entry's fields, each field is also a redbRedis.Stream.<field> header, and the entry id is in redbRedis.MessageId;
the consumer name defaults to the machine name.

Several nodes sharing a group, each with its own consumer name, split the stream between them: every entry goes to exactly one of them. That is how one stream is processed by a cluster without duplicates.

Acknowledgement and a second attempt

A group consumer acknowledges an entry (XACK) once the whole unit of work has finished well, the route transaction included. An entry whose route failed is not acknowledged and stays in the group's pending list, which is what makes this at-least-once.

StreamClaimMinIdle(ms) brings those entries back into work. On every poll the consumer claims (XAUTOCLAIM) the group's entries that have been pending for at least that long and processes them again: its own failed ones, and the ones left behind by a consumer on a node that died. Pick an idle time longer than your slowest route, or an entry will be claimed while it is still being worked on.

Where losing an entry is acceptable, there is StreamNoAck(): reading goes with NOACK, an entry counts as delivered the moment it is read and never enters the pending list (at-most-once). It does not combine with StreamClaimMinIdle, and an endpoint asking for both is refused.

A stream without a group

Without a group every consumer reads the whole stream and keeps the position itself, moving it past each entry it has read. That is how one log is handed to several independent readers, or replayed from the beginning:

From(Redis.XRead("orders").StreamStartPosition("0"))
    .To("direct://rebuild-projection");

With no StreamStartPosition the consumer starts at the entries added after it started, "0" reads from the first entry, and an entry id continues after that entry. The > position means "not yet delivered to this consumer group" and only means anything to a group: without one the consumer is not created. Nothing holds a failed entry back when there is no group, and reading moves on.

A list as a job queue

From("direct://enqueue-job")
    .To(Redis.LPush("jobs"));

From(Redis.Command("BRPOP", "jobs")
        .ProcessingList("jobs:processing")
        .PollDelay(500))
    .To("direct://run-job");

LPUSH at the head and reading from the tail give a queue in arrival order. The consumer takes items one at a time by polling and waits pollDelayMs while the list is empty. BLPOP and BRPOP here are the names of the operations rather than blocking commands: a blocking read would hold the shared connection the whole process works through.

With ProcessingList the queue is reliable. The item is moved (LMOVE) into the processing list and leaves it only after the route succeeded. A failed item goes back to the queue atomically, to the side it was taken from, and is processed again; whatever a previous run left in the processing list is returned to the queue at start. One consumer per processing list. LMOVE is available in Redis 6.2 and later.

Without ProcessingList the item is popped before processing and a failed one is gone. That suits work you can afford to lose, such as warming a cache.

Connections and secrets

The shortest path is parameters in the address: connectionString (localhost:6379 by default), database and password. The password is marked as a secret and is redacted in logs.

In production a named connection factory in the context registry is nicer, because secrets stay out of the route address:

context.AddToRegistry("prod", new RedisConnectionFactory
{
    ConnectionString = "redis-1:6379,redis-2:6379",
    User = "route",
    Password = secrets.RedisPassword,
    Ssl = true,
    SslProtocols = "Tls12, Tls13",
});

From("direct://rates")
    .To(Redis.Set("rates:usd").ConnectionFactory("prod").Ttl(300));

The factory knows about Redis 6+ ACL users, TLS with a choice of protocols, Sentinel (ServiceName), connect and operation timeouts, keep-alive, the reconnect policy (exponential or linear) and a channel prefix. Configuration mistakes are not masked: a factory name that is not in the registry, or a typo in SslProtocols, gives a clear error on connect instead of a silent fall back to the defaults. The same goes for address parameters: an option the endpoint does not know, or a value of the wrong type, is refused and named, not quietly dropped.

Every endpoint keeps one StackExchange.Redis connection and reconnects on its own. A lost and a restored connection are both logged.

Claim Check on Redis

The Claim Check pattern from the Enterprise Integration Patterns catalog takes a heavy body out of the message for the length of a route and brings it back when it is needed again. RedisClaimCheckRepository is the store for it:

var claims = new RedisClaimCheckRepository(
    new RedisConnectionFactory { ConnectionString = "redis.internal:6379" },
    defaultTtl: TimeSpan.FromHours(1));

From("direct://inbound-invoice")
    .ClaimCheck(claims, ClaimCheckOperation.Push)   // body into Redis, the key in the message
    .To("direct://route-by-headers")
    .ClaimCheck(claims, ClaimCheckOperation.Pop)    // body back, the entry removed
    .To("direct://archive");

The store writes the data as it is, in binary, and puts Redis's own time to live on the entry. Keys get the redb:claimcheck: prefix, which you can change. Read-and-remove is atomic: it runs as a single Lua script, so it works on Redis older than 6.2, where there is no GETDEL yet.

Observability and shutdown

Every producer call opens an OpenTelemetry span named redis <OPERATION>, tagged db.system=redis, with the resource and the operation on it. Redis calls show up in the same trace as the rest of the route.

Consumers stop gracefully, as they do in every other redb.Route connector: they stop taking new messages and finish the ones already in flight.

Getting it

dotnet add package redb.Route
dotnet add package redb.Route.Redis

Source: github.com/redbase-app/redb-route. The package is on NuGet.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

XML routes in redb.Route: a schema the editor picks up, checks at build time, modules swapped live

rinat kozin — Tue, 22 Sep 2026 18:27:21 +0000

"A route in .route.xml runs on the same engine as C#: completion in VS Code, package checks inside dotnet build, a graph editor and hot module swap.

An integration route outlives the code around it. A partner moves to another SFTP folder, an analyst asks for one more branch for a new file type, support wants to know which path last night's rejected request took. These are questions about the route, and answering them through a C# project, a build and a redeploy costs more than the question does.

In 4.0 redb.Route gained a second way to write a route: XML. A .route.xml file goes into a package, the package into a folder of a running service, and less than half a second later the first message travels the new route. XML here is not a separate engine and not a simplified builder "for analysts". The loader turns markup into the same DSL calls you would write in C#, and the definition tree comes out identical. For every example in the repository that is pinned by tests: the markup, the C# generated from it and the tree the loader builds all agree.

Around the format grew the things it was meant for: a schema the editor resolves by namespace, package checks that run inside dotnet build, a graph editor in VS Code, packages that contribute their own elements, and modules with no assembly at all that a worker swaps while it runs. Everything below is shown on a working example.

Where the example comes from

The serial numbers module came out of a GitHub discussion. A reader described a system of four Quartz jobs with status tables between them and asked how that maps onto redb.Route. The answer became SerialNumbersDemo: two partners (one over SFTP, one over AS2), SQL Server, objects in redb, a quota report on a schedule, and packaging into a .tpkg for Tsak. Its routes are written in C#.

Next to it lives the XML twin, SerialNumbers.Xml. Domain, services and database are shared; only the way the flow is described changed. That makes a fair comparison: one integration, two spellings, one engine.

One route, two spellings

The heart of the module is a serial number request. The file is validated against an XSD and unmarshalled into an object, the decision is taken inside a transaction, the response is queued, and only after the commit does the route log what has happened. In C#:

From(RouteUris.SerialNumberRequest)
    .RouteId("serial-number-request")
    .MessageHistory()

    .DoTry()
        .ValidateXsd(XmlSchemas.SerialNumberRequest)
        .Unmarshal<SerialNumberRequestXml>("application/xml")
    .DoCatch<ValidationException>()
        .ProcessWithRedb(IntakeRecorder.RecordInvalidAsync)
        .Log("${header.serials.partner}: ${header.serials.fileName} violates the schema", LogLevel.Warning)
        .Stop()
    .EndTryCatch()

    .Transacted()
        .ProcessWithRedb(SerialRequestService.RegisterAsync)
        .Choice()
            .When(e => DecisionOf(e) == MessageStatuses.Accepted)
                .ProcessWithRedb(SerialRequestService.AllocateAsync)
                .ProcessWithRedb(SerialRequestService.QueueResponseAsync)
            .When(e => DecisionOf(e) == MessageStatuses.Rejected)
                .ProcessWithRedb(SerialRequestService.RejectAsync)
                .ProcessWithRedb(SerialRequestService.QueueResponseAsync)
        .EndChoice()
    .EndTransaction()
    // ...logs after the commit

The same route in markup:

<route id="serial-number-request" description="A serial number request: validate, decide, respond">
  <from uri="direct://serial-number-request"/>
  <messageHistory/>

  <tryCatch>
    <try>
      <validateXsd file="SerialNumberRequest.xsd"/>
      <unmarshal format="application/xml"
                 target="SerialNumbers.Core.Integration.Xml.SerialNumberRequestXml, SerialNumbers.Core"/>
    </try>
    <catch exceptions="redb.Route.Validation.ValidationException">
      <to uri="bean:#intake?method=RecordInvalid"/>
      <log level="Warning">${header.serials.partner}: ${header.serials.fileName} violates the schema</log>
      <stop/>
    </catch>
  </tryCatch>

  <transaction>
    <to uri="bean:#serial-requests?method=Register"/>
    <choice>
      <when expr="header.serials.decision == 'Accepted'">
        <to uri="bean:#serial-requests?method=Allocate"/>
        <to uri="bean:#serial-requests?method=QueueResponse"/>
      </when>
      <when expr="header.serials.decision == 'Rejected'">
        <to uri="bean:#serial-requests?method=Reject"/>
        <to uri="bean:#serial-requests?method=QueueResponse"/>
      </when>
    </choice>
  </transaction>

  <log level="Debug">${messageHistory()}</log>
</route>

The correspondence is line by line. DoTry and DoCatch became <tryCatch> with <try> and <catch> branches, Transacted() became <transaction>, predicate lambdas became expressions in an expr attribute. The transaction boundary is visible to the eye: everything inside <transaction> commits together, everything outside it is written after the commit.

A service call, ProcessWithRedb(SerialRequestService.RegisterAsync), became bean:#serial-requests?method=Register. The bean is a thin wrapper that takes redb from the exchange and hands the exchange to the same service:

public sealed class SerialRequestBeans
{
    public Task Register(IExchange exchange, CancellationToken ct)
        => SerialRequestService.RegisterAsync(Redb(exchange), exchange, ct);

    public Task Allocate(IExchange exchange, CancellationToken ct)
        => SerialRequestService.AllocateAsync(Redb(exchange), exchange, ct);

    // Reject, QueueResponse: the same

    internal static IRedbService Redb(IExchange exchange)
        => exchange.ServiceProvider?.GetService<IRedbService>()
           ?? throw new InvalidOperationException("The exchange carries no IRedbService.");
}

The redb instance is the one ProcessWithRedb would have received: one connection per exchange, inside the transaction the route opened. The demo has ten such wrappers, and that is all the code the markup needed on top of the existing module. Markup decides when a step runs, the service decides what happens.

Not a second engine, a second spelling

The property that matters most is easy to say and hard to overvalue: the markup executes nothing by itself. The loader reads the document and calls Filter(...), When(...), SetHeader(...) and the rest of the DSL exactly as C# code would. What follows comes for free:

Everything the engine can do, the markup can do. Retries, transactions, message history, metrics, secret redaction in URIs, mocks in the test kit: all of it lives in the engine and behaves the same for both spellings.
A route converts back to C#. redb-route-xml csharp prints a RouteBuilder class. In the readable style, markup comments become code comments; in the machine style the code carries #line directives, so a debugger breakpoint lands on the line of the XML file.
The diagram comes from the same source. redb-route-xml mermaid builds a flowchart from the definition tree, so the picture in your documentation cannot drift from the running route.
Migration goes one route at a time. Both spellings live in one context, address the same direct: endpoints and beans, and there is no reason to convert everything at once.

For the examples in the repository this is pinned by tests: each one is loaded and compared against a reference description, and from each one C# is generated, compiled in a test project and checked to build the very same tree. The markup has 198 tests on three target frameworks, net8, net9 and net10.

A schema the editor picks up by itself

A markup document starts with a namespace:

<routes xmlns="urn:redb:route:1.0">

The VS Code extension hands the schema over for that namespace through an OASIS catalog, and from there Red Hat XML does the work: completion for elements and attributes, enum values offered, errors underlined. A file gets the schema from what its root says, under any name and in any folder. Somebody else's route.xml without our namespace is left alone.

The schema is not written by hand. It is generated from the same element registry the loader uses, so the editor knows exactly what the service will later accept: 77 elements including the ones packages contribute, and typed options of 48 schemes. That is every connector plus the engine's own schemes, the ones most routes start from: timer, direct, seda, vm. For sql: that means dataSource, placeholderStyle with its At, Colon and Question values, readOnly and the rest, each with its type and default. Options a connector marks as sensitive are shown as secrets.

The option catalog is built by reflection over connector assemblies. A connector gains an option, the catalog is regenerated, and the editor offers it without a line changing in the extension.

The graph editor: the text stays the truth

The second mode of the same extension opens a route as a graph: Open With, then redb Route Graph, or the button in the editor title. The graph is a projection of the text, not a second storage format. The file is never reserialized: every edit in the graph becomes a pinpoint replacement in the text, usually one line, and undoes with plain Ctrl+Z. Comments, indentation and attribute order stay as the author left them.

What it gives you:

Two orientations, one look. The route is drawn Mermaid-style: curved edges, diamonds for branchings, hexagons for conditions, brackets around filter and transaction. Top down it reads as a diagram; left to right it snakes, wrapping a long route onto a new row the way text wraps a line, with the carriage return drawn. Zoom is on buttons and Ctrl+wheel, and both the layout and the zoom are remembered per file.
A properties panel on click. Identifier and description first, enums and flags as dropdowns. An endpoint URI is broken out into the connector's own options: the ones you set on top, the rest below, secrets masked. A {{sftp.host}} placeholder is edited as written, with the value it resolves to from configuration shown next to it.
Editing with the mouse. A step is inserted through the "+" between steps and a palette built from the same element registry. A step is dragged to a new place or into a bracket. Right click opens a menu: insert before, after or inside, reveal in text, delete. A step travels with its comments.

An endpoint: a short URI or a structure

A short address is written as in C#:

<to uri="kafka://orders?key=${header.tripId}"/>

A long one reads better as a structure. SQL text goes into the element's content without escaping, parameters become their own lines:

<to>
  <sql dataSource="#main-db">
    <![CDATA[ INSERT INTO auth_log(login, at) VALUES (:#login, :#at) ]]>
    <param name="login" value="${header.login}"/>
    <param name="at" value="${dateformat(now(), 'o')}"/>
  </sql>
</to>

The loader assembles from that structure the same URI string a human would have typed, and one canonical form travels the pipeline: the endpoint cache, the statistics, mock patterns and secret redaction all see it. One rule keeps the format honest: no connector carries code written specially for XML. The structured form, option types, completion and validation are all derived from what the connector already has. A new connector gets markup support on the day it appears.

Factories, configuration and secrets

Objects the routes refer to are declared in the package's context.xml. Connection factories are the main kind: they already live in the context registry by name, and a URI points at them. So the factory is declared in markup, its settings arrive as placeholders from configuration, and not a single secret stays in the URI:

<bean name="main-db" type="redb.Route.Sql.Connection.SqlConnectionFactory, redb.Route.Sql">
  <constructorArg>
    <bean type="redb.Route.Sql.Connection.SqlConnectionOptions, redb.Route.Sql">
      <property key="ConnectionString" value="{{db.main.connection}}"/>
    </bean>
  </constructorArg>
</bean>

{{key}} and {{key:default}} resolve through the configuration chain before the value is converted, so {{ldap.port:636}} honestly becomes a number. Every placeholder without a default lands in the package manifest as a required key: whoever deploys the module sees the list of settings to provide before anything starts.

Some factories hold an object rather than a string. For AS2 those are certificates: your own with its private key, and the partner's. There a property takes a nested bean, and factoryMethod builds the object with a static method instead of a constructor:

<bean name="globex" type="redb.Route.As2.As2ConnectionFactory, redb.Route.As2">
  <property key="OurCertificate">
    <bean type="System.Security.Cryptography.X509Certificates.X509CertificateLoader, System.Security.Cryptography.X509Certificates"
          factoryMethod="LoadPkcs12FromFile">
      <constructorArg value="{{As2.CertificateDirectory}}/hub.pfx"/>
      <constructorArg value="{{As2.CertificatePassword}}"/>
    </bean>
  </property>
  <property key="As2From" value="{{As2.Id}}"/>
  <property key="As2To" value="GLOBEX"/>
  <property key="Sign" value="true"/>
  <property key="Encrypt" value="true"/>
</bean>

The AS2 partner is described in full: identifiers, the signing and encryption profile, the key material. The certificate password comes from the worker's configuration layer and never travels inside the package.

Packages bring their own elements

The element registry is open. A connector package can add its own elements to the markup, and the schema, the package check and the editor learn about them from its assembly. That is how <cache>, <rest>, <transformJson>, <payload> and the bridge to redb storage appeared: <redbGet>, <redbSave>, <redbQuery>, <redbDelete>, plus a <redb> block in the context.

Here is a route from the second demo, XmlDemo, which works with redb without a line of C#:

<route id="redbdemo-cycle" description="redb bridge: upsert one row by unique key, query it back">
  <from uri="timer://redbdemo?period={{demo.period:5000}}"/>
  <setBody expr='{"name":"xmldemo:heartbeat","value_unique":"xmldemo:heartbeat","properties":{"ProcessorName":"xmldemo","MessageKey":"heartbeat","Confirmed":true}}'/>
  <redbSave type="redb.Route.RedbCore.Models.IdempotentEntryProps, redb.Route.Core" byUnique="true"/>
  <redbQuery type="redb.Route.RedbCore.Models.IdempotentEntryProps, redb.Route.Core"
             where="ProcessorName == 'xmldemo' AND MessageKey == 'heartbeat'"
             orderBy="MessageKey" take="10"/>
  <setBody expr="REDBDEMO total=${body.Count}"/>
  <to uri="log://redbdemo"/>
</route>

<redbSave byUnique="true"> stores the object by a unique key: a row with that key is updated, no new one appears. <redbQuery> turns the where expression into a server-side redb query with ordering and a limit, rather than fetching everything and filtering in memory. An expression the database cannot run is rejected while the route loads, with a message that says why. The data schema is synchronised once, when the context starts:

<context xmlns="urn:redb:route:1.0">
  <redb>
    <syncScheme type="redb.Route.RedbCore.Models.IdempotentEntryProps, redb.Route.Core"/>
  </redb>
</context>

The routes project

XML routes live in an ordinary .NET project with a layout convention. One command creates it, redb-route-xml new, and from there the package is built out of what sits in its places:

Path	What is there
`routes/*.route.xml`	routes; load order comes from the manifest
`context.xml`	components, context beans, the one-off `<onInit>` pipeline
`resources/`	XSD, XSLT and anything routes reference by file name
`config/{Name}.config.json`	module identity: context name and autostart
`config/context.sample.json`	a sample of the settings for whoever deploys it
`schema/`	the schema for working without the extension, bound in `.vscode`

Configuration is layered. Only the module identity travels inside the package; settings come from the worker's shared configuration and its override layer, secrets from environment variables on the worker. The same .tpkg therefore goes to staging and to production without a rebuild: the environment changes, the package does not.

If a routes project references its own types (as SerialNumbers.Xml references the domain and the services), those are ordinary project and NuGet references. The build puts them in the output, and the package check sees them where the worker will.

Tests: the same test kit

An XML route is tested exactly as a C# one, with the same test kit and no test host of its own. Load the markup into a context, mock the external endpoint, send a message, assert. This is a test from the repository:

[Fact]
public async Task TheCanonicalTestKitFlow_WorksOverAnXmlRoute()
{
    _context.AddXmlRoutesFromContent("""
        <routes xmlns="urn:redb:route:1.0">
          <route id="under-test">
            <from uri="direct://tk-in"/>
            <setHeader name="seen" value="true"/>
            <to uri="kafka://orders"/>
          </route>
        </routes>
        """);
    _context.AdviceAllRoutes(a => a.MockEndpoints("kafka://*"));
    await _context.Start();

    var mock = _context.Mock("kafka://orders").ExpectMessageCount(1).ExpectHeader("seen", "true");
    await _context.SendBody("direct://tk-in", "payload");

    await mock.AssertIsSatisfiedAsync(TimeSpan.FromSeconds(2));
}

A route from a file loads the same way, with one line: AddXmlRoutes("routes/orders.route.xml").

WeaveById finds a step by its id attribute in the markup, so any step of an XML route can be replaced in a test without touching the file. A password from a URI never surfaces in mock names or test reports: redaction works on the same canonical URI string the running route uses.

Checks before you ship

Markup is edited by hand, so the format is built to surface a mistake as early as possible. The first line of defence is the loader: it collects all errors of a document into one list with line and column numbers instead of stopping at the first. A broken expression in a condition is caught at load, not on the first message.

The second line is the package check, redb-route-xml check and pack. It plugs straight into the build:

dotnet build -p:PackRouteOnBuild=true -p:Version=1.0.0

After the build the check runs against the fresh output and sees the real assemblies. What it verifies:

the whole document against the schema, with the position of every error;
registry references: #main-db with no declaration in the package is named, while a SQL parameter :#login is not a reference at all, the difference being where the hash sits;
beans against the real assemblies: the type exists, <property> properties are writable, the method in bean:#x?method=M is really there;
resources: the schema named by validateXsd file= is in the package;
secrets written literally, such as password=... in a URI;
required configuration keys, which go into the manifest.

Here is its answer to a typo in a bean method name and in an element name:

error: routes/serial-number-request.route.xml(22,10): The element 'catch' in namespace
       'urn:redb:route:1.0' has invalid child element 'stopp' ...
error: routes/serial-number-request.route.xml: bean 'serial-requests':
       type 'SerialNumbers.Xml.Beans.SerialRequestBeans' has no public method 'Regster'
       (bean:#serial-requests?method=Regster).

And to the corrected file:

serial-numbers-xml 0.0.0: 1 artifact(s), 0 required config key(s) - ok

A mistake in the markup becomes a build error, and a package with an error is not produced. A dangling reference, a missing method and a forgotten schema file are found on a developer machine, not in the log of a production worker.

Separately the gate warns, where the decision belongs to the author: a step that will never run (anything after <stop/> in the same list), an undeclared #name that module code may register at startup, and a condition that compares outside a placeholder. The last one is worth spelling out. Written expr="${header.kind} == 'order'", the line is rendered to text first, and non-empty text is true whatever it says, so that branch wins on every message and does so silently. Written without the braces, header.kind == 'order', it compares. The gate names the line where the braces are one pair too many.

The tool also watches itself: built against a different version of redb.Route.Xml than the project's assemblies, it stops and names both versions.

A module with no assembly of its own

A package holding only .route.xml, context.xml, resources and configuration is a complete Tsak module. Drop the .tpkg into the modules folder of a running worker and it picks it up. This is the log for XmlDemo:

03:33:17.720 [INF] Registered module xmldemo v4.0.1
03:33:17.743 [INF] Created context xmldemo
03:33:17.881 [INF] XmlRouteModule xmldemo: loaded 1 artifact(s)
03:33:18.091 [INF] Exchange [redbdemo-cycle]: Body: REDBDEMO total=1
03:33:18.095 [INF] Context 'xmldemo' started successfully: all 1 endpoints operational
03:33:23.108 [INF] Exchange [redbdemo-cycle]: Body: REDBDEMO total=1
03:33:28.134 [INF] Exchange [redbdemo-cycle]: Body: REDBDEMO total=1

From registering the module to the first exchange: 0.37 seconds. The counter stays at one tick after tick, because saving by a unique key updates the same row. Delete the file from modules and the module unloads just as quietly:

03:34:27.821 [INF] Package xmldemo-4.0.1.tpkg removed from disk, unloading 1 modules
03:34:27.857 [INF] Removed context xmldemo
03:34:27.871 [INF] Module xmldemo unloaded: context stopped, ALC released

A module that does need code ships its assembly in the same package. The worker brings up the module's code first and the markup second, so objects the code put in the registry are visible to the markup by #name. That is how SerialNumbers.Xml works: its assembly holds the bean wrappers and the bootstrap that raises the demo database, while the whole flow is described in XML. How Tsak assembles services out of modules is covered in the article on microservices with Tsak.

Expressions: one language, errors at load

The expression language is the same one C# routes use, covered in a separate article on redb.Route expressions. Three things matter for XML.

Position decides meaning. In a condition (filter, when, validate) the string becomes a predicate; in a value (setBody, setHeader, toD) it becomes an evaluated object; in uri and id it stays a literal. You never have to ask whether something is an expression: the attribute it sits in answers.

Whitespace around operators does not matter, which is convenient in XML specifically. In an attribute value the greater-than sign needs no escaping while less-than does, so a comparison without spaces is also shorter:

<filter expr="header.amount>1000"/>
<filter expr="header.amount &gt; 1000"/>
<filter expr="header.amount &lt; 10"/>

Diagnostics read as expressions. messageHistory() returns the trail of an exchange step by step: as a table, as a single line log > choice > to(http), as a JSON array, or as numbers ('count', 'totalMs', 'slowestMs'). stats('otel:...') reads counters of the OpenTelemetry layer that endpoint statistics do not carry: throttle delays, circuitbreaker trips, messages dropped by filter. In markup that turns into conditions which used to need code:

<setHeader name="trail" expr="messageHistory('compact')"/>
<choice>
  <when expr="messageHistory('slowestMs') &gt; 500">
    <log level="Warning">${routeId} slow: ${messageHistory()}</log>
  </when>
</choice>

The trail is recorded when <messageHistory/> is in the route or the engine option is on. Where it was never enabled, the function answers with an empty string and a zero: a diagnostic log must not bring the route down.

What to know before you start

Topology is written out. In the C# demo the SFTP consumer and the delivery route are created in a loop over the partners in the database. The markup describes each partner as its own route, so the topology is in the file rather than computed at startup. With a dozen partners that change by deployment, that reads better. When partners are a table that changes at runtime, a C# builder in the same package builds those routes while the markup describes the shared part of the flow. Both spellings live in one context.

Code stays where code belongs. Business logic lives in services, the markup calls them through beans. When a service signature differs from (IExchange) or (IExchange, CancellationToken), a two-line wrapper sits between them, as above.

XML is added, it does not replace. Routes move to markup one at a time while the rest keep running in C#. If a route outgrows the markup, redb-route-xml csharp turns it into code, and from there it evolves as an ordinary RouteBuilder.

Try it

The tool installs from NuGet:

dotnet tool install -g redb.Route.Xml.CodeGen

It creates a routes project by the package convention, with the schema and the editor binding in place:

redb-route-xml new Orders --context orders

The project builds and packs with the check, as shown above. From there the .tpkg goes into a Tsak worker's modules, and a route turns into C# or into a diagram whenever you want:

dotnet build -p:PackRouteOnBuild=true -p:Version=1.0.0
redb-route-xml csharp routes/main.route.xml --namespace Orders.Routes
redb-route-xml mermaid routes/main.route.xml

The markup library is the redb.Route.Xml package, version 4.1.0; the VS Code extension ships as a .vsix attached to the redb-route release on GitHub. Both demos live in the demo folder of the same repository.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

redb 4.0.1 is out: a reliability release for all four .NET products. Transactions, the SQL connector and cluster isolation got the most work. Pro stays free, no license key. https://redbase.app/ https://github.com/redbase-app

rinat kozin — Fri, 18 Sep 2026 14:34:54 +0000

redbase.app

redbase-app · GitHub

redbase-app has 4 repositories available. Follow their code on GitHub.

github.com

redb for business: your team writes business logic, the infrastructure is already built

rinat kozin — Tue, 15 Sep 2026 16:46:04 +0000

With redb your team writes only business logic: what the ecosystem replaces, what it costs, which risks it removes, and what to know before you start.

On September 12 we shipped redb 4.0.0. The ecosystem's four products (a data store, an integration engine, a runtime and an identity server) moved to a shared major version: 76 packages, an internal security audit, and builds on .NET 10, which Microsoft supports until November 2028. The engineering changes are covered in a separate post. This one is for the people deciding what to build a backend on: what the ecosystem replaces, what it costs, which risks it removes, and what to know before you start.

Here's what's in it, briefly:

redb.Core stores typed objects in PostgreSQL, MS SQL Server or SQLite. The data structure is a C# class, the database schema is synchronized from code, and there are no migrations.
redb.Route is an integration engine built on Enterprise Integration Patterns, in the same family as Apache Camel, with 29 transports ranging from Kafka, RabbitMQ and IBM MQ to SOAP, gRPC, AS2 and SFTP.
redb.Tsak is the runtime: modules deploy and swap live, nodes form a cluster, and everything is visible and managed from one dashboard.
redb.Identity is an OpenID Connect and OAuth 2.1 server with HTTP, gRPC and SOAP facades, running as a module of the same runtime.

Your team writes business logic, not infrastructure

In a typical project, before the first line of business logic, the team builds infrastructure: connections to brokers and external systems, retries and duplicate protection, database migrations, zero-downtime deployment, monitoring, authorization. That's months of work in which the business gets no new features, followed by years of maintaining all of it.

In redb that layer is already written, tested and maintained. A developer describes data as a class, a process as a route, and a connection to an external system as a connector address. Everything else goes into what the business actually pays for.

In numbers, counted on real systems:

2.2–4 person-years of infrastructure before the first line of business logic in a TMS built on WSO2 MI and EF Core, and 2.8–4.4 person-years in the payment platform model, plus 0.8–1.1 of a permanent headcount every year to maintain it. At a fully loaded $60–150 per engineering hour, that model comes to roughly $286k–$1.11M up front and $77k–$285k a year. With redb, that layer isn't yours to write or maintain.
Twenty minutes instead of half a day to a day to test a product idea that needs a new field or entity: no migration, no test environment, no rollback script. That's an order of magnitude, and the team gets to compare three to five options instead of shipping the first one that works.
Integration definitions four times shorter: in a migration off an ESB, 861 lines of WSO2 XML became 212 lines of route.

Development moves several times faster not because people type faster, but because they stop writing what has already been written. For the business, that means a shorter path from idea to production and fewer hires for the same roadmap.

What it replaces

Concern	The usual approach	In redb
Data	An ORM, migrations and a separate process to roll them out	The C# class is the schema
Integration	Apache Camel or WSO2 MI on Java, hand-written connectors	redb.Route routes
Runtime and operations	Home-grown glue: zero-downtime deployment, dead-letter queues, dashboards	redb.Tsak
Identity	Keycloak or WSO2 Identity Server on Java, with databases of their own	redb.Identity

The cost of that stack doesn't show up in license fees; it shows up in the sheer volume of someone else's infrastructure. In the breakdown of a real TMS on WSO2 MI and EF Core, the identity server alone brought 227 service tables across four databases, against 53 tables in the system's own business model. Every one of them has to be deployed, backed up and upgraded along with the product. redb.Identity keeps clients, tokens and keys in the same redb store as the rest of your data.

What it costs

Licensing. The open part is Apache 2.0. Pro features (optimized queries, partial change writes, parallel materialization, clustering) are free across the entire 4.x line, commercial use included. No license key, no registration, no per-core or per-node fees. The intent is to keep Pro free beyond 4.x as well. Pro packages are closed source, but the source is available on request to companies that adopt the ecosystem, for security audits, escrow or their own builds.

The real costs are people and hardware. As shown above, your people don't have to write and maintain the infrastructure, and the money side is worked out in the payment platform breakdown: the model is open, the assumptions are stated, and you can plug in your own hourly rate.

People: one language for the whole backend

The data store, integrations, runtime and identity server are written and maintained by one C# team. You don't need separate Java specialists for Camel, Keycloak or WSO2, and you hire from the broad .NET talent pool.

Three rules hold across the ecosystem: a data structure is a class, an entry point is a route, an integration is a connector. A new developer learns one approach instead of five that accumulated over the years, and code review doesn't burn time arguing about the right way to do things.

The integration vocabulary is shared with the industry. Pattern names come from the Enterprise Integration Patterns book that Apache Camel and WSO2 are built on, so an integrator's experience carries over without retraining. Since 4.0, integrations can be described without C# at all, as declarative XML routes in the spirit of WSO2 MI configurations. The markup gives you beans, SQL and the other connectors, reading, querying, saving and deleting objects in the redb store, and transactions. Such a route deploys to Tsak as a package containing a single XML file. Custom code, if you need it, is written in .NET, just as WSO2 MI extensions are written in Java.

Speed of change

A new data field is a new class property. The schema synchronizes at startup, with no migrations and no release windows for them.
Modules swap live, without restarting the node and without a maintenance window; there's a whole section on that below.
An integration can change without recompiling: since 4.0 a module can be a single XML route.
Integrations are tested without brokers. The test kit replaces calls to Kafka, queues and databases with mocks without touching the route itself, so regression testing no longer needs a dedicated environment.

Speed doesn't mean giving up control. Since 4.0, production schema changes can be handed entirely to your DBA: the application stops with a clear message and provides the upgrade script instead of running DDL on its own at startup.

Releases without a maintenance window

Hot deployment and graceful shutdown in redb aren't a feature you switch on before a release; they're the normal lifecycle of every module.

A new module version is checked before the running one is touched. The runtime loads it separately and confirms the package opens and contains a module; if it doesn't, the running version stays as it is. The old version then shuts down gracefully, stops accepting new messages and finishes the ones in progress, after which the new version starts. The brief pause affects only that module's context: the process doesn't restart, and modules in other contexts on the node don't notice the deployment.
Shutdown is graceful by default. A route first stops accepting new messages, then finishes in-flight ones within a configured timeout, and only then closes transports and database connections. Deployment, module removal and shutting down a whole node all work this way: a node first gives up cluster leadership, then stops its contexts, and one slow context can't cut short the shutdown of the others.
Nodes are serviced without downtime. A cordoned node takes no new work and hands its cluster routes to its peers, after which it can be upgraded or rebooted. A readiness check tells Kubernetes when a node can't do its work.
In money terms: in the payment platform model, preparing and running release windows costs about 450 hours a year. Here there's no window to run, and deployment stops being a late-night event with people on call.

All of this is about deploying your modules. Moving the platform itself to a new major version, such as 4.0, is planned separately: the nodes of one cluster group are upgraded together, and on a large database the schema upgrade is best done ahead of time in a quiet window.

Operations

No new database required. redb installs straight into an existing production database, PostgreSQL or MS SQL Server: its tables and functions sit next to yours and leave other objects alone. Backups, replication and database monitoring stay as they are, and if that's how your organization works, the DBA runs the install from a ready-made script.
A cluster without double processing. Since 4.0, uniqueness in the cluster is enforced by the database itself: one module is never assigned to two nodes, and two nodes starting on an empty database don't create two clusters.
Multiple clusters are built into the structure. The topology is a tree of cluster, group and node, and one database holds several clusters with their own groups and nodes. Inter-cluster communication, including across regions, is on the roadmap; until it lands, clusters in different regions are connected with redb.Route routes over a broker or HTTP.
One dashboard for every node: routes, errors, the dead-letter queue with replay, the audit log, logs. Prometheus scrapes the metrics, traces go to OpenTelemetry, and nodes deploy to Kubernetes.
A traffic spike doesn't take down its neighbors. Since 4.0 every entry point can have its own limit: excess requests are rejected before processing starts, while neighboring routes on the same port keep working. The dashboard shows which routes hit their limit, which tells you where capacity is short before errors appear.
High-frequency streams in the same database. Vehicle coordinates, sensor readings and other streams are ingested by redb.Route connectors: Kafka, MQTT, RabbitMQ and the rest. Raw points aren't written as redb objects but into ordinary time-series tables (partitioned ones, for example) in the same database, as bulk inserts batched by a route aggregator. The stream needs no separate database, and a foreign key to a redb object (a vehicle, a trip, a sensor) works as in any other table. Objects hold the entities and totals, time-series tables hold the raw stream, just as in the TMS architecture.
One build, two topologies. The same modules deploy as a monolith or as a cluster of microservices; the Tsak post covers this in detail.

Security and audit

Module code is signed. The runtime can be configured to refuse unsigned modules, and the signature is applied with your key in your build pipeline. The ecosystem's release archives are signed with cosign.
Third-party modules run separately. A module runs with the rights of the worker process, and a signature proves who built it but doesn't limit what it does. So a module from a vendor whose code you haven't reviewed is deployed in its own worker container: the OS and the container provide the isolation, and it's managed from the same dashboard as everything else.
The identity server is checked against the official OpenID Foundation conformance suite. The Config OP and Basic OP (35 modules) profiles pass with zero failures. The suite is open source and runs in Docker, and the results and run steps are published in the repository, so anyone can reproduce the check; here's how the run went. The protocol work for formal certification is done, while the OpenID Certified mark itself is granted by the Foundation through a separate paid application. Out of the box: multi-factor authentication, federation with external providers and LDAP, SCIM, DPoP.
Before 4.0, the Identity facades and the Tsak dashboard went through an internal security audit, and everything it found is fixed in this release. Since 4.0, access tokens carry an RFC 9068 audience, so a token issued for an external API doesn't open the identity server's management API.
A change journal for auditors. Since 4.0, write interceptors give the application "who changed what, and when" without touching business code, and in partial change-write mode (Pro) the per-field before and after values as well. Runtime administrator actions go to a separate journal.

Independence from the vendor

Everything runs inside your perimeter. No SaaS, no license server, and nothing goes back to the vendor: neither license checks nor usage data. The routes' own telemetry doesn't send itself anywhere: your Prometheus scrapes metrics from the node, and OpenTelemetry trace export is off by default and gets turned on with the address of your collector or Jaeger. Packages and images mirror into your internal feed and registry, after which no access to outside services is needed.
Your data stays yours. It lives in your PostgreSQL, MS SQL Server or SQLite, the storage layout is open and documented, and the database is readable with plain SQL. Built-in export and import cover the whole database, including between different database engines.
Data residency is decided by where you deploy. The system doesn't depend on an outside licensor or an external service, so regulated and air-gapped environments run it the same way as any other.

Who's behind it

redb is an ecosystem of four products on a single release line. The project already has an outside contributor, and we welcome more: contributions go through GitHub, with the rules in CONTRIBUTING.md. Its direction is shaped by feedback: GitHub discussions, external security reports, comments on articles.

What matters for a business isn't the size of the team; it's what stays in your hands whatever happens: open source under Apache 2.0, Pro source on request, data in your own database, and releases with signed artifacts and public changelogs for every product. Fixes are pinned by integration tests on three databases. Libraries target .NET 8, 9 and 10, and .NET 10 is supported by Microsoft until November 2028.

Deployment (including to the cloud), support, implementation help and SLAs are available by arrangement. So is a provider for another database, such as Oracle or MySQL; PostgreSQL, MS SQL Server and SQLite are supported out of the box. You can get in touch through redbase.app/about.

What to know before you start

redb is the platform your product is built on. Accounting, pricing, logistics and the rest of your domain logic are written by your team or by a third-party vendor by arrangement, while the infrastructure underneath is already in place.
It runs where you deploy it. redb runs on your servers or in your cloud rather than as a subscription service, and deployment and support can be handed to third-party vendors by arrangement.

Getting started

You don't have to adopt the whole ecosystem at once, and a phased rollout breaks nothing: redb sits next to what already runs, in the same database and alongside your existing systems. A pilot with a single module is a good way in: pick an integration or service with a clear boundary, move it over and compare the upkeep under real load. That's exactly what the ESB migration story did, counting the move of one module line by line. From there, modules move over one at a time, at whatever pace suits the business.

For a first look, the redb-tsak-stack image, with the runtime and dashboard in one container, is all you need. Packages are on NuGet and the source is on GitHub. To talk about a pilot, implementation or support, reach out through redbase.app/about.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

redb 4.0: XML routes, lazy references, unique keys and WS-Trust in one major release

rinat kozin — Mon, 14 Sep 2026 18:15:54 +0000

One major release across the ecosystem: what's new in redb.Core, Route, Tsak and Identity, what got fixed on security, and what breaks.

A lot of this release was built burning the midnight oil. Feedback from people using the stack kept feeding both the thinking and the fixes: comments on articles, GitHub discussions, reports from live deployments. After a marathon stretch of work, 4.0.0 is out.

This is a major release for the whole ecosystem at once. The redb.Core store, the redb.Route integration engine, the redb.Tsak runtime and the redb.Identity OpenID server all ship under one version number: 76 packages on NuGet (up from 66 in 3.7), seven container images on GHCR, and archives for Windows and Linux. The Pro edition stays free and needs no license key across the entire 4.x line.

There is enough here that each product deserves its own deep dive, and those are coming. This post is the short version, close to a list: what's new, what was fixed on security, and what to know before upgrading. Full changelogs are at redbase.app/releases.

redb.Route: XML routes, a test kit and data formats

Ten new packages and a noticeable rework of the engine itself.

A shared vocabulary: the EIP book

Wire Tap, Content-Based Router, Splitter, Aggregator, Content Enricher, Claim Check, Idempotent Receiver: none of these names were invented for redb.Route. They all come from Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe and Bobby Woolf (Addison-Wesley, 2003), a catalog of 65 messaging patterns. Apache Camel, WSO2 Micro Integrator and redb.Route are all built on it, which makes it the common vocabulary of integration: once you recognize a pattern by name, you recognize it in every one of these tools. The redb.Route DSL verbs follow Camel's spelling: WireTap, Choice, Split, Aggregate, Enrich, ClaimCheck, IdempotentConsumer.

The whole catalog is free on the authors' site, with a diagram and a short explanation for every pattern: enterpriseintegrationpatterns.com. The chapters on Message Routing and Message Transformation are the place to start: per-partner formats, translators between them and a single point of entry are covered there pattern by pattern.

New packages

redb.Route.Xml. A route is described declaratively in .route.xml and loads into the same fluent DSL you write in C#. The whole document is parsed at load time, and every problem comes back in one list: an unknown element with a suggestion, a malformed expression, an endpoint scheme that isn't registered. The XSD is generated from the element registry, context.xml declares the context's components and beans, and a generator prints the XML back out as C#. The redb store is reachable straight from the markup through <redbGet> and <redbQuery>.
redb.Route.TestKit. Test a route without its brokers and without touching the route: AdviceRoute with ReplaceFrom, MockEndpoints("kafka://*"), a MockEndpoint with expectations and scripted replies, NotifyBuilder. No dependency on any test framework.
redb.Route.Templates. Message bodies from Scriban or Liquid templates. The output type drives escaping of substituted values, templates compile at Start(), and the sandbox blocks method calls.
redb.Route.DataFormats.Csv, .Protobuf, .Avro, .Yaml, plus Base64, GZip and Zip in redb.Route itself. Protobuf and Avro support Confluent framing.
redb.Route.JsonTransform. JSON-to-JSON transformation with JSONata on a native .NET engine.
redb.Route.Cache. Cache as an EIP: a Cache(key, ttl)…EndCache() scope and a cache: component, in process or over any IDistributedCache.
redb.Route.XPath2. XPath 2.0 for what 1.0 can't do: regular expressions, sequences, if, for, some and every.

The engine

One expression language. Conditions, values and ${...} templates go through one grammar and one compiler instead of three engines. A broken condition now fails Start() instead of quietly turning into a constant on live traffic. New in the language: stats(target, metric) for endpoint statistics, format(value, pattern, culture), uuid() and the % operator.
The bean: component. Your own code as an ordinary endpoint: the object comes from the context registry or is created with constructor DI, and URI parameters bind to its properties.
Interception and completion in the Apache Camel shape: Intercept, InterceptFrom, InterceptSendToEndpoint, OnCompletion.
Aggregation without lambdas. A library of ready-made AggregationStrategies (collect bodies into a list, concatenate, sum, min and max by expression, merge into a header or property), Enrich without a strategy, and a throttle whose limit comes from the message. Aggregate, RecipientList, DynamicRouter, IdempotentConsumer, Enrich and PollEnrich accept an expression, not just a delegate.
REST DSL in redb.Route.Http. Rest("/api/orders").Get("/{id}") with JSON binding, Consumes/Produces, and OpenAPI 3.0.3 served at {basePath}/openapi.json.
Metrics from inside a route. IExchange.Context and UseMetricsSnapshot() make the OpenTelemetry layer readable in process.
No more Newtonsoft.Json. jpath moved to JsonPath.Net (RFC 9535) on System.Text.Json, which leaves redb.Route with a single JSON model.

Load and hosting

Per-endpoint admission limits. HTTP, SOAP, AS2 and gRPC gained maxConcurrentRequests and a wait queue: an excess request gets 429 with Retry-After before the pipeline runs, and a neighboring route on the same port keeps its own budget. Rejections land in a separate Rejected counter, so shedding load never looks like failing.
concurrentConsumers=auto on RabbitMQ, AMQP 1.0, IBM MQ, SQS and MQTT: one consumer per processor, with a minimum of two. The default is still one, which preserves message order.
WebSocket and SignalR on the shared Kestrel host, next to HTTP, gRPC, SOAP and AS2: one port, handshake authentication supplied by the host, and a backplane seam for scale-out.
Trusted proxies on the shared host. X-Forwarded-For is walked right to left past every listed proxy, X-Forwarded-Proto restores the client's scheme, and every transport on the host sees the real address. A reader's comment on the 3.7 write-up prompted this one.

Connectors

Kafka: a commit on partition revocation no longer loses a record that was read but not yet processed, and seekTo applies per partition.
S3: the consumer no longer loses objects, and connection settings actually reach the AWS SDK.
Firebase: FCM multicast and topic management, copy and signed URLs in Storage, streaming downloads, and databaseId for Firestore projects with several databases.
Llm: system prompt caching on Anthropic with cache token counters, a fixed preamble ahead of the conversation history, HTTP/2 keep-alive so long answers survive VPNs and NAT, speech-to-text through stt://, and file downloads from Telegram.
SOAP: the consumer serves TLS, a route knows who called it, and the route chooses the fault code.
TLS without a certificate no longer opens a plaintext port anywhere: WebSocket, SignalR and TCP refuse to start in that configuration.

redb.Core(.Pro): lazy references and unique keys

Everything below behaves the same on PostgreSQL, MSSQL and SQLite, in Free and in Pro.

Lazy references. A reference at the depth boundary arrives as a stub carrying id, scheme and hash, and the first access to Props loads exactly that object. A reference that should stay lazy at any depth is marked virtual, with EnableLazyReferences switched on. A collection of stubs loads in one query through LoadReferencesAsync, with no N+1. Saving the parent, computing hashes and serializing never wake a stub.
The ValueUnique object key. A readable string (order number, SKU, external id), unique within the scheme, with upsert by that key through SaveByUniqueAsync.
[RedbUnique] on Props fields. Uniqueness enforced by a database index for scalars of any type, including decimal, dates and byte[]; for fields of nested classes; for a whole nested class, array or dictionary, where the key is the content of the subtree; and for collection elements within one object or across the scheme. GetByUniqueAsync is a single index probe, and a violation surfaces as one RedbUniqueViolationException on all three databases. Duplicates already sitting in existing data go into a report instead of failing startup.
A contract for schema upgrades. The schema module is applied at application startup. If the application's role can't change the schema, startup stops with a typed RedbSchemaOutdatedException, and the script for your DBA comes from GetUpgradeScript() or redb schema --upgrade.
CancellationToken across the entire async API. A cancelled SaveAsync rolls back completely, and the token is never read after the commit: "cancelled" can never mean "but actually saved".
Write interceptors (IRedbSaveInterceptor) in the EF style: SavingAsync can adjust the object or veto the save, SavedAsync sees the outcome, and the same pair exists for deletes. Under ChangeTracking the interceptor receives the applied diff with property paths, ready to feed your own audit journal.
Transaction isolation level on demand through BeginTransactionAsync(IsolationLevel), with DbErrorClassifier.IsSerializationFailure for retrying serializable transactions.
Maintenance: AnalyzeAsync refreshes planner statistics in one call on any database, and GetIndexStatsAsync reads index health.
LoadJsonAsync(id, depth) returns an object as raw JSON with no CLR type, and RedbObject.ToString() returns the same canonical JSON.
byte[] is stored as one BLOB instead of a row per byte, and the old layout converts itself. A scheme remembers the namespace of its type, so two Order classes from different projects can no longer adopt each other's scheme.
Performance: indexes on the _values FK columns, faster ChangeTracking, SQLite hashes stored as BLOB(16), and an index on _value_string on MSSQL too.
Fixes that affected 3.x: saving a parent whose reference was set by id only overwrote the referenced object; DateOnly on existing databases read back as 0001-01-01; MSSQL cut nested references in LINQ results at depth 1 where PostgreSQL and SQLite used 10.

redb.Tsak: a cluster on database keys and modules from XML

The cluster rests on database unique keys. Locks, nodes, groups and module assignments are backed by redb.Core partial unique indexes. Two nodes starting on an empty database no longer create two cluster roots: the node that loses the race adopts the winner. Assignment writes carry the leader's epoch, so a stale leader can't place one module on two nodes. An existing database backfills its keys on the first start.
Modules from XML. A .tpkg can carry nothing but .route.xml and context.xml, with no assembly at all, and such a package hot-reloads just like a C# module. An XML route in the worker sees every markup element from the shared layer: <cache>, <rest>, <redbGet>.
The dashboard tells shedding from silence. A Shedding Routes panel shows routes that hit their admission limit, and endpoints now show Rejected and Bytes Out. Page auto-refresh survives a transient error, and monitoring charts no longer freeze under the cursor.
About page: dashboard and node versions, uptime, and for admins the list of redb assemblies actually loaded.
Trusted proxies: Tsak:Http:TrustedProxies takes addresses and CIDR networks, after which the API key throttle and module routes see the real client behind a chain of proxies.
The Audit and Dead-letter pages work on PostgreSQL. The daily tsak_audit_log retention sweep started working on PostgreSQL at the same time, so on a node that has lived on Postgres for a while, check the table's size before the first sweep runs.
Cluster protocol: a heartbeat no longer reverts a concurrent cordon, a cordoned node gets no new modules, and a clustered route stopped through the API starts again.
Cluster-ready defaults: on a shared database, Quartz ships in clustered mode (on SQLite it switches itself off, with a warning in the log), and a node's ApiEndpoint is detected automatically instead of every node registering localhost:9090.

redb.Identity: WS-Trust, token audiences and a console built from pages

WS-Trust facade (redb.Identity.Soap). A third transport after HTTP and gRPC: Issue, Validate, Cancel and Renew over SOAP, for systems that generate their clients from WSDL. Issuer, client registry and token store are shared, so a client registered over HTTP gets a token over SOAP. The facade refuses to start without TLS, and mTLS with certificate thumbprint pinning is available.
RFC 9068 audiences in access tokens. The resources of granted scopes and the application's new AccessTokenAudiences go into aud, and the management API accepts only tokens that carry its own audience.
Contracts in a carrier package of their own. Hot-reloading one module no longer splits the shared DTOs into two copies, which used to make typed bodies vanish on their way through direct-vm://.
Uniqueness on redb.Core primitives. On MSSQL, ClientId, ScopeName and ReferenceId are protected by an index for the first time, and an external identity can no longer be linked to two local users by a race.
An admin console built from pages instead of dialogs. Federation providers get a full page with claim mappings, a claim mapper's owner is now selectable, users are created through a wizard, and there's a new API resources catalog plus a dashboard with KPIs and audit by category.
DPoP behind a TLS-terminating proxy, and the correct client_id in the audit log for clients that send credentials in the form body.

Security

Some of these came in as external reports; others turned up in our own audit of the facades and the dashboard. The mechanics of each will be in the per-product deep dives. If you run 3.x in production, this list alone is a reason to upgrade.

redb.Identity: an HTTP header could impersonate any user. Internal-only headers are now stripped at the edge of all three facades: HTTP, gRPC and SOAP.
redb.Identity: the consent page took the application name and scope list from its own URL, and the consent form could be submitted on behalf of another user without their session. The page now renders from a server-signed ticket, the user comes only from the session, and the forms check Origin and refuse to be framed.
redb.Identity: introspection answered any authenticated client about any token. It now answers only members of the token's audience.
redb.Tsak: dashboard pages opened without a login. Every page is now protected, and access tiers match the worker API: audit and users for Admin, dead-letter and logs for Operator.
redb.Tsak: module signatures are verified on every load path, process startup included, and the bytes loaded are exactly the bytes verified. Passwords no longer reach the audit log, sessions are revalidated every five minutes, and the login throttle keys on username and client address together.
redb.Core: the default password hasher is now bcrypt. Legacy SHA256 hashes still verify and move to bcrypt on the next password change.

Breaking changes

The list is short, but go through it before upgrading.

redb.Tsak and redb.Identity clusters upgrade with a stop. Nodes of one group must not run mixed versions: stop the whole group, deploy 4.0, start it again. The first start backfills the keys.
A large database needs a maintenance window. The upgrade adds columns and builds partial indexes under exclusive locks. It's easier to apply GetUpgradeScript() ahead of time from one node and start the application with AutoApplyDatabaseUpgrades = false.
redb.Core: Props-level lazy loading is gone entirely (EnableLazyLoadingForProps, WithLazyLoading(), the lazyLoadProps parameter). _objects._value_string is capped at 450 characters, and a database holding longer values refuses the upgrade with a list of objects to move to _note or Props. Accessing Props on a reference beyond the depth now goes to the database; for Blazor WebAssembly there's LazyReferenceAccess = Throw. Assemblies compiled against 3.x need a rebuild.
redb.Route: the string forms of LoopExpression, DelayExpression and ThrottleExpression became Loop, Delay and Throttle, and SetBodyExpression("…") and its siblings became SetBody(Expr("…")). Newtonsoft.Json no longer arrives transitively. ${...} renders numbers and dates culture-invariant, ${...} in a consumer URI fails Start(), and zero, "0", "no" and "off" are now false in conditions.
redb.Identity: a backend that introspects another client's tokens (the classic SPA and BFF pair) must be added to that client's AccessTokenAudiences. The SQLite audit table changed its timestamp column type, so delete SQLite audit files before upgrading.

Getting it

dotnet add package redb.Core
dotnet add package redb.Postgres        # or redb.MSSql / redb.SQLite
dotnet add package redb.Postgres.Pro    # Pro: free, no license key

dotnet add package redb.Route
dotnet add package redb.Route.TestKit

Libraries target net8.0, net9.0 and net10.0; host applications and images are built on .NET 10. Tsak and Identity ship as archives for Windows and Linux (redb-tsak, redb-identity) and as images on GHCR. The easiest way to try Tsak is the redb-tsak-stack image, which runs the worker and the dashboard in one container.

Packages: nuget.org/profiles/relikt. Source: github.com/redbase-app.

What's next

Each product gets its own deep dive: redb.Core with code for lazy references and keys, redb.Route with the test kit and XML routes, and Tsak and Identity with their own stories. A VS Code extension for XML routes, with a text mode and a graph editor, is in the works.

Thanks to everyone who left comments, opened discussions and sent reports from production: a good part of this release grew out of them.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

Your access token was stolen. Now what: three layers of defence in your own OpenID provider

rinat kozin — Tue, 08 Sep 2026 17:48:00 +0000

A bearer token works for whoever holds it. Three layers in redb.Identity: BFF, key-bound tokens via DPoP, fast revocation. Plus what a stolen database yields.

An access token is a bearer token by default, and that phrase means exactly what it says: whoever holds it, is you. The server cannot tell the real owner from someone who pulled the token out of a browser that was not theirs. Until it expires, that token works from any machine, in any country, and in your logs it looks like perfectly ordinary traffic.

Then comes the unpleasant part. Access-token lifetimes in real configurations run from fifteen minutes to an hour. For all of that time the attacker has full access, and nobody gets a signal about it.

We built redb.Identity, our own OAuth 2.1 / OpenID Connect provider on .NET, and "what if the token gets stolen" was not a question we could answer with a slogan. What came out of it is three layers: do not let it be stolen, make the stolen thing worthless, kill it fast. Below is each one, with code and with the limits of where it applies.

After that, a nastier scenario: what is left for an attacker who walked off not with a token, but with the entire database.

Where token theft actually comes from

Discussions about stolen tokens tend to collapse into XSS. That is correct but incomplete. If a token lives in the browser, all of the following reach it:

XSS in your own application, the classic;
XSS through a dependency. A compromised npm package gets exactly the same access to the page as the code you wrote. You can build a flawless application and still hand out tokens, because somewhere in the transitive dependency tree one package changed hands. Auditing your own code does not close this one;
browser extensions. The user installs them, and they have access to the DOM and to the page's network calls;
localStorage. Readable by any JavaScript on the same origin, and it survives reloads and tab closes. Storing a token there is a discipline of its own, and enough has been written about it.

What all four share: the attacker walks away with the token itself. After that they need neither the victim's browser, nor their network, nor their session.

Layer one: do not let it be stolen

How most identity servers build their admin console

Take any mature identity server and look at how its admin console is built. Keycloak ships a React application. WSO2 Identity Server ships React too, for both Console and My Account. Both operate as public OAuth clients: the browser runs code+PKCE, receives an access token, and keeps it.

This is not sloppiness, it is the default that settled in for SPAs years ago. It has a price. An access token with administrator rights over the identity server sits in page memory that JavaScript can read. Any of the four vectors above, landing on that page, means leaking the admin token of the very server that grants access to every other system in the company.

Worth noting that the IETF's current BCP for browser apps, OAuth 2.0 for Browser-Based Applications, recommends the BFF pattern and describes browser-held tokens as something to move away from. The industry has voted; mature products are simply carrying compatibility with what was written earlier.

What a BFF is, briefly

Backend-for-Frontend: a server-side application sits between the browser and the API. It runs the OIDC exchange over the back channel, server to server, and keeps the tokens. The browser gets an HttpOnly session cookie. There is no token in the browser at all, and JavaScript cannot reach it by construction rather than by agreement.

How ours is built

redb.Identity.Web is the reference admin console and account portal. It runs on Blazor Server plus cookie authentication:

builder.Services.AddRazorComponents().AddInteractiveServerComponents();
// ...
.AddCookie(options =>
{
    options.Cookie.Name = "identity.web.session";
    options.Cookie.HttpOnly = true;
    options.Cookie.SecurePolicy = CookieSecurePolicy.Always;
    options.Cookie.SameSite = SameSiteMode.Lax;
    options.ExpireTimeSpan = TimeSpan.FromHours(8);
    options.SlidingExpiration = true;

After login the tokens go onto the authentication ticket, not out to the client:

AuthenticationTokenExtensions.StoreTokens(props, BuildTokens(result));

When server-side code needs an access token to call Identity, it takes it from the current context rather than from the browser's request:

public async Task<string?> GetAccessTokenAsync(CancellationToken ct = default)
{
    var ctx = _accessor.HttpContext;
    if (ctx?.User?.Identity is not ClaimsIdentity { IsAuthenticated: true })
        return null;

    return await ctx.GetTokenAsync(CookieAuthenticationDefaults.AuthenticationScheme, "access_token");
}

Intermediate state works the same way. The MFA challenge, the consent challenge and the impersonation state are separate HttpOnly cookies under DataProtection, not fields in localStorage and not query parameters:

SameSite = SameSiteMode.Lax,
HttpOnly = true,

Blazor Server adds a level that a classic BFF does not have. Markup renders on the server and only a diff travels to the browser over SignalR. There is physically no JavaScript bundle holding application state. There is nothing to attack, not because we hid it well, but because nothing is there.

Where BFF does not help

With a BFF, an attacker who achieves JavaScript execution on the page can still call the API as the victim: the browser attaches the cookie automatically. What they cannot do is take the token away and work with it later.

	Stolen token (SPA)	XSS behind a BFF
Where the attacker operates from	their own machine, anywhere	only from the victim's browser
For how long	the token's whole lifetime	while the page stays open
Survives closing the tab	yes	no
Can it be replayed later	yes, the token is gone	no, there is nothing to take
Odds of catching it by anomaly	higher: foreign IP, foreign user-agent, foreign geography	lower: IP, user-agent and working hours all match the victim

That last row runs against the BFF, and it deserves to be said out loud because it breaks the convenient picture. A stolen token is used from someone else's machine, which is precisely the signal impossible-travel detection exists for. XSS inside the victim's browser arrives from their own address, with their user-agent, during their working hours, and is behaviourally almost indistinguishable from the person. So on detectability the BFF loses, and the symmetric "different IP" argument does not work in our favour.

A caveat to the caveat: you can only cash in that advantage if you actually have anomaly detection. We do not have it yet, which gets its own section further down.

So the BFF wins four rows out of five and clearly loses the fifth. It converts a quiet hour of access from another country into an active session inside the victim's browser while they are looking at the screen. A smaller blast radius, not a cure, and certainly not a substitute for monitoring.

Take a cookie, remember CSRF

This consequence is easy to forget in the relief of having no token around. A browser never attaches a bearer header on its own, but it does attach a cookie to any request aimed at your domain, whatever page that request came from. In other words, moving to a BFF does not remove the risk, it changes its shape: token theft is replaced by cross-site request forgery.

So the BFF runs both lines of defence. Session cookies and every piece of intermediate state carry SameSite=Lax, HttpOnly and Secure. On top of that the pipeline enables app.UseAntiforgery(), and every form that changes anything carries <AntiforgeryToken />: login, MFA challenge, consent, email change, password reset, device verification, profile edit. Twelve forms, none without a token.

Both lines are needed together. SameSite=Lax blocks cross-site POSTs, but it is not a full replacement for an antiforgery token: it does not save you from an attack launched from your own subdomain, and it stops helping the moment some flow needs SameSite=None. The rule is simple. You took a cookie, so bring CSRF protection, no matter how good the other flags are.

What about mobile clients

The BFF is a browser pattern, and native applications do not fit into it. Telling a mobile app to embed a web view for the sake of a BFF would in fact violate RFC 8252 directly: it requires the system browser for authorisation, not a component living inside the app.

Native gets a different combination:

the system browser for authorisation per RFC 8252, plus PKCE, which OAuth 2.1 makes mandatory;
the operating system key store for tokens, Keychain and Keystore, rather than a file in the app sandbox;
layer two, meaning DPoP, and here it carries the main weight. The private key is generated inside the device's secure storage and never leaves it, so a token lifted out of the app is useless without it.

The precise way to put it: BFF solves the problem for the browser, DPoP solves it for everything else. A browser has somewhere to hide the token server-side, a mobile app does not, so what gets protected there is not storage but use.

Layer two: make the theft worthless

This is DPoP, RFC 9449, Demonstrating Proof-of-Possession. The idea is simple and radical: stop issuing bearer tokens.

The client generates a key pair and keeps the private half. When asking for a token it presents the public half, and the server stamps that key's thumbprint into the issued token. From then on every request to a resource carries a separate short JWT proof, signed with the private key and bound to a specific method and URL:

POST /api/orders HTTP/1.1
Authorization: DPoP eyJhbGciOiJSUzI1NiIsInR5cCI6...
DPoP: eyJ0eXAiOiJkcG9wK2p3dCIsImFsZyI6IkVTMjU2Iiwiandr...

Steal the token without the private key and you have a useless string. The server will demand a proof, and there is nothing to sign it with.

What is implemented on our side:

binding at issuance. The key thumbprint per RFC 7638 (SHA-256 over the JWK, base64url) goes into the cnf.jkt claim per RFC 7800. The token now knows which key it belongs to;
DPoP-Nonce per §8, built on HMAC-signed stateless nonces. The server can demand a fresh proof without holding per-client state;
a store of consumed jti values keyed by jkt. The same proof will not pass twice, even if it was intercepted whole;
asymmetric algorithms only, from an allow-list, as §4.2 requires. A symmetric "proof" proves nothing, because both sides know the key;
a separate package, redb.Identity.Resource.Dpop. This is the validator for your own resource APIs. Proofs must be checked somewhere other than the provider itself, otherwise the protection ends at the identity server's boundary while the data lives further along.

A note on comparisons. DPoP now exists in Keycloak and in WSO2 as well. It arrived there later, and in Keycloak it sat in preview status for a long while, but "they do not have it" would be wrong. The right question is different: check your specific version for whether it is on and what status it carries, because a preview feature in production is a separate conversation with your own security team.

Layer three: kill it fast

Even perfect binding does not remove the need to revoke. Somebody left the company, a device was lost, an incident was confirmed. From there only one question matters: how long until the revocation actually reaches every system.

Corporate requirement documents love this point, and it usually appears in them as an open question rather than as a decision.

What exists:

token revocation per RFC 7009, idempotent: a repeated revocation returns 200 as §2.1 requires, and does not turn into a source of script failures;
refresh-token rotation. A used refresh token is revoked, and presenting it again means somebody is working from a copy;
session idle timeout. Every session carries its last-activity time, refreshed on real actions: a refresh exchange, cookie validation, a userinfo call. A session that has idled past the limit is killed automatically;
backchannel logout in two modes. This is the interesting one, so in more detail.

Classic OIDC Backchannel Logout is push: the provider knocks on every registered application and reports that a session has ended. That works right up to the first network partition or crashed replica. An application the knock never reached goes on believing the session is alive.

So next to push we run a pull feed of revoked session identifiers, with a cursor:

POST /api/v1/identity/revoked-sids/add
GET  /api/v1/identity/revoked-sids/since?cursor=...

An application that came back from a crash, or lost connectivity for ten minutes, simply asks what has been revoked since its cursor and catches up. No revocation is lost because a node happened to be unreachable when the broadcast went out.

And a detail that ties layer one to layer three. The BFF checks that same list on every request while validating the session cookie:

options.Events.OnValidatePrincipal = async ctx =>
{
    var sid = ctx.Principal?.FindFirst("sid")?.Value;
    var sub = ctx.Principal?.FindFirst("sub")?.Value;
    // if sid/sub is in the cluster-wide revoked list, drop the cookie

So "sign out everywhere" kills not only tokens but the live UI session too, across every replica rather than just the one that happened to receive the logout call.

The routine around it

Three layers are the visible part. Below them sits work you only notice if you go looking. The parts I consider worth listing:

Password history. Configurable depth, reuse of a previous password is rejected, hashed with the same algorithm as the current one.

TOTP with replay protection. The problem with a stock TOTP implementation: a code is valid across a tolerance window, and the same code can be presented twice if you are quick. We store the last accepted step and reject a code from a step already used, even when it formally falls inside the window:

if (props.LastTotpStep.HasValue && step <= props.LastTotpStep.Value)
    return false;

On top of that the MFA row is taken under a lock before verification, so parallel attempts by one user cannot diverge on the read-then-write.

SMS and email one-time codes live server-side, not in state. The code itself is stored hashed (SHA-256) and marked consumed under LockForUpdate, while the encrypted challenge state carries only a reference to it. The client never holds anything the code could be reconstructed from.

Recovery codes are genuinely single-use. They are marked consumed inside the same transaction that creates the session. Not "mark first, create after", but atomically, otherwise the gap between the two operations becomes a window.

Constant-time secret comparison. CryptographicOperations.FixedTimeEquals appears in sixteen files: password-reset tokens, email verification, email change, server-side OTPs, the bootstrap secret. A timing attack on string comparison is not exotic, it is what automated scanners look for.

Rate limiting on three levels: per IP, per client_id through a token bucket, and a separate ceiling on failed attempts per (IP + user) pair, logged to a dedicated security channel.

Order of proxy-header processing. X-Forwarded-For is sanitised before the rate limiter and the lockout counter ever see it. Do it the other way around and an attacker forges the header to bypass both, and can also get somebody else's IP locked out. It only applies when the socket peer is on the trusted-proxy allow-list, otherwise the header is ignored entirely.

The idempotency cache sits after authorisation, not before. Otherwise a revoked token unlocks a cached response produced while it was still alive.

The __Host- prefix is emitted only when Secure=true. RFC 6265bis §4.1.3.2 requires it, and it is easy to get wrong: set the prefix, forget the flag, and end up with a cookie the browser silently drops while you hunt for the bug in your code.

SSRF protection on outbound fetches. There are exactly two situations where a client can make our server follow a link it supplied: resolving jwks_uri and fetching a request object by request_uri. Both go through one guard that refuses loopback, RFC 1918 private ranges, link-local including the 169.254.169.254 cloud metadata address, and RFC 6598 CGNAT space. Private targets open only behind an explicit flag, for single-host test rigs.

Refusal of alg:none. Our JAR implementation (RFC 9101) accepts a signed request object, verifies the signature against the client's keys and takes the parameters from inside the JWT. An unsigned object is rejected always, whatever the settings say: it throws away precisely the integrity guarantee JAR exists to provide. FAPI 2.0 forbids it outright.

One hundred and nine typed audit events across seven categories, into a flat table and optionally multicast to Kafka, Elasticsearch or RabbitMQ. Passwords and secrets never reach the audit: a client-secret rotation records the fact, not the value.

What if the whole database is stolen

So far this has been about one token. Now the worst case: the attacker has a full dump.

The threat model changes completely. No rate limits, no audit, no revocation. The attacker works offline, on their own hardware, for as long as they like, and you will not find out. The only thing protecting the data at that moment is the shape it is stored in.

Passwords

Hashed with Argon2id, using the OWASP 2023 parameters: 64 MiB of memory, 3 iterations, 4 lanes, a 16-byte per-user salt and a 32-byte hash.

Three properties that buys. The password is hashed rather than encrypted, so no key exists anywhere that could decrypt everything at once. Each user gets their own salt, so rainbow tables are useless and you cannot crack everyone whose password is qwerty123 in a single pass. The computation is deliberately slow, so instead of billions of attempts per second the attacker gets a handful.

Then memory, which is the real difference from bcrypt. Bcrypt needs roughly 4 KB per computation. On a GPU or a purpose-built ASIC that means tens of thousands of parallel instances, and the gap between a defender on an ordinary CPU and an attacker on a farm becomes enormous. Argon2id at 64 MiB breaks that arithmetic: 24 GB of video memory holds on the order of 380 lanes instead of tens of thousands. This is why Argon2id won the Password Hashing Competition and why OWASP lists it first.

Bcrypt has not gone anywhere, and that is deliberate. The redb.Core storage layer historically hashed passwords with bcrypt at work factor 12, and existing deployments have exactly that in their tables. So Identity registers a dispatcher that writes new hashes with Argon2id while still verifying old bcrypt and even ancient salted SHA-256:

builder.Services.TryAddSingleton<IPasswordHasher>(sp =>
{
    var argon2 = new Argon2idPasswordHasher(...);   // 64 MiB, t=3, p=4
    var bcrypt = new BcryptPasswordHasher(workFactor: opts.Bcrypt.WorkFactor);

    if (opts.Algorithm == PasswordHashAlgorithm.Bcrypt)
        return bcrypt;

    return new MultiFormatPasswordHasher(argon2, bcrypt);
});

Upgrade-on-login is wired too: on a successful sign-in the hasher is asked whether the stored format is stale, and if so the password is quietly rehashed to Argon2id and saved. The database migrates itself as people log in. No forced password reset for everyone at once, the kind users hate and support desks survive like a natural disaster.

Everything else in the database

Passwords are one row among many. The full inventory:

What is stored	In what form
User passwords	Argon2id, legacy bcrypt until first sign-in
Password history	same hasher as the current password
OAuth client secrets	bcrypt hash, not recoverable
TOTP secrets	encrypted
Recovery codes	PBKDF2-HMAC-SHA256, per-code salt, plus a pepper
SMS and email one-time codes	SHA-256, single-use, consumed under a lock
Password-reset, verification and email-change tokens	hashed, compared in constant time
Private signing keys	encrypted through DataProtection
DataProtection key ring	encrypted at rest

Two of those deserve their own paragraphs.

The pepper does not live in the database

Recovery codes are protected by more than a salt. The formula also takes a pepper, a separate secret supplied through an environment variable and never stored in a table.

That is exactly the difference between a salt and a pepper. A salt sits next to the hash and defeats rainbow tables, but it does nothing against brute force. A pepper does not sit next to it. With only a dump in hand there is nothing to brute-force recovery codes with, because the formula uses a value the dump does not contain.

The key chain ends at a root that is not in the dump

This is the part that matters most when an identity server's database is stolen. The worst outcome is not "the passwords were cracked", it is "the private signing key was obtained". With that, a token can be forged for anyone, administrators included, and no amount of password hardening helps.

Private signing keys are stored encrypted, in an EncryptedPem field protected by IDataProtector.Protect under a dedicated purpose. The obvious next question is what protects the DataProtection key ring itself, given that it also lives in the database. Otherwise you get a circle where the lock and the key sit in the same box.

There is no circle. The key ring is encrypted at rest by one of three means: an X.509 certificate, a 32-byte AES-GCM master key, or your own hook into a KMS or Vault. And this is not a recommendation in the documentation, it is a startup condition:

if (dp.RequireAtRestEncryption && !options.AllowEphemeralKeys)
{
    throw new InvalidOperationException(
        "DataProtection key-ring is unprotected at rest. Configure ONE of: ...");
}

RequireAtRestEncryption defaults to true. The server refuses to start in production if the key ring is unprotected. Turning the check off takes an explicit setting, named clearly enough that you will not do it by accident.

The net result: a dump yields encrypted signing keys, an encrypted key ring, and no root key. Ciphertext all the way down.

What a stolen database does achieve

Without this part the paragraphs above would be lying by omission.

Encryption does not undo a breach. A dump readily gives up personal data (names, emails, phone numbers, departments, managers, employee numbers), organisational structure (groups, roles, who has access to what), session metadata (IP addresses, devices, activity times, meaning who works from where) and the entire audit trail.

Under most data-protection regimes that is a personal-data breach with every obligation attached, regardless of the fact that not a single password was cracked. Saying "everything is encrypted" in that situation closes a conversation that must not be closed.

And a second, operational caveat. The whole chain rests on the root key living somewhere other than the database. If your backup is taken together with the configuration file, or with the container's environment variables, the protection collapses to zero: the attacker has the box and the key. Database backups and secret storage belong in different perimeters with different access rights. RequireAtRestEncryption protects against a stolen dump, not against a stolen backup, and those are different things.

What is actually exposed

A separate question, often more important than any cryptographic detail: which part of the system is reachable from the internet at all.

In most identity servers the admin console lives in the same process and on the same port as the protocol endpoints, under a path like /admin. It can be isolated, there are host settings and a reverse proxy for that, but the separation is by URL and by configuration. One mistake in the proxy rules or a regression in the hostname settings, and the management plane is outside.

Ours separates by process and by port.

The core is not networked at all. Every redb.Identity.Core endpoint lives on a direct-vm:// route, an in-process transport. This is not "listening on localhost": there is no network path to the core in principle, except through a facade you explicitly stood up.

Inside the HTTP facade the planes sit on different ports:

"Http": {
  "PublicPort": 5002,
  "ManagementPort": null
}

PublicPort carries only /connect/* and /.well-known/*. ManagementPort carries /api/v1/identity/* and /scim/*, and the setting's own comment states in plain words that production wants a separate firewalled port. SCIM is additionally gated by a flag and can stay down entirely.

The emergency bootstrap endpoint is separated further. POST /internal/bootstrap-admin lives on the management port but outside the /api/v1/identity/ base path, so a firewall rule can close it independently of everything else. It has no bearer authentication by design; protection is a header secret compared in constant time. CORS on it is disabled deliberately, because it is a back-channel operator tool and is never invoked from a browser.

The UI is a separate application. redb.Identity.Web is a standalone ASP.NET host that talks to Identity over a server-side HTTP client. A different machine, a different network segment, or not deployed at all: Identity does not stop working either way. It ships as source rather than as a package precisely because it is a reference you are meant to edit.

The point of the whole arrangement is the cost of a mistake. Exposing the management plane here requires deliberately publishing the management port or deploying the Web app into a DMZ. That is hard to do by accident.

An external arbiter

Everything above can be asserted about any server, and usually is. The trouble is that your own tests find exactly the bugs you already knew to look for.

We ran the server through the official OpenID Foundation conformance suite, the same one the Foundation certifies providers with. Config OP passes with no failures. Basic OP is thirty-five modules, and FAILED among them is zero.

Far more interesting is what it found in us. Scope-derived claims (phone, address) were landing in the id_token, and an id_token is forwarded to third parties and written to logs as proof of sign-in. So a user's phone number travelled considerably further than the client ever asked for. That is a PII leak, and none of our own tests caught it, because we did not know it was a bug. There is a full write-up of that run, including suite setup and the complete list of findings: running our OpenID server through the official suite.

Two modules in the report show as SKIPPED, and both concern unsigned request objects with alg:none. They are skipped because the server refuses to advertise the unsafe mode. In a conformance report you read the reason, not the colour of the row.

We do not carry the OpenID Certified™ mark. That is a trademark, granted by the Foundation through a separate paid submission. What is claimed is only what is true: the server is run against the official suite, and these are the results.

What is not there yet

The section without which this would be an advertisement.

Risk-based authentication. The inputs are already stored: session IP, user-agent, a readable device label, sign-in history in the audit trail, failed-attempt counters. What is missing is the rules engine and the escalation policy on top of them.

You can see the gap most precisely in acr. The server reports the level reached: acr of 1 for single-factor and 2 for verified multi-factor, plus amr with the concrete method (pwd, otp, mfa, hwk). What it does not act on is an incoming acr_values as a demand to raise the level. The parameter is advertised in discovery, but you cannot use it to make the server ask for a second factor. That is exactly what step-up needs, along with RFC 9470.

Trusted devices as a first-class entity. The device is recorded on the session, but there is no record saying "trust this device until such a date".

SAML 2.0. Not implemented in either direction. The decision is deliberate and written down: OIDC federation covers modern integrations, while full SAML means XML signatures, metadata exchange, three binding types and Single Logout with all its quirks. The conditions under which we would take it on are written down too, and the first is a real customer with an enterprise IdP that will not speak OIDC.

From the RFCs: Rich Authorization Requests (9396), the JWT profile for access tokens (9068, the typ=at+jwt header), the iss parameter in the authorisation response (9207), step-up (9470). Plus CIBA and OIDC Federation 1.0.

On mTLS specifically, because this one is easy to misrepresent. Transport mTLS exists: the gRPC facade can require a client certificate and pin it by thumbprint against an allow-list, and for the management port that is the recommended production setting. What does not exist is RFC 8705, which is a different thing: mTLS as a way of authenticating an OAuth client at the token endpoint, plus binding the issued token to the certificate through cnf.x5t#S256. The first protects the channel, the second would make the token as non-transferable as DPoP already makes it. We went the DPoP route.

FAPI 2.0 is not claimed as a profile. Individual bricks from it are in place: mandatory PAR can be switched on per client, alg:none is always rejected, and only asymmetric algorithms are allowed for request objects. But a profile is not a set of checkboxes, it is passing the corresponding conformance plan, and we have not run it. If your security team requires FAPI, treat that as a separate body of work, and better to learn it now.

What to take away

Three thoughts, if any survive the scroll.

First. A token in the browser is a choice, not a necessity. The BFF is the recommended pattern for browser applications, and moving to it changes not the probability of compromise but its radius: instead of a quiet hour of access from another country you get activity inside the victim's browser, bounded by an open tab. The price of the trade is that such activity is harder to catch by anomaly, which is worth holding in mind rather than treating the BFF as a free improvement.

Second. A bearer token is also a choice. DPoP turns a stolen string into a useless one, it switches on at the provider rather than by rewriting every client at once, and it works where a BFF cannot: mobile and desktop native, service-to-service calls. Check whether your server has it, and what status it carries.

Third. Revocation speed is an architecture question, not a token-lifetime question. Push notifications to applications break at the first network partition. A pull feed with a cursor survives partitions and crashed nodes, and costs very little.

Fourth, on database theft. Check two things about your own server, both of which take five minutes. One: what hashes your passwords and whether upgrade-on-login exists, because without it the database never migrates to a modern algorithm. Two, and this matters more: where the key lives that encrypts your private signing keys. If it is in the same database, the encryption is decorative, and the worst case of a stolen dump is wide open.

redb.Identity is Apache 2.0, runs on PostgreSQL, MS SQL and SQLite with no code changes, and can be hosted as an in-process module if you do not want a network between your services. Have a look on GitHub.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

WS-Trust over OAuth: a third transport for the OpenID server, for the estate that runs on SOAP

rinat kozin — Tue, 01 Sep 2026 20:07:53 +0000

redb.Identity gained a WS-Trust facade: Issue, Validate, Cancel, Renew on the same core routes. The same JWT, the same client registry, WSDL for the generator.

When we built the gRPC facade the claim was this: the logic of redb.Identity lives in the core behind direct-vm://identity-* addresses, and a transport is a thin translator on top of it. The second facade supported the claim. But two transports, both modern, both built on roughly the same assumptions about the world, test it gently. The real test is a third one built on entirely different assumptions.

It now exists. Next to HTTP and gRPC stands WS-Trust: Issue, Validate, Cancel, Renew over SOAP, on the same core routes, with the same issuer and the same token store. A token minted over SOAP is accepted over HTTP and gets the same verdict there.

Who this is for

There are estates where SOAP is not "legacy" but the current norm. Banking back offices, insurance, industrial control, government. They run WCF and CXF buses, their integrations are generated from WSDL, and the team maintaining them knows how to debug an envelope and does not know how to debug an OAuth redirect. Asking such an estate to "just fetch a token over HTTP with a form-encoded body" is asking it to write and maintain code for which it has neither the tooling nor the habits.

WS-Trust is the OAuth of that world. It is older, it is more verbose, it is about XML, but its job is exactly the same: a client arrives with its credentials and leaves with a token. If the authorization server speaks it, the integration stops being a project and becomes a client generated from WSDL.

From the consumer's side it looks like this:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Header>
    <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <wsse:UsernameToken>
        <wsse:Username>billing-service</wsse:Username>
        <wsse:Password>...</wsse:Password>
      </wsse:UsernameToken>
    </wsse:Security>
  </soap:Header>
  <soap:Body>
    <wst:RequestSecurityToken xmlns:wst="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
      <wst:RequestType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</wst:RequestType>
    </wst:RequestSecurityToken>
  </soap:Body>
</soap:Envelope>

Not a word about OAuth. Inside it is an ordinary client_credentials call to the same core that serves HTTP.

What was actually added

Four WS-Trust operations, laid onto core routes:

Operation	Where it goes	What it means
`Issue`	`identity-token`	mint a token
`Renew`	`identity-token`	refresh it
`Validate`	`identity-introspect`	is it still alive
`Cancel`	`identity-revoke`	revoke it

Client credentials arrive in a UsernameToken and are translated into client_id and client_secret, the same fields the core takes from everyone else. The facade has no authentication path of its own, and that is the point: a second path would mean a second place where it is decided who gets in.

One detail breaks the shape gRPC taught us. There, every operation is its own address and routing is free. WS-Trust puts all four on one address and tells them apart by the WS-Addressing Action header. Generated clients expect exactly that, so the operation is resolved by parsing the request rather than by the address it arrived on.

What goes into the RSTR

Our ordinary JWT, as it is, in a wsse:BinarySecurityToken with a value type declaring it a JWT.

That was a decision, not a default, and it was made before any code. The alternative was to build a SAML assertion: its own claims model, its own conditions, its own XML signature. The difference is not the writing effort but that the system would gain a second token format, and it would have to be taken through revocation and introspection, which today know one.

We chose to start with JWT and return to SAML when a client turns up that requires it, rather than on an assumption about the market. The cost is stated plainly: some WS-Trust clients expect SAML specifically and will not work with a JWT. For them this facade covers part of the niche, not all of it.

In exchange, what would otherwise have taken weeks works immediately: Validate and Cancel operate on a token minted over SOAP, because it is literally the same object that travels over HTTP.

A refusal arrives as a fault, not as a body

A client told the call succeeded has no reason to look inside the answer. So a refusal from the core has to arrive as a soap:Fault with a code of its own: generated WS-Trust clients branch on the code, and FailedAuthentication is not the same outcome to them as InvalidRequest.

The translation is arranged so as not to become a third independent copy. Reading the core's verdict lives once, in the shared IdentityVerdict; what remains on the SOAP side is rendering that verdict into its own vocabulary:

Core verdict	Fault
`Unauthenticated`, `Forbidden`	`wst:FailedAuthentication`
`InvalidScope`	`wst:InvalidScope`
`RateLimited`, `Unavailable`, `Timeout`, `ServerError`	`soap:Server`
everything else	`wst:InvalidRequest`

WS-Trust names considerably fewer outcomes than HTTP does, so several verdicts land on one fault. Where they do, the reason text carries what the code cannot, which is why it is never dropped.

A word on RateLimited. WS-Trust has no code for "wait", and soap:Server is the closer of the two available readings: the refusal is ours, not the caller's, and there is nothing in their request to rewrite.

The real test: one token, two transports, one verdict

Everything else can be attributed to careful implementation. This cannot.

A client is registered over HTTP through /connect/register. A token is fetched over SOAP. That token is then presented back over HTTP on an admin endpoint, and it is accepted. And the other way round: a read-only client is refused a write on both transports.

That is what the word "facade" means here. Not "a SOAP-compatible authorization server alongside the real one", but the same server with a third door.

Boundaries, named up front

No browser flows: authorize, the consent screen, MFA, device confirmation. The reason is not about SOAP; they need a browser, redirects and a cookie session. That niche has its own answer, WS-Federation, and it is filed as a separate decision rather than as an omission.

No DPoP either: RFC 9449 binds its proof to an HTTP method and URL.

No management surface in the first version. gRPC has one, and an estate arriving for WS-Trust usually needs tokens issued, not users administered over SOAP.

TLS is mandatory here, and the facade will not start without it

A UsernameToken in its ordinary form carries the client's password in clear text. An STS on plain HTTP therefore publishes credentials to everyone on the path.

We did not write a warning about that into the log. A warning is read afterwards, a refusal is read beforehand, so the facade simply does not start:

redb.Identity.Soap refuses to start without TLS: WS-Security UsernameToken carries
the client secret in clear text.

There is exactly one way out and it is named explicitly: AllowPlaintext, for a run behind a proxy that already terminates TLS. It is the operator's statement that the wire is protected some other way, not a way around the check. Setting it silently is not possible.

Client certificates are supported at the TLS layer, with modes and thumbprint pinning, the same way as on the gRPC facade.

The WSDL is served on GET

The audience for this facade builds clients with generators, and a generator needs a document. A GET on the same address serves the WSDL with soap:address rewritten to the URL it was fetched from. From there dotnet-svcutil, svcutil or wsimport do their work.

An STS that answers only POST is simply unusable for this audience.

How to see it in ten seconds

There is nothing to install: SOAP is ordinary HTTP with an XML body, so the demo is written against plain Invoke-WebRequest and prints everything that goes out and comes back.

pwsh -File demos/demo_soap_facade.ps1

Ten steps: the WSDL, registering a client over HTTP, Issue over SOAP, Validate, Cancel, a second Validate that now answers negatively, a refusal on a wrong secret with its fault code, an anonymous request, a junk body answered with wst:InvalidRequest, and finally a token minted over SOAP and spent over HTTP.

What covers it

Twenty-eight tests on the facade itself: parsing the RST, translating into core parameters, the fault table, end-to-end runs through a live listener into a real core, and separately a check that the per-IP rate limit actually works behind the facade.

That last one deserves a note. The core keys its rate limiting, its lockout after failed logins and its device metadata on the caller's address. With no address those checks do not fail, they quietly do nothing, which looks exactly like "no abuse". So forwarding the address was not enough; we had to see the limit actually trip behind the facade.

What had to be fixed in the connector itself

The facade is built on redb.Route.Soap, and working on it exposed three holes in the connector. All three are general, none specific to Identity.

A route could not choose its fault code. The consumer took only the text from the exception and always set the code to soap:Server. Every refusal caused by the caller therefore arrived as "we broke", leaving them to wait for us to fix what was theirs to fix.

The consumer could not serve TLS. It registered the listener with a "needs HTTPS" flag but passed neither certificate, nor password, nor client-certificate mode. And the listener DSL could not produce the soaps scheme at all. In that state an HTTPS SOAP endpoint could not be brought up.

A prefixed fault code was written without declaring the prefix. A fault code is a QName in both SOAP versions, and wst:FailedAuthentication with wst unbound reads fine to the eye and does not resolve in a strict client. Strict clients are precisely the ones that branch on the code.

The connector now has fifty-six tests.

Deployment

The facade ships as a separate Tsak module with its own identity.soap context and no compile-time reference to redb.Identity.Core at all: it talks to the core only through direct-vm://.

"identity.soap": {
  "IdentityTransport": {
    "Soap": {
      "Host": "0.0.0.0",
      "Port": 5021,
      "Path": "/sts",
      "Ssl": false,
      "AllowPlaintext": true,
      "ClientCertificateMode": "NoCertificate",
      "Wsdl": true,
      "EmitHttpCompatHeaders": true
    }
  }
}

The Ssl and AllowPlaintext values here are the ones that ship in the box, and they are meant for development. In production you set Ssl: true with a certificate and drop AllowPlaintext. We left them this way deliberately: a module that refuses to start on a fresh machine reads as broken rather than as intended.

The port is separate from the HTTP facade by default. Technically SOAP is ordinary HTTP/1.1 and could share a host. The separation buys something else: WS-Trust can be closed off from the internet while plain OIDC stays open, and the two surfaces get their own firewall rules.

EmitHttpCompatHeaders is on by default and should stay on, for the reason described above about rate limiting.

What comes next

The facade is built, covered by tests and a demo. It has not yet been exercised on a live worker, and until that run we do not call it proven in the field.

After that, as circumstances dictate. The management surface can be added with the same technique already worked out on gRPC. SAML assertions will appear when a client turns up that needs them specifically.

If you have a WCF or CXF estate, the cheapest check takes a minute: bring up the worker and feed your generator the address http://localhost:5021/sts?wsdl. It will build the client, and that client will bring back a real token from the same core that serves your HTTP.

There is a separate write-up on the second transport: the OpenID server gained gRPC alongside HTTP. And another on putting this same server through the official OpenID Foundation conformance suite: running our OpenID server through the official suite.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

Microservices in .NET without building a platform: a cluster of workers, one dashboard, hot-swapped module

rinat kozin — Mon, 31 Aug 2026 18:54:51 +0000

The same artifact deploys as a monolith and as a cluster of microservices. Same modules, one dashboard over every worker, hot swap with no container restart.

When a team splits a monolith into microservices, it pays for the split in operations. There used to be one process: one set of metrics, one log, one knob labelled "restart that thing". Now there are nine processes, and each has its own story about how to look at the dead-letter queue, how to stop one route without dropping the rest, and how to roll out a new version without hitting the window where nobody is running it.

Usually that operations layer gets built from scratch: Prometheus, Grafana, a hand-rolled health controller, a deploy script, a chat bot for restarts. It takes about as long as the split itself.

redb.Tsak offers a different trade: the operations layer lives in the runtime, and it is the same one whether you run one worker or nine. Cluster, dashboard, REST API, CLI, probes, metrics and tracing do not depend on the topology you picked. You decide how to carve up the processes, not how to operate them afterwards.

How a module becomes a running service with a dashboard and a deployment story was a separate article. It ended on a teaser about the cluster. This is that sequel.

One artifact, two topologies

Tsak is a worker (a container or a self-contained archive) plus the modules that arrive into it. A module is a .tpkg, an ordinary ZIP holding a manifest, DLLs and configuration.

Orders.tpkg
├── manifest.json          { "Name": "Orders", "EntryPoints": ["Orders.dll"] }
├── Orders.dll             entry point: ITsakModule or InitRoute.main
├── Orders.config.json     the module's business settings
└── Orders.Domain.dll      a private dependency

From there the topology decides, not the code.

As a monolith: every .tpkg goes into one worker's modules/. Each gets its own isolated AssemblyLoadContext, its own route context, its own property bag and its own lifecycle. One process, many independent parts. Calls between modules go through direct-vm://: the same exchange, no serialization, no loopback hop, no TLS handshake.

As microservices: the same .tpkg files spread across separate workers. One worker per module or per group of modules, a container each, all pointed at one database and joined into a cluster.

The module code does not change by a single line. What changes is where the file was put and what the environment variables say. That is the property that matters: "monolith or microservices" stops being an architectural decision you make on day one and becomes a deployment parameter you can change on a Monday.

Microservices are the recommended shape, and here is why.

What it looks like on a live project

A real dev topology from one project: three workers, a module each, plus one web console covering all three.

# Shared by every worker: one database, one cluster, one broker factory.
x-db: &db
  ConnectionStrings__Postgres: "Host=${PG_HOST};Database=app;Username=..."

x-cluster: &cluster
  Tsak__Cluster__Enabled: "true"
  Tsak__Cluster__ClusterName: app
  Tsak__Cluster__GroupName: app

x-rmq: &rmq
  Tsak__Contexts__default__RabbitMq__Host: ${RMQ_HOST}
  Tsak__Contexts__default__RabbitMq__Exchange: app.events

services:
  app-core:
    image: ghcr.io/redbase-app/redb-tsak-worker:3.7.2-net10
    environment:
      <<: [*db, *cluster, *rmq]
      Tsak__Modules__AssemblyPaths__0: /app/modules
      Tsak__Cluster__NodeId: app-core-1
      Tsak__Cluster__ApiEndpoint: http://app-core:9090
    volumes: [ "./output/core:/app/modules" ]

  app-api:
    image: ghcr.io/redbase-app/redb-tsak-worker:3.7.2-net10
    environment:
      <<: [*db, *cluster, *rmq]
      Tsak__Modules__AssemblyPaths__0: /app/modules
      Tsak__Cluster__NodeId: app-api-1
      Tsak__Cluster__ApiEndpoint: http://app-api:9090
    volumes: [ "./output/api:/app/modules" ]

  app-integration:
    image: ghcr.io/redbase-app/redb-tsak-worker:3.7.2-net10
    environment:
      <<: [*db, *cluster, *rmq]
      Tsak__Modules__AssemblyPaths__0: /app/modules
      Tsak__Cluster__NodeId: app-integration-1
      Tsak__Cluster__ApiEndpoint: http://app-integration:9090
    volumes: [ "./output/integration:/app/modules" ]

  # One web console for all three workers.
  tsak-web:
    image: ghcr.io/redbase-app/redb-tsak-web:3.7.2-net10
    environment:
      <<: [*db, *cluster]
      Tsak__Web__Mode: cluster
      Tsak__Web__ServiceApiKey: ${TSAK_WEB_API_KEY}

Two things there are worth noticing.

First, the three workers run the same image. The official redb-tsak-worker, not something built for this project. The only differences are the mounted modules folder and the NodeId. A custom image is what you build when you want the modules baked in rather than mounted, and that is a choice rather than a requirement.

Second, the web console does not list the nodes. In cluster mode it reads them from the cluster topology, which lives in the same database. Add a fourth worker and it shows up in the dashboard on its own.

The cluster forms around that shared database: leader election, node registration, heartbeats and module distribution are data in redb rather than separate infrastructure. No ZooKeeper, no etcd, no Consul, no Redis to install. A single worker with clustering on is already a working one-node cluster: it elects itself leader and starts handing out routes.

Why microservices are the better default

The monolith layout works honestly, and for a small system it is simpler. Past a certain size, though, separate processes give you what in-process isolation cannot.

Failure stops at the container boundary. AssemblyLoadContext isolation protects you from version conflicts and from one module reaching into another's statics. It does not protect you from an OutOfMemoryException, from a native library that took the process down, or from a leak that ate the working set. A separate container does.

Limits go on the thing that needs them. An integration module pulling hundred-megabyte XML and an API module returning JSON need different memory. In one process they share a single ceiling, and that ceiling is set by the worst case.

You scale what is loaded. Three API replicas and one integration replica is an ordinary picture. In a monolith you scale everything together, including the parts that did not ask for it.

Release cadences come apart. Updating one .tpkg does not make you think about the other eight.

Ownership boundaries match team boundaries. Each team gets its worker, its modules, its row in the dashboard and its set of endpoints.

The honest price. Splitting turns free in-process direct-vm:// calls into network calls. That is the cost, and it should be paid deliberately. The consolation is that the route code changes its address, not its logic: a producer publishes to a logical endpoint, and what that resolves to (seda: inside the process, rabbitmq: between processes) is configuration. The sensible tactic is to keep tightly coupled modules in one worker and cut along the boundary where the conversation is already asynchronous through a broker.

Microservices do not take hot reload away

Splitting into microservices usually costs you hot reload. The reasoning goes: a service is a container, an update is a new image, a new image is a pod restart. Want it faster, build your own plugin system.

In Tsak hot swap does not depend on topology. It lives in the worker, and there can be any number of workers.

# Updating a module on a running worker: replace one file.
cp ./output/Orders.tpkg /app/modules/

HotReloadService notices the changed timestamp and performs a graceful swap: it brings up a new AssemblyLoadContext, lets it settle, waits for the old context to finish its in-flight messages, stops it and releases it. Nothing is dropped, the process does not restart, and the other modules never notice.

Deleting a file is a first-class deployment operation rather than an error: rm modules/Orders.tpkg stops every module of that package atomically, closes its transports and connections and disposes its isolated context. Neighbouring packages keep running.

In a cluster RollingUpdate kicks in and nodes update in sequence. There is never a moment when zero nodes run the new version, and never a moment when in-flight messages are lost.

The knobs you actually turn in production:

Key	Default	What it does
`HotReload:ScanIntervalSeconds`	`10`	How often the module directories are scanned.
`HotReload:RollingUpdate`	`true`	In a cluster, nodes update one after another rather than together.
`HotReload:StartupTimeoutSeconds`	`60`	How long to let the new version settle before retiring the old one.
`HotReload:KeepVersions`	`2`	How many previous versions stay available for a one-command rollback.
`HotReload:RemovalDebounceScans`	`2`	How many consecutive scans a file must stay missing to count as deleted. Protects against atomic replacement, where the file vanishes for an instant.
`HotReload:AdditionStabilityScans`	`2`	How many consecutive scans a new file must hold the same size and timestamp before it is opened. Protects against reading a half-copied archive.
`HotReload:Collectible`	`false`	Full unload of the assembly context. Off deliberately, explained near the end.

The last two arrived in 3.7.0, after working out why a large archive an operator dropped in by hand was sometimes picked up half-written.

Configuration reloads hot as well: editing context.json or {Module}.config.json re-merges the layers and restarts the affected context. The worker stays up.

Groups, nodes, assignments: slice the isolation however you like

The topology is a three-level tree, and it lives in redb as ordinary objects:

cluster:default                      scheme _tsak_clusters
 └── group:default:default           scheme _tsak_groups
      ├── node:default:worker-1      scheme _tsak_nodes
      ├── node:default:worker-2
      └── node:default:worker-3

Every level is a boundary you can put to work.

A cluster separates environments and products. Different ClusterName values in one database do not see each other.

A group isolates leader election, assignment and rebalancing. Inside one cluster you can keep an edge group for the workers facing outward and a batch group for the nightly processing, and a re-election in one leaves the other alone.

A node is a worker. It registers itself, sends a heartbeat every 15 seconds, and is evicted from the registry after 60 seconds of silence. The leader lock is taken for 30 seconds and renewed; every state change is stamped with the leader's epoch, so a leader that lost the election cannot corrupt state after the fact.

A context is one more level, this time inside a worker. A named context joins several modules under one property bag and one lifecycle; an anonymous one is created for every module that was not assigned anywhere.

{ "Tsak": { "Contexts": {
    "api": { "Modules": ["Api.Orders", "Api.Catalog"], "AutoStart": true }
}}}

Out of those pieces almost any layout assembles: from one worker holding every module to nine workers holding one each, grouped by area of responsibility.

What the cluster adds on top:

Module distribution across nodes. The leader spreads contexts over the live nodes and reassigns them when a node joins or leaves. The strategy today is round-robin, weighted ones are planned.
Active-passive for a consumer route, out of the box. A route reading Kafka or firing on a schedule runs on exactly one node. The node dies, the lock expires, a neighbour takes over. No duplicates, no downtime.
Cordon and uncordon. A node steps out of assignment without stopping: tsak cluster cordon node-2, its modules move to the others, and the node can be serviced.
Scheduled work as cluster singletons. The built-in daily jobs (audit and dead-letter retention sweeps) are marked .Cluster(true) and run on one node rather than on every one.
Swappable coordination. ILeaderElection, IDistributedLock, INodeRegistry, IClusterCoordinator, IClusterBootstrap and IAssignmentManager are interfaces. If coordinating through the database is not what you want, register your own implementation, say one over Kubernetes Lease objects, with a single DI line before AddTsakCluster(). Nothing else changes.

One dashboard over every worker

The web console is a separate Blazor Server process that finds the nodes itself in cluster mode. Its sidebar splits into three groups: the whole cluster, the sections of the selected node, and settings. This is an operator's workplace: the console is where you act, not only where you look.

Cluster overview

Section	What is there
Dashboard	Node statuses, a status donut, metric sparklines, a sortable and filterable node table.
Cluster	The three-level topology tree, module assignments, per-node health, click-through into a node.

Inside a node, eleven sections, switched in the same sidebar without losing the selected node:

Section	What is there
Overview	Process cards: CPU, working set, managed memory, threads, thread-pool queue, collections per GC generation.
Contexts	The node's route contexts with status and endpoint count, plus start, stop and restart.
Endpoints	Consumer and producer endpoints per route.
Routes	Every route of every context at once: status, message count, error rate, click-through.
Route detail	One route in full: definition, current state, the exchanges in flight right now, recent diagnostics.
Watchdog	Suspected and hung routes with stop and restart buttons.
Modules	Loaded modules: name, version, status, dependencies, description.
Scheduler	Quartz jobs: group, cron, state, next fire time, pause, resume, fire now.
Monitoring	Four live Chart.js graphs: CPU, memory, threads, garbage collection. Refreshed every ten seconds, twelve hours of history.
Logs	The ring-buffer log viewer with search, level filter and tail mode.
Audit	The trail of admin actions, who pressed what.
Dead-letter	Failed exchanges: list, replay, discard.

Settings

Section	What is there
Auth & Users	API keys and accounts: create, revoke with confirmation.

Four items on that list deserve their own paragraph.

Per-node monitoring. The graphs are drawn from the worker's own metric history rather than from an external system. Prometheus and Grafana plug in alongside and are not going anywhere, but seeing what one node is doing right now does not require standing them up.

Exchanges in flight. The route detail shows which exchanges are sitting in a route at this moment. When a queue is not draining, "is it stuck or just slow" is answered by looking, not by attaching a debugger to production.

Watchdog. The service continuously classifies routes and tells suspected apart from hung. It will restart them on its own if you let it.

Replaying a failed exchange. The replay button in Dead-letter performs exactly one replay even if two operators press it at the same instant: the claim is taken with a conditional UPDATE, and the database decides which of the two won.

The console runs on its own design system, no Bootstrap, no MUI, no Tailwind: CSS variables, system fonts, dark and light themes, inline SVG icons.

Management, not just observation

The dashboard is one of three heads. Under all of them sits the same REST API, 70 endpoints across 16 controllers, all speaking JSON.

Group	Endpoints	About
`/api/health`	3	Kubernetes probes: startup, live, ready. Auth-exempt.
`/api/system`	6	Health, metrics, metric history, process info, effective configuration, loaded assemblies.
`/api/contexts`	8	List, start, stop, restart, reset route states, endpoints, remove.
`/api/contexts/{ctx}/routes`	8	Routes: start, stop, force-stop, in-flight exchanges, metrics.
`/api/modules`	6	List, remove, upload, validate signature, roll back.
`/api/scheduler`	8	Scheduler: status, scheduled jobs, running jobs, pause, resume, fire now.
`/api/cluster`	6	Status, nodes, rebalance, remove node, cordon, uncordon.
`/api/watchdog`	6	Status, alerts, enable, test alert.
`/api/exchanges`	3	Dead-letter queue: list, replay, discard.
`/api/logs`	3	Incremental tail, file list, download.
`/api/diagnostics`	2	Cluster-wide and per-route dumps.
`/api/auth`	3	API keys: create, list, revoke.
`/api/users`	5	Users.
`/api/audit`	1	The admin-action trail.
`/api/lifecycle`	1	The lifecycle event feed.
`/api/dashboard`	1	An aggregated snapshot for the console in one round-trip.

The API itself is built nicely: it is an ordinary route context called _system, one HTTP listener whose pipeline reads "header bridge, auth, controller dispatch". Tsak manages itself with the same engine it runs your routes on.

The second head is the CLI: tsak, one binary, 57 commands, connection profiles, tabular output for humans and JSON for CI.

tsak login http://prod-1:9090 --key $PROD_KEY --profile prod
tsak profile use prod

tsak context list                    # a table
tsak context list --output json      # for jq in a pipeline

tsak route force-stop orders route-1 # take down a hung route
tsak route inflight orders route-1   # see what is sitting in it
tsak dlq replay 42                   # replay a failed exchange
tsak cluster cordon node-2           # step a node out of assignment
tsak module deploy ./Orders.tpkg     # ship a module
tsak module rollback Orders          # go back one version

The third head is a typed C# client, for automating operations from your own code:

services.AddTsakClient(o => { o.BaseUrl = "http://tsak-prod:9090"; o.ApiKey = key; });

public class Ops(ITsakApiClient tsak)
{
    public async Task RestartFailedAsync(CancellationToken ct)
    {
        var contexts = await tsak.ListContextsAsync(ct);
        foreach (var c in contexts.Where(c => c.Status == "Failed"))
            await tsak.RestartContextAsync(c.Name, ct);
    }
}

On access: keys are stored as an HMAC-SHA256 hash, the raw key is never persisted, comparison is constant-time, a key carries roles, an expiry and a revocation, and a revoked key stops being accepted across every node within thirty seconds. Inter-node calls use the same authentication: there is no implicit trust between nodes.

Since 3.7.0 all of it is also closed by default. The management API binds 127.0.0.1 rather than 0.0.0.0, so exposing it is a deliberate act. A roleless key is denied instead of being treated as admin. The console has a real server-side cookie session, the password is checked against a BCrypt hash, and the login is rate-limited.

Kubernetes, Prometheus, Jaeger

External observability plugs in without a shim, because Tsak was written for containers from the start.

Three probes, not one. Split by pod lifecycle phase, all three auth-exempt.

startupProbe:   { httpGet: { path: /api/health/startup, port: 9090 } }
livenessProbe:  { httpGet: { path: /api/health/live,    port: 9090 } }
readinessProbe: { httpGet: { path: /api/health/ready,   port: 9090 } }

The difference between liveness and readiness is deliberate. Liveness intentionally does not check module health, otherwise a rolling update would turn into a restart loop. Readiness is stricter: any context in a non-running state takes the pod out of the load balancer without restarting it, and the cluster redistributes assignments meanwhile.

Prometheus with no extra port. With Tsak:Metrics:Prometheus:Enabled the metrics are served at /metrics on the same port as the API. The OpenTelemetry listener sits on loopback, and the facade route proxies it out. No second port to open, and on Windows no URL ACL, because Kestrel binds the sockets rather than HttpListener.

metadata:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port:   "9090"
    prometheus.io/path:   "/metrics"

The exporter is pre-flighted: if the loopback bind ever fails, Tsak logs a warning and runs without metrics. An optional exporter has no business taking the worker down.

Jaeger and any OTLP collector. Traces leave through the standard OTLP exporter, Tsak:Tracing:Otlp:Enabled plus an endpoint. The redb.Route ActivitySource is registered in the OpenTelemetry pipeline, so the spans opened by route processors and transports are collected on their own. The service name in Jaeger comes from Tsak:Tracing:ServiceName, which is exactly right in a microservice layout: each worker carries its own name, and the trace is stitched across the broker.

Ready-made artifacts. The repository ships Kubernetes manifests (a Deployment with the correct probes, a Service, a ServiceMonitor for Prometheus Operator), an importable Grafana dashboard, and a local Prometheus + Grafana + Jaeger stack behind one docker compose.

Pod identity. In a cluster the NodeId has to survive a restart or assignments drift. The downward API binds it to the pod name:

env:
  - name: Tsak__Cluster__NodeId
    valueFrom: { fieldRef: { fieldPath: metadata.name } }
  - name: POD_IP
    valueFrom: { fieldRef: { fieldPath: status.podIP } }
  - name: Tsak__Cluster__ApiEndpoint
    value: http://$(POD_IP):9090

Graceful termination. Set Tsak:Shutdown:TimeoutSeconds five seconds below terminationGracePeriodSeconds so that deregistering the node keeps its buffer. The order is SIGTERM, cluster deregistration, context drain, scheduler shutdown, log flush. SIGKILL never gets its turn.

On top of that the worker counts for itself: process metrics with twelve hours of history sampled every ten seconds (4320 points), per-context and per-route metrics (messages per second, error rate, in-flight count), a two-thousand-entry ring-buffer log queryable over REST and in the console, and diagnostic dumps per route and across the cluster.

Identity arrives as a package in the same worker

The same module format carries a finished product. A full OAuth 2.1 and OpenID Connect server ships as a set of .tpkg files.

Four packages: redb.Identity.Core (the server itself: schemes, stores, MFA, WebAuthn, federation, audit, key rotation) plus three transport facades, Http, Grpc and Soap. The facades are thin bridges with no business logic.

It lays out however you want, on exactly the same logic as your own modules:

One worker: Core + Http, and you have an OP on its port.
Two workers: Core + Http faces outward, Core + Grpc serves service-to-service calls. Different cluster groups, different limits, different exposure.
Inside your own worker: put redb.Identity.Core next to your module and call it straight from your route.

// No HTTP, no serialization, no loopback: the same exchange.
From("http:0.0.0.0:5090/api/login")
    .To("direct-vm://identity-token");

This is the shared-runtime benefit in its purest form: two products that know nothing about each other end up in one process and talk without a network, because both speak in route addresses.

The substance is tested and proven: OIDC Core, OAuth 2.1, introspection, dynamic client registration, Device Code, PAR, JAR, DPoP, backchannel logout, SCIM 2.0, TOTP, OTP over SMS and email, WebAuthn. The official OpenID Foundation conformance suite passes with zero failures on the Config OP and Basic OP profiles.

And it is observed by the same dashboard as everything else: Identity routes appear in the shared list, its metrics in the shared graphs, its logs in the shared buffer.

Your own worker, when you want one

The module stays yours and runs anywhere. The entry point is either a type implementing ITsakModule or a public static class InitRoute with a main(IRouteContext) method. The second is an Apache Camel style convention, and anyone can call it.

A debug host in full:

var services = new ServiceCollection();
services.AddLogging(b => b.AddSimpleConsole());
services.AddRedb(o => o.UseSqlite("Data Source=echo_demo.db"));

var sp = services.BuildServiceProvider();
await sp.GetRequiredService<IRedbService>().InitializeAsync(ensureCreated: true);

var ctx = new RouteContext(sp, contextId: "echo-worker");
ctx.AddService(typeof(ILoggerFactory), sp.GetRequiredService<ILoggerFactory>());

EchoModule.InitRoute.main(ctx);   // the exact method the Tsak worker calls

await ctx.Start();

Fifty lines, and the module runs under a debugger in your IDE with breakpoints and single-stepping. That beats attaching to a hot-loaded assembly context inside a running worker.

The same trick works when you want a host of your own: your DI, your configuration, your metric collection. The redb.Tsak sources are Apache 2.0, the packages are on NuGet, and there is no closed runtime in the middle.

Five configuration layers: why one image serves nine workers

This is the mechanism that lets the three services above start from a single image.

Layer 1: Tsak:Contexts:default                 base for every context
Layer 2: Tsak:Contexts:{name}                  settings of one context
Layer 3: modules/{Module}/context.json         the module's infrastructure defaults
Layer 4: modules/{Module}/{Module}.config.json the module's business settings
Layer 5: Tsak:Contexts:{name}:Override         operations get the final word

The layers deep-merge: nested objects complete each other rather than replacing wholesale. A module brings sensible defaults inside its own archive, operations overrides what has to differ in production, and the module's code anticipates none of it.

The practical consequence for secrets: LDAP and SMTP passwords and JWT signing keys arrive through the Override layer from environment variables and land in named connection factories. They are not in the .tpkg, not in endpoint URIs, not in the logs and not in the dashboard.

On top of that, the worker's shared layer already carries 28 redb.Route connectors: RabbitMQ, Kafka, AMQP, Azure Service Bus, IBM MQ, SQS, Redis, S3, Elasticsearch, SQL, gRPC, SOAP, AS2, SignalR, WebSocket, MQTT, TCP, mail, files, FTP, SFTP, LDAP, Telegram, Firebase, LLM and the rest. A module talking to Kafka does not carry the driver with it: the worker already has it.

What it costs

The boundaries worth knowing before you adopt it.

Assembly-context unloading is off by default. HotReload:Collectible = false, deliberately: Reflection.Emit (used by XmlSerializer, serialization generators and compiled regular expressions) does not survive an unload. The price is that old assembly contexts stay in memory until the process restarts. Their count is exposed as the LeakedAlcCount metric, so it is visible. For a worker updated once a week this is immaterial; for one updated twenty times a day, a nightly restart settles it.

There is one distribution strategy. Round-robin. Weighted strategies are planned, the IAssignmentManager interface for them already exists, and your own implementation can be plugged in today.

Coordination lives in the database. Leader election, locks and the node registry are rows in redb. The upside is that no separate membership infrastructure is needed. The downside is that the database becomes a participant in coordination. If that does not suit you, all six coordination interfaces are replaceable in DI, for instance with an implementation over Kubernetes Lease objects, leaving redb as storage for modules and keys only.

Quartz in a cluster wants a real database. RAMJobStore for development, AdoJobStore for production. The schema is created on first start, with no DBA action required.

Trying it out

The fastest path is a single redb-tsak-stack container: worker and web console in one image, in the spirit of rabbitmq:management.

docker run -p 9090:9090 -p 8085:8085 \
  -v ./modules:/app/worker/modules \
  ghcr.io/redbase-app/redb-tsak-stack:3.7.2

The API comes up on 9090 and the dashboard on 8085. Drop a .tpkg into ./modules and ten seconds later the module is in the list and its routes are on the graphs. Neither a database nor a broker is needed for a first run: the default storage is in-memory.

When it comes to spreading across machines, the separate redb-tsak-worker and redb-tsak-web images take over, as in the compose file above. Ready-made templates for all four cases (worker only, console only, stack, stack with PostgreSQL) live in the repository under publish/docker/.

Without Docker there is a self-contained archive: unpack, run, put the modules next to it. That is the monolith layout in one file, with the same dashboard and the same API. Image and archive signatures are verified with cosign.

The current number on the line is 3.7.2. The libraries target net8.0, net9.0 and net10.0; applications and images are built on .NET 10 and tagged -net10. Pro stays proprietary but free, with no licence key, across the whole 3.x line, clustering included: there is no node limit. The worker's main unit suite passes in full on this line, 647 of 647 on .NET 10.

Sources: github.com/redbase-app/redb-tsak
Releases and archives: redb-tsak/releases
Images: the stack in one container, everything on GHCR
NuGet packages: nuget.org/profiles/relikt
The ecosystem as a whole: redbase.app

Starting with the monolith layout is easier, and moving to microservices can wait until there is a reason. The pleasant part is that the move costs an edit to a compose file rather than a rewrite.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

SOAP in .NET without WCF: WS-Security, MTOM and ?wsdl as a route step

rinat kozin — Fri, 28 Aug 2026 18:00:42 +0000

SOAP with WS-Security, MTOM and WSDL publishing is now a native redb.Route connector: no WCF, no CoreWCF, no standalone gateway.

SOAP did not go away. If you sell airline tickets, you talk to Amadeus, Sabre and Travelport over SOAP. If you integrate with a bank, a government portal, an insurance backend, a telco billing system or almost any enterprise product bought before 2015, the contract is a WSDL and the wire is a <soap:Envelope>. WS-Security signatures, MTOM attachments, SOAP 1.1 next to SOAP 1.2, a soap:Fault on error: that world is alive, it pays the bills, and it is not migrating to REST because you asked nicely.

In .NET, calling it has become awkward. WCF as a full framework is gone. CoreWCF exists but every published version carries an unpatched crypto advisory, so pulling it in trades one problem for another. dotnet-svcutil generates a client from a WSDL at build time, but that is code generation glued to the side of your app, not an integration step. And hosting a SOAP endpoint, adding WS-Security by hand, or wiring MTOM tends to end in a pile of System.ServiceModel configuration nobody wants to own.

redb.Route.Soap takes a different route. SOAP becomes an ordinary step in a pipeline in your own .NET process. Call a service, get a typed reply. Host an endpoint, hand the body to a route, return a response. Everything is in-box: HttpClient, the shared Kestrel host, and System.Security.Cryptography.Xml for WS-Security. No WCF, no System.ServiceModel runtime, no vulnerable dependency, no separate gateway. Let us walk through how it is used, and why a native connector inside the ESB beats a codegen client or a standalone box.

SOAP in one minute

services.AddRedbRoute(route =>
{
    route.Services.AddRedbRouteSoap();
    route.AddRouteBuilder<MyRoutes>();
});

// Call a service
From("direct://get-fares")
    .To(Soap.Call("https://gds/air.svc").ConnectionFactory("amadeus").Operation("GetFares"));

// Host a SOAP endpoint
From(Soap.Listen("/svc/orders").Host("0.0.0.0").Port(4090))
    .Process(HandleOrder);

AddRedbRouteSoap() registers the soap and soaps schemes and shares one Kestrel receive server with every other HTTP-based connector in the process. In the default Payload mode the message body is the XML of <soap:Body>: send a fragment, receive a fragment, no code generation, works against any service.

The endpoint is a string (or a fluent builder)

Every endpoint reads two equivalent ways. Some people prefer the type-safe builder, some prefer a plain URI they can drop into config. They compile to the same thing.

// Fluent
.To(Soap.Call("https://gds/air.svc").ConnectionFactory("amadeus").Operation("GetFares"))

// String URI (identical)
.To("soaps://gds/air.svc?connectionFactory=amadeus&operation=GetFares")

The consumer side is the same story:

From(Soap.Listen("/svc/orders").Host("0.0.0.0").Port(4090).ConnectionFactory("orders"))
// or
From("soap:/svc/orders?host=0.0.0.0&port=4090&connectionFactory=orders")

soap is HTTP, soaps is HTTPS. The producer address is soap[s]://host/path; the consumer is soap:/path?host=&port= with the path kept intact. Because the URI is just a string, the endpoint can come from appsettings.json and change per environment without touching code.

A service binding is one object, not a scatter of parameters

Certificates, credentials, the SOAP version and the data format never live in a URI. They live on a SoapConnectionFactory, registered once by name. Routes reference it with .ConnectionFactory("name").

context.AddToRegistry("amadeus", new SoapConnectionFactory
{
    EndpointUrl = "https://gds/air.svc",
    SoapVersion = SoapVersion.Soap11,
    DefaultAction = "urn:GetFares",

    // WS-Security material, all optional
    SigningCert = ourPfx,      // our cert + PRIVATE key: signs outgoing, decrypts incoming
    EncryptCert = partnerCer,  // partner's PUBLIC cert: encrypts outgoing, authenticates their signature
    Username = "svc", Password = "secret",   // UsernameToken

    Mtom = true,               // MTOM/XOP attachments
    Wsdl = "contracts/air.wsdl",              // published on GET ?wsdl (consumer)
});

The password field is marked sensitive, so it is redacted in logs and in the runtime dashboard. Move from staging to production by swapping the registered object, not by editing the routes.

Three ways the route sees the message

camel-cxf has data formats; so does this connector, on SoapConnectionFactory.DataFormat.

Payload (default). The body is the inner <soap:Body> XML. Works with any service, no types, no codegen.

Message. A transparent proxy: the body is the whole envelope, in and out. Log SOAP traffic, forward it untouched, inspect it.

Pojo. Typed request and response objects via XmlSerializer, the .NET analogue of JAXB document/literal. The DTOs are ordinary XML-serializable types and may be generated from a WSDL with dotnet-svcutil.

context.AddToRegistry("air", new SoapConnectionFactory {
    EndpointUrl = "https://gds/air.svc",
    DataFormat  = SoapDataFormat.Pojo,
    ResponseType = typeof(GetFaresResponse),
});

From("direct://q")
    .Process(e => e.In.Body = new GetFaresRequest { Route = "JFK-LHR" })
    .To(Soap.Call("https://gds/air.svc").ConnectionFactory("air"));
// e.Out.Body is now a GetFaresResponse

You choose the level of typing. A generic pass-through pipeline stays in Payload; a service with a fixed contract goes Pojo and works with real objects.

WS-Security that authenticates, in-box

WS-Security is where SOAP integrations usually get stuck. Here it is three fields on the factory and it runs on System.Security.Cryptography.Xml, no CoreWCF anywhere near it.

UsernameToken: set Username (and Password); the producer prepends a <wsse:Security> header, the consumer surfaces the credentials to the route.
XML-Signature: with SigningCert set, the producer signs the <soap:Body> (Exclusive C14N, SHA-256, embedded X.509); the consumer verifies.
XML-Encryption: with EncryptCert set, the producer encrypts the Body to the partner; the consumer decrypts with its private key.

The verification is the part that matters. When the partner certificate is configured, signature verification is authenticated: the signer must be that exact certificate, and the signature must cover the Body, so a forged self-signed signature or a signature over a decoy element is rejected. That is the difference between "the XML was not tampered" and "this message is really from the partner", and it is easy to get wrong.

The encryption goes out in the layout real WS-Security stacks produce and expect: the EncryptedKey in the <wsse:Security> header, joined by a ReferenceList to the EncryptedData in the Body, AES-256 under an RSA-OAEP key wrap. It is not a private dialect. An independent crypto stack decrypts it end to end (more on that below).

Binary attachments: MTOM

Large binaries do not belong base64-inflated inside the XML. Set Mtom = true and attachments travel as multipart/related with XOP, on a side plane so the Body contract stays clean, the same way Camel keeps them on an AttachmentMessage.

var msg = new Message(
    "<Upload xmlns=\"urn:svc\"><file>" +
    "<xop:Include xmlns:xop=\"http://www.w3.org/2004/08/xop/include\" href=\"cid:doc-1\"/></file></Upload>");
msg.Headers[SoapHeaders.Attachments] =
    new List<SoapAttachment> { new("doc-1", "application/pdf", pdfBytes) };
await producer.Process(new Exchange(msg));

On the receiving side the route reads inbound attachments off redbSoap.attachments, and replies with its own list. The connector will not silently echo the caller's attachments back.

Publish your WSDL on ?wsdl

.NET Core has no runtime WSDL import, by design, so the connector does not pretend to. What it does do is publish a contract. Point SoapConnectionFactory.Wsdl at a file or inline XML on a consumer, and it is served on GET {path}?wsdl with the <soap:address> rewritten to the address the caller actually reached you on. Pair it with Pojo mode for typed request and response, and you have a contract-first SOAP service.

SOAP as a controller

Here is the part that does not exist in a codegen client. SOAP is a first-class controller transport in redb.Route, next to HTTP, gRPC and SignalR. Write a controller, dispatch by operation.

[Route("air")]
public class AirController : RedbController
{
    // Method name = SOAP operation; the XML body binds in, the typed reply serializes out.
    public Task<GetFaresResponse> GetFares([FromBody] GetFares req)
        => Task.FromResult(new GetFaresResponse { Price = Quote(req.Route) });

    [SoapOperation("HealthCheck")]           // explicit name when it differs from the method
    public string Health() => "ok";
}

// Behind a SOAP endpoint:
From(Soap.Listen("/svc/air").Host("0.0.0.0").Port(4090))
    .RedbSoapController<AirController>();

The operation the SOAP consumer parsed from the request maps to a method, the XML body binds to the parameter, and the return value goes back as the response. No HTTP attributes, no manual envelope handling, a soap:Fault on error. The same controller class works behind any transport, so a service can speak SOAP and REST at once with one implementation.

Where it is used, and why native beats a gateway

The scenarios are the boring, load-bearing ones. A travel platform querying GDS fares over SOAP and republishing them as JSON. A bank integration that receives a signed SOAP request, validates it, and writes to a ledger. A government or healthcare portal with a strict WSDL contract you must serve exactly. A legacy ERP that only speaks SOAP 1.1 with a UsernameToken.

With a codegen client or a standalone gateway, SOAP lives beside your integration. You generate a proxy, or you run a separate process, and then you still have to move the message into your actual pipeline, translate the fault, correlate the trace across a boundary. With redb.Route.Soap it is one step of a route:

From(Soap.Listen("/svc/orders").Host("0.0.0.0").Port(8443).ConnectionFactory("self"))
    .Validate(Body().Matches(orderSchema))
    .Unmarshal("xml").Marshal("json")
    .To("kafka://orders")
    .Process(e => e.Out.Body = "<Ack xmlns=\"urn:svc\">accepted</Ack>");

One process, one deployment, one trace. The SOAP request lands, gets validated, transformed, dropped into Kafka, and the partner gets an envelope back, and all of it shows up in the same OpenTelemetry span tree as every other step. A Client span on the call, a Server span on the endpoint, endpoint statistics, secret redaction: the same cross-cutting behavior every other connector in the family has.

Interop proven, not claimed

A SOAP connector that only talks to itself is not a SOAP connector. This one is checked against an independent stack, the Node.js soap library, which shares no code with it, in both directions: our producer against their server, their client against our consumer, plain SOAP and MTOM. And the WS-Security encryption is decrypted end to end by an independent crypto stack (Node's OpenSSL) straight off our envelope: RSA-OAEP key unwrap, AES-256-CBC, the XML-Encryption padding stripped as the spec mandates. The wire is standard, and that is demonstrated, not asserted.

Honest boundaries

Runtime WSDL import (a client introspecting a remote WSDL on the fly) is absent from .NET Core by design; the connector serves a static WSDL contract and pairs it with Pojo types. WSDL is not generated from CLR types by reflection; contract-first is the model. A few rare WS-* variants (external CipherReference, non-OAEP key transport, UsernameToken PasswordDigest) are not handled yet. None of that blocks the mainline: SOAP 1.1 and 1.2, faults, the two header planes, WS-Security with authenticated signatures and standard-layout encryption, MTOM, WSDL publishing, and controllers.

FAQ

Do I need CoreWCF or System.ServiceModel? No. The baseline is HttpClient, the shared Kestrel host, and System.Security.Cryptography.Xml. CoreWCF was deliberately avoided because every version carries an unpatched crypto advisory.

SOAP 1.1 or 1.2? Both. SoapVersion drives the Content-Type and the SOAPAction placement; faults are parsed for both, regardless of the namespace prefix a WCF or CXF peer uses.

Can one service speak SOAP and REST? Yes, through the controller transport: the same RedbController sits behind Soap.Listen(...) and Http.Listen(...).

Where do certificates go? On the SoapConnectionFactory, as X509Certificate2. The password is redacted in logs and the dashboard.

Install

dotnet add package redb.Route.Soap

The package is redb.Route.Soap on NuGet; the source and the full DSL reference are in the connector README. SOAP is one more transport in the redb.Route family, alongside Kafka, RabbitMQ, AS2, IBM MQ and the rest: the same From → … → To, the same EIPs, the same observability. The only difference is that the message on the wire is a <soap:Envelope> a twenty-year-old enterprise system is waiting for.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

redb 3.7.1: props search up to 100x faster. An alternative to EF Core, or a companion to it

rinat kozin — Wed, 26 Aug 2026 17:14:21 +0000

Query cost did not depend on what you searched for: the filter sat above GROUP BY. Now the cut happens before the aggregate, on three engines, with no application code changed. Up to 100x on a date range and 5.3x on a full result set in SQL Server.

There is a class of performance problem you cannot see on small data and cannot miss on large. Ours looked like this: a query hunting for one rare order number among a hundred thousand objects took exactly as long as a search that found nothing at all. Selectivity did not move the clock. At all.

The cause was the shape of the generated SQL. Props values live one per row, so the query first folds them into a wide row through GROUP BY and only then applies the filter. The condition sat above the aggregate, filtering the result of a fold rather than a column. No index can help there: by the time the condition runs, the engine has already read and folded every value of every object in the scheme. The trigram index on strings sat unused.

3.7.1 adds a step that narrows the object set before the aggregate runs.

How it works

The planner walks the filter tree and tries to express it as a condition on a single value row. A row belongs to exactly one structure, so a disjunction falls out naturally: "this is a Position row and it contains the needle, or this is a Department row and it contains the needle". That condition is spliced straight into the value scan, and the fold now receives an already narrowed set.

The key property: the prefilter is built as a superset. It may let extra objects through, it may never lose one. The authoritative filter stays exactly where it was, above GROUP BY. Anything the planner cannot analyse yields no prefilter and today's behaviour unchanged, so the worst outcome is the absence of a speedup rather than a change of results.

What already worked before

Without this caveat the numbers below read wrong, so it goes here rather than in a footnote.

Cutting before the aggregate has always existed in redb. Just not for props, but for the object's own fields: id, parent, creation and modification dates, name. Such a condition was spliced straight into the value scan:

AND v._id_object IN (SELECT o_src._id FROM _objects o_src
                     WHERE o_src._id_scheme = $1 AND <your condition>)

In code that is WhereRedb, and it is what production queries have been leaning on all along. In our own production and in redb.Identity such filters are almost everywhere: pick within a subtree, within a date range, within a set of ids, by owner. As long as objects were cut by _objects, the fold received an already narrowed set and ran fast.

The hole was exactly where there is nothing to cut by on _objects. Then the only selective condition left was props, and props sat above the aggregate and cut nothing: one rare order number cost the same as searching for nothing. 3.7.1 closes that case, and the two cuts now compose: first by objects, then by value rows.

What was measured

Two shapes, both plain LINQ, no special calls. And both without WhereRedb, which is the worst case: nothing to narrow by on _objects, the whole weight falls on props.

The first is one search box across several fields. This is what a UI search looks like when the user has typed a word and you have to look in both the job title and the department.

var found = await redb.Query<Employee>()
    .Where(e => e.Position.Contains("Design") || e.Department.Contains("Design"))
    .OrderByRedb(o => o.Id)
    .Take(100)
    .ToListAsync();

One needle, Design, two fields, an OR between them. In the table below this is the needle row. Before 3.7.1 that query read and folded the values of every employee in the scheme and only then checked for "Design". Now the rows that obviously cannot match are cut on the way in.

The second is a date range matching 3 288 objects out of 100 000, or 3.3%.

var hired = await redb.Query<Employee>()
    .Where(e => e.HireDate >= new DateTime(2030, 1, 1)
             && e.HireDate <  new DateTime(2031, 1, 1))
    .Take(100)
    .ToListAsync();

The numbers

All three engines seeded identically: 100 000 objects, roughly 8.4M value rows, statistics refreshed. Server-side time, best of three runs.

query	PostgreSQL	SQLite	SQL Server
one needle across two string fields, ordered	188 → 91 ms	1150 → 311 ms	55 → 17 ms
the same, whole result, no paging	338 → 156 ms	1201 → 314 ms	7037 → 1327 ms
date range, 3.3% of the scheme	154 → 1.5 ms	17 → under 1 ms	2 → 2 ms

Read this table as the upper bound of the gain rather than as an expectation for any query. If you have a WhereRedb, you were already fast and the absolute difference will be smaller. The gain grows the less there was to narrow by on _objects.

One caveat about what is being measured. This is query time, without materialising objects in memory. A real call adds materialisation on top, and materialisation is identical in both modes. So the search itself got faster by exactly the ratios above, while the share of that gain in the end-to-end call depends on how many objects you pull.

Which is where a second, independent lever comes in: projections. Select fetches only the fields you actually need instead of the whole object. One mechanism cuts the engine's work, the other cuts the materialiser's, and they compose.

Three engines, three different mechanisms

The same algebra, and every engine wins for its own reason. Worth knowing before predicting numbers on your own data.

PostgreSQL engages the trigram GIN and reads far fewer rows: the string index returns 40 958 rows where the structure index returns 200 000. SQLite engages a covering partial index and stops going back to the table. SQL Server reads the same pages but saves CPU instead, because the string column is NVARCHAR(MAX) and the comparison carries an explicit collation.

The spread shows most clearly on the date range: a hundredfold gain on PostgreSQL, sixteenfold on SQLite, and nothing whatsoever on SQL Server.

That last one is not a misfire, it is a sign that there was nothing left to cut. The condition matches 3.3% of the scheme, the query asks for a hundred rows, and the work is already bounded by the limit rather than by the size of the scheme: at that density you find a hundred matches after visiting roughly three thousand objects. Two milliseconds before, two after. Remove the limit, force a walk over the whole scheme, and the gain shows up there too: 88 ms against 65. That is a useful rule of thumb for your own data. The prefilter pays off where the engine would otherwise fold the entire scheme, and gives nothing where a limit already keeps the work small.

What this means in practice

We got very close to a flat table with an index. On simple conditions, an equality or a range on one field, the query now travels an index the way it would travel an ordinary column of an ordinary table.

And it keeps what a flat schema cannot have by construction. No row multiplication from joins: values fold per object in a single pass, and related collections do not turn a result set into a cartesian product you then have to collapse. No Include either: the graph loads by depth rather than by naming every branch by hand. On deep graphs that lands somewhere a classic ORM built on joins and Include does not reach.

Hence the two-sided positioning in the title. Nobody is asking you to replace EF Core wholesale: the two live side by side in one application, on one database. The relational part, where the schema is stable and the table is flat, stays with EF, which is the right home for it. redb takes what a flat model finds hard: shifting sets of fields, heterogeneous entities inside one scheme, trees and graphs that would otherwise become a chain of Include and a pile of duplicated rows. The choice is about the shape of your data, not about ideology.

There is no head-to-head benchmark against EF in this article, and I am not going to invent one. What is described here are properties of the execution model, not a measurement we did not take.

That measurement is coming separately. A deep dive is in preparation with code for both sides, execution plans and numbers: a simple condition against a flat indexed table, a graph several levels deep against Include, collections where a join multiplies rows, and what projections do about all of it. This post is short and about one change; that one will be long and about where this model wins and where it loses.

The boundaries, honestly

The prefilter is opt-in through EnablePvtPrefilter and off by default.

Today it covers a disjunction over selective fields and a range or equality on a single field. A top-level AND across different fields does not qualify: a value row belongs to one structure, and a conjunction of different fields is not expressible at row level. Filters over arrays, dictionaries, null checks, cross-field comparisons and computed expressions are recognised as unanalysable and yield no prefilter.

There is also a case where the prefilter had to step aside. On SQLite a query with a limit and no ordering at all streams without the prefilter and stops at the hundredth group; with it the planner switches to a multi-index OR, loses the ordering, needs a temporary B-tree and materialises everything. Measured: 8-12 ms against 388-521 ms, with no overlap between the ranges. So on SQLite, in that single shape, no prefilter is emitted. PostgreSQL knows no such trouble, and SQL Server cannot even produce the offending shape, because its paging is required to carry an ORDER BY.

A word on how this was verified. The invariant that results must match row for row is held by a suite of differential tests: every query runs twice, with the flag and without, over the same data, and the id sets are compared. That suite caught a defect that would otherwise have shipped: the row form cuts rows rather than objects, so an object qualifying through one branch of a disjunction lost the column values belonging to the other branches. The object set stayed intact, which is why fifteen hundred existing tests noticed nothing, while DistinctBy and OrderBy over such a field quietly lied. Fixed before publication.

What you get

Search finally costs what you are searching for. A rare condition is now cheap instead of costing the same as a query that finds nothing.

Three engines, one flag, zero changes in application code. The same LINQ, the same model, nothing to rewrite: turn it on and go.

up to 100x faster on a date range in PostgreSQL, 154 ms became one and a half;
3.7x on string search in SQLite and twice as fast in PostgreSQL;
5.3x on a full result set in SQL Server, where seven seconds became 1.3.

And not a single row of any result changed. The prefilter is a superset by construction, and the invariant is pinned by differential tests that run every query both ways and compare id sets exactly. That is not a figure of speech but a working tool, and it is what caught the defect that would otherwise have shipped.

Compose that with projections and you get two independent levers: one cuts the engine's work, the other the materialiser's. On simple conditions the query travels an index the way a flat table would, and on deep graphs you keep what a flat schema does not have: no rows multiplied by joins, no Include written by hand.

Where it lives

The prefilter lives inside redb and works on all three Pro providers: PostgreSQL, SQL Server, SQLite. One flag, EnablePvtPrefilter in RedbServiceConfiguration, turns it on. It is a step of query compilation rather than a separate mode: the same LINQ, the same result set, the same result, the same observability. The difference is that the aggregate now receives not the whole scheme, but only what could possibly match.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

redb.Route: Control Bus for .NET routes: stop, restart and react to route events at runtime

rinat kozin — Wed, 26 Aug 2026 09:31:00 +0000

Manage routes at runtime by sending a message: start, stop, suspend, restart, plus a consumer that turns route lifecycle events into messages.

A running integration is not a static thing. A deploy needs to drain a queue and stop a route before the new build takes over. A downstream service falls over, and the route hammering it should back off instead of piling up retries. An operator needs to suspend one branch of the flow without restarting the whole process. Something fails at 3am, and the on-call dashboard should light up on its own, not because someone was tailing logs.

Apache Camel solved the operational half of this a long time ago with the Control Bus pattern: you manage routes by sending a message to a special endpoint, the same way you move any other message. redb.Route brings that pattern to .NET, with the Camel command set, and adds one thing Camel does not have: a consumer that turns route lifecycle events back into messages you can route.

So there are two directions here. Outbound: tell a route to start, stop, suspend, resume or restart. Inbound: subscribe to what routes are doing and react. Both are ordinary steps of a pipeline. Let us look at the code.

Control Bus in one minute

The component is registered out of the box, there is no package to add. Address it by URI or by the fluent DSL.

// Stop a route by id
.To("controlbus:route?routeId=orders&action=stop")

// Same thing, fluent
.ControlBus(ControlBusAction.Stop, "orders")

That is a producer step. When a message reaches it, the named route is stopped. Everything else is variations on the action and one consumer for events.

Manage a route by sending a message

The action is the verb. The full set mirrors Camel:

Action	Effect
`Start`	Start the route.
`Stop`	Stop the route (consumer removed, the route stays registered).
`Suspend`	Suspend the route.
`Resume`	Resume a stopped or suspended route.
`Restart`	Stop, then start after `restartDelay` (default 1000 ms).
`Status`	Report the route's status.
`Stats`	Report the route's statistics.
`Fail`	Stop the route and mark it failed.

In the fluent form the action is an enum; as a URI it is the action query parameter.

using redb.Route.ControlBus;

.ControlBus(ControlBusAction.Suspend, "payments")
.ControlBus(ControlBusAction.Restart, "orders", async: true)   // fire-and-forget

// URI equivalents
.To("controlbus:route?routeId=payments&action=suspend")
.To("controlbus:route?routeId=orders&action=restart&async=true")

Because it is a normal route step, the control action can be the tail of any pipeline. A timer that suspends a batch route outside business hours. A webhook that restarts a route on a config change. A choice branch that stops a consumer when a poison message is seen. You are not calling a management API from the outside; you are routing a message, with all the same retries, error handling and observability as the rest of the flow.

A route can manage itself

Pass current as the route id and the action targets the route that is sending the message. This is how a route reacts to its own condition.

From("kafka://orders")
    .Process(CheckHealth)
    .Choice()
        .When(e => e.In.GetHeader<bool>("downstreamDown"))
            .ControlBus(ControlBusAction.Suspend, "current")   // back off, stop consuming
    .End();

A route that detects its downstream is unhealthy suspends itself instead of spinning through failures. Something external, a timer or an operator message, resumes it later.

The part Camel does not have: react to route events

This is the direction that is usually missing. In Camel the Control Bus is producer-only: you send commands, you do not subscribe to what happens. redb.Route adds controlbus:notify, a consumer that emits route and context lifecycle events as messages. You put it on the From side and route the events anywhere.

From("controlbus:notify")
    .Process(e =>
    {
        var evt   = e.In.GetHeader<string>(ControlBusHeaders.Event);    // RouteStarted, RouteStopped, RouteErrored, ...
        var route = e.In.GetHeader<string>(ControlBusHeaders.RouteId);
        var when  = e.In.GetHeader<DateTimeOffset>(ControlBusHeaders.Timestamp);
    })
    .To("kafka://route-events");

The events cover the lifecycle of routes and of the context itself: RouteStarted, RouteStopped, RouteSuspending, RouteErrored, ContextStarting, ContextStarted, ContextStopping, ContextStopped, ExchangeTimedOut. Each message carries the details as headers: the event name, the affected route id, a timestamp, and where relevant the error, the exchange id and the elapsed time.

Filter to what you care about with the events and routeId parameters:

// Only failures and stops, only for the orders route
From("controlbus:notify?events=RouteErrored,RouteStopped&routeId=orders")
    .To("slack://alerts");

Now the interesting part is that both directions compose. Events on the way in, commands on the way out, in one route: this is a self-healing loop with no external control plane.

From("controlbus:notify?events=RouteErrored")
    .Process(e => e.In.SetHeader("failed", e.In.GetHeader<string>(ControlBusHeaders.RouteId)))
    .Delay(TimeSpan.FromSeconds(30))
    .ControlBus(ControlBusAction.Restart, "current");   // or the captured route id

An errored route emits an event, a route consumes it, waits, and restarts the offender. The supervision logic is a pipeline, visible and testable like any other, not a hardcoded policy buried in the engine.

What this replaces

Without a Control Bus, runtime route management is a pile of one-off plumbing. A custom admin controller that reaches into the engine to stop a route. A BackgroundService that polls a flag table to know when to pause. A logging appender wired to an alerting SDK so someone hears about a failure. Each is a separate mechanism, with its own lifecycle, its own tests, its own way of being wrong.

The Control Bus turns all of that into routing. Management commands are messages To an endpoint. Lifecycle events are messages From an endpoint. They go through the same DSL, the same error handling, the same OpenTelemetry traces as your business flows, and they land in the same place your team already looks.

Zero-downtime deploys: suspend and drain a route on a signal, flip, resume.
Backpressure and circuit-breaking at the route level: a route suspends itself when its downstream is unhealthy.
Operational events as data: pipe RouteErrored and RouteStopped into Kafka, Slack, a metrics sink, an incident tool, an audit log.
Self-healing: notify in, restart out, in one small route.

The same control from a dashboard: redb.Tsak

Everything above is the programmatic face: a message drives a route's lifecycle. There is an operational face too. redb.Tsak, the runtime that hosts your routes, exposes the same route and context lifecycle in a dashboard, so an operator can start, stop, suspend or resume a route, and manage whole contexts, live, without writing a route or shipping a deploy. One capability, two entry points: a message from inside a flow, or a click from outside it, over the same engine.

Camel parity, and beyond

If you come from Camel, the command side is familiar: controlbus:route with routeId and action, plus controlbus:language for expression-based control, producer-only, the standard verbs. redb.Route matches that command set. The controlbus:notify consumer is the addition: Camel gives you EventNotifier as an SPI you implement in Java and register; redb.Route gives you the same information as a first-class endpoint you route from, no interface to implement, no wiring, just From("controlbus:notify").

FAQ

Is it a separate package? No. Control Bus is part of core redb.Route and is registered out of the box. Nothing to install.

Does stopping a route remove it? No. Stop and Suspend remove the consumer but keep the route registered, so Resume or Start brings it back. Fail stops it and marks it errored.

Can a route control another route, not just itself? Yes. Pass the target route id instead of current.

What events are available? Route lifecycle (RouteStarted, RouteStopped, RouteSuspending, RouteErrored), context lifecycle (ContextStarting through ContextStopped), and ExchangeTimedOut. Filter with events= and routeId=.

Is the command synchronous? By default yes; pass async=true (or async: true in the DSL) for fire-and-forget, which also avoids a route trying to stop itself synchronously mid-exchange.

Where it lives

Control Bus ships inside redb.Route on NuGet; the DSL and the controlbus:notify event set are in the documentation. It is one of the 30+ EIP patterns in the framework, and like the rest it is a step of a route: the same From → … → To, the same observability. The difference is that the message it carries is a route telling you what it just did, or you telling a route what to do next.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.

A directory is an integration: file exchange in .NET without cron, FileSystemWatcher or hand-rolled pollers

rinat kozin — Tue, 25 Aug 2026 21:39:03 +0000

Polling, waiting for a file to finish being written, idempotency and atomic temp+rename writes, as an ordinary route step.

Ask anyone who does integrations for a living what the money actually moves through, and files will be on the list. The bank drops a statement into a directory on a share. An ERP exports orders as XML on a schedule. An EDI partner pushes .edi files to an FTP server. Billing lands a couple of gigabytes of CDR files overnight. A retail chain sends stock levels as CSV because that is how it was in 2009 and nobody plans to change it. File exchange is not legacy on its way out. It is a working transport with exactly one property: it looks simpler than it is.

It looks simple because the first version takes an hour. Directory.GetFiles, a loop, File.ReadAllBytes, process, delete. Then real life starts. You picked the file up while the partner was still writing it, and half an order batch went into the database. Processing threw, and the file is already deleted. Two instances of the service grabbed the same file and created duplicate payments. Someone dropped an antivirus .tmp into the directory. FileSystemWatcher missed events once the directory moved to an SMB share. A deploy went out mid-processing and an 800 MB file ended up neither here nor there. Each of these gets its own workaround, and a year later you own a 2000-line half-framework nobody wants to touch.

redb.Route.File closes that list with a transport. A directory becomes an ordinary message source in a route, exactly like a Kafka topic or an HTTP endpoint: polling with filters, waiting until the file is fully written, idempotency, a policy for what happens after processing, atomic writes through a temp file. Same DSL, same EIP patterns after From, same telemetry. Here is what that looks like, and why a native connector inside the ESB beats a poller you wrote yourself.

Files in a minute

services.AddRedbRoute(route =>
{
    route.Services.AddRedbRouteFile();
    route.AddRouteBuilder<MyRoutes>();
});

using redb.Route.File.Fluent;

// Read incoming CSVs, hand the parsed result to a queue
From(FileDsl.Read("/data/incoming")
        .Include("*.csv")
        .MinAge(2000)
        .SortBy("Modified")
        .MoveTo("/data/processed"))
    .Log("Picked up ${header.redbFile.Name}, ${header.redbFile.Length} bytes")
    .To("direct://parse");

// Write the result out atomically
From("direct://export")
    .To(FileDsl.Write("/data/outgoing")
        .FileName("${header.orderId}.json")
        .TempPrefix(".tmp-"));

That is the whole setup. No external dependencies, one registration line, and the file scheme is then available both fluently and as a URI string.

An endpoint is a string, or a builder

Every endpoint reads two equivalent ways: a type-safe builder in code, and a plain URI that lives in appsettings.json and changes without a rebuild. They compile to the same thing, because the builder literally assembles the string.

// Fluent
From(FileDsl.Read("/data/incoming").Include("*.csv,*.xml").Recursive().Delete())

// String (identical)
From("file:///data/incoming?include=*.csv,*.xml&recursive=true&delete=true")

On Windows the path is written file:///C:/data/incoming and the leading slash is dropped for you. A small thing, and reliably the one that bites when a config moves between machines.

The central problem: a file that is still being written

No file system gives you a "writing finished" event. The partner opened a stream, is pushing 300 MB over a slow link, and your poller has already seen the name in the directory. The question is not whether this bites you, only how soon. The connector offers six independent mechanisms, and production setups usually combine two.

Minimum age. The cheapest one: leave the file alone until N milliseconds have passed since the last write.

FileDsl.Read("/data/incoming").MinAge(5000)

Wait for the size to settle. The Changed strategy watches size and modification time at an interval and releases the file only once it has stopped growing for a configured period. This is the right answer for large exports that take minutes to write.

FileDsl.Read("/data/incoming")
    .ReadLock("Changed")
    .ReadLockCheckInterval(1000)   // how often to look
    .ReadLockMinAge(5000)          // how long it must stay unchanged
    .ReadLockTimeout(120000)       // how long to wait at most

A marker file. The MarkerFile strategy creates name.redbLock next to the target using FileMode.CreateNew, which is atomic at the OS level. A second instance that sees the marker skips the file. This is the working answer to "two pods read the same share": exactly one wins the race, and the loser never even opens the file.

FileDsl.Read("/mnt/share/in").ReadLock("MarkerFile")

An exclusive handle. The FileLock strategy opens the file exclusively and holds the handle for the duration of processing. While it is held, nobody else opens that file for reading or writing. One detail worth spelling out: the handle is taken with delete permission, so your own post-processing can delete or move the file without waiting for the lock to drop, and the body is read through the handle that is already open rather than by opening the file a second time.

FileDsl.Read("/data/incoming").ReadLock("FileLock")

A rename. The Rename strategy moves the file to an internal name and works on it there. If the rename fails, someone else holds the file and the route leaves it alone. Pleasant side effect: while the file is being processed it is not in the directory under its original name, so another poller matching on a glob simply does not see it.

FileDsl.Read("/data/incoming").ReadLock("Rename")

A signal file from the sender. EDI classic: the partner writes order.csv, finishes it, and only then drops an empty order.csv.done. Until the signal is there, the file does not exist as far as the route is concerned.

FileDsl.Read("/data/incoming").DoneFileName("${file:name}.done")

${file:name} and ${file:name.noext} are both substituted, so the order.csv plus order.done convention works too. The signal file is removed for you once processing succeeds.

Separately: the consumer ignores internal names starting with .redb_ and anything starting with the configured tempPrefix. So a producer writing into the same directory through a temp file cannot feed itself an endless loop. That is the kind of detail a hand-rolled poller discovers on day three.

What happens to the file afterwards

Exactly one policy per endpoint, and trying to set two fails validation at startup rather than silently at runtime.

Policy	Effect	When to reach for it
`Noop()`	The file stays where it is	Read-only directory, or no write permission
`Delete()`	Removed after success	An inbox where only the fact of receipt matters
`MoveTo(dir)`	Moved to an archive	The production default: there is always something to show
`PreMove(dir)`	Moved BEFORE processing	Several consumers on one directory

PreMove deserves a note, because it solves a problem people usually solve badly. The file is moved into a working subdirectory first and read only afterwards. A move within one file system is atomic, so a second instance simply does not find it and moves on. As a bonus, the directory listing tells you at any moment what is currently in flight.

FileDsl.Read("/data/incoming")
    .PreMove("/data/processing")
    .MoveTo("/data/archive")

Now failure. If processing threw and the exception was not marked handled, post-processing does not run: the file stays exactly where it was and comes back on the next poll. That is deliberate, because "delete the file we could not process" is the worst available option. The flip side is that a genuinely broken file will be retried forever, which is why on a file route OnException is part of the construction rather than a nice-to-have:

OnException(typeof(FormatException))
    .Handled(true)
    .To(FileDsl.Write("/data/quarantine"));

A handled exception completes the exchange successfully, a copy lands in quarantine, and the original goes through the normal policy.

The same applies to a case people rarely plan for: the file was in the directory but could not be read. Someone holds it exclusively, permissions are wrong, the volume dropped. That is a failure for that file, not an empty message. The route will not receive zero bytes dressed up as a successful read, the file stays put, and the next poll picks it up. Obvious, right up until a hand-rolled poller hands you an empty byte[] and the import faithfully records zero orders.

Idempotency

A flag turns it on. The default key is built from the full path, the modification time and the size, so a file overwritten under the same name counts as new, while the same file is never processed twice.

FileDsl.Read("/data/incoming").Noop().Idempotent()

Noop() + Idempotent() gives you "read the directory, change nothing in it, each file exactly once". It is the only way to work with a directory you have no write access to, and it is also the most common shape when integrating with someone else's share.

The registry of processed keys lives in process memory, which is worth keeping in mind: after a restart it is empty. Combined with Delete or MoveTo that does not matter, because the processed file is no longer in the directory. Combined with Noop, a restart means a second pass, and de-duplication for that case belongs further down the route.

Selection: what to take, and in what order

Processing order almost always matters in file exchange. A nightly batch of 4000 files consumed in arbitrary order gives you stock levels that drift away from reality.

FileDsl.Read("/data/incoming")
    .Include("*.csv,*.xml")      // comma-separated globs, * and ?
    .Exclude("*.tmp,~*")
    .SortBy("Modified")          // Name, NameDesc, Modified, ModifiedDesc, Size, SizeDesc
    .MaxMessagesPerPoll(200)     // batch size per pass
    .Recursive()
    .Delay(1000)                 // poll interval, 500 ms by default

MaxMessagesPerPoll together with sorting is your load regulator. A partner comes back after a day of downtime and drops 40 000 files, and you work through them in predictable batches from oldest to newest instead of materialising a 40 000-element list and taking the process down.

Globs are matched case-insensitively, which on Linux saves you from the classic "they sent ORDERS.CSV and the mask says *.csv".

Writing: temp plus rename, not "however it goes"

The producer solves the mirror image of the same problem. If you write straight into a directory a partner is reading, sooner or later they read half a file.

From("direct://export")
    .To(FileDsl.Write("/data/outgoing")
        .FileName("orders-${header.batchId}.json")
        .TempPrefix(".tmp-")
        .AutoCreate());

The body goes to .tmp-orders-42.json and is renamed to the target name only once it is fully written. A rename inside a file system is atomic, so the partner sees either nothing or the finished file. There is no intermediate state. On an exception the temp file cleans itself up.

The target name comes, in priority order, from the FileName option (full expressions over headers and body), from the incoming redbFile.Name header, and failing both it is generated as redb-{guid}. That last fallback is handier than it sounds: a From(kafka).To(file) route works without configuring a name at all.

What to do when the target already exists is configured separately:

`FileExist`	Behaviour
`Override`	Overwrite (default)
`Append`	Append to the end
`Fail`	Throw
`Ignore`	Skip the write silently
`Move`	Set the existing one aside as `.bak`, then write
`TryRename`	Rename the existing one with a timestamp suffix

Ignore and Fail look exotic exactly until the first export where writing the same name twice means shipping the goods twice.

The file name is untrusted input

Look again at the priority order above: with no FileName set, the name arrives in the redbFile.Name header. And somebody put it there: an incoming partner file, an HTTP upload, a field from a queue message. Which means that in a typical From(http).To(file) route, the name of the file you create is chosen by whoever sends the request.

So by default the producer does not let a write escape its own directory. A name like ../../etc/cron.d/backdoor, or an absolute path, gets you an exception rather than a file somewhere surprising. The absolute path case is worth knowing about on its own: plain Path.Combine in .NET silently discards the base directory and hands back what it was given, so "but we concatenate with the base" is not a defence.

// default: writes stay inside /data/outgoing
FileDsl.Write("/data/outgoing")

// deliberately allow writing outside
FileDsl.Write("/data/outgoing").JailStartingDirectory(false)

The option has the same name on the local file system, FTP and SFTP, so it is one rule across all three transports.

An append log

A useful mode of its own: append with a separator. A receipt log, an audit trail, a cumulative daily CSV all become a single route step.

From("direct://audit")
    .To(FileDsl.Write("/var/log/app")
        .FileName("audit-${dateFormat(now(), 'yyyyMMdd')}.log")
        .FileExist("Append")
        .AppendChars("\n"));

Putting the date in the name expression means daily rotation falls out for free, with no separate scheduler.

Headers: the file name travels the whole route

The consumer puts a full set of metadata on the message under the redbFile. prefix, available downstream in any expression, predicate or processor.

Header	Contents
`redbFile.Name`	`order-42.csv`
`redbFile.NameOnly`	`order-42`
`redbFile.Extension`	`.csv`
`redbFile.AbsolutePath`	Full path at read time
`redbFile.RelativePath`	Path relative to the polled directory
`redbFile.Parent`	Parent directory
`redbFile.Length`	Size in bytes
`redbFile.LastModified`	`DateTimeOffset`

After writing, the producer sets redbFile.NameProduced to the path the file actually landed on. On top of that, ContentType is inferred from the extension for .json, .xml, .csv, .txt, .log and .html, so the next step already knows what it was handed.

The practical payoff shows up in routing by name, and file exchange has more of that than anyone would like:

From(FileDsl.Read("/data/in").Recursive().MoveTo("/data/archive"))
    .Choice(c => c
        .When(Header("redbFile.Extension").isEqualTo(".xml").Matches,
              w => w.To("direct://xml"))
        .When(Header("redbFile.Name").startsWith("INV_").Matches,
              w => w.To("direct://invoices"))
        .Otherwise(o => o.To("direct://unknown")));

Large files: do not read the whole thing

The body is a byte[] by default, which is convenient and completely unacceptable for a two-gigabyte CDR file. StreamBody hands over an open stream instead of an array, and the stream is closed along with the exchange.

From(FileDsl.Read("/data/cdr").Include("*.dat").StreamBody().MoveTo("/data/done"))
    .Split(ex => ReadLines((Stream)ex.In.Body!))
    .To("direct://cdr-record");

Streaming body plus splitter is what a splitter in an ESB exists for: the file is consumed line by line and memory holds one record instead of the whole file.

The file system as a cloakroom

There is an adjacent problem, and it shows up specifically in file integrations. The body is large, and the intermediate steps of the route never look at it. Downstream there is a broker, an HTTP call, another service, and each of them dutifully drags two hundred megabytes it has no use for through itself.

The Claim Check pattern (Hohpe and Woolf) handles this like a cloakroom: the body is checked in, and a ticket travels the route in its place. The body is unpacked where it is actually needed.

The store is pluggable, and for the file world the natural choice is the file system itself. FileClaimCheckRepository writes each claim as its own file alongside its metadata and TTL. For large bodies that is cheaper than memory, and on a shared file system it survives both a restart and the work moving to a neighbouring instance.

private readonly IClaimCheckRepository _claims =
    new FileClaimCheckRepository("/data/claims", TimeSpan.FromHours(6));

From(FileDsl.Read("/data/incoming").Include("*.zip").MoveTo("/data/archive"))
    .ClaimCheck(_claims, ClaimCheckOperation.Set, "${header.redbFile.NameOnly}")
    .To("direct://notify")                      // a ticket travels on, not the archive
    .ClaimCheck(_claims, ClaimCheckOperation.GetAndRemove, "${header.redbFile.NameOnly}")
    .Process(UnpackAndImport);                  // body restored, ticket redeemed

The original body type is remembered in headers, so what comes back is what went in rather than a bare byte array. The store can be registered under a name on the context and referenced by string, or left unspecified entirely, in which case steps share the context-wide store.

There is also a keyless stack mode, Push and Pop: check the body in before an enrichment call, take it back afterwards, nesting handled for you. Handy when the middle of a route needs a call that your two hundred megabytes only get in the way of.

Local directory, FTP and SFTP are the same route

This is the connector's main architectural bet. Polling, filters, sorting, idempotency, doneFileName, post-processing, atomic writes and the existing-file strategies are implemented once, in a shared redb.Route.GenericFile base. The local file system, FTP and SFTP are three implementations of file operations on top of it.

The practical consequence is direct. When a partner says "we no longer mount the share, collect from our SFTP", you change the source rather than rewriting the intake logic:

// before
From(FileDsl.Read("/mnt/partner/in").Include("*.edi").MoveTo("archive"))

// after
From(SftpDsl.Directory("/upload/in")
        .Host("sftp.partner.com").Username("edi").Password("{{sftp-pass}}")
        .Include("*.edi").MoveTo("archive"))

Option names, post-processing semantics and failure behaviour all match, because it is literally the same code. Migrating between transports takes a minute rather than a sprint.

Shutting down without losing a file

The consumer is built on the graceful shutdown machinery shared by every redb.Route transport: on Stop the directory polling stops first, then the engine waits for exchanges already in flight, and only then the route goes down. A file that was being processed during a deploy is neither abandoned halfway nor left suspended between PreMove and the archive.

For file exchange this matters more than it does for brokers: an unacknowledged broker message returns to the queue, while a file gets no second chance automatically.

Boundaries

An honest list of what the connector does not do, so you do not find out in production.

Boundary	How it is
Polling, not file system events	No `FileSystemWatcher`. Polling is predictable, survives network shares and loses nothing when a buffer overflows. The price: latency up to `delay`, 500 ms by default
The idempotency registry lives in process memory	It survives a restart only together with `Delete` or `MoveTo`, which take the file out of the directory. It is also not shared between instances: separate them with `preMove` or a `readLock`, not with hope
The idempotency key is built from file metadata	Path, modification time, size, or your own pattern with `${file:name}`. Header and body expressions are not available here: the key is needed before the file has been read, otherwise the check loses its point
Processing within a poll is sequential	One consumer works through a batch in order. Parallelism comes from further down the route, or from several routes on different globs
`readLock` strategies are local file system only	FTP and SFTP have `doneFileName`, `minAge` and `preMove` in their place
The polled directory is not created for you	If it is missing, the poll cycle is skipped quietly. Auto-creation exists on the producer side via `AutoCreate`

Where this plugs in

The value of a file connector is not in reading a directory. That is Directory.GetFiles. The value is that after From the rest of redb.Route is available: splitter and aggregator, content-based routing, de-duplication, retries, circuit breaker, transactions, distributed tracing. A typical nightly intake looks like this:

From(FileDsl.Read("/data/incoming")
        .Include("orders_*.csv")
        .DoneFileName("${file:name}.done")
        .SortBy("Modified")
        .MaxMessagesPerPoll(500)
        .PreMove("/data/processing")
        .MoveTo("/data/archive"))
    .Log("Batch ${header.redbFile.Name}")
    .Split(ex => ParseCsv(ex.In.Body))
        .To("sql:INSERT INTO orders(...) VALUES(...)?dataSource=#pg")
    .End()
    .To("kafka://orders-imported");

Twenty lines instead of a bespoke service, and every line states a decision rather than the mechanics of carrying it out. The producer also opens an OpenTelemetry span on write, so "where did that file go" stops being a question for the logs.

File exchange is never going to be fashionable. It is going to stay, and the difference between "we have a file integration" and "we have a reliable file integration" is measured in exactly the details listed above: file age, markers, atomic rename, order and batch size. It is pleasant when someone has already written them.

Package: redb.Route.File on NuGet; sources and the full option reference in the connector README. Files are one more transport in the redb.Route family, next to Kafka, RabbitMQ, SFTP, AS2 and the rest: same From → … → To, same EIP, same observability. The only difference is that the input is a directory somebody fills whenever it suits them.

If this was useful — a ⭐ on GitHub helps others find it.

More of my writing: redbase.app/articles, and on dev.to.