redb.Route 3.1.0 — LLM(AI) as just another connector: `.To("llm://claude")` and tools-as-routes

Series: redb ecosystem (announcement; deep-dive separate)

redb.Route 3.1.0 just shipped two new transports: redb.Route.Llm (the 24th) and redb.Route.Exec (the 25th). The LLM is now an addressable endpoint the same way Kafka, RabbitMQ and HTTP are: calling a model is .To("llm://claude"), an agent tool is a route with .AsLlmTool("shell") on it, a scheduled agent is From("llm://factory?schedule=5m"). Exec is a process spawner with allowlist, working-directory and timeout — both the backend that powers shell tools for agents and a first-class scheduled consumer in its own right (cron-less health probes, backups, deploy glue). No parallel "AI framework" sitting next to the integration framework — same DSL, same retry/throttle/circuit-breaker/audit, same OpenTelemetry traces, same dashboard endpoint counters.

This post is the announcement. A deep-dive on the agent loop, the governance hooks and the REDB storage internals will follow as a separate piece. Here: what landed, what it looks like in code, and what's honestly not done yet.

First time reading about redb.Route? Short context from earlier posts in the series:

• redb.Route — Apache Camel for .NET: 22 transports, 30+ EIP patterns, compiled DSL — why it exists, and what "Apache Camel for .NET" actually means
• RedBase / redb.Route / redb.Tsak 3.0.0 shipped — the 3.0.0 release that this LLM connector slots into
• Enterprise Integration Patterns in .NET — Part 1: the four in-memory channels (and the Exchange they carry) — how the runtime is built, if you want internals
• redb.Route 3.0.1 — flat DSL navigation, CRTP refactor, and a silent null fix — the previous patch right before 3.1.0

---

The shortest possible explanation

From("kafka://orders")
    .To(Llm.Factory("claude").Temperature(0.2).MaxTokens(1024).AsUri())
    .To("kafka://orders.translated");

One line, one full LLM call:

• the inbound message body becomes the user prompt;
• the agent runs (provider call → optional tool_use → next provider call → …) until EndTurn or MaxIterations;
• the assistant text lands in exchange.Out.Body;
• token usage, model id, stop reason, iteration count are pushed into headers;
• OpenTelemetry traces and metrics light up automatically;
• the endpoint shows up in the tsak.web dashboard with msg/sec, average duration, error rate and last-error info — like every other connector.

That's the whole reason "connector, not library" matters. If you already have an Apache Camel-class ESB with retry, breaker, idempotent consumer and audit, the only honest move is to turn the LLM into one more endpoint inside it. No re-implementing retries. No separate dashboard. No "AI infra" running parallel to the integration infra.

---

One adapter, 14 OpenAI-compatible APIs

Two production providers ship today:

• OpenAiProvider — one universal transport for 14 OpenAI-compatible APIs: openai, anthropic/claude (via Anthropic's official OpenAI-compat endpoint), groq, cerebras, openrouter, gemini (OpenAI-compat surface), github-models, mistral, together, huggingface, deepseek, ollama, lmstudio, plus a generic custom for any self-hosted gateway.
• StubProvider — deterministic echo for unit tests and CI runs with no keys.

A native AnthropicProvider (Messages API) is on deck for features the OpenAI-compat surface doesn't expose.

Switching provider is one DI registration:

services.AddLlmConnectionFactory("groq", f =>
{
    f.Provider = "groq";
    f.ModelId  = "llama-3.3-70b-versatile";
    f.ApiKey   = Environment.GetEnvironmentVariable("REDB_LLM_GROQ_KEY");
});

Change provider to "anthropic" and modelId to "claude-haiku-4-5" and the same .To("llm://...") is calling Claude. The DSL surface doesn't move a character.

---

Tools are routes

The central architectural call: a tool is an ordinary RouteBuilder route with one extra DSL aspect, .AsLlmTool("name"). Four properties fall out of that and would otherwise have to be paid for separately.

From("direct:tool-shell")
    .AsLlmTool("shell")
        .Description("Run a small shell command on the host. Input: {\"command\":\"<name>\",\"args\":[\"...\"]}.")
        .Input("""
            {
              "type":"object",
              "properties":{
                "command":{"type":"string"},
                "args":{"type":"array","items":{"type":"string"}}
              },
              "required":["command"]
            }
            """)
        .SideEffect(ToolSideEffect.ReadOnly)
        .Cost(ToolCostClass.Cheap)
    .Then()
    .To(ExecDsl.Run()
        .AllowedCommands("cmd", "pwsh")
        .WorkingDirectory(scratchDir)
        .TimeoutMs(5_000)
        .MaxStdoutBytes(8_192)
        .MaxStderrBytes(8_192));

What you get for free:

1. Tools-as-routes. The 30+ EIP processors are usable inside a tool — Splitter, Aggregator, CircuitBreaker, Throttle, Filter, TryCatch. A tool that talks to a fragile API gets wrapped in a breaker and a throttle without a line of runtime glue.
2. Tools from existing routes. Any From("http://...") or From("sql://...") you already have becomes a tool by adding .AsLlmTool("name"). Its auth, its audit trail, its metrics are already in place — because it's already a route.
3. Inventory-as-data. IToolDescriptorRegistry knows every tool by name with its JSON schema. Filter with ?tools=* or ?tools=lookup,shell on the URI; build "an agent that has every read-only tool" in one line.
4. Zero bumps on sibling connectors. The AsLlmTool() aspect lives in redb.Route.Llm.Abstractions. The other 22 connectors can be used as tools today without a NuGet bump on any of them.

---

A real example — a standalone "curl → Claude → shell → reply" demo

This is a complete program. Two files — Llm.HttpShell.csproj and Program.cs, top-level statements, no host abstractions on top. Drop your Anthropic key into one place and dotnet run brings up an HTTP endpoint on port 5088 where Claude Haiku can call a shell on the host and answer in the context of the previous turns.

csproj — five NuGet packages

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="redb.Route" Version="3.1.0" />
    <PackageReference Include="redb.Route.Http" Version="3.1.0" />
    <PackageReference Include="redb.Route.Llm" Version="3.1.0" />
    <PackageReference Include="redb.Route.Llm.Abstractions" Version="3.1.0" />
    <PackageReference Include="redb.Route.Exec" Version="3.1.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="9.0.0" />
  </ItemGroup>
</Project>

redb.Route is the ESB core, redb.Route.Http is the HTTP transport, redb.Route.Llm[.Abstractions] is the LLM connector and the .AsLlmTool() DSL aspect, redb.Route.Exec is the process spawner with allowlist and timeout. No redb.Core/redb.Postgres — the demo is self-contained and keeps conversation memory in process.

1. The key

const string ApiKey = "<your-key>";

The one you get from https://console.anthropic.com/. Switching to Groq is Provider = "groq", ModelId = "llama-3.3-70b-versatile" and a key from console.groq.com; the DSL below doesn't move.

2. DI and RouteContext

RouteContext ctx = null!;

var services = new ServiceCollection();
services.AddLogging(b => b
    .AddSimpleConsole(o => { o.SingleLine = true; o.TimestampFormat = "HH:mm:ss "; })
    .SetMinimumLevel(LogLevel.Information));
services.AddSingleton<IRouteContext>(_ => ctx);
services.AddSingleton<ILogger>(sp => sp.GetRequiredService<ILoggerFactory>().CreateLogger("redb.Route"));

var sp = services.BuildServiceProvider();
ctx = new RouteContext(sp, contextId: "llm-http-shell");
ctx.AddService(typeof(ILoggerFactory), sp.GetRequiredService<ILoggerFactory>());

RouteContext is the runtime container — routes, components, services. The _ => ctx closure registers IRouteContext as a DI service before the context itself is constructed (the context in turn needs the IServiceProvider). The last line hands ILoggerFactory to the context explicitly; without it .Log(...) steps in routes silently turn into no-ops (internals belong to the deep-dive piece).

3. Three components

ctx.AddComponent(new HttpComponent { ServerManager = new SharedHttpServerManager() });
ctx.AddComponent(new LlmComponent());
ctx.AddComponent(new ExecComponent());

A component in redb.Route is a transport plugin owning a URI scheme: http://, llm://, exec://. Pure registration, no business logic.

4. Connection factory for Claude Haiku

ctx.AddToRegistry("haiku", new LlmConnectionFactory
{
    Name        = "haiku",
    Provider    = "anthropic",
    ModelId     = "claude-haiku-4-5",
    ApiKey      = ApiKey,
    Temperature = 0.0,
    MaxTokens   = 512
});

"haiku" is the label the route will reference as Llm.Factory("haiku"). Same trick redb.Route uses framework-wide for named connection factories.

5. Agent engine + tool registry

var toolRegistry = new ToolDescriptorRegistry();
ctx.AddService(typeof(IToolDescriptorRegistry), toolRegistry);

var producerTemplate = new ProducerTemplate(ctx);
ctx.AddService(typeof(IProducerTemplate), producerTemplate);

var engine = new AgentEngine(
    logger:           null,
    producerTemplate: producerTemplate,
    observer:         new NoopAgentObserver(),
    budget:           new NoopBudgetEnforcer(),
    approval:         new AutoApproveGate(),
    redaction:        new NoopRedactionFilter(),
    shadow:           new NoopShadowRunner(),
    conversation:     new InMemoryConversationStore(),
    idempotency:      null,
    approvalStore:    null);
ctx.AddService(typeof(IAgentEngine), engine);

Every agent state surface — observability, budget, approval, redaction, shadow, conversation, idempotency — lives behind an interface. The demo wires Noop/InMemory implementations everywhere: the loop runs but nothing is persisted. Production swap is AddRedbLlmStorage() (covered below): replaces InMemoryConversationStore with RedbConversationStore, attaches RedbAuditObserver, etc.

6a. The HTTP route

var isWindows  = OperatingSystem.IsWindows();
var allowed    = isWindows ? new[] { "cmd", "pwsh", "powershell" } : new[] { "sh", "bash" };
var scratchDir = Path.Combine(Path.GetTempPath(), "redb-llm-shell");
Directory.CreateDirectory(scratchDir);

var systemPrompt =
    "You can run small shell commands through the 'shell' tool. " +
    $"The host is {(isWindows ? "Windows (use cmd /c)" : "Linux (use sh -c)")}. " +
    "Use the tool when the user asks about the system, files, or commands; " +
    "then summarise what you learned in one short sentence.";

ctx.AddRoutes(r =>
{
    r.From("http:0.0.0.0:5088/api/llm/shell?inOut=true")
        .RouteId("llm-http-shell")
        .ConvertBody<string>()
        .Process(e =>
        {
            e.In.Headers[LlmHeaders.SystemPrompt] = systemPrompt;
            e.In.Headers[LlmHeaders.ConversationId] =
                e.In.Headers.TryGetValue("X-Chat-Id", out var v) && v is string s && s.Length > 0
                    ? s : "default";
        })
        .Log("[LLM-SHELL] ▶ chat=${header.llm.conversation.id} prompt=${body}")
        .To(LlmDsl.Factory("haiku")
            .Tools("shell")
            .ConversationFromHeader()
            .MaxIterations(10)
            .Temperature(0.0)
            .AsUri())
        .Log("[LLM-SHELL] ◀ iters=${header.llm.tool.iterations} stop=${header.llm.stop_reason} " +
             "tokensIn=${header.llm.tokens.in} tokensOut=${header.llm.tokens.out}")
        .Log("[LLM-SHELL] ◀ reply=${body}");

The POST body is converted to a string and becomes the user prompt. Process puts the system prompt into the standard LlmHeaders.SystemPrompt header and resolves the conversation id from the client's X-Chat-Id (without it — default). .Tools("shell") is the only LLM-specific knob: "give the agent the tool registered under this name." MaxIterations(10) caps the tool loop (otherwise it's an unbounded ping between model and tool).

6b. The tool itself is a route

    r.From("direct:tool-shell")
        .AsLlmTool("shell")
            .Description(
                "Run a small shell command on the host. Input: " +
                "{\"command\":\"<name>\",\"args\":[\"...\"]}. Output: " +
                "{\"stdout\":\"...\",\"stderr\":\"...\",\"exitCode\":N}. " +
                $"Allowed commands: {string.Join(", ", allowed)}. " +
                $"Working directory is pinned to '{scratchDir}'.")
            .Input("""
                {
                  "type": "object",
                  "properties": {
                    "command": { "type": "string" },
                    "args":    { "type": "array", "items": { "type": "string" } }
                  },
                  "required": ["command"]
                }
                """)
            .SideEffect(ToolSideEffect.ReadOnly)
            .Cost(ToolCostClass.Cheap)
        .Then()
        .Log("[SHELL-TOOL] ▶ in=${body}")
        .To(ExecDsl.Run()
            .AllowedCommands(allowed)
            .WorkingDirectory(scratchDir)
            .TimeoutMs(5_000)
            .MaxStdoutBytes(8_192)
            .MaxStderrBytes(8_192))
        .Log("[SHELL-TOOL] ◀ exit=${header.redbExec.ExitCode} body=${body}");
});

Description and the JSON schema are what Claude sees as "this function is available, here's its signature." SideEffect.ReadOnly and Cost.Cheap are guidance for the governance hooks (budgets, approval gating). After .Then() it's an ordinary route: Log → ExecDsl.Run() with allowlist, working dir, 5-second timeout and a stdout/stderr cap → Log. No LLM-specific code below .Then(). It's still just a route — which means you can wrap it in a CircuitBreaker, a Throttle, a Transaction, a WireTap shadow, anything from the 30+ EIP processors redb.Route already ships.

7. Start and block

await ctx.Start();
producerTemplate.Start();

var stop = new ManualResetEventSlim();
Console.CancelKeyPress += (_, e) => { e.Cancel = true; stop.Set(); };
stop.Wait();

await ctx.DisposeAsync();

Run it

dotnet run

curl -d "how much free disk space?" http://localhost:5088/api/llm/shell
curl -d "what did you just say?" -H "X-Chat-Id: my-chat" http://localhost:5088/api/llm/shell

First request: Claude sees the system prompt ("you have a shell tool"), decides to check free space, calls the tool with something like {"command":"cmd","args":["/c","fsutil","volume","diskfree","C:"]}, gets the stdout back, summarises it for the human. Second request, with X-Chat-Id: my-chat, finds the previous turn in InMemoryConversationStore and answers in context.

The logs at that moment show the full path: [LLM-SHELL] ▶ prompt=... → [SHELL-TOOL] ▶ in={"command":"cmd",...} → [SHELL-TOOL] ◀ exit=0 body={"stdout":"...","exitCode":0} → [LLM-SHELL] ◀ iters=2 stop=end_turn tokensIn=... tokensOut=... reply=.... All of it is just the redb.Route .Log() step, not a separate LLM-tracing pipeline.

The command allowlist (cmd, pwsh, sh, bash) is the security envelope: anything outside it is rejected by redb.Route.Exec before a process starts.

---

redb.Route.Exec — process spawning as the 25th transport

In the demo above ExecDsl.Run() showed up as "the backend of the shell tool", but it's a connector in its own right, shipped in 3.1.0 alongside redb.Route.Llm. It closes a boring but ubiquitous gap: the framework already spoke 22+ transports (HTTP, Kafka, SQL, …), but reaching out to the operating system itself was missing.

Producer — .To(ExecDsl.Run(...)) — synchronous spawn. The command is resolved in this order: JSON body → headers redbExec.Command/redbExec.Args → URI options ?command=.... The Out body is JSON shaped for the LLM tool ABI:

{ "stdout": "...", "stderr": "...", "exitCode": 0, "timedOut": false }

This is exactly why the shell tool in the demo is .To(ExecDsl.Run().AllowedCommands(...)) with zero glue code in between. The model produces {"command":"cmd","args":[...]}, the producer parses it, runs it, returns structured JSON. Route-level features (CircuitBreaker, Throttle, OnException, audit) apply automatically — it's an endpoint like every other one.

Consumer — From(ExecDsl.Run(...).Schedule("5m")) — same spawner, but as a first-class source endpoint with a built-in scheduler. No cron, no separate worker:

routes.From(ExecDsl.Run("./scripts/health-check.sh")
                   .Schedule("5m")
                   .TimeoutMs(30_000))
      .Choice()
        .When(e => e.In.Headers["redbExec.ExitCode"]?.ToString() != "0")
          .To("http://alerts.internal/oncall")
        .Otherwise()
          .To("kafka://metrics.healthy");

Every 5 minutes the script runs, the route branches on exit code — one way to an HTTP webhook for the on-call, the other to a Kafka topic. ?schedule= accepts simple intervals (500ms, 30s, 5m, 1h). For cron expressions: From("quartz://<cron>").To(ExecDsl.Run(...)) — Quartz is already a scheduler, no point re-implementing it inside Exec.

The security envelope — what gives shell-as-a-tool the right to exist in the first place:

• AllowedCommands — case-insensitive on the file name, so /usr/bin/git and git.exe both match git. Anything off the list is rejected with UnauthorizedAccessException before the process starts.
• WorkingDirectory — pins the cwd; the spawned process can't escape it on its own.
• EnvironmentOverrides + ScrubEnvironment — start from an empty environment and apply only the KEY=VALUE pairs you set.
• TimeoutMs — wall-clock kill-switch that takes the whole process tree.
• MaxStdoutBytes / MaxStderrBytes — caps that protect the host from a runaway process.

Why a separate package, not part of redb.Route.Llm — three reasons: (1) Exec is useful with no LLM in sight (scheduled health probes, log rotation, deploy glue, backups); (2) redb.Route.Llm shouldn't drag a dependency on process spawning into projects whose agents only call HTTP or SQL tools; (3) the same allowlist/timeout/cap mechanism will be reused by future connectors that share the same security profile.

---

From("llm://...") — the scheduled agent consumer

This is the bit that has no equivalent in Camel's langchain4j-* family (where LLM is producer-only): the LLM endpoint is a first-class source, with the scheduler baked in.

From("llm://groq?schedule=5m" +
     "&systemPromptRef=#watchdog-system" +
     "&initialBodyRef=#daily-brief" +
     "&tools=*")
    .To("rabbitmq://alerts");

Every 5 minutes: a fresh agent run with the system prompt resolved from registry key #watchdog-system and the user prompt from #daily-brief. The reply goes to RabbitMQ. ?schedule= takes simple intervals (500ms, 30s, 5m, 1h); for cron use From("quartz://...").To("llm://...") — Quartz is already a scheduler, no point duplicating it inside the LLM consumer.

Use cases that fit this shape: watchdog agents, scheduled report generation, self-improving agents with conversation memory (same history across runs).

---

#-prompts — the dynamic registry

A # prefix on a URI parameter turns the value into a registry lookup. Some other route can rewrite the prompt by name; the next agent run picks it up without a redeploy:

host.Context.AddToRegistry("style.terse", "Reply in fewer than 5 words.");

r.From("direct:chat")
    .To("llm://scripted?systemPromptRef=#style.terse")
    .To("mock:done");

// ... later, from another route ...
host.Context.AddToRegistry("style.terse", "Reply in French only.");
// the next call sees the new value

Resolution: first IPromptTemplateRegistry (versioned, what eval replay needs), then the generic context registry (a plain string). Without # the value is literal and arrives at the provider unchanged.

This is the same #name registry convention redb.Route uses framework-wide for connection factories. Same convention, not a separate prompt-ref DSL.

---

Agent memory — AddRedbLlmStorage()

By default every state surface is in-memory (conversation transcripts, tool idempotency, approvals, cost ledgers, audit). That keeps AddRedbRouteLlm() zero-dependency — fine for tests and stateless agents, lost on restart.

One line swaps the defaults for REDB-backed stores:

services.AddRedbRoute(route =>
{
    route.Services.AddRedbRouteLlm();
    route.Services.AddRedbIdempotentRepository();   // required for idempotency
    route.Services.AddRedbLlmStorage();             // ← REDB stores on every surface
});

What AddRedbLlmStorage() replaces:

Interface	Default	REDB store
IConversationStore	InMemoryConversationStore	RedbConversationStore — tree-backed, parent linkage via native parent_id, message ids on indexed value_string
IToolIdempotencyStore	InMemoryToolIdempotencyStore	RedbToolIdempotencyStore (on top of IIdempotentRepository)
IApprovalStore	InMemoryApprovalStore	RedbApprovalStore
ICostBudgetStore	InMemoryCostBudgetStore	RedbCostBudgetStore — running totals per tenant/window
IAgentObserver	NoopAgentObserver	RedbAuditObserver — one row per tool call

Two non-obvious design choices:

• Per-row business identifiers live on _objects.value_string (an indexed column), not inside the props JSON. Lookups are O(log n) on one column, not a full scan with JSON decode.
• Transcript integrity is the tree's native parent_id (IRedbService.CreateChildAsync), not a soft FK in props. The tree primitive enforces it.

Stores lazy-sync their scheme on first use. No migration step.

---

What the engine carries for free

This is the central argument for "connector, not library." Everything redb.Route already does applies to LLM calls without extra code:

Concern	DSL primitive
Retry / backoff	RedeliveryPolicy, OnException
Rate limiting	Throttle
Resilience	CircuitBreaker
Idempotency	IdempotentConsumer
Compensation	Saga
Audit / shadow	WireTap, Multicast
Tracing & metrics	RouteActivitySource, RouteMetrics
Persistence	redb schemes (typed object engine)

A tool that calls an expensive API gets wrapped in a CircuitBreaker and a Throttle. Want a shadow run of a new system prompt alongside the old one? Multicast + WireTap. These aren't "features of the LLM connector" — they're the engine, and the LLM is attached to it as an endpoint.

---

What's live-tested, and what isn't

The honest status of the provider matrix (this is in the README, repeated here):

• Groq + Llama 3.3 70B — the most reliable free tier in scope. Carries the strict assertion in BasicChatTests and ToolRouteTests (the model has to produce a literal token).
• Mistral small-latest — stable for short-form replies; tool-use on the free tier is flaky, so tool tests assert philosophically.
• Gemini 2.0 Flash — 15 RPM on free; works on a quiet repo.
• Anthropic Claude (Haiku 4.5 / Sonnet 4.6) — via the OpenAI-compat endpoint, live-tested in ClaudeChatTests and the redb.Route.Demo host.
• OpenRouter / Cerebras — scaffolding works; individual free models rate-limit; tests gated on [EnvFact] and skipped without a key.

Honest skip-list, things not in this release:

• Embeddings and vector stores — Phase 2 (embed://, vector:// schemes are planned).
• RAG primitives, document loaders, web search — not first-class yet (a Tavily search tool is already shipped in redb.Route.Llm.Tools and plugs in as any other .AsLlmTool("web_search")).
• Sliding-window memory shapes (window-by-N-messages, window-by-K-tokens) — not first-class. Persistent transcripts via AddRedbLlmStorage() are shipped; a windowed shape on top is realizable today via Process + the conversation store, just not as a one-line option.
• Native AnthropicProvider — OpenAI-compat surface covers most scenarios; the native Messages API is for the features the compat surface doesn't expose.

---

Links

GitHub leads NuGet. Fresh bug fixes (e.g. the tool_use/tool_result
pairing recovery on conversation reload, or the Windows OEM-codepage
decoding fix in redb.Route.Exec) are already cut on main under version
3.1.1 — see CHANGELOG.md
in the public repo. NuGet packages are batched into the next release, so
if you hit a production bug — check main and the latest pre-release tag
first, then wait for the NuGet bump. This is normal practice, not a
process bug.

redb.Route on GitHub	github.com/redbase-app/redb-route
redb.Route.Llm on NuGet	nuget.org/packages/redb.Route.Llm
redb.Route.Exec on NuGet	nuget.org/packages/redb.Route.Exec
Full package README	redb.Route.Llm/README.md
USER-GUIDE (full walkthrough — DSL, governance, conversation)	redb.Route.Llm/doc/USER-GUIDE.md
STORAGE — REDB schemas for conversation / approval / audit / idempotency	redb.Route.Llm/doc/STORAGE.md
Exec connector README	redb.Route.Exec/README.md
Standalone Llm.HttpShell demo (curl → Claude → shell → reply)	demo/Llm.HttpShell/Program.cs
Full demo project (22+ routes)	demo/redb.Route.Demo/Routes/LlmHttpRoutes.cs
Discussions	github.com/redbase-app/redb-route/discussions

All Apache 2.0. A separate deep-dive will cover the agent tool-loop, the governance hook interfaces, and a tour of the REDB storage schemas. Questions in the comments are exactly what writes the next post — especially anything along the lines of "does the connector do X if I do Y?"