DEV Community: mgd43b

Dynamic Discovery in Agent Networks: From Hardcoded Routes to Capability Catalogs

mgd43b — Mon, 11 May 2026 14:00:00 +0000

The simplest way to connect two agent ensembles is a direct reference: ensemble A knows ensemble B's address and calls it. This works when you have two or three ensembles with stable relationships.

It stops working when you have ten ensembles, or when ensembles come and go, or when the same capability is provided by multiple ensembles and you want the caller to use whichever one is available. At that point, you need discovery -- a way for ensembles to find capabilities without knowing in advance who provides them.

The static wiring problem

In a statically wired agent network, every cross-ensemble call requires knowing the provider's identity and address:

// Static: caller knows exactly who to call
NetworkTask mealTask = NetworkTask.of("kitchen", "prepare-meal",
    "ws://kitchen:7329/ws");

This creates coupling. If the kitchen ensemble moves to a different port, every caller needs updating. If you add a second kitchen for capacity, callers need load-balancing logic. If the kitchen goes down, callers need fallback logic.

The fundamental issue is that callers should care about what they need (a meal preparation capability), not who provides it or where it runs.

Capability advertisement with tags

AgentEnsemble v3.0.0 introduces capability discovery. Ensembles advertise their shared tasks and tools with optional tags, and other ensembles discover providers at runtime.

Advertising capabilities

When building an ensemble, declare what it shares with the network:

Ensemble kitchen = Ensemble.builder()
    .chatLanguageModel(model)
    .task(Task.of("Manage kitchen operations"))
    .shareTool("check-inventory", inventoryTool, "food", "stock")
    .shareTask("prepare-meal", mealTask, "food", "cooking")
    .build();

kitchen.start(7329);

The shareTool and shareTask methods register capabilities in the network's capability registry. The trailing string arguments are tags -- metadata that classifies the capability for filtered discovery.

Discovering capabilities

Another ensemble can discover providers without knowing their identity:

// Discover by capability name
NetworkTool inventoryCheck = NetworkTool.discover("check-inventory", registry);

// Discover by tag
List<CapabilityInfo> foodCapabilities = registry.findByTag("food");

The registry returns the provider that currently offers the requested capability. If multiple providers offer the same capability, the registry can apply selection logic (round-robin, least-loaded, affinity-based).

Tag-based catalogs

Tags turn the capability registry into a searchable catalog. Rather than querying for specific capability names, you can query for categories:

// Find all capabilities tagged with "food"
List<CapabilityInfo> food = registry.findByTag("food");

// Find all capabilities tagged with both "food" and "stock"
List<CapabilityInfo> stockChecks = registry.findByTags("food", "stock");

Each CapabilityInfo includes the capability name, type (task or tool), provider ensemble name, and tags:

CapabilityInfo info = food.get(0);
String name = info.name();           // "check-inventory"
String type = info.type();           // "TOOL"
String provider = info.provider();   // "kitchen"
Set<String> tags = info.tags();      // ["food", "stock"]

This is useful for building dynamic agent systems where an orchestrating ensemble does not know in advance what capabilities are available. It can discover capabilities at runtime, filter by category, and wire them into its workflow dynamically.

The registry abstraction

The capability registry is part of the transport SPI, which means it has pluggable implementations:

Implementation	Backing	Use case
In-memory	`ConcurrentHashMap`	Development, testing
WebSocket-broadcast	Network messages	Multi-process, simple mode
Kafka-backed	Kafka topics	Production, durable

In development, capabilities are registered and discovered within a single process or across WebSocket connections. In production, the registry can be backed by Kafka for durability and horizontal scaling.

The application code that registers and discovers capabilities does not change between implementations.

Dynamic vs. static wiring

The choice between static and dynamic wiring is not binary. A practical network often uses both:

Static wiring for well-known, stable relationships (the front desk always calls the kitchen)
Dynamic discovery for capabilities that may be provided by different ensembles depending on deployment, capacity, or availability

// Static: known relationship
NetworkTask knownMealTask = NetworkTask.of("kitchen", "prepare-meal",
    "ws://kitchen:7329/ws");

// Dynamic: discover at runtime
NetworkTool discovered = NetworkTool.discover("check-inventory", registry);

The two approaches coexist. Static tasks bypass the registry entirely. Dynamic tasks use the registry for resolution. The agent using the task or tool does not know which approach was used to create it.

Capability lifecycle

Capabilities have a lifecycle that mirrors the ensemble lifecycle:

Registration -- when the ensemble starts and calls shareTask or shareTool
Discovery -- when other ensembles query the registry
Deregistration -- when the ensemble stops or the capability is removed

In simple mode, deregistration happens when the ensemble process exits and the in-memory registry is garbage collected. With WebSocket transport, the ensemble broadcasts a deregistration message on shutdown. With Kafka, a tombstone record is produced.

The lifecycle matters for production systems. A stale registry entry (pointing to an ensemble that no longer exists) causes request failures. The registry needs to handle stale entries, either through explicit deregistration, heartbeat-based expiry, or health-check-based cleanup.

Tradeoffs

Discovery adds a lookup step. Every dynamically discovered capability requires a registry query. In practice, this is cached -- the first lookup queries the registry, subsequent uses of the same capability reuse the resolved provider. But the initial resolution adds latency.

Tag semantics are convention-based. There is no schema for tags. If one ensemble tags a capability as "food" and another tags it as "cuisine", they will not discover each other. Tag conventions need to be agreed upon across teams.

Multiple providers create ambiguity. When two ensembles offer the same capability, the registry needs a selection strategy. The current implementation supports least-loaded selection (when capacity information is available), but more sophisticated strategies (affinity, cost-based, latency-based) would need to be built.

Registry availability is a dependency. If the registry is unavailable, dynamic discovery fails. Static wiring works regardless of registry state. For critical paths, consider falling back to static wiring when discovery is unavailable.

The design principle

The useful abstraction is separating what from who. An ensemble that needs a meal preparation capability should express that need ("I need prepare-meal") without specifying the provider ("specifically from the kitchen ensemble at ws://kitchen:7329/ws").

This separation enables the network to evolve. New providers can come online. Existing providers can be replaced. Capacity can be redistributed. The callers do not need to change.

Discovery is the mechanism. Tags make it searchable. The transport SPI makes it portable across deployment environments.

Capability discovery is part of AgentEnsemble. The discovery guide covers the full API including tag-based filtering.

I'd be interested in how others handle capability discovery in multi-agent systems -- whether you use service registries, hardcoded routes, or something else entirely.

Durable Transport for Agent Networks: Moving from In-Process Queues to Kafka

mgd43b — Sat, 09 May 2026 14:00:00 +0000

In-process queues are fine for development. They are fast, deterministic, and require zero infrastructure. But they have a property that becomes a liability in production: when the process dies, the queue contents disappear.

For agent networks that run as long-lived services -- handling work requests over hours or days -- losing queued requests on restart is not acceptable. The transport layer needs durability, and that means moving from in-process data structures to something that survives process failures.

What durability means for agent networks

An agent ensemble network has three communication patterns that need durable backing:

Work request delivery -- a request from one ensemble to another should not be lost if the receiving ensemble is temporarily unavailable
Response routing -- when an ensemble completes a request, the response needs to reach the original caller even if the caller restarted
Capability advertisement -- shared tasks and tools should remain discoverable across process restarts

Each of these has different durability requirements. Work requests are the most critical -- a lost request means lost work. Response routing needs correlation (matching responses to requests). Capability advertisement needs eventual consistency but not strict durability.

Kafka as the transport backing

The agentensemble-transport-kafka module implements the transport SPIs against Apache Kafka. All components share a single configuration:

KafkaTransportConfig config = KafkaTransportConfig.builder()
    .bootstrapServers("kafka:9092")
    .consumerGroupId("kitchen-ensemble")
    .topicPrefix("agentensemble.")
    .build();

Request queues

The KafkaRequestQueue produces work requests to a Kafka topic and consumes them with manual offset commits:

KafkaRequestQueue queue = KafkaRequestQueue.builder()
    .config(config)
    .ensembleName("kitchen")
    .build();

// Enqueue a work request (produces to Kafka)
queue.enqueue(workRequest);

// Poll for requests (consumes from Kafka)
Optional<WorkRequest> request = queue.poll(Duration.ofSeconds(5));

The topic name is derived from the ensemble name and prefix: agentensemble.kitchen.requests. Manual offset commits ensure that a request is only acknowledged after the ensemble has finished processing it. If the ensemble crashes mid-processing, the request will be redelivered on restart.

Delivery registry

The KafkaDeliveryRegistry tracks pending deliveries and routes responses back to callers:

KafkaDeliveryRegistry registry = KafkaDeliveryRegistry.builder()
    .config(config)
    .build();

// Register a pending delivery (before sending request)
CompletableFuture<String> future = registry.register(requestId);

// Complete the delivery when response arrives
registry.complete(requestId, responsePayload);

// Caller awaits the result
String response = future.get(30, TimeUnit.SECONDS);

The registry uses a Kafka topic for durability: pending deliveries are produced as records, and completions are produced as tombstones. On restart, the registry rebuilds its state by replaying the topic from the beginning.

Priority queues with aging

For workloads where some requests are more urgent than others, the PriorityRequestQueue adds priority levels with aging:

PriorityRequestQueue priorityQueue = PriorityRequestQueue.builder()
    .requestQueue(kafkaQueue)
    .levels(3)                    // 3 priority levels (0 = highest)
    .agingInterval(Duration.ofMinutes(5))
    .build();

// Enqueue with priority
priorityQueue.enqueue(urgentRequest, 0);   // highest priority
priorityQueue.enqueue(normalRequest, 1);   // normal priority
priorityQueue.enqueue(batchRequest, 2);    // lowest priority

Aging prevents starvation: requests that have waited longer than the aging interval are promoted to the next higher priority level. A batch request that has been waiting for 10 minutes (two aging intervals) gets promoted twice, eventually reaching the highest priority.

This is implemented as a layer on top of any RequestQueue implementation, so it works with both in-process and Kafka-backed queues.

What changes operationally

Moving from in-process to Kafka transport changes the operational profile of the ensemble network:

Startup behavior changes. With in-process queues, an ensemble starts with an empty queue. With Kafka, it may start with a backlog of unprocessed requests from before the restart. The ensemble needs to handle this gracefully -- processing the backlog before accepting new work, or processing both concurrently.

Failure modes change. In-process queue failures are process-fatal (if the process dies, the queue is gone). Kafka failures are infrastructure-level (broker unavailable, topic not found, authorization errors). The error handling needs to distinguish between transient failures (retry) and permanent failures (alert and skip).

Monitoring needs change. With in-process queues, queue depth is a simple counter. With Kafka, you need to monitor consumer lag, topic partition health, and broker connectivity. The ensemble's health check needs to include Kafka reachability.

Ordering semantics change. In-process queues provide strict FIFO. Kafka provides per-partition ordering, which means requests may be processed out of order if the topic has multiple partitions. For most agent workloads, this is fine -- requests are independent. But if your workflow depends on ordering, you need single-partition topics or application-level sequencing.

The configuration boundary

One design decision worth calling out: the Kafka transport configuration is separate from the ensemble configuration. The ensemble does not know it is using Kafka -- it interacts with the transport SPIs. The Kafka-specific configuration (bootstrap servers, consumer groups, topic prefixes) lives in the infrastructure layer.

// Infrastructure layer: Kafka-specific setup
KafkaTransportConfig kafkaConfig = KafkaTransportConfig.builder()
    .bootstrapServers("kafka:9092")
    .consumerGroupId("kitchen-ensemble")
    .build();

KafkaRequestQueue queue = KafkaRequestQueue.builder()
    .config(kafkaConfig)
    .ensembleName("kitchen")
    .build();

// Application layer: ensemble setup (transport-agnostic)
Ensemble kitchen = Ensemble.builder()
    .chatLanguageModel(model)
    .task(Task.of("Manage kitchen operations"))
    .requestQueue(queue)
    .build();

This separation means the same ensemble code works in development (with in-process queues) and production (with Kafka) without changes. The transport choice is an infrastructure decision, not an application decision.

Tradeoffs

Operational complexity. Kafka is infrastructure that needs to be provisioned, monitored, and maintained. For small deployments, the operational overhead may not be justified. The in-process transport with periodic state snapshots might be a simpler alternative.

Latency. Kafka adds millisecond-scale latency to every request delivery. For agent workloads where task execution takes seconds or minutes, this is negligible. For sub-second workflows, it may not be.

Topic proliferation. Each ensemble gets its own request topic. A network of 20 ensembles means 20+ Kafka topics. This is manageable but requires topic lifecycle management (creation, cleanup, retention policies).

Exactly-once is hard. The current implementation provides at-least-once delivery. A request may be processed twice if the ensemble crashes after completing the work but before committing the offset. For most agent workloads (which are non-deterministic anyway), this is acceptable. For workloads that require exactly-once, additional deduplication logic is needed.

When to use durable transport

The decision is straightforward:

Development and testing: in-process transport. Zero setup, fast, deterministic.
Single-node production: in-process transport with periodic state persistence. Simple, no external dependencies.
Multi-node production: Kafka transport. Durability, horizontal scaling, replay capability.
Edge or embedded: in-process transport. No infrastructure dependency.

The transport SPI lets you make this decision per-deployment without changing application code.

The Kafka transport module is part of AgentEnsemble. The durable transport guide covers the full configuration and operational details.

I'd be interested in whether others are using Kafka (or similar) for agent-to-agent communication, and what delivery guarantee level they find sufficient in practice.

Transport SPI: Making Agent Network Infrastructure Pluggable

mgd43b — Thu, 07 May 2026 14:00:00 +0000

When agent ensembles become long-running services that communicate over a network, the communication layer becomes infrastructure. And infrastructure has a property that application code should not: it varies by deployment environment.

Development uses in-process queues. Staging might use Redis. Production runs Kafka. The application code -- the agents, tasks, workflows -- should not change between these environments. The question is where to draw the abstraction line.

The transport problem

An ensemble network needs several communication primitives:

Request queues -- how work requests arrive at an ensemble
Delivery registries -- how responses get routed back to the requester
Capability registries -- how ensembles advertise and discover shared tasks and tools
Capacity tracking -- how ensembles report their current load

Each of these has a natural in-process implementation (maps, queues, lists) and at least one distributed implementation (Kafka topics, Redis streams, service registries). If these are hardcoded to a specific backing store, every deployment environment change requires code changes.

The SPI design

AgentEnsemble defines transport as a set of Java interfaces -- a Service Provider Interface -- with pluggable implementations:

// The transport factory -- entry point for all transport primitives
Transport transport = Transport.websocket("kitchen");

// Or for production with delivery guarantees
Transport transport = Transport.simple("kitchen", deliveryRegistry);

The Transport interface provides access to the individual primitives:

Primitive	Interface	Purpose
Request queue	`RequestQueue`	Inbound work request buffering
Delivery registry	`DeliveryRegistry`	Response routing back to callers
Capability registry	`CapabilityRegistry`	Shared task/tool advertisement

Each interface has a simple contract. RequestQueue, for example:

public interface RequestQueue {
    void enqueue(WorkRequest request);
    Optional<WorkRequest> poll(Duration timeout);
    int size();
}

The in-process implementation uses a LinkedBlockingQueue. The Kafka implementation produces to a topic and consumes with manual offset commits. Same interface, different backing.

Simple mode for development

The default transport uses in-process data structures:

Transport transport = Transport.websocket("kitchen");

This gives you a working ensemble network with WebSocket connections between ensembles, backed by in-process queues and maps. It is fast, requires no external infrastructure, and is appropriate for development and testing.

For local development that needs delivery tracking (ensuring responses reach their intended recipients), use the simple transport with a delivery registry:

DeliveryRegistry registry = new InMemoryDeliveryRegistry();
Transport transport = Transport.simple("kitchen", registry);

Why this matters for agent systems

The transport SPI is not unusual as an architectural pattern -- it is a standard dependency inversion. What makes it interesting in the agent context is what it enables.

Agent networks are inherently non-deterministic. Agents take variable time, produce variable output, and may fail in unpredictable ways. Adding infrastructure variability on top of that makes the system harder to reason about.

By isolating transport from application logic, you can:

Test with in-process transport -- no containers, no network, deterministic ordering
Develop locally with WebSocket transport -- real network behavior, zero infrastructure setup
Deploy to production with Kafka -- durability, horizontal scaling, replay capability
Switch between environments -- without touching agent code, task definitions, or workflow configuration

The agent code does not know whether its work requests arrive from an in-process queue or a Kafka topic. It processes them the same way.

The capability registry

One of the more interesting transport primitives is the capability registry. When an ensemble shares a task or tool on the network, that capability needs to be discoverable by other ensembles.

// Ensemble advertises capabilities
CapabilityRegistry registry = transport.capabilityRegistry();
registry.register("prepare-meal", CapabilityType.TASK, "kitchen");
registry.register("check-inventory", CapabilityType.TOOL, "kitchen");

// Another ensemble discovers capabilities
Optional<String> provider = registry.findProvider("prepare-meal");

In simple mode, this is an in-memory map. In production, it could be backed by a service registry, a shared database, or Kafka's consumer group protocol. The application code that registers and discovers capabilities does not change.

Tradeoffs

Abstraction leaks. In-process queues have different ordering and delivery guarantees than Kafka topics. The SPI abstracts the interface but cannot fully abstract the semantics. Code that depends on strict FIFO ordering will behave differently with Kafka partitions.

Configuration complexity. Each transport implementation has its own configuration (bootstrap servers, consumer groups, topic prefixes). The SPI does not unify configuration -- you still need environment-specific setup for each backing store.

Performance characteristics vary. In-process queues are nanosecond-scale. Kafka adds millisecond-scale latency. If your agent workflow is latency-sensitive, the transport choice matters and the abstraction cannot hide that.

Not all primitives are equally portable. Request queues map cleanly to most message systems. Delivery registries (correlating responses to specific requests) are harder to implement efficiently in some message brokers.

The design principle

The useful insight is that agent network communication has a small number of well-defined primitives (request queuing, response routing, capability registration), and these primitives have natural implementations at every scale (in-process, single-node, distributed).

Rather than building the network layer directly on top of a specific infrastructure choice, defining the primitives as interfaces lets the infrastructure decision be made at deployment time rather than at development time.

This is standard dependency inversion. It is not novel. But it is the foundation that makes everything else in the ensemble network possible -- durable transport, discovery, federation, and capacity management all build on these same interfaces.

The transport SPI is part of AgentEnsemble. The durable transport guide covers the Kafka implementation in detail.

I'd be interested in what transport backends others are using for agent-to-agent communication, and whether the primitive set (request queue, delivery registry, capability registry) feels complete or whether there are missing pieces.

Bridging MCP into Java Agent Systems: Reusing the Tool Ecosystem Without Leaving the JVM

mgd43b — Tue, 05 May 2026 14:00:00 +0000

The Model Context Protocol has created a growing ecosystem of tool servers -- filesystem operations, git integration, database access, API connectors. Most of these servers are written in TypeScript and communicate over stdio or SSE.

If you are building agent systems on the JVM, you face a choice: rewrite every tool in Java, or find a way to use what already exists. The useful answer is usually both -- and the bridge between them needs to be clean enough that the rest of your system does not care which approach a particular tool uses.

The integration problem

MCP servers expose tools through a well-defined protocol. LangChain4j (which AgentEnsemble builds on) already has MCP client support via McpClient and McpToolProvider. But there is a gap: LangChain4j's MCP integration produces tools for its AiServices abstraction, not for AgentEnsemble's AgentTool interface.

The bridge needs to:

Connect to any MCP server (stdio or SSE transport)
Discover available tools from the server
Adapt each MCP tool to the AgentTool interface
Manage the server subprocess lifecycle
Allow MCP tools and Java-native tools to coexist in the same agent's tool list

McpToolFactory

The agentensemble-mcp module provides McpToolFactory as the primary entry point. Connect to any MCP-compatible server and get back standard AgentTool instances:

try (StdioMcpTransport transport = new StdioMcpTransport.Builder()
        .command(List.of("npx", "--yes",
            "@modelcontextprotocol/server-filesystem", "/workspace"))
        .build()) {

    List<AgentTool> tools = McpToolFactory.fromServer(transport);

    Agent agent = Agent.builder()
        .role("File analyst")
        .goal("Analyze project structure")
        .tools(tools)
        .llm(model)
        .build();
}

The factory connects to the server, enumerates its tools, and wraps each one as an McpAgentTool. Because MCP tools already have typed parameter schemas, the wrapper passes those schemas through to LangChain4j's ToolSpecification directly -- no intermediate Java record needed.

You can also filter to specific tools:

List<AgentTool> tools = McpToolFactory.fromServer(transport,
    "read_file", "search_files", "directory_tree");

This is useful when a server exposes tools you do not want the agent to have access to -- write operations, for instance, when the agent should only read.

Convenience factories for common servers

The two most common MCP servers for coding workflows are the filesystem and git reference servers. McpToolFactory provides convenience methods that handle the subprocess setup:

try (McpServerLifecycle fs = McpToolFactory.filesystem(projectDir);
        McpServerLifecycle git = McpToolFactory.git(projectDir)) {
    fs.start();
    git.start();

    List<AgentTool> allTools = new ArrayList<>();
    allTools.addAll(fs.tools());
    allTools.addAll(git.tools());

    // Use allTools in any agent
}

The filesystem server provides: read_file, write_file, edit_file, search_files, list_directory, directory_tree, get_file_info.

The git server provides: git_status, git_diff_unstaged, git_diff_staged, git_diff, git_commit, git_add, git_log, git_branch, git_create_branch, git_checkout, git_show, git_reset.

Lifecycle management

MCP servers run as subprocesses. If you do not shut them down, you leak processes. McpServerLifecycle implements AutoCloseable so try-with-resources handles cleanup:

try (McpServerLifecycle server = McpToolFactory.filesystem(dir)) {
    server.start();
    // Use server.tools() ...
} // server is shut down here, subprocess is killed

For long-running ensembles, McpServerLifecycle also integrates with the ensemble's lifecycle listener. When the ensemble stops, any attached MCP servers are shut down automatically.

The lifecycle object exposes health checks:

if (server.isAlive()) {
    List<AgentTool> tools = server.tools();
}

Mixing MCP and Java-native tools

The most practical pattern is combining MCP tools with Java-native tools in the same agent. MCP provides the filesystem and git operations; Java-native tools handle domain-specific logic, calculations, or API calls:

try (McpServerLifecycle fs = McpToolFactory.filesystem(projectDir)) {
    fs.start();

    Agent agent = Agent.builder()
        .role("Code reviewer")
        .goal("Review code changes and check style compliance")
        .tools(fs.tools())                    // MCP filesystem tools
        .tools(List.of(                       // Java-native tools
            new StyleCheckerTool(),
            new MetricsCalculatorTool()))
        .llm(model)
        .build();
}

Both tool types implement the same AgentTool interface. The agent sees a flat list of tools with names and descriptions. It does not know or care which ones are backed by an MCP subprocess and which are pure Java.

This composability is the point. You can start with MCP servers for rapid capability acquisition, then replace individual tools with Java implementations when you need more control, better performance, or fewer runtime dependencies -- without changing the agent configuration.

The adapter pattern

Under the hood, each MCP tool is wrapped in an McpAgentTool:

public final class McpAgentTool implements AgentTool {
    private final McpClient client;
    private final String toolName;
    private final String toolDescription;
    private final JsonObjectSchema parameters;

    @Override
    public String name() { return toolName; }

    @Override
    public String description() { return toolDescription; }

    @Override
    public ToolResult execute(String input) {
        // Parse input JSON, call client.executeTool(), wrap result
    }
}

The adapter preserves the MCP tool's name, description, and parameter schema. The parameter schema flows through to the LLM's function-calling interface, so the model sees the same tool signature regardless of whether the tool is MCP-backed or Java-native.

Connecting to custom MCP servers

Any MCP-compatible server works -- not just the reference implementations. If you have a custom server that exposes domain-specific tools (database queries, API operations, internal services), connect it the same way:

// Custom MCP server over stdio
try (StdioMcpTransport transport = new StdioMcpTransport.Builder()
        .command(List.of("python", "-m", "my_custom_mcp_server"))
        .build()) {

    List<AgentTool> tools = McpToolFactory.fromServer(transport);
    // Use tools...
}

SSE transport works similarly for remote servers:

SseMcpTransport transport = new SseMcpTransport.Builder()
    .sseUrl("http://mcp-server:8080/sse")
    .build();

List<AgentTool> tools = McpToolFactory.fromServer(transport);

Tradeoffs

Subprocess overhead. Each MCP server is a separate process. For the reference servers, this means Node.js must be installed. The startup cost is measurable (typically 1-2 seconds for npx to resolve and start the server). For long-running agents, this is negligible; for one-shot scripts, it adds latency.

Debugging across process boundaries. When an MCP tool fails, the error comes back as a string from the subprocess. You lose Java stack traces and structured exception types. The bridge logs tool inputs and outputs at DEBUG level, but cross-process debugging is inherently harder.

Schema fidelity. MCP tool schemas are JSON Schema. The bridge passes these through as-is, which works well for LangChain4j's function-calling support. But if you need to validate inputs in Java before sending them to the server, you would need to add that validation layer yourself.

No hot-reload. If the MCP server crashes, the tools become unavailable. The bridge does not automatically restart servers. For production deployments, you would want health-check and restart logic around the lifecycle objects.

When to use MCP vs. Java-native tools

Consideration	MCP	Java-native
Ecosystem breadth	Large and growing	You build what you need
Runtime dependency	Node.js (for reference servers)	Pure JVM
Startup latency	1-2s per server	Instant
Debugging	Cross-process	Same-process stack traces
Customization	Limited to server's API	Full control
Integration with Java types	String-based	Native records, type safety

The practical pattern: start with MCP for rapid capability bootstrapping, move to Java-native tools for anything performance-sensitive or deeply integrated with your domain model.

The MCP bridge is part of AgentEnsemble. The MCP bridge guide covers the full API and transport options.

Curious whether others are mixing MCP and native tools in their agent systems, and where the boundary between the two tends to settle in practice.

Coding Agents on the JVM: Project Detection, Workspace Isolation, and Tool Composition

mgd43b — Sun, 03 May 2026 14:00:00 +0000

Most agent frameworks treat coding tasks the same as any other task: give the agent a file-read tool and a file-write tool and hope for the best.

In practice, an agent that can read and write files is not the same as an agent that can reliably work on a codebase. The gap between "can modify files" and "can fix a bug in a Gradle project" is significant, and it is mostly about context that the agent needs but does not have.

The missing context

A coding agent needs to know things that a general-purpose agent does not:

What kind of project is this? Is it Java with Gradle, Python with pip, TypeScript with npm? The build command, test command, and source layout all follow from this.
Where is the code? Source roots like src/main/java are conventions, not universal truths. The agent needs to know where to look.
How do I verify my changes? Running ./gradlew test is fundamentally different from running npm test. The agent needs the right command for the project.
How do I avoid breaking things? If the agent edits files directly in the user's working tree, a failed experiment leaves half-finished code behind.

Without this context, agents make predictable mistakes: they guess at build commands, search in wrong directories, and leave the codebase in a worse state than they found it.

Project detection as a first-class concern

The approach I've been working on in AgentEnsemble treats project detection as an explicit step before tool assembly.

ProjectDetector.analyze(Path) scans the project root for build-file markers and returns a ProjectContext that captures language, build system, source roots, and the commands needed to build and test:

Marker file	Language	Build command	Test command
`build.gradle.kts` / `build.gradle`	Java	`./gradlew build`	`./gradlew test`
`pom.xml`	Java	`mvn compile`	`mvn test`
`package.json` + `tsconfig.json`	TypeScript	`npm run build`	`npm test`
`pyproject.toml` / `requirements.txt`	Python	`python -m build`	`python -m pytest`
`go.mod`	Go	`go build ./...`	`go test ./...`
`Cargo.toml`	Rust	`cargo build`	`cargo test`

This is not magic -- it is a lookup table backed by file-existence checks. But it means the agent's system prompt includes the correct build and test commands for the specific project, rather than generic instructions that may or may not apply.

The detected context is injected into the agent's instructions automatically. The agent knows it is working on a Java/Gradle project with source at src/main/java and tests at src/test/java, and it knows that ./gradlew test is the verification command.

Workspace isolation via git worktrees

The harder problem is safety. A coding agent that writes directly to the user's working tree is an agent that can break your build, conflict with your uncommitted work, or leave half-finished refactoring behind if it fails partway through.

Git worktrees solve this cleanly. A worktree is a lightweight, branch-isolated copy of a repository that shares the same object store as the original. Creation is fast and disk-efficient because it does not duplicate the git history.

EnsembleOutput result = CodingEnsemble.runIsolated(model, repoRoot,
    CodingTask.implement("Add user profile endpoint"));

That runIsolated call:

Creates a git worktree from the current branch
Runs the coding agent inside the worktree
On success, preserves the worktree for review (you can inspect the changes, run tests again, then merge)
On failure, cleans up the worktree automatically

The key interface is Workspace:

public interface Workspace extends AutoCloseable {
    Path path();          // Absolute path to the isolated directory
    void close();         // Clean up (remove worktree)
}

For non-git projects, a DirectoryWorkspace creates a temporary directory and optionally copies source files. But for the common case -- a git repository -- worktrees provide isolation without the cost of a full clone.

The tradeoff is that worktrees require a git repository. If you are working on a non-git project or a freshly initialized directory, the fallback to temporary directories is less elegant. But for the vast majority of real codebases, worktrees are the right abstraction.

Composable tool backends

Different environments have different constraints. Some teams run pure-JVM deployments where Node.js is not available. Others already use MCP servers and want to reuse them. A coding agent framework should not force one approach.

AgentEnsemble provides three tool backends, selected via ToolBackend:

Backend	Description	Requires
`AUTO`	Detect best available backend	Nothing
`JAVA`	Java-native coding tools (glob, search, edit, shell, git, build, test)	`agentensemble-tools-coding` on classpath
`MCP`	MCP reference servers for filesystem + git	`agentensemble-mcp` + Node.js
`MINIMAL`	`FileReadTool` only	Always available

AUTO resolves in order: MCP > JAVA > MINIMAL. If neither optional module is on the classpath, the agent works with file-read only -- limited, but functional for read-only analysis tasks.

The Java backend provides purpose-built coding tools:

GlobTool -- find files by pattern across the project
GrepTool -- search file contents with regex
CodeEditTool -- surgical line-range replacement (not full-file overwrite)
ShellTool -- execute build/test commands with output capture
GitTool -- status, diff, stage, commit

The MCP backend starts the official MCP filesystem and git reference servers as subprocesses and adapts their tools to the AgentTool interface. Both backends produce the same tool interface, so the rest of the framework does not care which one is active.

The one-liner and the builder

For the common case, a single call handles everything:

EnsembleOutput result = CodingEnsemble.run(model, Path.of("/my/project"),
    CodingTask.fix("NullPointerException in UserService.getById()"));

That call detects the project, assembles tools, generates a coding-specific system prompt, and runs the agent with a higher iteration limit (75 vs the default 25 -- coding tasks typically need more rounds).

For more control, the builder exposes every knob:

Agent agent = CodingAgent.builder()
    .llm(model)
    .workingDirectory(Path.of("/my/project"))
    .toolBackend(ToolBackend.JAVA)
    .requireApproval(true)
    .maxIterations(75)
    .additionalTools(myCustomTool)
    .build();

The builder returns a standard Agent -- no subclassing, no special execution path. You can use it with Task, Ensemble, phases, or any other framework feature. The coding agent is composed from the same primitives as every other agent.

Pre-configured task types

Common coding workflows have predictable shapes. A bug-fix task needs different instructions than a feature implementation or a refactoring:

Task fix       = CodingTask.fix("NullPointerException in handler");
Task implement = CodingTask.implement("Add pagination to /api/users");
Task refactor  = CodingTask.refactor("Extract UserRepository interface");

Each returns a standard Task with appropriate description and expected-output templates. They can be further customized:

Task task = CodingTask.fix("Some bug")
    .toBuilder()
    .expectedOutput("Custom expected output")
    .build();

This is a convenience, not a requirement. You can always construct a Task manually and pass it to a coding agent.

Tradeoffs and limitations

Project detection is heuristic. It works for standard project layouts but will not detect custom build systems or unconventional directory structures. The fallback is explicit configuration via the builder.

Iteration limits are a blunt instrument. A higher limit gives the agent more chances to iterate, but it also means higher token costs if the agent goes in circles. There is no substitute for good prompting and appropriate task scoping.

Workspace isolation adds a step. The agent works in a worktree, but the user still needs to review and merge the changes. This is deliberate -- automated merge would undermine the safety guarantee -- but it does add friction to the workflow.

Tool backend selection is build-time. You choose your backend by including the right dependency. Runtime switching between Java and MCP backends is possible via AUTO, but you cannot hot-swap mid-execution.

The design principle

The useful abstraction is not "an agent that can code" but "a standard agent with the right tools and context for coding tasks." The coding agent is not a special type -- it is a regular agent, assembled with project-aware tools, operating in an isolated workspace, and configured with appropriate iteration limits.

This matters because it means coding agents compose with everything else in the framework: phases, delegation, human review, metrics, traces. There is no separate execution path to maintain.

The coding agent modules are part of AgentEnsemble. The coding agents guide and workspace isolation guide cover the full API.

I'd be interested in feedback on the tool backend abstraction -- whether three levels (MCP, Java, minimal) feels like the right granularity, or whether there are intermediate points that would be useful.

Humans as Participants, Not Controllers: Designing Agent Systems That Run Without You

mgd43b — Thu, 30 Apr 2026 14:00:00 +0000

Most human-in-the-loop designs treat humans as gatekeepers. The agent pipeline pauses, a notification fires, a human reviews and approves, the pipeline continues. If the human is not there, the system waits. If the human takes too long, the system times out.

This works for simple approval workflows. It does not work for systems that need to run autonomously for hours or days while humans come and go.

The harder design problem is: how do you build agent systems where humans are participants in the system rather than controllers of it? Where the system runs without them, benefits from their presence, and does not break when they leave?

The Controller Model vs the Participant Model

In the controller model, the human is a required step in the pipeline. The system cannot proceed without them. If the human is unavailable, the system blocks. Every approval gate is a potential bottleneck.

In the participant model, the human connects to a running system, observes its current state, provides input where useful, makes decisions that require their authority, and disconnects. The system keeps running.

The distinction is not about removing humans from the loop. It is about changing the default from "blocked, waiting for human" to "running autonomously, human welcome."

The Interaction Spectrum

Not all human interactions have the same urgency or the same blocking requirement. The design uses a five-level spectrum:

Level	Example	Behavior
Autonomous	Housekeeping cleans rooms after checkout	No human needed
Advisory	Manager says "prioritize VIP guest"	Human input welcomed but not required
Notifiable	"Water leak detected in room 305"	Alert a human, proceed with best-effort response
Approvable	Guest requests late checkout	Ask human if available, auto-approve on timeout
Gated	Opening the hotel safe	Cannot proceed without human authorization

Most interactions in a well-designed system should fall in the first three levels. The system handles them autonomously. Humans are notified of important events but do not need to take action for the system to continue.

The gated level is reserved for decisions that genuinely require human authority -- security decisions, compliance gates, large financial commitments. These are intentionally rare and intentionally blocking.

Gated Reviews with Role Requirements

When a task requires human authorization, the review specifies who can approve:

Task openSafe = Task.builder()
    .description("Open the hotel safe for cash reconciliation")
    .review(Review.builder()
        .prompt("Manager authorization required to open the safe")
        .requiredRole("manager")
        .timeout(Duration.ZERO)  // no timeout -- wait until a human decides
        .build())
    .build();

When this review fires and no qualified human is connected:

The review is queued
An out-of-band notification is sent (Slack, email, webhook)
The task waits
When a qualified human connects to the dashboard, they see the pending review immediately
They approve or reject, and the task resumes

The key design choice: timeout(Duration.ZERO) means the system waits indefinitely. This is appropriate for decisions that genuinely cannot be made without human authority. For less critical approvals, a timeout with auto-approve provides the fallback:

Review.builder()
    .prompt("Guest requests late checkout -- approve?")
    .requiredRole("front-desk")
    .timeout(Duration.ofMinutes(10))
    .timeoutDecision(ReviewDecision.APPROVE)
    .build()

If no human responds within 10 minutes, the system auto-approves. The human can still intervene within the window, but the system does not block indefinitely for a non-critical decision.

Human Directives

Humans can inject guidance into any ensemble they have access to:

{
  "type": "directive",
  "to": "room-service",
  "from": "manager:human",
  "content": "Guest in 801 is VIP, prioritize all their requests"
}

Directives are non-blocking. They do not pause the system or wait for acknowledgment. They are injected as additional context for future task executions. The next time room service processes a request related to room 801, the directive is included in the prompt context.

This models how human managers actually work. A hotel manager does not approve every room service order. They walk through the hotel, observe what is happening, and give occasional direction: "That table needs attention." "The VIP in the penthouse gets priority." Then they move on.

Control Plane Directives

Beyond natural language guidance, humans (or automated policies) can send structured control plane directives:

{
  "type": "directive",
  "to": "kitchen",
  "from": "cost-policy:automated",
  "action": "SET_MODEL_TIER",
  "value": "FALLBACK"
}

This switches the kitchen ensemble to a cheaper LLM model without restarting. The ensemble has configurable model tiers:

Ensemble.builder()
    .chatLanguageModel(gpt4)        // primary
    .fallbackModel(gpt4Mini)        // cheaper fallback
    .build();

Other control plane actions include pausing an ensemble, adjusting priority weights, enabling or disabling specific shared tasks, and changing queue depth limits. These are operational controls that affect ensemble behavior at runtime without redeployment.

Late-Join State Synchronization

When a human connects to the dashboard -- whether it is their first time today or they are reconnecting after a network interruption -- they need to see the current state of the system immediately. They should not have to wait for events to stream in before understanding what is happening.

The existing late-join mechanism (from v2.1.0's agentensemble-web module) extends to the network level. When a human connects:

The dashboard sends a hello message with the human's identity and roles
Each ensemble the human has access to sends a snapshotTrace -- the current state of all active tasks, pending reviews, queue depths, and recent events
Live events start streaming immediately

The human is caught up within seconds of connecting. Pending reviews that match their role are highlighted. They can start making decisions immediately without waiting for context to accumulate.

Operational Resilience

The participant model enables several operational patterns that the controller model cannot support:

Elastic scaling with human oversight. A conference weekend means higher load. The system scales automatically (K8s HPA watching queue depth). The human manager connects, observes the scaled-up state, adjusts priorities if needed, and disconnects. The system handles the load autonomously.

Operational profiles. Predefined configurations for known scenarios:

NetworkProfile sportingEvent = NetworkProfile.builder()
    .name("sporting-event-weekend")
    .ensemble("front-desk", Capacity.replicas(4).maxConcurrent(50))
    .ensemble("kitchen", Capacity.replicas(3).maxConcurrent(100))
    .ensemble("room-service", Capacity.replicas(3).maxConcurrent(80))
    .preload("kitchen", "inventory", "Extra beer and ice stocked")
    .build();

network.applyProfile(sportingEvent);

A human can apply a profile, or profiles can activate on a schedule or via rules.

Simulation and chaos engineering. Before the conference, simulate the expected load: "What happens if kitchen goes down during peak dinner service?" Run a simulation with mock LLMs, time-compressed. Get a capacity report. Then inject a kitchen failure as a chaos test. Assert that room service's circuit breaker opens within 30 seconds and the fallback activates within 1 minute. These are built into the framework, not bolted on.

Federation. Hotel A is at capacity. Hotel B across town has idle kitchen capacity. Overflow requests route to Hotel B automatically. The human manager sees both hotels on the same dashboard. This is the network-of-networks level -- multiple independent agent systems sharing capacity when needed.

Tradeoffs

Autonomy vs oversight. The more autonomous the system, the less opportunity for human correction before a mistake propagates. The mitigation is observability: the system runs autonomously but every decision is traced, logged, and visible. Humans review after the fact and inject directives to adjust future behavior.

Gating cost. Every gated review is a potential bottleneck and a source of latency. The design pressure is to minimize gated interactions -- reserve them for decisions that genuinely require human authority. If you find yourself gating routine operations, the system design needs revision, not more human approvals.

Notification fatigue. A system that notifies humans about everything trains them to ignore notifications. The notification levels (autonomous, advisory, notifiable, approvable, gated) exist to keep the signal-to-noise ratio high. Most things should be autonomous. Notifications should be reserved for things that actually need attention.

Simulation fidelity. Simulations use mock LLMs and time compression. The behavior will not perfectly match production. The value is in finding structural problems -- capacity bottlenecks, missing fallbacks, broken circuit breakers -- not in predicting exact outcomes.

This is the third and final post in the Ensemble Network architecture arc. The architecture is planned for AgentEnsemble v3.0.0. The previous posts cover ensembles as services and cross-ensemble delegation.

The design document at agentensemble.net/design/ensemble-network/ covers the full architecture including discovery, error handling, versioning, security, testing, and the phased delivery plan.

AgentEnsemble is open-source under the MIT license.

Task Sharing vs Tool Sharing: Cross-Ensemble Delegation in Distributed Agent Systems

mgd43b — Mon, 27 Apr 2026 14:00:00 +0000

MCP (Model Context Protocol) gives agents the ability to call tools hosted by other services. This is useful -- it is function-level interoperability. An agent calls a function, gets a result, continues.

But there is a level above function calls that most frameworks have not addressed: what happens when one autonomous agent system needs to delegate a complex, multi-step process to another autonomous agent system?

The distinction matters. Calling a tool is like borrowing a calculator. Delegating a task is like hiring a department.

Two Kinds of Sharing

When agent ensembles run as long-lived services on a network (as described in the previous post), they need to share capabilities with each other. There are two fundamentally different kinds of sharing:

Tool sharing exposes a single function. The calling agent invokes it in its ReAct loop, gets a result, and continues reasoning. The tool executes atomically -- there is no multi-step process, no internal agents, no review gates. This is what MCP provides.

Task sharing exposes a complete process. The calling ensemble delegates work to another ensemble, which runs its own agents, tools, memory, and review gates to produce the result. The caller does not know or control the internal process. It hands off work and gets back a result.

// Room service uses both kinds of sharing from kitchen
Ensemble roomService = Ensemble.builder()
    .name("room-service")
    .chatLanguageModel(model)
    .task(Task.builder()
        .description("Handle guest room service request")
        .tools(
            // Task sharing: delegates the full meal preparation process
            NetworkTask.from("kitchen", "prepare-meal"),

            // Tool sharing: calls a single function for inventory check
            NetworkTool.from("kitchen", "check-inventory"),
            NetworkTool.from("kitchen", "dietary-check"),

            // Task sharing: delegates repair work to maintenance
            NetworkTask.from("maintenance", "repair-request"))
        .build())
    .build();

Both NetworkTask and NetworkTool implement the same AgentTool interface. The agent calling them does not know whether a tool is local or remote, or whether it triggers a single function or an entire pipeline. The existing ReAct loop, tool executor, metrics, and tracing all work unchanged.

How Delegation Works

When an agent calls a shared tool, the flow is straightforward:

Agent calls check-inventory("wagyu beef")
NetworkTool serializes the call into a WorkRequest
Request is sent to the kitchen ensemble (WebSocket or queue)
Kitchen executes inventoryTool.execute("wagyu beef") locally
Result flows back: "Yes, 3 portions available"
Agent continues its ReAct loop

When an agent calls a shared task, the flow involves a full pipeline on the other side:

Agent calls prepare-meal("Wagyu steak, medium-rare, room 403")
NetworkTask serializes a WorkRequest with the full task context
Request is sent to kitchen
Kitchen runs its complete task pipeline -- agent synthesis, tool calls, execution, review gates
Result flows back: "Preparing now, estimated 25 minutes, ticket #4071"
Agent continues

The critical difference: in step 4 of the task delegation, the kitchen ensemble is running its own agents with its own tools and its own review gates. The room service agent is not involved in any of that. It delegated the work and is waiting for a result -- or continuing with other work if the request was async.

The WorkRequest Envelope

Every cross-ensemble message uses a standardized envelope:

public record WorkRequest(
    String requestId,           // Correlation + idempotency key
    String from,                // Requesting ensemble name
    String task,                // Shared task or tool name to execute
    String context,             // Natural language input/context
    Priority priority,          // CRITICAL / HIGH / NORMAL / LOW
    Duration deadline,          // Caller's SLA ("I need this within...")
    DeliverySpec delivery,      // How and where to return the result
    String traceContext,        // W3C traceparent for distributed tracing
    CachePolicy cachePolicy,    // USE_CACHED / FORCE_FRESH
    String cacheKey             // Optional, for result caching
) {}

A few design choices in this envelope are worth noting:

The context field is natural language. When maintenance asks procurement to order parts, the context is: "Order replacement valve for building 2 boiler." Not a typed JSON schema. Not a protobuf message. Natural language that the receiving ensemble's LLM interprets.

The deadline belongs to the caller, not the provider. The requester sets the SLA: "I need this within 30 minutes." The provider responds with an estimated completion time. If the estimate exceeds the deadline, the caller decides: accept the longer wait, try another provider (federation), or continue without.

Delivery is caller-specified. The requester tells the provider how to return the result -- WebSocket for real-time, a durable queue for reliability, a webhook for external integration, or a shared store for polling.

Natural Language as Contract

This is the design choice I find most interesting and most debatable.

In traditional microservice architectures, services communicate via typed schemas -- protobuf, OpenAPI, GraphQL. Schema versioning is a constant source of friction. A field name change breaks callers. A new required field breaks backwards compatibility. Teams spend significant effort on schema evolution, versioning policies, and migration tooling.

In the Ensemble Network, the contract between services is natural language. When maintenance tells procurement "order replacement parts for the boiler valve," it does not matter whether procurement's internal schema changed. The LLM on the receiving side interprets the request. Minor changes in wording do not break callers.

This works because the participants are LLMs, not deterministic parsers. An LLM that receives "order parts for the boiler" and an LLM that receives "purchase replacement components for the heating system" will produce equivalent behavior. The semantic intent is preserved even when the exact phrasing varies.

The tradeoff is real: you lose type safety. A typed schema guarantees that the data conforms to a specific shape. Natural language does not. If the receiving ensemble misinterprets the request, you get a wrong result, not a compile error. The mitigation is the same as elsewhere in agent systems: review gates, guardrails, and observability.

Three Request Modes

The caller decides how to wait for the result:

Mode	Behavior	Use case
Await	Block until result	Critical path: "Can't continue without this"
Async	Submit and continue; result delivered later	Non-critical: "Order towels when you get to it"
Await with deadline	Wait up to N; then continue with partial/no result	Balanced: "Wait 30 min, then proceed with what I know"

The await-with-deadline mode is the most operationally useful. It lets the caller set a budget for how long to wait before continuing. If the provider delivers within the deadline, the caller uses the result. If not, it makes a decision: retry, use a fallback, or proceed without.

Capacity Management

The provider's default response to load is accept and queue, not reject. LLM tasks are not real-time request/response -- they take seconds to hours. Everyone expects latency. The provider accepts the work into a priority queue and returns an estimated completion time:

{
  "type": "task_accepted",
  "requestId": "maint-7721",
  "queuePosition": 7,
  "estimatedCompletion": "PT45M"
}

Rejection only happens at hard limits -- the queue itself is full. This "bend, don't break" approach matches the reality of LLM workloads: capacity is elastic, latency is expected, and it is almost always better to queue work than to reject it.

Priority queuing ensures critical requests are processed first (CRITICAL > HIGH > NORMAL > LOW). Within the same priority, FIFO. Low-priority items age over time to prevent starvation.

Distributed Tracing

Every WorkRequest carries a W3C traceparent header. When maintenance delegates to procurement, which delegates to logistics, the trace context propagates across all three. Open Jaeger (or any W3C-compatible tracing backend) and you see the full chain: which ensemble originated the request, how long each step took, where the bottleneck was.

This is standard distributed tracing, not a custom solution. The same infrastructure teams use for HTTP microservices works here. The difference is that each span may represent an LLM call that takes 30 seconds instead of a database query that takes 3 milliseconds.

Tradeoffs

Loose coupling vs type safety. Natural language contracts are resilient to change but do not guarantee correctness. Typed schemas guarantee correctness but are brittle to change. The right choice depends on how stable the interface is. For evolving, exploratory agent systems, natural language is pragmatic. For stable, high-volume interfaces, a typed schema wrapper may be worth the friction.

Latency tolerance. Cross-ensemble delegation adds network hops and queuing delays. A task that takes 10 seconds locally may take 2 minutes when delegated across a network. The architecture assumes latency tolerance -- if your use case requires sub-second responses, delegation is the wrong pattern.

Failure modes. When the kitchen ensemble is down, room service's prepare-meal call fails. The circuit breaker opens. The agent needs a fallback -- suggest alternatives, queue the request for later, or inform the guest. Distributed systems fail in distributed ways. The framework provides the circuit breaker and fallback mechanisms, but the failure strategy is application-specific.

Observability cost. Every cross-ensemble request generates trace data, metrics, and log entries. In a busy network with many delegations, the observability overhead is non-trivial. The tracing infrastructure needs to handle the volume, and teams need dashboards that make sense of the flow.

This is the second post in a three-part arc on the Ensemble Network architecture. The next post covers human participation -- how humans connect to and interact with a network of autonomous ensembles without becoming bottlenecks.

The design document at agentensemble.net/design/ensemble-network/ covers the full architecture.

AgentEnsemble is open-source under the MIT license.

From Run-and-Exit to Always-On: When Agent Ensembles Become Services

mgd43b — Fri, 24 Apr 2026 14:00:00 +0000

Every multi-agent framework works the same way at its core. You define some agents, give them tasks, press go, get output. The agents exist for the duration of the run and then disappear.

This is fine for bounded problems: "research this topic and write a report." But it does not model how real work gets done in production systems that need to be always-on, multi-domain, and human-augmented.

The question I kept coming back to was: what changes when an ensemble stops being a script and starts being a service?

Scripts vs Services

A script runs and exits. You invoke it, it does work, it returns a result, the process terminates. Every multi-agent framework today -- CrewAI, AutoGen, LangGraph, AgentEnsemble v2.x -- operates in this mode.

A service runs continuously. It handles work as it arrives, communicates with peers, maintains state between requests, and survives restarts. The difference is not just about uptime -- it changes the entire interaction model.

When an ensemble is a script, it is invoked by something external. When an ensemble is a service, it participates in a network of other services. It can accept work from multiple sources, share capabilities with peers, and run proactive tasks on a schedule -- all without an external orchestrator telling it what to do.

The Hotel Model

Consider a hotel. It is composed of departments: front desk, housekeeping, kitchen, room service, maintenance, procurement. Each department is autonomous -- it has its own staff, processes, and expertise. These departments communicate with each other directly. Room service calls the kitchen to prepare a meal. Maintenance calls procurement to order spare parts.

The hotel runs continuously. The manager comes in at 8am, walks around, checks on things, gives some direction, handles decisions that require authority, and goes home at 6pm. The hotel does not stop when the manager leaves.

This maps directly to a distributed agent architecture:

Hotel concept	Agent system equivalent
A department	An ensemble -- long-running, autonomous
Staff within a department	Agents and tasks within the ensemble
The intercom / phone system	WebSocket mesh -- the message transport
A work order	A WorkRequest -- the standard message envelope
The hotel directory	Service registry -- ensembles discover each other
The duty manager	A human who connects via the dashboard to observe and intervene

The key observation: the hotel is not centrally orchestrated. There is no "manager agent" that routes every message. Departments handle their domain and communicate laterally.

Two Execution Modes

The existing one-shot mode remains unchanged:

EnsembleOutput output = Ensemble.run(model,
    Task.of("Research AI trends"),
    Task.of("Write a report"));

Tasks execute, output is returned, the ensemble is done. This is a "gig" -- a bounded unit of work.

The new long-running mode turns the ensemble into a service:

Ensemble kitchen = Ensemble.builder()
    .name("kitchen")
    .chatLanguageModel(model)
    .task(Task.of("Manage kitchen operations"))

    // Share capabilities to the network
    .shareTask("prepare-meal", Task.builder()
        .description("Prepare a meal as specified")
        .expectedOutput("Confirmation with preparation details and timing")
        .build())
    .shareTool("check-inventory", inventoryTool)

    // Scheduled proactive task
    .scheduledTask(ScheduledTask.builder()
        .name("inventory-report")
        .task(Task.of("Check current inventory levels and report shortages"))
        .schedule(Schedule.every(Duration.ofHours(1)))
        .broadcastTo("hotel.inventory")
        .build())

    .build();

kitchen.start(7329);  // WebSocket server, K8s Service fronts this

In long-running mode, the ensemble:

Registers shared tasks and tools on the network
Accepts incoming work requests via WebSocket, queue, HTTP, or topic subscription
Processes work through a priority queue
Delivers results via the caller-specified delivery method
Runs scheduled proactive tasks on configured intervals
Continues until explicitly stopped or drained

The start(port) call is the boundary between script and service. Before it, the ensemble is a configuration. After it, the ensemble is an active participant in a network.

Work Ingress

When an ensemble becomes a service, work can arrive from multiple sources simultaneously:

Source	Description
WebSocket	Direct from another ensemble (real-time)
Queue	Pull from durable queue (Kafka, SQS, Redis Streams)
HTTP API	`POST /api/work` (external systems, scripts, CI pipelines)
Topic subscription	React to events from other ensembles
Schedule	Internal cron/interval (proactive tasks)

All sources normalize into the same internal format before entering the ensemble's priority queue. The ensemble processes work by priority (CRITICAL > HIGH > NORMAL > LOW), with FIFO ordering within the same priority level.

This means an ensemble can simultaneously handle direct requests from peer ensembles, pull batch work from a queue, respond to events, and run scheduled health checks -- without any of these mechanisms knowing about each other.

Deployment Model

Each ensemble deploys as a Kubernetes service -- one or more pods behind a K8s Service resource. Ensembles discover each other via DNS name. This is standard infrastructure that operations teams already know how to manage.

Namespace: hotel-downtown
  +-- Service: kitchen
  +-- Service: room-service
  +-- Service: maintenance
  +-- Service: front-desk
  +-- Service: dashboard

Scaling is handled by Kubernetes HPA watching queue depth or request latency. Conference weekend with heavy kitchen load? Scale kitchen to 3 replicas. Off-peak Tuesday? Scale back to 1. The ensemble handles replica coordination through broadcast-claim delivery: a work request is offered to all replicas, and the first to claim it processes it.

What Changes

The shift from script to service changes several things:

Lifecycle management matters. A script that crashes restarts from scratch. A service that crashes needs graceful shutdown, drain logic, and state recovery. The ensemble supports a drain mode where it stops accepting new work, finishes in-flight tasks, and shuts down cleanly. On restart, it picks up queued work from durable sources.

Proactive work becomes possible. A script only does what you tell it to do. A service can schedule its own work -- periodic inventory checks, health assessments, report generation. These scheduled tasks run on internal timers and broadcast results to interested subscribers.

Observability changes. A script that runs for 30 seconds needs a log. A service that runs for months needs a dashboard. The existing web module (WebSocket server, live trace streaming, late-join snapshot) extends naturally to the long-running model.

The human relationship changes. A script blocks on human input and times out. A service has humans who connect and disconnect. They observe the current state, give direction, handle decisions that need authority, and leave. The system keeps running. This is a deep enough topic that the next post in this series will cover it in detail.

Tradeoffs

Complexity vs capability. A script is simple: invoke it, get a result. A service requires infrastructure -- Kubernetes, queues, monitoring, lifecycle management. If your workload is "run this pipeline once and give me the output," the service model is unnecessary overhead.

Always-on cost. A script uses resources only while it runs. A service uses resources continuously, even when idle. For intermittent workloads, the cost calculus favors one-shot execution with on-demand scaling.

State management. Scripts are stateless by nature -- they start fresh every time. Services accumulate state: queued work, scheduled tasks, shared memory, connection state. This state needs to be durable, recoverable, and observable.

When to use which. The one-shot mode is right for discrete, bounded problems. The long-running mode is right when the workload is continuous, when multiple domains need to communicate, when humans need to observe and participate without blocking, and when the system needs to be always-on.

Both modes coexist. An ensemble that runs as a long-running service can still execute individual tasks in one-shot mode internally. The architecture does not force a choice -- it extends the existing model.

This is the first post in a three-part arc on the Ensemble Network architecture planned for v3.0.0. The next post covers cross-ensemble delegation -- how ensembles share tasks and tools across service boundaries, and why the contract between them is natural language, not typed schemas.

The design document at agentensemble.net/design/ensemble-network/ covers the full architecture.

AgentEnsemble is open-source under the MIT license.

Token-Efficient Context Passing: Pluggable Serialization for Multi-Agent Pipelines

mgd43b — Tue, 21 Apr 2026 14:00:00 +0000

In a multi-agent pipeline, structured data flows between tasks at every step. Task outputs become context for downstream tasks. Tool results are appended to the conversation. Memory entries are injected into prompts. All of this data is serialized as text and counted against the model's context window.

The default serialization format is JSON. JSON is familiar, well-supported, and universally understood by LLMs. It is also verbose. Curly braces, quoted keys, commas, colons, and brackets consume tokens that carry no semantic value for the model. In a short pipeline with small payloads, this overhead is negligible. In a long pipeline with rich context -- multiple tool calls per task, structured outputs flowing forward, memory entries accumulating -- it compounds quickly.

The question I kept coming back to was: where does the serialization format actually matter in this pipeline, and can the framework make it pluggable without leaking complexity?

Where Tokens Go

In a typical multi-agent workflow, structured data appears in four places:

Location	What gets serialized	Token impact
Task context	Outputs from prior tasks injected into downstream prompts	Medium -- depends on output size
Tool results	JSON payloads returned by tool executions during ReAct loops	High -- tool results are often large and iterate multiple times
Memory entries	Structured content from memory scopes	Medium -- grows with pipeline length
Trace export	Execution traces serialized for analysis	None -- not sent to the LLM

Tool results tend to dominate. A tool that queries a database or calls an API returns a JSON payload. In a ReAct loop with multiple iterations, these payloads accumulate in the conversation history. Each iteration adds more serialized data to the context window.

The framework already controls every one of these serialization points. It builds prompts, formats tool results, injects memory, and exports traces. That means a single configuration point can control the format everywhere, without requiring changes to task definitions, tool implementations, or agent logic.

The Design

The goal was a pluggable serialization layer with three properties:

Opt-in: JSON remains the default. No existing code changes behavior.
Fail-fast: If a format is selected but its runtime dependency is missing, the ensemble fails at build time with a clear error, not at runtime mid-pipeline.
Single configuration point: One builder call controls all serialization points. No per-task or per-tool format settings.

The API surface is small:

public enum ContextFormat {
    JSON,
    TOON
}

public interface ContextFormatter {
    String format(Object value);
    String formatJson(String json);
}

ContextFormat is an enum selecting the serialization strategy. ContextFormatter is an interface with two methods: format(Object) for Java objects and formatJson(String) for re-encoding existing JSON strings. The distinction matters because tool results arrive as JSON strings that need to be converted, while task outputs and memory entries are Java objects that need to be serialized from scratch.

A factory class resolves the correct implementation:

ContextFormatter formatter = ContextFormatters.forFormat(ContextFormat.TOON);
String encoded = formatter.format(myObject);

TOON as a Concrete Alternative

TOON (Token-Oriented Object Notation) is a compact, human-readable serialization format designed specifically for LLM contexts. It combines YAML-like indentation with CSV-like tabular arrays and achieves 30-60% token reduction versus JSON.

JSON:

{"items":[{"sku":"A1","qty":2,"price":9.99},{"sku":"B2","qty":1,"price":14.5}]}

TOON:

items[2]{sku,qty,price}:
  A1,2,9.99
  B2,1,14.5

The token savings come from eliminating repeated key names in arrays (declared once in the header), removing quotes around keys, and using indentation instead of braces. For tabular data -- which tool results and structured outputs frequently contain -- the reduction is substantial.

JToon is the Java implementation. It is MIT-licensed, available on Maven Central, requires Java 17+, and supports Jackson annotations.

Wiring It In

Enabling TOON is one builder call:

EnsembleOutput result = Ensemble.builder()
    .chatLanguageModel(model)
    .contextFormat(ContextFormat.TOON)
    .task(researchTask)
    .task(analysisTask)
    .task(reportTask)
    .build()
    .run();

At build time, if TOON is selected, the framework verifies that the JToon class is loadable. If not, it throws an IllegalStateException with Maven and Gradle coordinates:

TOON context format requires the JToon library on the classpath.
Add to your build:
  Gradle: implementation("dev.toonformat:jtoon")
  Maven:  <dependency><groupId>dev.toonformat</groupId><artifactId>jtoon</artifactId></dependency>

This is a deliberate design choice. JToon is declared as compileOnly in the framework, so applications that never use TOON pay no dependency cost. Applications that do use it add one dependency and one builder call.

The resolved ContextFormatter is stored in ExecutionContext and passed to every component that serializes data for the LLM: the prompt builder, the tool result formatter, and the memory injector. No component needs to know which format is active -- it just calls formatter.format(value).

Integration Points

Prompt Building

When the prompt builder constructs the user message, context from prior tasks and memory entries flows through the configured formatter. If the format is TOON, the data arrives in the prompt as TOON. The LLM reads it as context -- it does not need to produce TOON output.

One important boundary: structured output schemas remain in JSON regardless of the context format. If a task has an outputType, the JSON schema in the prompt stays in JSON because the LLM needs to produce parseable JSON that the framework can deserialize. The context around the schema uses whatever format is configured.

Tool Results

Tool execution results are the highest-impact integration point. When a tool returns a JSON string, the framework can re-encode it via contextFormatter.formatJson(toolResultText) before appending it to the conversation. In a ReAct loop with multiple tool calls, each iteration's results are formatted, and the savings compound across the conversation.

Trace Export

ExecutionTrace gains TOON export methods alongside the existing JSON ones:

EnsembleOutput result = ensemble.run();

// JSON (always available)
result.getTrace().toJson(Path.of("trace.json"));

// TOON (requires JToon on classpath)
result.getTrace().toToon(Path.of("trace.toon"));

Trace export is not sent to the LLM, so the token savings do not apply here. The benefit is smaller trace files for storage and analysis.

Tradeoffs

Compatibility vs savings: JSON is universally understood by every LLM. TOON is newer and less widely tested across models. For models that handle structured text well (GPT-4o, Claude, Gemini), TOON works reliably as context. For smaller or less capable models, JSON may be safer.

Debuggability vs compactness: When you are inspecting prompts during development, JSON is more familiar. TOON is human-readable but less immediately obvious. During development, you might use JSON and switch to TOON for production workloads where cost matters.

Cost vs complexity: TOON reduces token usage, which directly reduces API cost. In a production pipeline processing thousands of runs, 30-60% fewer tokens in context passing translates to measurable cost savings. The complexity cost is one dependency and one builder call.

Schema boundary: The structured output schema stays in JSON. This means a prompt with TOON context and a JSON schema contains mixed formats. In practice, this works because the LLM treats the context section and the schema section as separate concerns. But it is worth being aware of if you are debugging prompt construction.

The format is opt-in and the default is unchanged. If you never set contextFormat, nothing changes. The pluggable design means future formats can be added -- a custom ContextFormatter implementation for a domain-specific format, or a future format optimized for a specific model family -- without changing the public API.

The guide at agentensemble.net/guides/toon-format/ covers setup, configuration, and usage patterns. The design document at agentensemble.net/design/toon-context-format/ covers the architectural decisions.

AgentEnsemble is open-source under the MIT license.

Typed Tool Inputs in Java Agent Systems: Records as Contracts

mgd43b — Sat, 18 Apr 2026 14:00:00 +0000

There is a design tension at the boundary between an LLM and a tool. The LLM generates a structured JSON call. Your code receives it. Between those two points, something has to parse the JSON, validate the fields, and turn it into something the tool can actually work with.

Most agent frameworks handle this with a single opaque string. The LLM gets a parameter named input. The tool receives a raw string. The tool author writes their own JSON parsing. Required-field validation is up to them. If the LLM passes malformed JSON or omits a required field, the tool finds out at runtime.

The question I kept coming back to was: why doesn't the framework own this? Java has records. Records have components with names and types. Those components can carry annotations. Everything needed to generate a typed JSON Schema, deserialize the call, and validate required fields is already there, waiting to be read.

The Problem With Single-String Tool Inputs

The original model is simple and universal. A tool implements one method: execute(String input). The LLM is told to format the input as JSON if the tool needs structured data. The tool parses it.

This works, but it puts a burden in the wrong place. Consider a web search tool:

public class WebSearchTool extends AbstractAgentTool {
    @Override
    public String name() { return "web_search"; }

    @Override
    public String description() {
        return "Performs a web search. Input: JSON with 'query' field.";
    }

    @Override
    protected ToolResult doExecute(String input) {
        JsonNode node = mapper.readTree(input);
        String query = node.get("query").asText();  // NPE if missing
        // ...
    }
}

A few problems here. The description must instruct the LLM about the expected JSON format, mixing tool contract with tool documentation. The tool code handles parsing. There is no schema — the LLM has no structured signal about what fields are expected, what their types are, or which are required. A missing query field produces a null pointer exception rather than a clean validation failure.

The tool input boundary is where type safety matters most. It is the handoff between model output and application code. A malformed call here does not just fail — it can silently corrupt state or produce errors that are hard to trace back to the model call.

Records as Input Contracts

The alternative is to declare the input contract as a Java record. The framework can derive everything it needs from the record's components and their annotations: the JSON Schema for the LLM, the deserialization logic, the required-field validation.

@ToolInput(description = "Parameters for a web search")
public record WebSearchInput(
    @ToolParam(description = "Search query string, e.g. 'Java 21 virtual threads'")
    String query
) {}

The tool then extends AbstractTypedAgentTool<WebSearchInput> instead of AbstractAgentTool:

public final class WebSearchTool extends AbstractTypedAgentTool<WebSearchInput> {

    @Override
    public Class<WebSearchInput> inputType() {
        return WebSearchInput.class;
    }

    @Override
    public ToolResult execute(WebSearchInput input) {
        String query = input.query().trim();
        if (query.isBlank()) {
            return ToolResult.failure("Search query must not be blank");
        }
        // ... perform search
    }
}

The @ToolParam annotation marks fields as required by default. Optional fields use @ToolParam(required = false). There is no JSON parsing in the tool body. The framework handles it.

What the Framework Does With the Record

At startup, ToolSchemaGenerator introspects the record class and produces a JsonObjectSchema for LangChain4j. The mapping covers the types you would expect:

Java type	JSON Schema type
`String`	`string`
`int`, `long`, `Integer`	`integer`
`double`, `float`, `BigDecimal`	`number`
`boolean`	`boolean`
`enum`	`string` with `enum` constraint
`List<T>`	`array`
`Map<String, V>`	`object`

When the LLM calls the tool, LangChain4jToolAdapter passes the full JSON arguments to AbstractTypedAgentTool, which uses ToolInputDeserializer to deserialize them into a record instance. Required fields that are missing or null produce a clean ToolResult.failure before the tool body executes.

The LLM receives a proper multi-parameter schema instead of a single input string. It can see the field names, their types, and their descriptions directly in the tool specification.

The Adapter Path

LangChain4jToolAdapter dispatches based on whether a tool implements TypedAgentTool:

if (tool instanceof TypedAgentTool<?> typed) {
    parameters = ToolSchemaGenerator.generateSchema(typed.inputType());
} else {
    // original single-parameter schema, fully backward compatible
    parameters = JsonObjectSchema.builder()
        .addStringProperty("input", "The input string for the tool")
        .required(List.of("input"))
        .build();
}

The same instanceof check applies at execution time — typed tools receive the full JSON arguments object; legacy tools receive the extracted "input" field value. No changes required to existing tool implementations.

Before and After: The Schema Difference

For the legacy WebSearchTool, the LLM received this schema:

{
  "properties": {
    "input": {
      "type": "string",
      "description": "The input string for the tool"
    }
  },
  "required": ["input"]
}

With TypedAgentTool, it receives:

{
  "properties": {
    "query": {
      "type": "string",
      "description": "Search query string, e.g. 'Java 21 virtual threads'"
    }
  },
  "required": ["query"]
}

The field is named correctly. The description is the one written for that field. For tools with multiple inputs, each parameter is a separate, named, typed entry in the schema rather than buried inside an opaque string that the tool must document and parse.

Migrating a Multi-Field Tool

The pattern is the same regardless of how many fields the record carries. A hypothetical HTTP request tool:

@ToolInput(description = "Parameters for an HTTP request")
public record HttpRequestInput(
    @ToolParam(description = "The URL to request") String url,
    @ToolParam(description = "HTTP method: GET, POST, PUT, DELETE") String method,
    @ToolParam(description = "Request body (optional)", required = false) String body
) {}

public final class HttpRequestTool extends AbstractTypedAgentTool<HttpRequestInput> {

    @Override
    public Class<HttpRequestInput> inputType() { return HttpRequestInput.class; }

    @Override
    public ToolResult execute(HttpRequestInput input) {
        // url and method are guaranteed non-null; body may be null
        // no parsing, no null checks on deserialization
    }
}

The body field is optional — it will be absent from the required array in the schema, and will be null if the LLM omits it. url and method are required — a call missing either produces a failure before execute is reached.

Tradeoffs

The typed system is opt-in for a reason. There are cases where it does not fit well.

Records only. The framework introspects Java records specifically. If an existing tool has complex input logic tied to its own parsing, migrating to a record may not be a clean fit.

Reflection at startup. Schema generation uses reflection on the record's components. For a large number of tools, this adds initialization cost. In practice it is small, but it is not zero.

Enum constraints. Enum fields generate a string schema with an enum list derived from Enum.name(). This aligns with Jackson's default deserialization behavior. If you have custom serialization names, the generated enum list may not match the LLM's expectations without additional configuration.

Not a substitute for description quality. The schema tells the LLM the structure. The field descriptions still determine whether the LLM understands how to fill that structure correctly. A typed schema with poor descriptions is only marginally better than a well-documented string input.

Five of the built-in tools were migrated to the typed system (FileReadTool, FileWriteTool, JsonParserTool, WebSearchTool, WebScraperTool). Four were left as legacy tools (CalculatorTool, DateTimeTool, HttpAgentTool, ProcessAgentTool) — either because their inputs are naturally single-valued, or because the migration cost was not justified by the gain.

Practical Takeaway

The tool input boundary is a contract. Java records are a natural way to express that contract, and the language gives you enough reflective machinery to derive everything you need from it without additional code generation or build-time processing.

The main gain is not ergonomics, though that improves. The main gain is that the LLM receives an accurate schema for the tool it is calling, which reduces hallucinated input formats and makes validation failures deterministic rather than buried in manual parsing code.

AgentEnsemble is a JVM-native framework for building multi-agent systems in Java. The typed tool input system is part of the core library.

Guide: https://agentensemble.net/guides/tools/
Source: https://github.com/AgentEnsemble/agentensemble
MIT licensed

If you have been working with agent tool systems and have found a different approach to the input boundary problem — typed DSLs, code generation, something else — I would be interested in the tradeoffs you have seen.

Self-Optimizing Agent Tasks: Persistent Reflection Loops in Java

mgd43b — Wed, 15 Apr 2026 14:00:00 +0000

Task definitions are written at compile time. You describe what the task should do, wire up a model, and run. The prompt stays fixed unless you go back and edit it.

In practice, you often discover after a few runs that the instructions could be more precise. The LLM misses an edge case you didn't anticipate. The output format drifts in ways you didn't specify. You revise the description, redeploy, and try again.

The harder version of this problem is: what if the instructions could improve themselves?

Task reflection is a persistent, automated feedback loop built into the task execution lifecycle. After a task completes successfully -- output accepted, guardrails passed, reviews approved -- an LLM-backed analysis step reviews whether the task's instructions could be improved. Improvements are stored in a ReflectionStore and injected into the task's prompt on subsequent runs. The original task definition is never modified.

This post covers how reflection works, what the API looks like, and where the tradeoffs sit.

Reflection vs Phase Review

These two mechanisms are often confused because both involve quality analysis. The distinction is in scope and timing:

	Phase Review	Task Reflection
Trigger	After phase completes	After task output accepted
Scope	Within a single `Ensemble.run()`	Across multiple `Ensemble.run()` calls
Purpose	Fix inadequate output this run	Improve instructions for future runs
Persistence	Transient -- lost after run	Persistent -- stored between runs
Initiated by	External reviewer	Automated LLM analysis

Phase review fixes output within a run. Task reflection improves instructions across runs. They compose: a task can have both.

Quick Start

Enable reflection with .reflect(true) on a task, and configure a ReflectionStore on the ensemble:

ReflectionStore store = new InMemoryReflectionStore();

Task research = Task.builder()
    .description("research the top 5 trends in cloud-native Java for 2026")
    .reflect(true)
    .chatModel(model)
    .build();

// Run 1: no prior reflections; task executes normally
EnsembleOutput run1 = Ensemble.builder()
    .tasks(List.of(research))
    .reflectionStore(store)
    .chatModel(model)
    .build()
    .run();

// Reflection fires after run 1 completes; improvements stored in `store`

// Run 2: prior reflections injected into the prompt automatically
EnsembleOutput run2 = Ensemble.builder()
    .tasks(List.of(research))
    .reflectionStore(store)
    .chatModel(model)
    .build()
    .run();

The store is the key. Pass the same store instance across run() calls and the accumulated reflections persist. The task definition -- the Task object -- is the same every run. The difference is what the reflection store contributes to the prompt.

The Execution Lifecycle

Reflection fires at the end of the task lifecycle, after all other post-processing:

1. Task executes (LLM call)
2. Guardrails evaluate output
3. Review gate runs (if configured)
4. Memory scopes write
5. [Reflection] LLM analyzes output; generates improvement; stores in ReflectionStore

If the task fails, guardrails reject the output, or the review gate retries, reflection does not fire. Reflection only fires on a fully accepted output.

On the next run of the same task:

1. ReflectionStore loads prior reflections for this task identity
2. Reflections injected into prompt
3. Task executes with improved instructions
4. [Reflection] New analysis fires; improvement stored

The task is identified by a TaskIdentity derived from its description. Two tasks with the same description share the same reflection history.

What Gets Injected

The reflection store contributes an additional section to the task's prompt:

[original task description]

## Instruction Refinements

Based on previous runs, the following refinements have been found to improve output quality:
- Be specific about the time range: results should cover events within the last 12 months only.
- Structure the output as a numbered list with a one-sentence summary per trend.
- For each trend, cite at least one concrete project or company as evidence.

The injection is additive. Original instructions are preserved. Reflections narrow, clarify, or extend them based on what previous outputs revealed.

Reflection Configuration

Bounding the History

By default, the framework uses all stored reflections for a task. You can bound the number injected via ReflectionConfig:

Task analysis = Task.builder()
    .description("analyze customer sentiment from support tickets")
    .reflect(true)
    .reflectionConfig(ReflectionConfig.builder()
        .maxReflections(5)
        .build())
    .chatModel(model)
    .build();

With maxReflections(5), only the 5 most recent reflections are injected. Older reflections remain in the store but are not included in the prompt. This prevents prompt bloat as the number of runs grows.

Custom Reflection Strategy

The default strategy uses an LLM call to analyze the task output and generate an improvement. You can substitute a custom ReflectionStrategy:

public class DomainReflectionStrategy implements ReflectionStrategy {

    @Override
    public Optional<String> reflect(ReflectionInput input) {
        String output = input.taskOutput();
        // custom analysis: check for required sections, format, length
        if (!output.contains("## Summary")) {
            return Optional.of("Always include a ## Summary section as the first heading");
        }
        // no improvement identified this time
        return Optional.empty();
    }
}

Task.builder()
    .description("write a technical design document")
    .reflect(true)
    .reflectionConfig(ReflectionConfig.builder()
        .strategy(new DomainReflectionStrategy())
        .build())
    .chatModel(model)
    .build();

A custom strategy can use deterministic rules, call a different model, or apply domain-specific analysis. Returning Optional.empty() skips storage for that run.

The Reflection Store

ReflectionStore is an interface with two methods:

public interface ReflectionStore {
    List<TaskReflection> load(TaskIdentity identity);
    void store(TaskIdentity identity, TaskReflection reflection);
}

InMemoryReflectionStore is included for development and testing. It holds reflections in a ConcurrentHashMap and loses state when the process stops.

For production, implement ReflectionStore against whatever persistence layer makes sense for your system -- a relational database, a document store, or a key-value store:

public class JdbcReflectionStore implements ReflectionStore {

    private final DataSource dataSource;

    @Override
    public List<TaskReflection> load(TaskIdentity identity) {
        // SELECT content FROM task_reflections WHERE task_id = ?
        // ORDER BY created_at DESC LIMIT maxReflections
    }

    @Override
    public void store(TaskIdentity identity, TaskReflection reflection) {
        // INSERT INTO task_reflections (task_id, content, created_at) VALUES (?, ?, ?)
    }
}

The TaskReflection record holds the improvement text and a timestamp. TaskIdentity includes the task description hash used for keying.

Disabling Reflection Selectively

Reflection is opt-in per task. Tasks without .reflect(true) are unaffected even if a ReflectionStore is configured on the ensemble. You can enable reflection for high-value tasks and leave it off for tasks where the instructions are stable or where the cost of an extra LLM call isn't justified.

Ensemble.builder()
    .tasks(List.of(
        stableDataFetchTask,       // no reflection
        evolving AnalysisTask,     // .reflect(true)
        stableFormattingTask       // no reflection
    ))
    .reflectionStore(store)
    .chatModel(model)
    .build()
    .run();

The store is queried and written only for tasks with reflection enabled.

Tradeoffs

Reflection adds an LLM call per reflective task per run. For tasks that run thousands of times, this adds up. The cost is bounded if reflections converge -- if the task's instructions become stable after a few runs, reflections may produce no new improvements and Optional.empty() returns more often.

Reflections can drift. If a task's purpose changes -- the description is updated, the downstream context changes, the data it processes shifts -- earlier reflections may no longer apply. maxReflections helps here by aging out old improvements. For significant task changes, clearing the stored reflections for that task is reasonable.

Reflection is not a substitute for good initial instructions. A task with fundamentally unclear instructions will accumulate reflections that patch around the ambiguity. The better use is to start with reasonable instructions and use reflection to sharpen them in response to real outputs over time.

The original task definition is never modified. All improvements live in the store. This is a deliberate choice: the source of truth for what a task does remains in code, not in a mutable prompt that silently drifts over time.

The guide at agentensemble.net/guides/task-reflection/ covers the full lifecycle, store implementation patterns, and configuration options. The design document at agentensemble.net/design/task-reflection/ covers the architectural decisions behind the feature.

AgentEnsemble is open-source under the MIT license.

Quality Gates on Agent Pipelines: Phase Review and Feedback Injection in Java

mgd43b — Sun, 12 Apr 2026 14:00:00 +0000

Most agent pipelines treat quality as a post-run concern. The pipeline runs, you look at the output, you decide if it's acceptable. If not, you re-run the whole thing or manually patch the result. That approach gets harder to sustain as pipelines grow in complexity and the cost of a bad output increases.

The question worth asking is: where in the pipeline should quality enforcement sit? And when a phase produces inadequate output, how should the feedback get back to the tasks responsible?

PhaseReview answers both questions. It attaches a quality gate to any phase, fires after the phase completes, and based on the reviewer's decision either approves the output, triggers a retry with injected feedback, pushes the work back to a predecessor phase, or rejects the pipeline entirely. The review task is itself a first-class AgentEnsemble task -- AI-backed, deterministic handler, or human-in-the-loop -- and the framework handles feedback injection and retry orchestration automatically.

Attaching a Review to a Phase

The minimal setup attaches a reviewer task to a phase via PhaseReview.of(reviewTask):

Task draftReport = Task.builder()
    .description("draft a summary report of the quarterly results")
    .chatModel(model)
    .build();

Task reviewReport = Task.builder()
    .description("""
        Review the report for completeness and accuracy.
        Output exactly one of:
        APPROVE
        RETRY:<specific feedback for the author>
        REJECT:<reason>
        """)
    .chatModel(model)
    .context(List.of(draftReport))
    .build();

Phase reporting = Phase.builder()
    .name("reporting")
    .tasks(List.of(draftReport))
    .review(PhaseReview.of(reviewReport))
    .build();

Ensemble.builder()
    .phases(List.of(reporting))
    .chatModel(model)
    .build()
    .run();

The review task reads the draft output via .context(List.of(draftReport)). After the phase tasks complete, the framework runs the reviewer, parses its decision, and acts on it. draftReport is retried with feedback. The reviewer never appears in the phase task list -- it runs as a gate after the phase completes.

The Four Decisions

A review task produces one of four outcomes:

Decision	Behavior
`APPROVE`	Phase output is accepted; pipeline continues
`RETRY:<feedback>`	Phase tasks are re-run; feedback injected into prompts
`RETRY_PREDECESSOR:<feedback>`	Predecessor phase re-runs first, then this phase re-runs
`REJECT:<reason>`	Pipeline fails with a `PhaseReviewRejectionException`

The APPROVE/RETRY/REJECT parsing is handled by the framework. RETRY_PREDECESSOR is useful when the inadequate output in the current phase is caused by incomplete or incorrect work in an earlier phase.

Feedback Injection

When a retry is triggered, the framework injects reviewer feedback directly into the prompt of the tasks being retried. The injected section looks like this:

[original task instructions here]

## Revision Instructions (Attempt 2)

The report is missing Q3 comparisons and does not address margin compression. Expand the analysis section with specific numbers.

Previous output:
[output from attempt 1]

The task sees the original instructions, the feedback, and its prior output. No changes to the task definition are required. The injection happens entirely in the prompt construction layer.

For attempt 3 and beyond, the section header updates to Attempt 3, and both the latest feedback and the most recent prior output are included.

Reviewer Types

AI Reviewer

The reviewer task is an AI-backed agent that reads the phase output via context() and generates a decision:

Task aiReviewer = Task.builder()
    .description("""
        You are a quality reviewer for financial reports.
        Read the draft report provided in context.
        Check for: completeness, accurate numbers, professional tone.
        Output exactly: APPROVE, RETRY:<specific instructions>, or REJECT:<reason>.
        """)
    .chatModel(model)
    .context(List.of(draftReport))
    .build();

The reviewer's LLM call is separate from the main phase tasks. You can use a different model for the reviewer if appropriate.

Deterministic Reviewer

For rule-based quality checks, a deterministic handler reads the phase output and returns a decision string:

Task qualityCheck = Task.builder()
    .description("programmatic-quality-check")
    .context(List.of(draftReport))
    .handler(ctx -> {
        String output = ctx.contextOutputs().get(0).getRaw();
        if (output.length() < 500) {
            return ToolResult.success("RETRY: output is too short -- expand each section to at least two paragraphs");
        }
        if (!output.contains("Q3") || !output.contains("Q4")) {
            return ToolResult.success("RETRY: report must include both Q3 and Q4 data");
        }
        return ToolResult.success("APPROVE");
    })
    .build();

Deterministic reviewers are useful when the quality criteria are precise and don't require LLM judgment.

Human Reviewer

For steps that need a human sign-off, the human review API blocks until the reviewer responds:

Task humanGate = Task.builder()
    .description("human-review-gate")
    .context(List.of(draftReport))
    .review(Review.required())
    .build();

The human reviewer sees the task output and enters APPROVE, RETRY with feedback, or REJECT in the console (or a custom reviewer UI). This integrates with the same retry loop.

Controlling Retry Limits

By default, PhaseReview.of(reviewTask) allows up to 2 self-retries and 2 predecessor retries. Both are configurable:

Phase reporting = Phase.builder()
    .name("reporting")
    .tasks(List.of(draftReport))
    .review(PhaseReview.of(reviewReport, 3))
    .build();

Or with the builder for full control:

Phase reporting = Phase.builder()
    .name("reporting")
    .tasks(List.of(draftReport))
    .review(PhaseReview.builder()
        .task(reviewReport)
        .maxRetries(3)
        .maxPredecessorRetries(2)
        .build())
    .build();

When the retry limit is exhausted, the framework treats the phase as failed and throws. The pipeline does not silently accept a low-quality output when retries are exhausted.

Predecessor Retry

When the reviewer determines that the current phase's inadequate output is caused by problems in an earlier phase, it can request a predecessor retry:

Task analysisReviewer = Task.builder()
    .description("""
        Review the analysis. If the data from the ingestion phase appears incomplete or incorrect,
        output: RETRY_PREDECESSOR:<what needs to be fixed in data ingestion>
        If the analysis itself is the problem, output: RETRY:<what needs to be revised>
        If acceptable, output: APPROVE
        """)
    .chatModel(model)
    .context(List.of(analysisTask))
    .build();

Phase ingestion = Phase.builder()
    .name("ingestion")
    .tasks(List.of(ingestTask))
    .build();

Phase analysis = Phase.builder()
    .name("analysis")
    .tasks(List.of(analysisTask))
    .after(ingestion)
    .review(PhaseReview.of(analysisReviewer))
    .build();

If the reviewer outputs RETRY_PREDECESSOR:, the framework re-runs the ingestion phase with the feedback injected into ingestion tasks, then re-runs the analysis phase. The review fires again after the second analysis completes.

The predecessor is the phase declared in .after(). Predecessor retry does not cascade further back automatically.

Accessing Review Results

After the ensemble completes, review results are available in the phase output:

EnsembleOutput result = ensemble.run();

PhaseOutput reportingOut = result.getPhaseOutputs().get("reporting");
ReviewRecord reviewRecord = reportingOut.reviewRecord();

System.out.println("Attempts: " + reviewRecord.attemptCount());
System.out.println("Decision: " + reviewRecord.finalDecision());

This is useful for logging, auditing, or downstream branching based on whether the output was approved on the first attempt or required iterations.

Tradeoffs

Review tasks add LLM calls to the pipeline. For AI reviewers, each retry cycle adds a reviewer call plus the retried task calls. In cost-sensitive pipelines, deterministic reviewers can enforce the most common criteria without adding model calls.

Feedback injection is prompt-based. Tasks see the feedback as text in their prompt, not as a structured signal. The quality of the retry depends on how well the reviewer communicates what needs to change.

Predecessor retry re-runs the entire predecessor phase, not individual tasks. If the predecessor phase is expensive, predecessor retries can be costly. Design predecessor phases with this in mind if predecessor retry is expected.

The review task reads phase outputs via context() declarations -- the same mechanism any task uses. Context resolution works across retries; the framework rebuilds task identity consistently so the reviewer always reads the most recent attempt's output.

The guide at agentensemble.net/guides/phase-review/ covers the full API including custom reviewer implementations and review event callbacks. The example source is runnable from the repository.

AgentEnsemble is open-source under the MIT license.