DEV Community: Jayakumar Ramalingam

Event-driven AI agents: Build multi-agent workflows that survive production failures

Jayakumar Ramalingam — Mon, 17 Aug 2026 01:44:41 +0000

AI agents become fragile when they are connected in long synchronous chains. An event bus lets them work independently, wait for people and tools, recover after restarts, and place policy between a model's recommendation and a real action.

Most AI agent demos fit inside a single request:

User -> Agent -> Tool -> Agent -> Response

The agent makes a plan, calls a tool, gets an answer, and returns a response. It is easy to understand and easy to demo.

Then the same agent meets a real workflow.

It needs data from several systems. One tool takes five minutes. Another agent has to review the result. A production change needs human approval. While the approval is pending, one of the services restarts.

The model is no longer the hardest part. Coordination is.

At that point, adding a better prompt will not fix the system. The agents need a way to work independently, survive failures, and resume after the original request has ended. That is an event-driven architecture problem.

Synchronous chains work until somebody has to wait

Imagine an operations agent investigating a slow checkout service. It needs to collect metrics, inspect dependencies, run a diagnostic job, propose a recovery action, wait for approval, execute the change, and verify recovery.

With direct calls, each component knows what comes next. The first agent waits for the second. The second waits for a tool. The request stays open while a person decides whether to approve the change.

This works when every step is fast and available. In production, that assumption does not last long.

A timeout may leave a tool running after its caller has given up. A retry may execute the same action twice. A restart can erase the current plan. Adding a security review means changing an integration that was already working.

Human approval exposes the problem most clearly. A decision may take minutes or hours. Holding an HTTP request open across that wait is a poor way to preserve a workflow.

Replace direct calls with facts

In an event-driven design, an agent publishes what happened. Other components decide whether that fact matters to them.

ServiceLatencyIncreased
        |
        v
DiagnosisRequested
        |
        v
DependencyFailureSuspected
        |
        v
RecoveryProposed
        |
        v
HumanApprovalRequired
        |
        v
RecoveryApproved
        |
        v
RecoveryCompleted

The service health agent can publish DiagnosisRequested without knowing which diagnostic agent will handle it. The diagnostic agent can publish DependencyFailureSuspected without calling the remediation agent directly.

An audit service, an observability pipeline, and a security agent can all react to the same event. If one consumer is temporarily unavailable, it can process the event after it recovers, subject to the broker's retention settings.

This is the useful part of an event bus. Components participate in the workflow without being wired directly to one another. It is the same producer and consumer separation described in AWS Prescriptive Guidance for event-driven AI, but the pattern is not tied to any cloud or broker.

An event is not a command

Agent systems get dangerous when every message is treated as interchangeable.

An event records a fact:

{
  "type": "ServiceLatencyIncreased",
  "eventId": "evt-204-01",
  "incidentId": "incident-204",
  "service": "checkout-api",
  "p95LatencyMs": 1840
}

The exact schema is up to the system. When events cross team or platform boundaries, the vendor-neutral CloudEvents specification provides a common envelope for event metadata.

A model proposal describes the action it wants to take. It does not choose the command ID or the idempotency boundary:

{
  "action": "ScaleService",
  "incidentId": "incident-204",
  "resource": "checkout-api",
  "targetState": {
    "replicas": 12
  }
}

An agent decision is a recommendation. It can include evidence and confidence, but it is still a proposal.

These distinctions matter. An LLM saying "scale the service" does not mean the service was scaled. It also does not mean the action was authorized.

A safer path is:

Event -> Agent proposal -> Policy check or human approval
      -> AuthorizedCommand -> Deterministic idempotency key
      -> Execution -> Outcome event

The agent interprets the situation and proposes an action. A policy layer checks permissions, limits, and approval requirements. When the action is allowed, the policy layer returns a canonical AuthorizedCommand.

That separation remains useful even when the model changes. The model can become more capable without quietly gaining permission to restart production or issue a refund.

Agent memory is not workflow state

Agent platforms talk a lot about memory. Conversational memory and execution state solve different problems.

Memory may contain a conversation summary, user preferences, or retrieved knowledge. Workflow state answers operational questions:

Which steps finished?
Which command is waiting for approval?
Was this event already processed?
What should resume after a restart?

A conversation transcript is a fragile place to store those answers. It may be summarized, truncated, or interpreted differently after a model update. If the only record that a refund was issued is a sentence in a context window, the system may eventually issue it again.

Store workflow state explicitly. Keep event IDs, command status, approvals, retry counts, and execution results in durable storage. Give the model only the context it needs for its current decision.

The broker will not make the system reliable for you

Moving work onto an event bus changes the failure modes. It does not remove them.

Duplicate events

Design consumers so they can see the same message more than once. Track stable event IDs, and require idempotency keys for tools that change state. Restarting a service twice or issuing the same refund twice is not a retry strategy.

Events arriving out of order

An approval may arrive after a proposal has expired. A recovery result may appear after a newer action has started. Put versions and timestamps on messages, and reject transitions that no longer match the current workflow state.

Agent feedback loops

Agent A publishes an event that wakes Agent B. Agent B responds with an event that wakes Agent A. Both agents can keep generating messages and spending tokens without completing useful work.

Track causation depth. Set limits for time, tokens, and workflow steps. Route repeated failures to human review.

Conflicting decisions

Two agents may propose opposite actions for the same resource. Do not allow both commands to run. Serialize changes for that resource or use a version check before execution.

The consumer itself should be boring. That is a compliment:

import hashlib
import json


def derive_idempotency_key(command):
    material = {
        "workflowId": command.workflow_id,
        "workflowVersion": command.workflow_version,
        "action": command.action,
        "resource": command.resource,
        "targetState": command.target_state,
    }
    canonical = json.dumps(material, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(canonical.encode()).hexdigest()


def handle(event, store, agent, policy, executor):
    if store.already_processed(event.id):
        return

    workflow = store.load_workflow(event.incident_id)

    if not workflow.accepts(event.type, event.version):
        store.mark_processed(event.id, outcome="stale")
        return

    proposal = agent.propose(workflow.context_for(event))
    verdict = policy.evaluate(proposal, workflow)

    if verdict.requires_human:
        store.request_approval(workflow, proposal)
        return

    if not verdict.allowed:
        store.record_denied(workflow, proposal, verdict.reason)
        return

    authorized = verdict.authorized_command
    idempotency_key = derive_idempotency_key(authorized)

    result = executor.run(
        authorized,
        idempotency_key=idempotency_key,
    )

    store.append_event_once(
        workflow,
        result.to_event(),
        dedupe_key=idempotency_key,
    )
    store.mark_processed(event.id, outcome=result.status)

The executor claims the idempotency key before it changes the external system. If the key already exists, it returns the stored result with a replayed status instead of running the action again. Otherwise, it applies the action and returns applied.

This does not give an arbitrary external API exactly once behavior. The tool must support idempotency, or the executor needs a way to reconcile an outcome when it cannot tell whether the action completed. The model can handle the ambiguous reasoning, while policy and execution remain deterministic and testable.

Not every agent needs an event bus

A document summarizer does not need a distributed control plane. A synchronous request is usually enough when one caller expects an immediate answer, the task finishes quickly, and a retry cannot cause harm.

Events begin to earn their cost when work continues past a request timeout, several agents act independently, a human interrupts the workflow, or an action has operational or financial consequences.

There is a price. Teams have to manage message schemas, retention, tracing, failed-message handling, and asynchronous debugging. Use an event-driven design when those costs buy reliability, not because the architecture sounds more sophisticated.

Start with the step that blocks

You do not need to rebuild a synchronous agent system all at once.

Find the step that blocks the longest. It is often a human approval or a slow external tool. Give that step a durable state record and an event that can resume the workflow. Then add an idempotency key to every command that changes something outside the agent system.

Take one agent workflow you already run and draw the waits, retries, approvals, and side effects. Where those boundaries are hidden inside a single synchronous chain, that is where the first event belongs.

Better models will improve agent decisions. They will not recover a lost approval, prevent a duplicate command, or resume a half-finished workflow. Once agents work across services and over time, they inherit distributed-system failures.

Failures still happen in an event-driven design. Durable workflow boundaries make it possible to stop safely and resume later.

Claude Managed Agents - How to Build a GitHub Repo Review Agent

Jayakumar Ramalingam — Mon, 20 Apr 2026 17:01:53 +0000

Creating an AI agent almost always requires much more than crafting a prompt. A model, infrastructure, execution logic, runtime, and hosting are typically required. That is where Claude Managed Agents transform the game.

By leaving the responsibility of the orchestration layer up to the Claude team, you can define the agent’s behavior by crafting prompts, providing tools, and establishing constraints. Then the Anthropic managed cloud runtime will take care of the execution session, execution environment, and many other aspects of the orchestration.

As an example, we’re going to create a GitHub Repository Review Agent that will check out a given repository, analyze its source code and documentation, find any possible problems, and generate a useful Markdown report based on that analysis.

What is important here, is that this approach puts the focus on prompts and behavior rather than infrastructure.

Why Claude Managed Agents feels different

When using a typical agent architecture, there is always the need for you to implement or adapt an orchestration mechanism, integrate calls to various tools, deal with the runtime infrastructure, make decisions on where the agent will be hosted, handle concurrent execution, consider file handling, and establish streaming methods for delivering results to users.

Claude Managed Agents takes care of many of those aspects in its managed environment. Instead of spending most of your time building scaffolding, you spend more time defining:

what the agent should do
what tools it can use
what environment it should run in
what output you want it to produce

That is the real shift. It is still engineering, but it is a more prompt-first and behavior-first development model.

What we’re building

We’ll create a workflow that does the following:

mounts a GitHub repository into a managed session
inspects repository structure, docs, and code layout
identifies architecture smells and risk areas
writes a Markdown report with a prioritized 30-day action plan

Prerequisites

You need:

Python 3.10+
an Anthropic API key
the anthropic Python package
a GitHub token if you want to mount a private repository

Step 1: Install the SDK

pip install anthropic

Step 2: Create a managed environment

The environment is the managed cloud container template where your session runs. It is where networking rules, packages, and mounted resources are controlled.

Here is a simple environment:

import anthropic

client = anthropic.Anthropic()

environment = client.beta.environments.create(
    name="repo-review-env",
    config={
        "type": "cloud",
        "networking": {
            "type": "unrestricted"
        },
    },
)

print(environment.id)

For real usage, I would usually tighten the network rules instead of leaving them fully open.

A more controlled version might look like this:

environment = client.beta.environments.create(
    name="repo-review-env-locked-down",
    config={
        "type": "cloud",
        "networking": {
            "type": "limited",
            "allowed_hosts": [
                "api.github.com",
                "raw.githubusercontent.com"
            ],
        },
    },
)

Step 3: Create the Claude Managed Agent

This is where the managed model becomes much clearer. Instead of building the full runtime loop yourself, you mainly define the behavior you want.

agent = client.beta.agents.create(
    name="Repository Review Agent",
    model="claude-opus-4-7",  // any model like sonnet etc
    system="""
You are a senior software architect performing an initial engineering review.

Your job:
1. Inspect the repository structure, documentation, and key code paths.
2. Identify architecture smells, risky areas, missing docs, and maintainability issues.
3. Distinguish between critical issues, medium-priority improvements, and optional cleanup.
4. Create a file at /mnt/session/outputs/review_report.md.

The report must contain:
- Executive summary
- Repository structure overview
- Risk areas
- Documentation gaps
- Suggested refactors
- A prioritized 30-day action plan

Be practical. Do not invent facts. Base findings only on the repository contents.
""",
    tools=[
        {
            "type": "agent_toolset_20260401",
            "default_config": {"enabled": True}
        }
    ],
)

print(agent.id, agent.version)

This is one of the biggest differences with Claude Managed Agents. You define the model, prompt, tools, and expected behavior, and the platform handles the managed runtime side.

Step 4: Start a session and mount a repository

Now we start a session and mount a GitHub repository into it.

import os

session = client.beta.sessions.create(
    agent={"type": "agent", "id": agent.id, "version": agent.version},
    environment_id=environment.id,
    title="Initial architecture review",
    resources=[
        {
            "type": "github_repository",
            "url": "https://github.com/owner/repo",
            "mount_path": "/workspace/repo",
            "authorization_token": os.environ["GITHUB_TOKEN"],
            "branch": "main",
        }
    ],
)

print(session.id, session.status)

At this point, the main infrastructure concern is no longer “where do I deploy this agent?” The session is already running inside the managed environment.

Step 5: Send the prompt and stream the results

Next, we stream the session and send the task.

with client.beta.sessions.stream(session_id=session.id) as stream:
    client.beta.sessions.events.send(
        session_id=session.id,
        events=[
            {
                "type": "user.message",
                "content": [
                    {
                        "type": "text",
                        "text": """
Review the mounted repository at /workspace/repo.

Focus on:
- architecture and module boundaries
- signs of tight coupling
- test coverage clues
- configuration and secrets handling
- developer experience issues
- the highest-value improvements for the next 30 days

Write the final report to /mnt/session/outputs/review_report.md
"""
                    }
                ],
            }
        ],
    )

    for event in stream:
        if event.type == "agent.message":
            for block in event.content:
                if block.type == "text":
                    print(block.text, end="", flush=True)

        elif event.type == "session.status_idle":
            print("\n--- Agent finished ---")
            break

        elif event.type == "session.status_terminated":
            print("\n--- Session terminated ---")
            break

This is where the workflow starts to feel different from a typical DIY agent setup. You are not building a custom orchestration loop with your own runtime machinery. You are giving the Claude Managed Agent a task and letting the managed system execute it.

Step 6: Download the generated report

Once the session is done, we can download the Markdown file the agent created.

import time

time.sleep(2)

files = client.beta.files.list(
    scope_id=session.id,
    betas=["managed-agents-2026-04-01"],
)

for f in files.data:
    print(f.filename, f.size_bytes)
    if f.filename == "review_report.md":
        content = client.beta.files.download(f.id)
        content.write_to_file("review_report.md")
        print("Saved review_report.md locally")

Now you have a report generated by the agent inside the managed environment.

Full example

Why this matters

When using Claude Managed Agents, you will still need to design your agent with all its considerations related to prompts, tools, capabilities, and outputs. However, there is no longer as much work in the runtime layer as before.

This is the change in focus:

less time spent on setting up the runtime environment
less concern regarding the necessary infrastructure
less trouble trying to manage the scalability of your own agent loop
more attention to getting the right behavior out of the box

Production note

In a production system, I would not initialize the environment and agent on each request. This should be done only once, then the same object should be used, and both should be carefully versioned with prompt and network access minimized.

I would also use a secure application layer in front of my workflow and not expose it directly.

"Managed" does not mean there is no architecture here. It only means it is raised to a new level.

Final thoughts

Claude Managed Agents are intriguing because they shift the developer's focus from implementation details to the behavior and outcome.

While you could spend a lot of time building the runtime for your own purposes, Managed Agents offer an easy way to focus solely on modeling what you are looking for: the model, prompt, tools, and the environment – with everything else taken care of by a managed system that runs your agent inside.