DEV Community: Amar Tinawi

ZOdyssey: making the plan-review gate a hard hook, not a prompt convention

Amar Tinawi — Mon, 10 Aug 2026 12:57:01 +0000

ZOdyssey: making the plan-review gate a hard hook, not a prompt convention

What if the "review the plan before executing" step in your agent pipeline was a hard gate, not a suggestion?

That is the question I kept coming back to after another session of watching an agent do the thing. You know the thing. The plan looked fine. The review step was in the prompt. The model nodded, said "looks good," and then immediately started editing files before anyone had actually approved anything — because nothing was actually stopping it. The review was a convention. Conventions are suggestions. Suggestions are not physics.

This post is about a small open-source project I just shipped called ZOdyssey. It is a multi-agent orchestration pipeline whose one architectural commitment is that the load-bearing invariants are enforced with code, not with prompts. I want to tell you why I built it, how it works, and — because I have read too many launch posts that hand-wave the limitations — exactly what it is not.

The problem, in four failure modes

If you have spent any real time driving coding agents on non-trivial tasks, you have hit all four of these. They are not exotic. They are the boring, recurring ways agent runs go sideways:

The model edits code before the plan is reviewed. You wrote "consult, then plan, then review, then execute" into the prompt. The model agreed to the order. Then a sub-task got exciting and an edit landed before the review step had actually returned a verdict. The plan was a draft; the code is now real.
The model over-engineers. You asked for a one-line fix. You got 50 spawned subagents, three refactors, a new abstraction, and a changelog. Parallelism is a superpower until it is a tax.
The model drifts off-scope. The plan said "edit auth.ts." The executor helpfully also touched session.ts, middleware.ts, and the README, because they were "related." Now your diff review is the scope review, after the fact.
A crashed run starts over from scratch. The agent got 40 minutes into a six-todo plan, hit a transient failure, and there is no checkpoint. You re-run. It re-plans. It re-edits. State is gone.

All four of these are deterministic. They are not about model intelligence. They are about invariants — properties of the run that should hold regardless of how clever or compliant the model feels today. And the dirty secret of most orchestrators (including ones I admire) is that they enforce these invariants the same way: by putting a sentence in the system prompt and hoping.

Prompts catch these failures most of the time. The model is usually cooperative. "Most of the time" is a rough profile when the failure is "unreviewed code landed in main."

The aha

The shift is embarrassingly small once you see it: enforce the gate with a PreToolUse hook, not a prompt convention.

A hook is not advice. It is a function that runs before a tool call is allowed to proceed, and it can return block. The model cannot argue with it, cannot "decide" to skip it, cannot get clever and route around it between tool calls in a single turn. If the hook says "no edit until state.review.verdict === "OKAY"," then no edit happens until that field is set. The model can write a brilliant argument for why it should be allowed to edit early. The hook does not read arguments. It reads state.

That is the entire delta. The pipeline shape — prime, triage, consult, plan, review, execute, verify, final wave — is the same shape omo and others already use, and ZOdyssey is openly built on that lineage. The cast of sub-agents (a consult agent, a planner, a reviewer, executors) is borrowed too. What ZOdyssey adds is the enforcement layer: the four invariants below are checked in code, on every relevant tool call, for the entire duration of a run.

The framing I keep coming back to is this: prompts guide choices; code enforces invariants. Use prompts for the stuff that is genuinely a judgment call (which skill to reach for, how to phrase the plan, when to ask the user). Use hooks for the stuff that must never be a judgment call (did the plan pass review, is this file in scope, are we over the parallel cap).

How it works

ZOdyssey runs an eight-phase state machine. The conductor (your main agent) drives it; sub-agents do the work; the hooks guard the invariants. Every phase transition checkpoints to a state.json file so a crashed run resumes instead of restarting.

The pipeline

Phase	Name	One-line job
−1	PRIME	A `prompt-master` pass refines your raw task into a sharp brief: intent, success criteria, surfaced constraints, ambiguities (ask up to 3, then commit), and a rewritten prompt that replaces the original. Runs first, before triage.
0	TRIAGE	The conductor does this directly. Trivial task → just answer and stop. Standard → single-track. Architecture-changing → full pipeline.
1	CONSULT	A `metis` agent reads prior learnings from a memory store, then returns intent classification, risks, questions, and directives. If it has user-facing questions, it surfaces them and waits.
2	PLAN	A `prometheus` agent writes the plan to `<repo>/.zcode/plans/<slug>.md`, one todo per block, each with `Files:`, `References:`, and executable acceptance criteria. It cannot edit product code — the hook blocks it.
3	REVIEW	A `momus` agent returns `OKAY` or `REJECT` with up to three blockers. This is the enforced gate. REJECT below the round cap (3) → re-plan and re-review. At the cap → stop and surface to the user. No unbounded loop.
4	EXECUTE	The conductor dispatches a `sisyphus-junior` per todo, parallel-by-default, scope-locked to that todo's declared `Files:`. On each return: tick the checkbox, write a checkpoint, update state.
5	VERIFY	The conductor runs each todo's acceptance commands itself. On failure, it re-dispatches with the error output attached.
6	FINAL WAVE	Independent passes against the full diff: F1 plan-compliance, F2 code-quality, F3 manual-QA, F4 scope-fidelity. All four must pass before the run is "done."

The four enforced invariants

These are the delta. They are all implemented as PreToolUse hooks in ~/.zcode/cli/config.json, and they are all no-ops unless an orchestration run is active — normal editing in your repo is never affected.

Review gate (the big one). Every edit-class tool call is intercepted. Is a run active? Is this path bookkeeping (.zcode/)? Has review returned OKAY? If not, block: "edits blocked until plan passes review." There is no override flag the model can set.
Scope-isolation boundary. The target file must be in the union of Files: declared in the plan. It fails closed if the plan is unreadable or empty — which is the fix for the real-world scope-creep failure where an executor widens its own scope by editing the plan after review. (The hook re-hashes the plan against the sha bound to the OKAY verdict, so post-review tampering does not silently widen scope.)
File-lock ledger. A per-file lock map. If another in-flight todo holds the lock for a path, the second edit blocks. Locks release when the todo is marked done (or get reaped by TTL). This is what makes parallel execution safe without the orchestrator having to reason about it.
Parallel cap. The dispatch tool (Task/Agent) is gated. The hook counts in-flight dispatches and blocks at the cap — default four. The model cannot bump state between tool calls in one turn, so the hook owns this counter; the orchestrator literally cannot over-spawn past the cap even if it tries.

There is a fifth, optional hook — a Bash write-gate that treats write-capable shell invocations (sed -i, redirects, git apply, …) the same way as direct edits. It is on by default and can be disabled with ZODYSSEY_UNGATE_BASH=1 when you want lower friction. The four above are the core delta.

The cast, and why the conductor matters

The sub-agents are narrow on purpose. metis consults. prometheus plans (and is one of only two agents that can write anything). momus reviews. sisyphus-junior executes — the only other writer. explore, librarian, and oracle are read-only research and advice. Each does one thing, returns a structured verdict, and gets out of the way.

The interesting part is not the cast, though — it is the capability routing. The orchestrator is the thing that knows to reach for the right tool before doing the activity the generic way. Logic implementation gets routed to a test-driven-development skill (non-negotiable for code todos). Hard multi-step reasoning gets sequential thinking. Codebase questions get codegraph_explore if there is a .codegraph/ index, else a dispatched explore agent. Library questions get Context7 plus a librarian. After two failed debugging attempts, the orchestrator asks an oracle for a fresh diagnosis instead of flailing. The point is that the conductor tells every dispatched agent which capability to use, rather than assuming a fresh sub-agent will reach for the right one on its own.

Two more things worth knowing

Checkpoint and resume. Every phase transition and every completed todo writes a checkpoint. /orchestrate resume <slug> reads the last checkpoint and picks up there, not from scratch. Durable execution was a hard requirement — a six-todo plan that dies on todo four should not cost you the first three.

An optional independent audit. After a run reaches done, /orchestrate-consult <slug> hands the plan plus the full git diff to a separate CLI process — fresh context, independent model — for an ACCEPT/REJECT audit. Because that auditor cannot inherit the run's assumptions, it catches things in-session reviewers miss. On REJECT, ZOdyssey re-arms the gates and loops until ACCEPT. (Honest caveat: the remediation loop runs after a terminal phase, so the gates are only re-armed because ZOdyssey explicitly flips state back to a remediate phase — the doc-vs-code gap here is itself a tracked item, and I would rather tell you that than pretend the cap is magic.)

Quick start

The reference implementation targets ZCode. If you are on ZCode, this is the whole install — zero npm dependencies, all scripts are ESM .mjs using only Node built-ins:

git clone https://github.com/amartinawi/zodyssey.git
cd zodyssey
node scripts/install.mjs            # copies into ~/.zcode/, registers hooks + MCPs
node scripts/install.mjs --verify   # health-check: hooks parse, MCP backends resolvable

Then start a new session and, in any repo, run:

/orchestrate <your task>

That is it. The installer also registers the pipeline MCPs (memory, sequential-thinking, codegraph, chrome-devtools, the model server) — each gated on its backend being on PATH, skipped with a hint if not. Full install, troubleshooting, and config live in docs/INSTALL.md.

Not on ZCode? The pattern is portable. Read docs/ADAPT.md — it is a concrete guide to bolting the four enforcement hooks onto omo, Claude Code, Cursor, or any harness that can run a PreToolUse hook. If you are already an omo user, start there: omo gives you the full pipeline and ergonomics, and you layer on the four hooks. That is the highest-leverage path for most people.

What it is NOT

I want this section to read as engineering integrity, not as an apology. The honest scope of v1 is narrower than the architecture implies, and you should know that before you install it.

It is not a replacement for normal agent operation. ZOdyssey is an opt-in mode you enter with /orchestrate. Everything else — quick questions, one-shot edits, the 95% of interactions that do not need a pipeline — is handled directly, the way you already work. The hooks are no-ops unless a run is active.
It is not multi-model in v1. There is a category routing field designed into the state machine (see docs/DESIGN.md), but in v1 routing reduces to effort and variant selection within one connected model. Wiring a second provider is on the roadmap, not in the binary.
It is not harness-agnostic in v1. The reference implementation targets ZCode; that is where the hooks, commands, and sub-agents are native. The pattern is portable — that is what docs/ADAPT.md is for — but I am not going to claim it drops into Claude Code or Cursor unchanged, because it does not.
It is not a team-mode orchestrator yet. Parallel multi-executor with mailboxes and git worktrees is designed but deferred to v2. v1 is single-executor-per-todo, dispatched in parallel waves under the cap. If you wanted multiple human collaborators in one run, that is not this version.

If any of those four are disqualifying for your use case, that is a perfectly good reason not to use it yet. I would rather you know now than find out mid-run.

What is next

The roadmap is the parts of the architecture that are designed for, not yet wired: multi-model routing (the category field becomes a real provider switch), team mode (parallel executors across mailboxes and worktrees), and broader harness support so the ADAPT path gets shorter. The enforcement pattern itself is stable — the four hooks are the load-bearing idea, and they are done. What changes next is what runs inside the gate, not the gate.

Provenance

ZOdyssey is a synthesis, not an invention, and I want the lineage visible because it is the honest framing. The pipeline shape and the sub-agent cast are modeled on omo — if you are not familiar, go read it; it is the cleaner expression of the orchestration pattern. The enforcement layer (the four hooks) is the differentiator, not a derivative. The thinking on multi-agent systems is grounded in Anthropic's multi-agent research post and the Building Effective Agents essay, with additional citations to LangChain's multi-agent architecture analysis and an arXiv context-engineering paper in docs/DESIGN.md. The routed skills (TDD, systematic debugging, brainstorming) come from obra/superpowers; the impact-analysis step uses codegraph. Full citations are in DESIGN.md §0 and §15.

It is MIT licensed. Take it, adapt it, use it. If the enforcement-gate pattern makes your orchestrator more reliable, that is the whole point.

The repo is github.com/amartinawi/zodyssey. The two things to read first are docs/DESIGN.md (the principle, load-bearing decisions, and research) and docs/ADAPT.md (how to bolt the delta onto whatever you are already running).

Multi-Tenant Agentic AI on AWS: Isolation and Cost Architecture

Amar Tinawi — Fri, 07 Aug 2026 14:10:24 +0000

Cross-posted from iqraa.tech — the full guide covers siloed vs pooled vs hybrid deployment, per-tenant cost attribution on Bedrock, and the noisy-neighbor problem for agent workloads.

Multi-tenant agents turn one agentic AI system into a SaaS serving many customers from shared infrastructure. Siloed, pooled, or hybrid deployment determines your unit economics and security.

Multi-Tenant Agents: What You’ll Learn

This guide maps the AWS-recommended deployment models, tenant-context propagation patterns, and isolation primitives across siloed, pooled, and hybrid topologies on Bedrock AgentCore.

By the end you should be able to justify a siloed, pooled, or hybrid topology for a given tenant mix, and trace a tenant identifier through every hop of a request from JWT to memory write.

You should also be able to explain the layered isolation controls (IAM, KMS, microVM, MCP credentials) that a security review will ask about. The goal is a mental checklist for any new tenant-facing agent feature, not one fixed blueprint.

Multi-Tenant Agents: The Core Challenge

Unlike a stateless web request, an agent invocation carries tenant-specific memory, tenant-scoped tools, and tenant-aware prompts. Every layer (API entry, LLM call, tool execution) must know which tenant it serves.

Three forces shape every architecture: isolation (tenant A never sees tenant B’s data), context (the agent knows which tenant is requesting), and attribution (every token is billable to the triggering tenant). No universally best pattern, only the best fit for your workload and regulatory environment.

The reason agents are harder to isolate than plain web apps is that an LLM call is not a single, auditable database query. It is a chain of tool invocations, memory reads, and prompt assemblies, and any one of those hops can silently pull in the wrong tenant’s data if the identifier is dropped.

A missing tenant_id at the tool layer does not throw an error; it just returns whatever the tool finds, which makes these bugs more dangerous because they fail silently instead of loudly.

Siloed vs Pooled Agent Deployment

The first and most consequential decision: a separate agent instance per tenant (siloed), or one shared instance with tenant context injected at runtime (pooled)?

This decision cascades into almost every later architecture choice: how you provision infrastructure, how you bill, how you patch, and how a security auditor will scope their review. Getting it wrong early is expensive to reverse because migrating live tenants between topologies means rebuilding the isolation boundary underneath production traffic, so it deserves more upfront analysis than most teams give it.

Dimension	Siloed	Pooled
Isolation	Strong (no shared state)	Weak (relies on context scoping)
Per-tenant cost	High	Low
Scaling	Linear (N = N instances)	Sublinear (1 serves all)
Customization	Trivial	Needs conditional logic
Upgrade velocity	Slow	Fast
Operational toil	High	Low
Best fit	Enterprise, regulated	Consumer, PLG

A hybrid model bridges them: most tenants share a pooled agent, but high-tier or regulated tenants get dedicated siloed instances, routed at API Gateway by the JWT tier claim.

When siloed wins

Siloed is right for regulated industries requiring per-tenant compute isolation, enterprise contracts with per-tenant fine-tuning or knowledge bases, and B2B SaaS with few high-value tenants where the cost premium is trivial relative to revenue.

The tell-tale sign you need siloed is a customer contract clause that names a specific isolation guarantee (dedicated compute, a named encryption key, or a right to audit the runtime) because those clauses are much easier to satisfy with a physically separate agent instance than with logical scoping inside a shared one.

When pooled wins

Pooled dominates consumer and PLG segments where a dedicated instance per tenant would sink the idle-cost baseline. It also wins when tenant customization is uniform. A new tenant is just a registry row, not a provisioning project.

Idle cost is the deciding variable: a siloed agent instance still consumes baseline compute even when a tenant sends zero requests overnight, and at thousands of low-usage tenants that idle cost dwarfs the actual inference spend. Pooled collapses that baseline to near zero because the shared instance is already running for other tenants’ traffic.

When hybrid wins

Hybrid is the 2026 default for enterprise SaaS: free tiers pooled, enterprise tiers siloed. The cost is running two topologies in parallel; the upside is matching isolation to willingness-to-pay. Most Bedrock AgentCore agents ship hybrid from day one.

Running two topologies means maintaining two deployment pipelines, two sets of IAM policies, and two on-call runbooks, which is real operational overhead. Teams that pick hybrid successfully treat the pooled and siloed paths as the same codebase with a routing decision at the edge, not two forked implementations that drift apart over time.

Three deployment models: siloed, pooled, hybrid.

Injecting Tenant Context into Agents

How does a pooled agent know which tenant is requesting? Tenant context injection attaches a tenant identifier to every step of execution. Done right, the agent behaves as if per-tenant. Done wrong, you ship a data leak. The canonical AWS pattern:

1. Caller authenticates → Cognito / OIDC issues a JWT
2. JWT carries a custom claim: { "tenant_id": "acme-corp" }
3. API Gateway authorizer validates JWT, extracts tenant_id
4. API Gateway forwards request to Lambda with tenant_id in header
5. Lambda reads tenant_id, scopes all DynamoDB queries with it
6. Lambda invokes Bedrock Agent with tenant_id in sessionMetadata
7. Bedrock Agent's tools receive tenant_id, scope their own queries
8. Memory store (Bedrock AgentCore Memory) keys sessions by tenant_id
9. Response returns to caller; no cross-tenant state leaked

The most common mistake is stopping at the API layer: validating tenant at API Gateway but forgetting to scope the DynamoDB query, S3 prefix, or Knowledge Base retrieval. Bedrock AgentCore exposes session metadata as a first-class concept: attach tenant_id once and it propagates to every tool call, memory write, and observability event. The Lambda authorizer that extracts tenant_id from the JWT:

Session metadata matters because it removes the temptation to pass tenant_id as an ordinary function argument that a future refactor could quietly drop. Once tenant_id lives on the session object, every downstream component (the tool executor, the memory writer, the trace exporter) reads it from the same place, so there is exactly one code path to audit instead of dozens of call sites that each need to remember to forward the value correctly.

import json, jwt  # PyJWT
TENANT_CLAIM = "custom:tenant_id"
JWT_AUDIENCE = "agent-api"
def lambda_handler(event, context):
    auth_header = event["headers"].get("authorization", "")
    token = auth_header.replace("Bearer ", "")
    try:
        # Verify signature + audience against Cognito JWKS
        decoded = jwt.decode(
            token,
            algorithms=["RS256"],
            audience=JWT_AUDIENCE,
            jwks_url="https://cognito-idp.us-east-1.amazonaws.com/"
                     + os.environ["USER_POOL_ID"] + "/.well-known/jwks.json",
        )
        tenant_id = decoded[TENANT_CLAIM]
        tier = decoded.get("custom:tier", "free")
    except jwt.PyJWTError as e:
        return {"principalId": "denied", "policyDocument": deny_policy()}
    # Allow + propagate tenant_id / tier to integration via context
    return {
        "principalId": tenant_id,
        "policyDocument": allow_policy(event["methodArn"]),
        "context": {"tenant_id": tenant_id, "tier": tier},
    }

Tenant context resolved at the perimeter, enforced at each interaction.

Tenant Onboarding and Lifecycle Management

Multi-tenant agents need a control plane separate from the application plane: the control plane handles onboarding, tiering changes, and offboarding; the application plane serves real-time requests. They communicate through a shared tenant registry (typically DynamoDB global tables) and use different IAM roles so a control-plane bug cannot serve tenant traffic.

A reference onboarding flow on Bedrock AgentCore:

import boto3
control_plane = boto3.client('events')  # EventBridge for lifecycle events
agentcore = boto3.client('bedrock-agentcore')
def onboard_tenant(tenant_id: str, tier: str, admin_email: str):
    # 1. Create tenant-scoped IAM role
    role_arn = iam.create_role(
        RoleName=f"agent-tenant-{tenant_id}",
        AssumeRolePolicyDocument=trust_policy_for_agentcore(),
        Description=f"Tenant {tenant_id} ({tier})",
    )['Role']['Arn']
    # 2. Provision tenant-scoped memory in AgentCore
    memory_id = agentcore.create_memory(
        name=f"mem-{tenant_id}",
        strategy='semantic',
        encryptionKeyArn=tenant_kms_key(tenant_id),
    )['memoryId']
    # 3. Register tenant in control-plane registry
    dynamodb.put_item(
        TableName='Tenants',
        Item={'tenant_id': {'S': tenant_id},
              'tier': {'S': tier},
              'role_arn': {'S': role_arn},
              'memory_id': {'S': memory_id},
              'status': {'S': 'ACTIVE'}},
    )
    # 4. Emit lifecycle event for downstream systems (billing, analytics)
    control_plane.put_events(Entries=[{
        'Source': 'agent.control-plane',
        'DetailType': 'TenantOnboarded',
        'Detail': json.dumps({'tenant_id': tenant_id, 'tier': tier}),
    }])
    return {'tenant_id': tenant_id, 'role_arn': role_arn, 'memory_id': memory_id}

Every resource is tenant-scoped by name (IAM role, memory ID, registry record), so cost attribution, audit, and offboarding become trivial prefix-matching operations.

The onboarding function above also illustrates why the control plane and application plane need separate IAM roles: the control plane needs iam:CreateRole and bedrock-agentcore:CreateMemory permissions to provision new tenants, but the application plane that actually serves chat requests should never hold those permissions. A compromised or buggy request handler with IAM-creation rights is a far bigger blast radius than one that can only read a tenant’s own memory and DynamoDB rows.

The control plane orchestrates tenant onboarding atomically.

Isolation, Security, and Data Ownership

A single cross-tenant leak is market-ending. AWS recommends a layered model. Identity: every tenant gets its own IAM role. Bedrock AgentCore‘s per-tenant session tokens stop even a misconfigured tool at the IAM boundary. Data: DynamoDB composite keys with tenant_id, S3 prefixes include tenant_id, Knowledge Bases per-tenant or partitioned. Compute: AgentCore’s microVM runtime gives each session a hardened, ephemeral environment.

Regulated workloads add per-tenant KMS customer-managed keys: each tenant’s data is encrypted with a key only that tenant can authorize. The MCP protocol adds another surface. A pooled agent calling an MCP server must pass tenant-scoped IAM credentials explicitly, never trusted to defaults.

Per-tenant KMS keys also give you a clean offboarding lever: scheduling a key for deletion cryptographically shreds every object encrypted under it, even if a stray copy of the data survives somewhere in a backup or log you forgot to purge. That is a much stronger guarantee than relying on a delete script to find and remove every row, because the guarantee holds even against your own mistakes.

An MCP client passes tenant-scoped IAM credentials to the MCP server.

Cost Attribution and Resource Management

Every Bedrock inference, AgentCore session minute, Lambda invocation, and DynamoDB read must attribute to a tenant. The cleanest pattern is tag-based attribution: tag every control-plane resource with Tenant={tenant_id} at creation, and Cost Explorer breaks spend per tenant automatically. For per-request costs (tokens, session minutes), the agent emits a telemetry event with tenant_id.

The noisy neighbor problem, one tenant’s burst saturating model concurrency or DynamoDB capacity, is the dark side of pooled. The fix is per-tenant token-bucket throttling at API Gateway or AgentCore Gateway. Tiered resource allocation (smaller model and shorter memory retention for free tier; larger model and dedicated capacity for premium) is config in the control-plane registry.

Token-bucket throttling works well here because it tolerates short bursts, a tenant running a legitimate batch job for a few seconds, while still capping the sustained rate that would otherwise starve every other tenant sharing the same model endpoint. Setting the bucket size and refill rate per tier, rather than globally, means a premium tenant’s burst allowance does not have to be sized down to protect free-tier capacity.

Tenant Tiering and Pricing Strategy

Three pricing models dominate multi-tenant agents in 2026. Subscription (flat fee with tiered feature gates) is simplest but risks heavy users eroding margin. Usage-based (per invocation, per thousand tokens, per task) aligns revenue to cost but risks bill shock. Mitigate with per-tenant spending ceilings.

Outcome-based (pay only when the agent delivers a measurable result) is boldest and most aligned with the agent’s value, but requires strong attribution connecting agent actions to business metrics. Most products start with subscription or usage-based and migrate to outcome-based once attribution matures.

The practical reason most teams delay outcome-based pricing is that it needs a causal link between a specific agent action and a business result (a resolved support ticket, a closed deal) and that link is often ambiguous when a human also touched the workflow. Usage-based pricing sidesteps the ambiguity by billing on a countable signal (tokens, invocations) that the agent platform already measures precisely.

Observability for Multi-Tenant Agents

Aggregate metrics hide problems: p95 looks fine while one tenant suffers a 10x regression. Tag every observability event with tenant_id. AWS X-Ray supports annotation-based filtering; Bedrock AgentCore emits per-session traces with tenant metadata automatically. Three metrics matter: error rate (signals data issues), token efficiency (drops signal prompt drift), and guardrail intervention rate (high rates suggest tier upgrade or policy review).

Annotation-based filtering in X-Ray means you can slice a trace query down to a single tenant_id and replay exactly what that tenant experienced, which turns a vague support ticket like “the agent is slow for us” into a concrete trace you can inspect end to end. Without tenant tagging on every span, the same investigation means grepping through shared logs hoping the right request stands out.

Migrating Between Deployment Models

Most systems do not start in their final topology. A common trajectory: siloed for the first 10 enterprise customers, hit a cost ceiling around 50 tenants, migrate to hybrid, then consolidate to pooled-with-strong-isolation once per-tenant memory primitives mature.

Siloed to pooled is highest-risk because isolation shifts from physical to logical. Audit every tool for tenant scoping, every IAM role for least privilege, and write regression tests that attempt cross-tenant access. Ramp behind a feature flag (5% → two weeks → 100%).

Pooled to siloed is mechanically simple but expensive if frequent. Automate it. Siloed to hybrid requires the control plane to track each tenant’s deployment model. Pick the topology you expect at 24 months and build toward it; stability beats theoretical optimality.

The feature-flag ramp matters more than it sounds: migrating a tenant from siloed to pooled at 100% on day one means any isolation gap surfaces in production against a real customer immediately. Ramping through 5%, then a subset of low-risk tenants for two weeks, then everyone, gives the chaos tests and monitoring dashboards time to catch a leak before it reaches a tenant who would notice and escalate.

Compliance and Audit for Multi-Tenant Agents

SOC 2, HIPAA, ISO 27001, FedRAMP: the answer depends on the audit trail. Three artifacts matter: tenant-scoped access logs (CloudTrail and AgentCore traces tagged with tenant_id), isolation test reports (weekly chaos tests attempting cross-tenant access), and tenant offboarding certificates (signed records of memory deletion, IAM revocation, KMS destruction).

CloudTrail inherits tenant tags; AgentCore emits per-session traces with tenant metadata; Bedrock Guardrails records every intervention. Build the audit pipeline as part of the control plane from day one. Produce artifacts continuously, not on demand.

Building the audit pipeline early also avoids a familiar failure mode: an auditor asks for six months of tenant-scoped access logs, and the team discovers CloudTrail tagging was only turned on three weeks ago. Continuous artifact generation costs almost nothing at write time (it is a tag and a log line) but retrofitting it after the fact means the gap in history simply cannot be recovered.

Multi-Tenant Agents in Practice: A Worked Example

A SaaS “AI Customer Success Agent” for mid-market B2B. Each tenant has its own CRM, churn playbook, and chat branding. The company chooses hybrid: free-tier tenants share a pooled Bedrock AgentCore agent (Strands supervisor + three MCP tools); premium tenants get a siloed agent with Claude Sonnet, per-tenant memory, and tighter IAM. API Gateway routes by JWT tier.

from strands import Agent
from strands_tools import http_request
import boto3, os
# Bedrock AgentCore session carries tenant_id implicitly via session metadata
def build_agent(tenant_id: str, tier: str) -> Agent:
    model = "anthropic.claude-3-5-sonnet" if tier == "premium" else "anthropic.claude-3-5-haiku"
    system_prompt = f"""You are the Customer Success Agent for tenant {tenant_id}. Always scope CRM queries with tenant_id='{tenant_id}'. Never reveal data from other tenants.
Cite CRM record IDs in every answer."""
    return Agent(
        model=model,
        tools=[http_request, crm_lookup_tool(tenant_id), playbook_search_tool(tenant_id)],
        system_prompt=system_prompt,
    )
# Pooled mode: build once per request from cached agent instances by (tenant_id, tier)
# Siloed mode: long-running dedicated instance per tenant
agent = agent_pool.get_or_build(tenant_id, tier)
result = agent(customer_question)

The system prompt carries the tenant identifier; the CRM lookup tool receives tenant_id as a closure so it cannot query another tenant’s data. Success is measured by per-tenant margin, p95 latency by tier, cross-tenant leak incidents (target: zero), and onboarding time (under 5 minutes).

Closing over tenant_id in the tool constructor rather than reading it from the prompt text is the important design detail here: even if a malicious or careless prompt tried to ask the agent to look up a different tenant’s account, the tool itself has no code path to honor that request because the identifier it queries with was fixed when the tool was built, not parsed from the model’s output.

Multi-Tenant Agents: Common Mistakes to Avoid

Most cross-tenant incidents trace back to one of a handful of repeatable mistakes rather than an exotic new failure mode. Reviewing a pooled agent design against this list before launch catches the majority of isolation gaps a chaos test would otherwise have to find the hard way.

Stopping tenant scoping at the API layer: forgetting to scope the DynamoDB query, S3 prefix, or Knowledge Base retrieval.
Single shared memory store: works until prompt injection retrieves another tenant’s session. Use per-tenant memory for regulated workloads.
No per-tenant rate limiting: one noisy tenant saturates model concurrency. Enforce token buckets at API Gateway.
Shared IAM role: one misconfigured tool reaches every tenant’s data. Use tenant-scoped roles, even in pooled mode.

Multi-Tenant Agents: Best Practices

These practices are the operational checklist that follows from everything above: they are not new ideas so much as the concrete, repeatable version of the isolation, context-propagation, and attribution principles this guide has walked through. Treat them as the minimum bar for a production multi-tenant agent, not an aspirational list.

Default to hybrid (pooled free + siloed premium).
Propagate tenant_id via session metadata, not spoofable request fields.
Use per-tenant AgentCore Memory for regulated workloads; share only for low-stakes consumer products.
Enforce per-tenant token-bucket limits at API Gateway keyed on JWT tenant_id.
Tag every AWS resource with Tenant={tenant_id} at provisioning time.
Run weekly chaos tests attempting cross-tenant access from staging.

Multi-Tenant Agents: Frequently Asked Questions

When should I choose siloed over pooled?

Choose siloed for enterprise or regulated tenants, when customization is a differentiator, or when tenant count is small (under 50). Pooled wins above a few hundred tenants and for free tiers. If you are unsure, default to pooled and carve out siloed exceptions for the specific tenants whose contracts demand it, rather than starting siloed for everyone and paying the operational cost of migrating later.

How does Bedrock AgentCore support multi-tenancy?

Per-tenant session metadata, per-tenant memory, a gateway enforcing per-tenant rate limits, and MCP tool credentials passing tenant-scoped IAM roles. One deployment serves many tenants with strong isolation. The session metadata is the connective piece, because it travels with the invocation automatically, memory, tools, and observability all inherit the same tenant scope without extra plumbing in application code.

What is the noisy neighbor problem in multi-tenant agents?

One tenant’s bursty requests saturate shared capacity (model concurrency, DynamoDB read units, AgentCore session limits), degrading latency for others. Fix: per-tenant throttling at API Gateway or AgentCore Gateway. The problem is specific to pooled deployments; a siloed tenant can only ever exhaust their own dedicated capacity, which is exactly why regulated or usage-spiky tenants often justify the siloed cost premium.

Can multi-tenant agents be HIPAA-compliant?

Yes, with per-tenant KMS keys, per-tenant AgentCore Memory, tenant-scoped IAM roles, and BAAs with AWS for HIPAA-eligible services (Bedrock, AgentCore, S3, DynamoDB). The isolation and audit-trail requirements described throughout this guide are not extra work bolted on for HIPAA. They are the same controls a well-run multi-tenant agent needs regardless of vertical, just enforced without exception.

How do I attribute Bedrock token cost to a specific tenant?

Tag Bedrock invocations at the API layer with tenant_id (as request metadata), then aggregate per-request token counts in CloudWatch or a billing pipeline. Cost Explorer shows tag breakdowns natively. For usage-based billing, emit the token count and tenant_id as a structured event at invocation time rather than trying to reconstruct cost later from aggregate logs, since per-request granularity is what a billing dispute ultimately requires.

[ays_quiz id=65]

Multi-Tenant Agents: Key Takeaways

SaaS with extra dimensions: inherits SaaS challenges (isolation, attribution) and adds agent-specific ones (non-determinism, memory partitioning, MCP isolation).
Siloed vs pooled vs hybrid is the first architecture decision; hybrid is the canonical enterprise pattern.
Tenant context injection propagates through every layer: JWT, API Gateway, Lambda, AgentCore, MCP tool, memory.
Control plane vs application plane separation is mandatory.
Isolation is layered: IAM, KMS, per-tenant Memory, microVM runtime, MCP credentials.
Cost attribution is first-class: tag every resource with Tenant={tenant_id} and emit per-request telemetry.

Multi-tenant agents on AWS give SaaS unit economics with per-tenant isolation. Choose hybrid by default, propagate tenant context through every layer, lean on AgentCore’s per-tenant primitives. Multi-tenant agentic AI on AWS succeeds when tenant isolation is enforced at IAM, AgentCore Memory, and KMS, not only at the application layer. Every multi-tenant design choice (pooled or siloed, shared or dedicated AgentCore) flows from the compliance tier and cost ceiling the tenant contract demands.

What Agentic AI on AWS Actually Costs: A Unit-Economics Breakdown

Amar Tinawi — Fri, 07 Aug 2026 14:09:20 +0000

Cross-posted from iqraa.tech — the full guide includes the worked ROI calculation, the Lambda/Step Functions cost model, and the risk-economics framework.

Agentic AI economics compares total human costs against agent costs, introduces pay-per-outcome pricing, and gives you the ROI models to scale automation sustainably on AWS, answering when replacing human labor with agentic AI actually pays off, and where agentic AI economics tips in favor of automating.

Agentic AI Economics: What You’ll Learn

Agentic AI economics is the structured comparison of what human labor really costs against what an agentic AI system really costs, across the full task lifecycle. The AWS Prescriptive Guidance frames this as a shift away from simplistic “human cost minus AI cost” arithmetic toward a model weighing total economic impact, risk, decision quality, and long-term strategic value.

This guide covers both cost models, pay-per-outcome pricing, automation candidate criteria, and ROI metrics, then walks through a document-review worked example mapping Bedrock token pricing, Lambda per-invocation billing, and Step Functions state-transition costs onto real unit economics.

Treat economics as a continuous practice: measure before, after, and as prices shift. Retire what stops paying off; double down on what compounds.

Agentic AI Economics: The Core Question

The AWS guide opens with a line worth memorizing: no system is 100% right. The core question is not “is the agent cheaper?” but “given that both humans and agents err, which mix delivers the best outcome-adjusted return?”

A naive comparison fails fast. A human reviewer at $60/hour fully loaded handles eight documents per hour; a Bedrock agent costs $0.04/document in inference. On raw unit cost the agent wins by three orders of magnitude, until you account for the agent’s 3% hallucination rate at $2,000 per remediation.

Once error cost enters the equation, the cheaper system is not always the agent.

The question changes with volume. At ten documents a day the human’s fixed overhead dominates; at ten thousand, the agent’s near-zero marginal cost dominates. The model answers “when” and “where” as much as “whether,” and captures strategic value: the second automation costs far less once platform, guardrails, and observability exist.

Understanding Total Human Labor Costs

Most teams undercount human costs, biasing the comparison against automation before it begins. Fully loaded cost is not salary; it is salary plus benefits, payroll taxes, facilities, management overhead, hiring, onboarding, training, and turnover, typically 1.4 to 1.8 times base salary.

Two hidden costs dominate. Latency: humans work eight to ten hours a day and handle one task at a time, so backlogs wait and surges create overtime. Error rate: fatigue and consistency errors whose rework and compliance costs are real but rarely attributed.

Training recurs on every policy or product change, and when a tenured reviewer leaves their judgment leaves with them. And scaling human cost is linear and slow: doubling throughput doubles overhead and takes months. That linearity is the lever agentic AI economics exploits.

Agentic AI System Costs on AWS

Inference is the dominant variable cost. Bedrock prices per million tokens; Nova Micro or Claude Haiku cost fractions of a cent per thousand tokens, Sonnet or Nova Pro several times more, favoring tiered selection (route easy 80% to the cheap model, hard 20% to the expensive one).

Infrastructure: Lambda bills per invocation plus GB-second; Step Functions per state transition (a four-tool workflow costs four transitions); Bedrock AgentCore, OpenSearch, and S3 each add their own line. Prompt engineering is skilled labor but amortizes: one versioned prompt serves a million requests. Monitoring (CloudWatch, X-Ray, eval suites, sampled human review) typically adds 10 to 20% to operating cost.

Cost Factor	Human Worker	Agentic AI System
Per-task cost	$5 to $50 (fully loaded)	$0.001 to $0.10 (inference)
Throughput	8 to 10 hrs/day, 1 at a time	24/7, 1000s in parallel
Scaling cost	Linear (hire more)	Near-zero marginal
Training cost	Weeks, recurring	Prompt updates, hours
Error cost	Variable, recoverable	Variable, needs guardrails
Oversight needed	Low (self-monitoring)	Required (trust builds over time)

The comparison that matters is per completed task, not per hour. Re-baseline quarterly using Cost Explorer and CloudWatch telemetry. Token prices fall, and a table accurate in January can mislead by July.

Put a number on it: a customer-support triage agent that sends 600 input tokens and returns 80 output tokens per ticket, orchestrated through two Step Functions transitions and one Lambda invocation, lands near $0.003 per ticket on a mid-tier Bedrock model, before oversight sampling and error cost.

Multiply that by ticket volume and the agentic AI economics of the workload are visible in minutes, which is the entire point of building the per-task table before debating the automation decision in a meeting.

Pay-Per-Outcome: The New Pricing Paradigm

Customers are shifting toward pay-per-outcome models, where you pay only when the agent delivers a measurable business result (a resolved ticket, an approved claim, a classified document), not for the compute that attempted it.

The appeal is risk alignment: under upfront pricing the customer bears underperformance risk; under pay-per-outcome the vendor shares it. ROI becomes self-evident: if each outcome is priced below the human cost of producing it, every transaction is accretive.

The model demands strong attribution, which is why it pairs naturally with AgentOps (Part 9). The catch is outcome definition: “resolved ticket” is clear until you define “resolved” across ten segments. Pay-per-outcome fits high-volume structured tasks long before ambiguous knowledge work.

Which Jobs Are Good Candidates for Automation?

The strongest candidates are high-volume, structured, repetitive tasks with clear success criteria and bounded ambiguity: document classification, invoice extraction, first-line triage, log anomaly detection, form validation. They repeat enough to amortize integration cost, prompt reliably, and measure accuracy cleanly.

Weak candidates are creative strategy, high-stakes empathy decisions, one-off annual tasks, ambiguous edge cases. High-stakes tasks like medical diagnosis or legal advice belong in a third category: human oversight, where the agent drafts and a human approves: ROI comes from productivity gain (e.g. 70% review-time reduction), not headcount reduction.

Plot tasks on a 2×2 of volume vs ambiguity: high-vol/low-amb = automate-now; high-vol/high-amb = automate-with-overhead; low-vol/low-amb = automate-if-cheap; low-vol/high-amb = leave-alone. Revisit quarterly. Tasks become automatable as models improve.

How to Calculate Agentic AI ROI

Cost per task: fully loaded human vs fully loaded agent. Throughput multiplier: how much more the agent produces, which reframes “can we afford the agent?” as “can we afford not to?” Error rate + rework cost: the most-skipped, most-decisive metric: error rate × per-error remediation cost for both sides, treated as a first-class line item. Time-to-value: how fast integration cost is repaid.

Net annual value = (human cost per task − agent cost per task) × annual volume − one-time integration cost − annual error-cost delta. Benchmark against your own historical data, not vendor case studies.

Run the formula both ways before committing budget. If agentic AI economics only clear the bar at optimistic error rates, the program is under-priced for risk; if they clear it even at pessimistic error rates and volume, integration is the safe next step. The teams that get agentic AI economics right treat this formula as a living spreadsheet updated monthly, not a one-time slide built to win a budget approval.

Six-month cumulative cost trajectory: break-even within the first month, widening savings as volume grows. Adapted from AWS Prescriptive Guidance.

Scaling What Works: From Pilot to Production

The pattern: start with high-volume structured tasks, measure everything, scale winners, retire failures. Pilots look promising; production surfaces what pilots hid.

Run a 30-day pilot on one task for one team. Treat it as an investment in data, not savings. When it clears its ROI bar, ramp volume in stages (10%, 50%, 100%) watching unit cost (should fall) and error cost (rises if task mix drifts harder). Retire failures honestly: a retired pilot is a successful experiment.

Set the ROI threshold before the pilot and act on “stop.”

The portfolio view: a mature program runs dozens of agents on one dashboard and reallocates investment from declining to rising agents quarterly.

Risk Economics: The Cost of Agent Errors

Finance respects this part most; engineering underweights it most. An agent that hallucinates a plausible-but-wrong action can be cheaper per task than a human until you account for catching and reworking the error. Weigh agent errors against equivalent human errors honestly.

Four deployment approaches mapped to error tolerance. Adapted from AWS Prescriptive Guidance.

Treat hallucination as expected error cost: error rate × per-error cost. Humans also hallucinate: they misremember policy, apply yesterday’s rule, get tired at hour seven. Measure both sides; do not assume the human baseline is zero.

Bedrock Guardrails tilt risk economics in the agent’s favor: blocking harmful content, masking PII, denying out-of-scope topics, filtering hallucinations through contextual grounding checks before they reach a user. Treat guardrail config as an investment: reduction in expected error cost ÷ build cost. Human-in-the-loop is the other lever: find the sampling rate that catches errors that matter without erasing the throughput advantage.

Risk compounds with scale: a rate tolerable at one thousand tasks a day can be catastrophic at one million. Stress-test at projected scale.

This is where agentic AI economics and reliability engineering merge into one job. A guardrail that costs $0.002 per call but cuts the error rate in half is worth adding at almost any volume once you have priced the error it prevents; a guardrail that adds latency without moving the error rate is a cost with no return. Price every guardrail the same way you price the agent itself.

Agentic AI Economics in Practice: Automating Document Review with ROI Analysis

A financial services firm reviews 8,000 loan applications/month with twelve reviewers at $48/hour fully loaded. Average review time 20 minutes, error rate 4%, each error $1,200 to remediate. They build a Bedrock agent that extracts fields, checks policy, and recommends: inference $0.06/app, Lambda + Step Functions $0.01, human oversight samples 15% at $4 each.

# Agentic AI economics — document review ROI model
monthly_volume = 8000

# Human baseline (fully loaded)
human_hourly = 48
apps_per_hour = 3                      # 20 min each
human_cost_per_task = human_hourly / apps_per_hour   # $16.00
human_error_rate = 0.04
human_error_cost_per_task = human_error_rate * 1200  # $48.00
human_total_per_task = human_cost_per_task + human_error_cost_per_task  # $64.00
human_monthly = human_total_per_task * monthly_volume                # $512,000

# Agent baseline
inference = 0.06
infra = 0.01
oversight = 0.15 * 4                   # sample 15% at $4 each
agent_cost_per_task = inference + infra + oversight  # $0.67
agent_error_rate = 0.025               # measured in pilot
agent_error_cost_per_task = agent_error_rate * 1200  # $30.00
agent_total_per_task = agent_cost_per_task + agent_error_cost_per_task  # $30.67
agent_monthly = agent_total_per_task * monthly_volume                   # $245,360

# ROI
monthly_saving = human_monthly - agent_monthly         # $266,640
integration_cost = 180_000                            # one-time build
break_even_months = integration_cost / monthly_saving  # ~0.7 months

print(f"Human per task:  ${human_total_per_task:.2f}")
print(f"Agent per task:  ${agent_total_per_task:.2f}")
print(f"Monthly saving:  ${monthly_saving:,.0f}")
print(f"Break-even:      {break_even_months:.1f} months")

Agent: $30.67/task vs human $64.00, which is $33.33 saved per application, ~$266,000/month. Integration cost ($180K) is repaid in under a month.

But this only holds because error cost was modeled honestly. If agent error rate were 6% instead of 2.5%, error cost would rise to $72/task and the agent would lose.

Guardrails and eval suites hold error rate in the profitable range, which is why economics and engineering must build the model together.

Token Economics: From Price-Per-Million-Tokens to Cost-Per-Task

Bedrock bills separately for input and output tokens, and the two rarely cost the same: output tokens on most foundation models run three to five times the input rate, because generation is the expensive half of inference. A production agentic AI economics model has to price a task by its full token profile, not by a single blended rate pulled from a pricing page.

A document-review call that sends 1,800 input tokens of policy context and returns 220 output tokens of structured JSON is a fundamentally different cost shape than a chat turn that sends 200 tokens and returns 900.

Three levers move the token bill more than model choice alone. Prompt caching on Bedrock lets a repeated system prompt or a large policy document be cached across calls, cutting the input-token charge on every call after the first by roughly 90 percent for the cached portion, decisive for RAG-heavy agentic workloads where the same 2,000-token context accompanies every request.

Output-length discipline, constraining the agent to structured, schema-bound output instead of free-form prose, shrinks the more expensive half of the bill directly; a JSON-only response mode routinely cuts output tokens by half against an unconstrained one. Context trimming, summarizing or windowing conversation history instead of resending the full transcript every turn, keeps input tokens from growing quadratically across a long agent session.

None of this shows up in the sticker price of a model. Two teams running the same Claude or Nova model on the same workload can see per-task inference cost differ by 3 to 4x purely on prompt engineering discipline, which is why agentic AI economics treats token shape, not model choice, as the primary cost lever engineers actually control.

Orchestration and Hosting: Step Functions and SageMaker in the Same Model

Inference is rarely the whole bill. A four-step agentic workflow (retrieve, reason, call a tool, verify) orchestrated on Step Functions Standard costs one state transition per step at a fraction of a cent each; the same workflow on Step Functions Express, priced by request duration and memory rather than per-transition, is usually cheaper at high volume and short execution times, and more expensive at low volume with long-running steps.

Modeling both and picking per workload, not defaulting to one, is worth real money at scale, and belongs in the same agentic AI economics spreadsheet as the inference line, not a separate infrastructure budget nobody reconciles against it.

When an agent calls a custom fine-tuned model instead of a foundation model on Bedrock, the cost model shifts from per-token to per-instance-hour: a SageMaker real-time endpoint bills for the instance whether or not it is handling traffic, so a lightly used endpoint can cost more per task than a heavily used Bedrock on-demand call.

SageMaker Serverless Inference or Bedrock Provisioned Throughput close that gap for spiky or highly predictable traffic respectively. Provisioned Throughput trades a flat hourly commitment for a much lower per-token rate once volume clears the break-even point, typically in the tens of millions of tokens per month.

Include instance-hour or reserved-throughput cost in the per-task model exactly like Bedrock on-demand pricing; treating hosting as a fixed sunk cost outside the ROI calculation is how pilots that look profitable in a spreadsheet lose money in production.

Agentic AI Economics: Common Mistakes to Avoid

Comparing agent cost to salary, not fully loaded cost: biases the comparison either way.
Skipping error cost: the most-decisive line item; omitting it inflates every pilot.
Automating the hardest task first: start with high-volume structured tasks.
Scaling on enthusiasm without telemetry: hidden costs surface at full volume.
Flat pricing where pay-per-outcome fits: match pricing to outcome measurement maturity.
Retiring pilots into permanent production: set the ROI threshold before the pilot and act on “stop.”
Ignoring idle hosting cost: a SageMaker real-time endpoint or a Bedrock Provisioned Throughput commitment bills by the hour whether or not it is handling traffic; size it to measured pilot traffic, not a guess, or the fixed cost erases the savings.
Treating token price as the whole inference bill: output tokens, uncached context, and unconstrained response length routinely double or triple the per-task cost implied by a model's headline price-per-million-tokens.

Each of these mistakes shares a root cause: modeling the sticker price instead of the delivered cost. Agentic AI economics done well prices a task the way finance prices a product: fully loaded, including the hosting commitment, the token shape, the oversight sampling rate, and the error-remediation cost, not the number on a pricing page.

Teams that skip this step consistently overstate ROI in the pilot deck and understate it in the first monthly AWS bill, which is the single most common reason a promising pilot gets killed after month two instead of scaled.

Human vs agent cost models and the pay-per-outcome shift. Adapted from AWS Prescriptive Guidance.

Agentic AI Economics: Best Practices

Model fully loaded human cost before any automation decision.
Track an ROI dashboard: cost per task, throughput multiplier, error + rework cost, time-to-value.
Start with high-volume structured tasks; defer ambiguous, high-stakes, or low-volume work.
Price error cost as a first-class line item.
Use tiered model selection on Bedrock: cheap models for easy cases, larger for hard.
Instrument the attribution pipeline early to enable pay-per-outcome later.
Scale in stages (10%, 50%, 100%) watching unit cost and error cost.
Run the portfolio like an asset book: rebalance quarterly.

Cost governance turns these practices from good intentions into an enforced discipline. Tag every Bedrock, Lambda, Step Functions, and SageMaker resource by agent and by task type, then pull a Cost Explorer report grouped by tag monthly. Without tagging, agentic AI economics collapses into one opaque AWS bill line that nobody can attribute to a specific automation's ROI.

Set an AWS Budgets alert per agent so a runaway retry loop or a prompt regression that balloons token usage surfaces in hours, not at the end of the billing cycle.

Quantify the caching and batching levers instead of assuming they help. Bedrock prompt caching, batch inference for non-real-time workloads, and SageMaker Savings Plans for steady-state hosting each independently cut 20 to 60 percent off their respective cost lines when applicable, layered together across a mature agentic AI economics program, they are usually the difference between a pilot that clears its ROI bar and one that does not.

Re-run the model whenever AWS changes a price, a new model tier ships, or task volume moves by more than 20 percent; a cost table is a snapshot, not a constant.

ROI best practices: tiered models, honest error cost, staged scaling. Adapted from AWS Prescriptive Guidance.

Agentic AI Economics: Frequently Asked Questions

What is agentic AI economics and why does it matter?

The structured comparison of human labor cost against agentic AI system cost, accounting for total economic impact, risk, and strategic value. It matters because naive comparisons mislead: an agent can look cheaper per task and cost more once error is counted.

What is pay-per-outcome pricing?

You pay only when the agent delivers a measurable business result. It aligns vendor and customer risk, makes ROI self-evident, and is self-funding as volume grows. It demands strong attribution.

When does automation not make economic sense?

When volume is too low to amortize integration cost, ambiguity too high to measure success, or error cost overwhelms the per-task saving.

How do I calculate the ROI of an agentic AI workload on AWS?

Start with fully loaded human cost: salary, overhead, latency, error rate, recurring training. Compare against agent cost: Bedrock inference per million tokens, Lambda per invocation, Step Functions per state transition, AgentCore per session, plus monitoring overhead. Agentic AI economics rewards teams that measure outcome-adjusted return: a deflected support contact is worth more than a successful API call.

Net annual value equals per-task saving times volume, minus integration cost, minus annual error-cost delta. Run the dashboard monthly; retire agents whose ROI declines for two quarters.

What AWS services drive the biggest unit-cost savings for agentic AI?

Three levers dominate. Tiered model selection routes the easy 80 percent of traffic to Claude Haiku or Amazon Nova Micro at fractions of a cent per thousand tokens, reserving Claude Sonnet or Nova Pro for the hard 20 percent. Bedrock prompt caching cuts input token cost for RAG-heavy pipelines. Bedrock model invocation jobs handle batch workloads at reduced pricing.

Layered together, these agentic AI economics levers typically produce a 60 to 80 percent cost reduction versus a single-model baseline. Agentic AI economics on AWS rewards teams that measure cost-per-task monthly, retire agents whose agentic AI economics deteriorate, and double down on agents whose pay-per-outcome is rising.

The agentic AI economics stack (Bedrock per-token, Lambda per-invocation, Step Functions per-transition, AgentCore per-session) turns abstract AI cost into auditable line items.

[ays_quiz id=73]

Agentic AI Economics: Key Takeaways

Core question is outcome-adjusted return, not raw unit cost: weigh human and agent error honestly.
Count fully loaded human cost: salary, overhead, latency, error, recurring training.
Agent cost = inference + infra + prompts + monitoring, mapped to AWS primitives.
Pay-per-outcome aligns cost with result: shifts risk to vendor, makes ROI self-evident.
Automate high-volume structured tasks first; defer ambiguous or high-stakes work.
Measure ROI with a dashboard, never a single number.
Scale winners, retire failures: rebalance quarterly.
Risk economics is the deciding line item: invest in guardrails that hold error rate in the profitable range.

Agentic AI economics comes down to one discipline: price the fully loaded human alternative, price the fully loaded agent (inference, hosting, and error cost together) and let the honest gap decide what gets automated and when it scales. Get the token shape and the error-cost line right, and the agentic AI economics of a well-run program compound quarter over quarter.

Running Claude Code on AWS Bedrock: IAM, SCPs, and the Governance Model Most Teams Get Wrong

Amar Tinawi — Mon, 03 Aug 2026 12:00:00 +0000

Cross-posted with permission from iqraa.tech — the full guide has the complete IAM policy, SCP examples, and a 300-engineer rollout case study.

Claude Code has a Bedrock mode. Most people find out about it from a single environment variable — CLAUDE_CODE_USE_BEDROCK=1 — and assume that's the whole story. It isn't. The env var is the on-switch; the IAM policy, region strategy, and governance layer around it are where almost every enterprise rollout actually gets stuck.

I run through the deployment end to end below — the parts that matter for a platform team, not just "here's a flag."

Why route through Bedrock at all

The models are identical either way. What changes is procurement, governance, and audit:

Consolidated billing. An AWS spend commitment absorbs Bedrock usage instead of opening a separate Anthropic contract.
Data residency. Bedrock lets you pin inference to a region — direct Anthropic API gives you far less control over where a request lands.
SCP-level guardrails. You can restrict which IAM principals, accounts, or regions may invoke Bedrock at all. That control doesn't exist on the direct API.
CloudTrail auditability. Every invocation lands in CloudTrail with caller identity, source IP, region, and model ID — append-only and SIEM-ready.

The environment variables

export CLAUDE_CODE_USE_BEDROCK=1
export AWS_REGION=us-east-1
export ANTHROPIC_MODEL=us.anthropic.claude-sonnet-4-6-20250610-v1:0
export AWS_PROFILE=claude-code-prod

Note the model ID format difference — direct Anthropic uses claude-sonnet-4-6; Bedrock uses inference-profile IDs like us.anthropic.claude-sonnet-4-6-20250610-v1:0. The us. prefix means AWS load-balances the request across US regions for resilience. Drop the prefix and you're pinned to one region only. Mixing the two ID formats is the single most common "model not found" error teams hit on day one.

The IAM policy (minimum viable, not `bedrock:*`)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "InvokeClaudeModels",
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": [
        "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-sonnet-*",
        "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-*",
        "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-haiku-*"
      ]
    },
    {
      "Sid": "UseCrossRegionInferenceProfiles",
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": ["arn:aws:bedrock:us-east-1:*:inference-profile/us.anthropic.claude-*"]
    },
    {
      "Sid": "ListFoundationModels",
      "Effect": "Allow",
      "Action": "bedrock:ListFoundationModels",
      "Resource": "*"
    }
  ]
}

Two details that trip people up:

InvokeModelWithResponseStream is a separate permission from InvokeModel. Miss it and Claude Code falls back to one-shot invocation — it still works, but feels sluggish and streaming sessions fail.
Never grant Resource: "*" on Bedrock invoke actions. Bedrock also hosts Llama, Mistral, Nova, and marketplace models with their own cost/security profile. Scope to anthropic.claude-* ARNs only, so a leaked credential's blast radius stays limited to Anthropic models.

The gotcha nobody expects: model access provisioning

New AWS accounts don't have Claude models enabled by default. You have to explicitly request access per model per region in the Bedrock console, and approval can take anywhere from minutes to several hours — not instant, despite what the console implies. Request access to Haiku, Sonnet, and Opus up front, even if you're only using Sonnet today, so a future model switch doesn't get blocked by a multi-hour provisioning delay mid-rollout.

SCPs: the actual governance layer

IAM controls what a role can do; SCPs control what an entire AWS account can do, org-wide:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyBedrockOutsideApprovedRegions",
      "Effect": "Deny",
      "Action": "bedrock:*",
      "Resource": "*",
      "Condition": {
        "StringNotEquals": { "aws:RequestedRegion": ["us-east-1", "eu-west-1"] }
      }
    },
    {
      "Sid": "DenyNonAnthropicBedrockModels",
      "Effect": "Deny",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": "arn:aws:bedrock:*:*:foundation-model/*",
      "Condition": {
        "StringNotLike": { "bedrock:FoundationModelArn": "arn:aws:bedrock:*:*:foundation-model/anthropic.claude-*" }
      }
    }
  ]
}

Layer this with an IAM permission boundary and the inline invoke policy above, and you get the same three-layer model mature shops already use for IAM and KMS: SCP governs the org, permission boundary governs what roles developers can create, inline policy governs what Claude Code itself can invoke.

Cost levers that actually move the number

The headline per-token rate is the ceiling, not what you'll actually pay:

Prompt caching — cached tokens bill at ~10% of normal input rate. A stable CLAUDE.md (no dynamic timestamps/build IDs) keeps cache hits high; that alone routinely cuts effective input cost 80-90% on long sessions.
Model routing — Haiku input is $0.80/M tokens vs. Opus at $15/M. Review your actual model mix after 30 days; Opus dominating usually means either a CLAUDE.md that isn't giving Sonnet enough structure, or developers manually overriding the default.
--max-budget-usd in CI — without a hard cap, a runaway agent loop in a failed pipeline is the most common cost-overrun story in early adoption. $5/pipeline is a reasonable default for routine tasks.

Auth: use the credential chain you already have

Claude Code on Bedrock rides the standard AWS SDK credential chain — no custom auth to build. AWS SSO for developer workstations (aws sso login, short-lived creds, single point of revocation), OIDC federation for CI/CD (GitHub Actions/GitLab/Jenkins all support it — no static secrets), instance/task roles for EC2/ECS. If you're stuck on long-lived access keys, rotate quarterly via Secrets Manager + a rotator Lambda, not manually.

The five failure modes that generate the most support tickets

Skipping the model-access request (blocks launch for hours)
Wrong model ID format — direct vs. cross-region vs. single-region Bedrock IDs
Missing InvokeModelWithResponseStream
Wildcard IAM resources on Bedrock invoke actions
No SCP region pinning — developers will invoke from unintended regions if nothing stops them

Full write-up (including a worked 300-engineer financial-services rollout, VPC endpoint config, and provisioned-throughput break-even math) is here: Claude Code AWS Bedrock: Enterprise Setup Guide

Happy to answer questions on IAM/SCP specifics in the comments — I set up a few of these rollouts and the model-access delay catches everyone the first time.
``

EKS Diagnoses: The Swiss Knife

Amar Tinawi — Fri, 20 Mar 2026 22:40:35 +0000

The 2 AM wake-up call every Kubernetes engineer dreads — and the tool I built to make it less painful

It's 2 AM. PagerDuty fires. Pods are crashing across your EKS cluster. You SSH in, bleary-eyed, and start the ritual:

kubectl get pods --all-namespaces | grep -v Running
kubectl describe pod <that-one-pod>
kubectl get events --sort-by=.lastTimestamp
kubectl logs <pod> --previous
kubectl top nodes
kubectl describe node <node>
aws eks describe-cluster ...
aws logs filter-log-events ...

Twenty minutes in, you're staring at a wall of YAML. You've checked six things. You still don't know the root cause. You don't even know if the node pressure caused the evictions, or if the evictions caused the node pressure.

After living this loop for two years across dozens of EKS clusters, I built a tool to automate the entire diagnostic process. It runs 73 analysis methods in parallel, correlates findings across data sources, identifies the root cause with a confidence score, and hands you a single interactive report — in about 60 seconds.

This is the EKS Comprehensive Debugger, and here's why it exists and how it works.

The Problem: EKS Troubleshooting Is a Scavenger Hunt

Kubernetes failures are rarely isolated. A single root cause — say, a node running out of memory — cascades into a chain of symptoms: pod evictions, rescheduling failures, service endpoint gaps, and eventually user-facing 5xx errors. By the time you're paged, you're looking at the end of that chain.

The diagnostic challenge isn't running kubectl. It's knowing which of the 50+ things to check, in what order, and then correlating findings across completely different data sources — Kubernetes events, pod status, node conditions, CloudWatch metrics, control plane logs, VPC networking, IAM roles, and AWS service quotas — to find the one thing that started it all.

Most teams solve this in one of three ways:

1. Tribal knowledge. The senior engineer who's "seen this before" runs their mental playbook. Works great until they're on vacation.

2. Runbooks. Documented checklists. Better, but they go stale, they can't correlate across data sources, and nobody reads a 40-step runbook at 2 AM.

3. Observability platforms. Datadog, Grafana, New Relic. Excellent for monitoring, but they show you dashboards — they don't diagnose. You still need to interpret the data and connect the dots yourself.

None of these answer the question an on-call engineer actually needs answered: "What broke, why, and what do I do about it?"

What the Tool Does

The EKS Comprehensive Debugger is a single Python script that connects to your cluster and systematically checks everything. It pulls data from four sources simultaneously:

Kubernetes API (via kubectl)
AWS EKS API
CloudWatch Logs
CloudWatch Metrics

It runs 73 analysis methods in parallel, correlates findings to identify root causes, and generates two output files:

An interactive HTML dashboard for humans
An LLM-ready JSON file for AI analysis

One command, one minute, complete picture.

amartinawi / EKS_Dubugger

Production-grade Python diagnostic tool for Amazon EKS cluster troubleshooting

EKS Health Check Dashboard

A production-grade Python diagnostic tool for Amazon EKS cluster troubleshooting. Analyzes pod evictions, node conditions, OOM kills, CloudWatch metrics, control plane logs, and generates interactive HTML reports with LLM-ready JSON for AI analysis.

Version: 5.0.0 | Analysis Methods: 84 | Catalog Coverage: 100% | Tests: 384

Features

Comprehensive Issue Detection (84 Analysis Methods)

Pod & Workload Issues

CrashLoopBackOff - Container crash detection with exit code analysis
ImagePullBackOff - Registry authentication, rate limits, network issues
OOMKilled - Memory limit exceeded detection
Pod Evictions - Memory, disk, PID pressure analysis
Probe Failures - Liveness/readiness probe failures
Init Container Failures - Init container crash, timeout, dependency issues (v3.6.0)
Sidecar Health - Istio, Envoy, Fluentd sidecar failures (v3.6.0)
Stuck Terminating - Finalizer and volume detach issues
Deployment Rollouts - ProgressDeadlineExceeded detection
Jobs/CronJobs - BackoffLimitExceeded, missed schedules
StatefulSets - PVC issues, ordinal failures
PDB Violations - Pod Disruption Budget blocking drains…

View on GitHub

The 73 Checks: What It Actually Analyzes

The tool covers the full EKS stack in categories that map to how failures actually cascade:

Pod & Workload Issues

CrashLoopBackOff, OOMKilled, ImagePullBackOff, stuck terminating pods, failed init containers, broken sidecar proxies, deployment rollout failures, StatefulSet issues, PDB violations.

Node Health

NotReady nodes, disk/memory/PID pressure, resource saturation (a leading indicator at 90% allocation, before kubelet pressure triggers), PLEG issues, container runtime health, kubelet version skew, outdated AMIs.

Networking

VPC CNI IP exhaustion, CoreDNS failures, DNS ndots:5 amplification, services with no endpoints, missing Ingress backends, ALB health, conntrack table exhaustion, security group misconfigurations.

Control Plane

API server latency and rate limiting, etcd health, controller manager reconciliation failures, admission webhook timeouts, scheduler issues.

Storage

Pending PVCs, EBS CSI attachment failures, EFS mount issues, failed volume snapshots.

IAM & Security

RBAC errors, IRSA/Pod Identity credential failures, privileged containers, sensitive host path mounts, PSA violations.

Autoscaling

Cluster Autoscaler issues, Karpenter provisioning and drift, HPA metrics source health, topology spread constraint violations.

Each finding is classified as either:

Historical Event — something that happened during your scan window
Current State — what the cluster looks like right now

This distinction separates "what's happening now" from "what happened during the incident window."

The Part That Actually Matters: Root Cause Detection

Listing problems is easy. Any monitoring tool can tell you "5 pods crashed." The hard part is answering why.

The debugger's correlation engine connects findings across data sources using a 5-dimensional confidence scoring system:

Dimension	Weight	What It Measures
Temporal	30%	Did the cause happen before the effect?
Spatial	20%	Same node, namespace, or pod?
Mechanism	25%	Known causal relationship?
Exclusivity	15%	Only plausible explanation?
Reproducibility	10%	Pattern occurred multiple times?

These combine into a composite confidence score mapped to a tier: high (≥75%), medium (≥50%), or low (<50%).

Here's what a real detection looks like in the JSON output — a cluster upgrade identified as root cause with 92% confidence:

{
  "potential_root_causes": [{
    "correlation_type": "cluster_upgrade",
    "root_cause": "Cluster version upgrade in progress or recently completed",
    "confidence_tier": "high",
    "composite_confidence": 0.92,
    "confidence_5d": {
      "temporal": 1.0,
      "spatial": 0.8,
      "mechanism": 1.0,
      "exclusivity": 0.9,
      "reproducibility": 0.0
    }
  }]
}

temporal: 1.0 — the AWS API confirmed the upgrade timestamp preceded all other findings.
mechanism: 1.0 — "cluster upgrade causes transient failures" is a well-established causal relationship.
reproducibility: 0.0 — upgrades are one-time events. The other four dimensions still provide strong evidence.

This is the reasoning a senior SRE does intuitively. The tool makes it systematic, consistent, and available at 2 AM without waking anyone up.

Actionable Output: Not Just What's Wrong — What to Do

Every finding includes contextual remediation with:

Diagnostic commands to investigate further
Fix commands to resolve the issue
Both pre-populated with actual resource names from your cluster

The commands aren't generic templates — that's sg-0af46ef489f81f6d0, the actual security group from the cluster. Copy, paste, run.

The LLM-Ready JSON: Built for AI Analysis

The HTML report is for humans. The JSON is for AI.

Every run produces a structured JSON file optimized for feeding into an LLM. The schema includes:

Full analysis context (cluster, region, time range)
Findings with severity classifications
5D confidence-scored correlations
Prioritized recommendations

Practical workflow: paste the JSON into Claude or GPT and ask, "What's the most important thing to fix first and why?" The confidence tiers and spatial evidence give the model enough context to prioritize correctly.

# Run the analysis
python eks_comprehensive_debugger.py --profile prod --region eu-west-1 \
  --cluster-name production --days 1

# Two files generated:
# production-eks-report-20260301-035821.html    ← for humans
# production-eks-findings-20260301-035821.json   ← for AI

How to Run It

# Clone and install
git clone https://github.com/amartinawi/EKS_Dubugger
cd eks-debugger
pip install -r requirements.txt

# Basic usage (auto-detects cluster)
python eks_comprehensive_debugger.py --profile prod --region eu-west-1

# Incident investigation: last 2 hours
python eks_comprehensive_debugger.py --profile prod --region eu-west-1 \
  --cluster-name my-cluster --hours 2

# Post-mortem: specific time window
python eks_comprehensive_debugger.py --profile prod --region eu-west-1 \
  --cluster-name my-cluster \
  --start-date "2026-01-26T08:00:00" \
  --end-date "2026-01-27T18:00:00" \
  --timezone "America/New_York"

# Private cluster via SSM tunnel
python eks_comprehensive_debugger.py --profile prod --region eu-west-1 \
  --cluster-name my-cluster \
  --kube-context my-cluster-ssm-tunnel

Prerequisites: Python 3.8+, kubectl configured, AWS CLI with credentials. Read-only access to EKS, CloudWatch, EC2 — no write permissions required.

What I Learned Building This

Most EKS issues are knowable from existing data. The Kubernetes API and CloudWatch already have the information to diagnose 90% of problems. The bottleneck isn't data collection — it's knowing what to look for and how to connect the dots.

Root cause detection is about correlation, not classification. Classifying a finding as "critical" is easy. Determining that this node pressure caused those pod evictions on that node during this time window requires reasoning across multiple dimensions. Spatial correlation — matching cause and effect by node/pod/namespace identity — was the single biggest accuracy improvement.

The tool is most valuable when nothing is wrong. Running it proactively — after an upgrade, after a config change, as part of a weekly health check — catches issues before they page you. Version skew detection, deprecated API scanning, and resource saturation warnings are all leading indicators.

AI-ready output changes the workflow. Structured JSON with confidence-scored root causes means you can ask an AI assistant to explain findings, draft an incident report, or suggest an architecture change — and it has enough structured evidence to do it well.

What's Next

CI/CD integration — Run as a post-deployment check; fail the deployment if critical issues are detected.
Scheduled health reports — Weekly automated runs with delta reporting.
Multi-cluster support — Aggregate findings across clusters for fleet-wide visibility.

If you're managing EKS clusters and spending too much time on diagnostics, give it a try. The tool is open source, a single Python file, no infrastructure dependencies — just point it at your cluster and run.

GitHub: https://github.com/amartinawi/EKS_Dubugger

Issues, PRs, and feature ideas welcome.

DEV Community: Amar Tinawi

ZOdyssey: making the plan-review gate a hard hook, not a prompt convention

ZOdyssey: making the plan-review gate a hard hook, not a prompt convention

The problem, in four failure modes

The aha

How it works

The pipeline

The four enforced invariants

The cast, and why the conductor matters

Two more things worth knowing

Quick start

What it is NOT

What is next

Provenance

Multi-Tenant Agentic AI on AWS: Isolation and Cost Architecture

Multi-Tenant Agents: What You’ll Learn

Multi-Tenant Agents: The Core Challenge

Siloed vs Pooled Agent Deployment

When siloed wins

When pooled wins

When hybrid wins

Injecting Tenant Context into Agents

Tenant Onboarding and Lifecycle Management

Isolation, Security, and Data Ownership

Cost Attribution and Resource Management

Tenant Tiering and Pricing Strategy

Observability for Multi-Tenant Agents

Migrating Between Deployment Models

Compliance and Audit for Multi-Tenant Agents

Multi-Tenant Agents in Practice: A Worked Example

Multi-Tenant Agents: Common Mistakes to Avoid

Multi-Tenant Agents: Best Practices

Multi-Tenant Agents: Frequently Asked Questions

When should I choose siloed over pooled?

How does Bedrock AgentCore support multi-tenancy?

What is the noisy neighbor problem in multi-tenant agents?

Can multi-tenant agents be HIPAA-compliant?

How do I attribute Bedrock token cost to a specific tenant?

Multi-Tenant Agents: Key Takeaways

What Agentic AI on AWS Actually Costs: A Unit-Economics Breakdown

Agentic AI Economics: What You’ll Learn

Agentic AI Economics: The Core Question

Understanding Total Human Labor Costs

Agentic AI System Costs on AWS

Pay-Per-Outcome: The New Pricing Paradigm

Which Jobs Are Good Candidates for Automation?

How to Calculate Agentic AI ROI

Scaling What Works: From Pilot to Production

Risk Economics: The Cost of Agent Errors

Agentic AI Economics in Practice: Automating Document Review with ROI Analysis

Token Economics: From Price-Per-Million-Tokens to Cost-Per-Task

Orchestration and Hosting: Step Functions and SageMaker in the Same Model

Agentic AI Economics: Common Mistakes to Avoid

Agentic AI Economics: Best Practices

Agentic AI Economics: Frequently Asked Questions

What is agentic AI economics and why does it matter?

What is pay-per-outcome pricing?

When does automation not make economic sense?

How do I calculate the ROI of an agentic AI workload on AWS?

What AWS services drive the biggest unit-cost savings for agentic AI?

Agentic AI Economics: Key Takeaways

Running Claude Code on AWS Bedrock: IAM, SCPs, and the Governance Model Most Teams Get Wrong

Why route through Bedrock at all

The environment variables

The IAM policy (minimum viable, not bedrock:*)

The gotcha nobody expects: model access provisioning

SCPs: the actual governance layer

Cost levers that actually move the number

Auth: use the credential chain you already have

The five failure modes that generate the most support tickets

EKS Diagnoses: The Swiss Knife

The 2 AM wake-up call every Kubernetes engineer dreads — and the tool I built to make it less painful

The Problem: EKS Troubleshooting Is a Scavenger Hunt

What the Tool Does

amartinawi / EKS_Dubugger

Production-grade Python diagnostic tool for Amazon EKS cluster troubleshooting

EKS Health Check Dashboard

Features

Comprehensive Issue Detection (84 Analysis Methods)

Pod & Workload Issues

The IAM policy (minimum viable, not `bedrock:*`)