DEV Community: Andrew Mackay

I Tested AACP Against Four Agent Frameworks. Here Is What I Found.

Andrew Mackay — Wed, 17 Jun 2026 18:48:53 +0000

The setup

A few weeks ago I published an article about building AACP, a typed
coordination protocol for multi-agent LLM systems. The premise was
simple: agents currently coordinate in natural language, which is
verbose, non-deterministic, and produces no reliable audit trail.
AACP replaces that with typed, pipe-delimited packets.

The first article measured token reduction on a single payroll workflow.
The finding was a 22.9 percent reduction on Claude and 23.7 percent on
GPT-4o. Interesting, but limited to one framework and one workflow type.

So I ran a proper benchmark. Same workflow, same data, same model, four
frameworks. 59 coordination hops each. Here is what I found.

The benchmark

The test case is a five-workflow department day: JML onboarding,
payroll, sales lead qualification, customer service resolution, and
month-end close. Three iterations of most workflows. 59 total
coordination hops.

For each framework I ran two versions:

Without AACP: the orchestrator writes natural language instructions to specialist agents in whatever style that framework uses by default
With AACP: the orchestrator sends typed packets, rule-based encoder, zero LLM cost

Same agents. Same mock data. Same gpt-4o-mini model throughout.

The results

Framework      Hops    Coord. calls    Total saving
────────────────────────────────────────────────────
LangChain        59      59 → 0          18%
CrewAI           59      59 → 0          30%
AutoGen          59      59 → 0          55%
Pydantic AI      59      59 → 0          85%

In every case, all 59 coordination LLM calls were eliminated for known
workflows. The difference is in how much the agent cost dropped alongside
the coordination cost.

Why the numbers are so different

This is the part I found genuinely interesting.

LangChain at 18%

LangChain's default coordination is task-based. The orchestrator writes
a description of what to do, relatively concise. AACP packets are more
compact, so there is a token saving on the coordination side, and the
agents process them slightly more efficiently. The 18% reflects a modest
improvement in a framework that was already reasonably tight.

CrewAI at 30%

CrewAI assigns tasks to agents with explicit role and goal context baked
into each message. The natural language instructions are more verbose
than LangChain's. More verbosity in the baseline means a larger
proportional saving when you replace it with a compact packet.

AutoGen at 55%

AutoGen uses a GroupChat model where agents exchange conversational
messages. Without AACP, the orchestrator writes something like:

"Hi HR Agent, I need you to retrieve all active employee salary
records for the period ending March 2026. Please include each
employee's department, cost centre, base salary, any changes applied
this month, and their pension contribution rate. Return the data as
a JSON array. Let me know if you need any clarification."

That is the coordination overhead AACP eliminates. The conversational
preamble, the clarification offer, the verbose field list -- all of it
goes away. The agent receives a packet and acts on it.

Pydantic AI at 85%

This one required a different framing to understand.

Pydantic AI already types the result layer. Agents return validated
Pydantic models, not free-form text. That is genuinely good -- the
output of every agent call is structured and reliable.

But the instruction layer is still natural language. The orchestrator
still writes English to tell agents what to do. The result is typed;
the instruction is not.

AACP completes the picture. With AACP, the orchestrator sends a typed
packet. The agent processes it and returns a typed Pydantic model. Both
layers are now deterministic and validated.

The 85% saving is partly because Pydantic AI's natural language
coordination, while not conversational like AutoGen, is still verbose
enough that compact AACP packets drive a significant agent cost
reduction. But the more important point is that the full stack --
instructions and results -- is now typed end to end.

The pattern

The saving is not random. It scales with how verbose the framework's
default coordination layer is:

More conversational default → larger AACP saving
Less conversational default → smaller AACP saving

This makes sense. AACP is most valuable exactly where coordination
overhead is highest. The frameworks that encourage rich, natural
language agent-to-agent communication benefit most from a typed
replacement.

What the cost numbers actually mean

At gpt-4o-mini scale the absolute dollar differences are small. A 59-hop
department day costs fractions of a cent either way. The cost argument
becomes significant at volume -- workflows running hundreds or thousands
of times per day, across enterprise pipelines where coordination is a
meaningful fraction of total cost.

The more immediately useful properties are determinism and auditability.
Natural language coordination varies on every run. The same intent
produces different messages. Validation before transmission is
impossible. Audit trails require natural language processing to extract
meaning.

AACP packets are identical across runs. They validate against a schema
before transmission. Every packet is a machine-readable audit record
at zero additional cost.

The typed-framework case

The Pydantic AI result points at something broader. As agent frameworks
mature, more of them will move toward typed results. Pydantic AI is
ahead of the curve here, but structured outputs are increasingly common
across all frameworks.

The instruction side has not followed at the same pace. Orchestrators
still write natural language to coordinate agents, even when those
agents return typed results. That is a gap AACP is designed to fill.

What is available now

Four framework integrations, all benchmarked at 59 hops:

pip install aacp-langchain    # 18% saving
pip install aacp-crewai       # 30% saving
pip install aacp-autogen      # 55% saving (Python 3.11)
pip install aacp-pydantic-ai  # 85% saving
pip install aacp              # core SDK, encoders, RuleRegistry

241 pre-validated community rules for common business workflows at
registry.aacp.dev. IETF Internet-Draft draft-mackay-aacp-03.
Full spec at aacp.dev.

What I would like to know

These benchmarks used mock data on a single model. I am interested in
whether the results hold in production environments with real data,
longer workflows, and different model combinations.

Specifically:

Does the determinism property matter in practice or is variation in coordination messages genuinely harmless?
Where does the typed packet format break down -- what coordination patterns cannot be expressed in the AACP vocabulary?
Does the Pydantic AI result -- completing the typed stack -- reflect something you have felt as a gap in your own systems?

If you are running multi-agent workflows in production, I would
genuinely like to hear what you find.

Links

Full spec: aacp.dev
IETF Draft: draft-mackay-aacp-03
Python SDK: github.com/MackayAndrew/aacp
First article: I built a coordination protocol for multi-agent LLM systems

I Built a Coordination Protocol for Multi-Agent LLM Systems

Andrew Mackay — Wed, 10 Jun 2026 12:28:28 +0000

The honest origin

This started as a simple observation, not a grand plan.

I was building multi-agent workflows and noticed that every time one agent
needed to instruct another, it wrote a natural language message. Something like:

"Please retrieve the employee salary records for the period ending March 2026.
I need all active employees, their departments, cost centres, base salary,
any changes made this month, and pension contribution rates. Return as JSON array."

That varies in wording every time the workflow runs. It contains pleasantries the receiving agent does not need. And when something goes wrong, there is no clean audit trail, just a log file full of slightly different English sentences.

I started wondering whether this was actually the right approach, or just the default one.

The first question

The obvious question was whether you could compress those instructions. Replace verbose English with something more structured. I ran some benchmarks using live API usage_metadata and found consistent results: across a four-hop payroll workflow, coordination token usage dropped by 22.9 percent on Claude Sonnet 4.5 and 23.7 percent on GPT-4o versus equivalent English instructions.

That felt like a result worth building on. So I defined a packet format:

FETCH|HR|return:HR-Agent|p:1|aacp:1.1|res:emp_salary|period:2026-03|filter:status=active|fmt:json

Typed. Pipe-delimited. Self-describing. Produced identically on every run by a rule-based encoder at zero LLM cost. I called it AACP (Agent Action Compression Protocol) and published a Python SDK with encoders for common business workflows:
payroll, IT provisioning, invoice processing, contract review.

For known workflow types, the encoder produces the packet deterministically.
For novel instructions, a four-tier fallback routes to LLM encoding once, logs the result, and serves it from cache on every subsequent call. An amortisation benchmark across 240 encoding operations showed 91.6 percent cost saving versus per-call LLM encoding, with 6 LLM calls required across the full run.

I filed an IETF Internet-Draft, stood up a site at aacp.dev, and started
getting feedback.

What I got wrong about the value

The token reduction is real and measurable. But when I started running the
protocol through more complex workflows and getting feedback from engineers, it became clear that token reduction was not the most interesting thing about it.

The more interesting thing was determinism.

When an agent coordinates using AACP packets rather than natural language, the coordination layer becomes predictable. The same workflow produces the same packets on every run. You can validate a packet against the schema before sending it. You can log every coordination hop as a structured record without post-processing. You can replay a workflow and know the coordination instructions were identical to the original run.

Natural language coordination cannot offer any of those things reliably.

This matters more in some environments than others. For a simple two-agent
prototype, the difference is minimal. But for workflows that run repeatedly at scale, or where auditability matters, or where multiple frameworks and models need to interoperate, the coordination content layer starts to look like a genuine gap in the current stack.

Where it actually fits

To be direct about the architecture: AACP operates between the routing layer and the task execution layer.

MCP (Anthropic) handles agent access to external tools. A2A (Google) handles agent discovery and task routing. Both are excellent at what they do. Neither specifies what agents say to each other inside coordination messages. That is the layer AACP addresses.

This means AACP is complementary to both, not competitive with either. A packet can travel inside an MCP payload or be routed by A2A. The protocol is transport-agnostic and model-agnostic by design.

Honest about where it helps most

AACP is most useful in specific conditions. It is worth being direct about this.

Repetitive structured workflows benefit most. Payroll runs monthly. IT
provisioning follows the same steps for every new hire. Invoice processing
applies the same logic every time. For these workflows, a rule-based encoder produces coordination packets at zero LLM cost on every run, indefinitely.

Compliance-sensitive pipelines have a different but equally concrete benefit. When you need to demonstrate what instruction was sent to which agent, when, and in what form, typed packets are a significantly cleaner audit record than parsed natural language. The audit agent in the lab runs without LLM calls, it logs structured records from the packets themselves at zero cost.

Multi-model and multi-framework systems face an interoperability challenge that AACP directly addresses. The same packet format validated identically across Claude Sonnet 4.5, GPT-4o, GPT-4.1, and GPT-4.1-mini in lab tests without model-specific prompt tuning. When agents run on different models or were built with different frameworks, a shared coordination vocabulary reduces translation overhead.

For enterprises specifically, the token reduction argument is secondary. What matters is auditability, determinism, and the ability to adopt a vendor-neutral coordination format that works regardless of which model or framework is running underneath. The coordination layer does not change when you swap from one LLM provider to another.

For simpler systems, the case is weaker. Two agents doing open-ended
research have small coordination overhead relative to their task work. AACP
adds the most value where workflows are repetitive, structured, and run at volume.

The framework integration results

After publishing the core SDK, I built integrations for LangChain and CrewAI to test the protocol in real framework contexts.

The methodology was straightforward: run the same five-workflow department day scenario (JML onboarding, payroll, sales qualification, customer service resolution, and month-end close) with and without AACP, using identical agents, identical data, and identical models. 59 coordination hops per run.

In a standard LangChain implementation, every agent-to-agent coordination hop involves an LLM generating the instruction. Across 59 hops that means 59 LLM calls before any task work begins. With AACP, that drops to zero for known workflows. Total workflow cost reduction: 18 percent.

CrewAI showed a 30 percent reduction across the same 59-hop scenario. The
larger saving reflects that CrewAI's default natural language task descriptions are more verbose, so the per-hop saving is higher. In both frameworks, coordination messages became deterministic, schema-validated, and machine-readable audit records.

The QBR lab

The most recent addition is a Quarterly Business Review workflow that connects to real services. Five agents coordinate via AACP packets to fetch sprint metrics from Jira, extract decisions and action items from Notion, retrieve budget actuals from Google Sheets, consolidate everything into a single view, and draft an executive summary for human review.

The important design decision: the human review gate is explicit. The protocol coordinates the data gathering and consolidation. The recommendation in the output is clearly labelled as AI-assisted, not AI-decided. Deciding whether to continue, pause, or scale a project requires reading the room in ways the system cannot do. That boundary is intentional.

The same workflow runs at daily, weekly, monthly, and quarterly cadence by
changing a single field in the coordination packet.

What I built

The full stack as of today:

Python SDK (pip install aacp) with 8 workflow encoders across 6 domains
TypeScript SDK (npm install aacp-ts)
LangChain integration (pip install aacp-langchain) -- 18% saving, 59 hops
CrewAI integration (pip install aacp-crewai) -- 30% saving, 59 hops
241 pre-validated community rules across 7 domains at registry.aacp.dev
VS Code extension (Dispatch) for packet building and validation
QBR lab with live Jira, Notion and Google Sheets connections
IETF Internet-Draft: draft-mackay-aacp-02

Everything is MIT licensed and on GitHub at github.com/MackayAndrew.

What I think this could become

The most interesting long-term possibility is a shared coordination vocabulary that agent frameworks can adopt independently of each other. A packet format that any agent understands, regardless of whether it is running on Claude, GPT, or an open model, and regardless of whether it was built with LangChain, CrewAI, AutoGen, or a custom framework.

That is a reasonable description of what AACP already is, for the domains and task types it covers today. The question is whether it scales to a broader set of workflow types, and whether the community of people building multi-agent systems finds it useful enough to adopt.

That is what I am trying to find out.

If you are building multi-agent systems and the determinism or auditability
properties are relevant to your use case, the SDK is published and the spec
is at aacp.dev. I would genuinely like to hear whether it is useful and where it falls short.

Links

Spec and documentation: aacp.dev
IETF Draft: draft-mackay-aacp-02
Python SDK: github.com/MackayAndrew/aacp
LangChain integration: github.com/MackayAndrew/aacp-langchain
CrewAI integration: github.com/MackayAndrew/aacp-crewai
Community rules: registry.aacp.dev