"Same information in different places" is the cleanest framing of this I've read, and measuring it instead of joining the "MCP is dead, REST won" chorus is the right move. The placement point is the whole thing: MCP re-presents the tool surface every turn, REST pays the contract cost once and then leans on one generic call.
Where I'd expect the divergence to bite hardest is exactly where you say they're a tie — small surfaces — once you scale the tool count. Re-sending N tool schemas every turn is a per-turn token tax that grows with the surface, which is why everyone's suddenly pruning MCP servers from 15 tools to 4. REST's "contract once in the system prompt" sidesteps that, but trades it for the model having to construct correct paths from prose, which a smaller model fumbles. So the real axis might not be MCP-vs-REST but surface-size-vs-model-capability.
The ETag/304 gap is the most interesting asymmetry you surface — conditional reads are something tool use genuinely can't express today. Did the repeated-read task show REST pulling ahead on tokens once caching kicked in, or did the agent fail to exploit the cache without being told to?
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
Thanks Max! You're right on the surface-size point, and I actually measured it: on the 30-tool surface MCP spent about 35% more tokens for the same success rate, because it re-ships every schema every turn. I like your "surface size vs model capability" axis, it's a better way to put it.
On caching, the honest answer is: the agent was told. The REST system prompt says to send If-None-Match when re-reading, so I measured whether the protocol can express a conditional read, not whether a model discovers it on its own. And on tokens the cache didn't pay off: on repeat_read REST used a bit more tokens than MCP (the model has to write the header), but about 30% less bytes on the wire. With a small payload like an order that's noise. The win shows up with large, rarely changing resources, where a 304 skips the whole body. That's a fair gap in my test and a good follow-up experiment, thanks for pushing on it!
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
Your runner passes the system prompt and the tool configuration into every Converse call, and the REST column shows it: order_customer takes 4,500 input tokens over three model turns against 2,850 for order_status over two, so the third turn costs about 1,650 tokens, about as much as each of the first two. A contract read once could not do that.
So the gap in Result 2 reads as a difference in density rather than in placement. Going from six operations to thirty adds 422 input tokens per call on the REST side and 1,307 on the MCP side, the same on every task whose number of turns did not change between your small and large results files. A tool schema costs roughly three times what a contract line costs, and both are paid on every call.
That makes Result 2 and Result 4 one knob, how many tokens the description spends on every call. In the thirty-operation run each extra schema carries the same name, description and parameter as its contract line, so the threefold cost there is format alone, and in Result 4 the extra weight is the enum values that let MCP map Europe to EUR on the first call, which you note would bring the per-turn tax with them if pasted into the contract. The run that would separate protocol from format is the REST arm with the same thirty JSON schemas pasted into its contract: if its input tokens land next to the MCP arm, the 35 percent belongs to the format of the description, not to where it sits.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
Sorry for the slow reply! I wanted to run all the experiments myself before answering.
You're (absolutely) right, and thanks for actually digging into the results files 😀
Converse is stateless and I don't use prompt caching, so the REST system prompt goes out on every call, exactly like the tool config, "the contract is read once" is wrong for this harness.
So I ran the experiment you proposed: third arm, REST with the same thirty JSON schemas pasted into its contract. All 3 arms ran in the same session, 180 runs, 100% success.
Going from six to thirty operations adds this many input tokens per call:
terse REST: 422
MCP: 1,307
REST with the schemas: 1,384
Same schemas, same cost. REST is even a little more expensive because of the paths. So the 35% belongs to the density of the description, not to the protocol or to where it sits.
That also makes Result 2 and Result 4 one knob, as you said: how many tokens you spend describing the API on every call, and what you get back for them (first-call success on a strict API).
Code and numbers are in the repo (iface_rest_schema.py, RESULTS.md). I'll fix the post too.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
1,384 against 1,307 settles it more cleanly than I expected, and the 77 tokens the REST arm pays for its paths is a good detail to have measured rather than assumed.
If density is the knob, the next question is which part of the roughly 960 tokens per call that the schemas add over the terse contract buys the first-call success. Result 4 points at the enum values, the ones that let MCP map Europe to EUR on the first try. A fourth arm with the terse contract plus only those enum lists, and nothing else from the schemas, would show whether a few dozen tokens recover most of the first-call success or whether the descriptions and types around them are doing the work too. That would turn the trade in the post into a curve rather than two points.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
I ran it, five arms on the strict scenario, 150 runs, all at 100% final success. I enriched the terse REST contract one step at a time and measured the input tokens of the first call:
terse: 730 tokens, 0% first-call success
- enum lists only (metric, region, granularity): 783 tokens, 100%
- date format: 795 tokens, 100%
- full JSON schema: 895 tokens, 100%
MCP: 784 tokens, 100%
So 53 tokens of allowed values take first-call success from 0% to 100%, and everything after that is cost with no measurable return. The types and descriptions around the enums aren't doing the work here. The terse contract plus the enum lists costs the same as the MCP schema, one token apart.
One caveat on the date step: four of my five prompts already give the date as YYYY-MM-DD, and the model converted "October 20, 2026" correctly without being told. So that step doesn't really test the format. A messier date prompt would.
So the curve is closer to a step than a slope: the first few dozen tokens of the right kind buy almost everything. I'd guess that's true of strict APIs in general. Most of a schema is there for the validator, while the enums are what help the model. The code is in strict_curve.py and the numbers are in RESULTS.md in the repo. Thanks for pushing on this
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
A step rather than a slope is a better result for the post than the curve I expected, and it also says where the curve would bend. The enum lists here are short, three fields with a handful of values each, so 53 tokens buy the whole jump. For a field whose allowed values run into the hundreds, like ISO currency codes or IANA time zones, the list itself becomes the cost, and the trade in the post comes back at the level of one field: send every value on every call, or let the first call fail and pay for a second turn.
Your date caveat is the same point from the other side. The step only shows up where the model cannot already guess the right value, so a field it guesses correctly, like the ISO date here, costs tokens and buys nothing.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
I agree on where it bends, but your two examples land on the cheap side. ISO currencies and IANA time zones are long lists, but they're standards the model already knows, so by your date point they cost nothing to leave out.
The expensive case is a list that's long and private, like internal categories or this API's APC instead of APAC. For those I'd put the allowed values in the 400 error, which this backend already does, so you only pay when the model guesses wrong.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
GMT strategy and custom B2B outreach lists on request: people and companies by ICP, domain, profile URL, email or name. 1B+ profiles. Claude Code plugins, API, MCP. kevin@outricher.com
Did the 30-tool run load every definition up front? Some clients, Claude Code among them, defer MCP definitions until a tool is looked up, so that 35% gap may shrink there. The other cost I saw on a hosted server was an output schema inferred from sample rows, about 32K tokens for one tool. I have not tied either to success rate.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
Good catch Kevin. Yes, the harness loads all 30 definitions up front on every call, there is no deferred loading or tool search. So the 35% is the worst case. I haven't measured that and it's probably a fair limit of the test. The output schema point is new to me: my server doesn't send any outputSchema, so that cost isn't in my numbers at all. 32K tokens for a single tool is a lot. If you ever tie it to success rate I'd love to read it.
We're a place where coders share, stay up-to-date and grow their careers.
"Same information in different places" is the cleanest framing of this I've read, and measuring it instead of joining the "MCP is dead, REST won" chorus is the right move. The placement point is the whole thing: MCP re-presents the tool surface every turn, REST pays the contract cost once and then leans on one generic call.
Where I'd expect the divergence to bite hardest is exactly where you say they're a tie — small surfaces — once you scale the tool count. Re-sending N tool schemas every turn is a per-turn token tax that grows with the surface, which is why everyone's suddenly pruning MCP servers from 15 tools to 4. REST's "contract once in the system prompt" sidesteps that, but trades it for the model having to construct correct paths from prose, which a smaller model fumbles. So the real axis might not be MCP-vs-REST but surface-size-vs-model-capability.
The ETag/304 gap is the most interesting asymmetry you surface — conditional reads are something tool use genuinely can't express today. Did the repeated-read task show REST pulling ahead on tokens once caching kicked in, or did the agent fail to exploit the cache without being told to?
Thanks Max! You're right on the surface-size point, and I actually measured it: on the 30-tool surface MCP spent about 35% more tokens for the same success rate, because it re-ships every schema every turn. I like your "surface size vs model capability" axis, it's a better way to put it.
On caching, the honest answer is: the agent was told. The REST system prompt says to send If-None-Match when re-reading, so I measured whether the protocol can express a conditional read, not whether a model discovers it on its own. And on tokens the cache didn't pay off: on repeat_read REST used a bit more tokens than MCP (the model has to write the header), but about 30% less bytes on the wire. With a small payload like an order that's noise. The win shows up with large, rarely changing resources, where a 304 skips the whole body. That's a fair gap in my test and a good follow-up experiment, thanks for pushing on it!
Your runner passes the system prompt and the tool configuration into every Converse call, and the REST column shows it: order_customer takes 4,500 input tokens over three model turns against 2,850 for order_status over two, so the third turn costs about 1,650 tokens, about as much as each of the first two. A contract read once could not do that.
So the gap in Result 2 reads as a difference in density rather than in placement. Going from six operations to thirty adds 422 input tokens per call on the REST side and 1,307 on the MCP side, the same on every task whose number of turns did not change between your small and large results files. A tool schema costs roughly three times what a contract line costs, and both are paid on every call.
That makes Result 2 and Result 4 one knob, how many tokens the description spends on every call. In the thirty-operation run each extra schema carries the same name, description and parameter as its contract line, so the threefold cost there is format alone, and in Result 4 the extra weight is the enum values that let MCP map Europe to EUR on the first call, which you note would bring the per-turn tax with them if pasted into the contract. The run that would separate protocol from format is the REST arm with the same thirty JSON schemas pasted into its contract: if its input tokens land next to the MCP arm, the 35 percent belongs to the format of the description, not to where it sits.
Sorry for the slow reply! I wanted to run all the experiments myself before answering.
You're (absolutely) right, and thanks for actually digging into the results files 😀
Converse is stateless and I don't use prompt caching, so the REST system prompt goes out on every call, exactly like the tool config, "the contract is read once" is wrong for this harness.
So I ran the experiment you proposed: third arm, REST with the same thirty JSON schemas pasted into its contract. All 3 arms ran in the same session, 180 runs, 100% success.
Going from six to thirty operations adds this many input tokens per call:
Same schemas, same cost. REST is even a little more expensive because of the paths. So the 35% belongs to the density of the description, not to the protocol or to where it sits.
That also makes Result 2 and Result 4 one knob, as you said: how many tokens you spend describing the API on every call, and what you get back for them (first-call success on a strict API).
Code and numbers are in the repo (iface_rest_schema.py, RESULTS.md). I'll fix the post too.
1,384 against 1,307 settles it more cleanly than I expected, and the 77 tokens the REST arm pays for its paths is a good detail to have measured rather than assumed.
If density is the knob, the next question is which part of the roughly 960 tokens per call that the schemas add over the terse contract buys the first-call success. Result 4 points at the enum values, the ones that let MCP map Europe to EUR on the first try. A fourth arm with the terse contract plus only those enum lists, and nothing else from the schemas, would show whether a few dozen tokens recover most of the first-call success or whether the descriptions and types around them are doing the work too. That would turn the trade in the post into a curve rather than two points.
I ran it, five arms on the strict scenario, 150 runs, all at 100% final success. I enriched the terse REST contract one step at a time and measured the input tokens of the first call:
So 53 tokens of allowed values take first-call success from 0% to 100%, and everything after that is cost with no measurable return. The types and descriptions around the enums aren't doing the work here. The terse contract plus the enum lists costs the same as the MCP schema, one token apart.
One caveat on the date step: four of my five prompts already give the date as YYYY-MM-DD, and the model converted "October 20, 2026" correctly without being told. So that step doesn't really test the format. A messier date prompt would.
So the curve is closer to a step than a slope: the first few dozen tokens of the right kind buy almost everything. I'd guess that's true of strict APIs in general. Most of a schema is there for the validator, while the enums are what help the model. The code is in strict_curve.py and the numbers are in RESULTS.md in the repo. Thanks for pushing on this
A step rather than a slope is a better result for the post than the curve I expected, and it also says where the curve would bend. The enum lists here are short, three fields with a handful of values each, so 53 tokens buy the whole jump. For a field whose allowed values run into the hundreds, like ISO currency codes or IANA time zones, the list itself becomes the cost, and the trade in the post comes back at the level of one field: send every value on every call, or let the first call fail and pay for a second turn.
Your date caveat is the same point from the other side. The step only shows up where the model cannot already guess the right value, so a field it guesses correctly, like the ISO date here, costs tokens and buys nothing.
I agree on where it bends, but your two examples land on the cheap side. ISO currencies and IANA time zones are long lists, but they're standards the model already knows, so by your date point they cost nothing to leave out.
The expensive case is a list that's long and private, like internal categories or this API's APC instead of APAC. For those I'd put the allowed values in the 400 error, which this backend already does, so you only pay when the model guesses wrong.
180 tests is a serious comparison. Appreciate you sharing the results.
Thanks Jesse, glad it was useful! The harness is open if you want to run it on your own setup
Did the 30-tool run load every definition up front? Some clients, Claude Code among them, defer MCP definitions until a tool is looked up, so that 35% gap may shrink there. The other cost I saw on a hosted server was an output schema inferred from sample rows, about 32K tokens for one tool. I have not tied either to success rate.
Good catch Kevin. Yes, the harness loads all 30 definitions up front on every call, there is no deferred loading or tool search. So the 35% is the worst case. I haven't measured that and it's probably a fair limit of the test. The output schema point is new to me: my server doesn't send any outputSchema, so that cost isn't in my numbers at all. 32K tokens for a single tool is a lot. If you ever tie it to success rate I'd love to read it.