Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
Your runner passes the system prompt and the tool configuration into every Converse call, and the REST column shows it: order_customer takes 4,500 input tokens over three model turns against 2,850 for order_status over two, so the third turn costs about 1,650 tokens, about as much as each of the first two. A contract read once could not do that.
So the gap in Result 2 reads as a difference in density rather than in placement. Going from six operations to thirty adds 422 input tokens per call on the REST side and 1,307 on the MCP side, the same on every task whose number of turns did not change between your small and large results files. A tool schema costs roughly three times what a contract line costs, and both are paid on every call.
That makes Result 2 and Result 4 one knob, how many tokens the description spends on every call. In the thirty-operation run each extra schema carries the same name, description and parameter as its contract line, so the threefold cost there is format alone, and in Result 4 the extra weight is the enum values that let MCP map Europe to EUR on the first call, which you note would bring the per-turn tax with them if pasted into the contract. The run that would separate protocol from format is the REST arm with the same thirty JSON schemas pasted into its contract: if its input tokens land next to the MCP arm, the 35 percent belongs to the format of the description, not to where it sits.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
Sorry for the slow reply! I wanted to run all the experiments myself before answering.
You're (absolutely) right, and thanks for actually digging into the results files 😀
Converse is stateless and I don't use prompt caching, so the REST system prompt goes out on every call, exactly like the tool config, "the contract is read once" is wrong for this harness.
So I ran the experiment you proposed: third arm, REST with the same thirty JSON schemas pasted into its contract. All 3 arms ran in the same session, 180 runs, 100% success.
Going from six to thirty operations adds this many input tokens per call:
terse REST: 422
MCP: 1,307
REST with the schemas: 1,384
Same schemas, same cost. REST is even a little more expensive because of the paths. So the 35% belongs to the density of the description, not to the protocol or to where it sits.
That also makes Result 2 and Result 4 one knob, as you said: how many tokens you spend describing the API on every call, and what you get back for them (first-call success on a strict API).
Code and numbers are in the repo (iface_rest_schema.py, RESULTS.md). I'll fix the post too.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
1,384 against 1,307 settles it more cleanly than I expected, and the 77 tokens the REST arm pays for its paths is a good detail to have measured rather than assumed.
If density is the knob, the next question is which part of the roughly 960 tokens per call that the schemas add over the terse contract buys the first-call success. Result 4 points at the enum values, the ones that let MCP map Europe to EUR on the first try. A fourth arm with the terse contract plus only those enum lists, and nothing else from the schemas, would show whether a few dozen tokens recover most of the first-call success or whether the descriptions and types around them are doing the work too. That would turn the trade in the post into a curve rather than two points.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
I ran it, five arms on the strict scenario, 150 runs, all at 100% final success. I enriched the terse REST contract one step at a time and measured the input tokens of the first call:
terse: 730 tokens, 0% first-call success
- enum lists only (metric, region, granularity): 783 tokens, 100%
- date format: 795 tokens, 100%
- full JSON schema: 895 tokens, 100%
MCP: 784 tokens, 100%
So 53 tokens of allowed values take first-call success from 0% to 100%, and everything after that is cost with no measurable return. The types and descriptions around the enums aren't doing the work here. The terse contract plus the enum lists costs the same as the MCP schema, one token apart.
One caveat on the date step: four of my five prompts already give the date as YYYY-MM-DD, and the model converted "October 20, 2026" correctly without being told. So that step doesn't really test the format. A messier date prompt would.
So the curve is closer to a step than a slope: the first few dozen tokens of the right kind buy almost everything. I'd guess that's true of strict APIs in general. Most of a schema is there for the validator, while the enums are what help the model. The code is in strict_curve.py and the numbers are in RESULTS.md in the repo. Thanks for pushing on this
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
A step rather than a slope is a better result for the post than the curve I expected, and it also says where the curve would bend. The enum lists here are short, three fields with a handful of values each, so 53 tokens buy the whole jump. For a field whose allowed values run into the hundreds, like ISO currency codes or IANA time zones, the list itself becomes the cost, and the trade in the post comes back at the level of one field: send every value on every call, or let the first call fail and pay for a second turn.
Your date caveat is the same point from the other side. The step only shows up where the model cannot already guess the right value, so a field it guesses correctly, like the ISO date here, costs tokens and buys nothing.
Edoardo Dusi is a Senior Developer Advocate at AWS, part of the EMEA Developer Relations team. He spent years building web platforms and leading teams before moving into DevRel.
I agree on where it bends, but your two examples land on the cheap side. ISO currencies and IANA time zones are long lists, but they're standards the model already knows, so by your date point they cost nothing to leave out.
The expensive case is a list that's long and private, like internal categories or this API's APC instead of APAC. For those I'd put the allowed values in the 400 error, which this backend already does, so you only pay when the model guesses wrong.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Your runner passes the system prompt and the tool configuration into every Converse call, and the REST column shows it: order_customer takes 4,500 input tokens over three model turns against 2,850 for order_status over two, so the third turn costs about 1,650 tokens, about as much as each of the first two. A contract read once could not do that.
So the gap in Result 2 reads as a difference in density rather than in placement. Going from six operations to thirty adds 422 input tokens per call on the REST side and 1,307 on the MCP side, the same on every task whose number of turns did not change between your small and large results files. A tool schema costs roughly three times what a contract line costs, and both are paid on every call.
That makes Result 2 and Result 4 one knob, how many tokens the description spends on every call. In the thirty-operation run each extra schema carries the same name, description and parameter as its contract line, so the threefold cost there is format alone, and in Result 4 the extra weight is the enum values that let MCP map Europe to EUR on the first call, which you note would bring the per-turn tax with them if pasted into the contract. The run that would separate protocol from format is the REST arm with the same thirty JSON schemas pasted into its contract: if its input tokens land next to the MCP arm, the 35 percent belongs to the format of the description, not to where it sits.
Sorry for the slow reply! I wanted to run all the experiments myself before answering.
You're (absolutely) right, and thanks for actually digging into the results files 😀
Converse is stateless and I don't use prompt caching, so the REST system prompt goes out on every call, exactly like the tool config, "the contract is read once" is wrong for this harness.
So I ran the experiment you proposed: third arm, REST with the same thirty JSON schemas pasted into its contract. All 3 arms ran in the same session, 180 runs, 100% success.
Going from six to thirty operations adds this many input tokens per call:
Same schemas, same cost. REST is even a little more expensive because of the paths. So the 35% belongs to the density of the description, not to the protocol or to where it sits.
That also makes Result 2 and Result 4 one knob, as you said: how many tokens you spend describing the API on every call, and what you get back for them (first-call success on a strict API).
Code and numbers are in the repo (iface_rest_schema.py, RESULTS.md). I'll fix the post too.
1,384 against 1,307 settles it more cleanly than I expected, and the 77 tokens the REST arm pays for its paths is a good detail to have measured rather than assumed.
If density is the knob, the next question is which part of the roughly 960 tokens per call that the schemas add over the terse contract buys the first-call success. Result 4 points at the enum values, the ones that let MCP map Europe to EUR on the first try. A fourth arm with the terse contract plus only those enum lists, and nothing else from the schemas, would show whether a few dozen tokens recover most of the first-call success or whether the descriptions and types around them are doing the work too. That would turn the trade in the post into a curve rather than two points.
I ran it, five arms on the strict scenario, 150 runs, all at 100% final success. I enriched the terse REST contract one step at a time and measured the input tokens of the first call:
So 53 tokens of allowed values take first-call success from 0% to 100%, and everything after that is cost with no measurable return. The types and descriptions around the enums aren't doing the work here. The terse contract plus the enum lists costs the same as the MCP schema, one token apart.
One caveat on the date step: four of my five prompts already give the date as YYYY-MM-DD, and the model converted "October 20, 2026" correctly without being told. So that step doesn't really test the format. A messier date prompt would.
So the curve is closer to a step than a slope: the first few dozen tokens of the right kind buy almost everything. I'd guess that's true of strict APIs in general. Most of a schema is there for the validator, while the enums are what help the model. The code is in strict_curve.py and the numbers are in RESULTS.md in the repo. Thanks for pushing on this
A step rather than a slope is a better result for the post than the curve I expected, and it also says where the curve would bend. The enum lists here are short, three fields with a handful of values each, so 53 tokens buy the whole jump. For a field whose allowed values run into the hundreds, like ISO currency codes or IANA time zones, the list itself becomes the cost, and the trade in the post comes back at the level of one field: send every value on every call, or let the first call fail and pay for a second turn.
Your date caveat is the same point from the other side. The step only shows up where the model cannot already guess the right value, so a field it guesses correctly, like the ISO date here, costs tokens and buys nothing.
I agree on where it bends, but your two examples land on the cheap side. ISO currencies and IANA time zones are long lists, but they're standards the model already knows, so by your date point they cost nothing to leave out.
The expensive case is a list that's long and private, like internal categories or this API's APC instead of APAC. For those I'd put the allowed values in the 400 error, which this backend already does, so you only pay when the model guesses wrong.