Same correct total, 69% fewer tokens: what a shopping agent spends on WooCommerce

Originally published on the KaliCart blog.

In brief — With the same agent and the same tasks, KaliCart and the public WooCommerce Store API both reached correct delivered totals. The cleanest case is the variant series (M6): both paths verified 20 of 20 quotes, and the same agent still used a median of 281,000 tokens with KaliCart against 533,000 with the Store API (measured, not pre-registered). On tasks with variants, quantity limits, shipping thresholds and two stores, KaliCart got there with 69% fewer tokens, in half the time and with 4 failed calls against 26: an estimated $15 instead of $51 of model cost per 1,000 quote attempts. On simple products neither path showed a clear accuracy advantage. Calls fell 20–29%, short of the 40% we had set ourselves.

Complex tasks, 40 runs	KaliCart	Store API
Tokens per run (median)	215,000	698,000
Verified totals	20/20	17/20
HTTP calls (median)	10	14
Failed calls (total)	4	26
Time per run (median)	39 s	81 s
Estimated model cost per 1,000 quote attempts	about $15	about $51

The question: what does it cost an agent to get the right total?

Agents will keep getting better, and today's shopping tasks will become easy for all of them. What stays is the cost of getting there: tokens, round trips, time and failed calls. At scale that cost is the bill. If an assistant runs a million purchase searches a month, a few cents per task become tens of thousands of dollars.

So we did not ask "can the agent do it?". We asked: with the same agent and the same task, which path costs less to reach a total that the store itself confirms?

The baseline is the WooCommerce Store API, which every WooCommerce store exposes without credentials. It is a fair baseline: products, variations, a cart and shipping rates, all public. An agent can do the whole job with it. The question is how much work that takes. Our previous benchmark used scripted agents; this one uses a language model.

Method

Every run is one fresh agent session with one task, scored against the store's own WooCommerce cart.

The agent. A deliberately minimal tool-using agent: a chat model (DeepSeek deepseek-flash via API, default temperature) with exactly one tool, http_request(method, url, headers, body). No cookie jar: any cart token has to be carried by the agent from response headers. Response bodies are truncated at 30,000 characters. The harness, not the agent, counts calls and tokens.

The two arms. The prompt is identical except for one line:

• K: "Use only the store's KaliCart catalog: start from /wp-json/kalicart/v1/discovery."
• S: "Use only the public WooCommerce Store API: /wp-json/wc/store/v1/ (no credentials), cart and shipping included."

The agent must end with a JSON block: chosen items (product and variation IDs, quantities), destination, the delivered total, and whether that total was read from an API response or computed by the agent.

The stores. Four real WooCommerce stores, anonymized here: a lingerie store (bras sized by band and cup), a mattress and pillow store (sizes, firmness, purchase limits), a popcorn supplies store and a farm food store. Real catalogs, real stock, real shipping zones.

Scoring. For every run a script rebuilds the agent's basket and asks WooCommerce for its delivered total, using the store's own cart and shipping calculation, read-only. A run is verified only if all of these hold:

1. the task's constraints are met, checked against the catalog (size, dimensions, quantity, product type, number of stores);
2. the delivered total, shipping included, is within budget;
3. the agent's total matches WooCommerce's to the cent;
4. the agent says it read the total from an API response. A correct total the agent computed itself counts as deduced, not verified.

Pre-registration. Tasks, stores, number of runs and thresholds were written down and committed before the first run of each series. Every change after that is logged as a deviation with its reason (four in total, listed below). Each task was checked to be feasible with a real quote before it was registered, after one of our early tasks turned out to be impossible.

Results

Across all three series both paths reached correct totals; KaliCart got there with 47–71% fewer tokens and 20–29% fewer calls.

Series	Tasks	Verified K / S	Calls, median K / S	Tokens, median K / S	Threshold verdict
M5	2 simple products, single cart, 3 stores (15 + 15 runs)	14 / 12	8 / 10 (20% fewer)	107k / 373k (71% fewer)¹	Accuracy gain: no. Calls ≥40% fewer: no
M6	variants: sizes, dimensions, shipping counted, 2 stores (20 + 20)	20 / 20	10.5 / 14 (25% fewer)	281k / 533k (47% fewer)¹	Accuracy gain: no. Calls ≥40% fewer: no
M7	quantity limits, shipping threshold, missing size, two stores (20 + 20)	20 / 17	10 / 14 (29% fewer)	215k / 698k (69% fewer)	Tokens ≥40% fewer: yes. Accuracy not worse: yes. Calls ≥40% fewer: no

¹ Measured but not a registered measure in M5 and M6. In M5 the temperature was 0, so runs of the same task are close to copies of each other.

M5: simple products

Two products in one cart, three stores, 30 runs. Both paths were essentially even on accuracy (14 vs 12 verified). The pre-registered thresholds asked for at least 3 more verified runs and 40% fewer calls: neither was met. On the store with only simple products the Store API needed slightly fewer calls than KaliCart (8 vs 9).

M6: variants

Four tasks, 40 runs: two black briefs of the same model in sizes 2 and 3 plus a bra; one brief under the free-shipping threshold, so shipping counts against a 55 € budget; a single mattress 80×190 with two identical pillows; a double mattress 160×190 with a topper of the same size. Both paths verified 20/20. With the Store API the agent failed 24 calls, mostly 401s from adding to the cart without the nonce. KaliCart's 23 failed calls were all 429s from our own catalog rate limiter, in two runs at the end of a day of traffic from one IP.

M7: complex tasks

Four tasks, 40 runs, tokens registered as the primary measure before the first run:

Task	Tokens, KaliCart vs Store API
Four identical pillows (some models allow at most 2) + a double mattress, 650 €	63% fewer
Two different pillows, 85 € shipping included (shipping is free only from 80 € of goods)	69% fewer
A bralette in a size that does not exist, closest size allowed, + a brief	67% fewer
One item from each of two stores, two orders, 120 € in total	76% fewer

The three Store API runs that were not verified: one picked a more expensive mattress and ended at 687.41 € on a 650 € budget; two had the right total but computed it themselves instead of reading it from the store.

Cost. The 40 M7 runs used $1.31 of DeepSeek credit. Split by tokens, that is about $0.015 per KaliCart run and $0.051 per Store API run: roughly an estimated $15 vs $51 per 1,000 quote attempts, about 71% less. The test stops at the quote and creates no order; failed attempts use credit too. The split assumes one price per token; providers that discount repeated prompt text discount both arms.

What the runs taught us

The benchmark was also a bug report. Every place where the agent wasted calls on our side became a fix or an open item.

Found in	What the agent hit	What we did
An external agent test	Quoting a variable product without a variation failed with no hint	1.0.159: the quote answers variation_required with the product's variations (ID, attributes, price, availability)
An external agent test	/catalog/categories?per_page=50 returned 400	1.0.160: categories and meta accept every parameter valid on the other catalog endpoints; unknown ones are still rejected
Preparing M7	A quantity above stock returned a bare item_not_quotable, while the Store API publishes add_to_cart.maximum for the same product. Here the Store API told the agent more than we did.	1.0.160: one reason per line (out_of_stock, quantity_not_available, sold_individually…) with line, product, variation, next_step and max_quantity, the same maximum WooCommerce applies to its cart
M6	23 429 KALICART_CATALOG_RATE_LIMITED in two runs; the wait time is only in the Retry-After header, which our own harness was dropping	Open: put retry_after in the JSON body too, and measure the default limits against real agent traffic before changing them
M7	color=nero (Italian for black) rejected with 400: the filter vocabulary is English-only	Open: evaluate localized color values

What the quote now returns when the quantity is the problem:

{
  "status": "unknown",
  "reason": "quantity_not_available",
  "line": 0,
  "product_id": 1568,
  "variation_id": null,
  "max_quantity": 19,
  "next_step": "Not enough stock for this quantity: lower it to max_quantity or choose another product."
}

Exact stock in the catalog stays private unless the merchant chooses to show it; the quote only says how many the cart would accept, which is what the store's own cart says too.

What this does not show

• One model, one harness. All automated runs used DeepSeek deepseek-flash with a one-tool agent. Another model, or an agent with a browser or a built-in WooCommerce client, may close part of the gap. A capable model already gets the right total through the Store API almost every time; the difference we measured is the effort to get there.
• We measured our own product. KaliCart is ours, and the baseline is the public alternative. That is why the protocols, the harness and every run are public: an independent replication is the test that matters most.
• Four stores, Italian catalogs. Real stores and real data, but a small sample. Catalog size, plugins and shipping setup change the numbers.
• Calls missed our own bar. We set 40% fewer calls as a threshold in advance and measured 20–29%. Tokens moved much more than calls because KaliCart responses are shorter and fewer steps mean less conversation history resent to the model.
• The agent handles the cart session itself. With one generic HTTP tool, the agent has to carry cart tokens and nonces; many Store API failures in M6 were 401s for that reason, while our 429s count against KaliCart. That is the right test for a generic HTTP agent. A replication with a client that manages the session automatically would show how much of the saving comes from the shape of the data and the direct quote.
• Cost is an estimate. Dollars per 1,000 quote attempts come from the real credit used, split by tokens at one price per token.
• Our rate limiter is part of the result. Two M6 runs were slowed by it. We report them as they happened, not removed. Without them the M6 call saving is 29% instead of 25%, and that number is descriptive only.
• Deviations. Four changes after registration, each logged with its reason; none changes a threshold verdict: one infeasible budget raised (the cheapest product cost more than the budget), one run stopped by an over-cautious cost cap in our harness, one scorer fix (a padded triangle bra was not recognized as a bra), and one batch moved forward by a few hours.

Try it

• On a store: KaliCart Bridge is free on wordpress.org. Once active, an agent starts from https://<store>/wp-json/kalicart/v1/discovery; the delivered-total quote is POST /wp-json/kalicart/v1/checkout/quote with items and a destination. It creates no order and writes nothing.
• Reproduce the comparison: the harness (a one-tool agent, the scorer and a run script), the protocols and the anonymized run data are on GitHub: giuseppesocci-bot/kalicart-agent-bench. The harness refuses to run unless you confirm you own the stores, and it paces itself (at least 1 s between requests to a store, Retry-After honoured, at most 10 runs per store per day).
• Tell us where your agent wastes calls. Every fix above came from watching an agent fail. If yours hits something on our side, open an issue and we will measure it.