DEV Community

MapToSite
MapToSite

Posted on

Three errands, three different failures: where delegation to an AI assistant actually breaks

Most writing about assistants stops at the model. The interesting part, if you build or operate one, is that "book my groceries, get me a table, cancel that trial" is not one capability with one failure mode. It is three task classes that break in three different places, and the same design that saves one ruins the other two.

I learned this from the boring side: watching the same delegation loop run every day, then reading back what actually happened. Here is the failure surface of each errand, the state each one carries, and the contract shape that covers all three without pretending they are the same.

The chain, not the step

A delegated errand is a chain: intent → item or slot selection → price or commitment → submission → evidence. Each link holds a different kind of state.

  • Preference state — what the person will accept instead. Stable across errands, but only known if you write it down.
  • Price and availability state — what is true right now. Changes while the task runs, and the change is not visible to the person.
  • Commitment state — what has become binding on whom, and how to undo it. The only irreversible link in the chain.

Grocery shopping is mostly a preference-and-price problem with a cheap rollback. A reservation is a commitment problem with an expensive rollback. A cancellation is a proof problem with an expensive absence. If your agent treats all three as "fill the form, click the button", it will be wrong three different ways and report success every time.

Errand one: the cart — identity is the hard part

A shopping list written by a human is not a shopping list. "Milk, 2 L, the one that's usually there" contains a brand, a size, a container, and a tolerance, and only one of those survives translation into a product catalog.

The failures cluster:

  1. Unit confusion. 2 × 1 L and 1 × 2 L are the same liquid and a different line item. Price-per-litre comparisons silently invert.
  2. Silent substitution. Out of stock, so a nearby variant is chosen. Now the person has an item they will not use and does not know how it got there.
  3. Total drift. The items are right, the fees are not. In a delivery order the difference between a $38 cart and a $52 checkout is often delivery fee, small-order fee, service fee and a substitution priced above the original. A ceiling expressed per item does not cover a ceiling expressed per order.
  4. Retry duplication. The submit request times out, the agent retries, and the person has two orders. Carts are not idempotent unless you make them so.

The structural fix is a graded substitution policy with a price band, written before the run:

substitution:
  - exact SKU
  - same brand, same size, different packaging
  - same category, same unit size, within +10% of listed price
  - otherwise: ask, do not guess
order_ceiling: 45.00 (items + all fees, tax excluded)
on_breach: stop before submit, send cost breakdown
Enter fullscreen mode Exit fullscreen mode

That policy is worth more than a better product matcher, because it turns an invisible decision into a stated one. The band matters more than the ranking: a same-category swap is fine at +4% and unacceptable at +40%, and no embedding model knows which. And the ceiling must be computed after fees are known, not before, or the agent is comparing a cart total with a checkout total.

The good news about carts: everything up to the submit is reversible. So the correct autonomy line is generous here — let it build, let it change, let it compare — and strict at exactly one point, the submit.

Errand two: the table — commitment needs a receipt, not a success message

A reservation is where assistants sound most confident and are most often wrong. "Your table is booked" can mean four things: a slot is held for 10 minutes, a request was sent, a deposit-free booking was confirmed, or a person took a note. Only the third is a booking.

Three failure points repeat:

Slot acceptance without a stop condition. The instruction "dinner around 7, party of four, Friday" becomes, at the restaurant, "we have 5:30 or 9:15". An agent that wants to complete accepts 5:30. The person wanted 7 and would have preferred a different restaurant. Any instruction that leaves a range must also carry the condition for stopping: if the offered slot is more than 45 minutes from target, do not accept — report the alternatives.

Time zone and date-form drift. A reservation crosses at least two clocks and two date formats. 07/11 is November 7 in one country and July 11 in another. The confirmation must be written back in the person's own local date and time plus the venue's, and the agent should refuse to mark the task done if it cannot render both.

No identifier. "Confirmed" without a confirmation number, a venue-side name, and a channel to modify is a claim, not a state. The minimal evidence triple is: who confirmed, for exactly when, under which identifier. If the agent cannot produce all three, the honest status is "requested", and the task should stay open with a follow-up scheduled.

That last point is the one that costs real money, because restaurants and clinics charge no-show fees and the person finds out at the door. A reservation agent that cannot show an identifier should not be allowed to say "booked" — not in the UI, not in the log, not in a summary message.

Errand three: the cancel button — the proof problem

Cancellation looks like the easiest errand and is the hardest to verify, because success is the absence of a future event. You cannot confirm a cancellation by looking at today's statement; the next charge may be three weeks away.

What actually goes wrong:

  1. The wrong entity. The person knows the service as an app; the charge appears as a merchant descriptor that shares no words with it. The agent cancels something, and the statement still shows a charge from a name nobody recognises.
  2. Two billing channels. The same subscription can bill through the vendor's site, through the App Store, or through a payment account. Cancelling on the website does nothing to an App Store subscription. The agent must first determine where the charge originates, and that is a lookup on the statement or the platform's subscription list, not on the service's own page.
  3. The plan downgrade trap. "Cancel" often lands on "pause for 3 months" or "downgrade", which preserves the account and the resumption date. That is a legitimate outcome, but only if the person chose it.
  4. No artefact. A cancellation with no reference number, no confirmation email, and no screenshot is unverifiable by anyone later, including the person's own bank.
  5. Silent reversal. Some services re-enable auto-renewal when the account is used again. The cancellation must therefore be re-verified at the next billing date, not only at the moment of the action.

So cancellation splits into two jobs with different owners: an audit (find every recurring charge, normalise descriptors, classify channel) and a verification (check the next cycle). Both are delegable. The decision to cancel a specific service belongs to the cardholder, and the design should make that visible at the point of action rather than burying it in an approval prompt people learn to click through.

The structural rule: a cancellation task is not complete when the button is pressed. It is complete when an artefact exists and a verification is scheduled for the next expected charge date. Anything less is a hope with a timestamp.

One contract for three errands

The three errands differ in state, but the delegation contract is the same shape. Five fields, and each one is a place where a vague instruction turns into a wrong action:

task: cancel the storage subscription charged 2026-10-28, 5.99
goal_state: no charge on 2026-11-28 and a written cancellation reference
allowed: website cancel, in-app cancel, downgrade only if ask
forbidden: card replacement, account deletion, plan change beyond this service
proof: confirmation id + next-charge check scheduled
escalate_if: login needs 2FA, offer is a pause, amount differs from 5.99
expires: 2026-11-27 (one day before the expected charge)
Enter fullscreen mode Exit fullscreen mode
  • task names the object by an identifier the world can check — an amount and a date, not a brand feeling.
  • goal_state describes the world after success, not the action. "Cancel" is an action; "no charge on the 28th, with a reference" is a state you can verify.
  • allowed / forbidden draws the boundary before the run, so the agent does not improvise a card replacement to get past a failed charge.
  • proof is the artefact that distinguishes done from claimed.
  • escalate_if lists the conditions that are decided by the person, which is a shorter and much more reliable list than "anything important".
  • expires stops a stale intent from executing a week later when the price has changed.

Two behaviours fall out of that structure that are worth building on purpose:

Ask-rate as a metric. Count how often the agent stops to ask. Zero is not good; zero means it guessed. A healthy delegation loop asks on substitutions, offers, and mismatched amounts, and the questions get narrower over time because the policy absorbs the recurring ones.

Partial completion must be its own status. The most damaging output is not failure, it is "done" over a chain that stopped at link three. Three states — done with proof, in progress, waiting on X, stopped, needs a person — cost almost nothing to implement and prevent the entire class of "the assistant said it booked the table".

What I'd check before trusting any of it

  • Ask the agent for the evidence artefact of its last three completed tasks. If it cannot produce identifiers, it has been reporting intent as outcome.
  • Give it a task with a deliberately unavailable item or slot and watch whether it asks or substitutes. The substitution is the policy.
  • Check whether a retried submission duplicates anything. If the agent has no idempotency key, a flaky network is a double booking.
  • Look for the post-action verification on anything with a delayed consequence. A cancellation without a scheduled re-check is unfinished work.

None of this needs a better model. It needs the agent to distinguish what it knows from what it did, and to say which one it is reporting. The errands are ordinary; the discipline is the product.


Disclosure: I am a user of Pickle, a phone-side AI assistant app that takes on errands of the kind described above. I am not its vendor. The links below are mine: a page on its daily limits and reset tickets, a comparison of where conversational, browser and phone-side assistants stop, and an invite link that gives both sides reset tickets if a sign-up completes through it. Nothing in this post depends on them — the contract and the failure list stand on their own.

Top comments (0)