DEV Community

Cover image for Correct is a low bar: a document has to be executable by someone who won't ask
TuringCorp
TuringCorp

Posted on

Correct is a low bar: a document has to be executable by someone who won't ask

Correct is a low bar: a document has to be executable by someone who won't ask

The page came back at seven in the morning. Platform, transfer corridor, Exit B2, fifty metres east after leaving the station. Complete, ordered, and true. Every step was in the right sequence and none of it could be used.

The person waiting for it was seventy-three, arriving by high-speed train from a county town, changing once in a city she had visited twice in her life. The man who ordered the document was her son. He has lived in that city for eleven years and goes through that station several times a week. He had never once thought about which way you turn when you come out.

This post is about the eleven hours between that page arriving and it becoming something a person could actually walk through, and about what we changed in how we read our own output afterwards. Not the model — the document. The part nobody puts in the schema.

What was asked for

The request went in as a paragraph to a phone, not as a form. That matters less than it sounds. What matters is what he put in it.

He told it her age, the route, and the change in the middle. He told it that she reads fine but that words like "transfer" and "Exit B2" mean nothing to her, and that she won't ask anyone either — two years earlier she had asked twice, got two different directions, stood there for half an hour, and in the end followed a woman with a suitcase. He asked for a step-by-step account from getting on the train to getting out of the station. No jargon. Say where to wait, which way to walk, and what to look for before turning. Send it to the inbox by morning.

That last clause is the whole job. Not "draft me some travel tips" — a deliverable with a deadline, a shape, and a recipient. Our agent runs long jobs in the cloud, on our machines, and returns a finished document to an inbox or a chat; nothing is installed and nothing is read off the computer you are sitting at. So the person asking doesn't have to stay at the keyboard while it works. He asked at eleven at night and went to bed.

What came back at 07:00 was, by any reasonable measure, correct. It had understood the journey better than he had: the platform, the transfer corridor, which exit, how far after the exit. He read it once and changed every one of the words that carried the actual information.

"Exit B2" became: when you come down the stairs there's a man selling roast sweet potatoes, go left past him.

"Transfer corridor" became: follow the people with suitcases, five minutes, don't turn off.

Jargon is a compression the reader cannot decompress

"Exit B2" is not vague. It is the most precise string available to anyone who has been in that station. It is short because the reader is assumed to already hold the map — the letters are just the address of a place already in their head. Every technical term works the same way: it is a compression that is lossless only for people who own the dictionary.

The mother does not own the dictionary. To her, "Exit B2" is a string of characters that has to be matched against the world, and the world, standing in a packed concourse at nine in the morning with a suitcase, offers her a great many blue rectangles.

The substitution is not "use simpler words". A simpler word for Exit B2 does not exist, because the thing being described is not a word, it's a location in a visual field. The replacement has to name something the reader will see without being told to look: a man selling roast sweet potatoes, people with suitcases, stairs. It is a different kind of sentence — an instruction anchored in the reader's field of view rather than in the building's coordinate system.

That is why the rewrite had to be done by a person standing in neither place, and could not have been done by the model. Not a limitation to work around — a division of labour. The model had the coverage: fourteen steps in order, the parts he would have forgotten to mention, the escalator question he hadn't thought of. He had the vocabulary: which landmark in that station is stable, which one is only there at eight in the morning, what his mother would actually look at. He printed it on one sheet at the largest type the page allowed.

The failure mode people expect here is a bad draft. The failure mode that actually happened is a good draft — one that was complete enough to stop him from rewriting it and wrong at exactly the words that carry the route. Complete output is more dangerous than thin output, because competence at the shape of a document reads as competence at its content.

"Follow the crowd" and the fork it hides

The night before the trip he got off work and walked the route backwards at the station, from the exit to the platform, sheet in hand.

At the first fork he stopped. The sheet said: follow the crowd. Which is true. It is also exactly wrong at the one place where the instruction does work, because "the crowd" splits two ways there — upstairs is the metro, to the right is the taxi rank. Every person in front of her would be following a crowd. Half of them would be following the wrong one.

"Follow the crowd" is the sentence you write when you have never stood at the fork. It survives an audit by reading. It dies on contact with the place. He changed it to follow the people with suitcases.

This is the part worth taking to any long-running agent that hands work back to a person. Reading the output is not verification, and no amount of the model re-reading it would have caught this. The only verification that finds an ambiguity like that is acting on the instruction in the environment where it will be used. For a route, that means walking it. For a procedure, it means doing it once, on a real object, with the document open. For a document meant for someone else, it means the person who will act on it rehearsing it, not reading it.

We used to say a document was done when it was checked. Now we ask a second question: what is the first step of this document that cannot be carried out by someone who is alone, in a crowd, with no way to ask a follow-up question? Whatever that step is, it is the only step that needed the work.

The last line is the error handler

The draft ended with a line of its own: If you have difficulty, please ask a member of station staff.

He deleted it.

Not because it is bad advice. Because it is a fallback that assumes something about the reader that is not true: she won't ask. Two years earlier she had stood in that same station and asked twice and got two answers, and the experience did not teach her that asking works. An error handler written for a user who will report the error is only an error handler for that user.

Here is what happened instead. Thursday morning he was in a meeting with his phone on silent. When it ended there were three missed calls. He ran downstairs — ten minutes from the office to the station — and the exit was packed and she was sitting there, on the edge of the steps, on her own suitcase, the sheet spread across her knees with an unopened bottle of water holding it down.

She had followed the sheet. It said nothing about what to do if nobody picks up, so she did the one thing that was still safe: she stopped, in a place where she could be found, and stayed there.

He added the line afterwards, at the bottom, after all the steps:

When you get there, stay put, call me, I'll come down and get you.

Every instruction handed to a person acting alone needs a terminal state. Not a general one — a specific one, naming the actor, the action and the place: stay here, do this, wait for me. Written after the fact, from the failure. Which is the only place it could have come from, because the failure branch is exactly the part of the plan that the person who can be called never needs to think about.

That is the honest limit of what a document can be told in advance. The request said "no jargon, say where to wait, which way to walk, what to look for before turning" — it did not say "and what do I do if my son doesn't pick up", because at eleven at night he did not know that was the shape of the hole. He knew it at nine the next morning, holding a phone with three missed calls on it, and the sheet still has that line at the bottom.

What survived

On the way back she asked who wrote the sheet. He said: the agent wrote it, I changed it. She asked whether it was one of those monthly ones. He said you pay for the one time you use it, several times less than the monthly ones, and when you're done you close the page.

She said oh. Then she folded the sheet twice and put it in her bag. He said you don't need that any more, I'll write you a new one. She said she'd keep it. She'd be coming again.

The following Sunday she called to say she had put it under the glass on top of the TV cabinet.

The document that worked was not the one that came back at seven. That one was correct and would have put a seventy-three-year-old in front of a blue rectangle at nine in the morning. The one that worked had three of the model's sentences replaced with landmarks, one ambiguity fixed by walking backwards, and an error handler added by the person who failed. The agent's contribution was the thing a person cannot do alone: hold fourteen steps in order, at one in the morning, without forgetting the escalator.

Both halves were necessary and neither half is impressive on its own. That is most of what we have learned running a long-running agent for people who will not file a bug: the deliverable is not the page, it is the recipient's successful action, and there is no way to check the second thing from inside the first.

Where it sits

CloudPal runs long jobs on our machines and hands back a finished document. Nothing to install, nothing asked of your computer, nothing read from the files you already keep. You describe the job the way you would describe it to a colleague — badly, out of order, including the thing you only remember on the second telling — and the result arrives where you already look, on a schedule, whether or not you are at the keyboard when it lands.

It is in limited preview: new accounts get a dollar of credit, enough to run something real, and the number of new accounts per day is capped. It costs a few times less than the comparable tools and you pay for what you use, which matters most for the documents nobody wanted to write in the first place.

If you are building anything that turns a request into instructions another person has to carry out, the property worth measuring is not whether the output is complete. It is which sentence in it assumes a reader with a fallback path — and whether the reader you actually have possesses one.

CloudPal.TuringCorp.net — create an Agent Pass, sign in, and say what you want done.

Earlier in this series: the same three facts, in a different order, the second half of the draft was policy we never agreed to, it can't read anything on my laptop, a direction is not a spec, four sentences in, seventeen pages out, the requester is not the reader, the first run is a demo, the ninth is the product, the meter, not the price, we deleted the install step.

Top comments (0)