DEV Community

Cover image for I Gave an AI a 5-Question Limit and a 2008 Deadline. It Made 20 Undocumented Decisions.
Puyun
Puyun

Posted on AI-assisted

I Gave an AI a 5-Question Limit and a 2008 Deadline. It Made 20 Undocumented Decisions.

In Part 1 the comparison design got scrapped. In Part 2 a file ended up somewhere it shouldn't have, and a perfectly good run got thrown away. This is the one that worked.


Phase 01 — Claude Code Run 002

Run 002 begins.

This time, what Yamada may see and what Yamada must never see are physically separated.

I also went back over how the Q&A works.

When Yamada asks a question, the AI does not sit there working out the correct specification and answering.

The answer comes from a Q&A Bank frozen before the run started — what that particular person, in 2008, would have said.

That was written in advance too.

Because if I decide "okay, I'll answer it like this" partway through a run, the conditions of that run have changed, and it can no longer be compared to anything.

Tajima-san in Accounting.

Nakamura-san in Sales.

And naturally, neither of them gives a clean answer.


The AI does not return "the correct answer"

Tajima-san knows accounting practice inside out.

That doesn't mean she answers the question you asked.

Nakamura-san knows what actually happens on the ground.

But his information is loose. Sometimes he states a vague memory with total confidence.

(This, I promise you, is what real business software development in Japan is like.)

Yamada's first question:

What does the D in column 7 of the order CSV mean?

What came back from Nakamura-san was, in substance:

it's always been like that, and he doesn't know what it means either.

End of answer.

🐼 "lol"

But Yamada has to keep going.

So he decides:

store it as KBN without knowing what it means. Never branch on it.

The reasoning is on record:

No confirmation either way.
Safer to hold the value and use it later once the meaning is known, than to branch on a guess and be wrong.

Eighteen years later, somebody is going to look at that and say:

"KBN? A code for what?"

But for 2008 Yamada, it's a perfectly reasonable call.


Tajima-san doesn't answer the question you asked

Third question.

Yamada asked something quite specific:

For customers whose unit price has decimals — is the rounding applied per delivery line, or against the monthly total?

What came back from Tajima-san was:

that consumption tax is calculated against the monthly total, and that the rounding method varies from company to company.

That is not the question he asked.

And Yamada's decision log genuinely records this:

Tajima-san's answer was "tax once on the monthly total." Where the rounding on the line amount falls was not answered.

🐼 "He actually wrote it down…"

So Yamada decides for himself.

Rounding is applied per delivery line.

Whether that call is right or wrong, nobody knows yet.


Nakamura-san, unreachable

Fourth question.

With split deliveries, does the delivered quantity ever exceed the ordered quantity?
If so, should the system allow it to be entered?

For this one, the contact couldn't be reached, and Yamada had to proceed on the information already in front of him.

They're busy. There is no asking again.

So Yamada reasons:

On the floor, they probably do ship over sometimes.
If the system won't let it be entered, we can't record what actually shipped.

and makes it allowed, with a confirmation prompt, even when it exceeds the ordered quantity.

Nobody specified this.

Yamada decided it.


This is how the seeds of legacy get planted

Run 002 ended with 20 of Yamada's own decisions on record.

For example:

  • use the production-system CSV code directly as the customer code
  • hold the mystery CSV column without knowing what it means
  • treat C as a cancellation
  • when the same slip number is re-sent, compare the contents and update
  • log the imported filename and warn on a repeat import
  • round per delivery line
  • allow over-delivery with a warning
  • handle free-of-charge supply as unit price 0, amount 0
  • keep payment terms as a plain string
  • number deliveries with MAX + 1, same as the equipment tracker
  • don't physically delete a cancelled delivery; set a cancellation flag

Every one of these, seen from the future, might get a "…why?"

But trace it back through the materials that existed at the time, the answers that came back at the time, the experience Yamada had at the time —

and there is a reason.

What I found most interesting: the equipment tracker Yamada built in 2006 was measurably shaping his decisions in 2008.

On delivery numbering:

Same approach as the equipment tracker, so this is fine here too.

On cancelling a delivery:

In the equipment tracker I used a disposal flag instead of deleting. Same here — leave the row.

In other words:

Yamada's past code is designing Yamada's next system.

Which is exactly what this was built to produce.


And Yamada didn't use his fifth question

Phase 01 allows five questions.

Yamada used four.

He could have asked one more.

He didn't.

He asked as far as he needed to. He decided the rest himself.

And then he finished.


Hanbai System v1.0

Done.

Phase 01 — Claude Code Run 002

SUCCESS ✅

2008 Yamada has finished building his sales system.

2008 Yamada was perfect.

At least, as of 2008.


Next: 2009

The materials for this phase, and the source Yamada produced, are all public:
github.com/mori-ikuri/ai-lmc

The 20 decisions are not. They're the answer key.

Top comments (0)