There is a category of defect that cannot appear in your acceptance criteria. Not because
nobody thought of it, but because the shape of a requirement has no room for it.
A requirement describes a state and a rule. A customer can apply a valid promo code at
checkout. State: the code is valid. Rule: it is accepted. Both are evaluated at a single
instant, because a sentence has one tense.
Real systems do not have one instant. They have two, and sometimes a lot more.
The gap between checking and using
Take that promo code. The system validates it when the customer types it into the basket. The
system commits it when the customer pays. Between those two events sits an unbounded amount
of time — thirty seconds if they have their card handy, three days if they leave the tab open
on a laptop lid.
If the code expires in that gap, what happens?
The requirement cannot tell you. It never contemplated a gap, because it was written as one
sentence about one moment. And a test written by hand almost certainly cannot tell you either,
because a person writing a test naturally writes it the way they would perform it: enter code,
assert accepted, pay, assert charged. Three lines, one instant, no gap.
This is time-of-check to time-of-use. Most developers first meet it as a security problem —
access() then open(), and a symlink swapped in between. The same shape appears at business
timescale, and there it is far more common and far less discussed:
- Stock is reserved at basket, decremented at dispatch. Someone else buys the last one.
- A permission is checked when the page loads, enforced when the action fires. The role changed.
- A price is quoted at quote time, charged at renewal. The tariff moved.
- A rate limit is checked at admission, consumed at execution. The window rolled over.
- A feature flag is read at session start, branched on at submit. Someone flipped it.
- A token is validated at the gateway, used by a downstream call. It expired in flight.
Every one of those is a real defect class, and not one of them will be written into a
requirement, because in each case the requirement is correct. It is the world that has two
timestamps.
Why it survives to production
It survives because it passes every test anybody wrote, and because it is not reproducible on
a developer's machine at a developer's speed. You cannot see it by clicking through the flow,
because you click through the flow in eleven seconds.
It also survives because the failure is usually silent and plausible. The customer gets
charged full price and does not notice. The order ships without the discount and support
handles it as a one-off. Nothing goes red. You find out from a pattern in refunds, three
months later, and by then the pattern is the only evidence.
Testing for it deliberately
The technique is not complicated. It is just not what a hand-written test looks like, and you
have to decide to do it.
Split the transaction in your test. Instead of one continuous flow, make the two moments
explicit and put a mutation between them:
# not this
apply_code("SAVE10")
assert basket.discount == 10
pay()
assert charge == 90
# this
apply_code("SAVE10")
assert basket.discount == 10
expire_code("SAVE10") # move the world, not the clock in your test runner
result = pay()
assert result.status == "revalidated"
assert charge == 100 # or whatever you decided it should be
The important line is the third one. To write it you need the ability to change state
between two calls — which usually means a test seam your system does not have yet, and
which is the real reason this class goes untested. Adding it is the work.
Then pick the case that hurts. For each two-phase interaction, ask what changes in the
gap, and who changes it:
| Who moves the world | Example |
|---|---|
| The clock | the code, token or quote expires |
| Another user | the last unit of stock is bought |
| An administrator | a role, a flag or a price is changed |
| The system itself | a batch job settles or reconciles |
Four questions per interaction, not thirty. It is a tractable list once you accept it exists.
What to do about it in the design
Testing tells you it is broken. Fixing it is a design decision, and there are only really
three options:
- Re-check at commit. The honest default. Costs a round trip and can surprise the user at the worst moment, which is why it needs a typed result and a real UI state — not an exception that renders as "something went wrong".
- Hold the promise. Reserve the stock, lock the price, honour the code you already accepted. Correct from the customer's point of view and expensive from the business's, and it needs an expiry of its own or the reservations pile up.
- Make it atomic. Collapse the two moments into one so there is no gap. Usually only available for interactions inside a single system.
The one option that is never right is the accidental one: check at entry, commit at payment,
and never state which of the three you chose. That is not a design, it is a coin flip
resolved by whoever is unlucky.
The uncomfortable bit
If you have never tested this class, you almost certainly have some. Not because your team is
careless — because a requirement is a sentence, a hand-written test mirrors the requirement,
and neither has a second timestamp in it.
Pick your most valuable two-phase interaction. Write one test that changes the world in the
middle. See what happens.
*Disclosure: this post is published by 2SD Technologies, where we build TAI — a testing platform that generates cases from requirements, user stories and the application, including this category. *
Top comments (0)