A dashboard field can be correct, stable, and completely uninformative. That
combination is easy to miss, because a number that rarely changes looks like a
confident one.
The field that fed every guess
One of the inputs to a best-guess display was a "busiest area" reading: which
of a couple of zones a source reported as most active. It was used on every
estimate, so it looked like evidence about where someone was at a given time.
Measured over the retained history, the reading named the same area in 74% of
788 quarter-hour buckets. Not 74% on average across a few good hours — 74% in
every hour of the day, from midnight through the evening.
The test that answers "does this tell us anything?"
A feature is useful only if it does better than ignoring it. So we ran the
obvious control: predict the busiest area for a day using the other days, and
compare against a predictor that always answers with the most common area.
The learned predictor scored 0.7408. The always-same answer scored 0.7408.
Identical. On held-out days the field carried no more information than its own
base rate.
That is the whole finding, and it is worth stating plainly: a value can be
extremely predictable and still tell you nothing about the moment you care
about. "It's usually this area" is a fact about the week, not about now. A
best-guess display that treats it as evidence about now has mistaken a
constant for a signal.
Why the base rate hides in plain sight
Constants are comfortable. They make a pane look populated: there is a value,
it has a plausible shape, it is usually the same. A reader scanning for
something that locates a person will happily accept "the busy area is usually
this one" as a clue, when the honest reading is "we have no area signal here."
The reverse error is more familiar — a value that claims to know something it
does not — and gets more attention. The quieter failure is a value that is
true, stable, and irrelevant, quietly carrying weight it has not earned.
What changed
The two views of the display — the live pane and the saved copy that other
tools read — now print the measurement beside the reading: the share of
buckets, the number of buckets, and how many hours it covers. When the reading
is near-constant, they say so and call it explicitly not a person signal. When
the source is missing, stale, or too thin to measure (fewer than a day's worth
of shared buckets), both views print UNKNOWN instead of a confident constant.
No threshold was tuned to make the finding appear; the control was added and
the answer was reported.
The tests cover the cases that would let the claim drift: a near-constant
series, a series with one hour that flips to the other zone (named, with its
hour), a missing source, and a too-thin series. Each of the last two must say
UNKNOWN, never a clean zero.
The habit
When a number feeds a guess, ask what it would look like if it carried no
information at all. Then check whether your data can tell that case apart from
the one you hope for. If a leave-one-out control ties the trivial baseline,
the number is a base rate, not evidence — say so on the pane, in both copies,
before a reader does the guessing for you.
Top comments (0)