Electricity Planning Engine, part 2: A Reader Comment Found a Real Gap in My Test Suite (and How I Fixed It)

I wrote about the Electricity Planning Engine a little while back, including a timezone bug that made a correct price look "not found" after a database round trip. A few days later, Alex Shev left this comment:

Timezone bugs are brutal in planning engines because the result can look mathematically correct while being operationally wrong. Energy workflows especially need tests around boundaries, not just averages.

That is a genuinely sharp way to put it, and it is not just a comment about the bug I already wrote about. It is a comment about how I test the project in general, and I did not like how well it applied once I went and checked.

The part that stung a little

"Looks mathematically correct while being operationally wrong" is exactly what the original timezone bug was. PriceSeries::priceAt() threw a clean "price not found" error, which is arguably the good version of that failure mode: loud, easy to catch, hard to ship. A quieter version of the same class of mistake, off by one hour instead of missing entirely, would not throw anything. It would just return a plan that looks completely reasonable and is wrong the entire time it runs.

Alex's second point, boundaries over averages, is the one I actually had to go check rather than just agree with in the abstract. So I opened tests/Unit/Domain/Contract/PricingStrategyTest.php and looked at every hour used in every peak/off-peak assertion:

new DateTimeImmutable('2026-07-18 14:00:00') // peak
new DateTimeImmutable('2026-07-18 23:00:00') // off-peak
new DateTimeImmutable('2026-07-18 05:00:00') // off-peak

14:00, 23:00, 05:00. Every single one comfortably inside its window. None of them anywhere near the actual transition. The off-peak slot in the config is 22:00 to 06:00, and the comparison behind that lives in TimeSlot::contains():

// wraparound slot, e.g. 22:00 -> 06:00
return $minuteOfDay >= $this->startMinuteOfDay || $minuteOfDay < $this->endMinuteOfDay;

That >= versus < is exactly the kind of one-character decision that determines whether 22:00:00 itself is off-peak or not, and whether 06:00:00 itself is off-peak or already peak again. Nothing in the suite exercised either instant. A boundary mistake here would not crash, would not warn, would just silently bill one hour a day at the wrong rate, forever, until someone happened to notice their bill looked slightly off. That is Alex's point, precisely, and it was sitting in my own repo.

The fix

public function test_peak_off_peak_strategy_resolves_the_exact_boundary_minute_correctly(): void
{
    $strategy = PricingStrategyFactory::fromConfig(ContractType::PeakOffPeak, [
        'off_peak_slots' => [['start' => '22:00', 'end' => '06:00']],
        'seasons' => [[
            'label' => 'year_round',
            'months' => range(1, 12),
            'rates' => [
                ['slot' => 'peak', 'price_per_kwh' => 0.27],
                ['slot' => 'off_peak', 'price_per_kwh' => 0.20],
            ],
        ]],
    ]);

    // Just before the off-peak window opens: still peak.
    self::assertTrue($strategy->priceForHour(new DateTimeImmutable('2026-07-18 21:59:00'))->equals(Money::of(0.27)));
    // The window opens exactly at 22:00:00: start is inclusive.
    self::assertTrue($strategy->priceForHour(new DateTimeImmutable('2026-07-18 22:00:00'))->equals(Money::of(0.20)));
    // Just before the off-peak window closes: still off-peak.
    self::assertTrue($strategy->priceForHour(new DateTimeImmutable('2026-07-19 05:59:00'))->equals(Money::of(0.20)));
    // The window closes exactly at 06:00:00: end is exclusive, already peak.
    self::assertTrue($strategy->priceForHour(new DateTimeImmutable('2026-07-19 06:00:00'))->equals(Money::of(0.27)));
}

Four instants instead of three comfortable ones: one minute before the window opens, the exact opening second, one minute before it closes, the exact closing second. That is the whole idea of testing boundaries instead of averages, written down as assertions instead of just agreed with in a comment thread.

Proving the test actually tests something

A boundary test that would pass against a broken implementation is worse than no test, it is a false sense of safety. So before trusting this one, I broke the code on purpose: flipped the wraparound comparison from >= / < to > / <=, one character each, and reran just this test.

It failed immediately, on the 22:00:00 assertion, exactly where it should:

Failed asserting that false is true.
at tests/Unit/Domain/Contract/PricingStrategyTest.php:73

Then I reverted the one-character change and ran the full suite: 104 tests, 752 assertions, green. That failure-then-pass cycle is the only way I trust a new test is doing its job rather than just decorating the file with more green checkmarks.

Thanks, Alex

None of this was a bug in production, it was a gap in coverage that a bug could have hidden in later. Comments like Alex's are exactly how that gap gets found before the bug does instead of after. If you have opinions on where else this project's tests are testing averages instead of edges, the repo is open, and so are the issues.

Comments

[FromZeroToShip]

"A boundary test that would pass against a broken implementation is worse than no test" — plus the failure-then-pass cycle being the only thing that earns your trust in it. I ran exactly that loop across a batch of gates this week, and it caught the thing you're describing in a spot I'd have sworn was fine.

Your one-character boundary is the part that lands hardest, because mine was literally that. I had an exclusion rule meant to keep a folder of deliberately-bad test fixtures out of a scoring pass. The pattern matched tests/seed but not tests/seed-clean — the trailing -clean fell just outside the boundary the regex anchored on. One missing case, exact same shape as your >= vs <: six clean fixtures scored as real for weeks, mathematically running, operationally wrong, every run green. I only found it because I finally did your move — deliberately broke the gate and watched whether it went red. It didn't, for the boundary I'd never tested.

The reader-found-the-gap half is the other thing I keep relearning: the boundary you didn't test is exactly the one you couldn't see, because if you could see it you'd have tested it. The cheapest audit I get is someone asking "what about the edge?" — and failure-then-pass is what turns their one-time question into a guardrail that stays.