DEV Community

Sanjay Kumar Sah
Sanjay Kumar Sah Subscriber

Posted on • Originally published at github.com

The bug that differential fuzzing cannot find

I ported a 30-million-download-a-month cron parser from TypeScript to Go and ran 36,000 differential test cases with zero unexplained divergence. Then I found a bug that no amount of fuzzing could ever have caught — because the output wasn't actually a function of the input.


Written for Port Mortem 2026, a porting hackathon run by @partnerships_raptors.

I spent a weekend porting cron-parser, a TypeScript library responsible for roughly 30 million npm downloads a month, to Go.

The result was 2,234 lines of Go replacing 2,823 lines of TypeScript, with zero dependencies. The original relies on Luxon for its date and timezone handling.

But the port itself wasn't the interesting part.

Getting a library to compile in another language is mostly mechanical. The difficult question was:

How do you prove that two implementations behave the same when they don't share a runtime, standard library, or date/time implementation?

That question led me through 36,000+ differential tests, several real bugs, some deliberately strange timezones, and eventually a dependency whose behavior depended on something I wasn't testing at all:

the current time.


First problem: you can't just run the original tests

The hackathon required the original test suite to pass against the port.

For a TypeScript → Go port, that's not literally possible.

The upstream suite contains 302 Jest tests written in TypeScript. They call expect() and directly exercise the TypeScript implementation. A Go binary can't execute those tests.

There was another option: have Jest call into the Go implementation through Node FFI.

But FFI was explicitly prohibited.

And translating the tests into Go didn't solve the problem either. At that point, I would have a new test suite rather than the exact suite the judges provided. Passing my translation wouldn't prove that the original tests still passed.

So instead of translating the tests, I treated them as an oracle.

Turning the test suite into a conformance oracle

I wrote a small recorder that reads the untouched .test.ts files as text and extracts literal calls such as:

CronExpressionParser.parse('<expression>', { ...options })
Enter fullscreen mode Exit fullscreen mode

There were 130 such call sites.

Each call was executed against the original TypeScript implementation, and I recorded either:

  • the exact error string, or
  • the next 8 and previous 8 fire times as epoch milliseconds.

The Go implementation then replayed those exact inputs and compared the results.

The result:

130/130 matched.

The original test files were never modified. I verified that independently by running the suite itself — all 302 tests passed under TZ=UTC — and by committing SHA-256 hashes of all 24 upstream files.

That gave me a clean baseline.

And then the oracle immediately found three bugs my fuzzer hadn't found.


The bug my fuzzer could never see

The biggest one was the default timezone.

When no tz option is provided, the TypeScript implementation ultimately uses the process timezone, because Luxon falls back to the system zone.

I had defaulted the Go implementation to UTC.

On the machine I was developing on, that meant a difference of exactly 5 hours and 30 minutes.

91 of the 130 recorded call sites were silently wrong.

My differential fuzzer hadn't found it.

And this wasn't because I got unlucky.

It was structurally impossible for that particular fuzzer to find the bug.

Every generated test explicitly supplied a timezone.

So the fuzzer was exploring thousands of timezone combinations while completely bypassing the behavior that mattered.

Lesson #1: the original test suite knows things your generated corpus doesn't.

A generated test can only explore the assumptions encoded into its generator. If the generator always supplies a timezone, it can never discover what happens when the timezone is omitted.

That realization changed how I approached the rest of the port.


Then DST got weird

With the obvious bugs fixed, I built a proper differential fuzzer.

The basic idea was simple:

  1. Generate a random cron expression and date/time configuration.
  2. Run it against TypeScript.
  3. Run it against Go.
  4. Compare absolute epoch milliseconds.
  5. Repeat.

I used 16 timezones specifically chosen to exercise different kinds of DST transitions.

Not every DST transition is one hour.

Zone Transition
America/New_York 60 minutes
Antarctica/Troll 120 minutes
Australia/Lord_Howe 30 minutes
Pacific/Chatham 60 minutes, with a +12:45 base offset
America/Santiago 60 minutes, occurring at midnight

These cases exposed six separate places where Go's time package and Luxon behaved differently.

None of these differences were obvious from reading the TypeScript code.

They existed at the boundary between two languages' interpretations of time.

Three cases were particularly nasty.


Ambiguous local times

During the fall-back transition, a local time such as 01:30 can occur twice.

There are two valid instants that both look like:

01:30
Enter fullscreen mode Exit fullscreen mode

Go's time.Date chooses the first occurrence.

Luxon preserves the offset it already has.

My iteration logic couldn't escape the repeated hour and eventually hit its loop limit.

The problem wasn't simply "DST is complicated."

The problem was that the same wall-clock timestamp can represent two different points in time.


Month overflow

Go and Luxon also disagree about month arithmetic.

In Go:

January 31 + 1 month
Enter fullscreen mode Exit fullscreen mode

normalizes forward and becomes:

March 3
Enter fullscreen mode Exit fullscreen mode

Luxon clamps it to:

February 28
Enter fullscreen mode Exit fullscreen mode

That difference had a nasty consequence.

February could effectively disappear from some iteration paths, meaning:

L 2 *
Enter fullscreen mode Exit fullscreen mode

—the last day of February—

could fail to match anything.


Midnight transitions

Then there was America/Santiago.

Its DST transition occurs at midnight.

On September 6, 2026, the local day begins at 01:00.

There is no local 00:00.

This seemingly innocent code:

time.Date(2026, 9, 6, 0, 0, 0, 0, santiago)
Enter fullscreen mode Exit fullscreen mode

resolves to:

2026-09-05T23:00:00
Enter fullscreen mode Exit fullscreen mode

The previous day.

Now consider what happens when an iterator does something equivalent to:

Sep 5 midnight
        ↓ +1 day
Sep 6 midnight
        ↓ timezone resolution
Sep 5 23:00
Enter fullscreen mode Exit fullscreen mode

The resulting date is still September 5.

The iterator asks for September 5 again.

And again.

And again.

Eventually it hits the loop limit.

These weren't hypothetical edge cases.

The harness found them.

That is exactly what the fuzzer was supposed to do.


Then I hit a wall

At this point, the fuzzer was doing useful work.

But it kept producing another kind of failure.

After roughly 70 seconds of fuzzing, it would sometimes find a divergence of exactly one DST shift.

Then the next run wouldn't.

The failures were concentrated around ambiguous start times.

So I tried to reverse-engineer the rule.

Which of the two valid instants does Luxon choose?

I collected examples:

Zone Ambiguous time Luxon chooses
Antarctica/Troll 2026-10-25 01:15 Earlier
Europe/Berlin 2026-10-25 02:30 Earlier
America/New_York 2026-11-01 01:30 Earlier
Pacific/Auckland 2026-04-05 02:30 Later
Australia/Lord_Howe 2026-04-05 01:45 Later

I tried five different hypotheses:

  • choose the earliest instant
  • choose the latest instant
  • choose the larger offset
  • choose the smaller offset
  • choose the pre-transition offset

Every rule worked for some zones and failed for others.

Each experiment meant another code change, rebuild, and roughly two-minute fuzzing run.

That is where the six hours went.

Eventually, I stopped guessing and opened Luxon's source.


The answer was hiding in 40 lines of code

Inside luxon/src/datetime.js was this function:

// find the right offset a given local time. The o input is our guess, which
// determines which offset we'll pick in ambiguous cases
function fixOffset(localTS, o, tz) { ... }
Enter fullscreen mode Exit fullscreen mode

The comment was the clue.

Luxon wasn't following a universal rule like "always choose the earlier instant."

It was doing a guess-and-correct search.

And the initial guess mattered.

So where did the guess come from?

Another function:

function guessOffsetForZone(zone) {
  if (zoneOffsetTs === undefined) {
    zoneOffsetTs = Settings.now();
    // ...
  }
}
Enter fullscreen mode Exit fullscreen mode

There it was.

The guess was based on:

the zone's offset right now.


The output depends on the clock

This took me a while to accept.

For an ambiguous local timestamp — the repeated hour during a fall-back transition — the actual instant returned by parse() can depend on when the function is called.

Run the same parse in January and then in July, in a timezone that observes DST, and the ambiguous timestamp can resolve differently.

Same input.

Same code.

Different output.

The difference is the wall clock.

There was no hidden deterministic rule waiting to be reverse-engineered.

My five failed hypotheses weren't simply bad guesses.

They were attempts to fit a deterministic function to behavior that wasn't actually a function of the input alone.

Conceptually, I had been trying to reason about:

output = f(input)
Enter fullscreen mode Exit fullscreen mode

But the real behavior was closer to:

output = f(input, current_time)
Enter fullscreen mode Exit fullscreen mode

And my differential fuzzer only controlled the first variable.


The faithful fix

At that point, the right solution was no longer to infer Luxon's behavior.

It was to reproduce it.

I transcribed fixOffset into Go, including the Math.min / Math.max branch for invalid local times, and seeded the calculation from:

time.Now().In(loc)
Enter fullscreen mode Exit fullscreen mode

That means the Go port now inherits the same time-dependent behavior for ambiguous start times.

That's intentional.

I'm porting the library, not redesigning it.

If the original implementation behaves strangely, a faithful port needs to preserve that behavior unless the goal is explicitly to create a better implementation.


What differential fuzzing can — and cannot — prove

By the end, my harness had run:

36,040 cases across 106 rounds and 180 seconds.

That's strong evidence of equivalence.

But it could never have discovered the time-dependent behavior described above.

Not with 36,000 cases.

Not with 36 million.

Not with unlimited compute.

Because differential fuzzing fundamentally asks:

f(input) == g(input)
Enter fullscreen mode Exit fullscreen mode

But this behavior wasn't determined solely by input.

It depended on the wall clock.

That's the important limitation.

Randomized differential testing is still the best tool I know for this kind of port.

It found six real bugs in my implementation — several serious enough that I would have confidently shipped them.

But differential fuzzing has a shape:

It can only discover divergences that are reproducible from the inputs it controls.

Ambient state is invisible unless you deliberately model it.

That includes things like:

  • current time
  • process timezone
  • locale
  • environment variables
  • operating-system behavior
  • cached state
  • values initialized once at process startup

The fuzzer told me that something was wrong.

It couldn't tell me why.

And throwing more compute at it wouldn't have changed that.

Reading about 40 lines of dependency source code did.


So is the port equivalent?

I wouldn't claim that.

My honest claim is narrower:

The Go port matches 6,400 generated cases across 10 fixed seeds, all 130 recorded call sites from the original suite, and a 180-second continuous differential run — with one known residual edge case whose scope and reproduction are committed.

That's a weaker statement.

It's also a more truthful one.

For a compatibility project, "I tested a lot" is not the same thing as "I proved equivalence."


The decision I'd change

If I did the project again, I would change one major thing:

Build the oracle first.

I initially created 15 hand-picked regression cases based on what I saw while reading the DST implementation.

Those cases were useful.

They found three divergences in only 15 tests.

But they had a fundamental weakness:

they inherited my assumptions.

My generated corpus inherited those assumptions too.

Both always supplied an explicit tz.

That's why the biggest correctness bug — the default timezone behavior — survived all 36,040 fuzz cases.

91 call sites were wrong by 5 hours and 30 minutes.

The conformance oracle caught it almost immediately.

And I had initially treated that oracle as little more than bookkeeping.

That was backwards.

The original test suite is a corpus created by someone else, against assumptions I don't share.

That's precisely what makes it valuable.

Generated tests explore the space I thought of.

Recorded tests explore the space the library's authors thought of.

The second is often where your blind spots live.

So next time, my order would be:

Original suite
      ↓
Conformance oracle
      ↓
Differential fuzzing
      ↓
Targeted regression cases
      ↓
Source-level investigation
Enter fullscreen mode Exit fullscreen mode

Not because fuzzing is less useful.

Because each technique covers a different failure mode.


Two things I'd rather admit than hide

One residual divergence

There is one remaining divergence that occurs roughly 1 in 30,000 cases.

It involves backward iteration from an ambiguous instant in a timezone with a sub-hour DST transition.

I traced it to Luxon's endOf(unit) implementation:

startOf(unit)
    + 1 unit
    - 1ms
Enter fullscreen mode Exit fullscreen mode

That combines wall-clock truncation with absolute-time arithmetic.

During a 30-minute fall-back transition, the result can land an hour earlier than expected.

I attempted a faithful transcription.

It caused iteration to stall.

The remaining alternatives were either blind approximations or porting enough of Luxon's Duration and objToTS machinery to effectively create a second library.

I chose not to do either.

The divergence is documented at the relevant function and in the findings, including its observed rate.

A known, reproducible limitation is something judges can evaluate.

A hidden one isn't.


Two claims I got wrong

I also made mistakes during the investigation.

At one point, I claimed that Go's most popular cron library uses AND semantics for day-of-month/day-of-week, while POSIX specifies OR.

That was wrong.

I had sourced the claim from an issue tracker for a different library in a different language and never verified it.

It's OR.

I also gave two different incorrect explanations for one of the DST mechanisms before a proper sweep settled what was actually happening.

Those mistakes remain in the write-up.

That's intentional.

The entire point of this exercise is that running the code and checking the evidence should be allowed to prove you wrong.

Removing the incorrect claims would remove part of the evidence of that process.


What the port revealed about the original library

The port was the instrument.

The original library was where some of the more interesting problems surfaced.

1. An existing DST bug is more complicated than it looks

CronDate.ts detects DST changes by subtracting hour numbers — effectively doing wall-clock arithmetic.

That breaks for transitions that aren't exactly one hour.

In Antarctica/Troll, the transition is two hours, and the observed difference can become 3, overshooting the constant being checked.

In Australia/Lord_Howe, the transition is only 30 minutes.

Depending on which minute the iterator enters the hour, the calculated difference can be either 1 or 2.

So there isn't a single hour-based constant that can correctly represent the transition.

The consequence is serious:

a daily job can silently skip an entire day.


2. The proposed fix doesn't handle 30-minute transitions

There is an open PR proposing:

Math.floor((offset - prevOffset) / 60)
Enter fullscreen mode Exit fullscreen mode

That works for hour-based transitions.

But for Lord Howe:

floor(30 / 60) = 0
Enter fullscreen mode Exit fullscreen mode

A real 30-minute DST transition therefore receives zero compensation.

The deeper issue is that the fields being written to are expressed in hours.

This isn't a constant problem.

The unit itself needs to become minutes.


3. prev() can hang on a valid expression

I also found a previously unreported case where prev() can exhaust its loop limit.

For example:

CronExpressionParser.parse('0 0 0 * * *', {
  tz: 'Pacific/Chatham',
  currentDate: '2026-09-27T05:45:00',
}).prev();

// Error: Invalid expression, loop limit exceeded
Enter fullscreen mode Exit fullscreen mode

The expression is completely valid:

0 0 0 * * *
Enter fullscreen mode Exit fullscreen mode

means:

every day at midnight.

But iterating backward across the Chatham spring-forward gap eventually exhausts the 10,000-step limit.

The same behavior reproduces in Australia/Lord_Howe.

My port deliberately does not reproduce this hang.

Compatibility doesn't mean inheriting an infinite loop when the behavior is clearly pathological.

Instead, the case is recorded in a known-divergences file with a deliberately narrow predicate so it cannot silently hide a regression in the port.


One more undocumented disagreement

There is another interesting difference between cron implementations.

During the fall-back transition in America/New_York, consider:

30 1 * * *
Enter fullscreen mode Exit fullscreen mode

A daily job at 1:30.

Under cron-parser, it fires once.

Under Go's most popular cron library, it fires twice.

For wildcard hours, both implementations fire twice.

So the behavior differs depending on whether the hour field is literal or wildcard.

Neither library clearly documents this rule.

That means running the same schedule through both implementations can produce a real production difference:

one of your nightly jobs runs twice a year.


The boring win

And yes, the Go port is faster.

On my benchmark:

Parse() + 10 × Next()

Mean:  1330 µs → 37.6 µs
p99:   ~20× faster
Memory: ~5× lower
Dependencies: 0
Enter fullscreen mode Exit fullscreen mode

That's roughly 35× faster on the mean.

It's a nice result.

It's also the least interesting part of the project.

Rewriting a JavaScript library in Go and discovering that Go is faster isn't particularly surprising.

The difficult work was reproducing the semantics of two completely different approaches to time.


The real lesson

I started this project thinking the hard part would be porting TypeScript to Go.

It wasn't.

The hard part was discovering what the original implementation actually means by the same operations.

"Add a month."

"Start at midnight."

"Choose the timezone."

"Parse an ambiguous timestamp."

"Find the previous occurrence."

Those sound deterministic.

They aren't always.

They can depend on:

  • language semantics
  • standard-library behavior
  • timezone databases
  • DST transition width
  • wall-clock arithmetic
  • environment configuration
  • process timezone
  • and, surprisingly, the current moment

That's why no single testing strategy was enough.

The original tests exposed assumptions my generator didn't know existed.

Differential fuzzing found edge cases I hadn't imagined.

Source inspection explained behavior that neither could fully explain.

And the most important discovery was the one I couldn't have found by generating more inputs:

Sometimes the bug isn't hiding in the input space.

Sometimes the output depends on state you forgot was part of the program.

That's the bug differential fuzzing could never find.


Reproduce it

Repository: https://github.com/sanjaysah101/port-mortem-cron-parser

Demo: https://youtu.be/4O2HAY4QAoM

Everything in this write-up is reproducible with:

node run.mjs
Enter fullscreen mode Exit fullscreen mode

The script:

  1. clones the pinned upstream repository,
  2. runs the untouched upstream test suite,
  3. replays the conformance oracle,
  4. and runs the differential fuzzer.

Built for Port Mortem 2026 by Hackathon Raptors. Track C, TypeScript → Go.

Top comments (0)