DEV Community

LXSAIHUB
LXSAIHUB

Posted on

A 4-line regex took down our image pipeline for 6 hours. It was legal.

Our image validation service stalled for about 6 hours in April. Not slow — stalled, requests queueing, CPU pinned at 100% on every worker. The regex that caused it was 4 lines long and I still have it in a scratch file:

const matcher = /^(\w+\.){2,}\w+\$/
Enter fullscreen mode Exit fullscreen mode

That looks harmless. It is also catastrophically backtracking, and here is why.

The input that broke it

Attachments come from users, so filenames come from users. This is a perfectly reasonable thing to send us:

'a'.repeat(40) + 'X'
Enter fullscreen mode Exit fullscreen mode

40 characters, one character off the end of the pattern. The engine tries every possible way to divide the string among the {2,} repetitions of (\w+\.), then backtracks through all of them before it can conclude there is no match.

Input length Time to reject
12 0.4 ms
20 51 ms
26 2.9 s
28 12 s
30 >45 s (timed out)

The jump from 0.4 ms to 45 s between 12 and 30 characters is the signature of exponential backtracking. Growth is not quadratic; it is much worse, and that is what lets a short string become an outage.

Why the static checks all passed

This is the part that still annoys me. Before shipping, that line was reviewed by:

  • an ESLint rule for regex safety — clean
  • a Semgrep rule set for ReDoS — clean
  • a human reviewer, me, looking at it carefully — clean

All three missed it. The reason is that most ReDoS detection is pattern-list based: it flags known-bad shapes like (a+)+ or (a|a)+. ((\w+\.){2,}\w+) is not on those lists, and for good reason — it is not reliably dangerous in isolation. The danger only appears when the input domain is unconstrained.

If I had to fix the detector, not the regex, the change would be small: give it a time budget instead of a shape list.

// What I want: prove the pattern is linear by measuring it
function isSafe(re, samples, budgetMs = 50) {
  for (const s of samples) {
    const t0 = process.hrtime.bigint()
    re.test(s)
    const dt = Number(process.hrtime.bigint() - t0) / 1e6
    if (dt > budgetMs) return { safe: false, ms: dt }
  }
  return { safe: true }
}
Enter fullscreen mode Exit fullscreen mode

Feed it a length-ladder — 'a'.repeat(n) + 'X' for n climbing — and a vulnerable pattern announces itself at n=22 instead of n=30 in production. That is a 40x earlier warning, and it catches shapes no pattern list will ever enumerate.

The actual fix

Two characters. Nested quantifier made possessive:

const matcher = /^(\w+\.){2,}\w+$/     // before: backtracks
const matcher = /^(?:\w+\.){2,}\w+$/   // after: no nested repeat to explode
Enter fullscreen mode Exit fullscreen mode

In practice I went further and dropped the regex. Filename extension checking does not need a general matcher:

const ext = path.extname(name).toLowerCase()
const ok = ['.jpg', '.jpeg', '.png', '.webp'].includes(ext)
Enter fullscreen mode Exit fullscreen mode

No backtracking possible, because there is no backtracking. The regex was solving a problem that path.extname solves in one line without a risk class.

The rest of the ladder

The same length-ladder check, run across our own patterns, found two more before they reached production. Both were "safe on our test data" because our fixtures were all well-formed:

pattern                     n=10    n=18    n=26    n=30
^(\w+\.){2,}\w+$            0.4 ms  51 ms   12 s    timeout
^([a-z]+-)+[a-z]+$         0.3 ms  22 ms   6.1 s   38 s
^\s*(a|aa)+$                0.2 ms  9 ms    1.4 s   4.2 s
Enter fullscreen mode Exit fullscreen mode

The growth ratio between consecutive rungs is the useful signal, not the absolute time. Anything that more than triples per rung is exponential, and no amount of short input will reveal it.

Dialect differences that actually bite

The same pattern is safe in one engine and catastrophic in another, which is why "is this regex vulnerable" without a stated engine is not a question with an answer.

Construct JS / ECMAScript Python re Go regexp PCRE
(?>...) atomic group not supported needs (?>...) (3.11+) not supported supported
Possessive *+ not supported not supported not supported supported
Backreference \1 supported supported not supported supported
Named group syntax (?<n>...) (?P<n>...) (?P<n>...) (?<n>...) / (?P<n>...)

Two practical consequences. A pattern using atomic groups throws a syntax error on Go rather than degrading gracefully, so the same library has to branch on the engine. And a pattern relying on backreferences is simply unportable to Go, which is a RE2-based engine and will not grow one — that is a hard limit, not a missing feature.

I keep a per-dialect note next to every pattern in the codebase. It costs a comment and it has prevented at least one "works on my machine" bug in review.

What I changed in our checks

Timing became a gate, not a note. Every regex in the codebase now runs against a length ladder in CI. If worst-case time across the ladder grows by more than 3x between consecutive rungs, the build fails. That is a cheap, objective bar.

Input domain is part of the threat model. "Is this regex vulnerable" is unanswerable on its own. "Is it vulnerable for filenames from the internet" is answerable. The domain is the input.

Prefer no pattern. The single most effective ReDoS mitigation is deleting the regex. (a|b)+ over a known 4-value enum should be a switch or a Set.

What it is not

This is not a proof that a pattern is safe. No static check can do that — the general problem is undecidable, and heuristics will miss novel shapes. I would not trust any tool, mine included, to certify an unfamiliar pattern as safe.

What it does: catch the common catastrophic-backtracking shapes across JS, Python, Go, PCRE and Rust dialects, and call out the edge cases a given pattern has (anchoring, alternation depth, absence of a possessive/atomic group). It is a lint with a clear failure mode, not a guarantee.

I keep the timing check because a heuristic that says "I don't know" is worth more than one that says "probably fine" and takes your image pipeline down for 6 hours.

Top comments (0)