DEV Community

Libme
Libme

Posted on

Scrubbing Test Data Without Nulling It Out: The Reserved Ranges That Are Safe to Seed

If your scrub script sets phone = NULL and email = NULL, every feature that depends on those fields silently stops existing in your test environment — SMS opt-in, required-field validation, notification fan-out. The fix is not random generation, which can dial a stranger. Standards bodies and telecom regulators have set aside ranges that are syntactically valid but guaranteed unroutable, and writing those into your scrubbed data keeps the code paths alive while making delivery impossible.

A commenter on an earlier post made this point about phone numbers, and it generalizes to almost every PII column you touch.

What actually breaks when you NULL a column?

Three distinct failures, and only the first one is obvious.

Validation paths never execute. Your checkout form requires a phone number. In the scrubbed environment nobody has one, so the edit form renders a blank required field, the E.164 normalizer is never called, and the branch that handles "user has a number but it failed verification" is unreachable. You ship the bug that only fires for users who do have data.

Downstream code gets a type it did not plan for. A NULL phone reaches your SMS client and you get TypeError: Cannot read properties of null (reading 'replace') in your formatter — or, if the null makes it to the provider, a rejection like Twilio's 21211 Invalid 'To' Phone Number. Neither error tells you anything about the feature you were testing.

Randomly generated data is worse than NULL. The moment someone points a test environment at live credentials — a misconfigured secret, a provider client that defaults to production when an env var is missing — randomly generated digits are routable digits. Random data fails open: it reaches a real person. Reserved data fails closed.

The takeaway: NULL removes the code path, random data removes the safety, and reserved ranges remove neither.

Strategy Shape is valid Exercises the code path Safe if creds leak Deterministic
NULL everything No No Yes Yes
Random digits/strings Usually Yes No Only if seeded
Scrubbed production values Yes Yes No Yes
Reserved ranges Yes Yes Yes Yes, if hashed from the PK

Which ranges are actually reserved?

These are set aside by the relevant authority specifically so that fiction and documentation cannot collide with a real subscriber or host.

Data Reserved range Set aside by
Phone, North America 555-0100 to 555-0199 in any area code NANPA, for fictional use
Phone, UK mobile 07700 900000–900999 Ofcom drama range
Phone, UK landline 020 7946 0000–0999, 01632 960000–960999 Ofcom drama range
Email domain example.com, example.net, example.org RFC 2606
Hostname / TLD .test, .example, .invalid, .localhost RFC 2606
IPv4 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24 RFC 5737 (documentation)
IPv6 2001:db8::/32 RFC 3849 (documentation)
Card numbers Your gateway's published test numbers, sandbox only Payment provider docs

Two details that bite people. The whole 555 prefix is not reserved — only 555-0100 through 555-0199; 555-1212 is real directory assistance in much of the NANP. And other regulators publish their own drama ranges, but the specific blocks differ by country, so look them up at the regulator rather than pattern-matching from the US ones.

For email, example.com is safe to write but mail to it goes nowhere, which means you cannot open a signup confirmation. If your tests need to read the message, keep the reserved domain in the database and point your provider at a mail-capture inbox in the preview environment's config — separate concerns, separate places.

The takeaway: reserved ranges are valid enough to pass your validators and dead enough that no carrier or DNS resolver will deliver anything.

How do I generate them deterministically?

Hash the primary key. Same input, same fake value, every run — so a re-scrub does not churn your test fixtures, and the same user looks identical in users, invoices, and your search index.

-- Run only on a machine allowed to see production data.
CREATE OR REPLACE FUNCTION fake_phone_e164(seed text) RETURNS text AS $$
  SELECT '+1' || npa || '555' || lpad((100 + (h % 100))::text, 4, '0')
  FROM (
    SELECT h,
           (ARRAY['202','212','312','415','503','617','713','808'])
             [1 + ((h / 100) % 8)] AS npa
    FROM (SELECT ('x' || substr(md5(seed), 1, 6))::bit(24)::int AS h) x
  ) y;
$$ LANGUAGE sql IMMUTABLE;

UPDATE users SET
  phone      = fake_phone_e164(id::text),
  email      = 'user-' || substr(md5(id::text), 1, 12) || '@example.com',
  full_name  = 'Test User ' || substr(md5(id::text), 1, 6),
  last_ip    = ('203.0.113.' || (1 + (('x' || substr(md5(id::text), 1, 4))::bit(16)::int % 250)))::inet;
Enter fullscreen mode Exit fullscreen mode

bit(24)::int is always positive, which saves you from the abs() overflow footgun on the more common bit(32)::int version of this trick.

Keep the shape of the data, not just the type. If 20% of your users have no phone in production, preserve that: CASE WHEN phone IS NULL THEN NULL ELSE fake_phone_e164(id::text) END. A column where every row is populated tests a world you do not ship to.

The takeaway: derive fakes from the primary key so the scrub is idempotent and referentially consistent across tables.

How do I prove the scrub actually worked?

Assert it in the same transaction, and let the assertion fail the job. A scrub that half-ran is more dangerous than no scrub, because everyone downstream assumes it completed.

DO $$
DECLARE leaked int;
BEGIN
  SELECT count(*) INTO leaked FROM users
   WHERE phone IS NOT NULL AND phone !~ '^\+1\d{3}55501\d{2}$';
  IF leaked > 0 THEN
    RAISE EXCEPTION 'scrub incomplete: % rows outside reserved phone range', leaked;
  END IF;

  SELECT count(*) INTO leaked FROM users
   WHERE email IS NOT NULL AND email !~* '@example\.(com|net|org)$';
  IF leaked > 0 THEN
    RAISE EXCEPTION 'scrub incomplete: % rows with non-reserved email domain', leaked;
  END IF;
END $$;
Enter fullscreen mode Exit fullscreen mode

Run this as the last step of the export job, and again as the first step of the import job on the untrusted side. The second run is the one that catches a snapshot that was copied from the wrong bucket.

If you want the managed version of this, PostgreSQL Anonymizer is the open-source extension that lets you declare masking rules as security labels on the columns themselves, so the rule lives next to the schema instead of in a script that drifts. Its cost is real: it is a C extension, so it has to be available on your managed Postgres provider, and the rules need database-level privileges to manage. Neosync covers similar ground as a service with referential-integrity-aware transformers if you would rather not run an extension at all, with the usual tradeoff of sending your schema to a third party.

The takeaway: a scrub without a machine-checked assertion is a hope, and hopes do not fail CI.

What about unique constraints and volume?

This is where the approach has a hard edge. The North American reserved block gives you exactly 100 numbers per area code. Eight area codes is 800 distinct values; every valid NPA is on the order of tens of thousands. If users.phone carries a unique index and you have more rows than that, the scrub will fail on a duplicate key.

Do not widen the range to fix that. The right response is to shrink the dataset: a preview or test database needs hundreds of representative rows, not your full user table. Subset first, scrub second, and the ceiling stops mattering. If you genuinely need a full-size dataset for load testing, drop the unique constraint in that environment explicitly and write down why — as of late 2026 there is no reserved range large enough to satisfy a million-row unique phone column, and pretending otherwise means routable numbers.

The takeaway: running out of reserved values is a signal that your test dataset is too big, not that the range is too small.

FAQ

What phone numbers are safe to use for test data?
In North America, 555-0100 through 555-0199 in any area code are reserved by NANPA for fictional use and will not connect to a subscriber. In the UK, Ofcom reserves 07700 900000–900999 for mobile and 020 7946 0000–0999 for London landlines. Other prefixes, including the rest of 555, can be real.

Can I send test emails to example.com?
You can safely store @example.com addresses — RFC 2606 reserves the domain so nobody can register it or receive mail there. But nothing will be delivered, so if your test needs to open the email, route the preview environment's SMTP to a mail-capture inbox instead of relying on the address.

How do I keep fake data consistent across tables?
Generate every fake value from a hash of the row's primary key rather than from a random source. The same user ID produces the same phone number and email on every run and in every table that references it, which makes the scrub idempotent.

Bottom line

Null-based scrubbing is the default because it is one line of SQL, and it quietly deletes the test coverage you thought you had. Write reserved-range values instead: they pass validation, exercise the same code paths as real data, and cannot reach a human even if someone wires the environment to live credentials by mistake. Derive them deterministically from the primary key, assert the result with a query that can fail the job, and subset your data before you scrub it so unique constraints never force you outside the reserved block. If you would rather declare the rules than maintain a script, PostgreSQL Anonymizer is the sane starting point on self-managed Postgres.

Related reading

Top comments (0)