DEV Community

DSP
DSP

Posted on Edited on

Messy data in, clean records out: RecordScrub, a data-cleaning API

Every project eventually inherits the same spreadsheet: emails in three different cases, phone numbers formatted six ways, names like "DOE, john michael", and the same customer entered twice. Cleaning it by hand, or with a pile of regexes you don't quite trust, is nobody's favorite afternoon.

RecordScrub is a small API we built at DSP to do that job in one call. It's live on RapidAPI now, with a free tier.

What it does

  • Email validation & normalization, single or batch, with valid/invalid stats and a plain-language reason for every rejection
  • Phone normalization to E.164, powered by Google's libphonenumber, so (312) 555-0142 becomes +13125550142
  • Name normalization, "DOE, john michael" → John Michael Doe, with first/last split out
  • US address normalization, state codes, ZIP+4, standardized street suffixes
  • Batch record cleaning, send CSV-style records, get snake_case keys, per-field cleaning, and exact dedupe on the fields you choose, with a duplicate report showing what was dropped and why

Every dropped or altered value comes back with a _warnings explanation. No silent data loss.

A taste

curl -X POST https://record-scrub.p.rapidapi.com/v1/normalize/phone \
  -H "X-RapidAPI-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"phone": "(312) 555-0142"}'
Enter fullscreen mode Exit fullscreen mode
{"original": "(312) 555-0142", "normalized": "+13125550142",
 "valid": true, "country_code": 1, "national_format": "(312) 555-0142"}
Enter fullscreen mode Exit fullscreen mode

And the batch cleaner, deduplicating on email + phone:

{"records": [{"name": "John Doe", "email": "john@example.com",
              "phone": "+13125550142"}],
 "stats": {"received": 2, "returned": 1, "duplicates_removed": 1},
 "duplicates": [{"index": 1, "duplicate_of": 0, "record": { "...": "..." }}]}
Enter fullscreen mode Exit fullscreen mode

Two things worth stating plainly, prompted by a comment from Benjamin Fisher below. First, normalization is not verification: a valid result means an email is well-formed, not that a mailbox exists, and a standardized address is formatted, not verified deliverable. Second, on dedupe: matching is exact, on the cleaned values of the fields you choose. When two records normalize to the same key, the first record is kept, and the later one is dropped and listed in the duplicates report with its original index and which record it duplicated, so you can audit a merge before trusting it.

How it's built

FastAPI, completely stateless. Requests are processed in transit and nothing is stored, so there's no database to leak from. It runs on a free-tier host, so the first call after a quiet spell can take a few extra seconds while it wakes up; after that it's fast.

Pricing

Every endpoint is available on every plan, including the free tier. No feature gates, no Pro-only cleaners, no endpoint held back to force an upgrade. Upgrading simply gets you more requests per month: start free with 100 requests a month, move up to Pro at $10/month for 5,000, and scale from there if your batch volume grows. You pay for volume, not for access.

Try it: https://rapidapi.com/darksundarker/api/record-scrub

Feedback very welcome, especially the "I wish it also cleaned X" kind. That's how the v2 list gets written.

Top comments (2)

Collapse
 
ziwetaro870 profile image
Benjamin Fisher •

One thing I'd make explicit in the docs is that normalization isn't verification: a syntactically valid email may not have a working mailbox, and a standardized address isn't necessarily deliverable. It’d also help to show how dedupe behaves when two records normalize to the same value, since that can affect which records users choose to merge.

Collapse
 
darksun159 profile image
DSP •

Benjamin, fair on both. The shape-not-deliverability line does exist, but in the README and Terms, not in this post or the listing where people actually look. It should be explicit in both places: a valid result means the address is well-formed, not that a mailbox exists, and a standardized address is not a verified deliverable one. On dedupe: it is exact-match on the cleaned values of the fields you choose. When two records normalize to the same key, the first record is kept, and the later one is dropped and listed in the response's duplicates report with its original index and which record it duplicated, so you can audit the merge before trusting it. I will add both points to the listing and this post.