DEV Community

Vivek Kalyanarangan
Vivek Kalyanarangan

Posted on

I got 29x speedup rewriting Python's validators library in Rust

#rust #performance #python #opensource

I've been using Python's validators library for a while — it's great for quick checks without defining schemas. But it became a bottleneck when validating millions of URLs and emails in data pipelines.

So I rewrote it in Rust.

The results

Validator Speedup
ipv4 47.0x
ipv6 39.4x
url 35.9x
email 28.5x
domain 28.3x

Average: 29x faster across 48 validators.

Drop-in replacement 

# Change one line
import rapidvalidators as validators

validators.email("test@example.com")  # True
validators.url("https://github.com")  # True
validators.btc_address("1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa")  # True
Enter fullscreen mode Exit fullscreen mode

That's it. Same API, same behavior, just faster.

What I learned building this

PyO3 + maturin is mature. Publishing wheels for Python 3.8-3.13 across Linux/macOS/Windows was straightforward. GitHub Actions + maturin handled cross-compilation without drama.

Hand-rolled parsing beats regex for simple patterns. For IPv4 validation, a simple split-and-parse approach was significantly faster than regex. Rust's regex crate is fast, but nothing beats avoiding it entirely.

Edge cases are brutal. URL validation alone has dozens of edge cases — international domain names, punycode, IPv6 hosts, weird port numbers. I wrote 370 tests to ensure parity with the original library.

The hardest validators were international ones. Spanish NIE, Indian Aadhaar, Finnish SSN — each has its own checksum algorithm. Lots of Wikipedia rabbit holes.

What's included

  • Network: email, url, domain, ipv4, ipv6, mac_address, hostname
  • Finance: iban, card_number, visa, mastercard, amex, cusip, isin
  • Crypto: btc_address, eth_address, bsc_address, trx_address
  • Hashes: md5, sha1, sha256, sha512
  • Encoding: base16, base32, base58, base64
  • International: Spanish (CIF, NIE, NIF), Indian (Aadhaar, PAN), Finnish, French, Russian

48 validators total.

Links

If you're processing lots of data and validators is in your dependency tree, give it a try. Happy to hear feedback or add validators people need.

Top comments (0)