DEV Community

Michael Lip
Michael Lip

Posted on • Originally published at zovo.one

URL Encoding Is Not Optional (And Getting It Wrong Breaks Everything)

A URL can only contain a specific set of characters: letters, digits, and a handful of special characters (- _ . ~). Everything else must be percent-encoded. A space becomes %20. An ampersand becomes %26. A forward slash becomes %2F.

This is not a suggestion. It is a requirement of RFC 3986. If you put a raw space in a URL, the behavior is undefined. Some browsers will encode it for you. Some will not. Some servers will interpret it one way, others differently. The only way to guarantee correct behavior is to encode properly.

The encoding rules

Unreserved characters (safe, no encoding needed): A-Z a-z 0-9 - _ . ~

Reserved characters (have special meaning in URLs): : / ? # [ ] @ ! $ & ' ( ) * + , ; =

Everything else: must be percent-encoded as %XX where XX is the hex value of the byte.

The key distinction: reserved characters have special meaning in URL structure. A ? separates the path from the query string. A & separates query parameters. A # marks the fragment. If your data contains these characters, they MUST be encoded or they will be misinterpreted as URL structure.

Query parameter value: "cats & dogs"
Unencoded URL: https://example.com/search?q=cats & dogs
Interpreted as: q=cats, then " dogs" is a malformed second parameter

Encoded URL: https://example.com/search?q=cats%20%26%20dogs
Interpreted correctly: q=cats & dogs
Enter fullscreen mode Exit fullscreen mode

encodeURIComponent vs. encodeURI

JavaScript provides two functions, and using the wrong one is the second most common URL encoding bug.

encodeURIComponent: Encodes everything except unreserved characters. Use this for query parameter values, path segments, and any data that goes INTO a URL.

encodeURI: Encodes everything except unreserved characters AND reserved characters that are part of URL structure. Use this for encoding a complete URL where the structure (://?#&=) should be preserved.

const param = "price=10&currency=USD";

encodeURIComponent(param); // "price%3D10%26currency%3DUSD"
// Correct for a parameter value: treats = and & as data

encodeURI(param);          // "price=10&currency=USD"
// Wrong for a parameter value: preserves = and & as structure
Enter fullscreen mode Exit fullscreen mode

Using encodeURI on parameter values is a security vulnerability. If the value contains &key=malicious, it will be interpreted as a separate parameter.

Double encoding

The most common encoding bug: encoding something that is already encoded. %20 becomes %2520 (the % gets encoded to %25).

Original:       hello world
Single encode:  hello%20world
Double encode:  hello%2520world  (broken!)
Enter fullscreen mode Exit fullscreen mode

This happens when you encode a URL, then pass it through a function that encodes it again. It is extremely common in redirect chains and when building URLs from already-encoded components.

The fix: encode raw values, not already-encoded strings. If you are unsure whether a string is encoded, decode it first (which is safe if it was not encoded), then encode it.

Unicode and percent encoding

URLs are byte-based. Unicode characters must first be converted to UTF-8 bytes, then each byte is percent-encoded.

The character "cafe" with an accent: cafe (U+0301)
UTF-8 encoding of U+0301: 0xCC 0x81
Percent-encoded: %CC%81

So "cafe" becomes "caf%C3%A9" (UTF-8 encoding of e with accent is C3 A9).

CJK characters are typically 3 bytes in UTF-8, so each character becomes three percent-encoded bytes: a single Japanese character might become %E4%BA%BA.

The practical tool

I built a URL encoder/decoder at zovo.one/free-tools/url-encoder-decoder that encodes and decodes URLs and URL components. It distinguishes between full-URL encoding and component encoding, detects double-encoding, and shows the raw bytes for Unicode characters. Paste a URL and see every encoded character highlighted and decoded, or paste raw text and get the properly encoded version.

I'm Michael Lip. I build free developer tools at zovo.one. 500+ tools, all private, all free.

Top comments (0)