URL encoding from zero: %20, encodeURIComponent vs encodeURI, and query strings

Every developer hits this bug at least once. You build a URL by gluing a value onto the end of it, ship it, and somewhere down the line a customer searches for "tom & jerry" and half their query vanishes. The & split their one value into two parameters and the space cut the URL short. The fix is URL encoding, and for something we lean on constantly, most of us only half understand it.

So for day 29 of SolveFromZero I built a small, offline URL toolkit and wrote down exactly how the encoding works. Here is the short version.

A URL has a tiny legal alphabet

RFC 3986 splits characters into groups. The unreserved set is the only thing that is always safe: A-Z, a-z, 0-9, and - _ . ~. Then there are reserved characters, which are the URL's punctuation: : / ? # & = @ and friends. They separate the scheme from the host, the path from the query, one parameter from the next. Everything else, including spaces, accents, and emoji, has no place in a URL at all.

Percent-encoding is the escape hatch. Take whatever character you have, get its bytes, and write each byte as a % followed by two hex digits. A space is byte 0x20, so it becomes %20. An & is 0x26, so %26. That's the whole mechanism. Decoding just runs the map backwards: see a %, read the next two hex digits, that's your byte.

Unicode is where it gets interesting

Percent-encoding works on bytes, and the web encodes text as UTF-8. In UTF-8 a single character can be one to four bytes, and you get one %XX per byte:

• a is one byte
• é is two bytes, so %C3%A9
• 日 is three bytes, so %E6%97%A5
• 😀 is four bytes, so %F0%9F%98%80

This is why é encodes to %C3%A9 and not %E9. %E9 is its old Latin-1 code point; %C3%A9 is its UTF-8 encoding. Get the byte layer wrong and the person on the other end decodes garbage. The tool has a little table that explodes any string you type into its code points, its raw UTF-8 bytes, and the final %XX, so you can watch one emoji become four escapes.

The one that trips everyone: encodeURIComponent vs encodeURI

JavaScript ships two encoders and picking the wrong one is the number one URL bug.

encodeURIComponent is strict. It escapes the reserved characters too, / ? : @ & = + $ , ; #, because it assumes you are encoding a single piece: one query value, one path segment. It must not let that piece accidentally introduce structure.

encodeURI is gentle. It leaves those reserved characters alone because it assumes you handed it a whole, already-assembled URL and it should only make it legal without breaking the link.

The rule of thumb: component for the parts, URI for the whole. Encode each value with encodeURIComponent, then assemble the URL yourself. Never run a finished URL through encodeURIComponent, or you will escape the :// and the ? and turn your link into mush. The tool shows both encoders side by side on the same input so the difference is obvious.

The + vs %20 gotcha

There are actually two space conventions. Generic URL encoding uses %20. But application/x-www-form-urlencoded, the format HTML forms submit and the one many query strings follow, encodes a space as +. So a query-string parser has to replace + with a space before it percent-decodes. decodeURIComponent("a+b") leaves the + alone, which surprises people. In a path a + is a literal plus; in a query it means space. Same byte, different meaning depending on where it sits.

Building and parsing query strings

The golden rule for building: encode each key and each value separately, then join with the raw = and & yourself. Never encode the assembled string, or you escape the very characters that give it structure.

Parsing is the mirror. Drop everything up to the ?, cut off any #fragment, split on &, split each pair on the first = only (values can contain an encoded one), then decode both halves with the + rule applied. The toolkit has a builder that adds key/value rows live, and a parser that takes a pasted URL and prints a decoded table.

Two last traps worth burning in: double-encoding (encode an already-encoded string and %20 becomes %2520, so decode first), and never trust raw input (an unencoded value can smuggle an extra &role=admin into your URL, or a CRLF into a redirect). Encode on the way out, decode exactly once on the way in, wrap decodeURIComponent in a try/catch, and validate redirect targets against an allow-list.

Play with all of it here, no install, runs entirely in the browser: https://dev48v.infy.uk/solve/day29-url-encoder.html