DEV Community

CheBit
CheBit

Posted on AI-assisted

URL encoding, explained by the bugs it causes

A link that works when you click it can still break when you paste it somewhere else.

https://app.example.com/login?next=https://app.example.com/dashboard?tab=billing
Enter fullscreen mode Exit fullscreen mode

That looks like a redirect to a dashboard tab. But look at the second ?. Is tab=billing part of the next value, or a second parameter on the login URL? Different parsers answer differently. One of them sends the user to the right place; another sends them to a 404.

The fix is percent-encoding. And it is one of those things everyone uses and almost nobody can explain precisely, which is why the same three bugs keep showing up in production.

What percent-encoding actually does

Percent-encoding works on bytes, not characters. It leaves a small set of characters alone and rewrites everything else as % followed by two hexadecimal digits.

The characters that never need encoding — the "unreserved" set from RFC 3986 — are:

A-Z  a-z  0-9  -  .  _  ~
Enter fullscreen mode Exit fullscreen mode

Everything else becomes bytes, and each byte becomes %XX.

encodeURIComponent('a b/c');   // 'a%20b%2Fc'
encodeURIComponent('é');       // '%C3%A9'
encodeURIComponent('日');      // '%E6%97%A5'
Enter fullscreen mode Exit fullscreen mode

Notice that é is a single character but produces two escapes. That is not a quirk of the encoding function — é is two bytes in UTF-8, and percent-encoding escapes bytes. This is also why URL-encoded strings look so much longer than the text they came from.

The three characters behind most of the bugs

& separates parameters. If it appears inside a value, it has to be %26. Otherwise the parser starts a new parameter and the rest of your value becomes a stray key.

encodeURIComponent('a&b');     // 'a%26b'
encodeURIComponent('?');       // '%3F'
Enter fullscreen mode Exit fullscreen mode

+ means space only in form-encoded data. That convention comes from HTML form submission (application/x-www-form-urlencoded), not from the URL standard itself. In a query string most servers accept both + and %20 as a space. In a path segment, + is a literal plus, not a space.

% starts an escape. A literal percent sign must be encoded as %25. This is where double encoding begins.

The double-encoding trap

encodeURIComponent('%20');                        // '%2520'
encodeURIComponent(encodeURIComponent('a b'));    // 'a%2520b'
Enter fullscreen mode Exit fullscreen mode

If you encode a string that was already encoded, the % becomes %25 and the escape stops working. The server decodes once and gets %20 back as literal text, never a space. Strings containing %2520 in a browser address bar are almost always this bug.

The cause is usually two layers each assuming the other did not encode. A client encodes a value, then a library encodes the whole URL again. The rule is simple: encode exactly once, at the boundary where the value enters the URL.

encodeURI vs encodeURIComponent

JavaScript gives you two functions with confusingly similar names.

encodeURI('https://example.com/a b');
// 'https://example.com/a%20b'

encodeURIComponent('https://example.com/a b');
// 'https%3A%2F%2Fexample.com%2Fa%20b'
Enter fullscreen mode Exit fullscreen mode

encodeURI preserves the characters that give a URL its structure — : / ? # & = — so it is for a complete URL you want to keep intact. encodeURIComponent escapes all of those, so it is for a value you are inserting into a URL, like a search term or a redirect target.

One thing that surprises people: encodeURIComponent does not escape !, ', (, ), or *. Those are valid sub-delimiters, so the output is still correct, but it is not the strictest possible encoding.

The redirect example from the top, fixed:

const target = 'https://app.example.com/dashboard?tab=billing';
const login = 'https://app.example.com/login?next=' + encodeURIComponent(target);

// https://app.example.com/login?next=https%3A%2F%2Fapp.example.com%2Fdashboard%3Ftab%3Dbilling
Enter fullscreen mode Exit fullscreen mode

Now the inner ? cannot be mistaken for the outer one.

Spaces: %20 or +

new URLSearchParams({ q: 'a b' }).toString();   // 'q=a+b'
new URLSearchParams({ q: 'a+b' }).toString();   // 'q=a%2Bb'
Enter fullscreen mode Exit fullscreen mode

URLSearchParams follows the form convention, so it writes spaces as + and escapes a literal plus as %2B. That is fine for query strings, and it is a good reason to use the API instead of building query strings by hand.

The slash that is not a slash

/ can legally be encoded as %2F. That does not mean every server treats it as data. Many frameworks decode the path before routing, then see a path separator. Proxies and CDNs differ in whether they decode at all. The result is a class of confusing bugs: a URL that returns 200 locally, 404 behind a load balancer, and 403 in a WAF.

If you need an arbitrary identifier inside a path segment, avoid relying on %2F. Use base64url, a UUID, or an opaque ID instead. And when validating decoded input, decode once and check the result — never decode repeatedly, because that is how encoded traversal sequences slip through.

The short version

  • Only A-Z a-z 0-9 - . _ ~ are unreserved. Everything else gets percent-encoded.
  • Encoding works on UTF-8 bytes, so é becomes %C3%A9.
  • Use encodeURIComponent for values and encodeURI for whole URLs.
  • Encode once. %2520 means someone encoded twice.
  • + means space only in form-encoded query strings.
  • Do not put %2F in a path and expect a normal slash.

I put together a browser-based URL encoder and decoder for checking these cases without sending anything to a server — useful when the URL contains a token or an internal hostname you would rather not paste into someone else's tool.

If you have ever wondered why a hash cannot be "decrypted", I covered the neighbouring question in an earlier post: hashing is one-way, encoding is reversible, and encryption is the one that needs a key.

I build small, privacy-first developer tools that run entirely in the browser. Most of them exist because I needed something for five minutes and every alternative wanted an upload or an account first.

Top comments (0)