DEV Community: CheBit

URL encoding, explained by the bugs it causes

CheBit — Thu, 22 Oct 2026 14:43:59 +0000

A link that works when you click it can still break when you paste it somewhere else.

https://app.example.com/login?next=https://app.example.com/dashboard?tab=billing

That looks like a redirect to a dashboard tab. But look at the second ?. Is tab=billing part of the next value, or a second parameter on the login URL? Different parsers answer differently. One of them sends the user to the right place; another sends them to a 404.

The fix is percent-encoding. And it is one of those things everyone uses and almost nobody can explain precisely, which is why the same three bugs keep showing up in production.

What percent-encoding actually does

Percent-encoding works on bytes, not characters. It leaves a small set of characters alone and rewrites everything else as % followed by two hexadecimal digits.

The characters that never need encoding — the "unreserved" set from RFC 3986 — are:

A-Z  a-z  0-9  -  .  _  ~

Everything else becomes bytes, and each byte becomes %XX.

encodeURIComponent('a b/c');   // 'a%20b%2Fc'
encodeURIComponent('é');       // '%C3%A9'
encodeURIComponent('日');      // '%E6%97%A5'

Notice that é is a single character but produces two escapes. That is not a quirk of the encoding function — é is two bytes in UTF-8, and percent-encoding escapes bytes. This is also why URL-encoded strings look so much longer than the text they came from.

The three characters behind most of the bugs

& separates parameters. If it appears inside a value, it has to be %26. Otherwise the parser starts a new parameter and the rest of your value becomes a stray key.

encodeURIComponent('a&b');     // 'a%26b'
encodeURIComponent('?');       // '%3F'

+ means space only in form-encoded data. That convention comes from HTML form submission (application/x-www-form-urlencoded), not from the URL standard itself. In a query string most servers accept both + and %20 as a space. In a path segment, + is a literal plus, not a space.

% starts an escape. A literal percent sign must be encoded as %25. This is where double encoding begins.

The double-encoding trap

encodeURIComponent('%20');                        // '%2520'
encodeURIComponent(encodeURIComponent('a b'));    // 'a%2520b'

If you encode a string that was already encoded, the % becomes %25 and the escape stops working. The server decodes once and gets %20 back as literal text, never a space. Strings containing %2520 in a browser address bar are almost always this bug.

The cause is usually two layers each assuming the other did not encode. A client encodes a value, then a library encodes the whole URL again. The rule is simple: encode exactly once, at the boundary where the value enters the URL.

encodeURI vs encodeURIComponent

JavaScript gives you two functions with confusingly similar names.

encodeURI('https://example.com/a b');
// 'https://example.com/a%20b'

encodeURIComponent('https://example.com/a b');
// 'https%3A%2F%2Fexample.com%2Fa%20b'

encodeURI preserves the characters that give a URL its structure — : / ? # & = — so it is for a complete URL you want to keep intact. encodeURIComponent escapes all of those, so it is for a value you are inserting into a URL, like a search term or a redirect target.

One thing that surprises people: encodeURIComponent does not escape !, ', (, ), or *. Those are valid sub-delimiters, so the output is still correct, but it is not the strictest possible encoding.

The redirect example from the top, fixed:

const target = 'https://app.example.com/dashboard?tab=billing';
const login = 'https://app.example.com/login?next=' + encodeURIComponent(target);

// https://app.example.com/login?next=https%3A%2F%2Fapp.example.com%2Fdashboard%3Ftab%3Dbilling

Now the inner ? cannot be mistaken for the outer one.

Spaces: %20 or +

new URLSearchParams({ q: 'a b' }).toString();   // 'q=a+b'
new URLSearchParams({ q: 'a+b' }).toString();   // 'q=a%2Bb'

URLSearchParams follows the form convention, so it writes spaces as + and escapes a literal plus as %2B. That is fine for query strings, and it is a good reason to use the API instead of building query strings by hand.

The slash that is not a slash

/ can legally be encoded as %2F. That does not mean every server treats it as data. Many frameworks decode the path before routing, then see a path separator. Proxies and CDNs differ in whether they decode at all. The result is a class of confusing bugs: a URL that returns 200 locally, 404 behind a load balancer, and 403 in a WAF.

If you need an arbitrary identifier inside a path segment, avoid relying on %2F. Use base64url, a UUID, or an opaque ID instead. And when validating decoded input, decode once and check the result — never decode repeatedly, because that is how encoded traversal sequences slip through.

The short version

Only A-Z a-z 0-9 - . _ ~ are unreserved. Everything else gets percent-encoded.
Encoding works on UTF-8 bytes, so é becomes %C3%A9.
Use encodeURIComponent for values and encodeURI for whole URLs.
Encode once. %2520 means someone encoded twice.
+ means space only in form-encoded query strings.
Do not put %2F in a path and expect a normal slash.

I put together a browser-based URL encoder and decoder for checking these cases without sending anything to a server — useful when the URL contains a token or an internal hostname you would rather not paste into someone else's tool.

If you have ever wondered why a hash cannot be "decrypted", I covered the neighbouring question in an earlier post: hashing is one-way, encoding is reversible, and encryption is the one that needs a key.

I build small, privacy-first developer tools that run entirely in the browser. Most of them exist because I needed something for five minutes and every alternative wanted an upload or an account first.

Why your base64-encoded image got bigger

CheBit — Sun, 18 Oct 2026 04:18:23 +0000

A 40 KB icon becomes a 53 KB data URI the moment you inline it. The page still works, the browser still renders it, but the HTML got bigger for no obvious reason.

That is usually the first time anyone notices Base64 is doing something, and it is also where the two most common misconceptions start: that Base64 compresses data, and that it hides it.

It does neither. Base64 is a transport encoding. Once you see what it actually does, both myths fall apart.

Three bytes in, four characters out

Base64 takes the raw bytes of your input and rewrites them using only 64 characters: A-Z, a-z, 0-9, + and /. That is a deliberately boring alphabet — nothing in it needs escaping inside an HTML attribute, a JSON string or an email header. (+ and / do still need encoding in a URL, which is exactly why the URL-safe variant exists; more on that below.)

The arithmetic is the whole story. Each byte is 8 bits. Base64 regroups the stream into 6-bit chunks, and 6 bits can address exactly 64 characters. Three bytes (24 bits) become four characters (4 × 6 bits).

Three in, four out. That is where the 33% expansion comes from — it is not a bug or a bad implementation, it is the definition.

const input = 'hello';                                  // 5 bytes
const encoded = Buffer.from(input).toString('base64');  // 8 characters

encoded;  // 'aGVsbG8='

The = at the end is padding. When the input length is not a multiple of three, the last group is incomplete, and = fills the gap so the output length stays a multiple of four. One or two =, never more.

Not encryption: there is no key

Base64 has no secret. The alphabet is public, the algorithm is public, and every step is reversible by anyone holding the string. There is nothing to "decrypt" because nothing was encrypted.

This matters in one place more than any other: JWTs. A JWT is three Base64url segments joined by dots, and the middle one is the payload. People regularly assume it is encrypted because it looks like noise.

const token = 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjMiLCJyb2xlIjoiYWRtaW4ifQ.signature';
const payload = token.split('.')[1];

JSON.parse(Buffer.from(payload, 'base64url').toString());
// { sub: '123', role: 'admin' }

No key was needed. Anyone who intercepts that token can read the payload. JWTs are signed, not encrypted — the signature proves the token was not altered, it does not hide the contents.

If you need to hide something, encode it after you encrypt it:

plaintext -> encrypt (AES-GCM) -> base64 -> transport

The Base64 step here exists only because binary ciphertext cannot travel safely through text channels. The secrecy comes entirely from the encryption step.

Not compression: it makes things bigger

The 33% expansion is measurable and predictable.

const bytes = Buffer.alloc(100000);           // 100 KB of bytes
const encoded = bytes.toString('base64');

encoded.length;   // 133336  (~133 KB)

Compression reduces size by finding repetition. Base64 does the opposite: it takes a dense byte stream and spreads it across a larger set of characters. It cannot reduce entropy, because it does not try.

The practical consequence is that inlining assets as data URIs is a trade-off, not a free win. For a 1 KB SVG icon it can be worth it — you save an HTTP request and the overhead is negligible. For a 500 KB photo it is usually the wrong call: you pay 33% more HTML on every page load, and a large inline URI holds up the parser before anything renders.

The rule of thumb: inline things under a few kilobytes, link to everything else.

Where Base64 is the right tool

It exists because some channels only accept text. Base64 turns arbitrary bytes into something those channels can carry unchanged.

Data URIs. src="data:image/png;base64,..." for small icons that should not cost a request.
Email attachments (MIME). Binary files are Base64-encoded so they can travel through text-based mail protocols. MIME also wraps the output at 76 characters per line.
Binary payloads in JSON. Many APIs accept a file as a Base64 string instead of a multipart body.
Base64url. A URL-safe variant that swaps + for - and / for _, and usually drops the padding. JWTs and OAuth parameters use it.
Basic auth. Authorization: Basic base64(user:password) — again encoded for transport, not protected.

In every one of those cases, the reason is the same: the data is safe to carry as text, not secret and not smaller.

The trap: btoa does not handle Unicode

Base64 operates on bytes, not characters. That distinction bites JavaScript developers because btoa() predates Unicode-aware APIs and only accepts Latin-1.

btoa('hello');   // 'aGVsbG8='
btoa('héllo');   // 'aOlsbG8='  — Latin-1 bytes, not UTF-8
btoa('€');       // throws: InvalidCharacterError

The second line is the dangerous one. Because é sits inside Latin-1, btoa does not complain. It silently encodes the single Latin-1 byte 0xE9 instead of the two UTF-8 bytes 0xC3 0xA9, so you get valid Base64 that decodes to something different from what you intended. Only characters above U+00FF, like € or 日本, actually throw.

The fix is to convert the string to UTF-8 bytes first:

const toBase64 = (str) =>
  btoa(String.fromCharCode(...new TextEncoder().encode(str)));

const fromBase64 = (b64) =>
  new TextDecoder().decode(
    Uint8Array.from(atob(b64), (c) => c.charCodeAt(0))
  );

toBase64('héllo');          // 'aMOpbGxv'
fromBase64('aMOpbGxv');     // 'héllo'

This is also why two people can encode "the same text" and get different results: UTF-8 and UTF-16 produce different bytes, and Base64 faithfully encodes whichever bytes you gave it.

The short version

Base64 is a reversible, keyless, 33%-larger representation of bytes, designed to move binary data through text-only channels. It is not compression and it is not encryption.

Use it when a channel cannot carry bytes. Do not use it to shrink files, and never confuse it with protecting them.

I put together a browser-based Base64 encoder and decoder for the everyday cases — text in, Base64 out, and back again, with the Unicode handling above taken care of. It runs locally, so nothing you paste is uploaded anywhere.

If you have ever wondered whether a hash counts as encryption, I covered that question in a separate post: MD5 is a hash, it cannot be reversed, and it is not a secret in any meaningful sense.

What MD5 actually does (and why it isn't encryption)

CheBit — Tue, 13 Oct 2026 15:32:36 +0000

This snippet shows up in code review all the time, and on its face it looks entirely reasonable:

public User register(String email, String rawPassword) {
    User user = new User();
    user.setEmail(email);
    user.setPassword(DigestUtils.md5Hex(rawPassword));
    return userRepository.save(user);
}

The comment above it said the password was "encrypted before storing". Three things in that snippet are wrong, and pulling them apart is a good way to nail down a distinction that still trips up working developers: encoding, hashing and encryption are three different operations, and only two of them are reversible.

Three operations, three purposes

Encoding changes the representation of data so it can survive transport. It is reversible by design and involves no secret. Base64 is the standard example: it turns arbitrary bytes into a safe subset of ASCII so they can travel through systems that expect text. Anyone can decode it.

Hashing reduces input of any length to a fixed-size digest. It is one-way by design and involves no key. There is nothing to reverse, because the digest is not a compressed copy of the input. It is a summary that many different inputs could have produced.

Encryption transforms data so that only a holder of the key can read it. It is reversible, and the key is the entire point.

The practical rule that follows: if you need the data back, you want encryption. If you only need to check that data has not changed, you want a hash. If you need to move data through a text-only channel, you want encoding.

What MD5 is, concretely

MD5 takes any input and returns 128 bits, conventionally written as 32 lowercase hex characters.

// Node.js
const crypto = require('crypto');

crypto.createHash('md5').update('hello').digest('hex');
// '5d41402abc4b2a76b9719d911017c592'

# Python
import hashlib

hashlib.md5(b'hello').hexdigest()
# '5d41402abc4b2a76b9719d911017c592'

printf 'hello' | md5sum
# 5d41402abc4b2a76b9719d911017c592  -

Three properties matter here. It is deterministic: the same input always produces the same digest. It is fixed-length: one character and a 10 GB disk image both produce 32 hex characters. And it has a strong avalanche effect: changing one bit of input changes about half the output bits.

crypto.createHash('md5').update('hello').digest('hex');
// '5d41402abc4b2a76b9719d911017c592'

crypto.createHash('md5').update('Hello').digest('hex');
// '8b1a9953c4611296a827abf8c47804d7'

One capital letter, and the two digests share nothing. That is exactly what you want when checking whether a file changed, and exactly why you cannot use a hash as storage for something you need to read back.

Why "MD5 decrypt" is a contradiction

Search for "MD5 decrypt" and you will find dozens of sites that will happily decrypt a hash for you. None of them are reversing the function.

What they are actually doing is a lookup. 5f4dcc3b5aa765d61d8327deb882cf99 is the MD5 of password, and that fact has been sitting in public tables for twenty years. These services hash a huge dictionary of candidate inputs, store the results, and hand you back whatever matched.

The distinction matters because the two failure modes are different. You cannot reverse MD5. You can guess the input, and when the input is a human-chosen password, guessing is cheap. MD5 was designed to be fast, and in this context fast is a liability.

MD5 has been broken since 2004

A collision is a pair of different inputs that produce the same hash. For a hash to be useful in any security-sensitive setting, collisions have to be hard to find.

For MD5 they are not. Practical collision attacks were demonstrated in 2004, and by 2008 researchers had used them to forge a certificate that a certificate authority would sign, which is about as serious as a hash break gets.

Practical guidance that follows:

Never hash passwords with MD5. Use bcrypt, scrypt or Argon2id, all of which are deliberately slow and salted.
Never rely on MD5 for signatures or certificates. Use SHA-256 or SHA-3.
Do not treat MD5 as a security boundary. If an attacker has a reason to forge a value, MD5 is the wrong tool.

// Don't
user.setPassword(DigestUtils.md5Hex(password));

// Do
user.setPassword(new BCryptPasswordEncoder().encode(password));

Where MD5 is still perfectly fine

Checksums, cache keys, and detecting accidental corruption. In all three cases the threat model excludes a deliberate attacker who controls the input, and MD5 is fast and universally available.

The most common real-world use is verifying a download:

# macOS
md5 yourfile.iso

# Linux
md5sum yourfile.iso

# Windows
certutil -hashfile yourfile.iso MD5

Compare the output against the value the publisher lists. If it matches, the file arrived intact. If it does not, the download was truncated or swapped, and you should not run it.

The nuance worth keeping in mind: a match proves you have the file the publisher hashed, not that the publisher is trustworthy. And because collisions are cheap, it is a weak guarantee against tampering. For anything adversarial, checksums published over HTTPS with SHA-256 are the better choice.

The short version

Encoding is reversible and takes no key. Encryption is reversible and needs a key. Hashing is neither, and that is the feature, not a limitation.

MD5 is a hash. It is fine for checking that a file survived a download, and unfit for anything an attacker has a reason to forge. If you have MD5 anywhere near passwords, that is a bug worth fixing this week.

I put together a browser-based MD5 hash generator for the legitimate uses. It computes the digest locally, so whatever you paste never leaves your machine, which is useful when the string you are hashing is a token or a key you would rather not paste into someone else's server.

I build small, privacy-first developer tools that run entirely in the browser. Most of them exist because I kept wanting a tool that wouldn't ask me to upload a config file or create an account first.