Hardi

Posted on Oct 28

URL Encoder/Decoder: Master URL Encoding for Web Development

#javascript #tutorial #webdev

URL encoding (also called percent-encoding) is essential for building robust web applications. From handling special characters in URLs to implementing proper API endpoints, understanding URL encoding prevents bugs, security vulnerabilities, and broken links. Let's master URL encoding and decoding for modern web development.

Why URL Encoding Matters

The URL Special Character Problem

// The fundamental web problem
const urlProblems = {
  issue: 'URLs can only contain specific ASCII characters',
  restricted: 'Spaces, special characters, non-ASCII break URLs',

  exampleFailures: {
    search: 'https://example.com/search?q=hello world',     // ✗ Space breaks it
    userInput: 'https://example.com/user?name=John & Jane', // ✗ & confuses parser
    international: 'https://example.com/search?q=你好',      // ✗ Non-ASCII fails
    special: 'https://example.com/file?path=/docs/file#1'   // ✗ # ends URL
  },

  solution: 'URL encoding converts unsafe characters to %XX format',

  afterEncoding: {
    search: 'https://example.com/search?q=hello%20world',
    userInput: 'https://example.com/user?name=John%20%26%20Jane',
    international: 'https://example.com/search?q=%E4%BD%A0%E5%A5%BD',
    special: 'https://example.com/file?path=%2Fdocs%2Ffile%231'
  }
};

console.log('URL encoding: Make any string URL-safe');

Real-World Impact

// Before URL encoding: Broken search
const brokenSearch = {
  userQuery: 'javascript tutorials',
  url: `https://api.example.com/search?q=${userQuery}`,
  result: 'https://api.example.com/search?q=javascript tutorials',
  problem: 'Space breaks URL, searches for "javascript" only',
  bugReports: 'Countless'
};

// With URL encoding: Perfect search
const fixedSearch = {
  userQuery: 'javascript tutorials',
  url: `https://api.example.com/search?q=${encodeURIComponent(userQuery)}`,
  result: 'https://api.example.com/search?q=javascript%20tutorials',
  outcome: 'Full query preserved, search works correctly',
  bugReports: 'Zero'
};

// Security vulnerability example
const xssVulnerable = {
  userInput: '<script>alert("XSS")</script>',
  badUrl: `https://example.com/search?q=${userInput}`,
  result: 'XSS attack possible',

  fixed: `https://example.com/search?q=${encodeURIComponent(userInput)}`,
  result2: 'https://example.com/search?q=%3Cscript%3E...',
  outcome: 'Attack neutralized'
};

// Real production incident:
// E-commerce site with product URLs
// Product name: "Coffee & Tea Set"
// Without encoding: /product?name=Coffee & Tea Set
// Result: "& Tea Set" treated as separate parameter
// Lost sales: Thousands per month
// Fix: encodeURIComponent('Coffee & Tea Set')
// Result: /product?name=Coffee%20%26%20Tea%20Set

How URL Encoding Works

Percent-Encoding Explained

// The encoding algorithm
function explainURLEncoding(text) {
  console.log(`\nEncoding: "${text}"\n`);

  const encoded = encodeURIComponent(text);

  console.log('Character-by-character breakdown:\n');
  console.log('Char | ASCII | Hex  | Encoded | Rule');
  console.log('-----|-------|------|---------|-----');

  Array.from(text).forEach(char => {
    const code = char.charCodeAt(0);
    const hex = code.toString(16).toUpperCase().padStart(2, '0');

    // Determine if character needs encoding
    let encodedChar = char;
    let rule = 'Safe';

    if (char === ' ') {
      encodedChar = '%20';
      rule = 'Space → %20';
    } else if (/[^A-Za-z0-9\-_.~]/.test(char)) {
      encodedChar = '%' + hex;
      rule = 'Special → %' + hex;
    }

    const display = char === ' ' ? '·' : char;
    console.log(`${display.padEnd(4)} | ${code.toString().padEnd(5)} | 0x${hex} | ${encodedChar.padEnd(7)} | ${rule}`);
  });

  console.log(`\nFinal: ${encoded}\n`);
  return encoded;
}

// Example
explainURLEncoding('Hello World!');

// Output:
// H    | 72    | 0x48 | H       | Safe
// e    | 101   | 0x65 | e       | Safe
// l    | 108   | 0x6C | l       | Safe
// l    | 108   | 0x6C | l       | Safe
// o    | 111   | 0x6F | o       | Safe
// ·    | 32    | 0x20 | %20     | Space → %20
// W    | 87    | 0x57 | W       | Safe
// o    | 111   | 0x6F | o       | Safe
// r    | 114   | 0x72 | r       | Safe
// l    | 108   | 0x6C | l       | Safe
// d    | 100   | 0x64 | d       | Safe
// !    | 33    | 0x21 | %21     | Special → %21
// Final: Hello%20World%21

Reserved vs Unreserved Characters

// RFC 3986 character classification
const urlCharacters = {
  unreserved: {
    chars: 'A-Z a-z 0-9 - _ . ~',
    rule: 'Never encoded (safe in all contexts)',
    examples: ['Hello', 'Test-File', 'version_2', 'user.name', 'file~1']
  },

  reserved: {
    general: {
      chars: ': / ? # [ ] @ ! $ & \' ( ) * + , ; =',
      rule: 'Have special meaning, encode in data',
      examples: {
        ':': 'Separates scheme/host/port',
        '/': 'Path separator',
        '?': 'Starts query string',
        '#': 'Starts fragment',
        '&': 'Separates query parameters',
        '=': 'Separates key/value'
      }
    }
  },

  mustEncode: {
    chars: 'Space and everything else',
    rule: 'Always encode these',
    examples: [' ', '<', '>', '"', '%', '{', '}', '|', '\\', '^', '`']
  },

  unicode: {
    rule: 'Encode as UTF-8 bytes, then percent-encode',
    example: {
      char: '你',
      utf8: [0xE4, 0xBD, 0xA0],
      encoded: '%E4%BD%A0'
    }
  }
};

// Safe characters (RFC 3986)
const SAFE_CHARS = /[A-Za-z0-9\-_.~]/;

function shouldEncode(char) {
  return !SAFE_CHARS.test(char);
}

console.log('Should encode space?', shouldEncode(' '));      // true
console.log('Should encode "A"?', shouldEncode('A'));        // false
console.log('Should encode "/"?', shouldEncode('/'));        // true
console.log('Should encode "-"?', shouldEncode('-'));        // false

JavaScript URL Encoding Functions

The Three Encoding Functions

// JavaScript provides three URL encoding functions
class URLEncodingGuide {
  static compare(input) {
    console.log(`\nInput: "${input}"\n`);

    const results = {
      escape: escape(input),                    // Deprecated!
      encodeURI: encodeURI(input),
      encodeURIComponent: encodeURIComponent(input)
    };

    console.log('Function             | Result');
    console.log('---------------------|-------');
    console.log(`escape (deprecated)  | ${results.escape}`);
    console.log(`encodeURI            | ${results.encodeURI}`);
    console.log(`encodeURIComponent   | ${results.encodeURIComponent}`);
    console.log('');

    return results;
  }

  static explainDifferences() {
    console.log('\n=== URL Encoding Function Guide ===\n');

    console.log('1. escape() - DEPRECATED, DO NOT USE');
    console.log('   - Uses %XX for ASCII, %uXXXX for Unicode');
    console.log('   - Not RFC 3986 compliant');
    console.log('   - Use encodeURIComponent() instead\n');

    console.log('2. encodeURI() - Encode full URL');
    console.log('   - Preserves URL structure: :/?#[]@!$&\'()*+,;=');
    console.log('   - Use for: Complete URLs');
    console.log('   - Example: encodeURI("http://example.com/path name")');
    console.log('   - Result:  "http://example.com/path%20name"\n');

    console.log('3. encodeURIComponent() - Encode URL parts');
    console.log('   - Encodes everything except: A-Za-z0-9-_.~');
    console.log('   - Use for: Query parameters, path segments');
    console.log('   - Example: encodeURIComponent("name=John & Jane")');
    console.log('   - Result:  "name%3DJohn%20%26%20Jane"\n');
  }
}

// Compare the three functions
URLEncodingGuide.compare('Hello World & Test=123');
URLEncodingGuide.explainDifferences();

// Practical examples
const examples = {
  // Use encodeURI for full URLs
  fullURL: {
    original: 'http://example.com/search?q=hello world',
    encoded: encodeURI('http://example.com/search?q=hello world'),
    result: 'http://example.com/search?q=hello%20world',
    note: 'Preserves URL structure (://?)'
  },

  // Use encodeURIComponent for URL parts
  queryParam: {
    param: 'hello world & test',
    encoded: encodeURIComponent('hello world & test'),
    result: 'hello%20world%20%26%20test',
    note: 'Encodes & and = for safety'
  }
};

// CRITICAL RULE:
console.log('\n⚠️  ALWAYS use encodeURIComponent() for query parameters!');
console.log('✓ Use encodeURI() only for complete URLs');
console.log('✗ Never use escape() - it\'s deprecated\n');

Implementation Methods

1. Complete JavaScript URL Encoder

// Production-ready URL encoder/decoder
class URLEncoder {
  // Encode query parameter
  static encodeParam(value) {
    return encodeURIComponent(value);
  }

  // Decode query parameter
  static decodeParam(value) {
    try {
      return decodeURIComponent(value);
    } catch (e) {
      // Handle malformed sequences
      return value;
    }
  }

  // Build URL with query parameters
  static buildURL(base, params) {
    const queryString = Object.entries(params)
      .map(([key, value]) => 
        `${this.encodeParam(key)}=${this.encodeParam(value)}`
      )
      .join('&');

    return queryString ? `${base}?${queryString}` : base;
  }

  // Parse query string
  static parseQueryString(queryString) {
    const params = {};

    // Remove leading ? if present
    const query = queryString.startsWith('?') 
      ? queryString.substring(1) 
      : queryString;

    query.split('&').forEach(param => {
      const [key, value] = param.split('=');
      if (key) {
        params[this.decodeParam(key)] = value ? this.decodeParam(value) : '';
      }
    });

    return params;
  }

  // Extract query parameters from URL
  static getQueryParams(url) {
    const queryStart = url.indexOf('?');
    if (queryStart === -1) return {};

    const queryString = url.substring(queryStart + 1);
    const hashIndex = queryString.indexOf('#');
    const query = hashIndex !== -1 
      ? queryString.substring(0, hashIndex) 
      : queryString;

    return this.parseQueryString(query);
  }

  // Update query parameter in URL
  static updateParam(url, key, value) {
    const [base, rest] = url.split('?');
    const params = rest ? this.parseQueryString(rest) : {};

    params[key] = value;

    return this.buildURL(base, params);
  }

  // Remove query parameter from URL
  static removeParam(url, key) {
    const [base, rest] = url.split('?');
    if (!rest) return url;

    const params = this.parseQueryString(rest);
    delete params[key];

    return this.buildURL(base, params);
  }

  // Validate URL encoding
  static isProperlyEncoded(str) {
    try {
      return str === encodeURIComponent(decodeURIComponent(str));
    } catch (e) {
      return false;
    }
  }

  // Find encoding issues
  static findEncodingIssues(url) {
    const issues = [];

    // Check for unencoded spaces
    if (url.includes(' ')) {
      issues.push('Contains unencoded spaces');
    }

    // Check for unencoded special characters in query
    const queryStart = url.indexOf('?');
    if (queryStart !== -1) {
      const query = url.substring(queryStart + 1);
      const unsafe = /[<>"{}|\\^`\[\]]/.exec(query);
      if (unsafe) {
        issues.push(`Contains unencoded special character: ${unsafe[0]}`);
      }
    }

    // Check for double encoding
    if (url.includes('%25')) {
      issues.push('Possibly double-encoded (contains %25)');
    }

    return {
      hasIssues: issues.length > 0,
      issues
    };
  }
}

// Usage examples
const params = {
  search: 'javascript tutorials',
  category: 'web & mobile',
  page: 1
};

const url = URLEncoder.buildURL('https://example.com/search', params);
console.log('Built URL:', url);
// https://example.com/search?search=javascript%20tutorials&category=web%20%26%20mobile&page=1

const parsed = URLEncoder.getQueryParams(url);
console.log('Parsed params:', parsed);
// { search: 'javascript tutorials', category: 'web & mobile', page: '1' }

const updated = URLEncoder.updateParam(url, 'page', 2);
console.log('Updated URL:', updated);

// Check for issues
const problematic = 'https://example.com/search?q=hello world&user=<script>';
const issues = URLEncoder.findEncodingIssues(problematic);
console.log('Issues:', issues);

2. Express API for URL Operations

const express = require('express');
const app = express();

app.use(express.json());

// Encode text for URL
app.post('/api/url/encode', (req, res) => {
  try {
    const { text, type = 'component' } = req.body;

    if (!text) {
      return res.status(400).json({ error: 'Text required' });
    }

    let encoded;
    if (type === 'component') {
      encoded = encodeURIComponent(text);
    } else if (type === 'uri') {
      encoded = encodeURI(text);
    } else {
      return res.status(400).json({ error: 'Invalid type' });
    }

    res.json({
      original: text,
      encoded,
      type,
      length: {
        original: text.length,
        encoded: encoded.length
      }
    });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

// Decode URL-encoded text
app.post('/api/url/decode', (req, res) => {
  try {
    const { text } = req.body;

    if (!text) {
      return res.status(400).json({ error: 'Text required' });
    }

    const decoded = decodeURIComponent(text);

    res.json({
      encoded: text,
      decoded,
      length: {
        encoded: text.length,
        decoded: decoded.length
      }
    });
  } catch (error) {
    res.status(400).json({ 
      error: 'Invalid URL encoding',
      details: error.message
    });
  }
});

// Build URL with parameters
app.post('/api/url/build', (req, res) => {
  try {
    const { base, params } = req.body;

    if (!base) {
      return res.status(400).json({ error: 'Base URL required' });
    }

    const queryString = Object.entries(params || {})
      .map(([key, value]) => 
        `${encodeURIComponent(key)}=${encodeURIComponent(value)}`
      )
      .join('&');

    const url = queryString ? `${base}?${queryString}` : base;

    res.json({
      base,
      params,
      url,
      queryString
    });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

// Parse URL
app.post('/api/url/parse', (req, res) => {
  try {
    const { url } = req.body;

    if (!url) {
      return res.status(400).json({ error: 'URL required' });
    }

    const urlObj = new URL(url);
    const params = {};

    urlObj.searchParams.forEach((value, key) => {
      params[key] = value;
    });

    res.json({
      original: url,
      protocol: urlObj.protocol,
      host: urlObj.host,
      hostname: urlObj.hostname,
      port: urlObj.port,
      pathname: urlObj.pathname,
      search: urlObj.search,
      hash: urlObj.hash,
      params
    });
  } catch (error) {
    res.status(400).json({ 
      error: 'Invalid URL',
      details: error.message
    });
  }
});

// Validate URL encoding
app.post('/api/url/validate', (req, res) => {
  const { url } = req.body;

  if (!url) {
    return res.status(400).json({ error: 'URL required' });
  }

  const issues = [];

  // Check for spaces
  if (url.includes(' ')) {
    issues.push('Contains unencoded spaces');
  }

  // Check for unsafe characters
  const unsafe = /[<>"{}|\\^`\[\]]/.exec(url);
  if (unsafe) {
    issues.push(`Contains unsafe character: ${unsafe[0]}`);
  }

  // Check for double encoding
  if (url.includes('%25')) {
    issues.push('Possibly double-encoded');
  }

  // Try to parse
  let parseable = true;
  try {
    new URL(url);
  } catch (e) {
    parseable = false;
    issues.push('Not a valid URL');
  }

  res.json({
    url,
    valid: issues.length === 0 && parseable,
    parseable,
    issues
  });
});

app.listen(3000, () => {
  console.log('URL Encoder API running on port 3000');
  console.log('POST /api/url/encode - Encode text');
  console.log('POST /api/url/decode - Decode text');
  console.log('POST /api/url/build - Build URL with params');
  console.log('POST /api/url/parse - Parse URL');
  console.log('POST /api/url/validate - Validate encoding');
});

3. Python Implementation

from urllib.parse import quote, unquote, urlencode, parse_qs, urlparse, parse_qsl

class URLEncoder:
    @staticmethod
    def encode(text, safe=''):
        """Encode text for URL (like encodeURIComponent)"""
        return quote(text, safe=safe)

    @staticmethod
    def decode(text):
        """Decode URL-encoded text"""
        return unquote(text)

    @staticmethod
    def build_url(base, params):
        """Build URL with query parameters"""
        if not params:
            return base

        query_string = urlencode(params)
        return f"{base}?{query_string}"

    @staticmethod
    def parse_query_string(query):
        """Parse query string to dictionary"""
        # parse_qs returns lists, parse_qsl returns tuples
        return dict(parse_qsl(query))

    @staticmethod
    def get_query_params(url):
        """Extract query parameters from URL"""
        parsed = urlparse(url)
        return dict(parse_qsl(parsed.query))

    @staticmethod
    def parse_url(url):
        """Parse URL into components"""
        parsed = urlparse(url)
        return {
            'scheme': parsed.scheme,
            'netloc': parsed.netloc,
            'path': parsed.path,
            'params': parsed.params,
            'query': parsed.query,
            'fragment': parsed.fragment,
            'query_params': dict(parse_qsl(parsed.query))
        }

# Usage
text = "Hello World & Test"
encoded = URLEncoder.encode(text)
print(f"Encoded: {encoded}")
# Hello%20World%20%26%20Test

decoded = URLEncoder.decode(encoded)
print(f"Decoded: {decoded}")
# Hello World & Test

# Build URL
params = {
    'search': 'python tutorial',
    'category': 'programming',
    'page': 1
}
url = URLEncoder.build_url('https://example.com/search', params)
print(f"URL: {url}")

# Parse URL
parsed = URLEncoder.parse_url(url)
print(f"Parsed: {parsed}")

4. Quick Online Encoding/Decoding

For rapid testing, debugging URLs, or learning how encoding works, using a URL encoder/decoder can quickly encode/decode without writing code. This is particularly useful when:

Debugging APIs: Decode error messages with encoded URLs
Testing endpoints: Encode query parameters correctly
Learning: See encoding patterns visually
Client support: Help debug URL issues

For production applications, always use proper encoding functions in your codebase to prevent security issues and bugs.

Real-World Use Cases

1. Search Functionality

// Proper search implementation
class SearchHandler {
  static buildSearchURL(query, filters = {}) {
    const params = {
      q: query,
      ...filters
    };

    return URLEncoder.buildURL('/api/search', params);
  }

  static handleSearch(userInput) {
    // User types: "javascript & react tutorial"
    const searchURL = this.buildSearchURL(userInput, {
      category: 'web development',
      sort: 'relevance'
    });

    console.log('Search URL:', searchURL);
    // /api/search?q=javascript%20%26%20react%20tutorial&category=web%20development&sort=relevance

    return fetch(searchURL);
  }
}

// Without encoding - BROKEN
const badSearch = `/api/search?q=${userQuery}`;  // ✗ Breaks with spaces/special chars

// With encoding - CORRECT
const goodSearch = `/api/search?q=${encodeURIComponent(userQuery)}`;  // ✓ Works perfectly

2. OAuth Redirect URLs

// OAuth callback URL handling
class OAuthHandler {
  static buildAuthURL(clientId, redirectUri, scope) {
    // redirectUri MUST be encoded
    const params = {
      client_id: clientId,
      redirect_uri: redirectUri,  // Will be encoded by buildURL
      response_type: 'code',
      scope: scope.join(' ')  // Scopes separated by spaces
    };

    return URLEncoder.buildURL('https://oauth.example.com/authorize', params);
  }

  static example() {
    const authURL = this.buildAuthURL(
      'my-app-123',
      'https://myapp.com/callback?state=xyz',  // Complex redirect
      ['read', 'write', 'delete']
    );

    console.log('OAuth URL:', authURL);
    // https://oauth.example.com/authorize?
    //   client_id=my-app-123&
    //   redirect_uri=https%3A%2F%2Fmyapp.com%2Fcallback%3Fstate%3Dxyz&
    //   response_type=code&
    //   scope=read%20write%20delete
  }
}

3. File Download with Special Characters

// Handle filenames with spaces and special characters
class FileDownloader {
  static createDownloadURL(filename, folder) {
    // Filename: "Project Report (Final) - 2024.pdf"
    // Without encoding: BREAKS
    // With encoding: WORKS

    const encodedFilename = encodeURIComponent(filename);
    const encodedFolder = encodeURIComponent(folder);

    return `/api/download/${encodedFolder}/${encodedFilename}`;
  }

  static handleDownload() {
    const filename = 'Project Report (Final) - 2024.pdf';
    const folder = 'Reports & Documents';

    const url = this.createDownloadURL(filename, folder);
    console.log('Download URL:', url);
    // /api/download/Reports%20%26%20Documents/Project%20Report%20(Final)%20-%202024.pdf

    window.location.href = url;
  }
}

4. Social Media Sharing

// Share URLs with pre-filled content
class SocialShare {
  static shareOnTwitter(text, url) {
    const params = {
      text: text,
      url: url
    };

    const shareURL = URLEncoder.buildURL('https://twitter.com/intent/tweet', params);
    window.open(shareURL, '_blank');
  }

  static shareOnFacebook(url) {
    const params = { u: url };
    const shareURL = URLEncoder.buildURL('https://www.facebook.com/sharer/sharer.php', params);
    window.open(shareURL, '_blank');
  }

  static shareViaEmail(subject, body) {
    // mailto: links also need encoding!
    const mailtoURL = `mailto:?subject=${encodeURIComponent(subject)}&body=${encodeURIComponent(body)}`;
    window.location.href = mailtoURL;
  }

  static example() {
    this.shareOnTwitter(
      'Check out this amazing article about web development! 🚀',
      'https://example.com/article?id=123&ref=social'
    );
    // Opens: https://twitter.com/intent/tweet?
    //   text=Check%20out%20this%20amazing%20article%20about%20web%20development!%20%F0%9F%9A%80&
    //   url=https%3A%2F%2Fexample.com%2Farticle%3Fid%3D123%26ref%3Dsocial
  }
}

5. Analytics and Tracking

// UTM parameters for campaign tracking
class AnalyticsTracker {
  static addUTMParams(url, campaign) {
    const params = {
      utm_source: campaign.source,
      utm_medium: campaign.medium,
      utm_campaign: campaign.name,
      utm_content: campaign.content,
      utm_term: campaign.term
    };

    // Remove undefined values
    Object.keys(params).forEach(key => 
      params[key] === undefined && delete params[key]
    );

    return URLEncoder.buildURL(url, params);
  }

  static example() {
    const trackedURL = this.addUTMParams(
      'https://example.com/products',
      {
        source: 'email newsletter',
        medium: 'email',
        name: 'summer sale 2024',
        content: 'top banner'
      }
    );

    console.log('Tracked URL:', trackedURL);
    // https://example.com/products?
    //   utm_source=email%20newsletter&
    //   utm_medium=email&
    //   utm_campaign=summer%20sale%202024&
    //   utm_content=top%20banner
  }
}

Common Pitfalls and Solutions

// Pitfall #1: Double encoding
const WRONG1 = {
  input: 'hello world',
  step1: encodeURIComponent('hello world'),  // 'hello%20world'
  step2: encodeURIComponent('hello%20world'),  // 'hello%2520world' ✗ WRONG!
  problem: 'Encoded twice, breaks decoding'
};

const RIGHT1 = {
  input: 'hello world',
  encoded: encodeURIComponent('hello world'),  // 'hello%20world' ✓
  rule: 'Encode once, at the right time'
};

// Pitfall #2: Encoding the entire URL
const WRONG2 = {
  url: 'https://example.com/search?q=test',
  bad: encodeURIComponent('https://example.com/search?q=test'),
  result: 'https%3A%2F%2Fexample.com%2Fsearch%3Fq%3Dtest',  // ✗ Unusable!
  problem: 'Encoded URL structure characters'
};

const RIGHT2 = {
  base: 'https://example.com/search',
  param: 'test value',
  good: `https://example.com/search?q=${encodeURIComponent('test value')}`,
  result: 'https://example.com/search?q=test%20value',  // ✓ Perfect!
  rule: 'Only encode parameter values, not URL structure'
};

// Pitfall #3: Forgetting to encode
const WRONG3 = {
  input: 'hello world',
  url: `https://example.com/search?q=${input}`,  // ✗ Space breaks it
  problem: 'Special characters break URL'
};

const RIGHT3 = {
  input: 'hello world',
  url: `https://example.com/search?q=${encodeURIComponent(input)}`,  // ✓
  rule: 'ALWAYS encode user input'
};

// Pitfall #4: Using wrong function
const WRONG4 = {
  param: 'hello&world',
  bad: encodeURI('hello&world'),  // 'hello&world' ✗ & not encoded!
  problem: 'encodeURI doesn\'t encode &'
};

const RIGHT4 = {
  param: 'hello&world',
  good: encodeURIComponent('hello&world'),  // 'hello%26world' ✓
  rule: 'Use encodeURIComponent for parameters'
};

console.log('⚠️  Common URL Encoding Mistakes:');
console.log('✗ Double encoding');
console.log('✗ Encoding entire URL');
console.log('✗ Forgetting to encode');
console.log('✗ Using encodeURI for parameters');
console.log('\n✓ Solution: Always use encodeURIComponent for values!');

Testing URL Encoding

// Jest tests
describe('URL Encoding', () => {
  test('encodes special characters', () => {
    const input = 'hello world & test';
    const encoded = encodeURIComponent(input);

    expect(encoded).toBe('hello%20world%20%26%20test');
    expect(encoded).not.toContain(' ');
    expect(encoded).not.toContain('&');
  });

  test('decodes correctly', () => {
    const encoded = 'hello%20world%20%26%20test';
    const decoded = decodeURIComponent(encoded);

    expect(decoded).toBe('hello world & test');
  });

  test('handles unicode', () => {
    const input = '你好世界';
    const encoded = encodeURIComponent(input);
    const decoded = decodeURIComponent(encoded);

    expect(decoded).toBe(input);
  });

  test('builds URL correctly', () => {
    const url = URLEncoder.buildURL('/search', {
      q: 'hello world',
      category: 'web & mobile'
    });

    expect(url).toContain('q=hello%20world');
    expect(url).toContain('category=web%20%26%20mobile');
  });

  test('parses query string', () => {
    const params = URLEncoder.parseQueryString('q=hello%20world&page=1');

    expect(params.q).toBe('hello world');
    expect(params.page).toBe('1');
  });
});

Conclusion: URL Encoding for Robust Web Apps

URL encoding isn't optional—it's essential for building reliable web applications. From search functionality to OAuth flows, file downloads to social sharing, proper URL encoding prevents bugs, security vulnerabilities, and broken user experiences.

✅ Prevents broken URLs (spaces, special chars)

✅ Enables international text (Unicode support)

✅ Security (prevents injection attacks)

✅ Data integrity (preserves exact values)

✅ Protocol compliance (RFC 3986 standard)

✅ Universal support (every language/platform)

✅ Reversible (perfect encoding/decoding)

✅ Required for APIs (REST, OAuth, webhooks)

Critical Rules:

✓ ALWAYS use encodeURIComponent() for query parameters
✓ Use encodeURI() only for complete URLs
✓ Encode user input before adding to URLs
✓ Never double-encode
✓ Don't encode URL structure (://?&#)
✗ Never use escape() - it's deprecated
✗ Never forget to encode - security risk!

The Bottom Line:
URL encoding is fundamental web development knowledge. Every broken search, failed OAuth flow, or corrupted filename can be traced back to improper URL encoding. Master encodeURIComponent(), use it religiously for user input, and you'll prevent countless bugs and security issues. It takes 2 seconds to encode a parameter—it takes hours to debug why URLs are breaking in production.

What URL encoding bugs have you encountered? Share your war stories in the comments!

webdev #url #encoding #javascript #api

DEV Community