DEV Community: tinali7564-eng

The Definitive Guide to JWT Security in 2026: HS256 vs RS256 vs ES256, The alg: none Exploit, kid Parameter Attacks, and Token

tinali7564-eng — Fri, 23 Oct 2026 02:03:06 +0000

The Definitive Guide to JWT Security in 2026: HS256 vs RS256 vs ES256, The alg: none Exploit, kid Parameter Attacks, and Token Rotation Architecture

JSON Web Tokens (RFC 7519) are the de facto industry standard for stateless identity assertion, powering microservice service-to-service calls, Single Sign-On (SSO) gateways, and modern OAuth 2.0 / OpenID Connect (OIDC) flows.

Yet despite their ubiquity, JWTs remain one of the most frequently misconfigured primitives in modern application security:

Security researchers routinely discover enterprise authentication endpoints vulnerable to algorithm confusion attacks, where an attacker signs a token using the server's public key as an HMAC symmetric secret.
Naive token verification libraries still fall prey to the infamous "alg": "none" exploit, granting arbitrary administrative privileges to attackers who delete the signature segment.
Attackers compromise identity providers using kid (Key ID) header injections, abusing SQL injections or directory path traversals to force servers into signing tokens with predictable local files (such as /dev/null).
Engineering teams store sensitive tokens in browser localStorage, exposing entire user sessions to single-line Cross-Site Scripting (XSS) credential theft.

This comprehensive architectural guide deconstructs how JSON Web Tokens work under the hood, analyzes the cryptographic mechanics of HS256 vs. RS256 vs. ES256 vs. EdDSA (Ed25519), dissects five catastrophic vulnerability vectors with reproducible proof-of-concepts, and provides a production-grade blueprint for stateless authentication and Refresh Token Rotation (RTR).

1. Deconstructing the 3-Segment Anatomy of a JWT

A compact JWT is a URL-safe string composed of three distinct base64url-encoded segments separated by periods (.):

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c3JfMTIzNCIsImFkbWluIjp0cnVlLCJpYXQiOjE3OTIwNzIwMDB9.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c
└──────────────────┬────────────────┘.└───────────────────────┬──────────────────────┘.└───────────────────────┬──────────────────────┘
                   │                                          │                                          │
            1. HEADER (JSON)                          2. PAYLOAD (JSON)                        3. CRYPTOGRAPHIC SIGNATURE

1. The Header: Metadata & Cryptographic Instructions

The header contains metadata informing the parser how to evaluate the signature:

{
  "alg": "HS256",
  "typ": "JWT",
  "kid": "auth-key-2026-q4"
}

alg: The cryptographic algorithm used to sign the token (HS256, RS256, ES256, etc.).
typ: Token type (conventionally JWT).
kid: Key Identifier, used by multi-tenant systems or key-rotation services to locate the correct public or symmetric key from a JWKS (JSON Web Key Set).

2. The Payload: Registered & Custom Claims

The payload contains the verifiable statements (claims) about the subject:

{
  "iss": "https://auth.dailytoolbox.org",
  "sub": "usr_987456123",
  "aud": "https://api.dailytoolbox.org",
  "exp": 1792075600,
  "nbf": 1792072000,
  "iat": 1792072000,
  "roles": ["admin", "billing"]
}

Registered Claims (RFC 7519 §4.1):
- iss (Issuer): Principal issuing the token.
- sub (Subject): Principal identity (User ID).
- aud (Audience): Intended recipient backend service.
- exp (Expiration Time): Unix timestamp beyond which the token is invalid.
- nbf (Not Before): Unix timestamp before which the token must not be accepted.
- iat (Issued At): Timestamp of generation.

3. The Signature: Cryptographic Integrity Proof

The signature is computed over the base64url-encoded header and payload concatenated with a period:
$$\text{Signing Input} = \text{base64url}(Header) \mathbin{\Vert} \text{"."} \mathbin{\Vert} \text{base64url}(Payload)$$

$$\text{Signature} = \text{Sign}_{Key}(\text{Signing Input})$$

The Fundamental Golden Rule: A JWT is signed, NOT encrypted. Anyone who possesses the token can paste it into an in-browser decoder and inspect every claim in cleartext within 10 milliseconds. Never place sensitive secrets, passwords, or unencrypted personally identifiable information (PII) inside a JWT payload!

2. Cryptographic Algorithm Matrix: HS256 vs. RS256 vs. ES256 vs. Ed25519

Selecting the wrong algorithm is the root cause of countless security vulnerabilities and CPU scalability bottlenecks:

Dimension	HS256 (HMAC-SHA256)	RS256 (RSA-SHA256)	ES256 (ECDSA P-256)	EdDSA (Ed25519)
Type	Symmetric (Shared Secret)	Asymmetric (Public/Private)	Asymmetric (Elliptic Curve)	Asymmetric (Edwards Curve)
Key Size	256 bits (32 bytes min)	2048 to 4096 bits	256 bits	256 bits
Signature Size	32 bytes	256 to 512 bytes	64 bytes	64 bytes
Header Overhead	Minimal	Heavy (~35% larger token)	Compact	Compact
Signing Speed	Extremely fast (~1.2 µs)	Slow (~1,800 µs)	Moderate (~80 µs)	Blazing fast (~25 µs)
Verify Speed	Extremely fast (~1.1 µs)	Fast (~45 µs)	Moderate (~120 µs)	Very fast (~40 µs)
Architecture Fit	Monolithic backends, internal single-tenant services	Multi-tenant OAuth2, legacy third-party integrations	Modern high-throughput microservices	State-of-the-art APIs (2026 standard)

                                  JWT SIGNING PIPELINE
Symmetric (HS256):
Auth Server [Shared Secret "K"] ────► Sign ────► Token ────► Resource Server [Shared Secret "K"] (Verify)
                                                                 ▲
                                                                 └─ DANGER: Resource Server can FORGE tokens!

Asymmetric (RS256 / ES256 / EdDSA):
Auth Server [Private Key (Secret)] ──► Sign ───► Token ────► Resource Server [Public Key (Open)] (Verify)
                                                                 ▲
                                                                 └─ SAFE: Resource Server CANNOT forge tokens!

Why Asymmetric Cryptography Wins in Distributed Architectures

With HS256, every downstream microservice that validates the token must possess the shared secret key. If a minor reporting microservice is breached, the attacker extracts the shared key and can now forge root administrator tokens valid across your entire infrastructure.

With RS256 / ES256 / EdDSA, only the central Authentication Server possesses the private key. Downstream microservices only possess the Public Key (typically published via a public /.well-known/jwks.json endpoint). Downstream services can verify signatures with mathematical certainty, but are structurally incapable of forging new tokens.

3. The 5 Major Attack Vectors (and How Attackers Exploit Them)

Attack Vector 1: The `alg: none` Vulnerability

RFC 7519 allows an algorithm identifier of "none" for unsecured tokens (e.g. when tokens travel through an already-encrypted channel like an internal IPC pipe).

Naive or poorly configured verification libraries check the algorithm parameter supplied inside the token's own untrusted header:

// Attacker crafts forged admin token:
{
  "alg": "none",
  "typ": "JWT"
}
.
{
  "sub": "admin",
  "role": "superuser"
}
.
[Empty signature string or omitted]

If the library evaluates if (header.alg === 'none') return valid;, the attacker bypasses authentication entirely with zero keys.

The Fix: Explicitly whitelist the permitted algorithms on your verification middleware. Never allow untrusted token headers to dictate verification behavior:

// SECURE: Enforce algorithm pinning
jwt.verify(token, publicKey, { algorithms: ['RS256'] }); // Rejects 'none' immediately

Attack Vector 2: The Algorithm Confusion Attack (RS256 to HS256)

This subtle attack occurs in systems using asymmetric keys (RS256) where the public key is publicly accessible (e.g. via https://auth.example.com/.well-known/jwks.json).

                              ALGORITHM CONFUSION ATTACK
1. Attacker downloads public key: "-----BEGIN PUBLIC KEY----- MIIBIjANBgkq..."
2. Attacker modifies JWT header: {"alg": "HS256"}
3. Attacker signs the token using the PUBLIC KEY STRING as an HMAC-SHA256 secret!
4. Flawed backend checks:
   - Backend reads header: "Oh, it says HS256."
   - Backend calls: jwt.verify(token, key)
   - Flawed library passes the same public key variable into HMAC verification!
   - HMAC verifies using the public key as the secret: MATCH! Attacker is Admin!

The Fix: Always verify that the key format matches the expected algorithm:

// SECURE: Strictly type-check key algorithm matching
if (tokenHeader.alg !== 'RS256') {
  throw new UnauthorizedException('Unsupported cryptographic algorithm');
}

Attack Vector 3: The `kid` (Key ID) Parameter Injection

In systems supporting key rotation, the kid header tells the server which file or database row holds the verification key:

{
  "alg": "HS256",
  "kid": "/dev/null"
}

Directory Path Traversal

If the backend fetches keys using naive string concatenation:

// VULNERABLE CODE:
const key = fs.readFileSync(`/var/keys/${header.kid}`);
jwt.verify(token, key);

An attacker sets "kid": "../../../../../dev/null". On Linux systems, reading /dev/null returns an empty string (0 bytes). The attacker signs the forged token with an empty string as the secret key (""), and the server validates it successfully!

SQL Injection via `kid`

If the backend queries a database:

-- VULNERABLE:
SELECT secret_key FROM api_keys WHERE key_id = '$header.kid';

An attacker injects:

{
  "kid": "key1' UNION SELECT 'my_known_attacker_secret'--"
}

The database returns the attacker's injected secret, allowing their forged signature to validate cleanly.

The Fix: Treat kid as untrusted input. Validate it against a strict whitelist or regex (^[a-zA-Z0-9_-]{1,64}$), and look up keys from a strictly typed in-memory Map rather than the filesystem or raw SQL queries.

Attack Vector 4: Weak HMAC Secrets & Offline Brute-Forcing

Because JWT signatures are computed using standard cryptographic hashes, an attacker who intercepts a single legitimate JWT signed with HS256 can take it offline and run GPU brute-force dictionaries using tools like Hashcat:

hashcat -m 16500 jwt_token.txt /usr/share/wordlists/rockyou.txt

If your HMAC secret is a human-readable phrase like supersecretkey or company_jwt_secret_2024, modern consumer graphics cards (like an RTX 4090) test billions of candidate passwords per second, recovering the secret in less than 30 seconds.

The Fix: Generate cryptographically secure random secrets with minimum 256 bits of entropy:

# Generate a production-ready 256-bit symmetric key:
openssl rand -hex 32

Attack Vector 5: Storage Vulnerabilities (LocalStorage vs. HttpOnly Cookies)

┌─────────────────────────────────────────────────────────────────────────────┐
│                       WHERE SHOULD TOKENS BE STORED?                        │
├─────────────────────┬───────────────────────────────┬───────────────────────┤
│ Storage Target      │ XSS Risk                      │ CSRF Risk             │
├─────────────────────┼───────────────────────────────┼───────────────────────┤
│ localStorage        │ 🚨 CATASTROPHIC: Stolen with   │ ✅ Completely immune   │
│                     │ window.localStorage.getItem() │ to CSRF               │
├─────────────────────┼───────────────────────────────┼───────────────────────┤
│ Regular Cookie      │ 🚨 CATASTROPHIC: Stolen with   │ ⚠️ Vulnerable unless  │
│                     │ document.cookie               │ SameSite=Strict       │
├─────────────────────┼───────────────────────────────┼───────────────────────┤
│ HttpOnly, Secure,   │ ✅ SECURE: JavaScript CANNOT  │ ✅ Immune when paired │
│ SameSite=Strict     │ access cookie bytes           │ with SameSite + CSRF  │
│ Cookie              │                               │ token                 │
└─────────────────────┴───────────────────────────────┴───────────────────────┘

OWASP 2026 Recommendation: Never store access tokens or session credentials in localStorage or sessionStorage. Any third-party analytics script, CDN bundle compromise, or client-side npm dependency vulnerability can extract tokens instantly. Always store authentication tokens in HttpOnly; Secure; SameSite=Strict cookies.

4. Production Architecture: Refresh Token Rotation (RTR)

Stateless access tokens have an inherent flaw: they cannot be easily revoked before they expire. If an access token with a 24-hour lifetime is compromised, the attacker has unrestricted API access for 24 hours.

The industry-standard architecture pairs short-lived Access Tokens with Refresh Token Rotation (RTR):

┌─────────┐                                           ┌──────────────┐
│ Browser │                                           │ Auth Gateway │
└────┬────┘                                           └──────┬───────┘
     │                                                       │
     │ 1. POST /api/login (credentials)                      │
     │──────────────────────────────────────────────────────►│
     │                                                       │
     │ 2. Return Access Token (5 min) + Refresh Token RT_1   │
     │    [RT_1 stored in Secure HttpOnly Cookie + Database] │
     │◄──────────────────────────────────────────────────────│
     │                                                       │
     │ 3. API Requests with Access Token (Fast, Stateless)   │
     │──────────────────────────────────────────────────────►│
     │                                                       │
     │ 4. Access Token Expires (HTTP 401)                    │
     │◄──────────────────────────────────────────────────────│
     │                                                       │
     │ 5. POST /api/auth/refresh (Sends RT_1)                │
     │──────────────────────────────────────────────────────►│
     │                                                       │
     │    [Server Invalidates RT_1, Generates RT_2]          │
     │                                                       │
     │ 6. Returns New Access Token + New RT_2                │
     │◄──────────────────────────────────────────────────────│

The Magic of Automatic Reuse Detection

What happens if an attacker intercepts RT_1 and attempts to use it after the legitimate user has already exchanged it for RT_2?

The server receives a refresh request presenting RT_1.
The database checks RT_1 status: "Already Consumed".
AUTOMATIC COMPROMISE TRIGGERED: The system immediately recognizes that token theft has occurred!
The server immediately revokes the entire family of tokens for that user account, invalidates RT_2, and forces all active sessions to re-authenticate.

5. Zero-Dependency TypeScript JWT Decoder & Expire Auditor

Here is a clean, zero-dependency TypeScript implementation demonstrating how to safely parse, decode, and audit a JWT without running untrusted code:

export interface DecodedJWT<T = Record<string, unknown>> {
  header: {
    alg: string;
    typ?: string;
    kid?: string;
    [key: string]: unknown;
  };
  payload: T & {
    iss?: string;
    sub?: string;
    aud?: string | string[];
    exp?: number;
    nbf?: number;
    iat?: number;
  };
  signature: string;
  isExpired: boolean;
  expiresInSeconds: number | null;
}

export function parseBase64Url(str: string): string {
  // Convert Base64URL to standard Base64
  let base64 = str.replace(/-/g, '+').replace(/_/g, '/');
  // Pad with '=' to make length multiple of 4
  while (base64.length % 4 !== 0) {
    base64 += '=';
  }
  // Universal decoding (Node.js Buffer or browser atob)
  if (typeof Buffer !== 'undefined') {
    return Buffer.from(base64, 'base64').toString('utf8');
  }
  return decodeURIComponent(
    atob(base64)
      .split('')
      .map((c) => '%' + ('00' + c.charCodeAt(0).toString(16)).slice(-2))
      .join('')
  );
}

export function decodeJWT<T = Record<string, unknown>>(token: string): DecodedJWT<T> {
  const segments = token.trim().split('.');
  if (segments.length !== 3) {
    throw new Error(`Invalid JWT format. Expected 3 segments, found ${segments.length}`);
  }

  const [rawHeader, rawPayload, signature] = segments;

  let header: DecodedJWT['header'];
  let payload: DecodedJWT<T>['payload'];

  try {
    header = JSON.parse(parseBase64Url(rawHeader));
  } catch (err) {
    throw new Error('Failed to parse JWT header JSON');
  }

  try {
    payload = JSON.parse(parseBase64Url(rawPayload));
  } catch (err) {
    throw new Error('Failed to parse JWT payload JSON');
  }

  const nowEpoch = Math.floor(Date.now() / 1000);
  let isExpired = false;
  let expiresInSeconds: number | null = null;

  if (typeof payload.exp === 'number') {
    isExpired = nowEpoch >= payload.exp;
    expiresInSeconds = payload.exp - nowEpoch;
  }

  return {
    header,
    payload,
    signature,
    isExpired,
    expiresInSeconds,
  };
}

6. Production Security Checklist for JWTs

Before deploying your authentication service:

Algorithm Pinning: Explicitly specify allowed algorithms (['RS256'] or ['ES256']). Never allow the token header to choose the verification algorithm.
Short Access Lifetimes: Set access token exp to between 5 and 15 minutes.
Mandatory Claims Validation: Always validate iss (Issuer), aud (Audience), exp (Expiration), and nbf (Not Before).
Never Put Secrets in Claims: Passwords, API tokens, credit card metadata, or sensitive personal data must never be stored in token payloads.
Secure Storage: Store tokens in HttpOnly; Secure; SameSite=Strict cookies. Keep localStorage strictly for non-sensitive UI settings.
Implement Key Rotation: Use standard JWKS endpoints (/.well-known/jwks.json) with automated key retirement schedules.

Try It in Your Browser

Need to inspect, decode, and verify a JSON Web Token right now? Our in-browser JWT Decoder parses headers, claims, and signatures locally on your client machine with zero server communication. You can also hash secrets with our HMAC Generator, generate public/private keypairs with our RSA Key Generator, or verify password digests with our Bcrypt Generator.

👉 Open Free In-Browser JWT Decoder →

FAQ

Why should I choose ES256 (ECDSA) over RS256 (RSA)?

ES256 uses 256-bit elliptic curve keys that provide equivalent cryptographic strength to a 3072-bit RSA key. The resulting token signature is significantly smaller (64 bytes vs. 384 bytes), saving bandwidth across high-frequency HTTP headers while reducing CPU overhead on edge gateways.

Can a user revoke their own JWT without database lookups?

A purely stateless JWT cannot be revoked prematurely without a lookup mechanism. Production architectures solve this by keeping access token expiration very short (5 minutes), while checking refresh tokens against a lightweight distributed Redis blacklist only during token renewal.

What is the difference between JWS, JWE, and JWT?

JWT (JSON Web Token): The overarching specification (RFC 7519) representing structured claims.
JWS (JSON Web Signature, RFC 7515): A signed, digitally verifiable JWT whose payload is visible in plaintext. (This is what 99% of developers mean when they say "JWT").
JWE (JSON Web Encryption, RFC 7516): An encrypted JWT whose payload is completely opaque and unreadable to intermediate parties without the decryption key.

Originally published on DailyToolbox.org

Unix Timestamp and Epoch Time Explained: 10-Digit vs 13-Digit Precision, Leap Second Smearing, Timezone Traps, and the Year

tinali7564-eng — Mon, 19 Oct 2026 02:44:20 +0000

Unix Timestamp and Epoch Time Explained: 10-Digit vs 13-Digit Precision, Leap Second Smearing, Timezone Traps, and the Year 2038 Problem (2026)

In distributed systems, few concepts appear simpler on the surface — or trigger more subtle production bugs — than the Unix timestamp.

At its core, Unix time represents the elapsed duration since January 1, 1970, at 00:00:00 UTC (the Unix Epoch). It is an integer that increases monotonically, completely indifferent to geopolitical timezones, daylight saving transitions, or Gregorian calendar anomalies.

Yet in modern web and backend engineering, the Unix timestamp is a constant source of friction:

A frontend engineer passes Date.now() (a 13-digit integer like 1791998400000) to a Python FastAPI backend expecting a 10-digit timestamp (1791998400), causing the database to store a timestamp in the year 58,742.
A high-frequency trading platform or distributed Cassandra database experiences out-of-order event anomalies during a Leap Second insertion, because POSIX time standard mathematically "repeats" second 86,400.
Embedded devices and legacy 32-bit systems are hurtling toward January 19, 2038 at 03:14:07 UTC, when standard 32-bit signed integers will overflow into negative numbers, wrapping systems back to December 13, 1901.

This architectural guide unpacks the physics and computer science behind Unix timestamps: the digit precision hierarchy (seconds vs. milliseconds vs. microseconds vs. nanoseconds), the astronomical mechanics of leap seconds and NTP smearing, database storage best practices in PostgreSQL and MySQL, and the engineering roadmap for the Year 2038 Problem.

1. What Is Unix Time? The 1970-01-01T00:00:00Z Origin Story

In the late 1960s, early operating systems (like Multics) tracked time by counting clock ticks (at 60 Hz) since system boot. When Ken Thompson and Dennis Ritchie architected Unix on the PDP-7 and PDP-11 at Bell Labs, they needed a universal, hardware-independent temporal coordinate system.

They defined Unix time as the number of elapsed seconds since the arbitrary datum point:
$$\text{Unix Epoch} = \text{1970-01-01T00:00:00Z}$$

Why 1970? At the time the PDP-11 kernel was written (early 1971), 1970 was a convenient, round historical baseline. The counter was originally implemented as a 32-bit integer incrementing at 60 Hz. Because a 60 Hz 32-bit counter would overflow after approximately 2.27 years, Ritchie and Thompson quickly modified the kernel to increment once every full second, extending the lifespan of the counter to $2^{31} - 1$ seconds (approximately 68.1 years).

2. The Digit Count Hierarchy: 10 vs. 13 vs. 16 vs. 19 Digits

The single most common bug when integrating multi-language microservices is precision mismatch. Different programming runtimes standardize on different subdivisions of time:

┌─────────────────────────────────────────────────────────────────────────────┐
│                       UNIX TIMESTAMP PRECISION HIERARCHY                    │
├─────────────┬────────────┬─────────────────────────────┬────────────────────┤
│ Precision   │ Digits     │ Example Value (Oct 2026)    │ Standard Runtimes  │
├─────────────┼────────────┼─────────────────────────────┼────────────────────┤
│ Seconds     │ 10 digits  │ 1791998400                  │ Python time.time() │
│             │            │                             │ Linux date +%s     │
│             │            │                             │ PostgreSQL EXTRACT │
│             │            │                             │ C time(NULL)       │
├─────────────┼────────────┼─────────────────────────────┼────────────────────┤
│ Milliseconds│ 13 digits  │ 1791998400000               │ JS Date.now()      │
│             │            │                             │ Java System.millis │
├─────────────┼────────────┼─────────────────────────────┼────────────────────┤
│ Microseconds│ 16 digits  │ 1791998400000000            │ Python time_ns//1k │
│             │            │                             │ C++ std::chrono    │
├─────────────┼────────────┼─────────────────────────────┼────────────────────┤
│ Nanoseconds │ 19 digits  │ 1791998400000000000         │ Go time.Now().Unix │
│             │            │                             │ Rust Instant / UTC │
└─────────────┴────────────┴─────────────────────────────┴────────────────────┘

The Silent "Year 58,742" Multiplication Bug

When JavaScript sends a millisecond timestamp (1791998400000) to a backend that interprets it as seconds:

$$\frac{1791998400000\text{ s}}{31,536,000\text{ s/year}} \approx 56,823\text{ years} + 1970 \approx \mathbf{\text{Year 58,793 AD}}$$

Conversely, when a backend sends a 10-digit second timestamp (1791998400) to a browser and the frontend runs new Date(1791998400):

JavaScript treats 1791998400 as milliseconds (only ~20.7 days past the epoch).
The user interface renders January 21, 1970!

Production Normalization Function (TypeScript)

Always normalize timestamps at API boundaries to prevent precision corruption:

export function normalizeToEpochSeconds(timestamp: number): number {
  // If timestamp has more than 11 digits, it is milliseconds, microseconds, or nanoseconds
  if (timestamp > 1e16) {
    return Math.floor(timestamp / 1e9); // Nanoseconds -> Seconds
  }
  if (timestamp > 1e13) {
    return Math.floor(timestamp / 1e6); // Microseconds -> Seconds
  }
  if (timestamp > 1e11) {
    return Math.floor(timestamp / 1e3); // Milliseconds -> Seconds
  }
  return Math.floor(timestamp); // Already seconds
}

export function normalizeToEpochMillis(timestamp: number): number {
  if (timestamp < 1e11) {
    return Math.floor(timestamp * 1000); // Seconds -> Milliseconds
  }
  if (timestamp > 1e16) {
    return Math.floor(timestamp / 1e6); // Nanoseconds -> Milliseconds
  }
  if (timestamp > 1e13) {
    return Math.floor(timestamp / 1e3); // Microseconds -> Milliseconds
  }
  return Math.floor(timestamp); // Already milliseconds
}

3. Leap Seconds: Why POSIX Time Breaks Astronomy (and How Big Tech Fixes It)

A standard calendar day is defined as 86,400 SI seconds ($24 \times 60 \times 60$). However, the Earth's rotational speed fluctuates due to tidal friction, glacial rebound, and atmospheric mass shifts.

To keep Coordinated Universal Time (UTC) synchronized with astronomical solar time (UT1), the International Earth Rotation and Reference Systems Service (IERS) historically injected Leap Seconds.

The POSIX Flaw

The IEEE POSIX standard (IEEE Std 1003.1) explicitly defines every day as having exactly 86,400 seconds. POSIX time does not recognize leap seconds:

$$\text{POSIX Days} = \frac{\text{Unix Time}}{86,400}$$

When a leap second is added to UTC (such as 23:59:60), standard POSIX systems repeat the 86,400th second:

UTC Clock:     23:59:58 ──► 23:59:59 ──► 23:59:60 ──► 00:00:00 (Next day)
POSIX Time:    1435708798 ─► 1435708799 ─► 1435708799 ─► 1435708800
                                             ▲
                                 DUPLICATE SECOND!

In 2012, this backward step caused high-profile global outages: the Linux kernel's futex subsystem panicked, causing Java application servers, Redis caches, and airline reservation platforms to spike to 100% CPU lockups.

The Solution: NTP Leap Smearing

To prevent clocks from stepping backward or repeating integers, Google, Cloudflare, Meta, and AWS developed Leap Smearing:

Instead of inserting an abrupt 1-second step at midnight, the NTP time servers spread the extra second continuously over a 24-hour window (usually 12 hours before and 12 hours after the leap event). Clocks run slightly slower (by $\approx 0.00115\%$, or ~11.6 ppm) over that day, ensuring timestamps remain strictly monotonic and smooth without duplicate integers.

(Note: In 2022, the General Conference on Weights and Measures resolved to abandon leap seconds by or before 2035, gradually phasing out this operational hazard).

4. Timezone Storage in Databases: PostgreSQL and MySQL Best Practices

A recurring source of data corruption is misunderstanding how relational databases store date and time values.

The PostgreSQL Distinction: `TIMESTAMPTZ` vs. `TIMESTAMP`

Data Type	What Is Stored Internally	Timezone Aware?	Display Behavior
`TIMESTAMP WITHOUT TIME ZONE`	Literal year, month, day, hour, min, sec (8 bytes)	❌ No	Returns exact literal digits regardless of client timezone
`TIMESTAMPTZ` (WITH TIME ZONE)	Microseconds since 2000-01-01 in UTC (8 bytes)	✅ Yes	Converts input to UTC on write; converts to client's local timezone on read

-- DANGER: TIMESTAMP WITHOUT TIME ZONE ignores client context!
CREATE TABLE orders_broken (
    id SERIAL PRIMARY KEY,
    created_at TIMESTAMP NOT NULL DEFAULT NOW() -- <--- BAD!
);

-- BEST PRACTICE: TIMESTAMPTZ always normalizes to UTC internally:
CREATE TABLE orders_secure (
    id SERIAL PRIMARY KEY,
    created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP -- <--- SAFE!
);

The Universal Database Rule: Always store timestamps in TIMESTAMPTZ (in Postgres) or TIMESTAMP / BIGINT (in MySQL). Never store local wall-clock times in database columns unless you are building a calendar scheduling application that must represent a recurring local appointment (e.g. "Every morning at 09:00 AM Paris Time").

5. The Year 2038 Problem (Y2038 / Epochalypse)

The Year 2038 Problem is the Unix equivalent of the Y2K bug, but rooted in computer hardware architecture rather than software formatting.

The 32-Bit Signed Integer Ceiling

In 32-bit Unix systems, time_t is declared as a signed 32-bit integer (int32_t).

The maximum positive value of a signed 32-bit integer is:

$$2^{31} - 1 = 2,147,483,647$$

Adding 1 to this value triggers integer overflow:

$$2,147,483,647 + 1 = -2,147,483,648$$

The Exact Moment of Overflow

The overflow will occur precisely at:
$$\mathbf{\text{2038-01-19 03:14:07 UTC}}$$

At the very next tick (03:14:08 UTC), affected systems will wrap to:
$$\mathbf{\text{1901-12-13 20:45:52 UTC}}$$

Timestamp:      01111111 11111111 11111111 11111111  (2,147,483,647 = 2038-01-19 03:14:07)
+ 1 second:     ──────────────────────────────────────────
Overflow:       10000000 00000000 00000000 00000000  (-2,147,483,648 = 1901-12-13 20:45:52)

Impact and Remediation Roadmap

64-bit Systems Are Safe: Modern 64-bit operating systems use a 64-bit signed time_t. A 64-bit signed integer will not overflow for 292 billion years (far outlasting the heat death of our Sun).
Embedded Systems and IoT Are at Risk: Millions of microcontrollers, industrial PLC sensors, network routers, automotive ECUs, and medical telemetry devices running 32-bit RTOS kernels will encounter silent crashes, broken TLS certificate validation, and billing lockouts.
Database Columns: If your database schema defines timestamps as INT (32-bit integer) instead of BIGINT (64-bit integer) or TIMESTAMPTZ, any record referencing a date after 2038 (such as a 30-year mortgage maturity date or 20-year infrastructure lease) will fail today!

-- Schema Migration: Check for 32-bit integer timestamps!
ALTER TABLE financial_transactions 
    ALTER COLUMN expiry_timestamp TYPE BIGINT;

6. Copy-Paste Code Patterns Across Languages

JavaScript / TypeScript (Node.js & Browser)

// Current Epoch Time
const seconds = Math.floor(Date.now() / 1000); // 10 digits
const millis = Date.now();                      // 13 digits

// Convert Timestamp to ISO-8601 UTC String
const isoString = new Date(seconds * 1000).toISOString();
// Example: "2026-10-14T22:00:00.000Z"

// Parse ISO-8601 String to Epoch Seconds
const epochSec = Math.floor(new Date("2026-10-14T22:00:00Z").getTime() / 1000);

Python 3

import time
from datetime import datetime, timezone

# Current Epoch Time
epoch_sec = int(time.time())                   # 10 digits
epoch_millis = int(time.time() * 1000)         # 13 digits

# Convert Timestamp to UTC Datetime
dt = datetime.fromtimestamp(epoch_sec, tz=timezone.utc)
iso_str = dt.isoformat()
# Example: '2026-10-14T22:00:00+00:00'

# Parse ISO-8601 to Epoch
parsed_epoch = int(datetime.fromisoformat("2026-10-14T22:00:00+00:00").timestamp())

PostgreSQL

-- Get current Unix timestamp in seconds:
SELECT EXTRACT(EPOCH FROM NOW())::BIGINT;

-- Convert Unix timestamp (seconds) to TIMESTAMPTZ:
SELECT TO_TIMESTAMP(1791998400);

-- Convert TIMESTAMPTZ to Unix timestamp (milliseconds):
SELECT (EXTRACT(EPOCH FROM created_at) * 1000)::BIGINT FROM orders;

7. Summary & Best Practices Checklist

When engineering temporal workflows across distributed microservices:

Always document the unit: In API contracts and protobuf schemas, never name a field simply timestamp. Name it created_at_seconds, timestamp_ms, or use the ISO-8601 string format created_at_utc.
Never store local time without an explicit offset: UTC is the only universal invariant in distributed systems.
Upgrade database columns to 64-bit: Audit all legacy database tables for INT columns holding timestamps and migrate them to BIGINT or native temporal types before 2038.
Use NTP with Leap Smearing: Configure internal cloud VPCs to synchronize with smearing NTP pools to safeguard distributed consensus algorithms against leap second regressions.

Try It in Your Browser

Need to convert an epoch timestamp, verify a 10-digit vs 13-digit integer, or inspect UTC calendar dates right now? Our in-browser Unix Timestamp Converter performs instantaneous bidirectional conversions between Unix seconds, milliseconds, human dates, and ISO-8601 strings locally without network latency. You can also calculate durations with our Time Between Dates or parse schedule expressions with our Cron Expression Parser.

👉 Open Free In-Browser Unix Timestamp Converter →

FAQ

Why did Unix time choose UTC instead of TAI (International Atomic Time)?

TAI counts atomic seconds without leap seconds, making it mathematically continuous. However, civilian human life follows the Earth's rotation relative to the Sun. Choosing UTC ensures that Unix midnight consistently aligns with astronomical solar midnight across historical epochs.

Can a Unix timestamp be negative?

Yes! A negative Unix timestamp represents a point in time before January 1, 1970. For example, -86400 corresponds exactly to December 31, 1969 at 00:00:00 UTC. Standard 64-bit systems handle negative timestamps smoothly back to before the Big Bang.

What is the difference between Unix Epoch and Windows Epoch?

Windows NT file systems (NTFS) and Active Directory use the Windows FileTime Epoch, which counts 100-nanosecond intervals since January 1, 1601. To convert between Windows FileTime and Unix timestamp, subtract 11644473600 seconds ($11,644,473,600,000,000$ intervals of 100ns).

Why do some systems show December 31, 1969 instead of January 1, 1970?

If a database or API returns a timestamp of 0 (or NULL cast to integer 0) and your client displays it in a Western timezone (e.g., US Eastern Time UTC-5), the epoch 1970-01-01T00:00:00Z converts to December 31, 1969 at 19:00:00. Seeing December 31, 1969 is the classic hallmark of a zeroed or uninitialized Unix timestamp!

Originally published on DailyToolbox.org

How Cron Expressions Really Work: The 5-Field vs 6-Field Syntax, Day-of-Week Traps (Sunday 0 vs 7), Daylight Saving Time Skips

tinali7564-eng — Sun, 18 Oct 2026 23:43:11 +0000

How Cron Expressions Really Work: The 5-Field vs 6-Field Syntax, Day-of-Week Traps (Sunday 0 vs 7), Daylight Saving Time Skips, and Production Failure Modes (2026)

Every backend engineer, DevOps specialist, and site reliability engineer has encountered the string 0 0 * * * pasted into a Kubernetes CronJob manifest, a GitHub Actions workflow, or an AWS EventBridge rule.

Invented by Ken Thompson in Unix Version 7 (1979) and later standardized by Paul Vixie in 1987, cron is one of the longest-surviving scheduling abstractions in computing. Yet behind its compact, space-delimited facade lies a minefield of subtle dialect incompatibilities, mathematical gotchas, and production failure modes.

Consider this real-world production incident:

A batch billing job scheduled with 30 2 * * * (2:30 AM every morning) silently failed to execute on March 8th when the clocks sprang forward for Daylight Saving Time.
Conversely, on November 1st when the clocks fell back, a payroll distribution job executed twice within 60 minutes, generating duplicate payout transactions.
In another cluster, a migration from a standard Linux crontab to a Java/Spring Quartz scheduler caused a scheduled report to run on Saturdays instead of Sundays because the two engines disagreed on whether Sunday is index 0 or 1.

This architectural guide demystifies how cron evaluation engines parse time specifications, compares the POSIX standard against enterprise Quartz engines, dissects special characters (*, /, ?, L, W, #), explains how to eliminate Daylight Saving Time outages, and provides production-ready concurrency patterns.

1. The Architectural Divergence: POSIX 5-Field vs. Quartz 6/7-Field Schedulers

The most frequent source of developer confusion is assuming that all cron expressions follow the same structural schema. In modern cloud architectures, you will encounter two fundamentally different formats:

Standard POSIX / Vixie Cron (5 Fields)

Used by: Linux crontab, Kubernetes CronJob, GitHub Actions, Node.js node-cron, Python celery.schedules, and Google Cloud Scheduler.

 ┌───────────── Minute (0 - 59)
 │ ┌───────────── Hour (0 - 23)
 │ │ ┌───────────── Day of the Month (1 - 31)
 │ │ │ ┌───────────── Month of the Year (1 - 12 or JAN - DEC)
 │ │ │ │ ┌───────────── Day of the Week (0 - 7 or SUN - SAT, where 0 and 7 are Sunday)
 │ │ │ │ │
 * * * * *

Enterprise / Quartz Cron (6 or 7 Fields)

Used by: Spring Framework (@Scheduled), AWS EventBridge (formerly CloudWatch Events), Quartz Scheduler, Apache Camel, and Jenkins.

 ┌───────────── Seconds (0 - 59)
 │ ┌───────────── Minute (0 - 59)
 │ │ ┌───────────── Hour (0 - 23)
 │ │ │ ┌───────────── Day of the Month (1 - 31)
 │ │ │ │ ┌───────────── Month (1 - 12 or JAN - DEC)
 │ │ │ │ │ ┌───────────── Day of the Week (1 - 7 or SUN - SAT, where 1 is Sunday!)
 │ │ │ │ │ │ ┌───────────── Year (Optional: 1970 - 2099)
 │ │ │ │ │ │ │
 * * * * * * *

Comparative Architecture Matrix

Property	POSIX / Linux Cron	Quartz / Spring Cron	AWS EventBridge
Field Count	5 fields	6 or 7 fields	6 fields
Sub-minute Resolution	❌ No (Minute-level minimum)	✅ Yes (Seconds field included)	❌ No (Seconds must be `0` or omitted)
Sunday Index	`0` and `7` both represent Sunday	`1` is Sunday, `7` is Saturday	`1` is Sunday, `7` is Saturday
Day Conflict Handling	Logical OR between Day-of-Month and Day-of-Week	Requires explicit `?` in one of the day fields	Requires explicit `?` in one of the day fields
Special Modifiers (`L`, `W`, `#`)	❌ Not supported in pure POSIX	✅ Supported (`5L`, `15W`, `2#2`)	✅ Supported (`L`, `W`, `#`)
Year Specification	❌ Not supported	✅ Optional 7th field	✅ Mandatory 6th field (or `*`)

Critical Danger: Pasting a 5-field expression like 0 12 * * ? into a Linux crontab will trigger a syntax error because ? is not a valid POSIX token. Conversely, pasting 0 12 * * * into AWS EventBridge will fail validation because AWS requires a ? in either Day-of-Month or Day-of-Week!

2. Decoding Special Characters: `*`, `/`, `-`, `,`, `?`, `L`, `W`, `#`

Cron engines rely on set-theoretical evaluations. Understanding how these symbols map to discrete integer intervals prevents unexpected triggers.

1. The Asterisk (`*`)

Represents the entire range of valid values for that field. In the minute field, * means "every minute from 0 through 59".

2. The Comma (`,`)

Defines a discrete list of explicit values.

15,30,45 * * * * triggers at minute 15, minute 30, and minute 45 of every hour.

3. The Hyphen (`-`)

Defines an inclusive continuous range.

0 9-17 * * * triggers at minute 0 of every hour between 9:00 AM and 5:00 PM inclusive (9 triggers per day).

4. The Slash (`/`) — Step Intervals

Defines incremental steps across a range. The syntax is <start>/<step>.

*/15 * * * * means starting from 0, step by 15: evaluates to 0, 15, 30, 45.
10/20 * * * * means starting from minute 10, step by 20: evaluates to 10, 30, 50.
The Zero-Minute Fallacy: Many engineers write * / 5 * * * * expecting it to run every 5 minutes. While this works, writing */5 * * * * is equivalent to 0,5,10...55. If you write 5/5, the first trigger of the hour occurs at minute 5, not minute 0.

5. The Question Mark (`?`) — The Quartz/AWS Disambiguator

In POSIX cron, if both Day-of-Month and Day-of-Week are specified, the engine evaluates them as a logical OR (the job runs if either condition matches).

In Quartz and AWS EventBridge, having wildcards in both fields is strictly forbidden because it introduces ambiguity. You must specify ? (meaning "no specific value / disregard this field") in one of the two day fields:

Run at 10:15 AM on the 1st of every month: 0 15 10 1 * ?
Run at 10:15 AM every Friday: 0 15 10 ? * FRI

6. The `L` Character — "Last"

When used alone in Day-of-Month (0 0 L * ?), it dynamically resolves to the last calendar day of the current month (28, 29, 30, or 31).
When used in Day-of-Week (0 0 ? * 5L in Quartz), it means the last Friday of the month.

7. The `W` Character — Nearest Weekday (Monday–Friday)

When specified with a day of the month (e.g. 15W), the engine triggers on the weekday closest to that date:

If the 15th is a Saturday, it triggers on Friday the 14th.
If the 15th is a Sunday, it triggers on Monday the 16th.
It will never jump over a month boundary: if 1W falls on a Saturday, it triggers on Monday the 3rd, not the previous month.

8. The `#` Character — Nth Weekday of the Month

The syntax is <day-of-week>#<occurrence>.

0 0 ? * 2#2 (in Quartz) translates to: The second Monday of the month (Day 2 = Monday, #2 = 2nd occurrence).
Indispensable for billing cycles, recurring monthly meetings, or maintenance windows that must follow calendar business rhythms (e.g., US Thanksgiving: fourth Thursday in November).

3. The Day-of-the-Week Incompatibility Matrix

The most dangerous cross-platform inconsistency is how engines index the days of the week:

Platform                 SUN    MON    TUE    WED    THU    FRI    SAT    SUN (Alt)
──────────────────────────────────────────────────────────────────────────────────
POSIX / Linux crontab     0      1      2      3      4      5      6        7
Kubernetes CronJob        0      1      2      3      4      5      6        7
Node-cron (npm)           0      1      2      3      4      5      6        7
Quartz Scheduler          1      2      3      4      5      6      7       N/A
Spring @Scheduled         1      2      3      4      5      6      7       N/A
AWS EventBridge           1      2      3      4      5      6      7       N/A
ISO-8601 Standard        N/A     1      2      3      4      5      6        7

Why This Triggers Outages

If you configure a backup job to run on Sunday using 0 0 * * 1 in Linux crontab, it will run on Monday (since Monday is 1).

If you then take that exact same string 0 0 ? * 1 and paste it into AWS EventBridge or a Spring Boot service, it will run on Sunday!

Best Practice: To make your cron expressions completely unambiguous and portable across cloud platforms, always use three-letter string abbreviations (SUN, MON, TUE, WED, THU, FRI, SAT) instead of numeric indices whenever supported!

4. The Daylight Saving Time (DST) Nightmare

Daylight Saving Time transitions cause catastrophic silent failures for time-scheduled batch jobs:

SPRING FORWARD (March): Clocks skip from 02:00:00 directly to 03:00:00
┌──────────────┐         [ 1-HOUR VOID ]         ┌──────────────┐
│ 01:59:59 AM  │ ──────────────────────────────► │ 03:00:00 AM  │
└──────────────┘                                 └──────────────┘
Any job scheduled between 02:00 and 02:59 NEVER RUNS!

FALL BACK (November): Clocks repeat 01:00:00 to 01:59:59 twice
┌──────────────┐         [ 1-HOUR REPEAT ]       ┌──────────────┐
│ 01:59:59 AM  │ ──(Clocks roll back to 1am)──►  │ 01:00:00 AM  │
└──────────────┘                                 └──────────────┘
Any job scheduled between 01:00 and 01:59 RUNS TWICE!

How to Immunize Your Architecture Against DST

The Golden Rule: Run All Schedulers in UTC Servers, container hosts, and database timestamps must be configured strictly to Etc/UTC (TZ=UTC). UTC is a monotonic time standard without seasonal jumps or duplicate hours.
If You Must Target Local Business Hours (e.g. 9:00 AM US Eastern):
- Use modern schedulers that natively accept explicit IANA Timezone identifiers (such as CRON_TZ=America/New_York or Kubernetes spec.timeZone: "America/New_York").
- Schedulers supporting IANA timezones (like Kubernetes 1.27+ and Node-cron) implement transition safety logic: during a Spring Forward gap, they fire the missed job immediately at the transition boundary (03:00:00) rather than dropping it entirely.
Avoid the 02:00–03:00 AM Window: If your system operates on local server time and you cannot migrate to UTC immediately, schedule overnight maintenance at 03:30 AM or 04:00 AM, well outside the DST transition window.

5. The Overlapping Execution Trap (Concurrency Control)

What happens if a job scheduled to run every minute (* * * * *) takes 75 seconds to process due to a slow database query?

By default, standard cron daemons and naive timers spawn a brand new process at the 60-second mark regardless of whether the previous instance has finished.

Time:   00:00       00:60       01:15       01:20       02:00
Job 1:  [=========================== DONE ]
Job 2:              [=======================================> (Slows down further)
Job 3:                                                  [===> (Pile up!)
                    ▲
         CONCURRENCY OVERLAP!
         Both jobs acquire database locks & exhaust connection pool.

This creates an exponential death spiral known as the thundering herd / scheduler cascade.

Production Concurrency Solutions

1. Linux Local File Locking (`flock`)

Wrap your crontab command with flock -n (non-blocking exclusive lock):

# If another instance is running, exit immediately with status 0 without blocking:
* * * * * /usr/bin/flock -n /var/lock/my_cron.lock /usr/local/bin/process_queue.sh

2. Kubernetes `concurrencyPolicy`

In Kubernetes CronJob specifications, explicitly set:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: queue-worker
spec:
  schedule: "* * * * *"
  concurrencyPolicy: Forbid  # <--- Drops new executions if previous pod is still running
  startingDeadlineSeconds: 30

3. Distributed Database Advisory Locks (PostgreSQL)

In distributed multi-instance environments, use transactional advisory locks:

-- Attempt to acquire an application-level advisory lock:
SELECT pg_try_advisory_lock(987654321);
-- If returns true, run batch processing. If false, terminate immediately!

6. Production Cron Patterns: 15 Copy-Paste Schedules

Below is a verified reference of common production schedules formatted for standard 5-field engines:

Business Requirement	Cron Expression	Explanation
Every 5 minutes	`/5 * * *`	Minute divisible by 5
Every 15 minutes	`/15 * * *`	At minutes 0, 15, 30, 45
Every hour on the hour	`0 * * * *`	Top of every hour
Every 2 hours	`0 /2 * *`	At 00:00, 02:00, 04:00...
Every weekday at 9:00 AM	`0 9 * * 1-5`	Monday through Friday at 09:00
Every Saturday at midnight	`0 0 * * 6`	Once per week at Saturday 00:00
Every Sunday at 3:30 AM	`30 3 * * 0`	Safe from DST transition
Twice daily (08:00 & 20:00)	`0 8,20 * * *`	Morning and evening sync
First day of every month	`0 0 1 * *`	Monthly billing / invoice run
First weekday of every month	`0 0 1-3 * 1-5`	Evaluates OR condition
Every quarter (Quarterly)	`0 0 1 1,4,7,10 *`	Jan 1, Apr 1, Jul 1, Oct 1
Twice a month (1st and 15th)	`0 0 1,15 * *`	Mid-month and end-month payroll
Working hours only (9-5 M-F)	`/30 9-17 * 1-5`	Every 30 mins during office hours
Midnight on New Year's Day	`0 0 1 1 *`	Once per year on January 1st
Every minute (High frequency)	`* * * * *`	Continuous polling worker

7. Zero-Dependency TypeScript Cron Evaluator & Next-Run Estimator

Here is a clean, dependency-free TypeScript module that parses standard 5-field cron expressions, expands step and range tokens, and computes the next scheduled execution timestamp:

export interface CronFields {
  minutes: number[];
  hours: number[];
  daysOfMonth: number[];
  months: number[];
  daysOfWeek: number[];
}

export function parseCronField(field: string, min: number, max: number): number[] {
  const values = new Set<number>();
  const parts = field.split(",");

  for (const part of parts) {
    if (part.includes("/")) {
      const [subRange, stepStr] = part.split("/");
      const step = parseInt(stepStr, 10);
      let start = min;
      let end = max;

      if (subRange !== "*") {
        if (subRange.includes("-")) {
          [start, end] = subRange.split("-").map(Number);
        } else {
          start = parseInt(subRange, 10);
        }
      }
      for (let i = start; i <= end; i += step) {
        values.add(i);
      }
    } else if (part.includes("-")) {
      const [start, end] = part.split("-").map(Number);
      for (let i = start; i <= end; i++) {
        values.add(i);
      }
    } else if (part === "*") {
      for (let i = min; i <= max; i++) {
        values.add(i);
      }
    } else {
      const num = parseInt(part, 10);
      if (!isNaN(num) && num >= min && num <= max) {
        values.add(num);
      }
    }
  }

  return Array.from(values).sort((a, b) => a - b);
}

export function parseCronExpression(expression: string): CronFields {
  const tokens = expression.trim().split(/\s+/);
  if (tokens.length !== 5) {
    throw new Error(`Invalid cron expression. Expected 5 fields, got ${tokens.length}`);
  }

  return {
    minutes: parseCronField(tokens[0], 0, 59),
    hours: parseCronField(tokens[1], 0, 23),
    daysOfMonth: parseCronField(tokens[2], 1, 31),
    months: parseCronField(tokens[3], 1, 12),
    // Standard POSIX: 0 and 7 are Sunday
    daysOfWeek: parseCronField(tokens[4], 0, 7).map((d) => (d === 7 ? 0 : d)),
  };
}

export function getNextRun(expression: string, fromDate = new Date()): Date {
  const fields = parseCronExpression(expression);
  const next = new Date(fromDate.getTime());
  next.setSeconds(0, 0);
  next.setMinutes(next.getMinutes() + 1); // Earliest possible next run is the next minute

  // Scan up to 5 years into the future to find matching minute
  const maxIterations = 60 * 24 * 365 * 5;
  let iterations = 0;

  while (iterations < maxIterations) {
    const month = next.getMonth() + 1; // 1-12
    if (!fields.months.includes(month)) {
      next.setMonth(next.getMonth() + 1, 1);
      next.setHours(0, 0, 0, 0);
      continue;
    }

    const date = next.getDate();
    const day = next.getDay(); // 0 (Sun) - 6 (Sat)
    const dayMatch = fields.daysOfMonth.includes(date) && fields.daysOfWeek.includes(day);

    if (!dayMatch) {
      next.setDate(next.getDate() + 1);
      next.setHours(0, 0, 0, 0);
      continue;
    }

    const hour = next.getHours();
    if (!fields.hours.includes(hour)) {
      next.setHours(next.getHours() + 1, 0, 0, 0);
      continue;
    }

    const minute = next.getMinutes();
    if (!fields.minutes.includes(minute)) {
      next.setMinutes(next.getMinutes() + 1);
      continue;
    }

    // Found match
    return next;
  }

  throw new Error("No upcoming execution found within 5 years");
}

8. Summary & Interactive Verification

Before deploying any scheduled task to production:

Count the fields: Are you writing for a 5-field POSIX engine (Linux/K8s) or a 6-field Quartz/AWS engine?
Clarify Sunday: Never use numeric 0 or 1 if your engine supports three-letter strings (SUN).
Lock your server to UTC: Never let Daylight Saving Time decide whether your billing pipeline runs once, twice, or never.
Implement concurrency guards: Always configure concurrencyPolicy: Forbid, flock -n, or database advisory locks to prevent overlapping queue storms.

Try It in Your Browser

Need to inspect, debug, or generate a human-readable cron expression right now? Our in-browser Cron Expression Parser instantly validates 5-field syntax, describes the exact execution schedule in plain English, and calculates the next 5 upcoming run times directly on your device. You can also convert between timezones with our Timezone Converter or generate timestamps with our Unix Timestamp Converter.

👉 Open Free In-Browser Cron Expression Parser →

FAQ

Why did my cron job stop running on the 31st of the month?

If you wrote 0 0 31 * *, your job will only run in January, March, May, July, August, October, and December. In 30-day months and February, the 31st does not exist, so the job is silently skipped. In Quartz, use 0 0 L * ? to target the last day of every month dynamically.

What is the difference between `* * * * ` and `/1 * * * *`?

They are semantically identical in every standard parser. Both trigger at minutes 0, 1, 2, 3... 59 of every hour.

Can cron run a task every 30 seconds?

Standard POSIX cron cannot; its minimum granularity is 1 minute. Schedulers that support 6 fields (like Quartz or systemd timers) can achieve sub-minute scheduling. On standard Linux, developers either use systemd timers (OnUnitActiveSec=30s) or a wrapper script with a sleep 30 loop.

Why does crontab execute with different environment variables than my shell?

The cron daemon executes jobs in a minimal, non-interactive environment (/bin/sh). Your interactive shell configuration files (~/.bashrc, ~/.zshrc, and custom PATH exports) are not loaded. Always specify absolute binary paths (e.g., /usr/bin/node instead of node) or source your environment file explicitly at the start of your crontab line.

Originally published on DailyToolbox.org

QR Code Error Correction Levels (L, M, Q, H): Capacity, Damage Tolerance, and Logo Embedding Guide

tinali7564-eng — Sun, 18 Oct 2026 02:48:22 +0000

QR Code Error Correction Levels (L, M, Q, H): Capacity, Damage Tolerance, and Logo Embedding Guide

You scan a restaurant QR code covered in greasy fingerprints, crumpled on laminated cardstock, or branded with a high-resolution corporate logo punched right through its center — and your smartphone camera decodes the URL in under 40 milliseconds.

How does a two-dimensional matrix barcode survive having nearly a third of its area destroyed, obscured, or replaced by arbitrary art without losing a single character of payload?

The secret lies in the Reed-Solomon error correction algorithm defined in ISO/IEC 18004. Unlike linear checksums (such as CRC32 or SHA-256) that merely detect data corruption and abort, QR codes generate forward error correction (FEC) parity codewords capable of mathematically reconstructing missing and corrupted data.

However, error correction is not free. Every bit of redundancy consumes module real estate, drives up the QR code version matrix, shrinks individual printed dot sizes, and degrades scanning distance. Choosing the wrong level can lead to unscannable packaging, bloated SVGs, or marketing campaigns with broken logo integrations.

This guide provides a comprehensive breakdown of the mathematics behind QR error correction, the exact damage recovery metrics for levels L, M, Q, and H, and production rules for embedding logos safely without breaking optical decoders.

1. Anatomy of a QR Code: Where Error Correction Lives

Before understanding error correction levels, consider how a QR code matrix is organized. Every QR code (Versions 1 through 40) is composed of two primary functional domains:

Function Patterns (Non-correctable Overhead):
- Finder Patterns: The three distinctive triple-nested square targets at the top-left, top-right, and bottom-left corners that scanners use for position, rotation, and skew calibration.
- Separators & Quiet Zone: Solid white borders isolating finder patterns from user data. The Quiet Zone requires an uninterrupted margin of at least 4 modules on all four sides.
- Timing Patterns: Alternating dark/light rows connecting the finder patterns to calibrate the module coordinate grid.
- Alignment Patterns: Smaller concentric squares embedded across Versions 2 through 40 to correct optical lens distortion on curved surfaces.
- Format & Version Information: Crucial metadata blocks encoding the selected Mask Pattern and Error Correction Level (protected by dedicated BCH error correction codes).
Encoding Region (Data + Error Correction Codewords):
- The remaining modules store raw user data interleaved with Reed-Solomon parity codewords.
- These codewords are divided into distinct interleaving blocks to distribute localized physical damage across multiple independent mathematical polynomials.

+-----------------------------------------------------------+
| [Finder] [Quiet Zone]                     [Finder]        |
|  (Position & Orientation)                  (Position)     |
|         \                                 /              |
|          +--- [Format Info: Level + Mask] +               |
|                                                           |
|       [Timing Pattern: Alternating Dots]                  |
|                                                           |
|    +-----------------------------------------------+      |
|    |  Interleaved Data Block 1 + Parity Codewords  |      |
|    |  Interleaved Data Block 2 + Parity Codewords  |      |
|    |  [Alignment Patterns for Curvature Correction]|      |
|    +-----------------------------------------------+      |
|                                                           |
| [Finder]                                                  |
|  (Position)              [Quiet Zone Margin >= 4 modules] |
+-----------------------------------------------------------+

2. The Mathematics: How Reed-Solomon Solves Damage

QR codes operate over the Galois Field $GF(2^8)$ with the primitive polynomial defined by ISO/IEC 18004:

$$p(x) = x^8 + x^4 + x^3 + x^2 + 1$$

In this field, every byte (8 bits) represents a polynomial coefficient between 0 and 255. When data is encoded, the input stream is treated as an information polynomial $I(x)$. A generator polynomial $G(x)$ is created by taking roots from powers of the field generator $\alpha$:

$$G(x) = \prod_{i=0}^{2t - 1} (x - \alpha^i)$$

Dividing $I(x) \cdot x^{2t}$ by $G(x)$ yields the remainder polynomial $R(x)$. The coefficients of $R(x)$ are the parity codewords appended directly to the data stream.

Erasures vs. Errors: The $2t$ vs. $t$ Rule

In digital signal processing, corrupted codewords fall into two distinct categories:

Erasures (Known Locations): The scanner detects that a physical module is blocked or unreadable (for example, covered by a centered logo or torn away), but its exact coordinate index is known. A code with $2t$ parity codewords can repair up to $2t$ erasures.
Errors (Unknown Locations): The scanner reads dirty or inverted pixels with high contrast noise, but does not know which specific bytes are false. Solving both the error location polynomial and error evaluator polynomial (using Berlekamp-Massey and Chien search algorithms) requires double the redundant data: $2t$ parity codewords can correct up to $t$ unknown errors.

In real-world conditions, a QR code faces a combination of both:

$$2 \cdot E + S \le 2t$$

Where:

$E$ is the number of unknown errors (scratches, poor contrast, specular glare).
$S$ is the number of known erasures (deliberate logo cutout).
$2t$ is the total number of Reed-Solomon parity codewords generated.

If you punch a center logo that consumes 25% of your code\'s codewords, you convert that entire margin into known erasures ($S$), leaving almost zero reserve ($E$) for real-world optical noise such as smudges or camera glare!

3. The 4 Error Correction Levels Compared (L, M, Q, H)

ISO/IEC 18004 defines four standard error correction levels. Each level represents a different mathematical trade-off between usable payload capacity and damage recovery tolerance:

Level	Identifier	Binary Bits in Format Info	Recovery Percentage (Approx.)	Raw Payload Efficiency	Primary Use Cases
Level L	Low	`01`	~7%	Maximum (~93% data)	Clean digital screens, large payloads (vCards, Wi-Fi keys), ultra-dense displays
Level M	Medium	`00`	~15%	High (~85% data)	Standard default for web links, mobile ticketing, clean printed paper
Level Q	Quartile	`11`	~25%	Moderate (~75% data)	Industrial packaging, courier shipping labels, outdoor posters exposed to weather
Level H	High	`10`	~30%	Lower (~70% data)	Branded QR codes with centered logos, harsh industrial factory floors

Practical Payload Trade-off by Level

Consider an alphanumeric string (such as a standard HTTPS URL) encoded at Version 3 ($29 \times 29$ grid) versus Version 10 ($57 \times 57$ grid):

QR Version	Total Codewords	Level L Capacity	Level M Capacity	Level Q Capacity	Level H Capacity
Version 2 ($25 \times 25$)	44 bytes	77 chars	61 chars	47 chars	34 chars
Version 4 ($33 \times 33$)	100 bytes	187 chars	149 chars	111 chars	80 chars
Version 10 ($57 \times 57$)	346 bytes	652 chars	513 chars	364 chars	271 chars

Notice: Moving from Level L to Level H drops your usable character capacity by nearly 55% for the exact same physical grid size!

If your URL is long (e.g. includes extensive UTM marketing parameters and session tracking tokens), using Level H will force the generator into a much higher QR Version, resulting in tiny, densely packed modules that low-end smartphone cameras cannot resolve in dim lighting.

4. The Version vs. Module Density Trap

Every QR code version increments the matrix size by 4 modules along each axis:

$$\text{Module Count} = 17 + 4 \times \text{Version}$$

Version 1: $21 \times 21$ modules
Version 5: $37 \times 37$ modules
Version 10: $57 \times 57$ modules
Version 40: $177 \times 177$ modules

Minimum Print Size Formula

To ensure optical decoders can distinguish dark and light modules reliably, a printed module should never measure smaller than 0.4 mm to 0.5 mm ($0.016$ to $0.02$ inches) for consumer smartphone cameras at hand-held distance (~20 cm to 30 cm):

$$\text{Minimum Print Width} = (17 + 4 \times \text{Version}) \times 0.45\text{ mm}$$

When you blindly select Level H for a 120-character URL, the code might jump from Version 4 ($33 \times 33$) to Version 7 ($45 \times 45$). If printed on a small business card with an allocated size of $20 \times 20\text{ mm}$:

At Version 4 ($33 \times 33$): Module size = $\frac{20}{33} \approx 0.60\text{ mm}$ (Easy scan)
At Version 7 ($45 \times 45$): Module size = $\frac{20}{45} \approx 0.44\text{ mm}$ (Fails or stutters on older lenses or budget sensors)

Architectural Tip: To embed logos without inflating module density, shorten your URL first using a clean redirect or trim unnecessary query parameters before generating the matrix!

5. The Golden Rules for Embedding Logos (Without Breaking Scanners)

Designers frequently want to superimpose a brand logo, an app icon, or a glyph over the center of a QR code. Because error correction treats obscured modules as localized damage, a centered graphic works mathematically — provided you follow these non-negotiable rules:

Rule 1: Always Select Level H (or Level Q as an Absolute Minimum)

Never attempt to superimpose graphics on Level L (7%) or Level M (15%). A modest logo covering just 12% of the center area instantly wipes out 80% to 100% of Level M redundancy, rendering the code fragile to the slightest camera smudge or ambient shadow.

Rule 2: Limit Logo Area to $\le 20\%$ of the Matrix

Although Level H mathematically supports ~30% recovery, you must reserve an optical buffer. If your logo occupies 20% of the area, you leave approximately 10% of recovery margin for physical scratches, print bleed, specular reflections, and lens aberrations.

$$\text{Maximum Logo Dimension} \le 0.44 \times \text{Total QR Width}$$

$$\text{Target Safe Logo Dimension} \le 0.35 \times \text{Total QR Width} \quad (\approx 12\%\text{ to }15\%\text{ area})$$

Rule 3: Keep a Strict Exclusion Zone Around Functional Patterns

Optical decoders scan the image using a deterministic sequential pipeline. If any of the following patterns are infringed by even 1 pixel, the code will fail to decode entirely:

Do NOT touch the 3 Finder Patterns (the large $7 \times 7$ square corners).
Do NOT overlap the Format Information strips directly adjacent to the finder squares.
Do NOT occlude Alignment Patterns on Version 2+ codes.
Center placement is safest because Reed-Solomon interleaves codewords throughout the matrix, ensuring a centralized cutout distributes errors across many independent blocks rather than devastating one single block.

Rule 4: Add an Uninterrupted Solid Margin Around the Logo

A transparent PNG with thin floating graphic elements will blend with the underlying black/white data modules, causing decoders to misread module boundaries.

Always place a solid circular or rounded-rectangular background plate (usually pure #FFFFFF with a 2–3 module safety border) directly behind your logo icon before compositing.

Rule 5: Respect the 4-Module Quiet Zone

Many modern web designers strip the white outer margin to make the QR code \'flush\' with card edges or dark hero banners. This breaks the decoder\'s edge-detection algorithm. The Quiet Zone must remain at least 4 modules wide and match the background color.

6. Real-World Troubleshooting: Why Your Branded QR Code Fails

Symptom	Underlying Cause	Production Remedy
Scans in bright light, fails in dim room	Contrast ratio is below $4:1$, or subtle logo transparency is bleeding into dark modules	Ensure background is `#ffffff` and foreground is `#000000` or a very dark hue ($L^* < 20$ in CIELAB). Add solid cutout beneath logo.
iPhone camera scans instantly, Android fails	The QR code was color-inverted (white modules on black background)	Traditional optical barcode decoders search strictly for dark-on-light finder targets. Never invert your QR colors for production print.
URL is truncated or scrambled	Logo is too large, exceeding the Reed-Solomon $2t$ boundary, triggering an uncorrectable block failure	Shrink logo size to $< 18\%$ total surface area and confirm error correction is explicitly configured to `Level H`.
Scanner opens the wrong page	Custom shortened URL wrapped by an unstable third-party redirect service that expired	Generate direct target URLs or manage your own 301 redirects on your primary domain.
Modules look distorted or blurry in print	Exported as low-resolution 72 DPI PNG rather than a vector SVG or 300+ DPI asset	Always export and print vector SVG or high-resolution PNG using integer module sizing (no fractional pixel anti-aliasing).

7. Zero-Dependency Client-Side SVG Generator

Here is a clean, dependency-free TypeScript implementation demonstrating how to build an SVG QR code string with custom sizing, level indicators, and an optional safe centered punch-out area:

// Types for QR error correction levels
export type QRErrorCorrectionLevel = "L" | "M" | "Q" | "H";

export interface QRCodeSVGOptions {
  modules: boolean[][];        // Matrix from your QR encoder
  size?: number;               // Pixel width/height (default 256)
  margin?: number;             // Quiet zone modules (default 4)
  darkColor?: string;          // Hex or CSS color (default #000000)
  lightColor?: string;         // Hex or CSS color (default #ffffff)
  includeLogoPunchout?: boolean;
  logoRatio?: number;          // Relative width of center punchout (default 0.22)
}

export function renderQRCodeToSVG(options: QRCodeSVGOptions): string {
  const {
    modules,
    size = 256,
    margin = 4,
    darkColor = "#000000",
    lightColor = "#ffffff",
    includeLogoPunchout = false,
    logoRatio = 0.22,
  } = options;

  const moduleCount = modules.length;
  const totalGridSize = moduleCount + margin * 2;
  const cellSize = size / totalGridSize;

  // Calculate punchout bounds if reserving space for a centered logo
  const centerModule = moduleCount / 2;
  const punchoutRadiusModules = (moduleCount * logoRatio) / 2;
  const minPunch = centerModule - punchoutRadiusModules;
  const maxPunch = centerModule + punchoutRadiusModules;

  const rects: string[] = [];

  for (let r = 0; r < moduleCount; r++) {
    for (let c = 0; c < moduleCount; c++) {
      // Check if module is inside dark matrix
      if (modules[r][c]) {
        // If center logo punchout is enabled, skip dark modules in center
        if (
          includeLogoPunchout &&
          r >= minPunch &&
          r <= maxPunch &&
          c >= minPunch &&
          c <= maxPunch
        ) {
          continue;
        }

        const x = (c + margin) * cellSize;
        const y = (r + margin) * cellSize;
        rects.push(
          `<rect x="${x.toFixed(2)}" y="${y.toFixed(2)}" width="${cellSize.toFixed(2)}" height="${cellSize.toFixed(2)}" fill="${darkColor}"/>`
        );
      }
    }
  }

  return [
    `<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 ${size} ${size}" width="${size}" height="${size}" shape-rendering="crispEdges">`,
    `  <rect width="100%" height="100%" fill="${lightColor}"/>`,
    `  ${rects.join("\n  ")}`,
    `</svg>`,
  ].join("\n");
}

8. Summary Checklist Before Going to Print

Before submitting packaging, billboards, or stickers to a commercial press:

Verify your redundancy level: Use Level M for standard URLs; switch to Level H if you plan to place any logo, icon, or artwork in the center.
Measure the logo footprint: Keep the logo area strictly under 20% of total surface area with a solid background barrier.
Protect the Quiet Zone: Leave at least 4 white modules of clear space on all edges.
Test in physical reality: Print a test sheet at actual size on your office laser printer. Test scanning under three conditions:
- Direct sunlight / bright overhead illumination.
- Low ambient room light (under 50 lux).
- Angled scan from a 45-degree tilt to verify alignment recovery.
Use vector assets: Always deliver vector SVG format to printers to ensure module edges remain razor-sharp without pixelation or compression artifacts.

Try It in Your Browser

Need to generate a clean, private QR code right now? Our in-browser QR Code Generator generates scalable SVGs and high-resolution PNGs locally on your device without sending your sensitive links or Wi-Fi passwords to a remote server. You can also prepare and optimize your assets using our Image Converter or inspect your site metadata with our Meta Tags Analyzer.

👉 Open Free In-Browser QR Code Generator →

FAQ

Can I place a logo in the corner of a QR code instead of the center?

No. The top-left, top-right, and bottom-left corners contain the mandatory $7 \times 7$ Finder Patterns. If any of these three patterns is partially covered, the scanner cannot detect the presence, boundaries, or orientation of the QR code, causing total scan failure regardless of error correction level.

Does error correction help if my QR code is blurry or out of focus?

Only slightly. Blur causes pixel bleeding across adjacent black and white modules. While Reed-Solomon corrects localized bit flips, severe optical blur destroys the Finder Patterns and Timing Patterns first, preventing the scanner from ever calculating the module grid.

Why do some online QR generators charge a monthly subscription?

Many commercial generators do not give you a direct QR code; instead, they embed their own proprietary tracking redirect URL. If you cancel your subscription, they disable the redirect and your printed QR codes stop working. By using an in-browser generator that encodes your actual target URL directly, your QR code remains permanently valid forever with zero recurring costs.

What is the maximum character limit for a QR code?

A Version 40 QR code with Error Correction Level L can store up to 7,089 numeric characters, 4,296 alphanumeric characters, or 2,953 binary bytes (8-bit data). However, high-version codes are difficult for mobile phone cameras to scan; keeping payloads under 150 characters is strongly recommended for consumer use cases.

Originally published on DailyToolbox.org

How to Generate a Secure Password You'll Actually Remember

tinali7564-eng — Sat, 17 Oct 2026 21:36:38 +0000

How to Generate a Secure Password You'll Actually Remember

Your company's password policy demands 12 characters, uppercase, lowercase, a number, and a symbol. So you create P@ssw0rd123!\. It passes the policy check. It would also fall to a dictionary attack in seconds — it's just "password" with the most predictable substitutions (@\ for a\, 0\ for o\) and the most predictable suffix (123!\).

Meanwhile you have 87 other accounts. You reuse three password variations across all of them because remembering 87 unique strings is impossible. One breach, and the attacker has the keys to everything.

There's a better way. It starts with understanding what actually makes a password strong — the math, not the policy checkbox.

The Math: Entropy Is What Matters

Password strength isn't about character types. It's about entropy — the number of guesses an attacker needs, measured in bits. The formula:

\entropy = length × log2(character set size) \\

A few concrete numbers:

Password	Charset	Length	Entropy	Crack time (offline, fast hash)
`P@ssw0rd123!\`	~70 chars	12	~73 bits theoretical, ~20 bits practical*	Seconds (dictionary)
`Tr7$kQ9!mZ2@\`	~70 chars	12	~73 bits	Centuries (brute force)
`correct horse battery staple\`	~7776 words	4 words	~52 bits	Centuries (brute force)
`correct horse battery staple extra\`	~7776 words	5 words	~65 bits	Effectively never

*Theoretical entropy assumes random characters. P@ssw0rd123!\ isn't random — it's a dictionary word with predictable substitutions, so its practical entropy is far lower. This is the critical distinction password policies miss.

The takeaway: length beats complexity. A 25-character passphrase of plain words is stronger than a 12-character soup of symbols — and you can actually remember it.

Method 1: The Passphrase (Best for Humans)

This is the approach security researchers have recommended for years, and it's finally going mainstream.

How it works:

Pick 4–5 random words from a word list. Not a sentence that makes sense — random words. "correct horse battery staple" works because the words are unrelated; "my dog likes treats" doesn't, because it's grammatically predictable.
Join them with a separator: spaces, hyphens, or just smash them together.
Optionally capitalize one word or add a number for sites with annoying complexity requirements.

Concrete example:

Take the Diceware word list (7,776 common English words, each mapped to a dice roll). Roll five dice four times, look up each result:

Roll 1: correct\
Roll 2: horse\
Roll 3: battery\
Roll 4: staple\

Your password: correct-horse-battery-staple\

That's 4 × log2(7776) ≈ 52 bits of entropy. An attacker guessing a billion times per second would need about 140,000 years. And you can remember it after typing it three times.

Need 5 words for extra margin? Add one more roll: correct-horse-battery-staple-extra\ → ~65 bits. Now you're beyond what any brute-force attack can touch.

Where to get the word list: the EFF's word lists are the standard. Don't use song lyrics, quotes, or Bible verses — those are in every attacker's dictionary.

Method 2: Random + Password Manager (Best Overall)

If you use a password manager (Bitwarden, 1Password, KeePassXC — all solid), you don't need to remember passwords at all. Generate maximum-entropy random strings and let the manager handle them:

\K7$mQ9!vZ2@pL4#nX8&dF5 \\

20 characters from a 70-character set = ~122 bits. That's not just strong — it's overkill in the best way. You'll never type it by hand, so memorability is irrelevant.

You only need to remember two things:

Your password manager's master password — use a 5-word passphrase (Method 1)
Nothing else. That's the point.

This is objectively the strongest setup. The passphrase protects the vault; the vault holds uncrackable random passwords for everything else. If you're not using a password manager yet, this is the single highest-impact security change you can make.

Method 3: The Hybrid (For Passwords You Type Often)

Some passwords you type daily — your laptop login, your password manager master password. For these, pure random strings are painful. Use a hybrid:

Start with 3 random words: ocean-bridge-lamp\
Add a personal-but-not-obvious anchor: a number that's meaningful to you but not your birthday (ocean-bridge-lamp-47\)
Optionally transform one word: ocean-bridge-lamp47!\

The result is memorable (you can picture an ocean, a bridge, a lamp) but not guessable (no attacker knows your anchor number or which words you picked).

What NOT to use as an anchor: birthdays, anniversaries, pet names, addresses, phone numbers. All of these are either public or easily discoverable. Pick something arbitrary — the number of books on your shelf, a random page number, anything without a paper trail.

What About Those Complexity Requirements?

Some sites still demand "one uppercase, one number, one symbol." This is outdated advice (NIST dropped it from their guidelines in 2017), but you're stuck with it on those sites. The pragmatic approach:

Take your passphrase: correct-horse-battery-staple\
Capitalize the first letter: Correct-horse-battery-staple\
Swap the last separator for a number + symbol: Correct-horse-battery9#\

You've satisfied the policy without meaningfully weakening the password. The entropy barely changes because the core — four random words — is doing all the heavy lifting.

Common Mistakes

"Clever" character substitutions. P@ssw0rd\ isn't clever. Attackers have used a→@\, e→3\, o→0\, s→$\ in their dictionaries for 20 years. Every substitution you can think of is already in the attacker's wordlist. Randomness beats cleverness, always.

Reusing passwords across sites. This is the mistake that causes the most real-world damage. When Site A gets breached (and it will — breaches are a matter of when, not if), attackers try your email + password on every major site automatically. This is called credential stuffing, and it's fully automated. Unique passwords per site aren't optional.

Password rotation policies. "Change your password every 90 days" sounds secure but backfires. Research shows it leads to weaker passwords — people just increment a number (Spring2026!\ → Summer2026!\). NIST now recommends against forced rotation. Change passwords when there's a breach, not on a schedule.

Security questions as backup. "What was your first pet's name?" is not security — it's a second, weaker password that's often publicly known. If a site forces security questions, treat the answers as additional passwords: generate random strings and store them in your password manager. "Mother's maiden name: X7#kP2$mQ9!\" is perfectly fine.

Writing passwords in a notes app or spreadsheet. An unencrypted notes file is barely better than a sticky note. If you must write them down (and for a password manager master password, a paper backup in a safe is actually reasonable), use paper, not a cloud-synced notes app.

Trusting "password strength meters." Most strength meters just count character types and length. They'll rate P@ssw0rd123!\ as "Strong" because it has all four character types. It isn't. Strength meters can't detect dictionary patterns. Ignore them.

Passwords Are Only Half the Story: Enable 2FA

Even the strongest password can be phished, keylogged, or leaked in a breach. Two-factor authentication (2FA) adds a second barrier: something you have (your phone) in addition to something you know (your password).

Not all 2FA is equal:

Method	Security	Notes
Hardware key (YubiKey)	Highest	Phishing-resistant. The gold standard.
Authenticator app (TOTP)	High	Codes change every 30 seconds. Works offline.
SMS codes	Low	Vulnerable to SIM swapping. Better than nothing.
Email codes	Low	If your email is compromised, so is this.

The priority order: enable 2FA on your email first (it's the recovery point for everything else), then banking, then password manager, then everything else. An authenticator app takes 30 seconds to set up per site and blocks the vast majority of account takeover attempts.

SMS 2FA gets criticized, and rightly — SIM swapping attacks are real. But SMS 2FA still blocks automated credential stuffing, which is the most common attack by volume. If SMS is your only option, use it. Don't let perfect be the enemy of good.

The Future: Passkeys

Passkeys (FIDO2/WebAuthn) are the industry's answer to passwords. Instead of a shared secret, your device generates a cryptographic key pair: the public key goes to the site, the private key stays on your device, protected by biometrics or a PIN.

Why they're better:

Phishing-resistant. The private key never leaves your device and is bound to the site's domain. A fake login page can't trick it.
Nothing to remember. No passwords to generate, store, or type.
No server-side secrets. Breaches leak public keys, which are useless to attackers.

The catch: adoption is still ramping up. Not every site supports passkeys yet, and cross-device sync (getting your passkeys onto a new phone) is still maturing. Use passkeys where offered — Google, Apple, GitHub, and others support them now — but keep your password manager for everything else.

Passkeys won't kill passwords overnight. But they're the direction things are moving, and enabling them where available is strictly an upgrade.

How to Actually Migrate: A Practical Plan

Knowing what to do and doing it are different things. Here's a weekend migration plan:

Saturday morning (1 hour): Pick a password manager and set it up

Install Bitwarden (free, open source) or 1Password (paid, polished). Both work on every platform.
Create your account with a 5-word passphrase as the master password. Write it on paper, store it somewhere safe.
Enable 2FA on the password manager itself — authenticator app, not SMS.
Install the browser extension.

Saturday afternoon (2 hours): Migrate your critical accounts
Start with these five, in order:

Email — the master key to all password resets. Unique 20-character random password + 2FA.
Banking / financial — unique random password + 2FA.
Password manager — already done, but verify 2FA works.
Cloud storage (Google Drive, iCloud, Dropbox) — unique random password + 2FA.
Social media — unique random password + 2FA where available.

For each: log in, go to security settings, generate a new random password in your manager, save it, enable 2FA. The manager's browser extension auto-fills from here on.

Sunday (1 hour): Bulk-migrate the rest
Go through your remaining accounts. For each one, generate a unique random password. You don't need to do all 87 in one sitting — do 20 per weekend and you'll be done in a month. Prioritize anything tied to money, identity, or work.

Ongoing: when you create a new account, let the password manager generate the password. Never type a password by hand again (except your master passphrase).

Total time investment: ~4 hours. What you get: every account protected by a unique, uncrackable password, with 2FA on the important ones. That's a bigger security upgrade than most companies achieve with six-figure budgets.

Try It

You don't need to roll dice or memorize word lists to get started. DailyToolbox's free password generator runs entirely in your browser — generate cryptographically random passwords or memorable passphrases, customize length and character sets, and copy with one click. Nothing is sent to a server or stored anywhere, which is exactly how a password tool should work.

👉 Try the Password Generator →

FAQ

How long should my password be?
For random passwords: 16+ characters. For passphrases: 4+ words (5 is better). These give you 80+ and 52+ bits of entropy respectively, both well beyond brute-force range. Longer is always better — there's no downside to a 6-word passphrase except typing time.

Are password managers safe? Can't they get hacked?
Reputable password managers (Bitwarden, 1Password, KeePassXC) encrypt your vault locally with your master password before anything touches their servers. Even if their servers are breached, the attacker gets encrypted blobs they'd need your master password to decrypt. This is vastly safer than reusing passwords. Use a strong master passphrase and enable 2FA on the manager itself.

What if I can't use a password manager (work computer, shared device)?
Use passphrases (Method 1). Four random words are typeable, memorable, and strong enough for any account. For your most critical accounts (email, bank), use 5 words. Your email is the most important — password resets go there, so it's the key to everything else.

Do I really need unique passwords for every site?
Yes. Credential stuffing is automated and relentless — breached credentials are tested against hundreds of sites within hours. A password manager makes unique passwords effortless. Without one, at minimum use unique passphrases for email, banking, and anything tied to your identity or money.

Originally published on DailyToolbox.org

JSON Formatter vs Validator: What's the Difference?

tinali7564-eng — Sat, 17 Oct 2026 10:40:53 +0000

JSON Formatter vs Validator: What's the Difference?

You're staring at an API response. It's one line, 40,000 characters, no whitespace. Somewhere in that wall of text is the field you need, and possibly a bug. You paste it into an online tool. But which one — a JSON formatter or a JSON validator?

They look interchangeable. They're not. Here's what each actually does, when to reach for which, and the overlap that confuses everyone.

What a JSON Formatter Does

A formatter takes valid JSON and makes it human-readable. That's it. Its job is presentation.

Input (what the API gives you):
\json {"users":[{"id":1,"name":"Ada","roles":["admin","editor"],"lastLogin":"2026-10-09T14:22:00Z"},{"id":2,"name":"Grace","roles":["viewer"],"lastLogin":null}],"total":2,"page":1} \\

Output (what the formatter gives you):
\json { "users": [ { "id": 1, "name": "Ada", "roles": [ "admin", "editor" ], "lastLogin": "2026-10-09T14:22:00Z" }, { "id": 2, "name": "Grace", "roles": [ "viewer" ], "lastLogin": null } ], "total": 2, "page": 1 } \\

Same data, same meaning. The formatter added indentation (usually 2 or 4 spaces) and line breaks so you can scan the structure, find nested fields, and spot anomalies. It can also do the reverse — minify — collapsing pretty JSON back to one line for production payloads where every byte counts.

A formatter assumes your JSON is already valid. If it isn't, the formatter fails — which brings us to the interesting part.

What a JSON Validator Does

A validator answers one question: is this valid JSON? It doesn't care about readability. It parses the input against the JSON specification (RFC 8259) and reports pass or fail.

If it fails, a good validator tells you where:

\Error: Unexpected token } in JSON at position 142 Line 6, column 18 \\

That position pointer is the whole point. When you're debugging a 500-line config file, "invalid JSON somewhere" is useless. "Line 6, column 18" tells you exactly where to look.

Validators don't reformat. They don't pretty-print. They check syntax and stop. Some also validate against a JSON Schema — a separate specification that defines what shape the JSON should have (required fields, types, value ranges). That's a different layer: syntax validation asks "is this JSON?", schema validation asks "is this the JSON I expected?"

The Overlap That Confuses Everyone

Here's the thing most tutorials skip: a formatter is implicitly a validator.

A formatter can't pretty-print invalid JSON. To add indentation, it has to parse the structure first. If parsing fails, it errors out — just like a validator would. So in practice:

	Formatter	Validator
Pretty-prints valid JSON	✅	❌
Minifies JSON	✅	❌
Detects syntax errors	✅ (as a side effect)	✅ (its main job)
Shows error location	Sometimes	✅ (always)
Validates against a schema	❌	Some do

If you paste broken JSON into a formatter, you'll get an error message. It's not as detailed as a dedicated validator's, but it tells you something's wrong. This is why most developers just use a formatter for everything — it covers 90% of cases.

When to Use Each

Reach for a formatter when:

Debugging an API response — you need to read the data, find a field, understand nesting
Reviewing a config file before deploying — visual scan catches misplaced values
Preparing a payload to send — write it readable, then minify for the actual request
Learning an unfamiliar API — pretty-printed responses reveal the data model faster than docs

Reach for a validator when:

Something fails to parse and you need the exact error location — the validator's position pointer saves you from eyeballing 500 lines
CI/CD pipelines — a validation step in your build catches broken JSON configs before they ship (formatters are interactive tools; validators run headless)
You're generating JSON programmatically and want a quick syntax gate before writing to disk
Validating against a JSON Schema — checking that the structure matches a contract, not just that the syntax is legal

The practical rule: if you're looking at JSON with your eyes, use a formatter. If a machine needs to check JSON without human involvement, use a validator.

A Real Debugging Walkthrough

Say your frontend throws Unexpected token in JSON at position 0\ when calling /api/users\. You grab the raw response body:

\{"users": [{"id": 1, "name": "Ada",}, {"id": 2, "name": "Grace"}]} \\

Paste it into a formatter. It fails. Now paste it into a validator:

\Error: Unexpected token } at position 38 \\

Position 38 — that's the comma after "Ada",\. Trailing comma. JSON doesn't allow them (unlike JavaScript object literals, which is exactly why this mistake is so common). Remove the comma, re-format, and you get clean output.

This is the typical loop: formatter for reading, validator for pinpointing, formatter again to confirm the fix. Most good online tools combine both, which is why the distinction feels blurry — but knowing which capability you're actually using helps when one of them isn't enough.

Common Mistakes

Trailing commas. The #1 JSON syntax error, full stop. {"a": 1,}\ is invalid. JavaScript allows it, Python allows it, JSON does not. Linters and formatters in your editor should catch this before it reaches production.

Single quotes. {'name': 'Ada'}\ is not valid JSON. Keys and string values must use double quotes. This bites everyone who writes JSON by hand after writing JavaScript all day.

Comments. JSON has no comments. Not //\, not /* */\. If you need to annotate a config, use a _comment\ field (a convention, not a standard) or switch to JSON5/YAML for human-edited files. But the moment it goes over the wire, it must be pure JSON.

Assuming formatted means correct. A formatter will happily pretty-print {"price": "free"}\ — valid syntax, wrong type. Syntax validation doesn't check that price\ should be a number. That's what JSON Schema is for. Don't confuse "parses successfully" with "contains the right data."

NaN and Infinity. {"value": NaN}\ looks reasonable if you come from JavaScript, but it's invalid JSON. JSON only supports null\, true\, false\, numbers, strings, arrays, and objects. No NaN\, no Infinity\, no undefined\. APIs that need these must encode them as strings or null.

BOM and invisible characters. A UTF-8 BOM (byte order mark) at the start of a file makes some strict parsers choke. Copy-pasting from Word documents or rich text editors can inject smart quotes ("\ instead of "\), non-breaking spaces, or zero-width characters. If your JSON "looks fine" but won't parse, check for invisible characters first.

Command-Line Tools: jq and Friends

If you work with JSON daily, learn jq\. It's the Swiss Army knife for JSON on the command line, and it handles both formatting and validation.

Pretty-print a file:
\bash jq . response.json \\

Validate without printing (exit code tells you pass/fail):
\bash jq empty response.json && echo "Valid" || echo "Invalid" \\

Extract a nested field from minified JSON:
\`bash
curl -s https://api.example.com/users | jq '.users[0].name'

Output: "Ada"

`\

Minify for production:
\bash jq -c . pretty.json > minified.json \\

jq\ is available on every platform (brew install jq\, apt install jq\, or download the binary). For quick validation in scripts, Python's built-in tool works without any install:

\bash python3 -m json.tool response.json > /dev/null && echo "Valid" \\

And in Node.js, a one-liner validates from stdin:

\bash echo '{"a": 1}' | node -e "JSON.parse(require('fs').readFileSync(0, 'utf8')); console.log('Valid')" \\

These are the tools to reach for in CI pipelines, pre-commit hooks, and shell scripts — anywhere a human isn't looking at the output.

JSON Schema: When Syntax Isn't Enough

A formatter confirms your JSON parses. But {"price": "free"}\ parses fine — and it's wrong if price\ should be a number. This is where JSON Schema comes in: a standard for describing what valid JSON should look like, not just whether it's syntactically legal.

Here's a schema for a user object:

\json { "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "required": ["id", "name", "email"], "properties": { "id": { "type": "integer", "minimum": 1 }, "name": { "type": "string", "minLength": 1 }, "email": { "type": "string", "format": "email" }, "roles": { "type": "array", "items": { "type": "string" } }, "lastLogin": { "type": ["string", "null"], "format": "date-time" } }, "additionalProperties": false } \\

This says: the object must have id\, name\, and email\. id\ must be a positive integer. email\ must look like an email. No extra fields allowed (additionalProperties: false\).

Now validate data against it (using Python's jsonschema\ library):

\`python
import json, jsonschema

schema = json.load(open('user-schema.json'))
data = json.load(open('user.json'))

try:
jsonschema.validate(data, schema)
print("Valid against schema")
except jsonschema.ValidationError as e:
print(f"Schema violation: {e.message}")
print(f"Failed at: {' -> '.join(map(str, e.absolute_path))}")
`\

When you need JSON Schema:

Designing a public API — publish the schema so consumers know the contract
Validating webhook payloads — reject malformed data at the boundary, not deep in your business logic
Configuration files — catch {"port": "8080"}\ (string instead of integer) before the server fails to start
Testing — assert API responses match the expected shape, not just that they parse

When you don't: quick debugging, one-off data exploration, internal scripts where you control both ends. A formatter is faster and you don't need the ceremony.

A Second Debugging Scenario: The Invisible Character

A teammate pastes a JSON snippet into Slack. You copy it, paste it into your code, and it won't parse. The validator says:

\Error: Unexpected token in JSON at position 0 \\

Position 0. The very first character. But the JSON starts with {\ — you can see it.

What's happening: Slack (or the teammate's editor) inserted a zero-width no-break space (U+FEFF, the BOM character) at the start. It's invisible in every editor, but the JSON parser sees it and chokes.

The fix: retype the first character, or pipe through something that strips it:

\`bash

Strip BOM and validate

tail -c +4 broken.json | jq empty && echo "Valid after BOM strip"
`\

This class of bug — invisible characters from copy-paste — accounts for a surprising number of "but it looks fine!" JSON errors. When the validator points at position 0 and the JSON looks correct, suspect invisible characters before anything else.

Try It

DailyToolbox's free JSON formatter runs entirely in your browser — paste minified JSON, get it pretty-printed instantly, minify it back when you're done. Invalid JSON gets a clear error with position info, so you get formatting and validation in one place. Nothing leaves your machine, which matters when you're debugging API responses with real user data in them.

👉 Try the JSON Formatter →

FAQ

Is JSON the same as a JavaScript object?
No, though they're related. JSON is a text format inspired by JavaScript object syntax. A JavaScript object lives in memory; JSON is the string you send over the wire. Key differences: JSON requires double quotes on keys, no trailing commas, no comments, no functions, no undefined\. JSON.parse()\ and JSON.stringify()\ convert between the two.

Why does my valid-looking JSON fail to parse?
The usual suspects, in order: trailing comma, single quotes instead of double, a comment someone left in, smart quotes from copy-pasting, or an invisible character (BOM, zero-width space). Paste it into a validator — the error position will point you to the culprit.

Should I validate JSON in my CI pipeline?
Yes, if you have JSON config files, fixtures, or mock data in your repo. A one-line check (python -m json.tool file.json > /dev/null\ or jq empty file.json\) catches syntax errors before they become runtime failures. It takes seconds to set up and saves real debugging time.

What's JSON Schema and do I need it?
JSON Schema is a separate standard for describing what valid JSON should look like — required fields, types, formats, value constraints. You need it when you're designing an API contract and want to validate incoming payloads automatically, not just check syntax. For quick debugging, a formatter/validator is enough. For API design, learn JSON Schema.

Can a formatter fix my broken JSON?
No — and be wary of tools that claim to. A formatter can only rearrange valid JSON. If your JSON has a syntax error, no amount of reformatting will fix it; something has to change the actual content (remove the trailing comma, fix the quotes). Some "smart" tools silently correct errors, which is dangerous: you might ship JSON that parses but doesn't mean what you intended. Always fix the source, don't let a tool guess.

Originally published on DailyToolbox.org

How to Reduce Image Size Under 100KB Without Losing Quality

tinali7564-eng — Sat, 17 Oct 2026 02:41:01 +0000

How to Reduce Image Size Under 100KB Without Losing Quality

Your homepage hero image is 2.4 MB. On a decent 4G connection that takes about 8 seconds to load. Half your visitors leave before they see anything. Google's Core Web Vitals flag your Largest Contentful Paint as "Poor," and your search ranking quietly slides.

The fix isn't buying a faster server. It's shrinking the image. Most web photos can land under 100 KB and look identical to the original on screen. Here's the exact process, with real numbers.

Why 100KB Is the Target

100 KB isn't magic, but it's a good line in the sand:

A 100 KB image loads in roughly 0.3 seconds on 4G (vs 8 seconds for 2.4 MB).
Google's LCP threshold for "Good" is 2.5 seconds. Images are usually the LCP element. A sub-100KB hero gives you headroom for everything else on the page.
On mobile data plans, every megabyte costs your user real money in some markets.

You don't need every image under 100 KB — thumbnails and icons should be far smaller, and a full-screen photographic background might legitimately be 150–200 KB. But if your content images are routinely over 100 KB, you're leaving performance on the table.

Step 1: Resize to the Actual Display Size

This is where most of the savings come from, and most people skip it.

That 2.4 MB photo? It's probably 6000×4000 pixels straight from a camera or stock site. Your page displays it at 800 pixels wide. You're shipping 56 times more pixels than the screen shows.

The rule: resize the image to the largest size it will actually display, then add a small margin.

Display context	Target width
Blog content image	800–1200 px
Hero/banner	1600–1920 px
Thumbnail	300–400 px
Full-screen background	1920 px (rarely more)

For retina/HiDPI screens, serve at 2× the CSS display width — a 400px-wide slot gets an 800px image. Going to 3× or beyond has diminishing returns; the human eye can't resolve the difference at normal viewing distances, and you're tripling the file size for nothing.

Concrete example: A 6000×4000 JPEG at 2.4 MB, resized to 1200×800, drops to roughly 400–500 KB before any other optimization. One step, ~80% reduction.

Step 2: Pick the Right Format

Format choice matters more than most people think. Here's the honest breakdown:

Photographs (complex, many colors):

WebP — your default choice in 2026. Typically 25–35% smaller than JPEG at the same visual quality. Supported by every modern browser (Chrome, Firefox, Safari 14+, Edge).
AVIF — 20–30% smaller than WebP, but encodes much slower and Safari support has quirks. Worth it for hero images you optimize once; overkill for bulk processing.
JPEG — the fallback. Use it when you need maximum compatibility (email clients, old systems).

Graphics, logos, screenshots with text:

PNG — only when you need transparency or pixel-perfect sharp edges. A photographic PNG can be 5–10× larger than the equivalent JPEG.
SVG — for logos and icons. It's code, not pixels. Always smaller, infinitely scalable.

The single biggest format mistake: saving a photograph as PNG. I see this constantly. A 2 MB PNG photo becomes a 180 KB JPEG with no visible difference. If your image has no transparency and isn't a logo, it probably shouldn't be PNG.

Step 3: Dial In the Quality Setting

Once resized and in the right format, the quality slider does the rest. Here's what actually happens at different settings:

Quality	JPEG/WebP file size (1200px photo)	Visible difference
100	~450 KB	None (don't do this)
90	~220 KB	None on screen
80	~130 KB	None on screen
70	~95 KB	Barely perceptible on close inspection
60	~70 KB	Slight artifacts in smooth gradients
50	~55 KB	Noticeable blockiness

The sweet spot for web is 75–82. At quality 80, a 1200px photo lands around 100–130 KB and looks identical to the original on any screen. Below 70, you start seeing artifacts in skies and gradients.

Don't trust the slider blindly — zoom to 100% and check smooth gradient areas (skies, skin, shadows). That's where compression artifacts show first. Text and sharp edges are more forgiving.

Step 4: Strip the Metadata

Photos carry EXIF data: camera model, GPS coordinates, timestamps, thumbnails. This adds 10–50 KB and — more importantly — can leak your location.

Most compression tools strip EXIF by default. If yours doesn't, make sure to enable it. The exception: photographers who need copyright metadata embedded should keep it, but that's not your blog hero image.

Stripping metadata on our example image saves another ~15 KB and removes a privacy risk. Free win.

Step 5: Serve Responsive Sizes (Don't Ship Desktop Images to Phones)

Even a well-optimized 1200px image is wasteful on a phone that displays it at 400px. Responsive images solve this with the srcset\ attribute — the browser picks the smallest file that fits the screen.

\html <img src="photo-800.jpg" srcset="photo-400.jpg 400w, photo-800.jpg 800w, photo-1200.jpg 1200w" sizes="(max-width: 600px) 400px, (max-width: 1000px) 800px, 1200px" alt="A mountain lake at sunrise" loading="lazy"> \\

Here's what each piece does:

srcset\ lists available files with their widths. The browser — not you — picks the best one based on screen size and pixel density.
sizes\ tells the browser how wide the image will display at different viewport widths, so it can choose before downloading.
loading="lazy"\ defers offscreen images until the user scrolls near them. Your initial page load only fetches what's visible.

The impact is dramatic. A mobile user on a 375px-wide phone gets the 400px version (~35 KB) instead of the 1200px version (~98 KB). That's a 64% reduction for the majority of web traffic — mobile accounts for roughly 60% of page views globally.

Generate three sizes per image: the full size, half, and quarter. Most build tools and CDNs automate this. If you're doing it manually, it's three exports from your compressor instead of one — an extra 30 seconds per image.

The `<picture>\` Element for Format Switching

srcset\ handles sizes. For formats, use <picture>\ to serve WebP or AVIF with JPEG fallback:

\html <picture> <source srcset="photo.avif" type="image/avif"> <source srcset="photo.webp" type="image/webp"> <img src="photo.jpg" alt="A mountain lake at sunrise" loading="lazy"> </picture> \\

The browser picks the first format it supports. Modern browsers get AVIF or WebP; older ones fall back to JPEG. You get the best compression available without breaking anyone's experience.

This is more markup to write, but the payoff is real: AVIF at quality 70 is roughly half the size of JPEG at quality 80, with comparable visual quality. For hero images that every visitor downloads, that difference matters.

WebP vs AVIF: An Honest Comparison

Both beat JPEG. But which should you actually use in 2026?

	WebP	AVIF
Size vs JPEG	25–35% smaller	40–50% smaller
Browser support	Universal (all modern browsers)	Near-universal, minor Safari quirks
Encode speed	Fast	5–10× slower than WebP
Decode speed	Fast	Slower on old devices
Tooling	Mature everywhere	Good, improving
Animation	Yes	Yes (better than GIF)
Transparency	Yes	Yes

My recommendation: WebP as your default. It's fast to encode, universally supported, and the tooling just works. Use AVIF selectively for your largest, most-viewed images (hero banners, og:image social cards) where the extra 20% savings justifies the slower encode and testing overhead.

Don't overthink this. The jump from JPEG to WebP captures most of the available savings. The jump from WebP to AVIF is incremental — nice to have, not transformative.

What About Image CDNs?

Services like Cloudinary, Imgix, and Cloudflare Images can automate everything above: upload one high-res source, and they serve resized, reformatted, quality-optimized versions on the fly via URL parameters.

\https://cdn.example.com/photo.jpg?w=800&format=webp&quality=80 \\

Change w=800\ to w=400\ and you get the mobile version. No manual exports, no build step.

The tradeoff: cost and complexity. These services charge by bandwidth or transformations. For a personal blog or small site, manual optimization (or a build-time tool like sharp\ or squoosh-cli\) is free and gives you full control. For a large site with thousands of images and a team, a CDN pays for itself in engineering time saved.

There's a middle ground: build-time optimization. Tools like Next.js's <Image>\ component, Gatsby's image plugins, or a simple sharp\ script in your build pipeline automatically generate responsive sizes and modern formats at deploy time. You write <Image src="photo.jpg" width={1200} />\ and the framework handles the rest. Zero runtime cost, zero manual work after setup.

Putting It Together: A Real Walkthrough

Starting point: a 6000×4000 stock photo, JPEG, 2.4 MB.

Resize to 1200×800 → ~480 KB (80% reduction)
Convert to WebP → ~310 KB (35% smaller than the resized JPEG)
Quality 80 → ~115 KB
Strip EXIF → ~98 KB

Final: 2.4 MB → 98 KB. A 96% reduction. Side by side on a monitor, you cannot tell them apart. I mean that literally — put the original and the compressed version next to each other at display size and try to spot the difference.

If you need to go further (say, under 50 KB for a thumbnail), drop to quality 65–70 and resize to 800px. You'll see minor artifacts if you pixel-peep, but at thumbnail size nobody will notice.

Common Mistakes

Compressing without resizing first. Running a 6000px image through a compressor at quality 60 gives you a 400 KB file that still looks bad. Resize first, then compress. Order matters.

Using PNG for photographs. I keep saying this because I keep seeing it. PNG is lossless — great for screenshots with text, terrible for photos. A photo saved as PNG is often 5–10× larger than necessary.

Quality 100 "to be safe." Quality 100 doesn't mean "best for web." It means "minimum compression," which produces files 3–4× larger than quality 80 with zero visible improvement on screen. You're paying for peace of mind with your users' bandwidth.

Upscaling small images. If your source is 800px and you need 1200px, don't upscale — you'll get a blurry 1200px image that's larger in file size than the sharp 800px original. Find a bigger source or accept the smaller size.

Forgetting about retina. Serving a 400px image in a 400px slot looks soft on a MacBook or modern phone. Serve at 2× (800px image, displayed at 400px CSS pixels). But don't go to 3× — the file size jump isn't worth the imperceptible sharpness gain.

Optimizing once and forgetting srcset. If you serve the same 1200px image to phones and desktops, mobile users download desktop-sized files. Use srcset\ to serve smaller versions to smaller screens:

\html <img src="photo-800.jpg" srcset="photo-400.jpg 400w, photo-800.jpg 800w, photo-1200.jpg 1200w" sizes="(max-width: 600px) 400px, (max-width: 1000px) 800px, 1200px" alt="..."> \\

This alone can cut mobile image payload by 60–70%.

Try It

You don't need Photoshop or a build pipeline for any of this. DailyToolbox's free image compressor runs entirely in your browser — drop in a photo, pick your target size and quality, and download the result. Nothing uploads to a server, which also means your images stay private.

👉 Try the Image Compressor →

FAQ

Will compressing to under 100KB work for all images?
For typical web photos at content sizes (800–1200px), yes. Very detailed images (dense foliage, crowds) compress less efficiently and might land at 120–150 KB at quality 80 — that's fine. Don't chase the number at the expense of visible quality.

Should I use WebP or AVIF?
WebP for now. It's universally supported and the tooling is mature. AVIF saves another 20–30% but encodes slowly and has edge cases in Safari. Revisit AVIF in a year or two when the ecosystem catches up.

Does image compression affect SEO?
Indirectly, yes. Google uses page speed (Core Web Vitals) as a ranking factor, and images are usually the biggest contributor to slow loads. Smaller images → faster LCP → better rankings. Also add descriptive alt\ text and filenames — that part matters for image search.

What about lazy loading?
Lazy loading (loading="lazy"\ on below-the-fold images) is complementary, not a substitute. It defers loading offscreen images, but the images still need to be small when they do load. Do both.

How do I handle user-uploaded images?
Validate on upload: reject files over a sane limit (say 5 MB), then process server-side or in a background job — resize to max dimensions, convert to WebP, strip EXIF. Never serve user uploads raw. Libraries like sharp\ (Node.js), Pillow (Python), or ImageMagick handle this in a few lines. Also consider limiting accepted formats to JPEG, PNG, and WebP to reduce attack surface.

Does compressing images affect print quality?
Web-optimized images (72–96 DPI equivalent, compressed) look bad in print. If your site offers downloadable/printable versions, keep a separate high-res original and serve the optimized version for screen display only. Don't try to make one file serve both purposes.

What's the deal with "Save for Web" in Photoshop?
It's the manual version of everything in this article — resize, format choice, quality slider, metadata stripping. It works fine, but it's 2026: use an automated pipeline or a browser tool instead. Manual "Save for Web" doesn't scale past a handful of images, and humans are inconsistent with quality settings.

Originally published on DailyToolbox.org

How TOTP Two-Factor Authentication Works Under the Hood: RFC 6238 Mathematics, Dynamic Truncation, Time Drift Windows, and

tinali7564-eng — Wed, 14 Oct 2026 22:54:28 +0000

How TOTP Two-Factor Authentication Works Under the Hood: RFC 6238 Mathematics, Dynamic Truncation, Time Drift Windows, and In-Browser 2FA Architecture (2026)

Every day, hundreds of millions of engineers, sysadmins, and security-conscious internet users open an authenticator app—such as Google Authenticator, 1Password, Bitwarden, or Authy—copy a 6-digit numeric code, and paste it into a login prompt before the 30-second countdown circle expires.

Yet if you put your smartphone into Airplane Mode with zero cellular reception, zero Wi-Fi, and Bluetooth disabled, the authenticator app continues to churn out perfectly valid 6-digit tokens like clockwork. When you type that offline code into a server located on the other side of the planet, the server accepts it instantly.

How can two devices on completely separate networks, without transmitting a single packet of data between them, agree on the exact same 6-digit number that changes every 30 seconds?

The answer lies in RFC 6238 (TOTP: Time-Based One-Time Password Algorithm), an elegant cryptographic protocol published by the Internet Engineering Task Force (IETF) in 2011.

While most high-level articles dismiss TOTP as "just hashing a timestamp with a secret key", the underlying engineering involves fascinating bitwise mechanics: big-endian binary time packing, HMAC-SHA1 cryptographic digestion, dynamic 4-bit truncation offsets, and sliding drift window tolerance.

In this comprehensive architectural guide, we unpack the mathematical derivation of RFC 6238, examine why Base32 encoding was chosen over Base64, dissect the security risks of time drift and replay attacks, and implement a production-grade, zero-dependency TOTP generator and verifier using modern Web Crypto standards.

1. The Architectural Evolution: From HOTP (RFC 4226) to TOTP (RFC 6238)

To understand TOTP, you must first understand its direct ancestor: HOTP (HMAC-Based One-Time Password), standardized in RFC 4226 in 2005.

The Foundation: RFC 4226 (HOTP)

HOTP was designed for physical hardware tokens (like RSA SecurID key fobs or YubiKeys). It relies on a shared secret key ($K$) and an event-based monotonic counter ($C$):

$$\text{HOTP}(K, C) = \text{Truncate}(\text{HMAC-SHA1}(K, C))$$

Every time a user presses the physical button on their hardware key:

The hardware increments internal counter $C \leftarrow C + 1$.
The key calculates the HMAC-SHA1 of counter $C$ using pre-shared key $K$.
The result is dynamically truncated into a 6-digit number.
The authentication server also increments its copy of counter $C$ and verifies the code.

HOTP (Event-Based Synchronization):
[Client Token (Button Press)] ──► Counter: 104 ──► Code: 492810
[Server Database]             ──► Counter: 104 ──► Code: 492810  (MATCH! Counter increments to 105)

The Desynchronization Failure Mode:
If the user presses the button 10 times in their pocket without logging in:
[Client Token] ──► Counter: 114 ──► Code: 839102
[Server DB]    ──► Counter: 104 ──► Code: 492810  (DESYNC! Server rejects code)

The fundamental flaw of HOTP is counter desynchronization. If the user accidentally triggers the hardware token or submits failed logins, the client's counter and the server's counter drift out of sync. Servers had to implement complex "lookahead windows" (e.g., checking $C+1$ through $C+50$) to re-synchronize, expanding the attack surface.

The Breakthrough: RFC 6238 (TOTP)

In 2011, the IETF solved the counter desynchronization problem with RFC 6238: replace the event counter ($C$) with current Unix epoch time ($T$).

Because physical clocks on smartphones and cloud servers both track the passage of time according to the universal standard of Coordinated Universal Time (UTC), time acts as an infinite, globally distributed, self-synchronizing counter.

$$\text{TOTP}(K, \text{now}) = \text{HOTP}(K, T)$$

Where the time-step counter $T$ is computed as:

$$T = \left\lfloor \frac{\text{Current Unix Time in Seconds} - T_0}{X} \right\rfloor$$

By standard convention:

$T_0 = 0$ (the Unix epoch: midnight January 1, 1970 UTC).
$X = 30$ seconds (the standard time step window).

Every 30 seconds, $T$ increments by exactly $1$. Whether your phone is in Tokyo, London, or a submarine with no cell service, $\lfloor \text{time} / 30 \rfloor$ produces the exact same integer on both client and server!

2. Step-by-Step Mathematical Derivation: How a 6-Digit Code is Born

Let's trace the exact binary operations required to turn a timestamp and a shared secret into the 6-digit code displayed on your screen.

High-Level TOTP Pipeline:

[Current Time: 1775437200s]
           │
           ▼  (Divide by 30s & floor)
[Time-Step Counter T = 59,181,240]
           │
           ▼  (Pack into 8-byte Big-Endian buffer)
[0x00, 0x00, 0x00, 0x00, 0x03, 0x87, 0x07, 0x98]
           │
           ├────────────────────────────◄ [Shared Secret K (Base32 Decoded)]
           ▼
[HMAC-SHA1(K, T)] ──► 20-byte cryptographic digest (160 bits)
           │
           ▼  (Dynamic Truncation: 4-bit offset lookup)
[Extract 4-byte Slice & Mask Bit 31 to 0] ──► 31-bit Unsigned Integer
           │
           ▼  (Modulo 1,000,000 & zero-pad to 6 digits)
[Final 6-Digit Token: "482091"]

Step 1: Compute and Pack the 64-Bit Time Counter ($T$)

Suppose current Unix epoch time is 1,775,437,200 seconds.
Dividing by 30 gives:
$$T = \lfloor 1,775,437,200 / 30 \rfloor = 59,181,240$$

In hexadecimal, $59,181,240$ is 0x03870798.

Per RFC 4226/6238, $T$ must be serialized as an 8-byte (64-bit) unsigned integer in Big-Endian (network byte order):

Byte Index:   [0]   [1]   [2]   [3]   [4]   [5]   [6]   [7]
Hex Values:  0x00  0x00  0x00  0x00  0x03  0x87  0x07  0x98

Step 2: Compute the HMAC Hash

Next, we take the binary decoded shared secret key ($K$) and compute the HMAC-SHA1 digest over the 8-byte counter buffer:

$$\text{Digest} = \text{HMAC-SHA1}(K, \text{CounterBuffer})$$

SHA-1 produces a digest of exactly 20 bytes (160 bits).

Suppose the resulting 20-byte hash in hexadecimal is:

Byte:   00  01  02  03  04  05  06  07  08  09  10  11  12  13  14  15  16  17  18  19
Hex:    1f  8b  4c  90  a2  13  d8  7e  9b  c4  55  2a  01  4f  89  d2  3b  7c  e5  5b

Step 3: Dynamic Truncation (DT)

A 20-byte cryptographic hash is far too long for a human to type. How do we extract a reliable 6-digit number from it?

If we always extracted the first 4 bytes, attackers analyzing output streams might spot structural bias. Instead, RFC 4226 introduces Dynamic Truncation: the hash dynamically tells us where to look.

1. Determine the Offset

Look at the last byte (byte 19) of the 20-byte hash:
In our example, byte 19 is 0x5b.

We apply a 4-bit bitmask (& 0x0F) to isolate the lowest 4 bits:
$$\text{Offset} = \text{Hash}[19] \ & \ \text{0x0F}$$
$$\text{0x5b} \ & \ \text{0x0F} = \text{0x0b} \ (\text{decimal } 11)$$

Because a 4-bit number can only range from 0 to 15 (0x0 to 0xF), and we need a 4-byte window from a 20-byte buffer:
$$\text{Max Offset} + 4 = 15 + 4 = 19 \le 19$$
The 4-byte slice is guaranteed to never overflow the 20-byte buffer.

2. Extract the 4-Byte Window

Starting at index 11, extract 4 consecutive bytes:

Byte 11: 0x2a
Byte 12: 0x01
Byte 13: 0x4f
Byte 14: 0x89

The combined 32-bit big-endian chunk is 0x2a014f89.

3. Mask Bit 31 to Zero (The Signed Integer Safeguard)

Why does RFC 4226 mandate & 0x7FFFFFFF?
In legacy C and Java runtimes, 32-bit signed integers represent negative numbers when the most significant bit (bit 31) is 1. Performing modulo operations on negative integers yields negative results (-482091), which breaks formatting.

By masking bit 31 with 0x7FFFFFFF:
$$\text{BinaryCode} = (\text{Slice} \ & \ \text{0x7FFFFFFF})$$
$$\text{0x2a014f89} \ & \ \text{0x7FFFFFFF} = \text{0x2a014f89} \ (\text{decimal } 704,737,161)$$

Step 4: Modulo Reduction to Desired Digits

To convert the 31-bit integer into a $d$-digit code (typically $d = 6$):

$$\text{OTP} = \text{BinaryCode} \pmod{10^d}$$
$$\text{OTP} = 704,737,161 \pmod{1,000,000} = 737,161$$

If the resulting number has fewer than 6 digits (e.g., 4523), it is zero-padded on the left: "004523".

Result: The user sees "737161" on their screen!

3. The Shared Secret: Why Base32 and Not Base64?

When you enable 2FA on GitHub, Google, or AWS, you are presented with a QR code and a fallback text string like:

JBSWY3DPEHPK3PXP

Why is this secret key encoded in Base32 (RFC 4648) rather than standard Base64 or Hexadecimal?

The Ergonomics of the Base32 Alphabet

The Base32 alphabet uses only 32 uppercase characters: A–Z and digits 2–7.

Standard Base32 Alphabet (RFC 4648):
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 2 3 4 5 6 7

Problem in Base64 / Hex	How Base32 Solves It
Ambiguous Glyphs	Digit `0` (zero) and letter `O` look identical on mobile screens. Number `1`, lowercase `l`, and uppercase `I` look identical in sans-serif fonts. Base32 completely excludes `0`, `1`, `8`, and `9`, preventing transcription errors.
Case Sensitivity	Base64 uses both lowercase and uppercase (`a` vs `A`). Base32 is strictly case-insensitive, so users typing on mobile keyboards without autocorrect will never fail due to capitalization.
Punctuation Characters	Base64 requires symbols like `+`, `/`, and `=`. These trigger awkward keyboard layout shifts on iOS and Android keyboards. Base32 is purely alphanumeric.

The Standard `otpauth://` URI Scheme

Under the hood, every 2FA QR code encodes a standard URI defined by Google's Key Uri Format:

otpauth://totp/DailyToolbox:alice%40example.com?secret=JBSWY3DPEHPK3PXP&issuer=DailyToolbox&algorithm=SHA1&digits=6&period=30

type: totp (or hotp for event counters).
label: Issuer:AccountName (displayed in the authenticator app UI).
secret: The unpadded Base32 encoded secret key.
algorithm: SHA1 (default), SHA256, or SHA512.
digits: 6 (default) or 8.
period: Time-step duration in seconds (30 default).

4. Time Drift, Clock Skew, and the Replay Attack Threat Model

In a theoretical world, all computer clocks are synchronized perfectly via atomic clocks. In reality, client clocks and server clocks inevitably drift.

A smartphone left in a desk drawer for three weeks, a user traveling on an airplane across timezones, or an uncalibrated virtual machine clock can easily be 10 to 45 seconds off from UTC.

Handling Clock Skew: The Sliding Window

If servers only accepted the exact current time step $T$, a user whose phone clock was running just 3 seconds fast would see their codes rejected 100% of the time.

To provide robust user experience, RFC 6238 specifies a verification window:

$$\text{Valid Timesteps} = [T - W, \dots, T, \dots, T + W]$$

Where $W$ is the window tolerance parameter.

Window Tolerance W = 1 (Standard Production Setting):

[Time T - 1: Previous 30s] ──► Valid (covers slightly fast server / slow phone)
[Time T    : Current  30s] ──► Valid (nominal sync)
[Time T + 1: Next     30s] ──► Valid (covers slightly slow server / fast phone)

Total Effective Code Lifetime: 30s * 3 = 90 seconds

Setting $W = 1$ allows for up to $\pm 30$ seconds of clock drift while maintaining high security. Setting $W > 2$ is considered an insecure anti-pattern, as it expands the window of vulnerability to over 2.5 minutes.

The Silent Vulnerability: Replay Attacks Within the Active Window

Because a single TOTP code is valid for 30 seconds (and up to 90 seconds under a $W = 1$ drift window), what happens if an eavesdropper intercepts the 6-digit code via man-in-the-middle (MitM) or shoulder-surfing?

Can the attacker replay that same code 5 seconds later before the timer expires?

CRITICAL SECURITY RULE: A One-Time Password must strictly be used EXACTLY ONCE.

To prevent Token Replay Attacks, production authentication systems must track consumed tokens:

-- Production Database Schema Example
CREATE TABLE user_totp_state (
    user_id UUID PRIMARY KEY REFERENCES users(id),
    secret_key BYTEA NOT NULL,
    last_verified_step BIGINT NOT NULL DEFAULT 0,
    failed_attempts INT NOT NULL DEFAULT 0
);

Verification Algorithm with Replay Protection:

Receive incoming code from user.
Calculate time step $T = \lfloor \text{now} / 30 \rfloor$.
Check steps $t \in [T-1, T, T+1]$.
If code matches step $t$:
- Check: Is $t \le \text{last_verified_step}$?
- If YES: REJECT IMMEDIATELY! (Token has already been consumed).
- If NO: Update last_verified_step $\leftarrow t$, clear failed attempts, and allow login!

5. Zero-Dependency TypeScript Implementation (Web Crypto API)

Here is a clean, production-grade implementation of RFC 6238 TOTP using the native browser and Node.js Web Crypto API. Zero external npm dependencies.

/**
 * RFC 4648 Base32 Decoder
 */
function base32ToBytes(base32: string): Uint8Array {
  const alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";
  const clean = base32.toUpperCase().replace(/=+$/, "").replace(/\s/g, "");

  let bits = 0;
  let value = 0;
  const output: number[] = [];

  for (let i = 0; i < clean.length; i++) {
    const idx = alphabet.indexOf(clean[i]);
    if (idx === -1) {
      throw new Error(`Invalid Base32 character: ${clean[i]}`);
    }
    value = (value << 5) | idx;
    bits += 5;

    if (bits >= 8) {
      output.push((value >>> (bits - 8)) & 0xff);
      bits -= 8;
    }
  }

  return new Uint8Array(output);
}

/**
 * Generate a 6-digit TOTP code per RFC 6238
 * @param secretBase32 The user's shared secret in Base32 format
 * @param timestampSeconds Unix epoch time in seconds (defaults to now)
 * @param step Window period in seconds (default 30)
 * @param digits Number of output digits (default 6)
 */
export async function generateTOTP(
  secretBase32: string,
  timestampSeconds: number = Math.floor(Date.now() / 1000),
  step: number = 30,
  digits: number = 6
): Promise<{ code: string; secondsRemaining: number }> {
  const counter = Math.floor(timestampSeconds / step);
  const secondsRemaining = step - (timestampSeconds % step);

  // 1. Pack 64-bit integer counter in Big-Endian
  const counterBuffer = new ArrayBuffer(8);
  const dataView = new DataView(counterBuffer);
  // High 32 bits
  dataView.setUint32(0, Math.floor(counter / 0x100000000), false);
  // Low 32 bits
  dataView.setUint32(4, counter & 0xffffffff, false);

  // 2. Decode Base32 secret key into raw bytes
  const keyBytes = base32ToBytes(secretBase32);

  // 3. Import key into Web Crypto API
  const cryptoKey = await crypto.subtle.importKey(
    "raw",
    keyBytes,
    { name: "HMAC", hash: { name: "SHA-1" } },
    false,
    ["sign"]
  );

  // 4. Compute HMAC-SHA1
  const hmacSignature = await crypto.subtle.sign("HMAC", cryptoKey, counterBuffer);
  const hashBytes = new Uint8Array(hmacSignature);

  // 5. Dynamic Truncation: extract lowest 4 bits of 20th byte
  const offset = hashBytes[19] & 0x0f;

  // 6. Extract 4-byte slice and mask bit 31
  const binaryCode =
    ((hashBytes[offset] & 0x7f) << 24) |
    ((hashBytes[offset + 1] & 0xff) << 16) |
    ((hashBytes[offset + 2] & 0xff) << 8) |
    (hashBytes[offset + 3] & 0xff);

  // 7. Modulo 10^digits
  const otpNumber = binaryCode % Math.pow(10, digits);
  const code = otpNumber.toString().padStart(digits, "0");

  return { code, secondsRemaining };
}

/**
 * Verify a user-submitted TOTP code with time-drift window support
 */
export async function verifyTOTP(
  secretBase32: string,
  userInputCode: string,
  windowTolerance: number = 1
): Promise<boolean> {
  const now = Math.floor(Date.now() / 1000);
  const step = 30;

  for (let w = -windowTolerance; w <= windowTolerance; w++) {
    const checkTime = now + w * step;
    const { code } = await generateTOTP(secretBase32, checkTime, step, userInputCode.length);
    if (code === userInputCode) {
      return true;
    }
  }

  return false;
}

6. Authentication Matrix: How TOTP Compares to Other 2FA Methods

Security Dimension	SMS One-Time Passwords	Email Verification Codes	TOTP (RFC 6238 Authenticator)	FIDO2 / WebAuthn (Passkeys & YubiKeys)
Phishing Resistance	❌ Vulnerable (MitM easily forwards SMS)	❌ Vulnerable (MitM forwards email code)	⚠️ Partial (Vulnerable to real-time reverse proxies like Evilginx)	🛡️ 100% Cryptographic Resistance (Bound to origin domain)
SIM Swapping Risk	🔴 Severe (Carrier social engineering bypasses SMS)	🟢 None	🟢 Zero (Key stored on local device)	🟢 Zero
Offline Reliability	❌ Fails without cell tower reception	❌ Fails without internet connection	✅ Works 100% offline in Airplane Mode	✅ Works 100% offline
Cost & Latency	🔴 Expensive (Carrier SMS gateway fees: ~$0.05/msg)	🟡 Moderate (Email delivery delays: 5-60s)	🟢 Free & Instantaneous (0 latency)	🟡 Requires hardware keys or modern OS
Regulatory Standing (NIST SP 800-63B)	❌ Deprecated / Discouraged	⚠️ Restricted	✅ Fully Approved (AAL2 Standard)	🏆 Gold Standard (AAL3 Highest)

7. Developer Tooling Matrix

When building or debugging two-factor authentication workflows, leverage DailyToolbox's suite of secure, client-side developer utilities:

Workflow Need	Recommended Tool	Architectural Application
2FA Testing & Generation	TOTP Authenticator Generator	Generate live 6-digit codes and verify time-step countdowns client-side without cloud transmission.
HMAC Verification	HMAC Generator	Calculate raw HMAC-SHA1, HMAC-SHA256, and HMAC-SHA512 hashes to audit cryptographic signatures.
Binary & String Conversion	Base64 Encoder / Decoder	Convert between raw hex strings, Base32, and Base64 representations.
Epoch Time Inspection	Unix Timestamp Converter	Inspect epoch timestamps and verify exact 30-second time-step bucket allocations.
Emergency Recovery Codes	Password Generator	Generate cryptographically random 16-character backup recovery codes with high entropy.

8. Frequently Asked Questions (FAQ)

Q1: Why does Google Authenticator still use SHA-1 instead of SHA-256 or SHA-512?

While collision attacks (like SHAttered) exist against SHA-1 in digital signature verification, HMAC-SHA1 remains cryptographically secure against pre-image attacks. In HMAC, the secret key is prepended and hashed twice ($H(K \oplus \text{opad} \parallel H(K \oplus \text{ipad} \parallel M))$), rendering collision attacks completely ineffective. Furthermore, because RFC 6238 only extracts a 31-bit integer out of the hash, the extra bit depth of SHA-256 offers virtually zero practical security gain for 6-digit numeric outputs.

Q2: What happens if an attacker steals my QR code or Base32 secret?

If an attacker obtains your Base32 secret key, they can clone your authenticator onto their device and generate the exact same codes. The security of TOTP rests entirely on the confidentiality of the shared secret. This is why backup codes, recovery phrases, and authenticator export QR codes must be guarded with the same rigor as master passwords.

Q3: Can an attacker reverse-engineer the secret key by intercepting multiple 6-digit codes?

Mathematically impossible. Each 6-digit code represents a one-way dynamic truncation and modulo reduction of a 160-bit HMAC output. Because the HMAC function is a cryptographic pseudo-random function (PRF), knowing hundreds of past 6-digit outputs yields zero computational advantage in discovering the underlying 128-bit or 160-bit secret key.

Q4: Is TOTP vulnerable to modern adversary-in-the-middle (AitM) phishing attacks?

Yes. While TOTP eliminates SIM swapping and credential stuffing, it is not origin-bound. If an attacker tricks a user into entering their username, password, and TOTP code onto a fraudulent phishing proxy (e.g., using tools like Evilginx), the proxy server can forward the 6-digit code to the legitimate service within the active 30-second window and establish an authenticated session. To defeat AitM phishing, organizations are actively migrating toward FIDO2 / WebAuthn Passkeys, which cryptographically bind authentication tokens to the browser's origin domain via public key cryptography.

Originally published on DailyToolbox.org

AES-GCM vs AES-CBC: Why CBC Mode Fails in Modern Web Apps, Galois Counter Architecture, and Zero-Dependency Web Crypto Implementation (2026)

tinali7564-eng — Mon, 12 Oct 2026 02:42:13 +0000

AES-GCM vs AES-CBC: Why CBC Mode Fails in Modern Web Apps, Galois Counter Architecture, and Zero-Dependency Web Crypto Implementation (2026)

In modern web security and backend architectures, symmetric key cryptography is the workhorse of data protection. Whether encrypting session tokens, securing client-side database storage (OPFS/IndexedDB), protecting REST API payloads, or establishing TLS records, the Advanced Encryption Standard (AES) is universal.

Yet, many legacy production codebases and developer tutorials still instantiate AES using Cipher Block Chaining (CBC) mode. Teams assume that simply picking a 256-bit key makes their data impenetrable.

In real-world distributed systems, confidentiality without integrity guarantees disaster.

This comprehensive technical guide dissects why AES-CBC fails under active network attacks (padding oracle and bit-flipping vulnerabilities), how Galois/Counter Mode (AES-GCM) solves this through Authenticated Encryption with Associated Data (AEAD), the underlying mathematics of GHASH and counter trees, critical pitfalls like IV/nonce collision catastrophic failures, and production-ready, zero-dependency implementations utilizing the standard Web Crypto API.

1. The Anatomy of AES: Block Ciphers vs. Block Cipher Modes

AES (specified in NIST FIPS 197) is strictly a block cipher. It transforms a fixed-length 128-bit block (16 bytes) of plaintext into a 128-bit block of ciphertext using a key of 128, 192, or 256 bits through repeated substitution-permutation rounds (SubBytes, ShiftRows, MixColumns, AddRoundKey).

Because real-world data is rarely exactly 16 bytes, block ciphers require an operational mode to handle arbitrary lengths:

+-----------------------------------------------------------------------------------------+
|                                    BLOCK CIPHER MODES                                   |
+-----------------------------------------------------------------------------------------+
| Mode    | Authentication? | Parallelizable? | Padding Needed? | Vulnerability Profile   |
+---------+-----------------+-----------------+-----------------+-------------------------+
| ECB     | No              | Yes             | Yes (PKCS#7)    | Catastrophic (Patterns) |
| CBC     | No (Conf. only) | No (Sequential) | Yes (PKCS#7)    | Padding Oracle & Bitflip|
| CTR     | No (Stream-like)| Yes (Both Enc/Dec)| No            | Bit-flipping attacks    |
| GCM     | Yes (AEAD)      | Yes (Hardware)  | No (Stream mode)| IV reuse breaks key/auth|
+-----------------------------------------------------------------------------------------+

2. Why AES-CBC Fails in Modern Web Architectures

To appreciate AES-GCM, one must first understand why AES-CBC has been systematically phased out by TLS 1.3, NIST SP 800-38D, and the Web Cryptography Working Group.

The Mechanics of AES-CBC

In CBC mode, each plaintext block $P_i$ is XORed with the preceding ciphertext block $C_{i-1}$ before encryption:

$$C_0 = IV$$
$$C_i = E_K(P_i \oplus C_{i-1}) \quad ext{for } i \ge 1$$

To decrypt:
$$P_i = D_K(C_i) \oplus C_{i-1}$$

Encryption in CBC:
 Plaintext P1            Plaintext P2
     │                       │
     ▼                       ▼
IV ─►(+)                 C1 ─►(+)
     │                       │
     ▼                       ▼
 ┌───────┐               ┌───────┐
 │ AES-K │               │ AES-K │
 └───────┘               └───────┘
     │                       │
     ▼                       ▼
 Ciphertext C1           Ciphertext C2

Vulnerability 1: The Padding Oracle Attack (POODLE / Vaudenay)

Because AES operates on 16-byte blocks, plaintext must be padded, almost universally using PKCS#7. If 5 bytes of padding are needed, the padding bytes are 0x05 0x05 0x05 0x05 0x05.

When a server decrypts an incoming message:

It runs $D_K(C_i) \oplus C_{i-1}$.
It inspects the trailing bytes to validate the PKCS#7 format.
If invalid, the server throws an error (e.g., HTTP 400 or HTTP 500 InvalidPaddingException).
If valid, it passes the data to the JSON/business parser. If the business parser fails, it returns an application-level error (e.g., HTTP 401 InvalidToken).

This timing or error differential creates an oracle. An attacker intercepting an encrypted cookie or session token can modify individual bytes of $C_{i-1}$. By iterating through all 256 possible byte values and observing whether the server returns a padding error or an application error, the attacker systematically decrypts the entire payload byte by byte without ever obtaining the secret key $K$.

Vulnerability 2: Malleability and Bit-Flipping

CBC mode provides zero integrity checks. Consider decryption:
$$P_i = D_K(C_i) \oplus C_{i-1}$$

Notice that if an attacker flips bit $b$ in ciphertext block $C_{i-1}$, that exact same bit $b$ flips in plaintext block $P_i$!
While block $P_{i-1}$ will be completely scrambled into random garbage during decryption, block $P_i$ is manipulated with surgical precision. If an attacker knows that block $P_2$ contains role=user;, flipping specific bits in $C_1$ alters the decrypted output to role=root;.

3. How AES-GCM Solves Integrity: The AEAD Revolution

AEAD (Authenticated Encryption with Associated Data) combines cryptographic confidentiality with cryptographic authenticity in a single unified primitive.

AES-GCM achieves this by marrying Counter Mode (CTR) with a high-speed universal hash function based on Galois field multiplication (GHASH).

High-Level Architecture of AES-GCM

                           Secret Key K
                                │
               ┌────────────────┴────────────────┐
               ▼                                 ▼
         Subkey Generation                CTR Engine (AES)
           H = AES_K(0^128)                      │
               │                                 │
               │                   Plaintext Blocks P1..Pn
               │                                 │
               │                                 ▼
               │                        Ciphertext Blocks C1..Cn
               │                                 │
               └───────────────► ┌─────────────┐ │
                                 │    GHASH    │◄┘
                                 │  (GF(2^128) │
                                 └─────────────┘
                                        │
                                        ▼
                                 Authentication
                                   Tag (128-bit)

The Counter (CTR) Pipeline

AES-GCM does not pad data. Instead, it treats AES as a pseudo-random stream generator:

Given a 96-bit Initialization Vector (IV / Nonce) and a 32-bit counter starting at 1, the cipher constructs 128-bit counter blocks: $$J_0 = ext{IV} \parallel 0^{31}1$$ $$J_i = ext{IV} \parallel ext{inc}{32}(J{i-1})$$
It encrypts each counter block with the key: $S_i = E_K(J_i)$.
Plaintext blocks $P_i$ are simply XORed with $S_i$: $$C_i = P_i \oplus S_i$$
The final block is truncated to match the exact byte length of the plaintext. No padding is ever added.

The GHASH Pipeline & Authentication Tag

To verify that neither the ciphertext nor the metadata has been tampered with:

A hash subkey $H$ is computed by encrypting a 128-bit zero block: $H = E_K(0^{128})$.
All Associated Data (AAD) (unencrypted headers, routing data) and Ciphertext blocks are fed into a polynomial evaluation over the Galois field $ ext{GF}(2^{128})$ with irreducible polynomial: $$f(x) = x^{128} + x^7 + x^2 + x + 1$$
The result of GHASH is XORed with $E_K(J_0)$ to produce the 128-bit Authentication Tag (Auth Tag).

If a single bit in the ciphertext or associated data is altered, tag verification mathematically fails with a probability of $1 - 2^{-128}$, causing the decryptor to immediately abort before any downstream code ever touches the decrypted buffer.

4. The Fatal Pitfall: The AES-GCM Nonce Reuse Catastrophe

While AES-GCM is virtually uncrackable when used correctly, it has one unforgiving Achilles' heel: Nonce (IV) Reuse.

In AES-GCM, the 96-bit (12-byte) IV must never, under any circumstance, be used more than once with the same key.

What Happens if an IV is Reused?

Plaintext Recovery via XOR: $$C_1 = P_1 \oplus S$$ $$C_2 = P_2 \oplus S$$ $$C_1 \oplus C_2 = P_1 \oplus P_2$$ XORing the two ciphertexts strips away the encryption keystream entirely, revealing the XOR difference between the two plaintexts. Statistical frequency analysis (or known-plaintext attacks) allows trivial extraction of both plaintexts.
Total Authentication Key Extraction (The "Forbidden Attack"): Reusing an IV reveals two equations sharing the same GHASH key $H$. Solving this polynomial in $ ext{GF}(2^{128})$ completely exposes the authentication subkey $H$. Once an attacker possesses $H$, they can forge valid authentication tags for arbitrary spoofed ciphertexts.

Safe IV Generation Rules

12-Byte (96-bit) Length: Always use exactly 12 bytes. Although GCM supports other IV lengths, non-96-bit IVs trigger an internal GHASH preprocessing step that increases latency and introduces subtle collision vectors.
Cryptographically Secure Randomness: Always use crypto.getRandomValues(new Uint8Array(12)) or a monotonic 64-bit integer counter paired with a unique 32-bit machine identifier. Never use Math.random().
Key Rotation Thresholds: Under random 96-bit IV generation, by the Birthday Paradox, encrypting more than $2^{32}$ messages with the same key raises collision risk above acceptable cryptographic margins. Rotate your master key well before reaching $2^{32}$ operations.

5. End-to-End Implementation: Production Web Crypto API in TypeScript

The Web Crypto API (globalThis.crypto.subtle) is built into all modern browsers (Chrome, Safari, Firefox, Edge) as well as Node.js (18+) and Edge runtimes (Cloudflare Workers, Deno, Bun). It leverages hardware AES-NI CPU instructions, executing cryptographic operations at gigabytes per second directly in C++ / machine code outside the JavaScript V8 heap.

Here is a hardened, production-grade implementation of AES-256-GCM featuring PBKDF2 password derivation, 96-bit random nonce generation, and binary payload serialization.

The Cryptographic Envelope Layout

To store or transmit encrypted data, bundle the metadata into a unified binary buffer:

+-------------------------------------------------------------------------+
|                  AES-GCM CRYPTOGRAPHIC ENVELOPE FORMAT                  |
+-------------------------------------------------------------------------+
| Salt (16 Bytes) | IV / Nonce (12 Bytes) | Ciphertext + Tag (N + 16 Bytes) |
+-------------------------------------------------------------------------+

Complete Zero-Dependency TypeScript Module

/**
 * Production-Grade In-Browser AES-256-GCM Engine
 * Zero external dependencies. Uses W3C Web Crypto API.
 */

export interface EncryptedPayload {
  combinedBase64: string; // Salt + IV + Ciphertext+Tag encoded as Base64
  saltHex: string;
  ivHex: string;
  tagHex: string;
}

const PBKDF2_ITERATIONS = 600_000; // Aligned with OWASP 2026 Password Storage Guidelines
const SALT_LENGTH_BYTES = 16;
const IV_LENGTH_BYTES = 12; // Strictly 96-bit for optimal GCM performance
const TAG_LENGTH_BITS = 128; // Standard 16-byte authentication tag

/**
 * Derives a 256-bit AES-GCM CryptoKey from a passphrase and salt using PBKDF2-HMAC-SHA256
 */
async function deriveKeyFromPassword(
  passphrase: string,
  salt: Uint8Array,
  usage: KeyUsage[]
): Promise<CryptoKey> {
  const enc = new TextEncoder();
  const rawKeyMaterial = await crypto.subtle.importKey(
    'raw',
    enc.encode(passphrase),
    'PBKDF2',
    false,
    ['deriveKey']
  );

  return await crypto.subtle.deriveKey(
    {
      name: 'PBKDF2',
      salt: salt,
      iterations: PBKDF2_ITERATIONS,
      hash: 'SHA-256'
    },
    rawKeyMaterial,
    { name: 'AES-GCM', length: 256 },
    false, // Do not allow key extraction from secure memory
    usage
  );
}

/**
 * Encrypts arbitrary text using AES-256-GCM with hardware acceleration
 */
export async function encryptAesGcm(
  plaintext: string,
  passphrase: string
): Promise<string> {
  const enc = new TextEncoder();
  const plaintextBytes = enc.encode(plaintext);

  // 1. Generate cryptographically secure random Salt and IV
  const salt = crypto.getRandomValues(new Uint8Array(SALT_LENGTH_BYTES));
  const iv = crypto.getRandomValues(new Uint8Array(IV_LENGTH_BYTES));

  // 2. Derive key
  const key = await deriveKeyFromPassword(passphrase, salt, ['encrypt']);

  // 3. Perform AES-GCM encryption
  // In Web Crypto API, the 16-byte Auth Tag is automatically appended to the end of ciphertext
  const ciphertextWithTag = await crypto.subtle.encrypt(
    {
      name: 'AES-GCM',
      iv: iv,
      tagLength: TAG_LENGTH_BITS
    },
    key,
    plaintextBytes
  );

  const ciphertextArray = new Uint8Array(ciphertextWithTag);

  // 4. Combine: Salt (16B) + IV (12B) + CiphertextWithTag (NB)
  const combinedBuffer = new Uint8Array(
    salt.length + iv.length + ciphertextArray.length
  );
  combinedBuffer.set(salt, 0);
  combinedBuffer.set(iv, salt.length);
  combinedBuffer.set(ciphertextArray, salt.length + iv.length);

  // 5. Convert to Base64 for transport/storage
  let binary = '';
  const len = combinedBuffer.byteLength;
  for (let i = 0; i < len; i++) {
    binary += String.fromCharCode(combinedBuffer[i]);
  }
  return btoa(binary);
}

/**
 * Decrypts a combined Base64 payload, verifying authentication tag before returning text
 */
export async function decryptAesGcm(
  combinedBase64: string,
  passphrase: string
): Promise<string> {
  // 1. Decode Base64 to binary buffer
  const binary = atob(combinedBase64);
  const combinedBuffer = new Uint8Array(binary.length);
  for (let i = 0; i < binary.length; i++) {
    combinedBuffer[i] = binary.charCodeAt(i);
  }

  // Minimum payload length: 16 (salt) + 12 (iv) + 16 (auth tag) = 44 bytes
  if (combinedBuffer.length < SALT_LENGTH_BYTES + IV_LENGTH_BYTES + 16) {
    throw new Error('Malformed ciphertext payload: buffer too short.');
  }

  // 2. Slice components
  const salt = combinedBuffer.slice(0, SALT_LENGTH_BYTES);
  const iv = combinedBuffer.slice(
    SALT_LENGTH_BYTES,
    SALT_LENGTH_BYTES + IV_LENGTH_BYTES
  );
  const ciphertextWithTag = combinedBuffer.slice(
    SALT_LENGTH_BYTES + IV_LENGTH_BYTES
  );

  // 3. Derive decryption key with matching salt
  const key = await deriveKeyFromPassword(passphrase, salt, ['decrypt']);

  // 4. Decrypt and verify tag
  try {
    const decryptedBuffer = await crypto.subtle.decrypt(
      {
        name: 'AES-GCM',
        iv: iv,
        tagLength: TAG_LENGTH_BITS
      },
      key,
      ciphertextWithTag
    );

    const dec = new TextDecoder();
    return dec.decode(decryptedBuffer);
  } catch (err) {
    // Web Crypto rejects the promise if authentication tag fails or data is corrupted
    throw new Error('Decryption failed: Integrity tag mismatch or invalid password.');
  }
}

6. Real-World Architectural Comparison: Performance & Threat Model

Dimension	AES-CBC (+ HMAC-SHA256)	AES-256-GCM
Integrity Model	Requires manual "Encrypt-then-MAC"	Native AEAD (Hardware GHASH)
Implementation Complexity	High (Requires 2 keys, 2 passes)	Low (Single standard API call)
Padding Oracle Vulnerability	Critical risk if padding handled before MAC	Immunized (No PKCS#7 padding used)
Bit-Flipping Malleability	High if unauthenticated	Immunized (Tag mismatch aborts)
Processing Throughput	Sequential encryption (~450 MB/s)	Parallelized SIMD / PCLMULQDQ (~3.8 GB/s)
Standards Compliance	Deprecated in TLS 1.3 / NIST SP 800-38D	Mandatory in TLS 1.3, WireGuard, IPsec

7. Frequently Asked Questions (FAQ)

Can I use a 16-byte (128-bit) IV with AES-GCM?

Technically yes, but never recommend it. When the IV length is anything other than 96 bits (12 bytes), NIST SP 800-38D mandates running the IV through GHASH to compress it into 96 bits. This wastes CPU cycles and creates theoretical hash collision vectors. Stick to 12 bytes.

Is AES-GCM vulnerable to quantum computers?

GCM mode itself is unaffected by quantum computers. For the underlying cipher, Grover's Algorithm reduces the effective security of symmetric keys by half ($N o \sqrt{N}$). Therefore, AES-128 offers 64-bit post-quantum security (vulnerable), whereas AES-256 provides 128-bit post-quantum security, which remains unbreakable by quantum computing architectures for the foreseeable decades. Always specify 256-bit keys for long-term data archival.

Why does `crypto.subtle.decrypt` throw a generic DOMException when the password is wrong?

This is an intentional cryptographic defense against side-channel analysis. The Web Crypto API does not distinguish between a wrong password, corrupted ciphertext, altered IV, or forged authentication tag. Any mismatch triggers an authentication failure, preventing attackers from learning why decryption failed.

Summary & Actionable Recommendations

Migrate Legacy CBC Immediately: Replace all internal AES-CBC routines with AES-256-GCM. If backward compatibility with CBC is strictly required, wrap CBC inside a strict Encrypt-then-MAC (HMAC-SHA256) pipeline.
Standardize on 96-Bit Nonces: Always supply a fresh, cryptographically random 12-byte IV (crypto.getRandomValues) for every single encryption call.
Never Strip the Auth Tag: Keep the 128-bit authentication tag attached to the ciphertext. Never truncate or discard the tag to save bandwidth.
Use Native Browser APIs: Avoid third-party pure-JavaScript crypto libraries like CryptoJS for heavy payloads. The native window.crypto.subtle API runs in hardware-accelerated C++ instructions with built-in constant-time guarantees.

Originally published on DailyToolbox.org

How Bcrypt Password Hashing Works: Eksblowfish Key Schedule, Cost Factor Mathematics, the 72-Byte Truncation Trap, and Argon2id Migration (2026)

tinali7564-eng — Sun, 11 Oct 2026 11:54:45 +0000

How Bcrypt Password Hashing Works: Eksblowfish Key Schedule, Cost Factor Mathematics, the 72-Byte Truncation Trap, and Argon2id Migration (2026)

In modern web application security, storing user passwords securely in databases is a fundamental requirement. Yet security teams still routinely uncover production architectures that store credentials using fast cryptographic digests like MD5, SHA-1, SHA-256, or naive salted hashes.

Fast hash functions are engineered for high-throughput checksum validation and data integrity. Modern desktop GPUs (such as an NVIDIA RTX 4090) compute over 25 billion SHA-256 hashes per second. If a database table containing salted SHA-256 hashes is leaked, an attacker can crack typical user passwords within minutes using tools like Hashcat.

Standardized by Niels Provos and David Mazières in 1999 for OpenBSD, Bcrypt was intentionally designed to be computationally expensive, memory-bound, and adaptively slow.

This comprehensive engineering guide dissects how Bcrypt works under the hood: the Eksblowfish (Expensive Key Schedule Blowfish) architecture, the anatomy of the 60-character modular crypt string, the exponential mathematics of cost factors, the dangerous 72-byte truncation trap, and modern migration paths toward Argon2id (aligned with OWASP 2026 password storage standards).

1. Why Fast Hashes (SHA-256 / MD5) Fail for Passwords

Cryptographic hash functions fall into two distinct engineering paradigms:

+-------------------------------------------------------------------------------------------------+
|                                 FAST HASHES vs. PASSWORD HASHES                                 |
+-------------------------------------------------------------------------------------------------+
| Property            | Fast Hashes (SHA-256, BLAKE3, MD5)      | Password Hashes (Bcrypt, Argon2id)      |
+---------------------+-----------------------------------------+---------------------------------+
| Design Objective    | Maximize throughput (GB/s on CPU/ASIC)  | Minimize cracker speed (Compute/Memory bound)|
| Hardware Scaling    | Billions of guesses/sec on consumer GPU | Hundreds of guesses/sec on GPU  |
| Adaptive Work Factor| No (Fixed computational steps)          | Yes (Exponentially tunable)     |
| Built-in Salt       | Requires manual application & storage   | Natively embedded in output     |
| Target Use Case     | File checksums, HMAC, Git commits       | Password verification, Key derivation |
+-------------------------------------------------------------------------------------------------+

When an attacker dumps a password table, they run offline brute-force attacks. Because they control the hardware, rate limits on your login endpoint cannot protect the database dump. Protection relies entirely on the per-hash computational cost imposed by the algorithm.

2. The Internal Mechanics of Bcrypt: Eksblowfish

Bcrypt is built on top of Bruce Schneier's Blowfish symmetric block cipher, replacing its key schedule with Eksblowfish (Expensive Key Schedule Blowfish).

Blowfish uses a Feistel network with sixteen 32-bit subkeys ($P$-array: $P_1 \dots P_{18}$) and four 256-entry S-boxes ($S_1 \dots S_4$, each containing 8-bit to 32-bit substitutions), initialized with the fractional digits of $\pi$.

                        Passphrase + 128-bit Salt
                                   │
                                   ▼
                       ┌───────────────────────┐
                       │   SetupState (π-hex)  │
                       └───────────┬───────────┘
                                   │
                                   ▼
                 ┌───────────────────────────────────┐
                 │ ExpandKey(state, salt, password)  │
                 └─────────────────┬─────────────────┘
                                   │
       ┌───────────────────────────┴───────────────────────────┐
       │                                                       │
       ▼                                                       ▼
 ┌───────────┐                                           ┌───────────┐
 │ Cost Iter │ ◄─────────────────────────────────────────┤ Round: 2^c│
 └─────┬─────┘                                           └───────────┘
       │
       ▼
 ┌─────────────────────────────────────────────────────────────┐
 │ Encrypt "OrpheanBeholderScryDoubt" 64 times (192 bits / 24B)│
 └─────────────────────────────┬───────────────────────────────┘
                               │
                               ▼
            Final 60-character Modular Crypt String

The Three-Phase Pipeline

State Initialization (SetupState): The $P$-array and $S$-boxes are loaded with the standard mathematical constant $\pi$ hex values.
Key Expansion with Salt & Password (ExpandKey): The 128-bit random salt and password are woven into the state. Then, the algorithm loops $2^{ ext{cost}}$ times, cyclically re-encrypting the state with the salt and the password. This guarantees that neither salt alone nor password alone can pre-compute subkeys.
Ciphertext Encryption (OrpheanBeholderScryDoubt): Once the subkeys are fully expanded, the cipher encrypts the fixed 24-byte ASCII string "OrpheanBeholderScryDoubt" 64 consecutive times using electronic codebook (ECB) mode. The resulting 192-bit ciphertext forms the final checksum.

Because Eksblowfish repeatedly alters all 4KB of S-box lookup tables in tight memory cycles, executing Bcrypt on GPUs is hindered by memory-divergent branch stalls, nullifying traditional GPU parallelism advantages over CPUs.

3. Dissecting the 60-Character Bcrypt Hash Format

A standard Bcrypt hash is serialized into the Modular Crypt Format (MCF), exactly 60 ASCII characters long:

 $2b$12$e8YvXmK.u2QeZ4J8oYpU7eM0xWb9N.Z1iV8jL2oQ4uG6wA8bC0dE
 └───┘──┘└────────────────────┘└──────────────────────────────┘
  │   │            │                          │
  │   │            │                          └─ Checksum (31 chars = 24 bytes)
  │   │            └─ 128-bit Salt (22 chars = 16 bytes)
  │   └─ Cost Factor (2 digits = 2^12 = 4,096 iterations)
  └─ Algorithm Prefix ($2a$, $2b$, or $2y$)

1. Algorithm Identifiers ( $2a$ , $2b$ , $2y$ )

$2$ (1999): Original OpenBSD implementation.
$2a$ (2002): Added specification for handling UTF-8 zero-termination.
$2y$ (2011): Introduced in PHP's crypt library to fix an unsigned vs. signed 8-bit character bug where characters with byte values > 127 caused hash mismatches.
$2b$ (2014): The modern, universal standard. Fixed an OpenBSD wraparound integer overflow bug. Always use $2b$ in modern codebases.

2. Radix-64 Encoding

Bcrypt uses a custom base-64 alphabet for serialization:
./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789
(Note: Standard RFC 4648 Base64 begins with A-Z, while Bcrypt Radix-64 begins with . and /).

4. The Critical Engineering Trap: 72-Byte Password Truncation

The most notorious architectural caveat of Bcrypt is its hard 72-byte password length ceiling.

In the Blowfish algorithm, the key schedule processes the key in 32-bit words up to 448 bits ($56 imes 8$ bits). Bcrypt extends this to a maximum internal buffer of 72 bytes (576 bits).

What Happens When Passwords Exceed 72 Bytes?

Any bytes beyond the 72nd byte are completely ignored.

// These two passwords generate the EXACT same Bcrypt hash:
const pass1 = "A".repeat(72) + "SuperSecretPassword123!";
const pass2 = "A".repeat(72) + "TotallyDifferentPassword999?";

// bcrypt.compareSync(pass1, hashOfPass2) === true!

The Multi-byte UTF-8 Hazard

This is a byte limit, not a character limit.

ASCII characters take 1 byte: "password123" = 11 bytes.
Accented European characters take 2 bytes: "café" = 5 bytes.
Chinese, Japanese, or Korean characters take 3 bytes: "我的密码很长安全吗" (9 chars) = 27 bytes.
Emojis take 4 bytes: 🔐🔑🛡️🚀 (4 chars) = 16 bytes.

A user providing an 18-character Japanese passphrase or a password with several emojis can inadvertently trigger silent truncation at 72 bytes.

How to Fix Truncation Properly

If your application allows passphrases of arbitrary length (e.g. up to 1,024 characters), pre-hash the password using SHA-256 or SHA-512 before feeding it into Bcrypt:

import crypto from 'node:crypto';
import bcrypt from 'bcrypt';

function safeHashPassword(plaintext: string, rounds = 12): Promise<string> {
  // Pre-hash with SHA-256 to collapse arbitrary-length input into exactly 32 bytes (64 hex characters)
  const preHashed = crypto.createHash('sha256').update(plaintext, 'utf8').digest('hex');
  return bcrypt.hash(preHashed, rounds);
}

async function safeVerifyPassword(plaintext: string, hash: string): Promise<boolean> {
  const preHashed = crypto.createHash('sha256').update(plaintext, 'utf8').digest('hex');
  return bcrypt.compare(preHashed, hash);
}

5. Cost Factor Mathematics and Sizing Guidelines (2026)

The cost parameter $c$ in $2b$c$ represents an exponential work scale:

$$ ext{Iterations} = 2^c$$

Every time you increment the cost factor by 1, you double the CPU time required to compute or verify a password.

+------------------------------------------------------------------------------------+
|                         BCRYPT COST FACTOR BENCHMARK TABLE                         |
+------------------------------------------------------------------------------------+
| Cost Factor | Iterations | Typical Server Latency (vCPU) | Acceptability (2026)    |
+-------------+------------+-------------------------------+-------------------------+
| 8           | 256        | ~4 ms                         | Insecure (Legacy only)  |
| 10          | 1,024      | ~18 ms                        | Bare minimum            |
| 12          | 4,096      | ~75 ms                        | Recommended for APIs    |
| 13          | 8,192      | ~150 ms                       | High-security systems   |
| 14          | 16,384     | ~310 ms                       | Batch / Admin accounts  |
+------------------------------------------------------------------------------------+

The 250ms Rule for Authentication Endpoints

Target Latency: Aim for 100ms to 250ms per password verification on your production infrastructure.
Denial of Service (DoS) Caution: Never set cost factors higher than 14 on synchronous public endpoints without strict IP rate-limiting; otherwise, malicious actors can send bursts of concurrent invalid logins to starve server CPUs.

6. Bcrypt vs. Modern Password Hashers: When to Choose Argon2id

In 2015, the Password Hashing Competition (PHC) selected Argon2 as the next-generation global standard. In 2026, OWASP recommends Argon2id as the primary choice for greenfield systems.

Feature	Bcrypt (1999)	PBKDF2-HMAC-SHA256 (2000)	Argon2id (2015/RFC 9106)
Primary Bound	CPU + Small Cache (4KB)	Pure CPU Compute	Memory + CPU + Threads
GPU Resistance	Moderate	Very Low	Extreme (Configurable RAM)
Side-Channel Defense	Good	High	Immune (Argon2id hybrid)
Max Password Length	72 bytes	Arbitrary	Arbitrary
OWASP 2026 Rank	Secondary Recommended	Acceptable Legacy	Top Recommended

When to keep Bcrypt:

If your runtime environment (e.g. AWS Lambda, Edge Workers, legacy enterprise Node/Python/Java) lacks native C bindings for Argon2.
If you have existing user tables already hashed with $2b$ .

7. Practical Node.js & TypeScript Implementation

Below is a production-grade authentication utility with automatic cost migration:

import bcrypt from 'bcrypt';

const RECOMMENDED_ROUNDS = 12;

export interface PasswordVerificationResult {
  isValid: boolean;
  needsRehash: boolean;
}

/**
 * Hash a password with modern bcrypt rounds
 */
export async function hashPassword(plainPassword: string): Promise<string> {
  const salt = await bcrypt.genSalt(RECOMMENDED_ROUNDS, 'b');
  return bcrypt.hash(plainPassword, salt);
}

/**
 * Verify password and detect if the stored hash uses outdated rounds
 */
export async function verifyPassword(
  plainPassword: string,
  storedHash: string
): Promise<PasswordVerificationResult> {
  const isValid = await bcrypt.compare(plainPassword, storedHash);
  if (!isValid) {
    return { isValid: false, needsRehash: false };
  }

  // Extract cost from $2b$10$...
  const parts = storedHash.split('$');
  const currentRounds = parts.length > 2 ? parseInt(parts[2], 10) : 0;
  const needsRehash = currentRounds < RECOMMENDED_ROUNDS;

  return { isValid: true, needsRehash };
}

Summary & Checklist for Developers

Use $2b$ : Never generate $2a$ or raw $2$ hashes in new deployments.
Set Cost $\ge 12$: Test on your production hardware so that verification takes between 75ms and 250ms.
Guard Against the 72-Byte Truncation: Validate password byte lengths, or apply a pre-hashing step (SHA-256) for unrestricted length fields.
Leverage Transparent Re-Hashing: When users log in, inspect their stored rounds. If below the current security standard, seamlessly hash with updated rounds and persist to the database.

Originally published on DailyToolbox.org

Regex Cheat Sheet for Developers: The 20% of Syntax That Covers 80% of Real Work (2026)

tinali7564-eng — Sun, 11 Oct 2026 02:55:16 +0000

Regular expressions have a reputation problem. Show a developer a pattern like ^(?:[a-z0-9!#$%&'*+/=?^_{|}~-]+(?:.[a-z0-9!#$%&'+/=?^_`{|}~-]+)|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\[\x01-\x09\x0b\x0c\x0e-\x7f])")@(?:(?:a-z0-9?.)+a-z0-9?|[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?).){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-][a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\[\x01-\x09\x0b\x0c\x0e-\x7f])+)])$` and watch them close the tab. That is a real email-validation regex, and it is a perfect example of what regex should not look like in practice.

Here is the secret experienced developers know: you do not need to master all of regex. A small core — anchors, character classes, a handful of quantifiers, groups, and alternation — handles the overwhelming majority of real-world tasks: validating input, extracting data from logs, transforming text, and parsing semi-structured formats. This guide covers that core deeply, gives you production-ready patterns with honest caveats, and then covers the two pitfalls that actually cause outages: greedy-vs-lazy confusion and catastrophic backtracking.

One note on dialects: regex flavors differ (PCRE, JavaScript, Python re, Java, Go's RE2). The syntax in this guide is the common subset shared by virtually all of them. Where flavors diverge in ways that matter, I will call it out.

The Core 20%: The Only Syntax Table You Need

Everything below composes. Learn these ~20 constructs and you can read and write most regex you will ever encounter.

Anchors — Where the Match Happens

Pattern	Meaning
`^`	Start of string (or start of line in multiline mode)
`$`	End of string (or end of line in multiline mode)
`\b`	Word boundary — between a word char and a non-word char

Anchors do not consume characters; they assert position. ^abc means "the string starts with abc", while abc alone means "abc appears anywhere". Forgetting anchors is the single most common validation bug: /\d{3}/ matches "abc123def" — probably not what you wanted when validating a 3-digit code.

Character Classes — What Can Match Here

Pattern	Meaning
`.`	Any character except newline (use the DOTALL/`s` flag to include newlines)
`\d`	Digit — see the Unicode caveat below
`\D`	Non-digit
`\w`	Word character: letters, digits, underscore
`\W`	Non-word character
`\s`	Whitespace: space, tab, newline, and friends
`\S`	Non-whitespace
`[abc]`	Any one of a, b, c
`[^abc]`	Any one character except a, b, c
`[a-z]`	Any lowercase ASCII letter
`[0-9]`	Any ASCII digit — not always the same as `\d` (see below)

Quantifiers — How Many Times

Pattern	Meaning
`*`	Zero or more (greedy)
`+`	One or more (greedy)
`?`	Zero or one (greedy)
`{n}`	Exactly n
`{n,}`	n or more
`{n,m}`	Between n and m
`*?`, `+?`, `??`, `{n,m}?`	Lazy versions — match as few as possible

Groups and Alternation — Structure

Pattern	Meaning
`(abc)`	Capturing group — groups and captures
`(?:abc)`	Non-capturing group — groups without capturing
`a\	b`
`\1`	Backreference to group 1 (in most flavors)

That is the core. Everything else — lookaheads, named groups, flags — is useful but situational. If you fully internalize the table above, you are already ahead of most developers who use regex daily.

Reading a Pattern: Anatomy of a Real One

Let's dissect ^\d{3}-\d{3}-\d{4}$, a US phone number validator, piece by piece:

^          Start of string — the match must begin here
\d{3}      Exactly 3 digits
-          A literal hyphen
\d{3}      Exactly 3 digits
-          A literal hyphen
\d{4}      Exactly 4 digits
$          End of string — nothing may follow

Against "555-123-4567": every piece lines up, full match. Against "Call 555-123-4567 now": the ^ fails immediately because the string does not start with digits. Against "555-123-456": the final \d{4} needs four digits but the string ends — no match.

The discipline here is to read patterns as a sequence of constraints, left to right. When a pattern misbehaves, walk through it this way against your failing input and you will usually find the culprit within seconds.

Real-World Patterns, With Honest Caveats

Below are patterns you can copy into production — but each comes with caveats, because regex validation is always a tradeoff between strictness and practicality. I would rather give you a pattern with known limits than a "perfect" one that breaks on real data.

Email Addresses: Why "Perfect" Is a Trap

You may have seen the monstrous RFC 5322-compliant email regex (it is thousands of characters long and still does not cover everything). Do not use it. Email syntax is genuinely complicated — quoted strings, comments, IP-literal domains — and no regex handles all of it sanely.

The pragmatic approach used by most production systems:

/^[^\s@]+@[^\s@]+\.[^\s@]+$/

Read it as: one or more non-space, non-@ characters, then @, then one or more non-space, non-@ characters, then a literal dot, then one or more non-space, non-@ characters.

What it catches: not-an-email, missing@domain, @nodomain.com, spaces in@email.com.

What it allows through: a@b.c (technically a valid format), user@123.123.123.123 without brackets, internationalized addresses it cannot judge.

The honest caveat: this validates shape, not existence. The only way to truly validate an email address is to send an email to it and get a response. For signup flows, the industry-standard practice is: run a shape check like the one above, then send a verification email. Do not reject users with an over-strict regex — you will lose real signups to false negatives.

URLs

/^https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-zA-Z0-9()]{1,6}\b([-a-zA-Z0-9()@:%_\+.~#?&//=]*)$/

This matches http:// or https:// URLs with reasonable paths and query strings. It requires a scheme — if you also want to accept www.example.com without a scheme, make the scheme group optional: ^(https?:\/\/)?.

Caveats: it does not validate that the TLD actually exists, it is generous with unusual-but-legal characters, and internationalized domain names (in Punycode form, xn--...) will match as plain ASCII — which is correct behavior, since Punycode is ASCII.

Phone Numbers

/^\+?1?[-.\s]?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}$/

Matches US formats: 5551234567, 555-123-4567, (555) 123-4567, +1 555.123.4567.

The honest caveat: phone number formats vary wildly by country, and regex cannot know numbering plans. If your application handles international numbers seriously, use Google's libphonenumber (or a port) instead of regex. Regex is fine for a rough US-format sanity check on a form; it is not a phone validation library.

Dates (ISO 8601 Format Check)

/^\d{4}-\d{2}-\d{2}$/

Matches 2026-10-10. But note: it also matches 2026-02-30 and 2026-13-99, which are not real dates. Regex checks format, not validity. The correct architecture is: regex for a fast format pre-check, then your language's date parser (datetime.strptime, new Date() with validation, etc.) for real validation. Trying to encode month lengths and leap years into regex is possible but produces unreadable patterns that the next developer will fear touching.

IPv4 Addresses

The quick version:

/^(?:\d{1,3}\.){3}\d{1,3}$/

Matches the shape of an IPv4 address — but also matches 999.999.999.999, which is not a valid address. The strict version constrains each octet to 0–255:

/^(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(?:\.(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}$/

Each octet alternative reads as: 250–255, or 200–249, or 100–199, or 0–99. This is a good example of regex doing genuine numeric-range validation — ugly, but precise and fast.

Greedy vs Lazy: The #1 Gotcha

Quantifiers are greedy by default: they match as much as possible while still allowing the overall pattern to succeed. Appending ? makes them lazy: match as little as possible.

Consider extracting HTML tags from "bold and italic":

/<.*>/     greedy

The greedy .* first consumes the entire string, then backtracks just enough to let the final > match. Result: one match — "bold and italic". Probably not what you wanted.

/<.*?>/    lazy

The lazy .*? consumes as little as possible, expanding only until the next > allows a match. Result: two matches — "" and "", then "" and "".

When to use which:

Greedy (default) is right when you want the longest plausible match: "([^"]*)" to capture a quoted string, .*\.jpg$ for a filename ending.
Lazy is right when you want the shortest match up to a delimiter: <.*?> for individual tags, ".*?" for the first quoted string.

A useful mental model: greedy asks "how much can I eat?", lazy asks "how little can I get away with?" If your matches are mysteriously too long, you probably need lazy. If they are mysteriously too short or failing, check whether lazy is starving a later part of the pattern.

Catastrophic Backtracking: How a Regex Hangs Your App

This is the regex pitfall with real production consequences — including a class of denial-of-service attacks (ReDoS) where an attacker submits a crafted string that sends your server's CPU to 100%.

The Mechanism

Consider:

/^(a+)+$/

against the input "aaaaaaaaaaaaaaaaaaaaaaaaaaaaa!" (29 a's, then an exclamation mark).

The inner a+ can match the a's in many ways, and the outer (...)+ can group those matches in many ways. For n a's, there are roughly 2ⁿ ways to partition them between the inner and outer quantifiers. The regex engine tries them one by one. When the final $ fails (because of the !), the engine backtracks through every partition before giving up.

For 29 characters, that is on the order of 2²⁹ ≈ 500 million paths. Your regex does not return "no match" — it effectively never returns. Add a few more a's and you are measuring in hours.

Why This Matters Beyond Toy Examples

Nested quantifiers hide in innocent-looking validation patterns: ^(\w+[\.-]?)+@... in an email regex, ^([a-zA-Z0-9]+)*$ in a username check. An attacker who finds an endpoint validating input with such a pattern can hang a server thread with a ~30-character string. This is ReDoS — Regular Expression Denial of Service — and it has taken down production systems.

Fixes

Simplify nested quantifiers. ^(a+)+$ is just ^a+$ — the nesting adds nothing but ambiguity. Most catastrophic patterns collapse this way once you look.
Use possessive quantifiers or atomic groups where your flavor supports them: (?>a+) (atomic group, PCRE/Java) or a++ (possessive, PCRE/Java) prevent the engine from backtracking into the group at all.
Bound your input length before regex ever sees it. A 10,000-character cap on the input string turns exponential blowup into a bounded (if still slow) computation.
Set regex timeouts. .NET has RegexOptions timeouts; Python 3.11+ has no built-in timeout, but you can run matching in a thread with a deadline. Know your platform's story here.
Prefer RE2-style engines (Go, Rust's regex crate) for untrusted input — they guarantee linear-time matching by construction and simply do not support backreferences.

Pitfalls Checklist: The Bugs You Will Actually Hit

Unescaped dots. /example.com/ matches "exampleXcom". In domains, versions, and filenames you almost always want \..
Missing anchors. /\d{4}/ validates that somewhere in the input there are four digits. Validation almost always wants ^...$.
\d is not always [0-9]. In Python 3, \d on a str pattern matches Unicode decimal digits (e.g., Arabic-Indic ٠١٢٣) — usually harmless, occasionally a validation hole if downstream code expects ASCII. In JavaScript, \d is ASCII-only. Know your flavor.
. does not match newlines by default. Multiline input silently fails to match across lines. Enable DOTALL (s flag: /./s) when you need it — or use [\s\S] as a portable "any character including newline" idiom.
^ and $ in multiline mode. With the m flag, ^/$ match at line boundaries, not just string boundaries. /^\d+$/m matches a string containing any all-digit line. For whole-string validation in multiline mode, use \A and \z (or \Z) where supported.
Case sensitivity. /[a-z]/ does not match A. Either add the i flag or expand the class. For emails and hostnames, case-insensitive matching is usually correct.
Forgetting that | has low precedence. /^cat|dog$/ means (^cat)|(dog$) — not ^(cat|dog)$. Group your alternations.

How to Test Regex Like an Engineer

A regex without tests is a guess. Here is a lightweight discipline that catches most problems:

1. Write positive and negative cases before finalizing the pattern.

import re

pattern = re.compile(r"^[^\s@]+@[^\s@]+\.[^\s@]+$")

valid = ["user@example.com", "a.b+tag@sub.domain.co", "x@y.zz"]
invalid = ["not-an-email", "@nodomain.com", "user@domain", "user @example.com", ""]

for s in valid:
    assert pattern.match(s), f"should match: {s!r}"
for s in invalid:
    assert not pattern.match(s), f"should NOT match: {s!r}"
print("all cases pass")

2. Test the boundaries, not just the happy path. Empty string, single character, maximum length, Unicode input, newline injection ("user@example.com\ninjected"), and the exact strings your pattern is supposed to reject. Attackers live at boundaries.

3. Test catastrophic inputs. If your pattern will ever see untrusted input, throw long repetitive strings at it ("a"*100 + "!") and time the match. If it does not return in milliseconds, you have a ReDoS risk — go fix the pattern before shipping.

4. Use a visual tester during development. Stepping through a match — seeing which group captured what, where backtracking happens — builds intuition far faster than staring at the pattern. It is also the fastest way to debug a pattern someone else wrote.

5. Pin the flavor. A pattern tested in a JavaScript tester may behave differently in Python (\d Unicode!) or Go (no backreferences). Test in the environment that will run it.

Try It

Theory is useful; reps build the skill. DailyToolbox's free regex tester runs entirely in your browser — paste a pattern, type test strings, and watch matches highlight live with capture groups broken out. Try the email pattern from this article against the tricky cases (user@domain, a@b.c, "quoted"@example.com), flip the greedy/lazy toggle on <.*> vs <.*?>, and feed (a+)+$ a long string of a's to watch backtracking happen in a safe sandbox instead of your production server.

👉 Try the Regex Tester →

Originally published on DailyToolbox.org

Base64 Encoding Explained: The 6-Bit Math, Padding Rules, and When Not to Use It (2026)

tinali7564-eng — Sat, 10 Oct 2026 10:43:00 +0000

Every developer has encountered a wall of gibberish like iVBORw0KGgoAAAANSUhEUgAA... pasted into source code, a config file, or an API response. That is Base64 — one of the most widely used and most misunderstood encodings in software. It shows up in email attachments, JWT tokens, data URIs, API keys, and countless config files, yet most developers treat it as a black box: bytes go in, ASCII comes out.

This article opens the box. You will learn exactly how the 6-bit grouping math works (with bit-level worked examples), why the = padding exists, how the URL-safe variant differs, where Base64 legitimately belongs in your architecture — and the critical ways it gets misused, including the dangerously common mistake of treating it as encryption.

Why Base64 Exists: Binary Data Meets Text-Only Channels

The problem Base64 solves is old and fundamental: many systems that move data around were designed for text, not binary.

The original motivation was email. SMTP, the protocol that still carries email today, was built for 7-bit ASCII text. If you tried to send a binary file — a photo, an executable, a PDF — raw bytes would pass through mail relays that might strip the high bit, interpret control characters, or mangle line endings. The file would arrive corrupted.

MIME (Multipurpose Internet Mail Extensions, RFC 2045) solved this by defining content transfer encodings: standard ways to represent arbitrary binary data using only safe printable characters. Base64 (RFC 4648) became the workhorse encoding for this job.

The same constraint appears constantly in modern systems:

JSON is a text format. It has no native binary type, so APIs that need to return a file, an image, or a cryptographic signature inside JSON must encode the bytes as text first.
URLs and filenames can only safely contain a limited character set. Raw binary would break parsing.
HTML and CSS are text. Embedding a small image directly in a stylesheet requires a text representation of its bytes.
HTTP headers (like Authorization: Basic ...) are text-only.

Base64's answer to all of these: take any sequence of bytes and represent it using exactly 64 safe ASCII characters. The output is slightly larger than the input (about 33% overhead), but it survives every text-only channel intact.

How It Works: The 6-Bit Grouping

Here is the core idea in one sentence: Base64 re-slices a byte stream from 8-bit groups into 6-bit groups, then maps each 6-bit value (0–63) to a printable character.

Why 6 bits? Because 2⁶ = 64, and 64 distinct characters fit comfortably in printable ASCII. Each output character carries exactly 6 bits of the original data.

The Alphabet

The standard Base64 alphabet (RFC 4648 §4) is:

Value	Char	Value	Char	Value	Char	Value	Char
0	A	16	Q	32	g	48	w
1	B	17	R	33	h	49	x
2	C	18	S	34	i	50	y
3	D	19	T	35	j	51	z
4	E	20	U	36	k	52	0
5	F	21	V	37	l	53	1
6	G	22	W	38	m	54	2
7	H	23	X	39	n	55	3
8	I	24	Y	40	o	56	4
9	J	25	Z	41	p	57	5
10	K	26	a	42	q	58	6
11	L	27	b	43	r	59	7
12	M	28	c	44	s	60	8
13	N	29	d	45	t	61	9
14	O	30	e	46	u	62	+
15	P	31	f	47	v	63	/

So A–Z cover values 0–25, a–z cover 26–51, 0–9 cover 52–61, and + and / cover 62 and 63.

Worked Example: "Man" → "TWFu"

Let's encode the 3-byte string Man step by step. This is the canonical example from RFC 4648 itself.

Step 1: Convert each byte to 8 bits.

Char	Decimal	Binary
M	77	`01001101`
a	97	`01100001`
n	110	`01101110`

Step 2: Concatenate into a 24-bit stream.

01001101 01100001 01101110

Step 3: Re-slice into four 6-bit groups.

010011 | 010110 | 000101 | 101110

Step 4: Convert each group to decimal, then look up the character.

6-bit group	Decimal	Character
`010011`	19	T
`010110`	22	W
`000101`	5	F
`101110`	46	u

Result: TWFu.

Notice why 3 bytes is the natural input unit: 3 bytes = 24 bits, and 24 is the least common multiple of 8 and 6. Three 8-bit bytes divide evenly into four 6-bit groups with nothing left over. The encoder processes input in 3-byte chunks, emitting 4 characters per chunk.

The 33% Overhead, Explained

Because 3 bytes become 4 characters, Base64 output is always 4/3 the size of the input (for inputs whose length is a multiple of 3). That is a 33.3% size overhead — a hard mathematical floor, not an implementation detail. A 3 MB image becomes a 4 MB string. This matters enormously when deciding where Base64 is appropriate, as we will see in the data URI section.

Padding: Why the `=` Exists

What happens when the input length is not a multiple of 3? The encoder cannot form a complete final group of 24 bits, so it pads the last group with zero bits — and then marks how many padding characters were added using =.

There are exactly two padding cases:

Input length mod 3	Remaining bytes	Output chars	Padding
0	3 (full chunk)	4	none
2	2	3 + `=`	`=`
1	1	2 + `==`	`==`

Worked Example: "Ma" → "TWE="

Two bytes remain: M (01001101) and a (01100001). Concatenated: 16 bits.

010011 | 010110 | 0001

The third group has only 4 bits (0001), so it is padded with two zero bits to make 000100 = 4 = E. We emit three real characters and one =:

TWE=

Worked Example: "M" → "TQ=="

One byte remains: M (01001101). Just 8 bits.

010011 | 01

The second group has only 2 bits (01), padded with four zero bits to make 010000 = 16 = Q. We emit two real characters and two = signs:

TQ==

Why Padding Matters

Padding serves two purposes. First, it tells the decoder exactly how many real bytes the final quantum contains — without it, a decoder cannot distinguish trailing zero bits from real data. Second, it keeps every Base64 output a multiple of 4 characters, which simplifies streaming decoders and length validation: if a supposedly-Base64 string's length is not divisible by 4, something is wrong.

One caveat: some variants (notably Base64URL as used in JWTs) deliberately omit padding. Decoders for those variants must infer the padding from the string length instead. When you write a decoder, always know which convention your input follows.

Base64URL: The URL-Safe Variant

Standard Base64 has a problem in URLs, filenames, and some protocols: the characters + and / are reserved. In a URL query string, + means "space"; / separates path segments. A standard Base64 string dropped into a URL would be misinterpreted.

RFC 4648 §5 defines base64url, which makes two changes:

Position	Standard	URL-safe
Value 62	`+`	`-`
Value 63	`/`	`_`
Padding	`=` required	usually omitted

So the same bytes encode slightly differently:

Standard:  a+b/c==
Base64URL: a-b_c      (padding stripped)

You will encounter base64url everywhere tokens travel in URLs: JWTs, OAuth code_challenge values in PKCE flows, and URL-safe identifiers. When converting between the two, remember it is a pure character substitution (plus padding handling) — the underlying bits are identical.

A common bug: feeding a base64url string (with - and _) into a standard Base64 decoder. Most decoders reject it outright; a few silently produce garbage. If your decoder chokes on a JWT segment, check the alphabet first.

Critical: Base64 Is Not Encryption

This is the single most important section of this article, because this is where Base64 causes real security incidents.

Base64 provides zero confidentiality. It is an encoding, like hexadecimal or URL-encoding — a reversible transformation with no key, no secret, and no security properties whatsoever. Anyone who recognizes Base64 (and every developer does) can decode it in under a second, using tools built into every operating system:

# "Decrypting" Base64 takes one command. There is no key.
$ echo "cGFzc3dvcmQxMjM=" | base64 -d
password123

The string cGFzc3dvcmQxMjM= looks opaque. It is the literal text password123. There is no step between "looks secret" and "is public" other than running a decoder.

How This Goes Wrong in Practice

"Encrypted" API tokens that are just Base64-encoded JSON. Anyone who decodes them sees user IDs, roles, and permissions in plaintext.
Obfuscated credentials in config files or mobile apps. Base64-encoding a hardcoded API key does not protect it from anyone who downloads the app — it only protects it from people who cannot recognize Base64.
"Secure" URL parameters carrying Base64-encoded user data. They are readable by every proxy, log, and browser history entry between the user and your server.

The rule is simple: if you would not print it on a billboard, do not Base64-encode it and call it protected. Encoding is not encryption.

What to use instead depends on the actual goal:

Goal	Use this, not Base64-alone
Keep data secret in transit	TLS (which you get from HTTPS)
Keep data secret at rest	Real encryption, e.g. AES-256-GCM with a proper key
Store passwords	A password-hashing function (bcrypt, scrypt, or Argon2) — never reversible
Prove data integrity	HMAC or a digital signature

Note that Base64 often appears alongside real cryptography — encrypted ciphertext is binary, so it is routinely Base64-encoded for storage in JSON or databases. That is a legitimate use: Base64 is the packaging, AES is the protection. Confusing the two is the error.

Data URIs: Embedding Binary in Text

One of the most visible uses of Base64 is the data URI scheme (RFC 2397), which lets you embed small binary resources directly in HTML or CSS:

data:[<mediatype>][;base64],<data>

A real example — a 1×1 transparent PNG embedded in CSS:

.icon {
  background-image: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==");
}

No separate HTTP request, no extra file to deploy. For a tiny icon, this is genuinely convenient.

When Data URIs Win — and When They Lose

They win for very small assets (icons, tiny patterns): you eliminate an HTTP round trip, which on high-latency connections can matter more than the 33% size penalty. They are also useful for single-file HTML documents (email templates, standalone reports) where external assets are not an option.

They lose as assets grow, for several compounding reasons:

The 33% bloat is real. A 100 KB image becomes ~133 KB of text inside your HTML or CSS.
It blocks parsing. Base64 in your HTML must be downloaded and parsed before the page renders; an <img> tag, by contrast, loads asynchronously.
No separate caching. Change one byte of your HTML and the browser re-downloads the embedded image too. A separate file would have been cached.
No progressive rendering or responsive variants. You cannot srcset a data URI easily, and the image cannot display until fully downloaded.

As a heuristic many frontend engineers use: data URIs for assets under a few kilobytes; real files (ideally modern formats like WebP or AVIF) for everything else. The exact threshold depends on your performance budget — measure, do not guess.

JWT: Base64URL in the Wild

JSON Web Tokens (RFC 7519) are the most prominent real-world consumer of base64url. A JWT has three parts separated by dots:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9   ← header (base64url, no padding)
.
eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ   ← payload
.
SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c   ← signature

Decode the header segment and you get:

{"alg":"HS256","typ":"JWT"}

Two things worth internalizing:

The payload is not encrypted. It is base64url-encoded and signed. Anyone who holds the token can decode the payload with zero keys — try pasting any JWT into a decoder. Never put secrets, passwords, or sensitive PII in a JWT payload. The signature proves the token was not tampered with; it does not hide its contents.
Verify the signature, and verify the algorithm. Early in JWT's history, several libraries accepted tokens with "alg":"none" — meaning "this token is unsigned, trust it anyway" — which allowed trivial token forgery. Modern libraries reject none by default, but the lesson generalizes: when you validate a JWT, pin the expected algorithm explicitly rather than trusting the token's own alg header.

When to Use Base64 — and When Not To

✅ Legitimate uses	❌ Misuses
Email attachments (MIME `Content-Transfer-Encoding: base64`)	"Encrypting" or "hiding" secrets
Binary data inside JSON/XML APIs	Compressing data (it expands data by 33%)
Data URIs for tiny assets (icons)	Large images or files in HTML
HTTP Basic auth credentials (`Basic dXNlcjpwYXNz`)	Storing blobs in a database text column (use a binary/BLOB column)
Packaging ciphertext or signatures as text	Replacing proper encryption
Embedding fonts in CSS (`@font-face` with small woff2)	Anything where the 33% overhead matters and alternatives exist

A useful mental model: Base64 is a packaging format, not a processing step. It answers "how do I carry these bytes through a text-only channel?" If your problem is not "bytes must survive a text channel," you probably do not need Base64.

Try It

Reading about the 6-bit grouping is one thing; watching it happen is better. DailyToolbox's free Base64 encoder/decoder runs entirely in your browser — paste text, see the Base64 output instantly, flip to decode mode, and try the worked examples from this article (Man → TWFu, Ma → TWE=) to verify them yourself. It also handles the URL-safe variant, so you can convert between standard and base64url alphabets without writing a script.

👉 Try the Base64 Encoder/Decoder →

Originally published on DailyToolbox.org

Value	Char	Value	Char	Value	Char	Value	Char
0	A	16	Q	32	g	48	w
1	B	17	R	33	h	49	x
2	C	18	S	34	i	50	y
3	D	19	T	35	j	51	z
4	E	20	U	36	k	52	0
5	F	21	V	37	l	53	1
6	G	22	W	38	m	54	2
7	H	23	X	39	n	55	3
8	I	24	Y	40	o	56	4
9	J	25	Z	41	p	57	5
10	K	26	a	42	q	58	6
11	L	27	b	43	r	59	7
12	M	28	c	44	s	60	8
13	N	29	d	45	t	61	9
14	O	30	e	46	u	62	+
15	P	31	f	47	v	63	/

Value	Char	Value	Char	Value	Char	Value	Char
0	A	16	Q	32	g	48	w
1	B	17	R	33	h	49	x
2	C	18	S	34	i	50	y
3	D	19	T	35	j	51	z
4	E	20	U	36	k	52	0
5	F	21	V	37	l	53	1
6	G	22	W	38	m	54	2
7	H	23	X	39	n	55	3
8	I	24	Y	40	o	56	4
9	J	25	Z	41	p	57	5
10	K	26	a	42	q	58	6
11	L	27	b	43	r	59	7
12	M	28	c	44	s	60	8
13	N	29	d	45	t	61	9
14	O	30	e	46	u	62	+
15	P	31	f	47	v	63	/

DEV Community: tinali7564-eng

The Definitive Guide to JWT Security in 2026: HS256 vs RS256 vs ES256, The alg: none Exploit, kid Parameter Attacks, and Token

The Definitive Guide to JWT Security in 2026: HS256 vs RS256 vs ES256, The alg: none Exploit, kid Parameter Attacks, and Token Rotation Architecture

1. Deconstructing the 3-Segment Anatomy of a JWT

1. The Header: Metadata & Cryptographic Instructions

2. The Payload: Registered & Custom Claims

3. The Signature: Cryptographic Integrity Proof

2. Cryptographic Algorithm Matrix: HS256 vs. RS256 vs. ES256 vs. Ed25519

Why Asymmetric Cryptography Wins in Distributed Architectures

3. The 5 Major Attack Vectors (and How Attackers Exploit Them)

Attack Vector 1: The alg: none Vulnerability

Attack Vector 2: The Algorithm Confusion Attack (RS256 to HS256)

Attack Vector 3: The kid (Key ID) Parameter Injection

Directory Path Traversal

SQL Injection via kid

Attack Vector 4: Weak HMAC Secrets & Offline Brute-Forcing

Attack Vector 5: Storage Vulnerabilities (LocalStorage vs. HttpOnly Cookies)

4. Production Architecture: Refresh Token Rotation (RTR)

The Magic of Automatic Reuse Detection

5. Zero-Dependency TypeScript JWT Decoder & Expire Auditor

6. Production Security Checklist for JWTs

Try It in Your Browser

FAQ

Why should I choose ES256 (ECDSA) over RS256 (RSA)?

Can a user revoke their own JWT without database lookups?

What is the difference between JWS, JWE, and JWT?

Unix Timestamp and Epoch Time Explained: 10-Digit vs 13-Digit Precision, Leap Second Smearing, Timezone Traps, and the Year

Unix Timestamp and Epoch Time Explained: 10-Digit vs 13-Digit Precision, Leap Second Smearing, Timezone Traps, and the Year 2038 Problem (2026)

1. What Is Unix Time? The 1970-01-01T00:00:00Z Origin Story

2. The Digit Count Hierarchy: 10 vs. 13 vs. 16 vs. 19 Digits

The Silent "Year 58,742" Multiplication Bug

Production Normalization Function (TypeScript)

3. Leap Seconds: Why POSIX Time Breaks Astronomy (and How Big Tech Fixes It)

The POSIX Flaw

The Solution: NTP Leap Smearing

4. Timezone Storage in Databases: PostgreSQL and MySQL Best Practices

The PostgreSQL Distinction: TIMESTAMPTZ vs. TIMESTAMP

5. The Year 2038 Problem (Y2038 / Epochalypse)

The 32-Bit Signed Integer Ceiling

The Exact Moment of Overflow

Impact and Remediation Roadmap

6. Copy-Paste Code Patterns Across Languages

JavaScript / TypeScript (Node.js & Browser)

Python 3

PostgreSQL

7. Summary & Best Practices Checklist

Try It in Your Browser

FAQ

Why did Unix time choose UTC instead of TAI (International Atomic Time)?

Can a Unix timestamp be negative?

What is the difference between Unix Epoch and Windows Epoch?

Why do some systems show December 31, 1969 instead of January 1, 1970?

How Cron Expressions Really Work: The 5-Field vs 6-Field Syntax, Day-of-Week Traps (Sunday 0 vs 7), Daylight Saving Time Skips

How Cron Expressions Really Work: The 5-Field vs 6-Field Syntax, Day-of-Week Traps (Sunday 0 vs 7), Daylight Saving Time Skips, and Production Failure Modes (2026)

1. The Architectural Divergence: POSIX 5-Field vs. Quartz 6/7-Field Schedulers

Standard POSIX / Vixie Cron (5 Fields)

Enterprise / Quartz Cron (6 or 7 Fields)

Comparative Architecture Matrix

2. Decoding Special Characters: *, /, -, ,, ?, L, W, #

1. The Asterisk (*)

2. The Comma (,)

3. The Hyphen (-)

4. The Slash (/) — Step Intervals

5. The Question Mark (?) — The Quartz/AWS Disambiguator

6. The L Character — "Last"

7. The W Character — Nearest Weekday (Monday–Friday)

8. The # Character — Nth Weekday of the Month

3. The Day-of-the-Week Incompatibility Matrix

Why This Triggers Outages

4. The Daylight Saving Time (DST) Nightmare

How to Immunize Your Architecture Against DST

5. The Overlapping Execution Trap (Concurrency Control)

Production Concurrency Solutions

1. Linux Local File Locking (flock)

2. Kubernetes concurrencyPolicy

3. Distributed Database Advisory Locks (PostgreSQL)

6. Production Cron Patterns: 15 Copy-Paste Schedules

7. Zero-Dependency TypeScript Cron Evaluator & Next-Run Estimator

8. Summary & Interactive Verification

Try It in Your Browser

Attack Vector 1: The `alg: none` Vulnerability

Attack Vector 3: The `kid` (Key ID) Parameter Injection

SQL Injection via `kid`

The PostgreSQL Distinction: `TIMESTAMPTZ` vs. `TIMESTAMP`

2. Decoding Special Characters: `*`, `/`, `-`, `,`, `?`, `L`, `W`, `#`

1. The Asterisk (`*`)

2. The Comma (`,`)

3. The Hyphen (`-`)

4. The Slash (`/`) — Step Intervals

5. The Question Mark (`?`) — The Quartz/AWS Disambiguator

6. The `L` Character — "Last"

7. The `W` Character — Nearest Weekday (Monday–Friday)

8. The `#` Character — Nth Weekday of the Month

1. Linux Local File Locking (`flock`)

2. Kubernetes `concurrencyPolicy`

What is the difference between `* * * * ` and `/1 * * * *`?

The `<picture>\` Element for Format Switching

Value	Char	Value	Char	Value	Char	Value	Char
0	A	16	Q	32	g	48	w
1	B	17	R	33	h	49	x
2	C	18	S	34	i	50	y
3	D	19	T	35	j	51	z
4	E	20	U	36	k	52	0
5	F	21	V	37	l	53	1
6	G	22	W	38	m	54	2
7	H	23	X	39	n	55	3
8	I	24	Y	40	o	56	4
9	J	25	Z	41	p	57	5
10	K	26	a	42	q	58	6
11	L	27	b	43	r	59	7
12	M	28	c	44	s	60	8
13	N	29	d	45	t	61	9
14	O	30	e	46	u	62	+
15	P	31	f	47	v	63	/