DEV Community: Max

yfinance 1.6.0: Practical Python Stock Data, Live Streaming, and Equity Screening

Max — Thu, 27 Aug 2026 17:01:25 +0000

I pulled yfinance stock data for the first time in 2021, back when the library was still fighting Yahoo's API changes every few months. In 2026, yfinance 1.6.0 landed on PyPI, and the gap between what it could do then and what it can do now is significant enough to revisit. This is not a documentation mirror — these are notes from working with it on a personal portfolio tracker and a couple of backtesting scripts.

What Changed Worth Knowing About

The most practically useful additions in recent releases are the live data components and the screening API. The WebSocket and AsyncWebSocket classes now expose real-time quote streaming. The Screener and EquityQuery objects let you build structured queries to filter equities without leaving Python. The Market class gives you status and session info for an exchange. Together these move yfinance closer to a self-contained data layer for personal projects.

One important caveat the PyPI page spells out: yfinance is an open-source tool using Yahoo's publicly available APIs, not affiliated with or endorsed by Yahoo, and their terms say the API is intended for personal use only. If you are building anything commercial, resolve that before writing a line of code.

pip install yfinance

The default install now includes curl_cffi as a request fallback. If that is a problem in some corporate proxies, older OS images, or constrained containers, the docs cover an alternative install path.

Fetching Historical Data Without Shooting Yourself in the Foot

The Ticker object is the entry point for single-instrument data:

import yfinance as yf

msft = yf.Ticker("MSFT")

# Last 3 months of daily data
hist = msft.history(period="3mo")
print(hist.tail())

# Specific date range
hist2 = msft.history(start="2026-01-01", end="2026-08-01")
print(hist2.shape)

history() returns a pandas DataFrame with Open, High, Low, Close, Volume, Dividends, and Stock Splits columns. The index is a timezone-aware DatetimeIndex — that trips up new users who compare it against naive datetime objects. Normalize your date comparisons or the filtering silently returns wrong results.

One thing I missed longer than I should have: history() auto-adjusts for splits and dividends by default. If you want raw unadjusted prices — say, when cross-referencing a broker's records — pass auto_adjust=False. The default is almost always right for portfolio math, but it matters when you are debugging discrepancies against another source.

For multiple tickers at once, use yf.download():

tickers = ["AAPL", "GOOGL", "BRK-B", "VTI"]
data = yf.download(tickers, period="1y", group_by="ticker")

The result has a MultiIndex column structure. If you only need closing prices, data["Close"] gives a clean DataFrame with one column per ticker — the shape most backtesting libraries expect.

Live Streaming and Where It Actually Helps

The WebSocket class is genuinely new territory. For personal projects that want live quote updates without paying for a proper market data feed, it is a workable starting point:

import yfinance as yf

ws = yf.WebSocket(["AAPL", "MSFT"])
ws.start()

for quote in ws.stream():
    print(quote)
    if some_condition:
        break

ws.stop()

Practical notes: the stream reflects Yahoo's quote-feed latency and hours. During pre/post-market the gaps between ticks can be longer than in regular session — do not bake hard latency assumptions into logic that processes this stream. Also test reconnection behavior before depending on it; a dropped connection mid-session should resume gracefully, but verify that on your network first.

The AsyncWebSocket variant is better if you are integrating into an async app — alongside FastAPI, an async queue, or asyncio orchestration it avoids threading headaches.

Screening Equities with EquityQuery

The EquityQuery + Screener combo is the most underused feature I see people miss. Instead of downloading a universe and filtering in pandas, push the filter criteria upstream:

from yfinance import EquityQuery, Screener

# Stocks with market cap over $10B in technology sector
q = EquityQuery("and", [
    EquityQuery("gt", ["marketcap", 10_000_000_000]),
    EquityQuery("eq", ["sector", "Technology"])
])

screener = Screener()
results = screener.set_predefined_body(q).fetch()
print(results["quotes"][:5])

The API mirrors Yahoo's internal screener query structure. Some fields behave differently than their names suggest — test on a small query first and validate the output shape before building business logic on top.

For backtesting infrastructure I run a weekly screener pull into a local SQLite database, then do all historical analysis offline. That keeps you out of rate-limit territory and gives reproducible inputs for strategy tests.

What It Cannot Do

yfinance does not give you tick data, Level 2 order-book depth, or historical intraday beyond what Yahoo's API exposes. For most personal finance projects and simple backtests, that is fine. Anything requiring precise intraday execution modeling needs a real market data vendor.

Rate limiting is worth understanding too. Yahoo publishes no official limits, and observed behavior varies with region, time of day, and residential vs cloud IP. If you fetch in bulk — five years of daily history for a thousand symbols — add delays and handle the occasional 429 or empty response gracefully:

import time
import yfinance as yf

def fetch_with_retry(ticker, retries=3, delay=2):
    for attempt in range(retries):
        try:
            t = yf.Ticker(ticker)
            data = t.history(period="1y")
            if not data.empty:
                return data
        except Exception as e:
            print(f"Attempt {attempt+1} failed: {e}")
            time.sleep(delay * (2 ** attempt))
    return None

A Practical Starting Project

If you want a concrete thing to build rather than experimenting in a notebook, try a weekly portfolio snapshot script. Once a week it downloads the last 52 weeks of price history for every position you hold, computes each position's percentage return against the index of your choice, writes to CSV, and optionally pushes a summary to a bot or local dashboard.

That forces you to handle the MultiIndex DataFrame, deal with corporate actions (splits and dividends) correctly, manage missing trading days across different exchanges, and think about where you store the output. Those four problems cover most of what you hit in more complex work.

yfinance 1.6.0 is a capable free data layer for personal finance work. It has real limits — not a production feed, personal-use terms — but within those limits it has grown into something genuinely useful. The screening API and live streaming components in particular are worth building time into if you have only ever used the historical-data path.

What are you pulling market data with these days — yfinance, a paid vendor, or something you rolled yourself?

I stopped uploading client images to compressors — here's the browser-only workflow I use now

Max — Mon, 24 Aug 2026 15:02:05 +0000

A while back I was prepping images for a client site. Some of the photos were under NDA — product shots that weren't public yet. And I caught myself doing what I'd done a hundred times: dragging them into an online compressor to shrink them before upload.

Then it hit me. I was uploading confidential client images to a random third-party server, just to save a few hundred KB. I had no idea where those files went, how long they were kept, or who could see them.

That's a bad habit, and if you build sites for clients, you probably have it too.

Why the usual tools are a quiet risk

Most "compress images online" tools upload your file to their server, process it there, and hand it back. That's fine for a meme. It is not fine for:

Client photos under NDA
Internal product screenshots
Anything with people's faces you don't have distribution rights to
Medical, legal, or financial documents saved as images

The compression is convenient. The upload is the problem.

The fix: compress in the browser, upload nothing

Modern browsers can resize and re-encode images entirely on your machine with the Canvas API. The file never leaves your laptop. You can prove it: open DevTools → Network tab, compress an image, and watch — zero requests go out.

Here's the minimal version of what's happening under the hood:

async function compress(file, quality = 0.8, maxWidth = 1920) {
  const bitmap = await createImageBitmap(file);
  const scale = Math.min(1, maxWidth / bitmap.width);
  const canvas = document.createElement('canvas');
  canvas.width = bitmap.width * scale;
  canvas.height = bitmap.height * scale;
  canvas.getContext('2d').drawImage(bitmap, 0, 0, canvas.width, canvas.height);
  return new Promise(res =>
    canvas.toBlob(res, 'image/webp', quality) // WebP: usually 60-80% smaller
  );
}

That's the whole trick. createImageBitmap decodes, canvas resizes, toBlob re-encodes to WebP/JPEG. No backend. No upload. It works offline.

If you don't want to wire it up yourself

I turned this into a small tool called QuickShrink — it's the browser-only version of TinyPNG, with presets for web/social/email/print, PNG→WebP conversion, and resize. Everything runs client-side; it's a PWA so you can install it and use it on a plane. Free, no account.

Tool: https://quickshrink.orthogonal.info/?ref=devto-nda

But honestly, whether you use my tool, Squoosh, or the 12 lines above — the point is the same: for anything confidential, compress locally. The upload is the part you should be paranoid about, not the compression.

The rule I follow now

Public/throwaway image → whatever tool is fastest is fine
Client, NDA, faces, or docs → browser-only, upload nothing

Takes the same amount of time. Removes an entire category of "wait, where did that file go?" risk.

What's your workflow for prepping client images? Still uploading, or local-only?

A Matching Checksum Doesn't Prove What You Think: Integrity vs Authenticity

Max — Thu, 20 Aug 2026 17:03:34 +0000

Last week I downloaded a CLI release and saw the familiar pair of links: a binary and its SHA256SUMS file. I hashed the binary, the digest matched, and I felt safe for about five seconds. Then I noticed the binary and its checksum came from the same server.

If that server were compromised, an attacker could have swapped both. The hash would still match perfectly. It just would not answer the question I actually cared about: did this file come from the real publisher?

That gap — integrity vs authenticity — trips up a lot of otherwise careful developers. Here's how I reason about it.

What a matching hash actually proves

A cryptographic hash turns any number of bytes into a fixed-length fingerprint. SHA-256 always returns 256 bits, shown as 64 hex characters. Flip one bit of input and the output changes unpredictably.

So when your computed SHA-256 equals the value a publisher lists, you have strong evidence that your local file is byte-for-byte identical to the file the publisher hashed. That catches:

an incomplete or resumed download
disk / transfer corruption
a broken or stale mirror
an accidental artifact replacement

That's real and useful. It is also the entire claim. A match says two byte sequences agree. It says nothing about who produced those bytes.

Integrity is not authenticity

Picture a release page serving tool.tar.gz and SHA256SUMS from the same origin. An attacker who controls that origin uploads a backdoored archive, recomputes its SHA-256, and overwrites the checksum file. Your comparison passes — because both malicious artifacts agree with each other.

HTTPS doesn't save you here. TLS protects the connection between your browser and the server. It says nothing about whether the server, build pipeline, maintainer account, or artifact was trustworthy before that connection began.

A checksum earns its keep when the reference value arrives over an independent, authenticated path: a signed checksum manifest, a package manager with signed metadata, a maintainer's verified channel, or a separately-controlled domain. The independence matters far more than the visual length of the hash.

MD5 and SHA-1 need careful language

MD5 and SHA-1 are broken for collision resistance. In 2017, the SHAttered work (Google + CWI Amsterdam) produced two visibly different PDFs with the same SHA-1 digest. MD5 fell years earlier.

But be precise about what that means, because people overstate it:

A collision = an attacker crafts two inputs that share a hash (they control both).
A second-preimage attack = an attacker takes your existing file and manufactures a different malicious file with the same digest. That's a harder, separate problem.

Collisions alone are still enough to make MD5/SHA-1 unacceptable whenever an adversary can shape both artifacts (chosen-prefix collisions). Keep MD5 around only for identifying old files or matching a legacy checksum — never as proof against an attacker. SHA-256 is the sensible default; no practical SHA-256 collision is known.

The download check I actually run

Download the artifact from the publisher's HTTPS release page.
Find the publisher's SHA-256 value — prefer a signed checksum manifest or an independent official channel, not the same box that served the binary.
Compute the digest locally.
Compare the full 64-char digest, not the first/last few characters.
If the project offers a signature, verify it as a separate step. A matching checksum does not replace signature verification.

For repeatable builds, pin the expected value and fail closed:

# macOS
shasum -a 256 tool.tar.gz

# Linux
sha256sum tool.tar.gz

# PowerShell
Get-FileHash .\tool.zip -Algorithm SHA256

Never paste a shortened digest into CI. A full SHA-256 is only 64 characters; truncating it deliberately throws away collision resistance. Store the expected value in reviewed source control, or consume a signed manifest.

When a signature is the real answer

A digital signature binds a digest to a private key, and verification proves the signer controlled that key. That's the authenticity property a bare checksum can't give you (GPG, platform code signing, or Sigstore in modern projects).

The catch just moves: how do you trust the public key or identity? A signature from an unknown key is worthless. Look for a fingerprint published on the project's established site, a verified maintainer identity, a package ecosystem's trust root, or Sigstore's identity + transparency-log checks.

(And HMAC is a different thing again — it authenticates data between parties sharing a secret, which is why it's great for webhook signatures but useless for public downloads: you can't safely hand every visitor the secret.)

A five-second threat model

Risk	Does a plain SHA-256 comparison help?	Better control
Accidental corruption	Yes	Checksum comparison
Broken / stale mirror	Yes, with an independent reference	Official checksum
Network tampering	Somewhat	HTTPS + independent checksum
Compromised download server	No, if it hosts both files	Signed release manifest
Malicious publisher / stolen signing key	No	Reproducible builds, transparency logs, key revocation

What I trust in practice

Low-risk utility: HTTPS + a SHA-256 from the official release page is fine.
Installer, firmware, wallet, security tool, production dependency: I want a signature or signed package metadata too.
High-impact infrastructure: reproducible builds or a transparency log on top.

One more privacy note: if the artifact is proprietary, contains customer data, or comes from an internal build, don't upload it to a random online checksum site just to answer a local math question — that creates a new disclosure risk. I keep integrity checks browser-only for exactly that reason: I built HashForge as a client-side hasher that runs crypto.subtle.digest() in your own browser (SHA-1/256/384/512, plus a local JS MD5 for legacy matching) so the file never leaves your machine. Use it when you need a fast manual check; use sha256sum + a signed manifest when a build must fail closed.

I wrote up the longer version — including the reproducible-builds and Sigstore angles — here.

Keep the claim precise: a matching hash proves two byte sequences agree. It does not tell you who created those bytes.

What's the worst "the checksum matched, so it's safe" assumption you've seen ship — and did anything catch it before prod?

Reading a JWT Offline: How to Spot alg:none and Algorithm Confusion Before They Bite

Max — Thu, 06 Aug 2026 17:03:13 +0000

A pentester friend sent me a JWT last month with a one-line note: "spot the bug in 10 seconds." I pasted the three segments into a decoder, flipped on URL-safe decoding, and read the header. The alg field said none. That token had no signature at all, and the backend was accepting it. Ten seconds, exactly.

Most JWT bugs aren't cryptographic. They're the kind you catch by just reading the token — if you can decode it without shipping it to a random website first. Here's how I read tokens offline and the three things I look for every time.

A JWT is three Base64url blobs, not encryption

People treat JWTs like ciphertext. They're not. A JWT is three chunks joined by dots:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NSIsInJvbGUiOiJ1c2VyIn0.dBjftJeZ4CVP-mB92K27uhbUJU1p1r_wW1gFWFOEjXk
[----- header -----].[--------- payload ---------].[------------- signature -------------]

The header and payload are plain Base64url-encoded JSON. Anyone holding the token can read them. The signature is the only part that's cryptographic, and it only proves the header and payload haven't been tampered with — it does not hide anything.

The catch: JWTs use Base64url, not standard Base64. RFC 7515 swaps + for -, / for _, and strips the trailing = padding so tokens survive inside URLs. Paste a raw JWT segment into a standard Base64 decoder and it often chokes on the missing padding or the -_ characters. That's why I keep a URL-safe toggle on — it undoes the substitution and re-adds padding before decoding, so each segment comes out as clean JSON.

Bug #1: alg is "none"

The original JWT spec allowed an alg value of none, meaning "this token is unsigned, trust it anyway." It was meant for cases where transport security already handled integrity. In practice it became one of the most reliable auth bypasses on the web.

The attack: take a valid token, change the payload to "role":"admin", set the header to {"alg":"none"}, and drop the signature entirely. Libraries that honored none would accept it. CVE-2015-9235 (jsonwebtoken), CVE-2016-5431, and a long tail of copycats all trace back to this.

So the first thing I decode is the header. Grab the part before the first dot and decode it:

// header segment
eyJhbGciOiJub25lIn0
// decoded
{"alg":"none"}

If you ever see none in production, that's a critical finding. Your validation library should reject it outright — modern versions of most libraries do, but only if you pin the expected algorithm on the verify call.

Bug #2: HS256 where you expected RS256

This one is subtler and still bites people in 2026. RS256 signs with a private key and verifies with a public key. HS256 signs and verifies with the same shared secret. The algorithm-confusion attack swaps RS256 for HS256, then signs the forged token using the server's public key as the HMAC secret — and the public key is, by definition, public.

If a verify function is written like this, it's vulnerable:

// BAD: trusts whatever alg the token claims
jwt.verify(token, keyOrSecret);

Because the library picks the algorithm from the attacker-controlled header. The fix is to pin it:

// GOOD: server dictates the algorithm
jwt.verify(token, publicKey, { algorithms: ['RS256'] });

Reading the header offline tells you instantly which algorithm a token claims. If your service issues RS256 tokens but you're staring at an HS256 header, someone is probing you.

Bug #3: secrets and PII sitting in the payload

The payload is not a secret. I've decoded production tokens and found full email addresses, internal user IDs, feature flags, and — twice — what looked like a hashed password stuffed into a custom claim. Anyone who intercepts the token, or pulls it out of a browser's localStorage, reads all of it.

Decode the middle segment and actually look at what you're shipping to the client:

{"sub":"12345","role":"user","email":"jane@corp.com","iat":1752000000,"exp":1752003600}

Check the exp claim too. It's a Unix timestamp. If it's missing, the token never expires, which turns a single leaked token into permanent access.

Why I decode offline, every time

The obvious way to read a JWT is to paste it into one of the popular online decoders. I stopped doing that, and I think you should too.

A JWT is a live credential. For as long as it hasn't expired, it is the logged-in session. Pasting a production token into a third-party website means handing your auth to whatever that site's server does with the request — logging, analytics, a compromised CDN, a curious employee. The token in the RFC 7519 examples is harmless. The one from your staging environment at 2am is not.

The property I want from anything touching a credential is that the decode happens entirely in the browser — no server round-trip, so the JSON never leaves your machine. You can confirm any client-side decoder does this: open the page, kill your network connection, and check that it still decodes. If it does, the token you paste to inspect stays local. (The one I built for this, Base64Lab, works offline for exactly that reason — but the test matters more than the tool.)

My 30-second token triage

When a token lands in front of me, the routine is always the same:

Split on the dots into three parts.
Decode the header (URL-safe on). Check alg — reject none, question anything that doesn't match what the service issues.
Decode the payload. Scan for PII or secrets that shouldn't be there. Confirm exp exists and is sane.
Leave the signature alone — you can't verify it without the key, and you don't need to for triage.

None of this requires a CLI, a library, or an internet connection to a decoder that logs your input. It's reading JSON. The only trick is a decoder that understands Base64url and keeps the data on your machine.

What's the worst thing you've found sitting in a JWT payload in production? I'll start: a hashed password in a custom claim, shipped to every browser. Curious what else is out there.

Parsing Trading Data With Regex: Test a Whole Suite of Cases at Once, Not One String

Max — Thu, 30 Jul 2026 17:04:01 +0000

Last quarter I was normalizing a feed of options trade confirmations from a brokerage API. The format looked clean at first:

2026-07-14 09:31:02 | AAPL | C | 190.00 | 2 | 3.45

Until it wasn't. Some rows used dots instead of pipes. Some had extra whitespace. A handful dropped the seconds off the timestamp entirely. My first regex ate the happy path and silently swallowed everything else — the worst failure mode, because nothing errored. I spent 45 minutes in Python's re module iterating the pattern: print intermediate results, tweak, reprint, repeat.

That's the loop I want to talk about, because most of the regex pain I see isn't "I don't know the syntax." It's "I can't see all my cases at once, so every fix quietly breaks a different row."

Why real-world data is especially hard to regex

Structured text from legacy systems each carries slightly different conventions. A few I've hit in the same project:

Timestamps that mix 2026-07-14 09:31:02, 14-JUL-2026 09:31, and 20260714T093102Z — sometimes in one file
Prices as 190.00, 190,00 (European locale), or $190.00
Ticker symbols that bleed into option codes: AAPL260718C00190000 (OCC format)
Account/reference fields that are sometimes all-numeric, sometimes alphanumeric with dashes

Writing a regex for any one of these in isolation is easy. Writing one that survives all the variants your real data actually contains is where it gets messy — and where iterating against a single pasted string falls apart.

The trick: a visible pass/fail test suite, not one string

Most regex testers let you paste a pattern + a string and highlight the match. Useful, but it only shows you one case at a time. What actually shortened my loop was defining a set of inputs, each tagged "should match" or "should not match," and watching them all evaluate simultaneously as I edit.

Take the timestamp column. My real requirements were:

09:31:02 → must match
09:31 → must also match (seconds optional upstream)
9:31:02 → must fail (no leading zero — the downstream DB column is strict HH:MM:SS)
25:00:00 → must fail (not a real hour)

The pattern I landed on:

^(\d{4})-(\d{2})-(\d{2})\s([01]\d|2[0-3]):([0-5]\d)(?::([0-5]\d))?$

Two parts people get wrong here:

([01]\d|2[0-3]) for hours, not \d{2}. The naive version happily accepts 25 and 99. Constrain the ranges or your "validation" validates nothing.
(?::([0-5]\d))? makes the seconds group optional — the (?: ... )? wraps both the colon and the digits so 09:31 passes but 09:31: (dangling colon) doesn't.

The point isn't the pattern — it's that when I changed the hour clause and it fixed 25:00:00, I could see at a glance that it hadn't broken the 09:31 (no-seconds) case. That's the feedback loop that turns a 45-minute grind into a 5-minute one.

OCC option symbols: a great regex stress test

OCC option symbology is a compact torture test. The format is [ROOT][YYMMDD][C/P][STRIKE×8], where the strike is 8 digits with 3 implied decimal places. Apple's $190 call expiring 2026-07-18 is:

AAPL260718C00190000

Decomposed:

^([A-Z]{1,6})(\d{2})(\d{2})(\d{2})([CP])(\d{5})(\d{3})$

Group 1: root, 1–6 uppercase letters
Groups 2–4: YY, MM, DD
Group 5: C or P
Groups 6–7: the 8-digit strike split into whole dollars (5) and thousandths (3)

You rebuild the strike as int(g6) + int(g7) / 1000 → 190 + 000/1000 = 190.0. A $7.50 strike encodes as 00007500 → 7 + 500/1000 = 7.5. Getting that split right on the first try is exactly the kind of thing worth verifying visually before it goes into parser code — use a replace template like $1 | $2$3$4 | $5 | $6.$7 and watch a handful of real symbols decompose cleanly.

Two habits that make the parser code readable

Named groups. Instead of:

(\d{4})-(\d{2})-(\d{2})

write:

(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})

Now it's m.group('year') in Python, not m.group(1). When a pattern has 8 capture groups (common with OCC symbols or log lines), named groups make the downstream code self-documenting — no comment needed to explain what group 6 was.

Know what regex can't do. I needed to flag rows where volume > 1000 and price < $5.00 (a low-priced, high-volume scanner filter). Regex can't compare numbers — but it can match digit ranges:

^\S+\s+[0-4]\.\d{2}\s+([1-9]\d{3})

[0-4]\.\d{2} is "0.00–4.99" and [1-9]\d{3} is "a 4-digit number that doesn't start with 0" (1000–9999). That's a deliberately narrow pattern for a specific range, not a general numeric comparison — and it's fine to reach for as long as you know that's what you're doing. The moment your ranges get irregular, drop back to code.

One thing that matters for regulated data: don't upload it

This is the part I care about most. Financial data is regulated, and I'm not pasting trade confirmations with account numbers into a cloud service whose logging policy I've never read. Whatever tester you use, prefer one that runs client-side — the JavaScript executes in your browser, the data never leaves the machine, and it keeps working offline after first load. I've iterated patterns against a brokerage CSV export on a flight with no connection. For anything touching account numbers or client data, "runs locally" isn't a nice-to-have.

The one I keep bookmarked for this is RegexLab — the multi-case test-suite mode is what actually shortened my loop, and it's browser-only so nothing gets uploaded. I wrote up the full trading-data walkthrough (locale-price handling, the OCC replace template, the timestamp suite) here.

But the tool is secondary to the habit: stop testing one string at a time. Build the failing cases into a visible suite so every fix has to survive all of them at once.

What's the nastiest real-world format you've had to regex — and what finally made it click?

Verifying Webhook Signatures by Hand: HMAC-SHA256 and Why Yours Keeps Failing

Max — Thu, 23 Jul 2026 17:00:52 +0000

A webhook fired at 2am, my handler 500'd, and the vendor's dashboard just said "delivery failed." No body, no signature, no clue. When I finally caught the payload, the first thing I needed to know was: is this actually from them, or is someone POSTing garbage at my endpoint?

That question is answered by one line of crypto — an HMAC-SHA256 signature — and you can check it by hand without pasting a production secret into some random website. This is about the boring, load-bearing part of webhooks nobody documents well: how the signature header is computed, why your comparison keeps failing, and how to verify one manually when a delivery breaks.

What the signature header actually is

Every serious webhook provider signs the request body. GitHub sends X-Hub-Signature-256. Stripe sends Stripe-Signature. Shopify sends X-Shopify-Hmac-Sha256. Different header names, same idea:

signature = HMAC-SHA256(secret, raw_request_body)

The provider and you both know a shared secret. They hash the exact bytes of the body with that secret and ship the result in a header. You recompute the same hash on your side. Match = authentic and untampered. No match = reject with a 401 and move on.

Why it matters: your webhook URL is public the moment you register it. Anyone who finds it can POST a fake "payment succeeded" event. Without signature verification, your app will happily believe them.

Verifying a GitHub signature by hand

GitHub's own docs use a concrete example, which makes it a perfect sanity check. Secret is It's a Secret to Everybody, body is Hello, World!. The expected signature is:

757107ea0eb2509fc211221cce984b8a37570b6d7586c22c46f4379c8b043e17

Run HMAC-SHA256 with that key and message and you get exactly that hex string. That's the whole verification. GitHub prefixes it with sha256= in the header, so the real value on the wire is sha256=757107ea... — strip the prefix before comparing.

Do this client-side (the Web Crypto API's crypto.subtle.sign keeps the secret in the tab — check the Network panel, there are no outbound requests). Pasting a webhook secret into a server-side "online HMAC generator" is the kind of thing that ends up in someone's access logs.

The three reasons your comparison fails

Manual verification exposes the bugs that silently break webhook handlers. In order of how often they've bitten me:

1. You hashed the parsed body, not the raw body. This is the big one. Frameworks love to parse JSON for you. But JSON.stringify(JSON.parse(body)) is not the original bytes — key order changes, whitespace vanishes, unicode gets re-escaped. The signature is over the exact bytes the provider sent. In Express you need the raw buffer:

app.post('/webhook',
  express.raw({ type: 'application/json' }),
  (req, res) => {
    const sig = req.get('X-Hub-Signature-256');
    const expected = 'sha256=' + hmacSha256(secret, req.body);
    // req.body is a Buffer here, not a parsed object
  }
);

If your handler works in tests but fails in production, this is almost always why — a body parser upstream mangled the bytes before you hashed them.

2. Wrong key encoding. Most providers treat the secret as a UTF-8 string. Some — a few payment and banking APIs — give you a hex or base64 secret that must be decoded to raw bytes first. Hashing the literal hex characters instead of the decoded bytes gives a completely different result.

3. Non-constant-time comparison. Once the bytes are right, don't compare signatures with ===. A naive string compare returns early on the first mismatched character, which leaks timing information an attacker can measure. Use a constant-time compare:

const crypto = require('crypto');
function safeEqual(a, b) {
  const ba = Buffer.from(a), bb = Buffer.from(b);
  if (ba.length !== bb.length) return false;
  return crypto.timingSafeEqual(ba, bb);
}

Stripe adds a timestamp — and so should you

Stripe's Stripe-Signature header isn't just the HMAC. It looks like:

t=1699999999,v1=5257a869e7ecebeda32affa62cdca3fa...

The signed payload is t + "." + body, not the body alone. You concatenate the timestamp, a literal dot, and the raw body, then HMAC-SHA256 that whole string with your signing secret.

The timestamp exists to stop replay attacks. Someone who captures a valid signed request can't resend it a day later, because you also check that t is within a few minutes of now. If you're building your own webhook sender, copy this pattern — sign the timestamp alongside the body and reject stale ones.

When to reach for manual verification

You don't do this on every request — your code handles the happy path. Manual HMAC checking earns its keep in exactly three moments:

First integration. Before you trust your verification code, confirm it against a known payload. Recompute the signature and diff it against what your handler produced. If they disagree, your handler is wrong, not the provider.
A specific delivery failed. Grab the raw body and the signature header from the provider's delivery log, recompute by hand, and you'll immediately see whether it's a body-encoding bug or a genuinely bad signature.
Rotating secrets. After changing a signing secret, verify one real event manually before you trust the pipeline again.

Four rules

Compute the HMAC on the raw bytes, compare in constant time, decode the key to the right encoding, and check the timestamp. Four rules, and your webhook endpoint stops trusting strangers.

I wrote the full walkthrough — including the byte-level view of what's actually being hashed and a browser-based way to reproduce each provider's signature without a network request — over on my blog.

What's the weirdest webhook signature scheme you've had to reverse-engineer? A few payment APIs out there do genuinely cursed things with the payload ordering.

Strip GPS and EXIF From a Whole Folder of Photos Before You Publish (Local, No Upload)

Max — Thu, 23 Jul 2026 15:01:34 +0000

Every JPEG your phone takes can carry the exact GPS coordinates of where you stood. Post a few "harmless" photos online and you may be broadcasting your home address, your kid's school, your gym. Most people find this out the hard way.

You don't need a website that "cleans metadata" — you can strip every image in a folder locally in one command. Nothing leaves your machine.

The one-liner (exiftool)

# macOS: brew install exiftool  |  Debian/Ubuntu: sudo apt install libimage-exiftool-perl
exiftool -all= -overwrite_original -r ./photos

-all= removes all metadata (GPS, timestamps, camera serial, software).
-overwrite_original skips the .jpg_original backups.
-r recurses into subfolders.

Verify nothing sensitive remains:

exiftool -gps:all -make -model ./photos

Empty output = clean.

Keep orientation, drop everything else

Stripping -all= can rotate photos that relied on the EXIF orientation flag. Preserve just that one tag:

exiftool -all= --Orientation -overwrite_original -r ./photos

A tiny reusable script

Save as clean-photos.sh:

#!/usr/bin/env bash
set -euo pipefail
target="${1:-.}"
echo "Stripping metadata from images in: $target"
exiftool -all= --Orientation -overwrite_original -r \
  -ext jpg -ext jpeg -ext png -ext webp "$target"
echo "Done. Verifying GPS is gone:"
exiftool -gps:all -r "$target" | grep -i gps || echo "No GPS tags found. ✅"

chmod +x clean-photos.sh
./clean-photos.sh ~/Pictures/to-publish

Why local matters

Uploading photos to an online "EXIF remover" means handing your original, geotagged images — the exact thing you're trying to protect — to a third-party server. Do it locally and the sensitive data never leaves your disk.

If you only have one or two images and don't want to touch a terminal, QuickShrink strips metadata (and compresses) entirely in your browser — the file never uploads. But for a whole folder, the exiftool script above is faster and scriptable.

TL;DR

Photos carry GPS. Assume every image does until proven otherwise.
exiftool -all= --Orientation -overwrite_original -r ./photos cleans a folder locally.
Verify with exiftool -gps:all.
Never upload the originals you're trying to protect.

Batch-Convert a Folder of Images to WebP with a Makefile (cwebp, No SaaS)

Max — Wed, 22 Jul 2026 15:00:35 +0000

Batch-Convert a Folder of Images to WebP with a Makefile (cwebp, No SaaS)

Shipping WebP instead of JPEG/PNG is one of the cheapest web-perf wins there is — usually 25–35% smaller at the same visual quality. But nobody wants to hand-run cwebp on 200 files, and uploading a client's product photos to a random web tool is a non-starter.

Here's a tiny, local, dependency-light Makefile that converts every image in assets/ to WebP, skips files that are already up to date, and never touches the network.

1. Install cwebp

# Debian/Ubuntu
sudo apt install webp
# macOS
brew install webp

2. The Makefile

Drop this in your project root:

SRC_DIR ?= assets
OUT_DIR ?= assets/webp
QUALITY ?= 80

SOURCES := $(wildcard $(SRC_DIR)/*.jpg $(SRC_DIR)/*.jpeg $(SRC_DIR)/*.png)
TARGETS := $(patsubst $(SRC_DIR)/%,$(OUT_DIR)/%.webp,$(SOURCES))

.PHONY: webp clean
webp: $(TARGETS)

$(OUT_DIR)/%.webp: $(SRC_DIR)/%
    @mkdir -p $(OUT_DIR)
    cwebp -quiet -q $(QUALITY) "$<" -o "$@"
    @echo "→ $@ ($$(du -h "$@" | cut -f1))"

clean:
    rm -rf $(OUT_DIR)

3. Run it

make webp              # convert everything at q80
make webp QUALITY=90   # higher quality
make clean             # nuke the output dir

Because targets depend on their source files, make only reconverts images that changed — so re-running it in a build pipeline is basically free.

4. Serve with a fallback

Let the browser pick WebP when it can, JPEG when it can't:

<picture>
  <source srcset="/assets/webp/hero.jpg.webp" type="image/webp">
  <img src="/assets/hero.jpg" alt="Hero" width="1200" height="600">
</picture>

When you just need one file, fast

The Makefile is for repos and CI. When I just need to shrink or convert a single image by hand — pick a quality, preview the result, download — I use QuickShrink, which does the compression in the browser (the file never leaves your machine). Same privacy property as the local cwebp approach, no install.

Why bother

Smaller payload → faster LCP, better Core Web Vitals.
Local → client images never hit a third-party server.
Incremental → make skips unchanged files, so it's cheap in CI.

That's the whole thing. No SaaS, no upload, ~15 lines of Makefile.

Auto-Optimize Images in a Git Pre-Commit Hook (Local, No Upload)

Max — Mon, 20 Jul 2026 15:00:47 +0000

Every repo I inherit has the same problem: someone dragged a 4.2 MB hero.png straight out of Figma into /public, committed it, and now every clone and every deploy drags that weight around forever. Git never forgets, so even after you shrink it later, the fat blob lives in history permanently.

The fix isn't "remember to compress images." Humans don't remember. The fix is a pre-commit hook that optimizes any staged image before it ever enters a commit — locally, with no upload to a third-party service.

Here's a setup you can paste in today.

The one-file version (no dependencies beyond what's already on your machine)

If you have pngquant and jpegoptim installed (both are in Homebrew and apt), this is the whole thing. Save it as .git/hooks/pre-commit and chmod +x it:

#!/usr/bin/env bash
# Optimize staged images before they enter a commit. All local, no uploads.
set -euo pipefail

# Only look at files that are staged (added/modified/copied)
staged=$(git diff --cached --name-only --diff-filter=ACM)

changed=0
while IFS= read -r file; do
  [ -f "$file" ] || continue
  case "${file,,}" in
    *.png)
      before=$(wc -c < "$file")
      pngquant --force --skip-if-larger --quality=65-90 --output "$file" -- "$file" 2>/dev/null || true
      after=$(wc -c < "$file")
      if [ "$after" -lt "$before" ]; then
        git add "$file"; changed=1
        echo "  png  $file  $((before/1024))KB -> $((after/1024))KB"
      fi
      ;;
    *.jpg|*.jpeg)
      before=$(wc -c < "$file")
      jpegoptim --strip-all --max=85 --quiet "$file" || true
      after=$(wc -c < "$file")
      if [ "$after" -lt "$before" ]; then
        git add "$file"; changed=1
        echo "  jpg  $file  $((before/1024))KB -> $((after/1024))KB"
      fi
      ;;
  esac
done <<< "$staged"

[ "$changed" -eq 1 ] && echo "Images optimized and re-staged."
exit 0

What it does:

Runs only on staged images, so it never rewrites files you didn't touch.
--skip-if-larger / the size check means it never makes a file bigger (compression can occasionally do that on already-tiny PNGs).
Re-stages the optimized file with git add so the smaller version is what actually gets committed.
strip-all removes EXIF — which also quietly deletes GPS coordinates from phone photos before they hit a public repo. (That alone has saved me from committing my home address in a screenshot's metadata.)

Make it team-wide with pre-commit

.git/hooks isn't version-controlled, so a raw hook only protects your machine. To enforce it for everyone, use the pre-commit framework and commit the config:

# .pre-commit-config.yaml
repos:
  - repo: local
    hooks:
      - id: optimize-images
        name: optimize staged images
        entry: bash scripts/optimize-images.sh
        language: system
        types_or: [png, jpeg]
        pass_filenames: true

Now pre-commit install wires it up for every contributor, and CI can run pre-commit run --all-files to catch anyone who skipped the hook with --no-verify.

Why local-only matters here

The lazy alternative is a SaaS compressor: upload the image, download the smaller one. That's fine for a one-off marketing PNG. It's a bad default inside an automated dev workflow for three reasons:

Latency + rate limits. A network round-trip per image in a pre-commit hook makes committing feel broken. The free tiers (TinyPNG = 500/mo) run out fast in CI.
Secrets/NDA exposure. Product screenshots, client mockups, internal dashboards — a lot of images in private repos are things you contractually cannot ship to a random third-party server.
Determinism. A hook that depends on someone's API being up is a hook that will fail your commit at the worst possible moment.

pngquant/jpegoptim run on your machine, offline, deterministically. That's the right tool for the automated path.

When you just need a quick manual squish

Hooks are for the repo. For the one-off "designer Slacked me a 6 MB PNG and I need it web-ready right now," a CLI install is overkill. I built a browser-only compressor for exactly that case — it runs the compression in the tab via the Canvas API, nothing is uploaded, and it has web/social/email presets so it's one click:

QuickShrink — drop an image, get a smaller one, close the tab. Same privacy property as the CLI (no upload), just without installing anything.

Takeaway

Don't rely on discipline to keep image weight out of your repo — automate it at the commit boundary. A ~30-line pre-commit hook using local tools will quietly shave megabytes off every PR, strip EXIF/GPS as a bonus, and never leak a private image to a third party. Ship the hook once; benefit on every commit forever.

If you're already doing this in CI, I'd like to hear how — especially anyone running it as a required GitHub Action gate rather than a local hook. What breaks at scale?

Your Photos Are Broadcasting Your Home Address (Strip EXIF GPS in the Browser)

Max — Tue, 14 Jul 2026 17:01:32 +0000

A friend sent me a photo of their new apartment and asked me to guess the neighborhood. I opened the JPEG in a terminal, ran exiftool, and read back their street address to two decimal places of latitude. They had never posted the location. The phone did it for them.

Most photos off a modern phone carry a GPS block inside the file the exact coordinates where the shutter fired, down to a few meters. Share that JPEG anywhere that doesn't re-encode it, and you're publishing your home, your kid's school, your office desk. Here's what's actually in that metadata, which platforms strip it and which don't, and why browser-only is the right fix.

What's actually inside the file

EXIF is a block of tags glued into the JPEG right after the start-of-image marker. Designed for camera settings shutter speed, ISO, focal length. But the spec also carries an entire GPS sub-directory, and phones fill it in by default.

What a single iPhone photo typically hands over:

GPS Latitude    : 37 deg 46' 29.88" N
GPS Longitude   : 122 deg 25' 9.84" W
GPS Altitude    : 14.2 m Above Sea Level
Create Date     : 2026:07:04 18:32:11
Make            : Apple
Model           : iPhone 15 Pro
Software        : 17.5.1

That lat/long pair drops a pin within ~5 meters. The timestamp says when you were standing there. The device model and OS version are a bonus for anyone fingerprinting you. None of it is visible in the picture. It rides along silently.

Under the hood: GPS coordinates are stored as three rational numbers (degrees, minutes, seconds), each a pair of 32-bit integers, referenced by an offset pointer in the main tag table. Tidy little format which is exactly why it's easy to both read and remove.

The "but platforms strip it" myth

The common reassurance is that social networks scrub metadata on upload. Some do. Many don't, and the behavior is inconsistent enough that I don't trust any of it:

Facebook, Instagram, Twitter/X: re-encode and drop EXIF. Generally safe but they replace it with their own tracking, and the re-encode wrecks quality.
Discord: keeps full EXIF on direct image attachments. That coordinate block ships straight through.
Slack: preserves the original file for downloads.
Email attachments: untouched. Whatever your camera wrote lands in the recipient's inbox.
Your own site / self-hosted gallery: serves the raw file unless you strip it yourself.
Cloud share links (Dropbox, Drive): hand over the original bytes.

The failure mode that bit my friend was a real-estate listing tool that just re-served the uploaded JPEGs. Coordinates intact. "The platform handles it" is not a plan. Stripping at the source is.

Why browser-only matters here

The obvious fix is exiftool, which is excellent. But telling a non-technical person to install a Perl utility and run exiftool -all= photo.jpg is a non-starter. The alternatives most people reach for are worse:

Online EXIF removers: you upload your geotagged photo to some stranger's server to have the location removed. Read that sentence again you just handed the coordinates to exactly the party you were hiding them from.
Desktop apps: fine, but overkill for "clean these 8 photos before I text them."
Phone share-sheet toggles: iOS has a "Remove Location" option buried in the share sheet's Options menu. Works, but only for location, only on Apple's terms, and most people never find it.

The mechanism for doing it client-side is simple enough to describe in a paragraph. A JPEG is a series of segments, each marked by 0xFF followed by a marker byte. EXIF lives in the APP1 segment (0xFFE1). To strip it, you parse the segment list, drop APP1 (and optionally APP0, XMP, and any color-profile junk), and re-concatenate the rest. The image pixels sit in the scan data, untouched so unlike the social-network approach, there's zero quality loss. No re-encode, no recompression artifacts. Same pixels, minus the tracking. Read the file with FileReader, walk the markers, rewrite without the metadata block all in the tab, no upload, no server round-trip, no log file with your coordinates in it.

I wrote up the full teardown plus a live browser tool that does exactly this here: Strip EXIF GPS in the Browser. Drag a photo in, download the clean copy, verify with exiftool if you're paranoid (I was the GPS block is gone, the pixels are byte-identical in the scan segment).

Fix it at the camera if you shoot a lot

Stripping after the fact works, but the cleaner move for anything sensitive is to not write the coordinates in the first place:

iOS: Settings > Privacy & Security > Location Services > Camera > Never.
Android: the camera app's own settings, usually "Location tags" or "Save location."

You lose the "photos on a map" feature that's the tradeoff.

But for the 90% case a few photos, right now, before you hit send a browser tab that never phones home is the right tool.

What's the sketchiest "upload it here to make it private" tool you've seen someone actually use?

Check If a Password Was Breached Without Sending It (HIBP k-Anonymity)

Max — Tue, 07 Jul 2026 17:01:51 +0000

A junior dev on my team once wanted to add a "check if your password was breached" feature to our signup form. His first instinct: POST the plaintext password to Have I Been Pwned and warn the user if it came back dirty.

I stopped him before the PR got anywhere. Sending a user's raw password to a third party to prove it isn't compromised is the kind of irony that ends up in a postmortem.

The good news is that HIBP solved this exact problem years ago with a technique called k-anonymity, and it's genuinely clever. You can check any password against 900+ million breached credentials without ever sending the password, its full hash, or anything that identifies it. Let me walk through how it works, show the actual bytes on the wire, and explain why this is one of the few "phone home" security checks I trust in a browser.

The problem with a naive breach check

The obvious design is: hash the password, send the hash, get back yes/no. But a SHA-1 hash of a password isn't anonymous. SHA-1 is fast and unsalted here, and breach corpuses are massive. If you send the full hash 5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8, the server — or anyone sniffing the request — can reverse it in microseconds against a rainbow table. That hash is literally the word password. You've leaked the credential.

You need a way to ask "is this password in your list?" where the server learns nothing useful about which password you asked about. That's what k-anonymity buys you.

How the range API actually works

The trick is to send only the first 5 characters of the SHA-1 hash. Here's the full flow for the password password:

SHA-1("password") = 5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8
                    └─┬─┘└──────────────┬──────────────────┘
                   prefix (5)        suffix (35)

GET https://api.pwnedpasswords.com/range/5BAA6

You send 5BAA6. The server responds with every breached-hash suffix that shares that prefix — the tail 35 hex characters plus a breach count, one per line:

003D68EB55068C33ACE09247EE4C639306B:29
00658BFD1E05761042698D19D32CD9F1A8F:15
...
1E4C9B93F3F0682250B6CF8331B7EE68FD8:52372427
...

That last line is the one you care about. Your browser — not the server — scans the response for your suffix 1E4C9B93F3F0682250B6CF8331B7EE68FD8, finds it, and reads the count: 52,372,427. The word "password" has appeared in 52 million breached records.

The server never saw which suffix you were looking for. It handed back roughly 800–1,000 candidates and let you do the final match locally. When I hit that prefix, I got 1,977 hash suffixes back. Any one of them could have been "your" password. That's the anonymity set.

Doing it yourself in ~15 lines

No API key, no rate limit worth worrying about, and CORS is wide open so this runs fine from browser JavaScript. Here's the whole thing in Python so you can see there's no magic:

import hashlib, urllib.request

def pwned_count(password):
    h = hashlib.sha1(password.encode()).hexdigest().upper()
    prefix, suffix = h[:5], h[5:]
    url = f"https://api.pwnedpasswords.com/range/{prefix}"
    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
    body = urllib.request.urlopen(req).read().decode()
    for line in body.splitlines():
        s, count = line.split(":")
        if s == suffix:
            return int(count)
    return 0

print(pwned_count("password"))                  # 52372427
print(pwned_count("123456"))                     # 210461208
print(pwned_count("correcthorsebatterystaple"))  # 4173
print(pwned_count("xK9#mQ2vLp8$wZ4nR7tB"))       # 0

Those are real numbers I pulled today, not made up. A couple are worth sitting with. 123456 shows up 210 million times — the single most breached string on the internet. And the famous XKCD passphrase correcthorsebatterystaple? Pwned 4,173 times. The moment a password becomes advice, it becomes a dictionary entry. Randomness is the only thing that keeps you at zero.

The JavaScript version is nearly identical, using the built-in crypto.subtle.digest("SHA-1", ...):

async function pwnedCount(password) {
  const buf = await crypto.subtle.digest(
    "SHA-1", new TextEncoder().encode(password)
  );
  const hash = [...new Uint8Array(buf)]
    .map(b => b.toString(16).padStart(2, "0")).join("").toUpperCase();
  const prefix = hash.slice(0, 5), suffix = hash.slice(5);

  const res = await fetch(`https://api.pwnedpasswords.com/range/${prefix}`);
  const body = await res.text();
  for (const line of body.split("\n")) {
    const [s, count] = line.trim().split(":");
    if (s === suffix) return parseInt(count, 10);
  }
  return 0;
}

This is exactly the kind of thing SubtleCrypto is good at — unlike MD5, which the Web Crypto API flatly refuses to compute.

The padding option most people miss

There's a subtle leak in the basic scheme. Response sizes vary — a prefix might return 400 suffixes or 1,200. A network observer counting bytes can sometimes narrow down which prefix you requested, and popular prefixes correlate with common passwords.

HIBP added a fix: send the header Add-Padding: true and the server pads every response with a random number of fake, zero-count entries.

curl -s "https://api.pwnedpasswords.com/range/5BAA6" \
     -H "Add-Padding: true" -H "User-Agent: Mozilla/5.0"

# ...real entries...
DBB7A2BC0BCFAC5BF1E8B50FFC97A118303:0   ← decoy
...

When I added the header, the response grew from 1,977 to 2,122 lines — 144 of them decoys with a count of :0. Your matching code already ignores anything with count zero, so the padding is invisible to you but blows up traffic-analysis attacks. If you're building this into a product, turn padding on. It costs a few KB.

Why browser-only matters here

k-anonymity protects you from the HIBP server, but it doesn't protect you from your own backend if you route the check through it. The cleanest design is to hash and query entirely client-side, so the plaintext never leaves the tab.

That's the same principle behind every tool I build: the file, the password, the hash never touches a server I control. I wrote up the full teardown with a live browser demo here if you want to poke at the bytes yourself.

One thing this whole exercise reframed for me: a "check if breached" feature is only as trustworthy as its data flow. The clever part isn't the hashing — it's that the question itself is anonymized.

What's the sketchiest "we'll just send it to a third party to be safe" security feature you've seen ship? I've got a couple of horror stories.

Pasting a JWT Into an Online Base64 Decoder Is a Credential Leak — Here's the Browser-Only Fix

Max — Tue, 30 Jun 2026 17:02:29 +0000

Last month I watched a teammate debug an auth bug by pasting a production JWT into the first "base64 decode online" result on Google. The token was a live bearer credential — valid for another 50 minutes, signed for our payments service. He pasted it into a text box on a server he'd never heard of, hit decode, and read the payload. The bug got fixed. The token also got handed to a stranger's web server, where it sat in request logs neither of us will ever see.

That's the quiet problem with online base64 tools, and it's worth understanding why it happens — plus the two things even experienced devs get wrong when they try to skip the tool and just use the browser console.

Why pasting a JWT into a random decoder is a credential leak

A JWT is three base64url segments joined by dots: header, payload, signature. The first two decode to plain JSON. The third is the HMAC or RSA signature. Decoding it doesn't "crack" anything — but that misses the point: the whole string is the credential. If your decoder runs server-side, you just POSTed a working bearer token to a third party.

Most "free online" decoders are server-side. You can tell because:

they still work with JavaScript disabled, or
the network tab shows a request firing on every keystroke.

Some are honest hobby projects. Some are ad-funded and log everything. You have no way to know which, and "it's probably fine" is not a security model when the input is a live session token, an API key in a config blob, or a base64-encoded .env file.

The fix isn't a better-behaved server. It's not using a server at all. atob, btoa, and TextDecoder have shipped in every browser for years — the decode can happen entirely in your tab, with zero requests carrying your data. Open the network tab while a properly client-side tool decodes a 2 MB file and you'll see exactly that: nothing leaves.

The URL-safe gotcha that breaks the browser console

Here's the part that trips up even experienced devs. You might think "I don't need a tool, I'll just run atob() in the console." Try it on a real JWT segment and watch it throw.

// A JWT payload segment is base64URL, not standard base64
atob("eyJzdWIiOiIxMjM0NTY3ODkwIn0")  // ok here

// But base64url uses - and _ instead of + and /
// and usually drops the trailing = padding:
atob("-_-_Pj_4")
// Uncaught DOMException: Failed to execute 'atob':
// The string to be decoded is not correctly encoded.

Base64url swaps two characters from the standard alphabet: + becomes -, / becomes _, and trailing = padding is usually dropped. The browser's atob only understands the standard alphabet with correct padding, so it rejects exactly the strings you most often need to decode — JWTs, OAuth state params, anything that travels in a URL.

The fix is a normalization step on every decode:

function decode(str) {
  let n = str.replace(/-/g, '+').replace(/_/g, '/').replace(/\s/g, '');
  while (n.length % 4 !== 0) n += '=';      // re-add stripped padding
  const raw = atob(n);
  try { return decodeURIComponent(escape(raw)); }  // UTF-8 aware
  catch { return raw; }                            // fall back to raw bytes
}

Tested against the canonical jwt.io token: the header decodes to {"alg":"HS256","typ":"JWT"} and the payload to {"sub":"1234567890","name":"John Doe","admin":true,"iat":1516239022} — and the same input throws Invalid character through bare atob. That replace/repad dance is the whole reason a dedicated decode beats the raw console call.

The UTF-8 trap, and the emoji that proves it

The second thing naive decoders get wrong is multi-byte text. atob hands you a binary string where each character is one byte. Decode UTF-8 content like café and a naive reader shows you cafÃ©, because it's reading the two UTF-8 bytes for é as two separate Latin-1 characters.

The decodeURIComponent(escape(raw)) trick handles it: escape percent-encodes each byte, then decodeURIComponent reads those percent groups as UTF-8. Encoding runs the mirror image:

btoa(unescape(encodeURIComponent(data)))

It's an old idiom, but it round-trips correctly, and the try/catch means raw binary that isn't valid UTF-8 falls through untouched instead of corrupting silently. I ran a string of emoji through encode → decode and got byte-identical output the other side.

Where browser-only beats the command line too

I live in a terminal, so I'll be honest about when base64 -d is the right call: scripting, pipes, CI. But three things push me back to a browser tab more often than I expected.

Auto-detect direction. Paste base64, it decodes; paste plain text, it encodes. No flipping a -d flag and re-running.
Per-line mode. A file of base64 strings, one per line, decodes row-by-row instead of being treated as one stream. macOS base64 won't do that without a while read loop.
Image preview. Paste a data:image/png;base64,... URI and render the actual image — the fastest way to sanity-check an inline asset.

And if it's a PWA with a service worker, it works offline: load it once, kill wifi, still decodes — exactly the posture you want for a tool that touches secrets.

The honest limitation

Base64 is encoding, not encryption. Decoding a JWT shows you the claims; it does not verify the signature or let you forge one. If you need to validate signatures or test signing keys, that's a different job — reach for a proper JWT library, not a base64 tool.

If you want a client-side one to poke at, I put the working version of all of the above (base64url normalization, UTF-8 round-trip, per-line, image preview, offline) into a free browser-only tool: Base64Lab. Network tab stays empty by construction. Full write-up with the byte-level details is here.

What's the worst credential you've watched someone paste into a random online tool? I'll start: a live Stripe restricted key, into a "JSON pretty print" site, on a shared screen.