<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Max</title>
    <description>The latest articles on DEV Community by Max (@orthogonalinfo).</description>
    <link>https://dev.to/orthogonalinfo</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3847175%2F78878eb1-022c-4880-ba72-cde851bc87d8.png</url>
      <title>DEV Community: Max</title>
      <link>https://dev.to/orthogonalinfo</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/orthogonalinfo"/>
    <language>en</language>
    <item>
      <title>Verifying Webhook Signatures by Hand: HMAC-SHA256 and Why Yours Keeps Failing</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Thu, 23 Jul 2026 17:00:52 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/verifying-webhook-signatures-by-hand-hmac-sha256-and-why-yours-keeps-failing-2d55</link>
      <guid>https://dev.to/orthogonalinfo/verifying-webhook-signatures-by-hand-hmac-sha256-and-why-yours-keeps-failing-2d55</guid>
      <description>&lt;p&gt;A webhook fired at 2am, my handler 500'd, and the vendor's dashboard just said "delivery failed." No body, no signature, no clue. When I finally caught the payload, the first thing I needed to know was: is this actually from them, or is someone POSTing garbage at my endpoint?&lt;/p&gt;

&lt;p&gt;That question is answered by one line of crypto — an HMAC-SHA256 signature — and you can check it by hand without pasting a production secret into some random website. This is about the boring, load-bearing part of webhooks nobody documents well: how the signature header is computed, why your comparison keeps failing, and how to verify one manually when a delivery breaks.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the signature header actually is
&lt;/h2&gt;

&lt;p&gt;Every serious webhook provider signs the request body. GitHub sends &lt;code&gt;X-Hub-Signature-256&lt;/code&gt;. Stripe sends &lt;code&gt;Stripe-Signature&lt;/code&gt;. Shopify sends &lt;code&gt;X-Shopify-Hmac-Sha256&lt;/code&gt;. Different header names, same idea:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;signature = HMAC-SHA256(secret, raw_request_body)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The provider and you both know a shared secret. They hash the exact bytes of the body with that secret and ship the result in a header. You recompute the same hash on your side. Match = authentic and untampered. No match = reject with a 401 and move on.&lt;/p&gt;

&lt;p&gt;Why it matters: your webhook URL is public the moment you register it. Anyone who finds it can POST a fake "payment succeeded" event. Without signature verification, your app will happily believe them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Verifying a GitHub signature by hand
&lt;/h2&gt;

&lt;p&gt;GitHub's own docs use a concrete example, which makes it a perfect sanity check. Secret is &lt;code&gt;It's a Secret to Everybody&lt;/code&gt;, body is &lt;code&gt;Hello, World!&lt;/code&gt;. The expected signature is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;757107ea0eb2509fc211221cce984b8a37570b6d7586c22c46f4379c8b043e17
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run HMAC-SHA256 with that key and message and you get exactly that hex string. That's the whole verification. GitHub prefixes it with &lt;code&gt;sha256=&lt;/code&gt; in the header, so the real value on the wire is &lt;code&gt;sha256=757107ea...&lt;/code&gt; — strip the prefix before comparing.&lt;/p&gt;

&lt;p&gt;Do this client-side (the Web Crypto API's &lt;code&gt;crypto.subtle.sign&lt;/code&gt; keeps the secret in the tab — check the Network panel, there are no outbound requests). Pasting a webhook secret into a server-side "online HMAC generator" is the kind of thing that ends up in someone's access logs.&lt;/p&gt;

&lt;h2&gt;
  
  
  The three reasons your comparison fails
&lt;/h2&gt;

&lt;p&gt;Manual verification exposes the bugs that silently break webhook handlers. In order of how often they've bitten me:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. You hashed the parsed body, not the raw body.&lt;/strong&gt; This is the big one. Frameworks love to parse JSON for you. But &lt;code&gt;JSON.stringify(JSON.parse(body))&lt;/code&gt; is not the original bytes — key order changes, whitespace vanishes, unicode gets re-escaped. The signature is over the &lt;em&gt;exact bytes&lt;/em&gt; the provider sent. In Express you need the raw buffer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;/webhook&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nx"&gt;express&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raw&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;application/json&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;}),&lt;/span&gt;
  &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;res&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;sig&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;X-Hub-Signature-256&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;expected&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;sha256=&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nf"&gt;hmacSha256&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;secret&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="c1"&gt;// req.body is a Buffer here, not a parsed object&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If your handler works in tests but fails in production, this is almost always why — a body parser upstream mangled the bytes before you hashed them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Wrong key encoding.&lt;/strong&gt; Most providers treat the secret as a UTF-8 string. Some — a few payment and banking APIs — give you a hex or base64 secret that must be decoded to raw bytes first. Hashing the literal hex characters instead of the decoded bytes gives a completely different result.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Non-constant-time comparison.&lt;/strong&gt; Once the bytes are right, don't compare signatures with &lt;code&gt;===&lt;/code&gt;. A naive string compare returns early on the first mismatched character, which leaks timing information an attacker can measure. Use a constant-time compare:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;crypto&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;crypto&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;safeEqual&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ba&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;Buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="nx"&gt;bb&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;Buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ba&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="nx"&gt;bb&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;crypto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;timingSafeEqual&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ba&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;bb&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Stripe adds a timestamp — and so should you
&lt;/h2&gt;

&lt;p&gt;Stripe's &lt;code&gt;Stripe-Signature&lt;/code&gt; header isn't just the HMAC. It looks like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;t=1699999999,v1=5257a869e7ecebeda32affa62cdca3fa...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The signed payload is &lt;code&gt;t + "." + body&lt;/code&gt;, &lt;strong&gt;not&lt;/strong&gt; the body alone. You concatenate the timestamp, a literal dot, and the raw body, then HMAC-SHA256 that whole string with your signing secret.&lt;/p&gt;

&lt;p&gt;The timestamp exists to stop replay attacks. Someone who captures a valid signed request can't resend it a day later, because you also check that &lt;code&gt;t&lt;/code&gt; is within a few minutes of now. If you're building your own webhook &lt;em&gt;sender&lt;/em&gt;, copy this pattern — sign the timestamp alongside the body and reject stale ones.&lt;/p&gt;

&lt;h2&gt;
  
  
  When to reach for manual verification
&lt;/h2&gt;

&lt;p&gt;You don't do this on every request — your code handles the happy path. Manual HMAC checking earns its keep in exactly three moments:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;First integration.&lt;/strong&gt; Before you trust your verification code, confirm it against a known payload. Recompute the signature and diff it against what your handler produced. If they disagree, your handler is wrong, not the provider.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A specific delivery failed.&lt;/strong&gt; Grab the raw body and the signature header from the provider's delivery log, recompute by hand, and you'll immediately see whether it's a body-encoding bug or a genuinely bad signature.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Rotating secrets.&lt;/strong&gt; After changing a signing secret, verify one real event manually before you trust the pipeline again.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Four rules
&lt;/h2&gt;

&lt;p&gt;Compute the HMAC on the &lt;strong&gt;raw bytes&lt;/strong&gt;, compare in &lt;strong&gt;constant time&lt;/strong&gt;, decode the key to the &lt;strong&gt;right encoding&lt;/strong&gt;, and check the &lt;strong&gt;timestamp&lt;/strong&gt;. Four rules, and your webhook endpoint stops trusting strangers.&lt;/p&gt;

&lt;p&gt;I wrote the full walkthrough — including the byte-level view of what's actually being hashed and a browser-based way to reproduce each provider's signature without a network request — &lt;a href="https://orthogonal.info/verifying-webhook-signatures-by-hand-hmac-sha256-in-the-browser-with-hashforge/" rel="noopener noreferrer"&gt;over on my blog&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What's the weirdest webhook signature scheme you've had to reverse-engineer? A few payment APIs out there do genuinely cursed things with the payload ordering.&lt;/p&gt;

</description>
      <category>security</category>
      <category>javascript</category>
      <category>webdev</category>
      <category>node</category>
    </item>
    <item>
      <title>Strip GPS and EXIF From a Whole Folder of Photos Before You Publish (Local, No Upload)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Thu, 23 Jul 2026 15:01:34 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/strip-gps-and-exif-from-a-whole-folder-of-photos-before-you-publish-local-no-upload-246n</link>
      <guid>https://dev.to/orthogonalinfo/strip-gps-and-exif-from-a-whole-folder-of-photos-before-you-publish-local-no-upload-246n</guid>
      <description>&lt;p&gt;Every JPEG your phone takes can carry the exact GPS coordinates of where you stood. Post a few "harmless" photos online and you may be broadcasting your home address, your kid's school, your gym. Most people find this out the hard way.&lt;/p&gt;

&lt;p&gt;You don't need a website that "cleans metadata" — you can strip every image in a folder locally in one command. Nothing leaves your machine.&lt;/p&gt;

&lt;h2&gt;
  
  
  The one-liner (exiftool)
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# macOS: brew install exiftool  |  Debian/Ubuntu: sudo apt install libimage-exiftool-perl&lt;/span&gt;
exiftool &lt;span class="nt"&gt;-all&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nt"&gt;-overwrite_original&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; ./photos
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;-all=&lt;/code&gt; removes &lt;strong&gt;all&lt;/strong&gt; metadata (GPS, timestamps, camera serial, software).&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;-overwrite_original&lt;/code&gt; skips the &lt;code&gt;.jpg_original&lt;/code&gt; backups.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;-r&lt;/code&gt; recurses into subfolders.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Verify nothing sensitive remains:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;exiftool &lt;span class="nt"&gt;-gps&lt;/span&gt;:all &lt;span class="nt"&gt;-make&lt;/span&gt; &lt;span class="nt"&gt;-model&lt;/span&gt; ./photos
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Empty output = clean.&lt;/p&gt;

&lt;h2&gt;
  
  
  Keep orientation, drop everything else
&lt;/h2&gt;

&lt;p&gt;Stripping &lt;code&gt;-all=&lt;/code&gt; can rotate photos that relied on the EXIF orientation flag. Preserve just that one tag:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;exiftool &lt;span class="nt"&gt;-all&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nt"&gt;--Orientation&lt;/span&gt; &lt;span class="nt"&gt;-overwrite_original&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; ./photos
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  A tiny reusable script
&lt;/h2&gt;

&lt;p&gt;Save as &lt;code&gt;clean-photos.sh&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;#!/usr/bin/env bash&lt;/span&gt;
&lt;span class="nb"&gt;set&lt;/span&gt; &lt;span class="nt"&gt;-euo&lt;/span&gt; pipefail
&lt;span class="nv"&gt;target&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;1&lt;/span&gt;&lt;span class="k"&gt;:-&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"Stripping metadata from images in: &lt;/span&gt;&lt;span class="nv"&gt;$target&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
exiftool &lt;span class="nt"&gt;-all&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nt"&gt;--Orientation&lt;/span&gt; &lt;span class="nt"&gt;-overwrite_original&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-ext&lt;/span&gt; jpg &lt;span class="nt"&gt;-ext&lt;/span&gt; jpeg &lt;span class="nt"&gt;-ext&lt;/span&gt; png &lt;span class="nt"&gt;-ext&lt;/span&gt; webp &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$target&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"Done. Verifying GPS is gone:"&lt;/span&gt;
exiftool &lt;span class="nt"&gt;-gps&lt;/span&gt;:all &lt;span class="nt"&gt;-r&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$target&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-i&lt;/span&gt; gps &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"No GPS tags found. ✅"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;chmod&lt;/span&gt; +x clean-photos.sh
./clean-photos.sh ~/Pictures/to-publish
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Why local matters
&lt;/h2&gt;

&lt;p&gt;Uploading photos to an online "EXIF remover" means handing your original, geotagged images — the exact thing you're trying to protect — to a third-party server. Do it locally and the sensitive data never leaves your disk.&lt;/p&gt;

&lt;p&gt;If you only have one or two images and don't want to touch a terminal, &lt;a href="https://quickshrink.orthogonal.info/?ref=devto-exif" rel="noopener noreferrer"&gt;QuickShrink&lt;/a&gt; strips metadata (and compresses) entirely in your browser — the file never uploads. But for a whole folder, the exiftool script above is faster and scriptable.&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Photos carry GPS. Assume every image does until proven otherwise.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;exiftool -all= --Orientation -overwrite_original -r ./photos&lt;/code&gt; cleans a folder locally.&lt;/li&gt;
&lt;li&gt;Verify with &lt;code&gt;exiftool -gps:all&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Never upload the originals you're trying to protect.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>security</category>
      <category>privacy</category>
      <category>bash</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Batch-Convert a Folder of Images to WebP with a Makefile (cwebp, No SaaS)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Wed, 22 Jul 2026 15:00:35 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/batch-convert-a-folder-of-images-to-webp-with-a-makefile-cwebp-no-saas-4mhd</link>
      <guid>https://dev.to/orthogonalinfo/batch-convert-a-folder-of-images-to-webp-with-a-makefile-cwebp-no-saas-4mhd</guid>
      <description>&lt;h1&gt;
  
  
  Batch-Convert a Folder of Images to WebP with a Makefile (cwebp, No SaaS)
&lt;/h1&gt;

&lt;p&gt;Shipping WebP instead of JPEG/PNG is one of the cheapest web-perf wins there is — usually 25–35% smaller at the same visual quality. But nobody wants to hand-run &lt;code&gt;cwebp&lt;/code&gt; on 200 files, and uploading a client's product photos to a random web tool is a non-starter.&lt;/p&gt;

&lt;p&gt;Here's a tiny, local, dependency-light &lt;code&gt;Makefile&lt;/code&gt; that converts every image in &lt;code&gt;assets/&lt;/code&gt; to WebP, skips files that are already up to date, and never touches the network.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Install cwebp
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Debian/Ubuntu&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;apt &lt;span class="nb"&gt;install &lt;/span&gt;webp
&lt;span class="c"&gt;# macOS&lt;/span&gt;
brew &lt;span class="nb"&gt;install &lt;/span&gt;webp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  2. The Makefile
&lt;/h2&gt;

&lt;p&gt;Drop this in your project root:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight make"&gt;&lt;code&gt;&lt;span class="nv"&gt;SRC_DIR&lt;/span&gt; &lt;span class="o"&gt;?=&lt;/span&gt; assets
&lt;span class="nv"&gt;OUT_DIR&lt;/span&gt; &lt;span class="o"&gt;?=&lt;/span&gt; assets/webp
&lt;span class="nv"&gt;QUALITY&lt;/span&gt; &lt;span class="o"&gt;?=&lt;/span&gt; 80

&lt;span class="nv"&gt;SOURCES&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="p"&gt;$(&lt;/span&gt;wildcard &lt;span class="p"&gt;$(&lt;/span&gt;SRC_DIR&lt;span class="p"&gt;)&lt;/span&gt;/&lt;span class="k"&gt;*&lt;/span&gt;.jpg &lt;span class="p"&gt;$(&lt;/span&gt;SRC_DIR&lt;span class="p"&gt;)&lt;/span&gt;/&lt;span class="k"&gt;*&lt;/span&gt;.jpeg &lt;span class="p"&gt;$(&lt;/span&gt;SRC_DIR&lt;span class="p"&gt;)&lt;/span&gt;/&lt;span class="k"&gt;*&lt;/span&gt;.png&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nv"&gt;TARGETS&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="p"&gt;$(&lt;/span&gt;patsubst &lt;span class="p"&gt;$(&lt;/span&gt;SRC_DIR&lt;span class="p"&gt;)&lt;/span&gt;/%,&lt;span class="p"&gt;$(&lt;/span&gt;OUT_DIR&lt;span class="p"&gt;)&lt;/span&gt;/%.webp,&lt;span class="p"&gt;$(&lt;/span&gt;SOURCES&lt;span class="p"&gt;))&lt;/span&gt;

&lt;span class="nl"&gt;.PHONY&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;webp clean&lt;/span&gt;
&lt;span class="nl"&gt;webp&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;$(TARGETS)&lt;/span&gt;

&lt;span class="nl"&gt;$(OUT_DIR)/%.webp&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;$(SRC_DIR)/%&lt;/span&gt;
    &lt;span class="p"&gt;@&lt;/span&gt;&lt;span class="nb"&gt;mkdir&lt;/span&gt; &lt;span class="nt"&gt;-p&lt;/span&gt; &lt;span class="p"&gt;$(&lt;/span&gt;OUT_DIR&lt;span class="p"&gt;)&lt;/span&gt;
    cwebp &lt;span class="nt"&gt;-quiet&lt;/span&gt; &lt;span class="nt"&gt;-q&lt;/span&gt; &lt;span class="p"&gt;$(&lt;/span&gt;QUALITY&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$&amp;lt;&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-o&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$@&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
    &lt;span class="p"&gt;@&lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"→ &lt;/span&gt;&lt;span class="nv"&gt;$@&lt;/span&gt;&lt;span class="s2"&gt; (&lt;/span&gt;&lt;span class="p"&gt;$$(&lt;/span&gt;&lt;span class="s2"&gt;du -h "&lt;/span&gt;&lt;span class="nv"&gt;$@&lt;/span&gt;&lt;span class="s2"&gt;" | cut -f1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;)"&lt;/span&gt;

&lt;span class="nl"&gt;clean&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
    &lt;span class="nb"&gt;rm&lt;/span&gt; &lt;span class="nt"&gt;-rf&lt;/span&gt; &lt;span class="p"&gt;$(&lt;/span&gt;OUT_DIR&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  3. Run it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;make webp              &lt;span class="c"&gt;# convert everything at q80&lt;/span&gt;
make webp &lt;span class="nv"&gt;QUALITY&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;90   &lt;span class="c"&gt;# higher quality&lt;/span&gt;
make clean             &lt;span class="c"&gt;# nuke the output dir&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because targets depend on their source files, &lt;code&gt;make&lt;/code&gt; only reconverts images that changed — so re-running it in a build pipeline is basically free.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Serve with a fallback
&lt;/h2&gt;

&lt;p&gt;Let the browser pick WebP when it can, JPEG when it can't:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;picture&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;source&lt;/span&gt; &lt;span class="na"&gt;srcset=&lt;/span&gt;&lt;span class="s"&gt;"/assets/webp/hero.jpg.webp"&lt;/span&gt; &lt;span class="na"&gt;type=&lt;/span&gt;&lt;span class="s"&gt;"image/webp"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;img&lt;/span&gt; &lt;span class="na"&gt;src=&lt;/span&gt;&lt;span class="s"&gt;"/assets/hero.jpg"&lt;/span&gt; &lt;span class="na"&gt;alt=&lt;/span&gt;&lt;span class="s"&gt;"Hero"&lt;/span&gt; &lt;span class="na"&gt;width=&lt;/span&gt;&lt;span class="s"&gt;"1200"&lt;/span&gt; &lt;span class="na"&gt;height=&lt;/span&gt;&lt;span class="s"&gt;"600"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/picture&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  When you just need one file, fast
&lt;/h2&gt;

&lt;p&gt;The Makefile is for repos and CI. When I just need to shrink or convert a single image by hand — pick a quality, preview the result, download — I use &lt;a href="https://quickshrink.orthogonal.info/?ref=devto-webp" rel="noopener noreferrer"&gt;QuickShrink&lt;/a&gt;, which does the compression in the browser (the file never leaves your machine). Same privacy property as the local &lt;code&gt;cwebp&lt;/code&gt; approach, no install.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why bother
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Smaller payload&lt;/strong&gt; → faster LCP, better Core Web Vitals.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Local&lt;/strong&gt; → client images never hit a third-party server.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Incremental&lt;/strong&gt; → &lt;code&gt;make&lt;/code&gt; skips unchanged files, so it's cheap in CI.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That's the whole thing. No SaaS, no upload, ~15 lines of Makefile.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>tutorial</category>
      <category>devops</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Auto-Optimize Images in a Git Pre-Commit Hook (Local, No Upload)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Mon, 20 Jul 2026 15:00:47 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/auto-optimize-images-in-a-git-pre-commit-hook-local-no-upload-m28</link>
      <guid>https://dev.to/orthogonalinfo/auto-optimize-images-in-a-git-pre-commit-hook-local-no-upload-m28</guid>
      <description>&lt;p&gt;Every repo I inherit has the same problem: someone dragged a 4.2 MB hero.png straight out of Figma into &lt;code&gt;/public&lt;/code&gt;, committed it, and now every clone and every deploy drags that weight around forever. Git never forgets, so even after you shrink it later, the fat blob lives in history permanently.&lt;/p&gt;

&lt;p&gt;The fix isn't "remember to compress images." Humans don't remember. The fix is a &lt;strong&gt;pre-commit hook&lt;/strong&gt; that optimizes any staged image &lt;em&gt;before&lt;/em&gt; it ever enters a commit — locally, with no upload to a third-party service.&lt;/p&gt;

&lt;p&gt;Here's a setup you can paste in today.&lt;/p&gt;

&lt;h2&gt;
  
  
  The one-file version (no dependencies beyond what's already on your machine)
&lt;/h2&gt;

&lt;p&gt;If you have &lt;code&gt;pngquant&lt;/code&gt; and &lt;code&gt;jpegoptim&lt;/code&gt; installed (both are in Homebrew and apt), this is the whole thing. Save it as &lt;code&gt;.git/hooks/pre-commit&lt;/code&gt; and &lt;code&gt;chmod +x&lt;/code&gt; it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;#!/usr/bin/env bash&lt;/span&gt;
&lt;span class="c"&gt;# Optimize staged images before they enter a commit. All local, no uploads.&lt;/span&gt;
&lt;span class="nb"&gt;set&lt;/span&gt; &lt;span class="nt"&gt;-euo&lt;/span&gt; pipefail

&lt;span class="c"&gt;# Only look at files that are staged (added/modified/copied)&lt;/span&gt;
&lt;span class="nv"&gt;staged&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;git diff &lt;span class="nt"&gt;--cached&lt;/span&gt; &lt;span class="nt"&gt;--name-only&lt;/span&gt; &lt;span class="nt"&gt;--diff-filter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;ACM&lt;span class="si"&gt;)&lt;/span&gt;

&lt;span class="nv"&gt;changed&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0
&lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="nv"&gt;IFS&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;read&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; file&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;do&lt;/span&gt;
  &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="nt"&gt;-f&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="k"&gt;continue
  case&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;file&lt;/span&gt;&lt;span class="p"&gt;,,&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt;
    &lt;span class="k"&gt;*&lt;/span&gt;.png&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="nv"&gt;before&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;wc&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; &amp;lt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;
      pngquant &lt;span class="nt"&gt;--force&lt;/span&gt; &lt;span class="nt"&gt;--skip-if-larger&lt;/span&gt; &lt;span class="nt"&gt;--quality&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;65-90 &lt;span class="nt"&gt;--output&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;--&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; 2&amp;gt;/dev/null &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;true
      &lt;/span&gt;&lt;span class="nv"&gt;after&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;wc&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; &amp;lt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;
      &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$after&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-lt&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$before&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
        &lt;/span&gt;git add &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nv"&gt;changed&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1
        &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"  png  &lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;  &lt;/span&gt;&lt;span class="k"&gt;$((&lt;/span&gt;before/1024&lt;span class="k"&gt;))&lt;/span&gt;&lt;span class="s2"&gt;KB -&amp;gt; &lt;/span&gt;&lt;span class="k"&gt;$((&lt;/span&gt;after/1024&lt;span class="k"&gt;))&lt;/span&gt;&lt;span class="s2"&gt;KB"&lt;/span&gt;
      &lt;span class="k"&gt;fi&lt;/span&gt;
      &lt;span class="p"&gt;;;&lt;/span&gt;
    &lt;span class="k"&gt;*&lt;/span&gt;.jpg|&lt;span class="k"&gt;*&lt;/span&gt;.jpeg&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="nv"&gt;before&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;wc&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; &amp;lt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;
      jpegoptim &lt;span class="nt"&gt;--strip-all&lt;/span&gt; &lt;span class="nt"&gt;--max&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;85 &lt;span class="nt"&gt;--quiet&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;true
      &lt;/span&gt;&lt;span class="nv"&gt;after&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;wc&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; &amp;lt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;
      &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$after&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-lt&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$before&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
        &lt;/span&gt;git add &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nv"&gt;changed&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1
        &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"  jpg  &lt;/span&gt;&lt;span class="nv"&gt;$file&lt;/span&gt;&lt;span class="s2"&gt;  &lt;/span&gt;&lt;span class="k"&gt;$((&lt;/span&gt;before/1024&lt;span class="k"&gt;))&lt;/span&gt;&lt;span class="s2"&gt;KB -&amp;gt; &lt;/span&gt;&lt;span class="k"&gt;$((&lt;/span&gt;after/1024&lt;span class="k"&gt;))&lt;/span&gt;&lt;span class="s2"&gt;KB"&lt;/span&gt;
      &lt;span class="k"&gt;fi&lt;/span&gt;
      &lt;span class="p"&gt;;;&lt;/span&gt;
  &lt;span class="k"&gt;esac&lt;/span&gt;
&lt;span class="k"&gt;done&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&amp;lt;&amp;lt;&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$staged&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;

&lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$changed&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-eq&lt;/span&gt; 1 &lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"Images optimized and re-staged."&lt;/span&gt;
&lt;span class="nb"&gt;exit &lt;/span&gt;0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;What it does:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Runs only on &lt;strong&gt;staged&lt;/strong&gt; images, so it never rewrites files you didn't touch.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;--skip-if-larger&lt;/code&gt; / the size check means it never makes a file &lt;em&gt;bigger&lt;/em&gt; (compression can occasionally do that on already-tiny PNGs).&lt;/li&gt;
&lt;li&gt;Re-stages the optimized file with &lt;code&gt;git add&lt;/code&gt; so the smaller version is what actually gets committed.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;strip-all&lt;/code&gt; removes EXIF — which also quietly deletes GPS coordinates from phone photos before they hit a public repo. (That alone has saved me from committing my home address in a screenshot's metadata.)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Make it team-wide with pre-commit
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;.git/hooks&lt;/code&gt; isn't version-controlled, so a raw hook only protects &lt;em&gt;your&lt;/em&gt; machine. To enforce it for everyone, use the &lt;a href="https://pre-commit.com/" rel="noopener noreferrer"&gt;&lt;code&gt;pre-commit&lt;/code&gt;&lt;/a&gt; framework and commit the config:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# .pre-commit-config.yaml&lt;/span&gt;
&lt;span class="na"&gt;repos&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;repo&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;local&lt;/span&gt;
    &lt;span class="na"&gt;hooks&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;optimize-images&lt;/span&gt;
        &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;optimize staged images&lt;/span&gt;
        &lt;span class="na"&gt;entry&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;bash scripts/optimize-images.sh&lt;/span&gt;
        &lt;span class="na"&gt;language&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;system&lt;/span&gt;
        &lt;span class="na"&gt;types_or&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;png&lt;/span&gt;&lt;span class="pi"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;jpeg&lt;/span&gt;&lt;span class="pi"&gt;]&lt;/span&gt;
        &lt;span class="na"&gt;pass_filenames&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now &lt;code&gt;pre-commit install&lt;/code&gt; wires it up for every contributor, and CI can run &lt;code&gt;pre-commit run --all-files&lt;/code&gt; to catch anyone who skipped the hook with &lt;code&gt;--no-verify&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why local-only matters here
&lt;/h2&gt;

&lt;p&gt;The lazy alternative is a SaaS compressor: upload the image, download the smaller one. That's fine for a one-off marketing PNG. It's a bad default inside an automated dev workflow for three reasons:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Latency + rate limits.&lt;/strong&gt; A network round-trip per image in a pre-commit hook makes committing feel broken. The free tiers (TinyPNG = 500/mo) run out fast in CI.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Secrets/NDA exposure.&lt;/strong&gt; Product screenshots, client mockups, internal dashboards — a lot of images in private repos are things you contractually cannot ship to a random third-party server.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Determinism.&lt;/strong&gt; A hook that depends on someone's API being up is a hook that will fail your commit at the worst possible moment.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;code&gt;pngquant&lt;/code&gt;/&lt;code&gt;jpegoptim&lt;/code&gt; run on your machine, offline, deterministically. That's the right tool for the &lt;em&gt;automated&lt;/em&gt; path.&lt;/p&gt;

&lt;h2&gt;
  
  
  When you just need a quick manual squish
&lt;/h2&gt;

&lt;p&gt;Hooks are for the repo. For the one-off "designer Slacked me a 6 MB PNG and I need it web-ready right now," a CLI install is overkill. I built a browser-only compressor for exactly that case — it runs the compression in the tab via the Canvas API, nothing is uploaded, and it has web/social/email presets so it's one click:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://quickshrink.orthogonal.info/?ref=devto-precommit" rel="noopener noreferrer"&gt;QuickShrink&lt;/a&gt;&lt;/strong&gt; — drop an image, get a smaller one, close the tab. Same privacy property as the CLI (no upload), just without installing anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaway
&lt;/h2&gt;

&lt;p&gt;Don't rely on discipline to keep image weight out of your repo — automate it at the commit boundary. A ~30-line pre-commit hook using local tools will quietly shave megabytes off every PR, strip EXIF/GPS as a bonus, and never leak a private image to a third party. Ship the hook once; benefit on every commit forever.&lt;/p&gt;

&lt;p&gt;If you're already doing this in CI, I'd like to hear how — especially anyone running it as a required GitHub Action gate rather than a local hook. What breaks at scale?&lt;/p&gt;

</description>
      <category>git</category>
      <category>devops</category>
      <category>webdev</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Your Photos Are Broadcasting Your Home Address (Strip EXIF GPS in the Browser)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Tue, 14 Jul 2026 17:01:32 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/your-photos-are-broadcasting-your-home-address-strip-exif-gps-in-the-browser-3fcj</link>
      <guid>https://dev.to/orthogonalinfo/your-photos-are-broadcasting-your-home-address-strip-exif-gps-in-the-browser-3fcj</guid>
      <description>&lt;p&gt;A friend sent me a photo of their new apartment and asked me to guess the neighborhood. I opened the JPEG in a terminal, ran &lt;code&gt;exiftool&lt;/code&gt;, and read back their street address to two decimal places of latitude. They had never posted the location. The phone did it for them.&lt;/p&gt;

&lt;p&gt;Most photos off a modern phone carry a GPS block inside the file the exact coordinates where the shutter fired, down to a few meters. Share that JPEG anywhere that doesn't re-encode it, and you're publishing your home, your kid's school, your office desk. Here's what's actually in that metadata, which platforms strip it and which don't, and why browser-only is the right fix.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's actually inside the file
&lt;/h2&gt;

&lt;p&gt;EXIF is a block of tags glued into the JPEG right after the start-of-image marker. Designed for camera settings shutter speed, ISO, focal length. But the spec also carries an entire GPS sub-directory, and phones fill it in by default.&lt;/p&gt;

&lt;p&gt;What a single iPhone photo typically hands over:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;GPS Latitude    : 37 deg 46' 29.88" N
GPS Longitude   : 122 deg 25' 9.84" W
GPS Altitude    : 14.2 m Above Sea Level
Create Date     : 2026:07:04 18:32:11
Make            : Apple
Model           : iPhone 15 Pro
Software        : 17.5.1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That lat/long pair drops a pin within ~5 meters. The timestamp says when you were standing there. The device model and OS version are a bonus for anyone fingerprinting you. None of it is visible in the picture. It rides along silently.&lt;/p&gt;

&lt;p&gt;Under the hood: GPS coordinates are stored as three rational numbers (degrees, minutes, seconds), each a pair of 32-bit integers, referenced by an offset pointer in the main tag table. Tidy little format which is exactly why it's easy to both read and remove.&lt;/p&gt;

&lt;h2&gt;
  
  
  The "but platforms strip it" myth
&lt;/h2&gt;

&lt;p&gt;The common reassurance is that social networks scrub metadata on upload. Some do. Many don't, and the behavior is inconsistent enough that I don't trust any of it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Facebook, Instagram, Twitter/X:&lt;/strong&gt; re-encode and drop EXIF. Generally safe but they replace it with their own tracking, and the re-encode wrecks quality.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Discord:&lt;/strong&gt; keeps full EXIF on direct image attachments. That coordinate block ships straight through.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Slack:&lt;/strong&gt; preserves the original file for downloads.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Email attachments:&lt;/strong&gt; untouched. Whatever your camera wrote lands in the recipient's inbox.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Your own site / self-hosted gallery:&lt;/strong&gt; serves the raw file unless you strip it yourself.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cloud share links (Dropbox, Drive):&lt;/strong&gt; hand over the original bytes.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The failure mode that bit my friend was a real-estate listing tool that just re-served the uploaded JPEGs. Coordinates intact. "The platform handles it" is not a plan. Stripping at the source is.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why browser-only matters here
&lt;/h2&gt;

&lt;p&gt;The obvious fix is &lt;code&gt;exiftool&lt;/code&gt;, which is excellent. But telling a non-technical person to install a Perl utility and run &lt;code&gt;exiftool -all= photo.jpg&lt;/code&gt; is a non-starter. The alternatives most people reach for are worse:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Online EXIF removers:&lt;/strong&gt; you upload your geotagged photo to some stranger's server to have the location removed. Read that sentence again you just handed the coordinates to exactly the party you were hiding them from.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Desktop apps:&lt;/strong&gt; fine, but overkill for "clean these 8 photos before I text them."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phone share-sheet toggles:&lt;/strong&gt; iOS has a "Remove Location" option buried in the share sheet's Options menu. Works, but only for location, only on Apple's terms, and most people never find it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The mechanism for doing it client-side is simple enough to describe in a paragraph. A JPEG is a series of segments, each marked by &lt;code&gt;0xFF&lt;/code&gt; followed by a marker byte. EXIF lives in the APP1 segment (&lt;code&gt;0xFFE1&lt;/code&gt;). To strip it, you parse the segment list, drop APP1 (and optionally APP0, XMP, and any color-profile junk), and re-concatenate the rest. The image pixels sit in the scan data, untouched so unlike the social-network approach, there's &lt;strong&gt;zero quality loss&lt;/strong&gt;. No re-encode, no recompression artifacts. Same pixels, minus the tracking. Read the file with &lt;code&gt;FileReader&lt;/code&gt;, walk the markers, rewrite without the metadata block all in the tab, no upload, no server round-trip, no log file with your coordinates in it.&lt;/p&gt;

&lt;p&gt;I wrote up the full teardown plus a live browser tool that does exactly this here: &lt;a href="https://orthogonal.info/your-photos-are-broadcasting-your-home-address-strip-exif-gps-in-the-browser/" rel="noopener noreferrer"&gt;Strip EXIF GPS in the Browser&lt;/a&gt;. Drag a photo in, download the clean copy, verify with &lt;code&gt;exiftool&lt;/code&gt; if you're paranoid (I was the GPS block is gone, the pixels are byte-identical in the scan segment).&lt;/p&gt;

&lt;h2&gt;
  
  
  Fix it at the camera if you shoot a lot
&lt;/h2&gt;

&lt;p&gt;Stripping after the fact works, but the cleaner move for anything sensitive is to not write the coordinates in the first place:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;iOS:&lt;/strong&gt; Settings &amp;gt; Privacy &amp;amp; Security &amp;gt; Location Services &amp;gt; Camera &amp;gt; Never.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Android:&lt;/strong&gt; the camera app's own settings, usually "Location tags" or "Save location."&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;You lose the "photos on a map" feature that's the tradeoff.&lt;/p&gt;

&lt;p&gt;But for the 90% case a few photos, right now, before you hit send a browser tab that never phones home is the right tool.&lt;/p&gt;

&lt;p&gt;What's the sketchiest "upload it here to make it private" tool you've seen someone actually use?&lt;/p&gt;

</description>
      <category>security</category>
      <category>javascript</category>
      <category>webdev</category>
      <category>privacy</category>
    </item>
    <item>
      <title>Check If a Password Was Breached Without Sending It (HIBP k-Anonymity)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Tue, 07 Jul 2026 17:01:51 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/check-if-a-password-was-breached-without-sending-it-hibp-k-anonymity-5j1</link>
      <guid>https://dev.to/orthogonalinfo/check-if-a-password-was-breached-without-sending-it-hibp-k-anonymity-5j1</guid>
      <description>&lt;p&gt;A junior dev on my team once wanted to add a "check if your password was breached" feature to our signup form. His first instinct: &lt;code&gt;POST&lt;/code&gt; the plaintext password to Have I Been Pwned and warn the user if it came back dirty.&lt;/p&gt;

&lt;p&gt;I stopped him before the PR got anywhere. Sending a user's raw password to a third party to &lt;em&gt;prove&lt;/em&gt; it isn't compromised is the kind of irony that ends up in a postmortem.&lt;/p&gt;

&lt;p&gt;The good news is that HIBP solved this exact problem years ago with a technique called &lt;strong&gt;k-anonymity&lt;/strong&gt;, and it's genuinely clever. You can check any password against 900+ million breached credentials &lt;strong&gt;without ever sending the password, its full hash, or anything that identifies it.&lt;/strong&gt; Let me walk through how it works, show the actual bytes on the wire, and explain why this is one of the few "phone home" security checks I trust in a browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem with a naive breach check
&lt;/h2&gt;

&lt;p&gt;The obvious design is: hash the password, send the hash, get back yes/no. But a SHA-1 hash of a password isn't anonymous. SHA-1 is fast and unsalted here, and breach corpuses are massive. If you send the full hash &lt;code&gt;5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8&lt;/code&gt;, the server — or anyone sniffing the request — can reverse it in microseconds against a rainbow table. That hash is literally the word &lt;code&gt;password&lt;/code&gt;. You've leaked the credential.&lt;/p&gt;

&lt;p&gt;You need a way to ask "is this password in your list?" where the server learns nothing useful about &lt;em&gt;which&lt;/em&gt; password you asked about. That's what k-anonymity buys you.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the range API actually works
&lt;/h2&gt;

&lt;p&gt;The trick is to send only the &lt;strong&gt;first 5 characters&lt;/strong&gt; of the SHA-1 hash. Here's the full flow for the password &lt;code&gt;password&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;SHA-1("password") = 5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8
                    └─┬─┘└──────────────┬──────────────────┘
                   prefix (5)        suffix (35)

GET https://api.pwnedpasswords.com/range/5BAA6
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You send &lt;code&gt;5BAA6&lt;/code&gt;. The server responds with every breached-hash suffix that shares that prefix — the tail 35 hex characters plus a breach count, one per line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;003D68EB55068C33ACE09247EE4C639306B:29
00658BFD1E05761042698D19D32CD9F1A8F:15
...
1E4C9B93F3F0682250B6CF8331B7EE68FD8:52372427
...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That last line is the one you care about. &lt;strong&gt;Your browser — not the server — scans the response&lt;/strong&gt; for your suffix &lt;code&gt;1E4C9B93F3F0682250B6CF8331B7EE68FD8&lt;/code&gt;, finds it, and reads the count: &lt;code&gt;52,372,427&lt;/code&gt;. The word "password" has appeared in 52 million breached records.&lt;/p&gt;

&lt;p&gt;The server never saw which suffix you were looking for. It handed back roughly 800–1,000 candidates and let you do the final match locally. When I hit that prefix, I got 1,977 hash suffixes back. Any one of them could have been "your" password. That's the &lt;strong&gt;anonymity set&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Doing it yourself in ~15 lines
&lt;/h2&gt;

&lt;p&gt;No API key, no rate limit worth worrying about, and CORS is wide open so this runs fine from browser JavaScript. Here's the whole thing in Python so you can see there's no magic:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;pwned_count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;password&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;h&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sha1&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;password&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;()).&lt;/span&gt;&lt;span class="nf"&gt;hexdigest&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;upper&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;prefix&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;suffix&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;h&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;h&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;:]&lt;/span&gt;
    &lt;span class="n"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.pwnedpasswords.com/range/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;prefix&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;req&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;User-Agent&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Mozilla/5.0&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="n"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;urlopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;read&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;decode&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;splitlines&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;count&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;suffix&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;count&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;

&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;pwned_count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;password&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;                  &lt;span class="c1"&gt;# 52372427
&lt;/span&gt;&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;pwned_count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;123456&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;                     &lt;span class="c1"&gt;# 210461208
&lt;/span&gt;&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;pwned_count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;correcthorsebatterystaple&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;  &lt;span class="c1"&gt;# 4173
&lt;/span&gt;&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;pwned_count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;xK9#mQ2vLp8$wZ4nR7tB&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;       &lt;span class="c1"&gt;# 0
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Those are real numbers I pulled today, not made up. A couple are worth sitting with. &lt;code&gt;123456&lt;/code&gt; shows up &lt;strong&gt;210 million&lt;/strong&gt; times — the single most breached string on the internet. And the famous XKCD passphrase &lt;code&gt;correcthorsebatterystaple&lt;/code&gt;? Pwned &lt;strong&gt;4,173&lt;/strong&gt; times. The moment a password becomes advice, it becomes a dictionary entry. Randomness is the only thing that keeps you at zero.&lt;/p&gt;

&lt;p&gt;The JavaScript version is nearly identical, using the built-in &lt;code&gt;crypto.subtle.digest("SHA-1", ...)&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;pwnedCount&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;password&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;buf&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;crypto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;subtle&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;SHA-1&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;TextEncoder&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;password&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;hash&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[...&lt;/span&gt;&lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Uint8Array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;buf&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;padStart&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;""&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;toUpperCase&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;prefix&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;hash&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="nx"&gt;suffix&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;hash&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;res&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`https://api.pwnedpasswords.com/range/&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;prefix&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;res&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;text&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;line&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;count&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;:&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="nx"&gt;suffix&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;parseInt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is exactly the kind of thing &lt;code&gt;SubtleCrypto&lt;/code&gt; is good at — unlike MD5, which the Web Crypto API flatly refuses to compute.&lt;/p&gt;

&lt;h2&gt;
  
  
  The padding option most people miss
&lt;/h2&gt;

&lt;p&gt;There's a subtle leak in the basic scheme. Response sizes vary — a prefix might return 400 suffixes or 1,200. A network observer counting bytes can sometimes narrow down which prefix you requested, and popular prefixes correlate with common passwords.&lt;/p&gt;

&lt;p&gt;HIBP added a fix: send the header &lt;code&gt;Add-Padding: true&lt;/code&gt; and the server pads every response with a random number of fake, zero-count entries.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="s2"&gt;"https://api.pwnedpasswords.com/range/5BAA6"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
     &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"Add-Padding: true"&lt;/span&gt; &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"User-Agent: Mozilla/5.0"&lt;/span&gt;

&lt;span class="c"&gt;# ...real entries...&lt;/span&gt;
DBB7A2BC0BCFAC5BF1E8B50FFC97A118303:0   ← decoy
...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When I added the header, the response grew from 1,977 to 2,122 lines — 144 of them decoys with a count of &lt;code&gt;:0&lt;/code&gt;. Your matching code already ignores anything with count zero, so the padding is invisible to you but blows up traffic-analysis attacks. If you're building this into a product, &lt;strong&gt;turn padding on.&lt;/strong&gt; It costs a few KB.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why browser-only matters here
&lt;/h2&gt;

&lt;p&gt;k-anonymity protects you from the HIBP server, but it doesn't protect you from &lt;em&gt;your own backend&lt;/em&gt; if you route the check through it. The cleanest design is to hash and query entirely client-side, so the plaintext never leaves the tab.&lt;/p&gt;

&lt;p&gt;That's the same principle behind every tool I build: the file, the password, the hash never touches a server I control. I wrote up &lt;a href="https://orthogonal.info/check-breached-password-without-sending-hibp-k-anonymity/" rel="noopener noreferrer"&gt;the full teardown with a live browser demo here&lt;/a&gt; if you want to poke at the bytes yourself.&lt;/p&gt;




&lt;p&gt;One thing this whole exercise reframed for me: a "check if breached" feature is only as trustworthy as its data flow. The clever part isn't the hashing — it's that the &lt;em&gt;question itself&lt;/em&gt; is anonymized.&lt;/p&gt;

&lt;p&gt;What's the sketchiest "we'll just send it to a third party to be safe" security feature you've seen ship? I've got a couple of horror stories.&lt;/p&gt;

</description>
      <category>security</category>
      <category>javascript</category>
      <category>webdev</category>
      <category>python</category>
    </item>
    <item>
      <title>Pasting a JWT Into an Online Base64 Decoder Is a Credential Leak — Here's the Browser-Only Fix</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Tue, 30 Jun 2026 17:02:29 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/pasting-a-jwt-into-an-online-base64-decoder-is-a-credential-leak-heres-the-browser-only-fix-lmo</link>
      <guid>https://dev.to/orthogonalinfo/pasting-a-jwt-into-an-online-base64-decoder-is-a-credential-leak-heres-the-browser-only-fix-lmo</guid>
      <description>&lt;p&gt;Last month I watched a teammate debug an auth bug by pasting a production JWT into the first "base64 decode online" result on Google. The token was a live bearer credential — valid for another 50 minutes, signed for our payments service. He pasted it into a text box on a server he'd never heard of, hit decode, and read the payload. The bug got fixed. The token also got handed to a stranger's web server, where it sat in request logs neither of us will ever see.&lt;/p&gt;

&lt;p&gt;That's the quiet problem with online base64 tools, and it's worth understanding &lt;em&gt;why&lt;/em&gt; it happens — plus the two things even experienced devs get wrong when they try to skip the tool and just use the browser console.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why pasting a JWT into a random decoder is a credential leak
&lt;/h2&gt;

&lt;p&gt;A JWT is three base64url segments joined by dots: header, payload, signature. The first two decode to plain JSON. The third is the HMAC or RSA signature. Decoding it doesn't "crack" anything — but that misses the point: &lt;strong&gt;the whole string is the credential.&lt;/strong&gt; If your decoder runs server-side, you just POSTed a working bearer token to a third party.&lt;/p&gt;

&lt;p&gt;Most "free online" decoders &lt;em&gt;are&lt;/em&gt; server-side. You can tell because:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;they still work with JavaScript disabled, or&lt;/li&gt;
&lt;li&gt;the network tab shows a request firing on every keystroke.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Some are honest hobby projects. Some are ad-funded and log everything. You have no way to know which, and "it's probably fine" is not a security model when the input is a live session token, an API key in a config blob, or a base64-encoded &lt;code&gt;.env&lt;/code&gt; file.&lt;/p&gt;

&lt;p&gt;The fix isn't a better-behaved server. It's not using a server at all. &lt;code&gt;atob&lt;/code&gt;, &lt;code&gt;btoa&lt;/code&gt;, and &lt;code&gt;TextDecoder&lt;/code&gt; have shipped in every browser for years — the decode can happen entirely in your tab, with zero requests carrying your data. Open the network tab while a properly client-side tool decodes a 2 MB file and you'll see exactly that: nothing leaves.&lt;/p&gt;

&lt;h2&gt;
  
  
  The URL-safe gotcha that breaks the browser console
&lt;/h2&gt;

&lt;p&gt;Here's the part that trips up even experienced devs. You might think "I don't need a tool, I'll just run &lt;code&gt;atob()&lt;/code&gt; in the console." Try it on a real JWT segment and watch it throw.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// A JWT payload segment is base64URL, not standard base64&lt;/span&gt;
&lt;span class="nf"&gt;atob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;eyJzdWIiOiIxMjM0NTY3ODkwIn0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  &lt;span class="c1"&gt;// ok here&lt;/span&gt;

&lt;span class="c1"&gt;// But base64url uses - and _ instead of + and /&lt;/span&gt;
&lt;span class="c1"&gt;// and usually drops the trailing = padding:&lt;/span&gt;
&lt;span class="nf"&gt;atob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;-_-_Pj_4&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;// Uncaught DOMException: Failed to execute 'atob':&lt;/span&gt;
&lt;span class="c1"&gt;// The string to be decoded is not correctly encoded.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Base64url swaps two characters from the standard alphabet: &lt;code&gt;+&lt;/code&gt; becomes &lt;code&gt;-&lt;/code&gt;, &lt;code&gt;/&lt;/code&gt; becomes &lt;code&gt;_&lt;/code&gt;, and trailing &lt;code&gt;=&lt;/code&gt; padding is usually dropped. The browser's &lt;code&gt;atob&lt;/code&gt; only understands the &lt;em&gt;standard&lt;/em&gt; alphabet with correct padding, so it rejects exactly the strings you most often need to decode — JWTs, OAuth &lt;code&gt;state&lt;/code&gt; params, anything that travels in a URL.&lt;/p&gt;

&lt;p&gt;The fix is a normalization step on every decode:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;decode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;n&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;str&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/-/g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;+&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/_/g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;/&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\s&lt;/span&gt;&lt;span class="sr"&gt;/g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;''&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nx"&gt;n&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;=&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;      &lt;span class="c1"&gt;// re-add stripped padding&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;raw&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;atob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;n&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;decodeURIComponent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;escape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;raw&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;  &lt;span class="c1"&gt;// UTF-8 aware&lt;/span&gt;
  &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;raw&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;                            &lt;span class="c1"&gt;// fall back to raw bytes&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Tested against the canonical jwt.io token: the header decodes to &lt;code&gt;{"alg":"HS256","typ":"JWT"}&lt;/code&gt; and the payload to &lt;code&gt;{"sub":"1234567890","name":"John Doe","admin":true,"iat":1516239022}&lt;/code&gt; — and the &lt;em&gt;same input&lt;/em&gt; throws &lt;code&gt;Invalid character&lt;/code&gt; through bare &lt;code&gt;atob&lt;/code&gt;. That &lt;code&gt;replace&lt;/code&gt;/repad dance is the whole reason a dedicated decode beats the raw console call.&lt;/p&gt;

&lt;h2&gt;
  
  
  The UTF-8 trap, and the emoji that proves it
&lt;/h2&gt;

&lt;p&gt;The second thing naive decoders get wrong is multi-byte text. &lt;code&gt;atob&lt;/code&gt; hands you a &lt;em&gt;binary string&lt;/em&gt; where each character is one byte. Decode UTF-8 content like &lt;code&gt;café&lt;/code&gt; and a naive reader shows you &lt;code&gt;cafÃ©&lt;/code&gt;, because it's reading the two UTF-8 bytes for &lt;code&gt;é&lt;/code&gt; as two separate Latin-1 characters.&lt;/p&gt;

&lt;p&gt;The &lt;code&gt;decodeURIComponent(escape(raw))&lt;/code&gt; trick handles it: &lt;code&gt;escape&lt;/code&gt; percent-encodes each byte, then &lt;code&gt;decodeURIComponent&lt;/code&gt; reads those percent groups as UTF-8. Encoding runs the mirror image:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nf"&gt;btoa&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;unescape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;encodeURIComponent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;)))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It's an old idiom, but it round-trips correctly, and the &lt;code&gt;try/catch&lt;/code&gt; means raw binary that isn't valid UTF-8 falls through untouched instead of corrupting silently. I ran a string of emoji through encode → decode and got byte-identical output the other side.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where browser-only beats the command line too
&lt;/h2&gt;

&lt;p&gt;I live in a terminal, so I'll be honest about when &lt;code&gt;base64 -d&lt;/code&gt; is the right call: scripting, pipes, CI. But three things push me back to a browser tab more often than I expected.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Auto-detect direction.&lt;/strong&gt; Paste base64, it decodes; paste plain text, it encodes. No flipping a &lt;code&gt;-d&lt;/code&gt; flag and re-running.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Per-line mode.&lt;/strong&gt; A file of base64 strings, one per line, decodes row-by-row instead of being treated as one stream. macOS &lt;code&gt;base64&lt;/code&gt; won't do that without a &lt;code&gt;while read&lt;/code&gt; loop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Image preview.&lt;/strong&gt; Paste a &lt;code&gt;data:image/png;base64,...&lt;/code&gt; URI and render the actual image — the fastest way to sanity-check an inline asset.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And if it's a PWA with a service worker, it works offline: load it once, kill wifi, still decodes — exactly the posture you want for a tool that touches secrets.&lt;/p&gt;

&lt;h2&gt;
  
  
  The honest limitation
&lt;/h2&gt;

&lt;p&gt;Base64 is &lt;strong&gt;encoding, not encryption.&lt;/strong&gt; Decoding a JWT shows you the claims; it does not verify the signature or let you forge one. If you need to validate signatures or test signing keys, that's a different job — reach for a proper JWT library, not a base64 tool.&lt;/p&gt;

&lt;p&gt;If you want a client-side one to poke at, I put the working version of all of the above (base64url normalization, UTF-8 round-trip, per-line, image preview, offline) into a free browser-only tool: &lt;a href="https://base64lab.orthogonal.info/" rel="noopener noreferrer"&gt;Base64Lab&lt;/a&gt;. Network tab stays empty by construction. Full write-up with the byte-level details is &lt;a href="https://orthogonal.info/base64lab-decode-jwt-base64url-browser-only/" rel="noopener noreferrer"&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What's the worst credential you've watched someone paste into a random online tool? I'll start: a live Stripe restricted key, into a "JSON pretty print" site, on a shared screen.&lt;/p&gt;

</description>
      <category>security</category>
      <category>javascript</category>
      <category>webdev</category>
      <category>node</category>
    </item>
    <item>
      <title>Reverse-Engineering SEC EDGAR's Full-Text Search API (One Undocumented GET Request)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Tue, 23 Jun 2026 17:04:58 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/reverse-engineering-sec-edgars-full-text-search-api-one-undocumented-get-request-4ie6</link>
      <guid>https://dev.to/orthogonalinfo/reverse-engineering-sec-edgars-full-text-search-api-one-undocumented-get-request-4ie6</guid>
      <description>&lt;p&gt;The official SEC EDGAR full-text search box is great if you're a human clicking around. It's useless if you want to pull 200 filings that mention "going concern" into a script.&lt;/p&gt;

&lt;p&gt;So I opened the network tab, watched what the search page actually calls, and rebuilt the request myself. It turns out the entire thing runs on one &lt;strong&gt;undocumented GET request&lt;/strong&gt; that returns clean Elasticsearch JSON. No API key, no signup, no OAuth dance. The SEC quietly shipped one of the better free financial-data APIs and never put a docs page on it.&lt;/p&gt;

&lt;p&gt;Here's the exact request, the response fields nobody explains, and the gotchas that cost me an afternoon.&lt;/p&gt;

&lt;h2&gt;
  
  
  The endpoint and its real parameters
&lt;/h2&gt;

&lt;p&gt;The page is a thin React front end. Every search fires a GET to &lt;code&gt;https://efts.sec.gov/LATEST/search-index&lt;/code&gt; and gets back raw Elasticsearch JSON.&lt;/p&gt;

&lt;p&gt;One trap before you copy anything: &lt;strong&gt;the path casing matters.&lt;/strong&gt; &lt;code&gt;/LATEST/&lt;/code&gt; is uppercase; a lowercase &lt;code&gt;/latest/&lt;/code&gt; 404s.&lt;/p&gt;

&lt;p&gt;The query parameters that actually do something:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;q&lt;/code&gt; — the search term. Wrap a phrase in URL-encoded double quotes (&lt;code&gt;%22climate+risk%22&lt;/code&gt;) for an exact match, or it tokenizes into an OR search.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;forms&lt;/code&gt; — comma-separated filing types: &lt;code&gt;10-K&lt;/code&gt;, &lt;code&gt;8-K&lt;/code&gt;, &lt;code&gt;SC 13D&lt;/code&gt;, etc. Leave it off to search everything.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;startdt&lt;/code&gt; and &lt;code&gt;enddt&lt;/code&gt; — date bounds in &lt;code&gt;YYYY-MM-DD&lt;/code&gt;. Both required if you want a window.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;from&lt;/code&gt; — pagination offset.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;ciks&lt;/code&gt; — restrict to a specific company by its zero-padded CIK number.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A complete request:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-A&lt;/span&gt; &lt;span class="s2"&gt;"your-app your-email@example.com"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="s2"&gt;"https://efts.sec.gov/LATEST/search-index?q=%22machine+learning%22&amp;amp;forms=8-K&amp;amp;startdt=2026-01-01&amp;amp;enddt=2026-06-01"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;The User-Agent header is not optional.&lt;/strong&gt; SEC's fair-access policy rejects requests with a generic or empty agent — you'll get a 403. Put your app name and a contact email in there. I learned this the hard way after my first ten curls returned nothing but an HTML block page.&lt;/p&gt;

&lt;h2&gt;
  
  
  The two fields that unlock everything
&lt;/h2&gt;

&lt;p&gt;The response is the Elasticsearch envelope, untouched. A single hit looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"0001193125-26-032000:ionq-ex99_2.htm"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"_source"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"ciks"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"0001824920"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"display_names"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"IonQ, Inc.  (IONQ)  (CIK 0001824920)"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"root_forms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"8-K"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"form"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"8-K"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"file_date"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"2026-01-30"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"adsh"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"0001193125-26-032000"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"file_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"EX-99.2"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two fields do all the work:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;_id&lt;/code&gt;&lt;/strong&gt; is &lt;code&gt;{accession}:{filename}&lt;/code&gt;. Split on the colon and you can build a direct link to the document.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;adsh&lt;/code&gt;&lt;/strong&gt; is the accession number — the join key you feed into the rest of EDGAR's data and XBRL endpoints to pull the full filing.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Turning a hit into a clickable filing URL means stripping the dashes from the accession number for the folder path:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;filing_url&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hit&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;adsh&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;fname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;hit&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;_id&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;cik&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hit&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;_source&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ciks&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;  &lt;span class="c1"&gt;# int() drops leading zeros
&lt;/span&gt;    &lt;span class="n"&gt;folder&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;adsh&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;-&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://www.sec.gov/Archives/edgar/data/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;cik&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;folder&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;fname&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A few &lt;code&gt;_source&lt;/code&gt; fields are worth knowing because the docs never mention them:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;items&lt;/code&gt;&lt;/strong&gt; — 8-K item codes. This is the fast filter for event-driven work: &lt;code&gt;2.02&lt;/code&gt; is earnings, &lt;code&gt;5.02&lt;/code&gt; is an exec change, &lt;code&gt;1.01&lt;/code&gt; is a material agreement.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;root_forms&lt;/code&gt;&lt;/strong&gt; — use this, not &lt;code&gt;form&lt;/code&gt;, when you want amendments grouped with originals (&lt;code&gt;8-K/A&lt;/code&gt; rolls up under &lt;code&gt;8-K&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;file_date&lt;/code&gt; vs &lt;code&gt;period_ending&lt;/code&gt;&lt;/strong&gt; — filing date vs the period the filing covers. For "what was disclosed today" you want &lt;code&gt;file_date&lt;/code&gt;; for fundamentals you want &lt;code&gt;period_ending&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;display_names&lt;/code&gt;&lt;/strong&gt; — a pre-formatted &lt;code&gt;Name (TICKER) (CIK …)&lt;/code&gt; string. Regex the ticker out instead of doing a second lookup.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;There's also a free bonus: every response carries an &lt;code&gt;aggregations&lt;/code&gt; block with &lt;code&gt;form_filter&lt;/code&gt;, &lt;code&gt;entity_filter&lt;/code&gt;, &lt;code&gt;sic_filter&lt;/code&gt;, and &lt;code&gt;biz_states_filter&lt;/code&gt; faceted counts — whether you asked for them or not. You can build a filings dashboard's sidebar without a single extra request.&lt;/p&gt;

&lt;h2&gt;
  
  
  A scraper that actually paginates
&lt;/h2&gt;

&lt;p&gt;Pagination is the one thing that trips people up. Each request returns at most 100 documents in &lt;code&gt;hits.hits&lt;/code&gt;; there's no &lt;code&gt;size&lt;/code&gt; parameter the backend honors past that. You walk the result set with &lt;code&gt;from&lt;/code&gt;, step by 100, and watch &lt;code&gt;hits.total.value&lt;/code&gt; for when to stop.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="n"&gt;EFTS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://efts.sec.gov/LATEST/search-index&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="n"&gt;HEADERS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;User-Agent&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;orthogonal-research max@orthogonal.info&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;search_all&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;q&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;startdt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;enddt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;max_results&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
    &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;max_results&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;params&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;q&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;q&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;from&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;   &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;forms&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;startdt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;startdt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;startdt&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;enddt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;   &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;enddt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;enddt&lt;/span&gt;

        &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;EFTS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;HEADERS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;hits&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;hits&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;hits&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;hits&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;break&lt;/span&gt;
        &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;extend&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hits&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;100&lt;/span&gt;
        &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;0.15&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  &lt;span class="c1"&gt;# stay under ~10 req/sec
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;

&lt;span class="n"&gt;filings&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;search_all&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'"&lt;/span&gt;&lt;span class="s"&gt;going concern&lt;/span&gt;&lt;span class="sh"&gt;"'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;10-K&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                     &lt;span class="n"&gt;startdt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;2026-01-01&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;enddt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;2026-06-01&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;filings&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;src&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;_source&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;src&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;file_date&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;src&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;form&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;src&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;display_names&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;time.sleep(0.15)&lt;/code&gt; keeps you under SEC's documented limit of ~10 requests/sec. Go faster and you'll get temporary IP blocks lasting about ten minutes. There's &lt;strong&gt;no &lt;code&gt;X-RateLimit&lt;/code&gt; header&lt;/strong&gt; to watch — the only signal is a sudden 403, so it's better to throttle up front than to detect and back off.&lt;/p&gt;

&lt;h2&gt;
  
  
  The gotchas that cost me time
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Phrase vs token search.&lt;/strong&gt; A bare &lt;code&gt;q=climate risk&lt;/code&gt; matches documents containing "climate" OR "risk" anywhere — that returned 40x more noise than I expected. The quoted form &lt;code&gt;q=%22climate risk%22&lt;/code&gt; is the exact phrase, and it's what you almost always want.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The 10,000-result ceiling.&lt;/strong&gt; Elasticsearch caps deep pagination. Once &lt;code&gt;from&lt;/code&gt; passes 10,000 the endpoint errors out. If a query has more hits than that, narrow it with a tighter date range and stitch the windows together — there's no scroll cursor exposed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Full-text only covers 2001 onward.&lt;/strong&gt; The index starts in 2001. Older filings exist in EDGAR but won't show up here; for pre-2001 you're back to the structured submissions API.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It indexes exhibits, not just the main doc.&lt;/strong&gt; A single 8-K can return several hits — one per attached exhibit. Dedupe on the accession number (&lt;code&gt;adsh&lt;/code&gt;) if you only want one row per filing.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Where this fits
&lt;/h2&gt;

&lt;p&gt;I use this as the front door for a couple of projects: a script that flags new 8-K filings mentioning specific risk language, and an insider-buying alerter that cross-references full-text hits against Form 4 data. The full-text endpoint &lt;em&gt;finds&lt;/em&gt; the filings; the structured EDGAR APIs &lt;em&gt;pull&lt;/em&gt; the details.&lt;/p&gt;

&lt;p&gt;I wrote up the full field-by-field decode of the &lt;code&gt;_source&lt;/code&gt; envelope (every key in a real &lt;code&gt;forms=8-K&lt;/code&gt; response) &lt;a href="https://orthogonal.info/sec-edgar-full-text-search-api-efts-python/" rel="noopener noreferrer"&gt;here&lt;/a&gt; if you want the complete reference.&lt;/p&gt;

&lt;p&gt;The whole thing is one undocumented GET request returning clean JSON — no key, no cost. What other "human-only" search boxes are quietly sitting on a clean JSON API? I keep finding them in network tabs.&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>tutorial</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Why Math.random() Is a Security Bug in Password Generators (and the Web Crypto Fix)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Thu, 11 Jun 2026 17:04:37 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/why-mathrandom-is-a-security-bug-in-password-generators-and-the-web-crypto-fix-3li4</link>
      <guid>https://dev.to/orthogonalinfo/why-mathrandom-is-a-security-bug-in-password-generators-and-the-web-crypto-fix-3li4</guid>
      <description>&lt;p&gt;Last week I was reviewing a small auth service and found this one-liner generating password-reset tokens:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;token&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Array&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;length&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;16&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;CHARS&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;floor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;random&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;CHARS&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;
&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;''&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It runs. It produces things like &lt;code&gt;xK9$mLp2@nQ7vR4w&lt;/code&gt;. It also happens to be a real security bug.&lt;/p&gt;

&lt;p&gt;That exact pattern is the one I deliberately avoided when I built a small browser-only password generator — and the reason is worth a few hundred words, because almost every "roll your own" password snippet on the web gets it wrong in the &lt;em&gt;same&lt;/em&gt; way. Here's what's broken about &lt;code&gt;Math.random()&lt;/code&gt; for secrets, the fix, and the two gotchas that bite people who try to fix it themselves.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;code&gt;Math.random()&lt;/code&gt; is predictable by design
&lt;/h2&gt;

&lt;p&gt;In V8 — the engine behind Chrome and Node — &lt;code&gt;Math.random()&lt;/code&gt; has used an algorithm called &lt;strong&gt;xorshift128+&lt;/strong&gt; since version 4.9.40 (late 2015). It has 128 bits of internal state, a period of 2^128 − 1, and it passes the TestU01 statistical suite. Statistically, the numbers &lt;em&gt;look&lt;/em&gt; random.&lt;/p&gt;

&lt;p&gt;But "looks random" and "unpredictable" are different properties.&lt;/p&gt;

&lt;p&gt;xorshift128+ is a &lt;strong&gt;pseudo&lt;/strong&gt;-random generator: every output is a deterministic function of that 128-bit state, and the state is recoverable. Feed enough consecutive outputs into a system of linear equations and you can solve for the internal state — there are public tools on GitHub that recover it from as few as &lt;strong&gt;64–128 consecutive&lt;/strong&gt; &lt;code&gt;Math.random()&lt;/code&gt; calls. Once an attacker has the state, every future output is known. Every "random" password you generate after that point is predictable.&lt;/p&gt;

&lt;p&gt;For a UI animation or a Monte Carlo sim, who cares. For a password, an API key, or a session token, that's the whole ballgame.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;code&gt;crypto.getRandomValues()&lt;/code&gt; is the actual fix
&lt;/h2&gt;

&lt;p&gt;Browsers ship a cryptographically secure RNG (CSPRNG) through the Web Crypto API. It pulls from the OS entropy pool (&lt;code&gt;/dev/urandom&lt;/code&gt; on Linux, &lt;code&gt;BCryptGenRandom&lt;/code&gt; on Windows) and is built so that observing past output tells you nothing about future output. There's no recoverable internal state to solve for.&lt;/p&gt;

&lt;p&gt;The core is four lines:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;secureRandom&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;max&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;arr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Uint32Array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="nx"&gt;crypto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getRandomValues&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;arr&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;arr&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="nx"&gt;max&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Read a fresh 32-bit unsigned integer from the CSPRNG, reduce it into the range you need, done. Swap &lt;code&gt;Math.random()&lt;/code&gt; for this and the prediction attack above is gone.&lt;/p&gt;

&lt;p&gt;But notice that &lt;code&gt;% max&lt;/code&gt; — that's gotcha number one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Gotcha 1: modulo bias is real (but size matters)
&lt;/h2&gt;

&lt;p&gt;When you take a random integer modulo your alphabet size, the ranges usually don't divide evenly, so some characters come up more often than others. I wanted to see how bad it actually is, so I generated &lt;strong&gt;6.2 million random bytes&lt;/strong&gt; and bucketed &lt;code&gt;byte % 62&lt;/code&gt; (a typical alphanumeric set):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;expected per character: 100,000&lt;/li&gt;
&lt;li&gt;lowest-frequency char: ~96,900 hits&lt;/li&gt;
&lt;li&gt;highest-frequency char: ~121,400 hits&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;ratio: 1.25&lt;/strong&gt; — a 25% skew&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It happens because &lt;code&gt;256 % 62 = 8&lt;/code&gt;, so byte values 0–7 each give one extra shot to the first eight characters.&lt;/p&gt;

&lt;p&gt;The textbook fix is &lt;strong&gt;rejection sampling&lt;/strong&gt;: throw away any byte in the biased tail and draw again. Rejecting values ≥ 248 dropped the skew to a 1.02 ratio in my test, at the cost of discarding about 3.1% of draws.&lt;/p&gt;

&lt;p&gt;But here's the part the "always use rejection sampling" advice skips: &lt;strong&gt;the bias depends entirely on how big your random integer is relative to the alphabet.&lt;/strong&gt; If you don't read a single byte but a full &lt;code&gt;Uint32&lt;/code&gt; (range 0 to ~4.29 billion), then for a 94-character symbol set, &lt;code&gt;Uint32 % 94&lt;/code&gt; makes the favored characters more likely by roughly &lt;strong&gt;1 part in 45 million&lt;/strong&gt; — a bias of 0.0000022%.&lt;/p&gt;

&lt;p&gt;For a password, that's noise far below anything that matters. So you can skip rejection sampling on purpose and keep the code simple, &lt;em&gt;because a 32-bit draw already makes the bias irrelevant&lt;/em&gt;. If you're minting cryptographic keys, add the rejection step; for human passwords, a wide draw is enough.&lt;/p&gt;

&lt;h2&gt;
  
  
  Gotcha 2: the 64KB quota wall
&lt;/h2&gt;

&lt;p&gt;The second surprise showed up while running that bias test. My first attempt asked &lt;code&gt;getRandomValues()&lt;/code&gt; to fill one big buffer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;crypto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getRandomValues&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Uint8Array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;620000&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
&lt;span class="c1"&gt;// QuotaExceededError: The requested length exceeds 65,536 bytes&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;getRandomValues()&lt;/code&gt; refuses any request over &lt;strong&gt;65,536 bytes (64 KB)&lt;/strong&gt; in a single call. It's in the spec and every browser enforces it. If you're generating one 16-character password you'll never hit it, but the moment you batch-generate or fill a large buffer, you have to chunk:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;fillSecure&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;buf&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nx"&gt;buf&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;65536&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;crypto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getRandomValues&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;buf&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;subarray&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;65536&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Undocumented in most tutorials, and a hard failure rather than a silent one — which is at least honest of it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why browser-only matters here
&lt;/h2&gt;

&lt;p&gt;A password generator that does the work &lt;strong&gt;server-side&lt;/strong&gt; is a service that has seen your password in plaintext. The only design that makes sense for a secret is to build it on the user's machine, from their OS entropy, so it never touches a network. Open dev tools, watch the Network tab while you click generate, and you should see exactly zero requests.&lt;/p&gt;

&lt;p&gt;If you want to poke at a working version, here's the &lt;a href="https://orthogonal.info/free-password-generator-online/" rel="noopener noreferrer"&gt;browser-only password generator&lt;/a&gt; I built around these exact decisions — everything runs client-side.&lt;/p&gt;

&lt;h2&gt;
  
  
  One layer is never enough
&lt;/h2&gt;

&lt;p&gt;A strong, truly-random password fixes the "guessable" problem. It does &lt;strong&gt;nothing&lt;/strong&gt; about phishing, reused credentials, or a leaked database. Generate unique passwords, store them in a real manager, and gate the important accounts with hardware 2FA. Three cheap layers beat one strong one.&lt;/p&gt;

&lt;p&gt;The lesson I keep relearning: in security, the code that "works" and the code that's &lt;em&gt;correct&lt;/em&gt; are often the same length and completely different.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;Math.random()&lt;/code&gt; works. &lt;code&gt;crypto.getRandomValues()&lt;/code&gt; is correct.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;How do you handle modulo bias in your own token/ID generators — always reject, or do you size the draw so it doesn't matter? Curious what others do in practice.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>javascript</category>
      <category>webdev</category>
      <category>node</category>
    </item>
    <item>
      <title>How to Compress Images From the Command Line (and in CI) — No Upload, No Account</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Tue, 09 Jun 2026 15:24:08 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/how-to-compress-images-from-the-command-line-and-in-ci-no-upload-no-account-481m</link>
      <guid>https://dev.to/orthogonalinfo/how-to-compress-images-from-the-command-line-and-in-ci-no-upload-no-account-481m</guid>
      <description>&lt;p&gt;Most "compress your images" advice ends with &lt;em&gt;"...now drag your files into this website."&lt;/em&gt; That's fine for a one-off. It's useless when you have a &lt;code&gt;/public/images&lt;/code&gt; folder with 300 PNGs, or a build step that should never ship a 4 MB hero image again.&lt;/p&gt;

&lt;p&gt;I wanted image compression that lives where the rest of my tooling lives: the terminal and CI. No upload, no account, no clicking. Here's the workflow I landed on, plus a tiny CLI I built to make it one command.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem with web-based compressors in a dev workflow
&lt;/h2&gt;

&lt;p&gt;TinyPNG, Squoosh, and friends are great tools. But in a real project they have three issues:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;They don't script.&lt;/strong&gt; You can't put "open a browser and drag files" in a &lt;code&gt;package.json&lt;/code&gt; or a GitHub Action.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;They upload.&lt;/strong&gt; For a lot of teams, shipping customer/product images to a third-party server is a non-starter.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;They're one-at-a-time-ish.&lt;/strong&gt; Batch + recursive folders + keeping structure is exactly the boring part you want automated.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;What you actually want: &lt;code&gt;compress ./images&lt;/code&gt; → done, locally, every time.&lt;/p&gt;

&lt;h2&gt;
  
  
  Option 1: raw &lt;code&gt;sharp&lt;/code&gt; in a script
&lt;/h2&gt;

&lt;p&gt;If you just want the engine, &lt;a href="https://sharp.pixelplumbing.com/" rel="noopener noreferrer"&gt;&lt;code&gt;sharp&lt;/code&gt;&lt;/a&gt; (libvips bindings) is the workhorse. A minimal batch script:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// compress.js&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nx"&gt;sharp&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;sharp&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;glob&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;glob&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;node:path&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nx"&gt;fs&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;node:fs/promises&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;files&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;glob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;images/**/*.{jpg,jpeg,png}&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;fs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;mkdir&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;dist&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;recursive&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;file&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;files&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;dist&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;basename&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;file&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;extname&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;file&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;.webp&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;sharp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;file&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resize&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1600&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;withoutEnlargement&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt; &lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;webp&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;quality&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;80&lt;/span&gt; &lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toFile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;✓&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This works. But you'll quickly want flags (quality, format, max-width), parallelism across cores, "don't enlarge," metadata stripping, dry-run, and preserved folder structure — and now you're maintaining a tool instead of shipping your app.&lt;/p&gt;

&lt;h2&gt;
  
  
  Option 2: a tiny CLI that already does all that
&lt;/h2&gt;

&lt;p&gt;So I packaged exactly that into a small, MIT-licensed CLI called &lt;strong&gt;QuickShrink&lt;/strong&gt;. It's a thin, well-tested wrapper over &lt;code&gt;sharp&lt;/code&gt;, focused on the batch-folder workflow. Run it once with &lt;code&gt;npx&lt;/code&gt;, no global install:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# compress every image in ./images → ./compressed&lt;/span&gt;
npx https://quickshrink.orthogonal.info/cli/quickshrink.tgz ./images
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Convert a whole folder to WebP and cap the width for web (the single most impactful thing you can do for page weight):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx https://quickshrink.orthogonal.info/cli/quickshrink.tgz ./photos &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-o&lt;/span&gt; ./web &lt;span class="nt"&gt;--format&lt;/span&gt; webp &lt;span class="nt"&gt;--max-width&lt;/span&gt; 1600 &lt;span class="nt"&gt;--quality&lt;/span&gt; 80
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Recurse into subfolders and keep the structure:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx https://quickshrink.orthogonal.info/cli/quickshrink.tgz ./assets &lt;span class="nt"&gt;-o&lt;/span&gt; ./out &lt;span class="nt"&gt;--recursive&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Preview before you touch anything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx https://quickshrink.orthogonal.info/cli/quickshrink.tgz ./photos &lt;span class="nt"&gt;--dry-run&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Typical output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;  ✓ out/hero.webp        1.38 MB → 42.7 KB  (-97%)
  ✓ out/sub/banner.webp  3.10 MB → 31.1 KB  (-99%)

Done: 2 ok, 0 failed.
Total: 4.47 MB → 73.8 KB  (saved 4.40 MB, 98.4%)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Flags it supports:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Flag&lt;/th&gt;
&lt;th&gt;Does&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;-o, --out &amp;lt;dir&amp;gt;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Output directory (default &lt;code&gt;./compressed&lt;/code&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--format &amp;lt;fmt&amp;gt;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;jpeg&lt;/code&gt; \&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--quality &amp;lt;1-100&amp;gt;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Encoder quality (default 80)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;--max-width&lt;/code&gt; / &lt;code&gt;--max-height&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Resize down, never up&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--recursive&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Walk subfolders&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--dry-run&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Show the plan, write nothing&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Everything runs &lt;strong&gt;locally&lt;/strong&gt; — your images never leave the machine. It uses all your CPU cores and strips metadata by default.&lt;/p&gt;

&lt;h2&gt;
  
  
  Option 3: wire it into CI
&lt;/h2&gt;

&lt;p&gt;Because it's a single command, dropping it into a GitHub Action is trivial. Here's a step that compresses everything under &lt;code&gt;public/images&lt;/code&gt; and fails loudly if compression errors:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# .github/workflows/images.yml&lt;/span&gt;
&lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Compress images&lt;/span&gt;
&lt;span class="na"&gt;on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;pull_request&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;paths&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;public/images/**"&lt;/span&gt;&lt;span class="pi"&gt;]&lt;/span&gt;

&lt;span class="na"&gt;jobs&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;shrink&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;runs-on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ubuntu-latest&lt;/span&gt;
    &lt;span class="na"&gt;steps&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/checkout@v4&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/setup-node@v4&lt;/span&gt;
        &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;{&lt;/span&gt; &lt;span class="nv"&gt;node-version&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="nv"&gt;20&lt;/span&gt; &lt;span class="pi"&gt;}&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Compress images&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
          &lt;span class="s"&gt;npx -y https://quickshrink.orthogonal.info/cli/quickshrink.tgz \&lt;/span&gt;
            &lt;span class="s"&gt;./public/images -o ./public/images --format webp --max-width 1600&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Commit if changed&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
          &lt;span class="s"&gt;git config user.name "image-bot"&lt;/span&gt;
          &lt;span class="s"&gt;git config user.email "bot@users.noreply.github.com"&lt;/span&gt;
          &lt;span class="s"&gt;git add -A&lt;/span&gt;
          &lt;span class="s"&gt;git diff --cached --quiet || git commit -m "chore: compress images"&lt;/span&gt;
          &lt;span class="s"&gt;git push&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now nobody on the team can accidentally ship a 4 MB screenshot again. The bot quietly WebP's and resizes on every PR that touches images.&lt;/p&gt;

&lt;h2&gt;
  
  
  A package.json shortcut
&lt;/h2&gt;

&lt;p&gt;For local use, alias it so teammates don't need to remember the URL:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"scripts"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"images"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"npx -y https://quickshrink.orthogonal.info/cli/quickshrink.tgz ./src/assets -o ./public/img --format webp --max-width 1600"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;npm run images&lt;/code&gt; and you're done.&lt;/p&gt;

&lt;h2&gt;
  
  
  What about serverless / "I can't install native deps"?
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;sharp&lt;/code&gt; ships prebuilt binaries, so the CLI works in most CI runners and locally. The one place it gets awkward is constrained serverless functions or runtimes where native libs are a pain. For that case I'm building a small &lt;strong&gt;hosted compression API&lt;/strong&gt; (key-based, metered) so you can &lt;code&gt;POST&lt;/code&gt; an image and get bytes back without bundling libvips. It's in private beta — if that's your use case, there's a note + email on the &lt;a href="https://quickshrink.orthogonal.info/cli/" rel="noopener noreferrer"&gt;CLI page&lt;/a&gt; and I'd genuinely like the feedback on what limits/pricing make sense.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prefer a GUI for one-offs?
&lt;/h2&gt;

&lt;p&gt;For the occasional "just shrink this one screenshot" moment, there's a &lt;a href="https://quickshrink.orthogonal.info/" rel="noopener noreferrer"&gt;browser version&lt;/a&gt; that does the same thing client-side (the compression runs in your browser via Canvas — also no upload). But for anything repeatable, the CLI is the move.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx https://quickshrink.orthogonal.info/cli/quickshrink.tgz ./images &lt;span class="nt"&gt;--format&lt;/span&gt; webp &lt;span class="nt"&gt;--max-width&lt;/span&gt; 1600
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Local, scriptable, batch, free, MIT. That's the whole pitch. If you put it in CI, I'd love to hear how it goes — and what flag you wish it had next.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>javascript</category>
      <category>devops</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Your Online SQL Formatter Might Be Logging Your Database Password</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Thu, 04 Jun 2026 17:03:46 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/your-online-sql-formatter-might-be-logging-your-database-password-3n9p</link>
      <guid>https://dev.to/orthogonalinfo/your-online-sql-formatter-might-be-logging-your-database-password-3n9p</guid>
      <description>&lt;p&gt;Last month I watched a contractor paste a full Kubernetes secret manifest — base64 blobs and all — into the first "free YAML validator" that came up on Google. He just wanted to check the indentation. What he actually did was POST a production database password to a server he'd never heard of, run by people he'll never meet, with a privacy policy he didn't read.&lt;/p&gt;

&lt;p&gt;That's the part of online dev tools nobody talks about.&lt;/p&gt;

&lt;p&gt;A SQL formatter, a YAML validator, a JSON beautifier — they &lt;em&gt;feel&lt;/em&gt; disposable, like a calculator. But a huge number of them send whatever you paste to a backend for processing. If that paste contains a connection string, an API key, or a customer record, you just leaked it. No breach required. You handed it over.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why "format my SQL" is a data exfiltration path
&lt;/h2&gt;

&lt;p&gt;Here's the mechanic. Server-side tools work like this: your text goes into a &lt;code&gt;&amp;lt;textarea&amp;gt;&lt;/code&gt;, JavaScript fires an HTTP request to &lt;code&gt;/api/format&lt;/code&gt;, the server runs the actual formatting, and the result comes back. Simple to build, which is exactly why so many sites do it that way.&lt;/p&gt;

&lt;p&gt;The problem is what travels in that request body.&lt;/p&gt;

&lt;p&gt;I tested a handful of popular online formatters with my browser's Network tab open. Several of them sent the &lt;strong&gt;entire input payload&lt;/strong&gt; to their own domain. One sent it to a third-party API. The query I pasted was harmless test data, but the request was real — my text left my machine.&lt;/p&gt;

&lt;p&gt;Now picture the realistic version. You're debugging a failing migration at 11pm. You copy the offending query straight out of your ORM logs to "just clean it up." That query has a hardcoded credential a teammate left in six months ago. You paste, you format, you move on.&lt;/p&gt;

&lt;p&gt;The credential is now in someone's request logs, maybe their analytics, maybe an LLM training pipeline if the tool resells data. You will never know.&lt;/p&gt;

&lt;p&gt;This isn't paranoia. It's the same threat model that makes pasting code into random pastebins a fireable offense at most security-conscious shops. We just don't apply it to "format" tools because they feel too small to matter.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to actually verify a tool is client-side
&lt;/h2&gt;

&lt;p&gt;Don't take any tool's word for it — including the one I'm about to mention. Verifying is a two-minute job and every developer should know how.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Watch the Network tab.&lt;/strong&gt; Open DevTools (F12), go to the Network panel, clear it, then paste your text and hit format. If you see a new XHR/fetch request fire with your input in the payload, the tool is server-side. If nothing happens on the network, the work is local.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight http"&gt;&lt;code&gt;&lt;span class="err"&gt;// What a server-side formatter looks like in the Network tab:
POST /api/format-sql
Request Payload:
{ "query": "SELECT * FROM users WHERE token='sk_live_...'" }

// What a client-side tool looks like:
// (nothing — no request fires when you click format)
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;2. Kill your connection.&lt;/strong&gt; The bluntest test there is. Load the page, then turn off Wi-Fi or drop into airplane mode. If the tool still formats your text, it's running entirely in the browser. If it spins or errors, it needed a server. I do this with any tool before I trust it with anything sensitive.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Check for a service worker.&lt;/strong&gt; Truly offline-capable tools register a service worker so they work with no connection at all. In DevTools, look under &lt;strong&gt;Application → Service Workers&lt;/strong&gt;. Its presence is a strong signal the developer designed for offline-first, which usually means client-side processing too.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where this fits in a real workflow
&lt;/h2&gt;

&lt;p&gt;A few concrete cases where I reach for browser-only tools &lt;em&gt;specifically&lt;/em&gt; because of the data:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Reviewing a teammate's config PR.&lt;/strong&gt; Diffing two Helm values files that contain registry credentials — done locally, nothing logged anywhere.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cleaning up a query from prod logs.&lt;/strong&gt; Format it to read it, without shipping whatever sensitive &lt;code&gt;WHERE&lt;/code&gt; clause it carries to a stranger's server.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Validating a CI secrets file.&lt;/strong&gt; Checking that a GitHub Actions YAML parses before you commit, without exposing the encrypted values to a validation API.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;On a locked-down network.&lt;/strong&gt; Some client environments block external dev-tool domains entirely. Offline-capable tools just keep working.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The broader point: treat every "paste your text here" box as a potential outbound network call until you've proven otherwise. Most of the time it's fine. The one time it isn't, it's a leaked credential you can't un-leak.&lt;/p&gt;

&lt;h2&gt;
  
  
  The structural fix
&lt;/h2&gt;

&lt;p&gt;The fix is structural, not procedural. Don't rely on &lt;em&gt;remembering&lt;/em&gt; to scrub secrets first — use tools that physically can't send your data anywhere, because all the work happens in your tab.&lt;/p&gt;

&lt;p&gt;That's the reason I built &lt;a href="https://orthogonal.info/free-sql-formatter-online/" rel="noopener noreferrer"&gt;our formatters&lt;/a&gt; as single-file, client-side apps — a &lt;a href="https://orthogonal.info/free-sql-formatter-online/" rel="noopener noreferrer"&gt;SQL Formatter&lt;/a&gt;, a &lt;a href="https://orthogonal.info/free-yaml-validator-formatter-online/" rel="noopener noreferrer"&gt;YAML Validator&lt;/a&gt;, and a &lt;a href="https://orthogonal.info/free-diff-checker-online/" rel="noopener noreferrer"&gt;Diff Checker&lt;/a&gt; where the parsing runs in JavaScript on your device. There is no &lt;code&gt;/api/format&lt;/code&gt; endpoint. The text in your textarea never crosses the network because there's nowhere for it to go.&lt;/p&gt;

&lt;p&gt;But browser-only tools only remove &lt;em&gt;one&lt;/em&gt; exfiltration path. Defense in depth still applies: rotate the credentials that have already been pasted into who-knows-what, and put a &lt;a href="https://orthogonal.info/i-caught-14-leaked-secrets-in-my-git-history-heres-the-pre-commit-setup-that-stops-it/" rel="noopener noreferrer"&gt;pre-commit secret scanner&lt;/a&gt; in front of your repos so the hardcoded ones never ship in the first place.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;How do you handle this on your team — do you have a policy on pasting into online dev tools, or is it the wild west? Curious what threat models other people apply here.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>devops</category>
      <category>webdev</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>I Made an Image Compressor That Never Sees Your Images (100% Client-Side)</title>
      <dc:creator>Max</dc:creator>
      <pubDate>Fri, 29 May 2026 15:02:23 +0000</pubDate>
      <link>https://dev.to/orthogonalinfo/i-made-an-image-compressor-that-never-sees-your-images-100-client-side-309e</link>
      <guid>https://dev.to/orthogonalinfo/i-made-an-image-compressor-that-never-sees-your-images-100-client-side-309e</guid>
      <description>&lt;p&gt;Ever notice how most "free" image compressors upload your files to their servers?&lt;/p&gt;

&lt;p&gt;I got fed up with this — especially when compressing screenshots that might contain sensitive data — so I built &lt;strong&gt;QuickShrink&lt;/strong&gt;: an image compressor that runs entirely in your browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why client-side matters
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Privacy: Your images never leave your device&lt;/li&gt;
&lt;li&gt;Speed: No upload/download round-trip. Compression is instant&lt;/li&gt;
&lt;li&gt;Works offline: It's a PWA. Install it and use it without internet&lt;/li&gt;
&lt;li&gt;No account needed: Just drag, drop, done&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Compress JPEG/PNG/WebP with adjustable quality (1-100)&lt;/li&gt;
&lt;li&gt;Batch compress multiple images at once&lt;/li&gt;
&lt;li&gt;Convert between formats (PNG to WebP for 60-80% size reduction)&lt;/li&gt;
&lt;li&gt;Resize images with aspect ratio lock&lt;/li&gt;
&lt;li&gt;See before/after comparison with file sizes&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The tech
&lt;/h2&gt;

&lt;p&gt;Built with vanilla JavaScript + Canvas API + OffscreenWorker for non-blocking compression. The entire app is about 50KB. No React, no framework, no build step.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://quickshrink.orthogonal.info" rel="noopener noreferrer"&gt;quickshrink.orthogonal.info&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Free tier: 5 compressions/day&lt;br&gt;
Pro: Unlimited + batch + format conversion ($4.99 one-time)&lt;/p&gt;

&lt;p&gt;I'd love feedback on compression quality vs file size. Is the default quality slider (80%) a good default for web images?&lt;/p&gt;

&lt;p&gt;Built this as a weekend project after TinyPNG's free tier dropped to 20 images/month. If you're compressing screenshots or blog images regularly, give it a shot.&lt;/p&gt;

</description>
      <category>showdev</category>
      <category>webdev</category>
      <category>javascript</category>
      <category>privacy</category>
    </item>
  </channel>
</rss>
