DEV Community: Mahdyar

How I Built Secret-Scrub: A Zero-Dependency Pre-Commit Secret Scanner with Shannon Entropy Analysis

Mahdyar — Mon, 21 Sep 2026 23:20:53 +0000

Every developer has had that split-second wave of cold panic:

You type git commit -m "update config" && git push, and a second later you realize:
"Wait... did I just push my .env file or hardcoded AWS secret key to a public GitHub repo?"

Even if you delete the commit or force-push, git history scanners and malicious scrapers detect leaked credentials within seconds of hitting GitHub. Once an API key is leaked, it must be rotated immediately.

To protect codebases before secrets ever leave your local machine, I built Secret-Scrub — a blazing-fast, zero-dependency Node.js CLI scanner with a 1-click Git pre-commit hook installer and Shannon entropy analysis.

Architecture: Dual-Layer Detection Engine

Regex alone is never enough for secret scanning. Vendor formats change, and raw credentials don't always carry known prefixes. Secret-Scrub combines two complementary inspection layers:

[Target File / Staged Diff]
            │
            ▼
┌──────────────────────────────────────────────┐
│ Layer 1: High-Fidelity Signature Patterns     │
│ └── AWS (AKIA...), GitHub (ghp_...), Stripe,  │
│     OpenAI (sk-...), Private Keys (RSA/SSH)  │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────┐
│ Layer 2: Shannon Entropy Analysis             │
│ └── H(X) = -Σ P(x) * log2(P(x))               │
│     Calculates character randomness to catch │
│     unstructured API keys & high-entropy pwds│
└──────────────────────┬───────────────────────┘
                       │
                       ▼
        [Exit Code 1: Block Git Commit]

1. Vendor Signature Matching (18+ Cloud Providers)

Scans for well-known token formats including:

AWS Access Keys (AKIA...)
GitHub Personal Access Tokens (ghp_..., gho_...)
Stripe Secret & Restricted Keys (sk_live_..., rk_live_...)
OpenAI API Keys (sk-...)
Slack Webhooks, Google API Keys, JWT tokens, and PEM Private Keys.

2. Shannon Entropy Math for Unknown Secrets

Attackers and developers frequently use unstructured high-randomness tokens that have no vendor prefix. Secret-Scrub measures information entropy:

$$H(X) = -\sum_{i=1}^n P(x_i) \log_2 P(x_i)$$

High entropy strings (e.g., dGhpcy1pcy1hLXJhbmRvbS1zZWNyZXQ=) score high randomness and are flagged even without a pattern match, while normal camelCase code variables score low and are ignored.

Git Workflow Integration (`--staged`)

Scanning thousands of files on every commit slows down developers. Secret-Scrub solves this with --staged mode:

npx secret-scrub --staged

Instead of scanning entire directories, it queries git diff --cached and only inspects files currently staged for commit. This takes less than 40 milliseconds, making it imperceptible in day-to-day coding.

1-Click Pre-Commit Hook Installer

You don't need complex external hook runners. You can install native Git protection directly:

npx secret-scrub install-hook

This automatically configures .git/hooks/pre-commit. If anyone on your team attempts to commit a secret, the commit is automatically aborted locally before it can be pushed.

Quick Start

Test your current directory or staged commits immediately:

# Scan whole directory
npx secret-scrub .

# Scan only staged git files
npx secret-scrub --staged

# Machine-readable JSON output for CI pipelines
npx secret-scrub . --format json

Source Code & Community

Secret-Scrub is 100% open-source under the MIT license.

🔗 GitHub Repository: github.com/mahdyarmonfared/secret-scrub

I'd love to hear your thoughts:

What provider signatures or secret formats would you like added?
How does your team enforce secret prevention before CI/CD?

If you find this tool helpful for securing your repositories, feel free to drop a ⭐ on GitHub!

How I Built QuickShare-QR: Share Files from Terminal to Phone via Instant ASCII QR Codes

Mahdyar — Mon, 21 Sep 2026 23:17:28 +0000

Every developer encounters this awkward friction almost every week:

You have an APK file, a configuration snippet, an invoice PDF, or a test video on your development machine, and you need to send it to your phone right now.

What do most of us do?

Email the file to ourselves.
Send it over a Telegram "Saved Messages" chat.
Upload it to Google Drive or Dropbox and wait for cloud syncing.

All these methods upload your local files to external third-party cloud servers just to travel 30 centimeters across your desk over the same local Wi-Fi router.

To eliminate this friction, I built QuickShare-QR — a zero-setup Node.js CLI tool that spins up a secure, ephemeral local HTTP server and prints a scannable ASCII QR code directly into your terminal.

How it Works

[Developer Machine]                                   [Smartphone / Tablet]
       │                                                       │
$ npx quickshare-qr test.pdf                                   │
       │                                                       │
       ├── 1. Discovers Local LAN IP (192.168.1.X)             │
       ├── 2. Spins up Ephemeral HTTP Server on random port    │
       └── 3. Renders ANSI QR Code directly in Terminal        │
                       │                                       │
                       └────────── [Scan with Camera] ─────────┤
                                                               │
                                         Direct LAN Streaming Download
                                        (No app installation needed!)

1. Zero Cloud, Zero External Servers

The transfer happens 100% peer-to-peer over your local Wi-Fi / LAN network. Files never leave your local router. Speed is limited only by your local Wi-Fi bandwidth (often 30–80 MB/s).

2. Native Camera Scanning (No Client App Required)

Because QuickShare-QR serves standard HTTP with proper Content-Disposition headers, the receiver does not need to install any app.
Just point the phone's default camera at the terminal QR code, tap the notification, and the browser immediately begins downloading.

3. Direct Streaming with Ephemeral Auto-Shutdown

The CLI streams files chunk-by-chunk using Node.js read streams:

import http from 'node:http';
import fs from 'node:fs';

const server = http.createServer((req, res) => {
  res.writeHead(200, {
    'Content-Type': 'application/octet-stream',
    'Content-Disposition': `attachment; filename="${fileName}"`,
    'Content-Length': fileStat.size,
  });

  const fileStream = fs.createReadStream(filePath);
  fileStream.pipe(res);

  fileStream.on('end', () => {
    console.log('Transfer complete! Shutting down server...');
    setTimeout(() => server.close(), 1000);
  });
});

As soon as the file finishes downloading, the server gracefully shuts down and releases the port.

Security & Path Traversal Protection

Allowing devices on the local network to fetch files requires strict security boundaries:

Path Confinement: Prevents directory traversal attacks (../) by strictly binding the HTTP route to the single selected target file.
Dynamic Ephemeral Ports: Binds to arbitrary available ports to prevent conflict and sniffing.
Optional Single-Use Tokens: Guarantees that only the device that scanned the terminal QR code can initiate the download.

Quick Start in 5 Seconds

You don't need to install anything permanently:

# Share any file instantly over LAN
npx quickshare-qr ./my-presentation.pdf

# Share an archive or photo
npx quickshare-qr ./assets.zip

Point your phone's camera at your terminal screen, and the transfer starts immediately!

Source Code & Community

QuickShare-QR is open source under the MIT license.

🔗 GitHub Repository: github.com/mahdyarmonfared/quickshare-qr

I'd love your feedback:

Would you like support for receiving files (uploading from phone back to terminal)?
What other local sharing protocols or encryption layers would make this even better?

Feel free to open an issue or drop a star ⭐ on GitHub!

How I Built HashDup: A Fast, Memory-Safe Duplicate File Finder CLI in Node.js

Mahdyar — Mon, 21 Sep 2026 23:14:15 +0000

Over time, every workstation accumulates duplicate files: duplicated photo backups, repeated downloads of large zip archives, cloned repositories, and redundant ISOs. They quietly eat gigabytes of expensive SSD storage.

Most people write a simple script that reads all files and compares their hashes. But on large drives, that naive approach has two fatal problems:

Performance bottleneck: Hashing thousands of files wastes CPU and disk I/O when 90% of files have unique sizes and can never be duplicates.
Memory crashes (OOM): Reading multi-gigabyte files into memory (fs.readFileSync) crashes the Node.js event loop with out-of-memory errors.

To solve this efficiently, I built HashDup — a fast, memory-safe CLI duplicate finder built with native Node.js streams and a smart 2-phase scanning engine.

The 2-Phase Scanning Architecture

HashDup uses an algorithmic filtering pipeline that avoids unnecessary disk reads:

[All Target Files]
       │
       ▼
Phase 1: Size-First Grouping (O(1) filter)
  └── Discard all files with unique byte sizes without reading contents!
       │
       ▼ (Only files with identical sizes remain)
Phase 2: Chunked Streaming SHA-256 Hashing
  └── Streams files in 64KB buffers (Flat O(1) memory consumption)
       │
       ▼
[Identified Duplicate Sets & Reclaimable Space]

1. Phase 1: Size-First Filtering

Before reading any file content, HashDup runs a lightweight metadata scan using fs.promises.stat().

If a file has a unique size in bytes, it is physically impossible for it to be a duplicate of any other file. We discard these immediately. On a drive with 10,000 files, this eliminates 90%–95% of candidates in milliseconds!

2. Phase 2: Memory-Safe Streaming SHA-256

For candidate files that share the exact same byte size, we compute cryptographic SHA-256 digests.

Instead of buffering whole files into RAM, HashDup uses Node.js streams:

import fs from 'node:fs';
import crypto from 'node:crypto';

function hashFile(filePath) {
  return new Promise((resolve, reject) => {
    const hash = crypto.createHash('sha256');
    const stream = fs.createReadStream(filePath);

    stream.on('data', chunk => hash.update(chunk));
    stream.on('end', () => resolve(hash.digest('hex')));
    stream.on('error', reject);
  });
}

This ensures that whether a file is 2 KB or 10 GB, Node.js memory usage stays constant at around 30MB of RAM.

Safety & Reporting Features

Waste Analytics: Displays exact duplicate groupings, file paths, and calculates total reclaimable space (MB/GB).
Dry-Run by Default: Inspect duplicates in a clean CLI table before touching or deleting anything.
Zero Heavy Dependencies: Built entirely with Node.js built-ins (crypto, stream, fs/promises).

Quick Start

You can test it right now on any directory using npx:

# Scan current directory and preview duplicates
npx hashdup-cli scan .

# Scan with custom output format or dry-run
npx hashdup-cli scan ~/Downloads --min-size 1MB

Source Code & Community Feedback

HashDup is open source under the MIT license.

🔗 GitHub Repository: github.com/mahdyarmonfared/hashdup-cli

I'd love to hear from other CLI developers:

How do you handle file comparisons on massive datasets (e.g., partial hashing vs full SHA-256)?
Would you find an interactive terminal cleanup wizard (TUI) useful for safe one-click deletion?

Feedback, PRs, and ⭐ stars on GitHub are always appreciated!

How I Built CleanDrop: A Safe, Zero-Dependency CLI to Tame Your Downloads Folder

Mahdyar — Mon, 21 Sep 2026 23:09:54 +0000

Every developer’s Downloads folder eventually becomes a chaotic digital landfill. Screenshots, random PDF receipts, compressed archives, test datasets, and CLI binaries pile up until finding anything becomes a chore.

Like most engineers, I initially relied on ad-hoc Bash one-liners. But quick scripts have major flaws:

They fail silently or overwrite files when file names collide.
One wrong flag or typo can accidentally move or delete critical files.
There is no rollback mechanism if you change your mind.

To solve this once and for all, I built CleanDrop — a lightweight, zero-dependency Node.js CLI utility designed to organize cluttered folders safely, predictably, and reversibly.

Key Design Principles

When designing CleanDrop, developer safety and ergonomics were the top priorities:

1. Dry-Run by Default Mentality (`--dry-run`)

Moving hundreds of files without knowing where they will land is terrifying. CleanDrop allows you to simulate the entire sorting process first:

npx cleandrop --dry-run

It prints an exact audit table of source paths, target categories, and planned file actions without touching disk storage.

2. Built-in Undo Support (Atomic Transaction Log)

Accidents happen. CleanDrop logs batch actions to a local history journal. If you run a sort and realize you wanted to keep certain files in place, you can reverse the operation with a single command:

npx cleandrop --undo

3. Non-Destructive Conflict Resolution

CleanDrop never silently overwrites an existing file. If invoice.pdf already exists in Documents/, the tool renames the incoming file safely (e.g., invoice (1).pdf), ensuring zero data loss.

4. Zero Heavy Dependencies

The entire CLI is powered by native Node.js filesystem APIs (fs/promises, path). It starts instantly and doesn't require bloated npm dependency trees.

How it Categorizes Files

CleanDrop inspects file extensions and maps them into clean, logical target folders:

Category	Extensions Handled
Documents	`.pdf`, `.docx`, `.txt`, `.xlsx`, `.pptx`, `.csv`, `.md`
Images	`.png`, `.jpg`, `.jpeg`, `.gif`, `.svg`, `.webp`
Archives	`.zip`, `.tar.gz`, `.rar`, `.7z`, `.tar`
Code & Data	`.js`, `.ts`, `.py`, `.json`, `.sql`, `.html`
Media	`.mp4`, `.mov`, `.mp3`, `.wav`, `.mkv`

System files, hidden dotfiles (.DS_Store, .git), and already organized subdirectories are safely ignored.

Quick Start in 5 Seconds

You don't even need to install it globally. Run it directly with npx:

# 1. Preview changes first
npx cleandrop --dry-run

# 2. Run the organization
npx cleandrop

Source Code & Contributing

CleanDrop is 100% open-source under the MIT license.

🔗 GitHub Repository: github.com/mahdyarmonfared/cleandrop

I would love to get your thoughts:

What safety features do you look for in automated file management tools?
What file types or custom rule options would you like to see next?

If you find it useful, feel free to drop a star ⭐ on GitHub or open an issue with your suggestions!

DEV Community: Mahdyar

How I Built Secret-Scrub: A Zero-Dependency Pre-Commit Secret Scanner with Shannon Entropy Analysis

Architecture: Dual-Layer Detection Engine

1. Vendor Signature Matching (18+ Cloud Providers)

2. Shannon Entropy Math for Unknown Secrets

Git Workflow Integration (--staged)

1-Click Pre-Commit Hook Installer

Quick Start

Source Code & Community

How I Built QuickShare-QR: Share Files from Terminal to Phone via Instant ASCII QR Codes

How it Works

1. Zero Cloud, Zero External Servers

2. Native Camera Scanning (No Client App Required)

3. Direct Streaming with Ephemeral Auto-Shutdown

Security & Path Traversal Protection

Quick Start in 5 Seconds

Source Code & Community

How I Built HashDup: A Fast, Memory-Safe Duplicate File Finder CLI in Node.js

The 2-Phase Scanning Architecture

1. Phase 1: Size-First Filtering

2. Phase 2: Memory-Safe Streaming SHA-256

Safety & Reporting Features

Quick Start

Source Code & Community Feedback

How I Built CleanDrop: A Safe, Zero-Dependency CLI to Tame Your Downloads Folder

Key Design Principles

1. Dry-Run by Default Mentality (--dry-run)

2. Built-in Undo Support (Atomic Transaction Log)

3. Non-Destructive Conflict Resolution

4. Zero Heavy Dependencies

How it Categorizes Files

Quick Start in 5 Seconds

Source Code & Contributing

Git Workflow Integration (`--staged`)

1. Dry-Run by Default Mentality (`--dry-run`)