DEV Community

Cover image for DotEnvy Aegis: Building a 4-Layer AI Secret Detection Pipeline for VS Code
freerave
freerave

Posted on

DotEnvy Aegis: Building a 4-Layer AI Secret Detection Pipeline for VS Code

#vscode #security #typescript #python

How I architected a multi-layer secret scanner combining regex, truncated composite community hashing, Shannon entropy gating, and a 35-feature neural classifier — cutting LLM overhead by ~80%.

TL;DR: DotEnvy Aegis introduces a 4-layer secret detection pipeline (Regex → Truncated Composite Community Blacklist → Shannon Entropy Gate → Neural Context Classifier), migrates encryption keys to the OS keychain, auto-authorizes backups with project master keys, and syncs hashes to an anti-poisoning community blacklist on Railway — all without transmitting raw secret payloads.

What is DotEnvy?

DotEnvy is an open-source VS Code extension designed to bring enterprise-grade security to local .env file management:

  • Encrypted Variable Management (AES-256-GCM, PBKDF2 with 310,000 iterations)
  • Multi-User Envelope Encryption (each teammate decrypts using individual credentials)
  • Cloud Synchronization (native Doppler, Vercel, and AWS integrations)
  • AI-Powered Secret Detection (the focus of this article)
  • Git Pre-Commit Hooks (blocking credential leaks before commits land in git history)
  • Environment Analytics, Timeline, & Diff panels

The extension communicates with Aegis (python-llm), a custom Python microservice deployed on Railway that runs contextual neural classification on candidate secrets.

The Problem: One Layer is Never Enough

In our earliest experimental prototype, secret detection was a blunt instrument: every candidate token triggered an HTTP request to the backend model.

This naive architecture fell apart under real-world development workloads:

  1. Latency Overhead: Every network round-trip added 200–450ms of latency, quickly degrading editor responsiveness during live typing.
  2. Availability Dependency: If the network dropped or the backend restarted, secret detection failed silently.
  3. The False Positive/Negative Dilemma:
    • A high-entropy random string (like an image hash or UUID) looks like a secret to entropy analyzers, but is harmless.
    • A low-entropy or structured token looks harmless to statistical math, but may grant full database admin rights.

Context matters. Regex patterns matter. Math matters. Community intelligence matters.

Aegis solves this by decoupling detection into an L1 → L4 pipeline: four independent, complementary layers where each layer filters out noise before invoking the more expensive layer downstream.

The L1–L4 Architecture 

Candidate: (secretValue, contextLine, variableName)
                     │
                     ▼
┌──────────────────────────────────────────┐
│  L1: Regex Pattern Engine                │ ← Sub-millisecond, in-process
│  Matches well-known vendor prefixes      │   High-confidence match → HIGH
└────────────────────┬─────────────────────┘
                     │ no match
                     ▼
┌──────────────────────────────────────────┐
│  L2: Community Blacklist Cache           │ ← O(1) in-memory Set<string>
│  Truncated composite hash lookup         │   Known leaked hash → HIGH
└────────────────────┬─────────────────────┘
                     │ not in blacklist
                     ▼
┌──────────────────────────────────────────┐
│  L3: Shannon Entropy Gate                │ ← Pure mathematical calculation
│  Normalized entropy < 3.5 → LOW (skip)   │   Filters ~80% of candidate tokens
└────────────────────┬─────────────────────┘
                     │ entropy ≥ 3.5
                     ▼
┌──────────────────────────────────────────┐
│  L4: Contextual Neural Classifier        │ ← Remote FastAPI service
│  35-feature vector + attention layers    │   Deep semantic risk analysis
└──────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

In our current workload benchmarks across typical codebases, approximately 20% of extracted candidates reach L4. The remaining ~80% are resolved in-memory with sub-millisecond execution times, saving substantial network bandwidth and cloud compute.

Layer 1: In-Process Regex Engine (Instant Confidence)

Known secret formats (like AWS access keys, Stripe live tokens, and GitHub personal access tokens) follow strict, deterministic grammars. Running an AI model to identify a string that starts with AKIA followed by 16 alphanumeric characters is wasteful.

const KNOWN_SECRET_PATTERNS = [
    { name: 'AWS Access Key',  regex: new RegExp(['A', 'KIA', '[0-9A-Z]{16}'].join('')) },
    { name: 'Stripe Live Key', regex: new RegExp(['sk', '_live_', '[0-9a-zA-Z]{24,}'].join('')) },
    { name: 'GitHub Token',    regex: new RegExp(['g', 'hp_', '[a-zA-Z0-9]{36}'].join('')) },
    { name: 'Google API Key',  regex: new RegExp(['AI', 'za', '[0-9A-Za-z\\-_]{35}'].join('')) },
];
Enter fullscreen mode Exit fullscreen mode

An L1 match immediately assigns high risk, bypasses all subsequent layers, and triggers an asynchronous hash report to the community blacklist.

Layer 2: Community Blacklist via Truncated Composite Hashing

When a secret is identified and verified across the community, other developers should benefit instantly without having to re-analyze it through the LLM. However, sharing leaked secrets raises an immediate privacy question: How do you cross-reference secrets without transmitting sensitive values over the network?

L2 utilizes truncated composite hashing:

public hashEntry(variableName: string, value: string): string {
    // Only capture the variable name and the first 8 characters of the value
    const prefix = value.slice(0, 8);
    return crypto
        .createHash('sha256')
        .update(`${variableName}:${prefix}`)
        .digest('hex')
        .substring(0, 16);
}
Enter fullscreen mode Exit fullscreen mode

Privacy Nuance & Threat Model

We deliberately avoid claiming that this hashing scheme is an "absolute privacy shield":

  • What it accomplishes: The full secret is never transmitted across the wire or stored in community databases.
  • The engineering trade-off: For very short secrets or known formats, truncated prefixes could theoretically be susceptible to targeted dictionary guessing. We treat this composite hash as an effective deduplication fingerprint for collaborative defense, not a one-way cryptographic commitment for sensitive data.

On extension activation, DotEnvy pulls the promoted blacklist hashes into an in-memory Set<string>. Any candidate lookup is an $O(1)$ memory check with zero network overhead.

Server-Side Anti-Poisoning Architecture

To prevent malicious clients from poisoning the blacklist or whitelisting actual credentials, submissions must pass a server-side reputation pipeline:

Client Submits Hash
        │
        ▼
Staging Queue (Requires 3 distinct environments + reputation weighting)
        │
        ▼
Server-Side Verification (Backend re-evaluates risk on the pattern)
        │
        ▼
Promoted to Community Blacklist
Enter fullscreen mode Exit fullscreen mode

Threat Model Note: The 3-environment quorum and historical accuracy weighting are designed to raise the operational cost of poisoning attacks, rather than claiming Byzantine Sybil-proof consensus. High-accuracy reporters gain vote weight, while repeated false reporters are banned.

Layer 3: Shannon Entropy Gate

Shannon entropy quantifies the unpredictability and information density of a string:

$$H(X) = -\sum_{i=1}^{n} P(x_i) \log_2 P(x_i)$$

In our feature extraction pipeline, Shannon entropy is calculated and normalized into a standard feature scalar:

const features = this.extractFeatures(secretValue, context, variableName);
const entropy = features[7] * 8.0; // Denormalize f[7]

if (entropy < 3.5) { 
    return 'low'; // Skip expensive L4 inference entirely
}
Enter fullscreen mode Exit fullscreen mode

Plain text words and standard identifiers (const databaseHost = "localhost") exhibit low entropy ($\approx 2.1 - 2.8$). Cryptographic secrets and API tokens (sk_live_9xQ8...) typically exhibit entropy values above $4.2$.

Filtering out low-entropy candidates at L3 eliminates the vast majority of false alarms before a single byte touches the network.

Layer 4: Contextual Neural Classifier (Aegis)

Tokens that survive L1–L3 enter L4. This layer evaluates a 35-dimensional feature vector capturing:

  1. Text Morphology (6): Uppercase ratio, digit ratio, symbol density, length normalization.
  2. Entropy Gradients (3): Shannon entropy, bigram entropy, trigram entropy.
  3. Pattern Signals (3): Base64 padding, hex character sets, character transition frequency.
  4. Context Semantics (5): Surrounding keywords (Bearer, Authorization, db_pass, private_key).
  5. Identifier Conventions (4): ALL_CAPS naming, presence of SECRET/TOKEN/KEY substrings.
  6. Structural Separators (4): Dot delimiters (JWTs), underscores, dashes, colons.
  7. Derived Interactions (10): Non-linear threshold flags combining entropy with context sensitivity. 
# python-llm/src/model.py
class CustomLLM:
    """Neural risk classifier with self-attention and backpropagation."""
    def __init__(self, config: ModelConfig):
        self.embedding = DenseLayer(35, config.hidden_dim)
        self.attention = MultiHeadSelfAttention(config.hidden_dim, num_heads=4)
        self.classifier = FeedForward(config.hidden_dim, config.num_classes)
        self.optimizer = Adam(learning_rate=0.001)
Enter fullscreen mode Exit fullscreen mode

At its current stage, the classifier is trained on an experimental curated dataset of 112+ labeled secret and non-secret samples. It runs backpropagation via Adam optimization, with model weights persisted in a PostgreSQL database surviving service redeploys.

Important Philosophy: Risk Signals, Not a Security Oracle

A vital design principle behind Aegis:

No layer is treated as an absolute security oracle.
The pipeline generates heuristic risk classifications, not mathematical proofs of secrecy. L1 provides deterministic pattern matching for known vendor formats, while entropy and neural inference provide weighted signals to aid human developers in reviewing their environments.

When the remote model is unreachable or rate limits are reached, the extension seamlessly falls back to local heuristic scoring without crashing or blocking the developer.

OS-Level Key Security: SecretStorage Migration

A secret detector is useless if the extension itself stores encryption keys insecurely.

In earlier versions, master keys lived in VS Code’s workspaceState (essentially unencrypted JSON storage). In Aegis, all master encryption keys and PBKDF2 salts were migrated to VS Code’s OS-level SecretStorage:

  • macOS: Apple Keychain
  • Windows: Windows Credential Manager
  • Linux: Secret Service API / gnome-keyring 
public static async ensureMasterKey(context: vscode.ExtensionContext): Promise<Buffer> {
    const secretsKey = `${this.SECRET_STORAGE_KEY_PREFIX}${workspace}`;

    // 1. Check OS SecretStorage
    const stored = await context.secrets.get(secretsKey);
    if (stored) { return Buffer.from(stored, 'base64'); }

    // 2. Zero-touch migration from legacy workspaceState
    const legacyKey = context.workspaceState.get<string>(secretsKey);
    if (legacyKey) {
        await context.secrets.store(secretsKey, legacyKey);
        await context.workspaceState.update(secretsKey, undefined); // Secure cleanup
        return Buffer.from(legacyKey, 'base64');
    }

    // 3. Cryptographically secure 256-bit key generation
    const key = crypto.randomBytes(32);
    await context.secrets.store(secretsKey, key.toString('base64'));
    return key;
}
Enter fullscreen mode Exit fullscreen mode

The migration is completely transparent to the user, executing silently upon extension activation.

Test Verification

Both the TypeScript extension and the Python backend are verified by end-to-end test suites:

DotEnvy Extension (VS Code) 

🧪 Running Complete DotEnvy Test Suite

✅ [Encryption Algorithms & PBKDF2]           8/8 passed
✅ [L1–L4 Multi-Layer AI Scanner & HMAC]      4/4 passed  
✅ [Master Key Lifecycle, Password Migration]  9/9 passed

🎉 21/21 TESTS PASSED
npm run compile   → 0 errors
npm run lint      → 0 warnings
Enter fullscreen mode Exit fullscreen mode

Aegis Backend Service (FastAPI / PyTorch) 

✅ Feature extractor: 35 features verified - OK
✅ Neural model architecture - loss=1.1534
✅ Forward pass evaluation - prediction: high
✅ Full analyze pipeline execution - confidence: medium
✅ Database round-trip test - weights save/load OK
Enter fullscreen mode Exit fullscreen mode

The Next Challenge: Shipping the Detector

Designing a 4-layer AI scanner was only half the journey.

When we attempted to package this extension into a .vsix bundle and distribute it through the VS Code Marketplace and Open VSX, we collided with two massive engineering road hazards:

  1. The VSIX Trap: How our attempt to authenticate extension requests with our backend accidentally created a critical secret leak vector.
  2. The Scanner Paradox: Why static analysis bots accused DotEnvy of leaking secrets simply because it contained the regex patterns needed to detect them!

👉 Read the complete post-mortem in Part 2:

The VSIX Packaging Trap: How I Eliminated Embedded Client Secrets from a VS Code Extension

Links & Resources

Written by FreeRave. Contributions and community feedback are always welcome.

Top comments (0)