DEV Community

Antigravity Dev Studio
Antigravity Dev Studio

Posted on

Building Anti-Detect Web Automation & Headless Fleets in Pure Python: CDP Injection, Coherent Fingerprints & SQLite Sessions

Building Anti-Detect Web Automation & Headless Fleets in Pure Python: CDP Injection, Coherent Fingerprints & SQLite Sessions

Web scraping and browser automation in 2026 have shifted from basic HTTP parsing to an adversarial battle against machine-learning anti-bot systems. Platforms like Cloudflare Bot Management, Datadome, Akamai, and HUMAN (PerimeterX) analyze hundreds of browser runtime integrity signals, WebGL hardware pipelines, and behavioral telemetry.

In this guide, we break down the architecture of StealthFleet Proโ€”a production-grade, zero-subscription Python framework for running undetectable browser automation and headless crawling fleets.


1. Why Naive Randomization Fails (The Coherence Rule)

A common mistake in web scraping is indiscriminately randomizing browser properties on every request. Novice scripts often pair a Windows User-Agent with an Apple M-series WebGL renderer, or randomize navigator.hardwareConcurrency between 2 and 64 cores across paginations.

Anti-bot neural networks immediately flag incoherent fingerprints:

[Incoherent Fingerprint Detected]
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) ...
Platform: Win32
WebGL Vendor: Apple Inc. โŒ [Anomaly: Apple GPU on Windows OS]
Screen: 1920x1080
Color Depth: 30-bit โŒ [Anomaly: Non-standard HDR depth on standard 1080p profile]
Result: Immediate Cloudflare Turnstile Challenge
Enter fullscreen mode Exit fullscreen mode

To remain undetected, fingerprints must be statistically coherent and stable per session profile.


2. Generating Coherent Hardware Profiles in Python

In stealthfleet_pro, we model realistic hardware distributions. When a Windows desktop profile is requested, our generator ensures that screen resolution, WebGL vendor (Google Inc. (NVIDIA) or Google Inc. (Intel)), Direct3D11 ANGLE renderers, hardware concurrency, and client hints strictly correlate:

import random
from typing import Optional
from pydantic import BaseModel, Field

class FingerprintProfile(BaseModel):
    platform: str
    user_agent: str
    viewport_width: int = 1920
    viewport_height: int = 1080
    hardware_concurrency: int = 8
    device_memory_gb: int = 16
    languages: list[str] = Field(default_factory=lambda: ["en-US", "en"])
    webgl_vendor: str = "Google Inc. (NVIDIA)"
    webgl_renderer: str = "ANGLE (NVIDIA, NVIDIA GeForce RTX 4070 Direct3D11 vs_5_0 ps_5_0, D3D11)"
    canvas_noise_seed: float = 0.12345
Enter fullscreen mode Exit fullscreen mode

3. Chrome DevTools Protocol (CDP) Pre-Document Injection

Injecting evasion scripts via DOMContentLoaded executes too lateโ€”anti-bot scripts execute at the root of the HTML document. Using CDP's Page.addScriptToEvaluateOnNewDocument, our overrides run before any site scripts evaluate:

async def attach_stealth_pipeline(page, fingerprint: FingerprintProfile):
    stealth_js = f"""
    (() => {{
        // 1. Remove navigator.webdriver cleanly
        Object.defineProperty(navigator, 'webdriver', {{
            get: () => undefined,
            configurable: true
        }});

        // 2. Align navigator.languages
        Object.defineProperty(navigator, 'languages', {{
            get: () => {repr(fingerprint.languages)},
            configurable: true
        }});

        // 3. Mask WebGL vendor and unmasked renderer
        const getParam = WebGLRenderingContext.prototype.getParameter;
        WebGLRenderingContext.prototype.getParameter = function(param) {{
            if (param === 37445) return '{fingerprint.webgl_vendor}';
            if (param === 37446) return '{fingerprint.webgl_renderer}';
            return getParam.apply(this, arguments);
        }};

        // 4. Subtle Canvas noise to prevent perceptual hash profiling
        const originalToDataURL = HTMLCanvasElement.prototype.toDataURL;
        HTMLCanvasElement.prototype.toDataURL = function(...args) {{
            const ctx = this.getContext('2d');
            if (ctx) {{
                ctx.fillStyle = 'rgba(255, 255, 255, 0.001)';
                ctx.fillRect(0, 0, 1, 1);
            }}
            return originalToDataURL.apply(this, args);
        }};
    }})();
    """

    cdp_session = await page.context.new_cdp_session(page)
    await cdp_session.send("Page.addScriptToEvaluateOnNewDocument", {"source": stealth_js})
Enter fullscreen mode Exit fullscreen mode

4. Serverless SQLite Session Store (Eliminating Repetitive Logins)

Logging in on every scrape execution triggers rate limits and SMS verifications. Instead, persistent cookies, localStorage tokens, and warmed session states are stored in a local SQLite lakehouse:

import sqlite3
import json

class SessionStore:
    def __init__(self, db_path="sessions.db"):
        self.conn = sqlite3.connect(db_path)
        self.conn.execute("""
            CREATE TABLE IF NOT EXISTS sessions (
                session_id TEXT PRIMARY KEY,
                domain TEXT NOT NULL,
                fingerprint_json TEXT NOT NULL,
                cookies_json TEXT NOT NULL,
                last_used_at REAL NOT NULL
            )
        """)
        self.conn.commit()

    def persist(self, session_id, domain, fp, cookies):
        self.conn.execute("""
            INSERT INTO sessions VALUES (?, ?, ?, ?, strftime('%s', 'now'))
            ON CONFLICT(session_id) DO UPDATE SET
                cookies_json=excluded.cookies_json,
                last_used_at=excluded.last_used_at
        """, (session_id, domain, fp.model_dump_json(), json.dumps(cookies)))
        self.conn.commit()
Enter fullscreen mode Exit fullscreen mode

5. Async Concurrency & Exponential Backoff

Unthrottled browser spawns quickly crash cloud VMs. An async fleet manager bounds concurrent processes with asyncio.Semaphore and enforces exponential backoff upon encountering HTTP 429 or CAPTCHA challenges:

import asyncio

class AsyncFleetManager:
    def __init__(self, max_concurrency: int = 5):
        self.semaphore = asyncio.Semaphore(max_concurrency)

    async def execute_task(self, url: str, max_retries: int = 3):
        async with self.semaphore:
            retries = 0
            while retries <= max_retries:
                try:
                    # Execute stealth scrape
                    return await self._scrape_with_retry(url)
                except Exception as ex:
                    retries += 1
                    if retries > max_retries:
                        raise ex
                    backoff = 0.5 * (2 ** (retries - 1))
                    await asyncio.sleep(backoff)
Enter fullscreen mode Exit fullscreen mode

Conclusion & Complete Codebase

Building resilient, autonomous web automation fleets in Python requires no expensive commercial scraping APIs. By understanding the browser runtime integrity checks and managing coherent fingerprints with session persistence, solo engineers can extract data at massive scale on $0 cloud infrastructure.

The complete commercial codebase (stealthfleet_pro) and the full 150-page guide are available as part of The Sovereign Software Architect Master Collection:

๐Ÿ‘‰ Download the Full Codebase & Master Collection:

https://antigone16.gumroad.com/l/sovereign-architect-quad-vault

(Use coupon code **LAUNCH20* for 20% off storewide).*

Top comments (0)