DEV Community: Regaan

Why I stopped treating WebSocket messages as the unit of work (and built WSHawk instead)

Regaan — Sat, 11 Jul 2026 09:29:50 +0000

Most automated web scanners are built around one assumption: you send a request, you get a response, you look at the response. That model falls apart the moment you point them at a WebSocket endpoint.

I kept running into this while testing realtime apps (chat, collaboration tools, trading dashboards, the usual SaaS suspects), so I built WSHawk, an open-source toolkit (v4.0.1, AGPL-3.0) for authorized security assessment of WebSocket and realtime web applications. This post is a tour of why each piece exists and how it fits together.

WSHawk is an offensive tool. It's for authorized testing only: systems you own, engagements you're contracted for, or bug bounty programs within scope. Full stop.

Why WebSockets break normal scanners

After the HTTP upgrade handshake, a WebSocket is just a long-lived, full-duplex TCP channel. Three things about that make classic tooling useless:

The connection is stateful. Auth, subscriptions, and app context are built up through an ordered sequence of messages. You can't just fire a payload at a URL, you have to connect, authenticate, subscribe, and then reach the injectable sink, all on the same socket.
Communication is async and bidirectional. A server's reaction to your injected message might be delayed, interleaved with unrelated push traffic, or arrive on a completely different channel. Sometimes it never comes back on the same socket at all.
Payloads are structured. JSON, XML, Protocol Buffers, MessagePack, CBOR. Throw a raw ' OR 1=1-- at a schema-validated message and it gets rejected long before it reaches anything interesting.

So the core design decision in WSHawk is this: the connection, not the individual message, is the unit of work.

Learn first, attack second

The scanner splits into a passive learning phase and active probing phases.

During learning, WSHawk keeps the socket open (five seconds by default), samples the traffic, and figures out:

the wire format and which fields are injectable,
the backend stack (it fingerprints languages, frameworks, and databases from response signatures),
a response baseline for later comparison.

Only after it understands the schema does it run the per-class probes: SQLi, XSS, command injection, path traversal, XXE. Each probe injects into a well-formed base message rather than spraying raw strings, which is the whole point: you have to look like valid traffic to get past structural validation.

Connect + state sequence
   -> Learning phase (sample, detect format, fingerprint)
   -> Per-class probing (SQLi, XSS, CMDi, XXE, traversal)
   -> Heuristic verifier => confidence
   -> High-confidence XSS? -> browser verify
   -> CVSS + report / evidence

Payloads that adapt to the server

There are two payload subsystems working together.

1. A strategy-scored mutation engine. Eight deterministic strategy families (encoding, case variation, comment injection, whitespace substitution, concatenation, filter-bypass, tag-breaking, polyglot). Each carries a weighted score. When a payload gets through unblocked, every strategy it used gets boosted. It also reads the room: it scans responses for WAF signatures (Cloudflare, Akamai, Imperva, F5, AWS WAF, ModSecurity) and generic block indicators, then steers future mutations around whatever's filtering.

2. Smart Payload Evolution (SPE), a genetic pipeline:

a context-aware generator that learns the message structure and embeds attack strings inside well-formed JSON/XML (it even emits type-confusion payloads like __proto__ prototype-pollution objects),
a feedback loop that classifies each response into ERROR, BLOCKED, REFLECTED, DELAYED, DIFFERENT, or NORMAL,
an evolver that treats successful payloads as a population, breeds them with crossover/mutation, and reallocates effort toward high-fitness individuals.

Fitness is an exponential moving average, with guardrails so it doesn't run away:

# fitness update (EMA)
f_new = 0.4 * score + 0.6 * f_old      # beta = 0.4
# anything scoring >= 0.7 goes to the hall of fame

Offspring are MD5-deduplicated, capped at 2000 chars, and the population is trimmed back to N=100 fittest each generation, so memory stays bounded no matter how long a run goes.

Here's the actual crossover operator (abridged):

def _crossover(self, p1, p2):
    strategy = random.choice(['split', 'interleave', 'wrap', 'inject'])
    if strategy == 'split':
        m1, m2 = len(p1)//2, len(p2)//2
        return (p1[:m1] + p2[m2:] if random.random() < 0.5
                else p2[:m2] + p1[m1:])
    if strategy == 'interleave':
        k = random.randint(2, 8); out = []
        for i in range(0, max(len(p1), len(p2)), k):
            out.append(p1[i:i+k] if i % (2*k) < k else p2[i:i+k])
        return ''.join(out)
    if strategy == 'wrap' and len(p1) > 4:
        m = len(p1)//2
        return p1[:m] + p2 + p1[m:]
    # inject: splice p2 at a random offset in p1
    pos = random.randint(0, len(p1))
    return p1[:pos] + p2 + p1[pos:]

Stacked oracles: cheap to expensive

Naive detection lies to you. Reflection isn't execution, blind bugs give no on-channel signal, and timing is noisy. So WSHawk layers its oracles by cost:

Heuristic verifier first (error signatures, timing, reflection with context analysis). One nice detail: timing uses time.monotonic(), not wall-clock, so a system clock change can't poison a time-based detection.
Out-of-band (OAST) callbacks for blind XXE and SSRF, the payload makes the target phone home to a collaborator; the callback is the proof.
A Playwright/Chromium browser pool for XSS, but only on high-confidence candidates. It renders the response in a sandboxed page that overrides alert/eval, installs a MutationObserver, and confirms actual script execution rather than mere reflection.

The browser oracle is expensive, so gating it behind the heuristic verifier's HIGH rating is a deliberate cost/precision tradeoff.

HTTP and WebSocket, same workflow engine

Modern apps split state across a browser-authenticated HTTP surface and a realtime socket. So WSHawk gives HTTP and WebSocket symmetric attack services, replay, authorization-diffing, race testing, subscription abuse, all sharing the same identities and a {{variable}} templating workflow engine.

That's what lets a single playbook chain an HTTP bootstrap into a privileged WebSocket action:

HTTP bootstrap (extract CSRF/tenant)
   -> variable map {{var}}
   -> WS replay (privileged action)
   -> bundle with HTTP + WS replay recipes

Six playbooks ship built-in: login bootstrap, CSRF/session capture, WS privilege escalation, stale-token reuse, tenant hopping, and authed HTTP replay.

The race tester is worth a mention: it launches N attempts per wave, all blocked on a shared asyncio.Event barrier so they release as close to simultaneously as possible, then flags a suspicious window when more than one attempt succeeds in a state-changing mode (duplicate action, replay-before-invalidation, stale-token window).

Knowing what protocol you're actually looking at

Realtime traffic is rarely "raw", it's usually one of a handful of recognizable families. WSHawk's protocol subsystem turns captured frames into an actionable map and fingerprints six families with dedicated target packs:

Pack	Signals
`graphql_ws`	subscribe/next/complete, operations, variables
`phoenix_channels`	topic/event/payload/join_ref
`actioncable`	command/identifier with embedded identifier JSON
`signalr`	record-separator framing, invocation IDs
`socket_io`	Engine.IO prefixes, namespaces, event arrays
`binary_realtime`	consumes protobuf/msgpack/CBOR analysis

Once it knows the family, it recommends concrete attacks and maps them to playbooks the workflow engine can actually run.

Evidence you can trust

A finding is only as good as the trail behind it. Everything (identities, traffic, findings, notes, replay recipes) lives in a project-backed SQLite store, with sensitive columns encrypted at rest. Exports are tamper-evident:

Project bundle -> redaction (mask secrets) -> canonical JSON
   -> per-list hash chains + Ed25519 signature (SHA-256)
   -> integrity + provenance block

The hash chain folds each item into the previous root, so any reorder is detectable, and the Ed25519 signature over canonical JSON catches any content edit. Verification is self-contained because the public key and chain roots travel inside the bundle. Redaction runs before export, so shared bundles don't leak secrets.

@classmethod
def _hash_chain(cls, items):
    previous = ""
    for item in items:
        previous = cls._sha256_hex({"previous": previous, "item": item})
    return {"count": len(items), "root": previous}

The rest of the platform

A quick lightning round of what else is in there:

Binary message handler with format auto-detection (zlib/gzip, Avro, BSON, protobuf, FlatBuffers, MessagePack, CBOR, plus a Shannon-entropy test that flags likely-encrypted payloads at >7.5 bits/byte) and format-aware mutation.
Endpoint discovery that probes common WS paths, parses HTML/JS for ws:///wss:// URLs and library signatures, and ranks hits by confidence.
Session-hijacking tester (token reuse, subscription spoofing, impersonation, channel violations, session fixation, privilege escalation).
A blue-team defensive suite (DNS-exfil egress filtering, bot detection, CSWSH Origin enforcement, WSS/TLS posture checks).
A process-isolated plugin system, plugins run in a separate process with a 5s timeout, size limits, and a sanitized env, so a crashing plugin returns a safe default instead of killing the scan.
Adaptive token-bucket rate limiting that backs off on latency and WebSocket close-code signals (1013 = strong rate-limit signal), plus an HTTP resilience layer with retry and circuit breaking.
Reporting in JSON/CSV/SARIF 2.1.0/HTML, and integrations for Jira, DefectDojo (with per-class CWE mapping), and Slack/Discord/Teams webhooks.

There's also an Electron desktop app (frame-by-frame interceptor, payload blaster, endpoint map, auth builder, evidence vault) and a Manifest V3 browser companion that pairs on localhost to ingest handshake context.

The honest part: what I haven't proven yet

I want to be straight about this, because security tooling has a credibility problem when it overclaims.

WSHawk does not yet have a detection-accuracy benchmark. The current validation is engineering-focused: a unit/integration test suite and three local validation labs (a full-stack realtime SaaS, a Socket.IO SaaS, and a GraphQL-subscriptions lab) that verify behavioral correctness and reproducibility. What they do not measure is true/false-positive rates against a labeled vulnerability corpus, throughput comparisons, or head-to-head results against other scanners.

Other limitations I'll name openly:

the XSS browser oracle depends on Playwright/Chromium (no Playwright = heuristic-only fallback),
CVSS scoring is a simplified v3.1 base-score computation, indicative, not a full calculator,
non-browser classes lean on error/timing/reflection heuristics, which are inherently prone to false positives/negatives,
binary parsing is heuristic and schema-less, so field extraction is best-effort.

Closing that empirical gap (a controlled evaluation with per-class oracle precision: heuristic vs. browser-verified vs. OAST-confirmed) is the top of the roadmap.

Takeaway

If you're building or testing realtime apps, the mental shift that matters is treating the connection as the thing you're testing, not individual messages. State preservation, schema-aware injection, layered oracles, and a signed evidence trail all fall out of that one decision.

Repo's here if you want to dig in : https://github.com/regaan/wshawk

Built by Regaan R // Rot Hackers. Authorized testing only, folks.

How I Built an AI That Breeds Its Own Jailbreaks Using Genetic Algorithms

Regaan — Wed, 18 Mar 2026 12:49:37 +0000

Static jailbreak lists are dead.

Every time a model provider patches their safety filters, your entire payload library becomes obsolete. Manual red teaming doesn't scale. And most AI security tools are just payload databases with a UI.

So I built something different.

The Problem

I tested 6 major LLM deployments last year. Every single one had a bypass within 5 prompts. The problem isn't that LLMs are insecure — it's how the industry tests them.

Most red teaming today looks like this:

Copy a jailbreak from a GitHub list
Paste it into the target
If it works, report it
If it doesn't, try the next one

That's not security testing. That's pattern matching. And it stops working the moment the model gets patched.

The Idea

What if adversarial prompts could evolve?

Not manually crafted. Not randomly generated. Actually evolved — like organisms under selection pressure.

The strong prompts survive. The weak ones die. The survivors mutate and reproduce. Each generation gets better at bypassing the target's specific defenses.

That's the core idea behind Basilisk.

How It Works

Basilisk introduces Smart Prompt Evolution for Natural Language (SPE-NL) — a genetic algorithm that treats adversarial prompts as organisms in a population.

Selection: Each prompt is scored by a multi-signal fitness function — did it bypass the guardrail? Did it extract sensitive data? Did it make the model contradict its system prompt?

Mutation: 10 mutation operators transform surviving prompts — semantic rewriting, context injection, authority spoofing, encoding shifts, persona layering, and more.

Crossover: 5 crossover strategies combine the strongest parts of two successful prompts into offspring that inherit the best traits of both parents.

Evolution: By generation 5, the evolved prompts achieved a 92% improvement in attack success rate over the original static payload library.

The prompts literally get smarter at breaking the specific target they're aimed at.

What It Covers

Basilisk maps to the OWASP LLM Top 10 with 29 attack modules across 8 categories:

Prompt injection (direct and indirect)
System prompt extraction
Data exfiltration
Tool/function abuse
Guardrail bypass
Denial of service
Multi-turn manipulation
RAG poisoning

It supports 100+ LLM providers through LiteLLM — OpenAI, Anthropic, Google, Mistral, Cohere, local models via Ollama, and any custom endpoint.

Differential Testing

One of my favorite features — point Basilisk at multiple models simultaneously and watch how they diverge.

The same evolved prompt might bypass Claude but fail on GPT. Or break Gemini's guardrails while Llama holds firm. This behavioral divergence analysis reveals which models share defense architectures and which have unique weak points.

Non-Destructive Posture Assessment

Not every engagement needs active exploitation. Basilisk includes a guardrail grading mode that scores your LLM's defenses from A+ to F without actually breaking anything. Safe enough for production environments.

The Results

On the benchmark targets I tested:

92% improvement in attack success rate by generation 5
Evolved prompts discovered bypass patterns that didn't exist in any public jailbreak database
Cross-provider differential testing revealed behavioral divergence invisible to single-target scanning

The genetic approach doesn't just find known bypasses faster — it discovers novel ones that static testing would never reach.

Try It

It's fully open source. One command to install:

pip install basilisk-ai

Point it at any LLM endpoint:

basilisk scan --target https://your-llm-endpoint.com --mode standard

The full research paper is published with a permanent DOI:
Basilisk: An Evolutionary AI Red-Teaming Framework for Systematic Security Evaluation of Large Language Models

GitHub: github.com/regaan/basilisk

What's Next

I'm currently researching a new attack class called Prompt Cultivation — a technique that doesn't use injection or commands at all. Instead, it exploits the model's own curiosity and reasoning to absorb it into a frame where safety guidelines become irrelevant. No override. No jailbreak. The model follows the idea, not the instruction.

Paper coming soon.

If you're building with LLMs, test them before attackers do. Your AI is already vulnerable. You just don't know it yet.

Star the repo if this was useful: github.com/regaan/basilisk