DEV Community: OBI EBUKA DAVID

83% of Financial Apps Ship a CSP That Doesn't Work

OBI EBUKA DAVID — Thu, 13 Aug 2026 17:44:48 +0000

We scanned 618 public apps from 347 African banks, fintechs and payment companies across 21 countries. One passive request each. No logins, no probing, nobody named.

Half graded C or below.

83% exposed to XSS. 39% with no clickjacking protection on the page customers log in through. 79 running WordPress under a bank brand. 14 on expired certificates today.

It took one afternoon from a laptop.

That is the same view an attacker has of your estate right now. The difference is they are running it with AI, across a whole market, before lunch.

And a third of these institutions publish a public API. Nothing in this report protects that part. No header stops a request that is perfectly well formed and simply should not have been allowed.

The research: https://www.nemesislabs.xyz/state-of-app-security/
Check your own app, free: https://www.nemesislabs.xyz/protect/

Autogon #Omniguard #Nemesis #ApplicationSecurity #Finance #AI

Your WAF was never built to see this: positive security vs signature WAFs, and how it stops IDOR/BOLA

OBI EBUKA DAVID — Wed, 12 Aug 2026 01:22:09 +0000

You are logged in. You hit your own billing page and the browser fires:

GET /api/invoices/1043 HTTP/1.1
Host: app.acme.com
Authorization: Bearer eyJhbGciOi...
Cookie: session=valid

Back comes your invoice. Fine. Now you change one digit:

GET /api/invoices/1044 HTTP/1.1
Host: app.acme.com
Authorization: Bearer eyJhbGciOi...
Cookie: session=valid

Invoice 1044 belongs to a different customer. If the response is their invoice, you just walked out with someone else's data using nothing but your own valid session and the up-arrow key. That is IDOR (Insecure Direct Object Reference), and in API terms it is BOLA (Broken Object Level Authorization). It is the number one item on the OWASP API Security Top 10.

Here is the part that catches people. Look at that second request again. There is no SQL in it. No <script>. No ../../etc/passwd. No union select, no encoded payload, no oversized header. It is a clean, well-formed, authenticated GET to a route your API absolutely serves. Every byte of it looks exactly like the first request, because it is exactly like the first request. One integer changed.

So walk it through your defenses. Your WAF looks at it and sees a valid method, a known path shape, a real bearer token, and a body with nothing suspicious in it. Pass. Your rate limiter sees one request. Pass. Your TLS terminates fine. Pass. The only thing in your entire stack that had the information needed to stop that request was the handler behind /api/invoices/{id}, which needed to check "does the caller who owns this session also own invoice 1044?" and did not. The authorization check was the one guard that could have caught it, and it was missing. Everything else waved it through because, to everything else, there was nothing to catch.

What a signature WAF actually does

A signature WAF (Cloudflare WAF, AWS WAF, ModSecurity with the OWASP Core Rule Set) works by matching your traffic against a list of known-bad patterns. The OWASP CRS is thousands of these rules: regexes and heuristics for SQL injection, cross-site scripting, path traversal, command injection, known scanner user-agents, malicious file uploads. Plus rate limiting and IP reputation on top. When a request matches enough badness, it gets blocked or scored up.

This model is genuinely good at what it is for. It filters out the constant background radiation of the internet: the automated bots spraying Log4Shell strings at every host they can find, the SQLi fuzzers, the WordPress exploit kits hammering /wp-login.php. If you run an API on the public internet without something soaking up that noise, your logs are unreadable and your odds are worse. I am not telling you to turn your WAF off.

I am telling you what it structurally cannot see. A signature WAF answers one question: does this request look like a known attack? IDOR does not look like a known attack. Neither does broken authentication, where a valid token is used in a context it should not be allowed in. Neither does business-logic abuse, like applying a coupon a thousand times, or calling a state-transition endpoint out of order to skip a payment step. These are the attacks that breach real APIs, and they are all made of well-formed requests. The pattern-matching model has nothing to match, because the maliciousness is not in the shape of the request. It is in the relationship between the caller and the object, and that relationship lives in your data, not in the bytes on the wire.

OWASP made this explicit. In the API Security Top 10 (the 2023 list), API1:2023 is Broken Object Level Authorization (BOLA), and API5:2023 is Broken Function Level Authorization (BFLA). BOLA is "I can access an object I do not own." BFLA is "I can call a function I am not entitled to call," like a normal user hitting an admin-only endpoint that happens to accept their token. The two authorization failures sit at the very top of the list precisely because they are so common and so invisible to the tooling most teams already run. A signature WAF is not on that list of controls because it was never designed to be.

The inversion: allow-list instead of deny-list

Positive security flips the question. Instead of does this request match something bad, it asks does this request match this app's own normal behavior? You build an allow-list of what your application actually does, then you block whatever deviates from it.

The trick is that "normal" is not a rule someone writes. It is learned from your app's real behavior. A positive-security layer watches traffic and builds a model: these are the routes that exist, this is the shape each one takes (method, path structure, which query params appear and of what kind), this is whether the caller is usually authenticated, and, importantly, these are the per-tenant access patterns. What does a normal session touch? Whose objects does a given identity normally reach?

Now replay the IDOR. Your session has a learned baseline: over hundreds of requests, this identity reads invoices that belong to this identity. Then a request comes in for an object outside that pattern. It is still a well-formed GET to a known route. But it deviates from this caller's learned normal, and deviation is exactly what a positive-security model is built to flag. It does not need a rule that says "invoice 1044 is off-limits to user 7." It just needs to have learned that user 7 does not reach into that neighborhood, and then notice when user 7 suddenly does. The catch comes from the baseline, not from a signature.

That is the whole difference in one sentence. A deny-list needs someone to have anticipated the attack. An allow-list only needs to know what your app normally does, and every real IDOR is, by definition, your app doing something it normally does not.

What this looks like in practice

This is the model behind Nemesis Shield, which is what I build, so I will show you the real integration rather than hand-wave. It is a per-app SDK you mount as outermost middleware. It computes a privacy-preserving shape of each request locally (method, the normalized path like /invoices/{int}, query-param structure, status, and whether the caller was authenticated) and never ships your bodies, responses, or secrets.

FastAPI, one line:

from nemesis_shield.asgi import SentinelMiddleware
app.add_middleware(SentinelMiddleware, token=os.environ["NEMESIS_TOKEN"])

Express, one line:

import { sentinel } from "@nemesis-shield-autogon/sentinel/express";
app.use(sentinel({ token: process.env.NEMESIS_TOKEN }));

Install is the obvious pip install nemesis-shield or npm install @nemesis-shield-autogon/sentinel, token comes from the environment, and there are the same one-liners for Go, Ruby, PHP, Java, .NET, Rust, and the edge runtimes. Mount it first so it sees every route, not just your API prefix.

Every app starts in observe mode. It blocks nothing. It only learns, building the per-app, per-tenant baseline from real traffic. That matters because the whole model rests on knowing your normal, and you do not want to enforce against a baseline that has not seen your app yet.

If you are shipping a brand-new API, there is no traffic to learn from, so you can drive it:

npx @nemesis-shield-autogon/learn --target http://localhost:3000 --app-token nsk_... --repo .

Nemesis Learn exercises every route (it can read your repo to discover them, log in to reach protected routes, and chain CRUD flows) so the baseline completes in minutes instead of waiting weeks on organic traffic. It reports coverage back so the console shows when you are actually ready.

Then you review. Learned behaviors show up in a queue with the evidence behind them, you approve the legitimate ones, and when you are satisfied you flip the app to enforce in the console. No redeploy. From that point, requests whose shape falls outside the approved, per-tenant baseline get a 403 and land as findings. The key word is per-tenant: this is not one global ruleset pretending every customer looks the same. The baseline is scoped to your app and its access patterns, which is the only way "this caller does not normally reach that object" means anything.

The honest limits

I am not going to tell you this replaces fixing your authorization. It does not, and anyone who says their WAF-shaped product does is selling you a nightlight and calling it a lock.

Positive security has real costs you should go in knowing:

It needs a clean learning period. If you learn from traffic that already contains abuse, you bake the abuse into "normal." Learn in dev/staging or a known-good window, and review what it learned before you trust it.
It is a complement, not a substitute. The actual fix for the invoice IDOR is an authorization check in the handler: does this caller own this object? Write that check. Positive security is defense in depth that catches the ones you missed and the new ones you have not written yet. It is not permission to skip the check.
False positives are real, which is why observe-first exists. A legitimate new feature is, to the model, a deviation. That is what the approve/deny review queue is for, and it is why enforcement is a deliberate flip and not a default. Managed well, the noise is bounded. Ignored, it will annoy you.

Business-logic abuse that stays entirely inside normal behavior (a real user doing a real thing too many times, within their own scope) is also not something an access-pattern baseline catches on its own. Different problem, different control.

Close

Your signature WAF is doing its job. It just was never built to see the request that changed one digit and walked off with someone else's invoice, because that request was never malformed. It was well-formed and unauthorized, and those are different problems that need different defenses.

Positive security is the defense for the second one, and it is free to start, one line to add, and safe by default because it observes before it ever blocks. If you build with an AI editor, you do not even add it yourself:

npx -y @nemesis-shield-autogon/mcp

Point Cursor or Claude Code at the Nemesis MCP server and it wires the SDK in as it builds. Then go write the authorization check too.

Author: David Obi

Your Supabase Edge Function needs a WAF too

OBI EBUKA DAVID — Wed, 12 Aug 2026 01:18:00 +0000

Here is the function that keeps me up at night. It is a Supabase Edge Function called checkout, maybe 60 lines of Deno, and it does what every checkout does: takes an order id, looks up the cart, confirms the price, writes a row. To read the cart across users it was given the service_role key, because RLS was getting in the way during a Friday afternoon and the fastest unblock was to step outside RLS entirely. That decision shipped. The function is now public at https://<project>.supabase.co/functions/v1/checkout, and the key inside it can read and write every row in every table, ignoring every policy you wrote.

Now the part people miss. Your app has a WAF. Cloudflare, or the Vercel firewall, or whatever sits in front of your Next.js origin. That WAF is doing real work on app.yourcompany.com. It is doing zero work on the checkout function, because the checkout function does not run behind it. It runs on Supabase's edge, at a different hostname, on infrastructure your CDN never sees. An attacker who hits the function URL directly walks straight past the thing you think is protecting you.

The edge is the blind spot on purpose

The whole selling point of edge and serverless functions is that they run close to the request and independent of your origin. Supabase Edge Functions, Vercel Edge, Cloudflare Workers: same shape. They spin up in a lightweight isolate, handle the request, and disappear. That independence is exactly why the WAF in front of your main app does not cover them. Different host, different path, different network. The firewall config you spent a day tuning applies to routes that terminate at your origin, not to a Deno isolate answering on functions.supabase.co.

And these functions are not the low-value part of your stack. They are where you put the logic too sensitive for the browser: signing URLs, charging cards, minting tokens, reading other users' data with elevated privileges. Supabase Edge Functions are public by default. There is no implicit login gate. Anybody who knows the URL can send a request, and the function itself is the only thing deciding whether that request is legitimate. It is the trust boundary, and most of the time there is nothing sitting on it except your own if statements.

I have watched teams add a WAF, feel covered, and never notice that the three functions holding the service_role key have no protection at all. The monolith got the guard. The functions that can drop a table got nothing.

A signature WAF would not save you here anyway

Say you did route the function through a classic WAF. It still would not catch the attacks that actually hit these things. A signature WAF (the negative-security model) works off a list of known-bad patterns. SQL injection strings, XSS payloads, the log4shell probe, a directory of CVE fingerprints. It is a bouncer with a list of banned faces.

The attacks against a checkout function do not have banned faces. GET /checkout/1001 when you are user 1002 is a textbook IDOR/BOLA, and every byte of it looks like a normal request, because it is one. It is a valid method, a valid path, valid syntax. There is no payload to match. Broken authentication looks like a request missing a header. Business-logic abuse (replaying a discount, ordering a negative quantity, calling steps out of order) is made entirely of legal requests in an illegal sequence. A pattern matcher has nothing to match on. The OWASP API Security Top 10 has been led by BOLA and broken auth for years precisely because signature tools are blind to them.

The model that works is the inverse. Positive security. Instead of listing what is bad, you learn what is normal for this specific function and block everything else. The function serves GET /products unauthenticated and POST /checkout authenticated with an integer order id. That is the allow-list. GET /checkout/1001 from an unauthenticated caller is not on it, so it does not get through, and nobody had to predict that particular attack in advance. The unknown request fails because it deviates from the baseline, not because someone wrote a rule.

The fix is one line around your handler

This is Nemesis Shield. The edge SDK is a single function, withShield, that wraps your Deno handler and puts the allow-list in front of it. Here is the checkout function before:

Deno.serve(async (req) => {
  const { orderId } = await req.json();
  const cart = await db.from("carts").select().eq("id", orderId).single();
  return new Response(JSON.stringify(cart), {
    headers: { "content-type": "application/json" },
  });
});

And after:

import { withShield } from "jsr:@nemesis-shield/edge";

Deno.serve(withShield(async (req) => {
  const { orderId } = await req.json();
  const cart = await db.from("carts").select().eq("id", orderId).single();
  return new Response(JSON.stringify(cart), {
    headers: { "content-type": "application/json" },
  });
}, { token: Deno.env.get("NEMESIS_TOKEN") }));

The import is the JSR package jsr:@nemesis-shield/edge (on npm runtimes like Cloudflare Workers or Vercel Edge it is @nemesis-shield-autogon/edge, same withShield signature). The token is read from an environment variable, never hardcoded. You set it once with the Supabase CLI:

supabase secrets set NEMESIS_TOKEN=nsk_your_app_token

Grab a free token at shield.nemesislabs.xyz. That is the whole integration. withShield(handler, { token }) returns a (Request) => Response handler, which is exactly what Deno.serve wants, so it drops in with no other changes.

What actually happens after you deploy

The thing I want to be clear about, because it is what makes this safe to ship on a Friday: it does not start blocking. On first deploy the function is in observe mode. Every request runs your handler exactly as before, and the SDK records the shape of the request (method, normalized route, the structure of query params, whether the caller was authenticated) so it can learn what normal looks like. Nothing is blocked. You cannot break production by adding it, because in observe mode there is no enforcement to break.

Learning from live traffic takes as long as it takes to see all your routes, which for a quiet function could be a while. So there is an agent that does it in minutes. Nemesis Learn exercises every route of your app in dev or staging and lets the baseline complete fast:

npx @nemesis-shield-autogon/learn --target http://localhost:54321 --app-token nsk_... --repo .

It sends representative, benign traffic (never attack payloads), can read your repo to discover routes from source, and posts a coverage report back so the console shows a baseline-readiness meter. When it looks complete, you open the app in the console, approve the learned behaviors, and flip the mode to enforce. No redeploy. The SDK refreshes its compiled policy lazily on the request path with a short TTL, so the next request on a cold isolate picks up the new mode on its own.

One property I lean on hard: it is fail-open. Look at the actual handler in the SDK and the entire policy-refresh-and-decide block is wrapped so that any internal error falls straight through to your code. If Shield is unreachable, if the policy fetch times out, if anything in the middleware throws, your function still runs. The only path that returns a 403 is an explicit, deliberate block in enforce mode. A security tool that can take down your checkout when its own backend has a bad day is worse than no tool. This one cannot.

The honest gotchas

It learns from the traffic you give it, so give it clean traffic. If you run Nemesis Learn against a staging environment that is already full of junk, or you leave observe mode running while someone is actively fuzzing the function, that noise becomes part of the baseline and you will approve behaviors you did not mean to. Exercise the real routes, review what it learned before you approve, and do not enforce on a baseline you did not look at.

It models structure, not values. The shape it records is method plus normalized route plus param names and kinds (is this an integer, a uuid, an email) plus an auth flag plus status. It never ships request bodies, and it never ships your users' data. That is a deliberate privacy choice, and it is also a limit: an attack that stays entirely inside your normal request shapes (a valid authenticated request that abuses logic the function itself permits) is not something a behavioral allow-list catches. That class needs business-logic scoring, which is a different tool. Do not let anyone sell you an allow-list as if it catches everything. It catches deviation from learned normal, which is a large and painful category, and not the whole of it.

And the free tier has limits on volume and app count. For a couple of functions it is genuinely free. If you are running a fleet, you will hit the paid tiers eventually. I would rather tell you that now than have you find out at request one million.

Add it from your editor

If you build with an AI editor (Cursor, Claude Code, Windsurf, anything that speaks MCP), you can have the agent wire this in for you. There is a local MCP server:

npx -y @nemesis-shield-autogon/mcp

Point your editor at it and "add Nemesis Shield to my checkout function" becomes a thing your agent can actually do, in observe mode, reading the token from the environment, mounted first so it sees every route. The free tools need no account.

The checkout function that reads every row with the service_role key is not going away. It is how you build on Supabase. Put something on the boundary it lives on.

David Obi

Add a real WAF to your FastAPI, Express, or Next.js app in one line

OBI EBUKA DAVID — Wed, 12 Aug 2026 01:06:27 +0000

The first time an app of mine got popped, it was not a clever attack. Someone changed a number in a URL. /api/orders/8123 became /api/orders/8124, and they were looking at a stranger's order. No SQL injection, no payload, nothing my WAF had ever been trained to recognize. Just a well-formed GET request for an object that was not theirs.

That is the part nobody tells you when you buy a WAF. The WAF you are paying for was built to catch a different decade's attacks.

What your WAF is actually doing

A traditional WAF is a signature engine. It carries a list of known-bad patterns, most of them descended from the OWASP Core Rule Set, and it matches every request against that list. SQL injection strings, cross-site scripting payloads, path traversal, a few thousand regexes. If a request matches, it blocks. If it does not, it passes.

This works great for the noise. Automated scanners hammering your login page with ' OR 1=1 get stopped cold, and that is worth something.

But look at what it cannot see. The order-number attack I described has no bad pattern in it. It is a legitimate request shape hitting a legitimate route with a legitimate session. The only thing wrong is that this user should not have access to that object, and a signature engine has no idea what any user should have access to. Broken object-level authorization sits at the very top of the OWASP API Security Top 10 for a reason. It is the most common way APIs get breached, and it is invisible to the model your WAF is built on.

Same story for broken auth, for business-logic abuse, for the endpoint you shipped last Tuesday that leaks a little more than it should. None of it looks like an attack. All of it is an attack.

The other way to do this

There is a different model, and it is old enough that the idea is not controversial: instead of listing everything bad and blocking that, list everything good and block the rest. Allow-list instead of deny-list. In security terms it is called positive security.

The reason nobody did it for web apps is that writing the allow-list by hand is miserable. Your app has hundreds of routes, each with its own request shapes, auth requirements, and access patterns, and they change every sprint. Nobody is going to maintain that by hand, and they were right not to try.

The thing that changed is that you no longer have to write it. You let the app teach you what normal looks like. Every route it actually serves, the shape of the requests that hit it, whether the caller was authenticated, what a real user's access pattern looks like. Learn that, and then anything that deviates from it gets flagged. The order-number attack deviates, because this user has never accessed that object and the access pattern does not fit. It gets caught for what it is, not because someone wrote a rule.

That is what I have been building with Nemesis Shield, so the rest of this is how you actually wire it up. It is genuinely one line, and it is free to start.

FastAPI

Install the package and add the middleware. That is the whole integration.

pip install nemesis-shield

from fastapi import FastAPI
from nemesis_shield.asgi import SentinelMiddleware

app = FastAPI()
app.add_middleware(SentinelMiddleware, token="nsk_your_app_token")

Your routes do not change. Your handlers do not change. The middleware sits in front, watches the request and response shapes, and reports them.

Express

npm install @nemesis-shield-autogon/sentinel

const express = require("express");
const { sentinel } = require("@nemesis-shield-autogon/sentinel/express");

const app = express();
app.use(sentinel({ token: process.env.NEMESIS_TOKEN }));

Register it before your routes so it sees every request, including the ones that match no handler. Fastify and Koa have the same one-liner if that is your stack.

Next.js

For Next, guard everything from the middleware:

// middleware.ts
import { withShield } from "@nemesis-shield-autogon/edge";
import { NextResponse } from "next/server";

export const config = { matcher: "/:path*" };
export default withShield(() => NextResponse.next(), {
  token: process.env.NEMESIS_TOKEN,
});

The same withShield wrapper works for Vercel Edge, Cloudflare Workers, and Supabase Edge Functions, which matters more than it sounds like, because those are usually the parts of an app that have no protection at all.

The part that keeps you from getting paged

Here is the thing that makes this safe to ship, and the thing I would have wanted to hear before turning any WAF loose on production traffic.

It starts in observe mode. Blocks nothing. It watches real traffic and builds the baseline, and everything it sees shows up in a review queue with the evidence attached. You look at what it learned, you approve it, and only then do you flip it to enforce. There is no moment where you deploy a wall and pray it does not brick your checkout flow.

If you do not want to wait on real traffic to fill out the baseline, point the Nemesis Learn agent at your app in staging and it exercises your routes for you:

npx @nemesis-shield-autogon/learn \
  --target http://localhost:3000 \
  --app-token nsk_your_app_token \
  --repo .

It reads your repo to find routes, drives them over HTTP, and reports coverage, so you can see exactly when the baseline is ready to enforce instead of guessing.

And it is fail-open. If the Nemesis service is ever unreachable, the SDK gets out of the way and your app serves the request. A security tool that can take your app down when its own backend has a bad day is not a security tool, it is a second outage waiting to happen. I have watched that movie. The SDK ships behavioral shapes only, the method, the shape of the path, the status, whether the caller was authenticated. Not your request bodies, not your secrets, not your source.

Where this fits, honestly

Positive security is not a replacement for writing correct authorization code. If your handler does not check that the order belongs to the user, fix the handler. The WAF is the layer that catches the mistake you did not know you made, and the zero-day in a dependency you cannot patch until Tuesday, and the endpoint a teammate shipped without telling you. It buys you time and it catches the class of attack a signature engine was never going to see. That is the whole pitch.

It also needs a clean learning window. If you first turn it loose on an app that is already under active attack, it will learn the attack as normal, so do the learning in staging or during a quiet period and approve what it shows you. That is the one real gotcha, and observe-first exists precisely so you never skip it.

If you are building with an AI editor, you do not even have to add the line yourself. The Nemesis MCP server gives your coding agent the tools to do it while it writes the code:

npx -y @nemesis-shield-autogon/mcp

Ask it to protect your app and it installs the SDK, runs the learn agent, and walks the baseline to enforce for you.

The free tier covers one app and does not need a card, which is enough to protect the side project that is quietly holding real user data. Change one number in one of your own URLs sometime and see what comes back. If it is somebody else's data, you already know which layer was supposed to stop that, and you already know it did not.

We built an LLM that runs real pentests. The hard part was stopping it from lying

OBI EBUKA DAVID — Wed, 08 Jul 2026 05:49:54 +0000

We built an autonomous pentester on top of a large language model. It runs recon, picks and drives real offensive tools, exploits what it finds, and writes the report. The engineering that actually mattered was not teaching a model to hack. Models will happily "hack." The hard part was stopping it from telling us it found things that were not there.

This is the build story of Nemesis Red, and specifically the problem everyone building LLM security tooling smashes into: hallucinated findings.

Why most LLM pentest demos are demoware

Point a model at a target, ask it to find vulnerabilities, and it will hand you a confident, well-formatted list. Some of it is real. Some of it is invented. From the text alone you cannot tell which, and in security a false positive is not a rounding error. It burns an analyst's afternoon, it erodes trust in the tool, and at scale it makes the output worthless.

A model that says "the login form is vulnerable to SQL injection" has told you nothing you can act on. A model that says "here is the injection, here is the exact request I sent, here is the database version it returned" has told you everything. The gap between those two sentences is the entire product.

ReasonChain: prove it or it did not happen

The core of Nemesis Red is an engine we call ReasonChain. The rule it enforces is easy to state and annoying to build: the model is never trusted on its own word. Every claim it makes gets executed against the real target and checked against ground truth before it counts as a finding.

The loop:

The model forms a hypothesis. "This parameter looks injectable."
It selects and runs a real tool to test that hypothesis, against the actual target, not a simulation.
The result is checked against an objective signal. Did the payload return the data. Did the shell open. Did the scoreboard flip.
Only a hypothesis that survives step 3 becomes a finding. Everything else is discarded, including the things the model was completely sure about.

This is closer to how a human pentester works than to how a chatbot works. You do not write "SQLi confirmed" in a report because the app looked suspicious. You write it because you pulled a row out of the database. ReasonChain holds the model to that same standard.

The effect on output quality is the whole point. The model is allowed to be wrong in private as many times as it likes. It is not allowed to be wrong in the report.

Driving 290 tools instead of reimplementing them

We did not try to teach the model to exploit things from scratch. The offensive tooling already exists and is battle-tested. Nemesis Red orchestrates around 290 tools from the Kali ecosystem across 17 categories, and the model's job is to be the operator: pick the right tool for the current phase and target type, run it, read the output, and decide the next move.

That framing matters. The model is not a hacker in a vacuum, it is a decision layer on top of tools that already work. Recon feeds exploitation feeds reporting in a closed loop, with the model choosing the path and the tools doing the actual work. When the model picks wrong, the verification step catches it, because a tool that finds nothing produces no ground-truth signal to promote into a finding.

We measured it, and kept the methodology attached

Claims about "AI finds more" are cheap, so here is ours with the method attached.

In a controlled ablation, Nemesis Red produced 24.4 times more valid findings than fixed-script automation on the same targets. The setup: 30 OWASP-class targets, 120 runs, across four conditions. The full system, then three ablations that each remove one capability: no replanning, no fusion of results across steps, and randomized tool order instead of reasoned order. Wilcoxon signed-rank, W = 465, p < 0.001.

The ablations are the interesting part, because they say where the lift comes from. Removing replanning hurts. Removing cross-step fusion hurts. Randomizing the order the model would otherwise reason its way to hurts. The gain is not "we called an LLM," it is the reasoning loop wrapped around the tools. Strip the reasoning and you are back to scripted scanning.

Receipts

On known-vulnerable targets during the study, Nemesis Red autonomously identified and exploited three critical vulnerabilities rated CVSS 9.8, and confirmed each with a working exploit rather than a scanner flag:

CVE-2024-38476 (Apache mod_rewrite SSRF)
CVE-2024-38474 (Apache mod_rewrite substitution)
CVE-2023-25690 (HTTP request smuggling)

To be precise about what that is and is not: these are known, published N-day CVEs, not novel zero-days. The result is that the system chained a full exploit against each on its own and proved it, with no human steering the steps. Novel zero-day discovery is a different and harder problem, and we treat it as a separate thing rather than blurring the two.

Local by default, because pentest data is sensitive

A detail that matters for anyone who would actually run this: you should not have to ship a client's live vulnerabilities to a third-party API to use the tool. Nemesis Red is bring-your-own-key for the frontier models (Anthropic, OpenAI, Gemini) if you want their reasoning, but it also runs fully local against an Ollama model (Qwen, Llama, or your own fine-tune). Local is slower and a little less sharp, but for engagements where the target data cannot leave the environment, "a little less sharp and it never leaves the box" is the correct trade.

Three ways to run it

The same engine runs at three levels of autonomy, depending on how much you want to hand over:

Copilot. A human drives, the model reads tool output and pre-fills the next moves. For testers who want speed without giving up the wheel.
Autonomous vulnerability assessment. The model runs the discovery and verification loop and hands you a verified findings list.
Autonomous penetration test. Recon to exploit to report, hands off.

What it gets wrong, honestly

Before verification, the model is exactly as overconfident as every other LLM. The entire architecture exists because raw model output is not trustworthy, and if you deleted the verification layer you would have the same demoware problem as everyone else. Verification also costs time and tool executions, so a run is not free. And autonomy needs guardrails: a system driving offensive tools on its own has to be scoped hard to the authorized target set, which is a first-class design concern, not an afterthought.

None of that is a reason not to build it. It is the reason to build it with proof in the loop instead of vibes.

Where this fits

Nemesis Red is the offensive half of what we are building. The defensive half, Nemesis Blue, is a Rust EDR with a completely different build story that I wrote up separately.

If you want to see it run or get your hands on it, we are at nemesislabs.xyz.

We built a Rust EDR. macOS notarization tried to kill it.

OBI EBUKA DAVID — Wed, 08 Jul 2026 02:19:08 +0000

Over the last few months we shipped Nemesis Blue, an endpoint detection and response agent written in Rust, notarized and running on macOS. This is the build story: why Rust, what notarization actually costs you, the entitlement bug that ate a week, and how you do detection engineering inside Apple's hardened runtime.

No marketing here. Just the parts that were hard.

Why Rust for an EDR

An EDR runs on every machine you protect, with high privilege, all the time. That job description rules a lot of things out. A garbage-collected runtime that pauses mid-scan is not acceptable. A memory-safety bug in an agent that sits just above the kernel is an incident waiting to happen, on your customer's box, with your signature on it.

Rust gave us three things that mattered:

No GC pauses. Scans run in bounded time, every time.
Memory safety without a runtime. The agent ships as a single static binary.
One codebase across platforms. Our crates (nemesis-core, nemesis-scan, nemesis-detect, and friends) build for macOS, Linux, and Windows sensors out of the same Cargo workspace.

That last decision paid off later: once the macOS agent was solid, the Linux build came up end to end in days instead of weeks, because everything below the platform layer was already shared and tested.

Notarization is the part nobody warns you about

If you want a macOS binary to run on someone else's machine without a scary Gatekeeper wall, you notarize it. To notarize, Apple requires the hardened runtime. And the hardened runtime quietly changes the rules your process runs under: no unsigned executable memory, no loading unsigned libraries, restricted debugging, and by default, no JIT.

That "no JIT" line is where we lost a week.

The SIGKILL that ate a week

Our scanning engine is built on YARA-X, the Rust rewrite of YARA. YARA-X compiles rules down and runs them on wasmtime. Wasmtime is a JIT: to execute, it allocates memory, marks it executable, and jumps into it.

Under a normal build, this is invisible. Under the hardened runtime, the kernel refuses to hand you executable-writable pages. The moment the scanner tried to run its first rule, the whole daemon died. Not an exception we could catch. An immediate SIGKILL, with Console showing a code-signature violation.

The symptom was maddening because everything worked in development and died the instant it was signed and hardened. Same binary, different fate, depending on entitlements.

The fix is one entitlement:

<key>com.apple.security.cs.allow-jit</key>
<true/>

Add it, re-sign, re-notarize, and wasmtime can allocate its JIT pages again. Obvious in hindsight. Not obvious at 2am when your only clue is a process that vanishes without a stack trace.

The lesson we took from it: on macOS, your entitlements are part of your runtime behavior, not a checkbox at the end. If a dependency touches executable memory, dynamic libraries, or the network, that shows up here or your process dies in ways your test suite never sees.

Detection engineering under the hardened runtime

With the engine actually running, the real work is detection. Nemesis Blue runs three layers:

Signatures. YARA-X rules for known families and behaviors. Fast, explainable, and the thing an analyst can read and trust.

Machine learning. An XGBoost model trained on the EMBER feature set for static classification of executables. The model is the interesting supply-chain problem: if an attacker can swap your model file, they own your verdicts. So model bundles are signed with Ed25519, and the agent verifies the signature before it will load a model. A model that does not verify does not run. Retraining happens weekly in CI, and it only publishes when the labeled corpus is large enough to be worth shipping. New models roll out to a canary slice of the fleet first, so a bad training run degrades a handful of machines instead of all of them.

IOC matching and quarantine. Indicator feeds for the known-bad, and quarantine to pull a file out of harm's way once something fires.

None of these are novel on their own. The engineering is in making all three run in bounded time, on someone else's laptop, without being the reason their fan spins up.

The 78-second freeze

Memory scanning is where we hit the ugliest performance bug. Our first pass at walking a process's memory regions shelled out and parsed the output. On small processes it was fine. On a large one, it serialized behind a single call that blocked for 78 seconds. For an agent that is supposed to be invisible, 78 seconds of a pegged process is not a bug, it is an eviction notice.

The fix was to stop treating memory enumeration as a subprocess and read the regions directly, with hard caps on region size and early skips for the mappings we never scan. Bounded work, off the hot path. The 78 seconds became milliseconds.

This is the recurring theme of EDR work: correctness is table stakes, but the thing that gets you uninstalled is being heavy. Every scan has to answer "what did this cost the user."

Shipping, and staying shipped

The agent ships as a notarized .pkg. But shipping once is not the hard part. Keeping a fleet current is.

Nemesis Blue agents check for updates at boot and every few hours after, on a stable channel, and self-install a newer signed and notarized package when one is published. Pushing a release is: bump the version, build the package, notarize it, publish the manifest. Because updates are gated behind signing and notarization, a compromised update server still cannot push a payload the agents will accept. The agents only run what Apple and our signing key both vouch for.

What it honestly does today

Here is the part most product posts skip. As of today, Nemesis Blue does detection: scanning, YARA-X, the ML classifier, IOC matching, quarantine, and monitoring, all in a notarized agent you can install right now.

What it does not do yet is real-time blocking. Full inline enforcement (killing a malicious process before it executes, streaming kernel events, process attribution) depends on Apple's Endpoint Security entitlement, which is a manual review with a multi-week queue. We have it submitted. Until it clears, enforce mode stays behind the flag, and we would rather say that plainly than imply a capability we cannot ship. The moment the entitlement lands, the blocking path is already built and waiting for it.

Takeaways

If you are building a signed, notarized native agent on macOS:

Treat entitlements as runtime behavior. allow-jit for anything wasm or JIT, and audit every dependency for what the hardened runtime will forbid.
Sign your models, not just your binary. A swapped model is a swapped verdict.
Measure what every scan costs the user. Heavy gets you uninstalled faster than wrong.
Be honest about what is gated. The entitlement queue is real, and pretending otherwise just burns trust with the exact people who will read your Console logs.

Nemesis Blue is the defensive half of what we are building at Nemesis Labs. The offensive half, Nemesis Red, is an autonomous pentest engine, and it has its own build story that is nothing like this one. That post is next.

If you want to follow along or kick the tires, we are at nemesislabs.xyz.