DEV Community

floworkos
floworkos

Posted on

Building a Security Scanner for AI Agents — Flowork's Threat Radar Architecture

#security #ai #devops #productivity

Why a Security Scanner Matters for AI Agents

An AI agent is a program that decides what to run, where to run it, and when. Unlike a human operator, it has no natural instinct to pause and ask "should I really do this?" Before shipping an agent into production — or even testing one — you need real visibility into the code it might execute. Flowork's Threat Radar is built-in security radar that watches the code your agents run, live.

No other agent framework ships this. Most treat security as an afterthought or a separate toolchain you bolt on later. Here, it's part of the microkernel.

The Live Dashboard

Open Threat Radar and you see a radar sweep on the left with three numbers:

  • Runs — how many scans have completed on this agent.
  • Findings — total issues discovered across all scans.
  • Critical — the severity of the worst active issue. This number is red if anything critical is live, green if clean. Critically, it reflects the latest scan of each target, so when you fix something, the red fades.

On the right: a Scan Log (every scan, newest first) and a Findings panel (click any run to drill into what it found). The dashboard refreshes every few seconds without you needing to refresh the page.

Two Scanning Modes

The ⊕ Scan Target button opens a form with four fields:

  1. Tool — which scanner to use (e.g., a static code auditor, a dynamic checker).
  2. Target — what to scan (your own agent code, or an authorized external target).
  3. Args — optional flags or configuration the tool needs.
  4. Category — either immune (hardening your own code) or pentest (an authorized external target you have permission to test).

Both the tool list and target list come from an owner-editable allowlist. Flowork refuses to run any tool or touch any target that isn't on it. There's no shell in the middle, no arbitrary command execution — only what the owner explicitly permits.

The Arsenal: Defensive Code Auditors + Thousands of Checks

At the top-right, ≣ Arsenal opens a catalog of everything the scanner can use:

  • Defensive code auditors (marked CORE) — the foundation, can't be uninstalled.
  • Tools — specific scanners you can enable or disable.
  • Detection checks — thousands of them, searchable, installable as packs.

This is where you customize what the scanner looks for. Install a new detection pack and it snaps into your arsenal instantly, without a restart.

Building Your Own Checks

A check is a nuclei template — a small YAML file that says "look for this pattern." Nuclei is battle-tested in the security industry; we use it as the language for checks because it's declarative, readable, and safe (templates run inert — they describe what to look for, not what to do).

A Simple HTTP Check

Here's a check that looks for exposed .env files:

id: exposed-env-file
info:
  name: Exposed .env file
  author: you
  severity: high
http:
  - method: GET
    path:
      - "{{BaseURL}}/.env"
    matchers:
      - type: word
        words:
          - "DB_PASSWORD"
Enter fullscreen mode Exit fullscreen mode

The template says: "Make a GET request to /.env on the target. If the response contains DB_PASSWORD, flag it as a high-severity issue."

Adding a Single Check

POST it to /api/scanner/checks/add with a JSON body:

{
  "name": "exposed-env-file",
  "yaml": "id: exposed-env-file\ninfo:\n  name: Exposed .env file\n  author: you\n  severity: high\nhttp:\n  - method: GET\n    path:\n      - \"{{BaseURL}}/.env\"\n    matchers:\n      - type: word\n        words:\n          - \"DB_PASSWORD\""
}
Enter fullscreen mode Exit fullscreen mode

The check runs through nuclei -validate on arrival. If it's valid, it lands in <nuclei-templates>/flowork-private/ and shows up in the Arsenal.

Bundling Many Checks into a Pack

For a larger scanner or a collection of related checks, create a .fwpack:

my-scanner.fwpack
├─ plugin.json
└─ checks/
   ├─ check-1.yaml
   ├─ check-2.yaml
   └─ ...
Enter fullscreen mode Exit fullscreen mode

The plugin.json defines the pack:

{
  "id": "my-scanner",
  "kind": "scanner",
  "scanner": {
    "name": "My Custom Scanner",
    "description": "Detects configuration flaws and credential leaks"
  }
}
Enter fullscreen mode Exit fullscreen mode

Install it with /api/scanner/packs/install. Every check is validated; the rest snap into the Arsenal instantly.

Safety Guarantees

  • Owner-only — only the owner can add tools, targets, or checks.
  • Every check is validated — all YAML is run through nuclei -validate before it lands in your system.
  • Templates run inert — a nuclei template describes what to look for, not what to execute. No shell escapes, no code injection.
  • Allowlist enforcement — scans only ever touch tools and targets explicitly on your allowlist. You control what the scanner can touch.
  • No external calls — everything stays on your machine unless you explicitly point a target to an external system (with your permission).

Why This Matters for Self-Hosted AI

A self-hosted agent runs code on your hardware. That code could be:

  • A tool call (fetch data, run a script, send a message).
  • An external MCP server the agent calls out to.
  • Scheduled automation that runs while you're not looking.

Threat Radar gives you real-time visibility into what's safe and what isn't. When your agent asks to run something, you have a scanner that:

  • Knows what bad looks like — thousands of rules for common flaws (exposed secrets, unpatched versions, common misconfigurations).
  • Catches problems early — before a mistake reaches production.
  • Stays under your control — no cloud, no external API, no third party holding your scan results.

For a self-hosted system, that's not a luxury — it's a requirement. You own the agent, you own the risks, and you own the scanner.

Scanning in Practice

A typical workflow:

  1. Agent is deployed → runs for a while → makes a tool call.
  2. You suspect something's wrong → open Threat Radar, click ⊕ Scan Target, pick the agent, and run a scan.
  3. Findings come back → you see issues ranked by severity.
  4. You fix the code → restart the agent.
  5. You scan again → the critical count drops back to green.

Or, if you prefer automation: wire Flowork's scheduler to run scans on a schedule (hourly, daily, after deploy), so new risks are caught before they live.

Trade-offs and Honesty

Nuclei templates are declarative and safe, but they describe what to look for, not what to fix. A scan tells you "there's a hardcoded password," but fixing it is still your job. We're not a SAST tool with rewrite suggestions — we're a radar that watches and alerts.

Also: a scanner is only as good as the checks it knows about. If you don't have a rule for a new vulnerability, the scanner won't catch it. The Arsenal starts with the most common patterns; you add domain-specific checks based on your own needs.

Glossary

  • Nuclei template — a YAML declaration of what to look for in a target.
  • Check — a single detection rule (one .yaml file).
  • Pack — a collection of checks bundled into a .fwpack.
  • Allowlist — the owner-editable list of tools and targets the scanner is allowed to use.
  • Category — immune (scan your own code), or pentest (scan an authorized external target).
  • Finding — an issue detected by a scan, with severity and context.
  • Arsenal — the catalog of all available tools and checks.

Threat Radar exists because self-hosted AI isn't safe if you can't see what's running. Use it.

Flowork is open source — both products:

💬 Join the Flowork community on Telegram: https://t.me/+55oqrk75lc43YWE1

Top comments (0)