Adding Humanbound's Firewall on Top of Your PyRIT Stack

Humanbound's firewall training can import Microsoft PyRIT's red team results directly (hb firewall train --import pyrit_results.json), turning a one-time PyRIT engagement into training data for an ongoing, self-improving production defense instead of a report that sits in a shared drive.

PyRIT, the Python Risk Identification Toolkit, comes from a different world than most AI security products. Microsoft's AI Red Team started building it as a set of internal scripts back in 2022, before most companies were thinking about generative AI risk at all, and open-sourced it in 2024. It's model- and platform-agnostic, supports multi-turn attack strategies like Crescendo and Skeleton Key, and includes a GUI for human-led red teaming alongside its automation. It's a serious research tool, built by and for people who do this for a living.

That's exactly why it pairs well with Humanbound instead of overlapping with it.

Research depth versus operational coverage

PyRIT is designed for red teamers who want deep control: composable building blocks, custom attack scenarios, and the flexibility to probe for novel harms that don't fit a pre-built template. It's most often used in dedicated, point-in-time red teaming exercises, run by a security researcher or red team, producing a rich but time-bound set of findings.

Humanbound is built for what happens on either side of that exercise. Before it, hb test runs OWASP-aligned adversarial and behavioral testing without requiring a dedicated red teamer to hand-craft each scenario. After it, continuous monitoring keeps testing the agent on a schedule, tracks whether findings get fixed or regress, and produces a posture score, 0 to 100, that turns a one-time PyRIT engagement into an ongoing trend line.

Neither replaces the other. A PyRIT engagement finds things a template-driven scanner might miss. Humanbound keeps testing for those things, and everything else, long after the engagement ends.

Where they connect: the same firewall pipeline as Promptfoo

Humanbound's firewall training explicitly supports importing PyRIT's scan output, auto-detected by its redteaming_data key:

hb firewall train --import pyrit_results.json

# Combine PyRIT with other sources in one training run
hb firewall train --import pyrit.json --import promptfoo.json

That means a PyRIT red team engagement doesn't have to end as a PDF report that sits in a shared drive. Its findings become training data for Humanbound's Tier 2 agent-specific classifier, the layer of the Humanbound Firewall that catches attack patterns generic models miss. The research your red team did manually gets encoded into a runtime defense that keeps working after the engagement is over.

What this looks like in practice

For teams with a dedicated security or AI red team function, a natural workflow looks like:

1. Run a PyRIT engagement against a new agent before launch, using its multi-turn strategies to probe for risks specific to that use case.
2. Import the results into Humanbound's firewall training, alongside Humanbound's own test logs, to seed the Tier 2 classifier with what the human researchers found.
3. Turn on continuous monitoring so the agent keeps getting tested on a schedule, with regressions and drift tracked automatically, not just re-assessed the next time someone schedules another PyRIT exercise.

This is particularly relevant for regulated or public-sector teams, where a documented, research-grade red team exercise is often expected as part of AI governance, and where Humanbound's compliance mapping (EU AI Act, NIST AI RMF) gives that exercise a continuous, auditable trail afterward instead of a single snapshot.

Try it yourself

Both tools are open source, so there's nothing stopping you from testing this pairing today:

pip install humanbound
hb firewall train --import pyrit_results.json

If PyRIT is already part of your red teaming practice, the fastest way to see this work is to take your next engagement's output and run it through hb firewall train --import. The research doesn't stop being useful the day the report is filed, it becomes the foundation the firewall keeps learning from.

Have you combined a manual red team exercise with an automated firewall or guardrail layer before? Curious how other teams are closing that loop, drop it in the comments.

Sources:

• PyRIT — Python Risk Identification Tool Documentation
• Announcing Microsoft's open automation framework to red team generative AI Systems