Crucible: Pytest for AI Agents

One thing becoming increasingly obvious while working with AI systems:

Testing functionality is not enough.

An AI agent may:

• complete tasks correctly

• respond fluently

• appear fully functional

…while still failing under adversarial or unexpected inputs.

That becomes dangerous once agents gain:

• memory
• tool access
• autonomy
• long-running execution

Traditional software testing doesn’t fully cover behavioral failures in AI systems.

So we started building Crucible:

An open-source security testing framework for AI agents.

Core capabilities:

• adversarial testing

• prompt injection evaluation

• behavorial monitoring

• security scanning for agentic systems

Simple setup:

**pip install crucible-security

crucible scan --target https://your-agent-endpoint.com
**

Built with:

Python • FastAPI • LangChain • Pydantic

Open-source under Apache 2.0.

GitHub:

https://github.com/crucible-security/crucible