DEV Community: Crucible Security

AI Security Doesn't End at Deployment

Crucible Security — Tue, 30 Jun 2026 16:49:16 +0000

Many software teams have embraced continuous integration and continuous deployment.

AI systems deserve the same mindset for security.

The moment an AI agent reaches production, its environment begins to change.

New users interact with it.

New tools are connected.

Enterprise data evolves.

Memory grows.

External systems change.

Even if nothing inside the model changes, the system around it does.

That means a security assessment performed before launch is only one snapshot in time.

As AI applications become more dynamic, security has to become continuous.

The future isn't just "secure by design."

It's secure throughout operation.

That's one of the long-term ideas behind Crucible: making AI security a continuous engineering practice rather than a one-time release checklist.

Pytest for AI Agents.

opensource

cybersecurity

python

buildinpublic

aiagents

Measuring an Open Source Project by Trust, Not Downloads

Crucible Security — Tue, 30 Jun 2026 16:22:58 +0000

Open source has plenty of metrics.

Stars.

Forks.

Downloads.

Installs.

Clones.

They're useful, but they don't tell the whole story.

The number we care about most is how many developers decide to give the project a chance.

Over the past 14 days, more than 180 unique developers cloned the Crucible repository.

During the last month, the package recorded 1,330+ PyPI installs.

Those numbers don't mean we've "made it."

They mean people believe AI agent security is a problem worth solving.

To everyone who installed Crucible, opened an issue, suggested an improvement, submitted a pull request, or simply shared feedback—

Thank you.

Open source is built by communities, not companies.

We're excited for what's next.

Pytest for AI Agents.

opensource

buildinpublic

cybersecurity

artificialintelligence

aiagents

AI Security 2.0 Starts Where Model Security Ends

Crucible Security — Tue, 30 Jun 2026 16:15:36 +0000

The first generation of AI security focused on one objective:

Protect the language model.

That solved an important problem.

But today's AI applications have become much more capable.

Modern AI agents:

execute tools
access enterprise systems
maintain memory
browse the web
orchestrate workflows
communicate with other services
operate over long conversations

Each capability introduces a new trust boundary.

That means the attack surface is no longer limited to prompt injection or jailbreaks.

It includes behavior.

Permissions.

Memory.

Infrastructure.

Integrations.

The next generation of AI security isn't about replacing model security.

It's about expanding it.

That's the direction behind Crucible: testing deployed AI agents as complete systems so developers can understand how they behave in production, not just how they respond to a single prompt.

As AI systems evolve, security has to evolve alongside them.

Pytest for AI Agents.

cybersecurity #artificialintelligence #opensource #githubopensource #security #aiagents

AI Security Isn't a Product. It's an Engineering Discipline.

Crucible Security — Tue, 30 Jun 2026 15:50:41 +0000

As AI systems become more capable, security can no longer be treated as a one-time activity.

A single scan before release isn't enough.

Production AI changes.

Models evolve.

Prompts change.

Tools are added.

Memory grows.

Integrations expand.

Security has to evolve alongside the system.

That's why we approached Crucible differently.

Instead of building another standalone AI security scanner, we focused on creating an engineering platform that integrates with existing development workflows. From HTTP-native testing and asynchronous execution to multiple reporting formats and CI/CD integration, the goal is to make AI security something teams can run continuously—not something they remember to do before a release. These capabilities are reflected throughout the platform's architecture and reporting pipeline.

Trustworthy AI isn't achieved with one successful scan.

It's built through repeatable engineering practices.

AI Security Has Evolved. Has Your Testing?

Crucible Security — Mon, 29 Jun 2026 16:56:24 +0000

For years, AI security focused on the language model.

That made sense.

The model was the application.

Today, that's no longer true.

Modern AI systems are agents.

They connect to APIs.

They use tools.

They store memory.

They browse the web.

They coordinate workflows.

They interact with enterprise systems.

The attack surface has expanded far beyond prompt injection.

Security testing has to expand with it.

Crucible was designed around this new reality. Instead of treating an AI application as a single model, it evaluates the complete system through its production interfaces, including tools, memory, browser interactions, MCP servers, and multi-turn behavior. That approach reflects the way AI agents are actually deployed and used.

The future of AI isn't just smarter models.

It's more capable agents.

And capable agents require more comprehensive security.

Pytest for AI Agents.

cybersecurity #artificialintelligence #opensource #githubopensource #security #aiagents

AI Models Aren't Your Product. AI Agents Are.

Crucible Security — Mon, 29 Jun 2026 16:46:30 +0000

When most people think about AI security, they think about testing a language model.

Can it resist prompt injection?

Can it resist jailbreaks?

Can it avoid hallucinations?

Those questions matter—but they're only part of the picture.

Modern AI applications are no longer just models.

They're agents.

Agents call tools.

They access APIs.

They maintain memory.

They connect to enterprise systems.

They interact with MCP servers.

They execute workflows.

Every one of those capabilities introduces new security boundaries that a model-only evaluation can miss.

That's why Crucible was built around deployed HTTP-accessible AI agents rather than raw LLMs. It evaluates the system the way users experience it in production: as a complete application with tools, permissions, memory, and behavior—not just a text generator.

The future of AI security isn't just testing what models say.

It's testing what agents do.

Pytest for AI Agents.

cybersecurity #artificialintelligence #opensource #githubopensource #security #aiagents

The Most Dangerous AI Payload Might Not Exist Yet

Crucible Security — Mon, 29 Jun 2026 14:54:54 +0000

Security testing often relies on historical attacks.

Known payloads.

Known jailbreaks.

Known prompt injections.

That's necessary—but it's not sufficient.

Attackers constantly adapt.

A small wording change.

A different structure.

An unexpected context.

Sometimes that's all it takes to bypass a defense that worked yesterday.

That's why modern AI security testing should move beyond replaying known attacks.

Crucible's Shadow Payload Generator is designed to create new adversarial payloads inspired by existing attack patterns, helping developers evaluate how resilient their systems are to variations—not just exact matches.

Because production AI shouldn't only defend against what we've already discovered.

It should be prepared for what comes next.

Pytest for AI Agents.

cybersecurity

artificalintelligence

opensource

githubopensource

security

buildinpublic

aiagents

Static Payloads Don't Represent Real Attackers

Crucible Security — Mon, 29 Jun 2026 14:43:49 +0000

Many AI security tools rely on fixed payload libraries.

That's a good starting point.

But attackers rarely repeat the exact same prompt.

They rewrite it.

They obfuscate it.

They change wording.

They manipulate context.

They keep adapting until something works.

Security testing should reflect that reality.

That's why Crucible includes a Mutation Engine that transforms base attack payloads into multiple variations before executing them. Instead of evaluating whether an agent blocks one exact prompt, it evaluates whether the underlying defense remains effective as the attack evolves.

This shifts testing from pattern matching to resilience testing.

Because defending against one prompt is useful.

Defending against an entire family of attacks is far more valuable.

Pytest for AI Agents.

cybersecurity

artificalintelligence

opensource

buildinpublic

aiagents

Why One Prompt Isn't Enough to Test an AI Agent

Crucible Security — Sun, 28 Jun 2026 16:30:44 +0000

Most AI security testing still revolves around a simple pattern:

Send one prompt.

Read one response.

Move on.

That works for testing isolated behavior.

It doesn't reflect how AI agents are actually used.

Real interactions unfold over multiple turns.

Context accumulates.

Memory influences decisions.

Trust builds over time.

Attackers know this.

Rather than attempting an obvious exploit immediately, they may gradually influence an agent's behavior through a sequence of seemingly harmless interactions.

That's why Crucible includes a Behavioral Drift Engine together with multi-turn attack strategies. Instead of evaluating a single response, it analyzes how an agent's behavior changes across an entire conversation, helping identify risks that only emerge after context has evolved.

Security testing should model the way AI is used in production—not just how it responds to one message.

Because one prompt tells you what an AI says.

Multiple turns tell you how it behaves.

Pytest for AI Agents.

cybersecurity

artificalintelligence

opensource

buildinpublic

aiagents

Why AI Security Needs More Than a Few Test Prompts

Crucible Security — Sun, 28 Jun 2026 16:21:10 +0000

A common mistake in AI security testing is assuming that a handful of prompts can accurately measure an agent's resilience.

In reality, attackers don't repeat the same technique.

They change tactics.

They chain attacks.

They exploit new trust boundaries.

That's why Crucible includes 170+ attack vectors spread across multiple security categories.

Rather than focusing on a single vulnerability class, it evaluates how an AI agent responds to a broad range of adversarial behaviors—from prompt injection and jailbreaks to memory poisoning, MCP security, browser agents, infrastructure escalation, and more. These attack vectors are organized into dedicated modules so testing remains structured while covering a wide range of AI-specific risks.

The goal isn't to overwhelm developers.

It's to make comprehensive security testing repeatable, automated, and practical.

Because real-world AI systems deserve more than a few manually written prompts.

Pytest for AI Agents.

cybersecurity #artificalintelligence #opensource #buildinpublic #aiagents

AI Agents Don't Have One Attack Surface

Crucible Security — Sun, 28 Jun 2026 15:59:39 +0000

One of the biggest misconceptions in AI security is that a single test is enough.

Run a few prompt injection payloads.

Check the responses.

Ship to production.

Modern AI agents are far more complex.

A production agent may interact with tools, memory systems, browsers, MCP servers, enterprise knowledge bases, cloud infrastructure, and external APIs. Each of those components introduces different security risks.

That's why Crucible organizes testing into 13 dedicated security modules, each focused on a different class of vulnerabilities. Instead of producing isolated results, the framework combines those findings into a single report, giving developers a broader view of their agent's security posture.

Security isn't about asking:

"Did we test prompt injection?"

It's about asking:

"What parts of our AI system haven't we tested yet?"

Comprehensive coverage is what turns security testing into production confidence.

Pytest for AI Agents.

cybersecurity

artificalintelligence

opensource

buildinpublic

aiagents

Why AI Security Testing Needs to Be Measured in Seconds, Not Hours

Crucible Security — Sun, 28 Jun 2026 15:14:14 +0000

The biggest challenge with AI security isn't knowing what to test.

It's finding the time to test everything.

Traditional security reviews are often manual:

Write a payload.
Send it.
Inspect the response.
Try another variation.

That approach quickly breaks down as AI systems grow more complex.

Crucible takes a different approach.

Instead of testing one prompt at a time, it automatically launches 90+ adversarial payloads in under 60 seconds against any HTTP-accessible AI agent and generates a structured security report.

Those payloads span multiple attack categories, including prompt injection, goal hijacking, jailbreaks, MCP security, memory poisoning, and tool injection. The framework is designed to integrate with CI/CD pipelines so testing can happen continuously instead of only before release.

Fast security testing isn't just about convenience.

It's what makes continuous AI security practical.

Because if testing takes hours, teams won't run it on every deployment.

If it takes under a minute, they can.

Pytest for AI Agents.