DEV Community

suraj kumar
suraj kumar

Posted on

swarm-test v0.3.2 — Write Your Own Multi-Agent Reliability Tests

#ai #python #testing #opensource

swarm-test v0.3.2 adds a plugin system. You can now write custom reliability tests for your specific multi-agent architecture.

The built-in tests cover universal failure modes — cascade failures, context leakage, intent drift, collusion, blast radius, timeout resilience, sensitive data detection, and contract violations. But every team has domain-specific risks that a generic tool can't anticipate. Maybe you need to check that your billing agent never communicates directly with your data deletion agent. Maybe you need to verify that no agent chain exceeds 5 hops. Maybe you have compliance requirements unique to your industry.

Now you can build those checks yourself and they run alongside everything else.

Writing a plugin takes about 10 lines:

from swarm_test.plugins import BasePlugin, PluginResult
from swarm_test.core.models import Finding

class MaxHopsPlugin(BasePlugin):
name = "max_hops_check"
version = "0.1.0"
description = "Warns if any agent chain exceeds N hops"

  def run(self, graph, agents, edges, config):
      findings = []
      # your test logic using the NetworkX graph
      return PluginResult(
          test_name=self.name,
          status="passed" if not findings else "failed",
          score=100,
          findings=findings,
          duration_ms=0.0
      )
Enter fullscreen mode Exit fullscreen mode

Register it in your package's pyproject.toml:

[project.entry-points."swarm_test.plugins"]
max_hops_check = "my_package:MaxHopsPlugin"

Install your package and swarm-test discovers it automatically:

swarm-test plugins list

Plugin findings appear everywhere — console output, JSON export, HTML reports, GitHub Action annotations, CI/CD gates. They respect the same YAML config filtering (enabled_tests/disabled_tests) as built-in tests. One failing plugin doesn't crash the rest of the run.

The graph object your plugin receives is a full NetworkX DiGraph with all agent nodes, edges, and metadata. You have access to every graph algorithm NetworkX provides — centrality, shortest paths, connected components, community detection. The agents and edges lists give you the full swarm-test model with roles, tools, health scores, and redundancy data.

What I'd love to see the community build: rate limit validation (does any agent path exceed API rate limits), cost estimation plugins (token counting per path), compliance-specific checks (HIPAA, SOC 2, GDPR agent isolation), framework-specific tests that go deeper than the generic adapters.

If you build a plugin, open an issue on the repo and I'll add it to a community plugins directory.

pip install swarm-test --upgrade
GitHub: github.com/surajkumar811/swarm-test

Top comments (0)