DEV Community

Anoymask
Anoymask

Posted on

Project Quarry: Open-Source DSL to Detect Multi-Step AI Agent Attacks Using Finite-State Automata

#security #threatintel

1. Basic Information

2. Executive Summary

Cisco has released Project Quarry, a tool that evaluates AI agent messages, tool calls, and results in chronological order to detect and explain distributed attack patterns by compiling them into symbolic finite-state automata.

This mechanism evaluates tool-use chains that cannot be seen in a single prompt. While the automaton portion is deterministic, the reproducibility of optional model scoring is separate, and no generalizable public benchmark has been provided.

3. Research Objective

  • Determining whether multi-step AI agent attacks can be detected while preserving event order and tool context.

4. Proposed Methodology

  • Compiling DSL rules into symbolic finite-state automata for deterministic evaluation. When using optional model scoring, this does not mean the model judgment itself becomes deterministic.

5. Inputs and Outputs

Inputs

  • Messages
  • Tool calls and results
  • Ordered events

Outputs

  • Matched rules
  • Witness evidence
  • Captures/counts
  • Explanations

6. Workflow

  1. Parse/validate
  2. Compile
  3. Evaluate event stream
  4. Extract evidence
  5. Return detection results and evidence to the caller. Alerting and blocking depend on the implementation and configuration of the system that integrates Quarry.

7. Evaluation Design

  • Public Benchmark: None
  • Counterfactual Case Study: Available
  • Note: Not executed against the 17,600-action incident data.

8. Results

  • Code, DSL, and API have been published.
  • Generalizable accuracy and performance results have not been published.

9. Practical Implications

  • Rules can be reused across runtime guardrails, post-hoc evaluation, and incident response.

10. Success Criteria

  • Required traces must be retained.
  • Benign and malicious chains must be distinguishable.

11. Limitations

  • Alpha status with potential compatibility changes.
  • No public benchmark.
  • Model judgments impact cost, latency, and reproducibility.
  • Keyword stubs in public semantic rule examples should not be viewed as production-grade semantic evaluation or performance validation.

Uncertainties and Areas for Further Investigation

  • No public benchmark has been provided to establish how the published examples compare with production rules.

12. Facts, Inference, and Hypothesis

Facts

  • Quarry is a detection DSL and engine that takes messages, tool calls, results, and event order as inputs.
  • Rules are compiled into symbolic finite-state automata after parsing and validation; failing rules are not loaded.
  • It supports RE2, structured fields, multi-turn flows, captures, counts, typed parameters, and optional model scoring.
  • It provides witness evidence, explanations, sync/async APIs, and resource/model-call limits.
  • The repository is in alpha, and the DSL may change prior to version 1.0.
  • The 17,600-action incident example is a counterfactual case study: Cisco did not run Quarry on that incident dataset.
  • Precision/recall, false-positive rates, and latency benchmarks have not been provided.
  • Public repository semantic rule examples use deterministic keyword stubs. These are for example/test purposes and do not demonstrate production model classifiers or actual detection accuracy in real environments.

Inference

  • Inference: Organizations must measure accuracy, latency, and cost using their own agent traces prior to production deployment.

Hypothesis

No additional hypotheses. Unconfirmed items are listed under uncertainties and areas for further investigation.

13. MITRE ATT&CK Mapping

Because the research framework itself is not an attack campaign, direct mapping is not applicable.

14. Application Guidance and Required Evidence

Application Guidance

  • Start with high-confidence rules in shadow mode.
  • Combine tool allowlists with approval gates.

Required Evidence

  • Compilation results
  • Matched event sequences
  • Precision/recall
  • Latency/cost
  • Version reproducibility

15. Implications for Defenders

It is necessary to preserve not only prompts but also tool calls, arguments, results, and ordering to make attack chains reproducible.

16. Summary by Role

  • SOCs: Integrate alerts with witness evidence into the SIEM.
  • Administrators: Configure version pinning, rule reviews, and resource limits.
  • Users: Monitoring design is primarily the responsibility of teams developing and operating AI agents.

Top comments (0)