Testing an AI Agent Governance Boundary with AgentGovBench | Agent Lab Journal

  AL
  Agent Lab Journal


  Guides
  Glossary

ADVANCED PRACTICE · GOVERNANCE AND SECURITY

Testing an AI agent governance boundary with AgentGovBench

      Level: advanced
      Reading and lab time: 75 minutes
      Outcome: 48 scenarios and a baseline scorecard

A model can produce the correct answer while the surrounding agent system commits a serious control failure. The request may lose the authenticated user identity on its way to a worker, a subagent may acquire permissions its parent never had, one tenant may reuse another tenant’s policy decision, and the audit log may be too incomplete to reconstruct what happened. AgentGovBench targets these system-level failures rather than the quality of the model’s prose.

What you will finish with. A pinned AgentGovBench installation, a validated inventory of 48 scenarios, an installation baseline, a run through your real enforcement path, a category-level scorecard, and an evidence-based failure review covering identity, delegation, scope inheritance, tenant isolation, rate limits, failure modes, and audit completeness.

Why ordinary model tests miss the problem

A conventional benchmark sends prompts to a model and evaluates the resulting text, classification, tool choice, or structured output. That is useful for measuring reasoning and instruction following, but it does not prove that the selected operation was authorized correctly.

Consider an apparently successful request:

User → orchestrator → mail worker → read_email → concise answer
                                      └──────────→ audit pipeline

The final answer can be accurate even when all of the following happened:

• The worker invoked the tool under a shared service account instead of the authenticated user.

• A user-specific prohibition was checked by the orchestrator but not by the worker.

• The tenant identifier came from model-controlled arguments rather than verified membership.

• The log recorded “tool completed” without the user, policy version, decision reason, or delegation chain.

A response-quality test marks the task as successful. A governance test asks different questions:

• Who was the effective subject at the tool boundary?

• Which tenant did the subject belong to?

• Which permissions were effective at that exact moment?

• Which component delegated the operation to the executor?

• Which rule allowed or rejected the operation?

• Did the tool actually receive the call?

• Is there enough durable evidence to investigate the decision later?

  The useful unit of governance testing is not the model’s answer. It is a transition across an authority boundary.
  

AgentGovBench should therefore run deterministic tool sequences wherever possible. Keeping the large language model out of the critical test path separates authorization defects from generation variance. Model-security tests remain necessary, but they answer a different question.

The 48-scenario coverage model

The target profile in this lab contains eight categories with six scenarios in each category. Before interpreting a result, verify that your pinned AgentGovBench revision actually exposes the expected profile. Do not silently add, remove, or rename scenarios to make the count fit.

          Category
          Count
          Primary assertion
          Typical failure




          identity_propagation
          6
          The verified user survives direct, queued, retried, and delegated calls
          An action is attributed to a worker or service account


          per_user_policy_enforcement
          6
          User-specific grants, denials, and revocations are applied at execution time
          A broad tenant rule overrides a personal prohibition


          scope_inheritance
          6
          Effective scope can stay equal or narrow, but cannot expand
          A child obtains an administrative capability requested in task data


          delegation_provenance
          6
          The complete delegation chain remains ordered and verifiable
          The audit event contains only the final worker


          audit_completeness
          6
          Allowed, denied, failed, and retried operations leave usable evidence
          The log proves activity but not the policy decision


          rate_limit_cascade
          6
          Rate limits follow the user and tenant across workers
          Fan-out creates a fresh budget for every child


          fail_mode_discipline
          6
          Declared fail-open and fail-closed behavior is enforced predictably
          A policy-service timeout accidentally disables protection


          cross_tenant_isolation
          6
          Tenant isolation covers data, policy, credentials, limits, cache, and audit
          A decision or credential from tenant A is reused in tenant B

Each category needs both positive and negative controls. A gateway that rejects every request may block every malicious case, but it is not a functioning governance system. The suite must prove that narrow authorized work still succeeds while unauthorized work is stopped.

Six scenarios per category should cover more than six textual variations of one request. A useful category includes direct execution, delegation, asynchronous execution, state change, an allowed control case, and an evidence check. Treat the source scenario definitions in your pinned revision as authoritative for the run.

Concrete case: Alice disappears at the queue boundary

Use a synthetic environment with two tenants. Alice belongs to tenant-a. She has email.read but does not have admin.grant_permission. An orchestrator receives her request, creates a mail worker, and sends the task through a queue.

The unsafe queue message carries only model-visible task data:

{
  "task": "Read the message and prepare a short summary",
  "mailbox": "[e-mail]",
  "agent_name": "mail-worker"
}

The worker then reconstructs authority from local defaults:

actor_uid = "service:mail-worker"
tenant_id = payload.get("tenant_id", DEFAULT_TENANT)
effective_scopes = SERVICE_ACCOUNT_SCOPES

This design has lost the authenticated user, verified tenant membership, original scope, and delegation provenance. If the policy engine evaluates the service account, Alice’s personal denial no longer matters. The service token can also expose capabilities that the user never possessed.

A safer execution envelope separates trusted authority from untrusted task input:

{
  "execution": {
    "request_id": "run-local-001-request-001",
    "tenant_id": "tenant-a",
    "subject": {
      "uid": "user-alice",
      "display": "[e-mail]"
    },
    "delegation": [
      {
        "from": "orchestrator",
        "to": "mail-worker",
        "scopes": ["email.read"]
      }
    ],
    "effective_scopes": ["email.read"],
    "policy_version": "lab-policy-v1"
  },
  "input": {
    "mailbox": "[e-mail]"
  }
}

The values are synthetic. They are not credentials and must not be replaced with production accounts during this exercise.

The input object is untrusted. It may select a mailbox or document within an already authorized boundary, but it cannot replace subject, tenant_id, delegation, or effective_scopes. The execution envelope must originate at an authenticated boundary and be integrity-protected across the queue.

Effective permissions for the child are calculated by intersection:

effective_child_scopes =
    authenticated_user_scopes
    ∩ parent_effective_scopes
    ∩ delegated_task_scopes
    ∩ tool_policy_scopes

They must never be calculated by union:

# Unsafe: requested data can add authority
child_scopes = parent_scopes | requested_scopes

This is an application of least privilege: a delegation may narrow authority for a task, but it cannot manufacture a permission that is absent from any required layer.

1. Prepare an isolated test environment

Do not point the initial run at production mailboxes, file stores, payment APIs, customer records, or live policy administration. Use tool fixtures that record requests but cannot perform external effects.

The minimum laboratory needs:

• a supported Python version for your pinned AgentGovBench revision;

• an isolated virtual environment;

• two synthetic tenants, such as tenant-a and tenant-b;

• at least three synthetic users with deliberately different permissions;

• safe fixtures for read, write, outbound, and administrative tool classes;

• a way to reset policies, caches, limits, queues, and logs between scenarios;

• an audit export independent of the benchmark runner’s in-memory state.

Obtain AgentGovBench from the source approved by your organization, check out the revision you intend to evaluate, and record that exact revision. Starting from an already cloned repository:

cd agentgovbench

git status --short
git rev-parse HEAD
git rev-parse HEAD > ../agentgovbench.commit

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e .

Do not continue from a dirty checkout unless the local changes are intentional and archived with the results. A commit hash alone cannot reproduce uncommitted modifications.

Inspect the command-line interface and repository structure instead of assuming that a command from another revision still applies:

agentgovbench --help
agentgovbench run --help

find . -type f \( -name '*.yaml' -o -name '*.yml' -o -name '*.json' \) \
  | sort \
  | sed -n '1,160p'

find . -type f -iname '*runner*.py' | sort

Locate the scenario directory for the selected profile and count scenario definitions using the layout of your revision. If each scenario is stored in a separate YAML file, a check may look like this:

find scenarios -type f \( -name '*.yaml' -o -name '*.yml' \) \
  | sort \
  | tee ../scenario-files.txt \
  | wc -l

The target for this article is exactly 48 runnable scenarios. If the command prints another number, determine whether you selected another profile, counted shared fixtures, or checked out a different revision. Do not alter the scorecard denominator manually.

2. Build fixtures that can prove side effects

A denial is not proven by the agent saying “I did not perform the action.” The fixture behind the policy boundary must show whether the request actually arrived.

A minimal fixture receipt can contain:

{
  "received_at": "2026-01-01T00:00:00Z",
  "run_id": "run-local-001",
  "request_id": "run-local-001-request-001",
  "tenant_id": "tenant-a",
  "tool": "read_email",
  "arguments_hash": "sha256-of-normalized-arguments"
}

The timestamp and identifiers above are examples of shape, not claimed benchmark output. Generate unique values during your run.

The fixture should enforce four properties:

• It never connects to a production service.

• It records only calls that actually reached the execution boundary.

• It does not trust a model-supplied tenant or actor field.

• It can be reset and queried by run_id and request_id.

Keep policy decisions and fixture receipts separate. A policy record proves what the control layer decided. A fixture receipt proves whether execution was attempted after that decision. Comparing the two reveals gaps such as “denied but executed” or “allowed but never attempted.”

3. Run an installation baseline

Use the no-governance or reference runner shipped by your pinned revision, if one exists. Its purpose is to verify scenario loading, runner startup, result serialization, and scoring—not to represent the security of your system.

Confirm the actual runner name with agentgovbench run --help. A representative invocation is:

mkdir -p results

agentgovbench run \
  --runner vanilla \
  --out results/vanilla-local.json

If your revision uses different flags, substitute only the documented CLI syntax. Do not rename a custom runner to vanilla merely to make this command work.

Verify the baseline operationally:

• All 48 scenarios were discovered.

• No installation, import, setup, or serialization exception was hidden.

• The output contains individual scenario records rather than only one total.

• Each scenario has a stable identifier, category, and status.

• The baseline and future system run will use the same benchmark revision.

Save the environment beside the output:

python --version > results/environment.txt
python -m pip freeze >> results/environment.txt
git rev-parse HEAD >> results/environment.txt

sha256sum results/vanilla-local.json \
  > results/SHA256SUMS

Do not copy a score from this article or another installation. The valid baseline is the result produced by your pinned code in your environment.

4. Connect the real control path through a runner

A runner translates abstract scenario actions into operations against your system. It creates test identities, applies policy, starts delegation, invokes tools, simulates control-plane failure, queries audit evidence, and resets state.

The runner must adapt interfaces without implementing the expected answer. If it returns “deny” because the scenario name contains deny, you are testing the adapter rather than the system.

          Scenario operation
          Required runner behavior
          Invalid shortcut




          Direct tool call
          Use the same enforcement path as the deployed agent
          Calling the fixture directly


          Delegation
          Create a real child execution context
          Appending a fake chain only during audit export


          Policy change
          Use the normal policy mechanism and wait for bounded propagation
          Changing only the runner’s expected value


          Parallel fan-out
          Create the requested logical children and concurrent calls
          Giving every child a separate user budget


          Gateway failure
          Make the decision service unavailable to test traffic
          Returning a preselected allow or deny from the runner


          Audit query
          Read the durable investigation log
          Constructing evidence from runner memory

Find the runner contract in your checkout:

grep -R "class BaseRunner" -n . --exclude-dir=.venv
grep -R "BaseRunner" -n . --exclude-dir=.venv | head -40
grep -R "def .*audit\\|def .*tool\\|def .*delegate" -n . \
  --exclude-dir=.venv \
  | head -80

Method names differ across revisions. Read the base class and the nearest included adapter before implementing yours.

Separate trusted context from model input

trusted_execution_context = {
    "run_id": scenario_run_id,
    "request_id": scenario_request_id,
    "actor_uid": authenticated_user.uid,
    "actor_display": authenticated_user.display,
    "tenant_id": verified_membership.tenant_id,
    "effective_scopes": computed_scopes,
    "delegation_chain": verified_chain,
    "policy_version": active_policy.version
}

untrusted_tool_request = {
    "tool": scenario.tool,
    "arguments": scenario.arguments
}

A test impersonation mechanism may be necessary to create synthetic subjects. Keep it isolated from production, protect it with a separate administrative capability, and record its use in the audit trail. Never accept an arbitrary actor_uid from an ordinary tool request.

Reset state explicitly

Before every scenario, reset or namespace:

• user and tenant policy;

• policy-decision caches;

• rate-limit buckets;

• queued jobs and retry counters;

• delegation records;

• fixture receipts;

• audit-query cursors.

Prefer unique run and scenario identifiers over broad deletion. The runner should also assert that the scenario starts from the intended empty or seeded state.

5. Debug one category before the full run

Begin with identity propagation because failures there contaminate policy, delegation, limits, isolation, and audit attribution.

agentgovbench run \
  --runner your_runner_name \
  --category identity_propagation \
  --out results/identity.json

Replace your_runner_name with the registered runner shown by your CLI. Then exercise the other high-impact boundaries:

agentgovbench run --runner your_runner_name \
  --category per_user_policy_enforcement \
  --out results/per-user-policy.json

agentgovbench run --runner your_runner_name \
  --category scope_inheritance \
  --out results/scope-inheritance.json

agentgovbench run --runner your_runner_name \
  --category delegation_provenance \
  --out results/delegation.json

agentgovbench run --runner your_runner_name \
  --category cross_tenant_isolation \
  --out results/cross-tenant.json

Stop interpretation if setup or teardown fails. A runner error is not evidence that the governance boundary passed or failed. Repair the adapter, reset the environment, and rerun the entire affected category.

6. Execute all 48 scenarios

After category-level debugging, reset all test state and run the complete profile:

agentgovbench run \
  --runner your_runner_name \
  --out results/your-system.json

sha256sum results/your-system.json \
  >> results/SHA256SUMS

Inspect the result structure without printing potentially sensitive arguments:

python - <<'PY'
import json
from pathlib import Path

path = Path("results/your-system.json")
data = json.loads(path.read_text(encoding="utf-8"))

print("root_type:", type(data).__name__)
if isinstance(data, dict):
    print("root_keys:", sorted(data))
elif isinstance(data, list):
    print("items:", len(data))
PY

Locate the collection that holds individual results. Verify:

• 48 records exist;

• all scenario identifiers are unique;

• eight expected categories exist;

• each category contains six records;

• every record has an interpretable status;

• declined and error were not converted into pass.

If your result uses a top-level scenarios array, this validation is suitable:

python - <<'PY'
import json
from collections import Counter
from pathlib import Path

data = json.loads(
    Path("results/your-system.json").read_text(encoding="utf-8")
)
rows = data["scenarios"]

ids = [row["id"] for row in rows]
counts = Counter(row["category"] for row in rows)

assert len(rows) == 48, f"received {len(rows)}, expected 48"
assert len(set(ids)) == 48, "duplicate scenario identifiers"
assert len(counts) == 8, counts
assert set(counts.values()) == {6}, counts

print("scenarios:", len(rows))
for category, count in sorted(counts.items()):
    print(f"{category}: {count}")
PY

If your pinned revision uses different keys, adapt only the extraction layer. Do not change scenario statuses or expected outcomes.

7. Build a baseline scorecard

An overall percentage is insufficient. One allowed cross-tenant operation may be unacceptable even when the other 47 scenarios pass. Preserve category counts and operational status separately.

          Category
          Pass
          Fail
          Declined
          Error
          Required review




          Identity propagation
          from output
          from output
          from output
          from output
          Subject loss and substitution


          Per-user policy enforcement
          from output
          from output
          from output
          from output
          Rule specificity and revocation


          Scope inheritance
          from output
          from output
          from output
          from output
          Permission intersection


          Delegation provenance
          from output
          from output
          from output
          from output
          Chain integrity


          Audit completeness
          from output
          from output
          from output
          from output
          Decision and execution evidence


          Rate-limit cascade
          from output
          from output
          from output
          from output
          User and tenant budget keys


          Fail-mode discipline
          from output
          from output
          from output
          from output
          Timeout and dependency behavior


          Cross-tenant isolation
          from output
          from output
          from output
          from output
          Release-blocking isolation defects


          Total
          calculated from the result file
          Ready / conditional / blocked

Use these status meanings consistently:

      Pass
      All required assertions were observed and the evidence is available.

      Fail
      The system produced observable behavior that contradicted an assertion.

      Declined
      The runner explicitly could not reproduce a required capability or topology.

      Error
      Setup, execution, collection, or teardown failed before an interpretable result existed.

Do not include declined or error scenarios in the pass count. Also do not quietly remove them from reporting. Preserve the reason, affected capability, owner, and proposed resolution.

A generic scorecard extractor for the example result shape is:

python - <<'PY'
import json
from collections import defaultdict
from pathlib import Path

rows = json.loads(
    Path("results/your-system.json").read_text(encoding="utf-8")
)["scenarios"]

statuses = ("pass", "fail", "declined", "error")
card = defaultdict(lambda: {status: 0 for status in statuses})

for row in rows:
    category = row["category"]
    status = row["status"].lower()
    if status not in statuses:
        raise ValueError(
            f"unknown status {status!r} in scenario {row['id']}"
        )
    card[category][status] += 1

print("category,pass,fail,declined,error")
for category in sorted(card):
    counts = card[category]
    print(
        category,
        counts["pass"],
        counts["fail"],
        counts["declined"],
        counts["error"],
        sep=","
    )
PY

A practical release barrier

Your system owner must choose the formal release policy. For high-impact boundaries, a useful starting barrier is:

• no confirmed bypass of a user-specific prohibition;

• no scope expansion during delegation;

• no allowed cross-tenant access or credential reuse;

• no fail-open behavior where policy requires fail-closed;

• an investigation-grade record for every allowed and denied tool request;

• no unexplained runner errors in a category used for release approval.

This barrier is an operational recommendation, not a claimed default rule of AgentGovBench. Align it with your own threat model and risk ownership.

Failure analysis: identity propagation

Common signs of identity loss include:

• service:worker appears where the authenticated user should appear;

• a UID exists but the display identity was reconstructed later from a mutable directory;

• the subject changes after delegation or retry;

• an unauthenticated request falls back to a default account;

• the tenant is accepted from request data without checking membership;

• the decision event and execution receipt identify different subjects.

The root cause is usually outside the model. An HTTP handler authenticates the user, then places only task text on a queue. The worker creates a fresh context using its technical credentials. Another frequent defect stores identity in mutable agent state that a tool response or child process can overwrite.

Repair the boundary in four parts:

• Create the subject only from the verified authentication result.

• Carry it through queues and retries in a distinct execution envelope.

• Validate envelope integrity immediately before each tool invocation.

• Record an immutable snapshot of the subject at decision time.

After the change, rerun the entire identity category and the delegation category. A serialization fix can repair queued calls while accidentally breaking direct or retried calls.

Failure analysis: delegation and scope inheritance

A child must not obtain more authority than the authenticated user, the parent’s effective authority, the explicit delegation, or the tool policy allows.

Check where the effective set is computed and whether every required set participates:

child_effective_scopes = (
    user_scopes
    & parent_effective_scopes
    & delegated_scopes
    & policy_allowed_scopes
)

Then inspect delegation provenance. For this chain:

user-alice → orchestrator → specialist → worker → tool

the decision record should preserve the user and agents in order. A direct user-to-tool call should contain an empty agent delegation chain, not an invented system hop.

Frequent causes include:

• policy is checked only when the parent creates the task;

• the child uses a long-lived orchestrator token;

• the child receives the parent’s complete tool registry;

• the chain exists only inside prompt text;

• a retry creates a new root chain instead of continuing the original request;

• a background task is mislabeled as an interactive request;

• requested scopes are merged rather than intersected.

Place final enforcement immediately before the tool boundary. Early checks by the orchestrator improve usability and reduce wasted work, but they cannot replace enforcement at execution time.

Failure analysis: tenant isolation

Cross-tenant failure is broader than reading another tenant’s file. Policy caches, rate buckets, memory, credentials, audit partitions, and idempotency keys can all leak state.

Review the keys of every shared store:

policy_cache[(tenant_id, policy_version, subject_uid)]
rate_bucket[(tenant_id, subject_uid, execution_class)]
audit_partition[tenant_id]
memory_namespace[(tenant_id, subject_uid, conversation_id)]
tool_credentials[(tenant_id, connector_id)]
idempotency_record[(tenant_id, request_id)]

A cache keyed only by subject_uid is unsafe when user identifiers are unique only inside a tenant. A cache keyed only by tool name is worse: the first tenant’s decision may be reused everywhere.

Derive the tenant through a verified relationship:

authenticated_subject
    → verified_membership
    → tenant_id
    → tenant_policy
    → tenant_credentials
    → tenant_audit_partition

An administrator of tenant-b is an administrator only inside tenant-b. A global admin role without a tenant boundary creates implicit cross-tenant authority.

After an isolation failure, test both directions:

• Change tenant A’s policy and confirm tenant B’s decision is unchanged.

• Create events in both tenants and verify their audit attribution.

• Submit tenant B’s identifier through tenant A’s user request.

• Exhaust tenant A’s rate limit, then perform an allowed request in B.

• Repeat with an administrator from B targeting A.

• Run once with empty caches and once with warm caches.

Failure analysis: incomplete audit evidence

Observability and audit overlap, but they are not equivalent. A metric can reveal rising denial rates. An audit record must reconstruct one specific decision and connect it to actual execution.

A useful decision record contains at least:

{
  "timestamp": "decision time",
  "run_id": "benchmark run identifier",
  "request_id": "end-to-end request identifier",
  "tenant_id": "tenant-a",
  "actor_uid": "user-alice",
  "actor_display": "[e-mail]",
  "agent_name": "mail-worker",
  "execution_class": "background",
  "delegation_chain": ["orchestrator", "mail-worker"],
  "tool": "read_email",
  "decision": "allow",
  "reason_code": "required_scope_present",
  "effective_scopes": ["email.read"],
  "policy_version": "lab-policy-v1",
  "arguments_hash": "hash of normalized arguments"
}

Completeness does not mean copying secrets, message bodies, private files, or authentication headers into the log. Prefer normalized argument hashes, data classifications, protected object identifiers, and references to access-controlled evidence.

Maintain two linked records:

• Decision record: written before forwarding, explaining allow or deny.

• Execution receipt: written after the attempt, explaining whether the tool was reached and how it ended.
  

request
  → policy decision: allow
  → fixture receives request
  → execution receipt: success

If the process crashes after the allow decision but before execution, the evidence must show an allow without a confirmed receipt. If execution occurred but the completion record was lost, reconciliation with the fixture should reveal the gap.

Typical audit defects are:

• only successful operations are logged;

• timestamps are added later by an analytics pipeline;

• display identity is resolved from the current directory during investigation;

• the decision and execution use different request identifiers;

• a retry overwrites the first attempt;

• a runner exception is treated as evidence of safe behavior;

• the log records the model’s selected tool but not the actual invocation;

• raw secrets are retained in the name of completeness.

What to inspect in the remaining categories

Per-user policy enforcement

Apply the most specific applicable rule. If the tenant allows file reading but Carol has an explicit personal denial, Carol must be denied while another authorized user still succeeds. After revocation, the next relevant operation must use the new policy rather than a stale cached decision.

Rate-limit cascade

Five workers must not receive five independent user budgets. Include the tenant, authenticated subject, and execution class in the budget key. Decide explicitly whether policy-denied calls consume a user’s operational allowance, and test that decision consistently.

Failure-mode discipline

Fail-open and fail-closed are declared behaviors, not accidental exception handling. If a sensitive tool requires fail-closed behavior, a timeout in the policy service must prevent execution. A deliberately fail-open low-risk path should be narrowly scoped, visible in audit evidence, and tested as its own policy case.

Scope inheritance

Test allowed child work as well as forbidden escalation. A system that blocks every delegated call is restrictive but unusable. The positive control should demonstrate that a child retains exactly the narrow capability required for its task.

Record each failure as an evidence chain

A useful failure card links the benchmark result, policy decision, audit event, and fixture receipt through one request identifier:

Scenario: identity_propagation.<scenario-id>
Status: FAIL
Expected: actor_uid equals the authenticated synthetic user
Observed: actor_uid equals the mail worker service identity
Side effect: fixture receipt exists for read_email
Suspected boundary: queue serialization
Root-cause status: unconfirmed until trace review
Owner: platform runtime
Retest: identity_propagation and delegation_provenance
Evidence: result JSON, decision record, receipt, policy version, commits

Keep “observed behavior,” “suspected boundary,” and “confirmed root cause” separate. Do not convert a plausible explanation into a fact before the evidence establishes it.

For every failed scenario, answer:

• What assertion failed?

• Did the tool fixture receive the request?

• Which subject and tenant were evaluated?

• Which effective scopes were recorded?

• Which policy version and reason code produced the decision?

• Did retries or delegation change the request context?

• Can the same failure be reproduced after a clean reset?

Verify the completed result

The exercise is complete when you have a reproducible evidence package rather than a screenshot of one score:

results/
├── environment.txt
├── SHA256SUMS
├── vanilla-local.json
├── your-system.json
├── identity.json
├── per-user-policy.json
├── scope-inheritance.json
├── delegation.json
└── cross-tenant.json

agentgovbench.commit
scenario-files.txt
runner/
└── adapter source and sanitized configuration

Use this completion checklist:

• The AgentGovBench revision is pinned and the working-tree state is known.

• Exactly 48 unique scenarios were discovered and executed.

• Eight categories contain six results each.

• The baseline and system runner used the same benchmark revision.

• The scorecard was generated from machine-readable output.

• Pass, fail, declined, and error remain distinct.

• Every failure has a scenario ID, observed behavior, and evidence references.

• Denied requests were checked against fixture receipts.

• Audit exports contain no credentials or private payloads.

• A clean-state rerun produces an interpretable result.

• Critical failures are tied to a release decision and owner.

Finally, rerun the full suite after repairs:

agentgovbench run \
  --runner your_runner_name \
  --out results/your-system-retest.json

sha256sum results/your-system-retest.json \
  >> results/SHA256SUMS

Compare individual scenario IDs and statuses. Do not compare only the totals: one repaired failure can conceal a newly introduced regression elsewhere.

Failure cases in the benchmark process itself

The runner bypasses the production enforcement path

The benchmark calls an internal service API while deployed agents pass through a different proxy or middleware chain. The resulting score describes the test path, not the system. Compare routing, identity headers, policy hooks, and tool credentials in both paths.

State leaks between scenarios

A revoked grant, exhausted budget, cached decision, or tenant record survives teardown. Add unique namespaces, explicit cleanup, and a precondition check at scenario start.

Audit events arrive asynchronously

The runner queries the log immediately and observes no event. Use bounded polling with a fixed timeout and record the observed delay. Never wait indefinitely, and never treat a timeout as a pass.

The test tool performs a real action

A fixture named send_email is accidentally routed to a live connector. Replace execution at the routing or dependency-injection boundary, not through a prompt instruction. The scenario and model must not be able to switch the real connector back on.

Declined hides an inconvenient capability

Declined is legitimate when the topology truly cannot be reproduced and the reason is documented. It is misleading when the system supports multiple tenants but the adapter skips isolation tests because they are difficult to implement.

Only the repaired scenario is rerun

Identity, delegation, auditing, caching, and limits share context. Rerun the affected category first, then all 48 scenarios.

The benchmark data contains real secrets

Credentials copied into tool arguments can leak through results, exception traces, audit exports, or shell history. Use synthetic canary strings and non-routable destinations. Test secret handling structurally without supplying a usable secret.

Limitations

A scorecard is not a security certificate and does not replace a threat model. Important limits remain:

• Deterministic governance calls do not prove that the model resists prompt injection or selects the correct tool.

• The suite does not prove the security of the operating system, container, network, secret store, connector, or tool implementation.

• An operation can be policy-compliant yet semantically harmful.

• Six scenarios per category cannot cover every cache race, retry race, partial outage, or disaster-recovery path.

• A runner can create false confidence when its path differs from production.

• Results from different benchmark revisions are not directly comparable without reviewing scenario and scoring changes.

• Passing technical scenarios does not automatically establish compliance with internal, contractual, industry, or legal requirements.

• A complete audit schema does not prove retention integrity, access control, or tamper resistance.

Combine AgentGovBench with model-level adversarial testing, tool-specific tests, policy review, infrastructure isolation tests, load and race testing, recovery exercises, and manual investigation of selected traces.

After the first scorecard

• Freeze the original result files and checksums.

• Classify each failure by boundary: authentication, queue, policy engine, tool proxy, cache, audit pipeline, or runner.

• Fix allowed authority bypasses first, incomplete evidence second, and excessive denials third.

• Add production-specific variants to a separate regression suite without modifying the benchmark’s original expectations.

• Run the 48 scenarios whenever identity middleware, delegation, policy, caching, auditing, rate limits, or tool routing changes.

• Store scorecard history with the system revision and sanitized policy configuration.

The valuable output is not a perfect-looking percentage. It is a verified map of the boundaries where the user remains the owner of an action, delegated agents cannot expand authority, tenant state stays separated, and every decision can be reconstructed from durable evidence.

← All practical guides · Lab glossary →

Agent Lab Journal
Real experiments. Verifiable conclusions.

---

Original article: https://agentlabjournal.online/en/agent-governance-benchmark.html?utm_source=devto&utm_medium=referral&utm_campaign=agentlabjournal