DEV Community: Mohith

Severance: An IAM Security Agent That Explains and Tests Its Fixes

Mohith — Sun, 18 Oct 2026 15:53:50 +0000

A security finding tells you something is exposed. It rarely tells you which relationship to change without disrupting legitimate access.

In an IAM environment, access can pass through users, groups, roles, and shared policies. A permission that looks harmless in isolation can become dangerous when combined with several other relationships. Removing the first connection you find might block the exposure, but it might also break access that should remain.

We built Severance to investigate that problem for Graph Hacks, entering Track 01: Agents That Act on Connected Data.

Severance is an AI agent that investigates an IAM graph, explains the path behind an exposure, proposes a specific relationship to remove, and tests that change before asking for approval.

The current implementation operates on an offline FalkorDB model. It does not change AWS permissions.

The graph supplies the evidence

We gave the language model responsibility for selecting tools and explaining their results. Reachability, paths, scores, and proposed changes come from deterministic code and FalkorDB queries.

The graph contains five node types:

Node	What it represents
Principal	A user, group, service, or modeled untrusted identity
Role	An IAM role or an explicit permission-carrier adapter
Policy	A managed or inline permission policy
Action	An operation scoped to an assessed target
Resource	A seeded resource or a concrete permission request

Relationships capture group membership, role access, role assumption, policy attachment, permitted actions, and their targets.

One public-input investigation follows this chain:

Compromised user
  → MEMBER_OF → Group
  → HAS_ROLE → Group permission carrier
  → GRANTS → Policy
  → PERMITS → s3:PutObject
  → TARGETS → Assessed object permission

The permission carrier adapts user and group policy attachments to our existing graph schema. It is not an additional AWS role.

That distinction matters: a graph should make the source data easier to reason about without silently inventing cloud infrastructure.

Where FalkorDB does the work

FalkorDB is the primary operational context layer.

Parameterized Cypher queries find the principals connected to a target through identity and permission relationships. For example, the reachability query follows the policy and action spine backward from a resource, then finds the identities that reach the relevant role:

MATCH (res:Resource {id: $rid})<-[:TARGETS]-(act:Action)
      <-[:PERMITS]-(:Policy)<-[:GRANTS]-(role:Role)
MATCH path =
      (p:Principal)-[:MEMBER_OF|HAS_ROLE|CAN_ASSUME*1..6]->(role)
RETURN p.id AS id,
       p.name AS name,
       p.kind AS kind,
       min(length(path)) + 3 AS hops,
       collect(DISTINCT act.name) AS via_actions

We use algo.SPpaths to obtain shortest exposure paths and sampled betweenness centrality to identify load-bearing roles and policies.

The planner then evaluates candidate relationship removals using GRAPH.COPY. Each candidate is tested on a disposable copy, allowing the same reachability queries to measure its effect.

Without FalkorDB, Severance cannot establish the exposure path, rank candidate fixes, simulate their effects, or apply the selected graph change.

Finding a path is only the beginning

Several different changes can block the same path. Choosing between them requires measuring what each change removes.

Our planner considers edges on the shortest untrusted paths and ranks candidate cuts using:

How many untrusted reachers remain.
How many existing shortest paths are closed.
How much other modeled access is lost.
The broker score of the roles or policies involved.
Relationship specificity and a stable final tie-break.

This gives us a declared, reproducible decision policy rather than asking the model to choose whichever fix sounds convincing.

The selected fix is simulated again before approval. Severance shows the supporting path, before-and-after measurements, and any collateral impact.

Approval produces a single-use token bound to the specific plan. The agent cannot apply a different edge deletion using that approval. Non-interactive investigations reject changes by default.

An append-only audit trail records tool calls, proposals, simulations, approval decisions, and applied changes.

Moving beyond our own seeded scenarios

We started with four deterministic scenarios containing known vulnerabilities. Those fixtures let us check exact paths and expected fixes.

We also wanted to test whether Severance could work with data maintained outside our project.

For that, we imported the public account-authorization example from Salesforce’s Cloudsplaining repository. The pinned input contains:

44 users.
5 groups.
78 roles.
84 managed policies, alongside inline policies.

The original file, source revision, checksum, and upstream license are included in the repository.

Each import assesses one concrete action/resource request and explicitly identifies a user or role session assumed to be compromised.

One case examines the OverprivilegedEC2 role. Its attached InsecurePolicy grants S3 write operations on wildcard resources. Severance traces that grant and proposes detaching the policy from the role.

Another case follows a user’s group membership to a policy granting s3:*. That investigation produces a five-edge explanation and proposes removing the selected user’s membership.

These are externally sourced example records, not a verified production-account export. The assessed object ARN is a concrete example request, not evidence that an object exists.

What the benchmark showed

We defined six cases from the source records before running the engine:

Case	Expected behavior
Role-attached S3 write	Find the path and detach the relevant policy
Group-inherited S3 write	Find the path and remove the selected membership
Group-inherited object ACL access	Find the path and remove the selected membership
Rejected approval	Preserve the graph
Allow and deny in one policy	Produce no positive exposure path or mutation
Allow and deny across attached policies	Respect the explicit deny

The final run passed 30 out of 30 deterministic engine evaluations and six out of six live Sonnet MCP-agent investigations.

Approved cases reduced the assessed untrusted reacher count from one to zero, with no collateral among the supported modeled principals. Each approved run deleted exactly one graph edge. Rejection and deny controls preserved the graph.

The repository also has 113 passing deterministic tests.

These results establish that the selected workflows behave as expected on these fixtures. Six agent calls are not a production accuracy estimate.

Earlier runs exposed resource-name lookup errors and a collision between concurrent benchmark graphs. We retained those results, fixed exact-name handling and graph isolation, and ran the final benchmark again.

The benchmark report and per-call evidence are available in the repository.

The most important limitation is authorization scope

An IAM snapshot is not the complete AWS authorization environment.

Our importer supports unconditional identity-policy allows and explicit denies, wildcard matching, default managed-policy versions, inline policies, and group inheritance.

Relevant conditions, unsupported negations, policy variables, missing policy documents, and permissions boundaries produce an unknown result. An unknown selected identity stops the import. Other unknown identities are reported and excluded from positive paths.

The projection does not evaluate role-trust transitions, resource policies, organization controls, session policies, or actual resource existence.

Consequently, a positive path means a supported identity-policy grant in the imported model. It does not prove that AWS would authorize a live request. Likewise, collateral measurements cover the assessed permission, not every permission or workload.

Keeping those boundaries visible was an important part of building the project.

What we learned

Resource scoping is essential. A globally shared s3:GetObject node could accidentally connect a policy to unrelated resources. Action nodes need to preserve their target scope.

Explicit denies also cannot be treated as ordinary graph edges alongside allows. Their effect must be resolved when constructing the supported permission projection.

Finally, graph-copy simulation made the agent’s proposed actions reviewable. Instead of presenting a recommendation alone, Severance can show what changed in the model and what remained reachable.

That is the interaction we wanted to build: an investigation whose evidence, decision policy, approval, and resulting change can all be inspected.

Explore Severance on GitHub, including local setup instructions, the graph model, essential queries, and benchmark evidence.

View the working-project recording, covering both a seeded investigation and a public-snapshot investigation through approval and application.

SentinelLM - A Proxy Middleware for Safer, Observable LLM Systems

Mohith — Sun, 01 Mar 2026 09:05:51 +0000

Large Language Models are powerful.

But most production AI apps today have a hidden problem:

They directly connect their application to the model API —
with no inspection layer in between.

No runtime safety scoring.
No structured logging of prompt/response risks.
No observability into failures.

That’s why I built SentinelLM.

GitHub: https://github.com/mohi-devhub/SentinelLM

What Is SentinelLM?

SentinelLM is an open-source middleware that sits between your application and any LLM backend.

Instead of:

App → OpenAI (or any LLM)

You run:

App → SentinelLM → LLM Provider

It acts as a proxy layer that intercepts every request and every response before it reaches the user.

Think of it as a safety + quality firewall for LLMs.

Why This Layer Matters

LLMs process untrusted input.

That includes:

User prompts
Retrieved documents (RAG pipelines)
Tool outputs
External API responses

Without a control layer, your system is blindly trusting model outputs in real time.

SentinelLM introduces:

Request evaluation
Injection pattern detection
Response scoring
Hallucination checks
Toxicity and policy violation flags
Structured logging
Real-time observability

Not as a replacement for model safety —
but as an additional enforcement layer.

Key Features

1. Interception & Evaluation Pipeline

Every request passes through a chain of evaluators before reaching the model.

Every response is analyzed before being returned to the user.

2. Pluggable Architecture

Evaluators can be extended or modified depending on your application needs.

Want stricter hallucination detection?
Add a custom evaluator.

3. Drop-In Integration

No major app changes required.

Just point your LLM client to:

http://localhost:8000/v1

SentinelLM mirrors standard LLM API formats, making integration simple.

4. Logging & Observability

All interactions are logged for:

Debugging
Risk auditing
Analytics
Monitoring

This makes your AI system observable — not a black box.

Design Philosophy

SentinelLM is built around one idea:

LLMs should be treated as powerful but untrusted components.

Just like we use:

API gateways
Reverse proxies
Web application firewalls

AI systems need runtime inspection layers.

Not because models are bad.

But because production systems require accountability.

Who Is This For?

SentinelLM is useful if you’re building:

AI assistants
AI agents
Copilots
RAG-based systems
Enterprise AI workflows
Internal AI tooling

Especially if:

You need audit trails
You handle sensitive data
You care about injection risks
You want measurable safety metrics

The Bigger Picture

As AI systems move from demos to production infrastructure, we need:

Observability
Defense-in-depth
Runtime evaluation
Transparent logging

SentinelLM is a step toward that future.

It’s open-source, extensible, and built for real-world AI engineering.

If you’re working on production AI systems, I’d love feedback.

What safety or quality checks do you think every LLM system should have by default?