In my last project, I built a LLM routing system that dynamically selects and switches between model providers based on task complexity, cost budgets, latency requirements, and quality thresholds.GitHub — mgbec/LLM-Router-deployed-to-AWS · GitHub . I was looking into AI compliance standards and added some controls that would meet ISO 42001.
This time around, I used the same base project but added SOC2 controls. With the rapidly evolving field of technology, our regulatory requirements have been changing as quickly. AI workflows transform much faster than lawmakers can create documentation and standards. Federal, state, and global differences make the issues even more complex.
Policy as Code has been around for a while and the nature of AI workflows and their probabilistic nature has made deterministic controls like this even more attractive. By translating human-readable laws, regulations, and corporate policies into machine-readable code, we can try to automatically check, enforce, and log compliance across their entire infrastructure.
Policy as Code vs Traditional Methods
The Old Way (manual + code-based)
-Rules live as if statements buried in application code
-Auditors have to read Python/Java to understand what’s enforced
-A developer can change a rule by editing one line, no review
-Proving “this was enforced at time X” requires digging through git blame
-One team’s code might enforce budget limits differently than another’s
The Policy-as-Code Way
-Rules are declarative, they describe WHAT, not HOW
-Rules are testable, OPA test proves correctness without running the full system
-Rules are auditable, an auditor reads deny if estimated_cost > max_cost and understands instantly. The evaluation engine records results into a log that can be used as evidence
-Rules are versioned, Git history shows exactly when each rule was added/changed
-Rules are separate from code, a policy change doesn’t require a code deploy
-Rules are enforceable, Pass/Fail Gates in CI fail if policy is violated, no human override possible
Adding Policy as Code
I added both Open Policy Agent (OPA)/Conftest and AWS AgentCore Policies into my design. In this project, OPA and AgentCore Policies work together, but at different layers, to meet more compliance and security needs.
Infrastructure Tier
Conftest is a utility built on top of the Open Policy Agent (OPA) to help you write and run tests against structured configuration data using OPA’s Rego policy language. In this case it does Terraform plan validation.
What: “Can we even deploy this safely?”
When: Before anything deploys (CI/CD pipeline, pre-apply)
Types of events it can catch:
-A developer removes encryption from a Kinesis stream
-Someone sets a CloudWatch log group to indefinite retention (compliance violation)
-A DynamoDB table ships without point-in-time recovery
-An S3 bucket accidentally allows public access
Compliance value: Auditors can look at the policy file and say “this rule is enforced automatically on every deployment — no human can bypass it.” That satisfies SOC 2 CC6.1 (logical access controls) and ISO 42001 requirements for documented, repeatable processes.
Business Logic Tier
Open Policy Agent (OPA) Open Policy Agent — Homepage | Open Policy Agent is a general purpose policy engine that can be used for many use cases. Here we are using in for routing decisions.
What: “Should this request be routed this way?”
When: Every request, evaluated in milliseconds at the routing layer
What it enforces:
- Budget limits: “This user’s policy caps requests at $0.05 — deny the Opus route”
-Data consent: “No external providers unless the caller passed data_consent: all-providers”
-Kill switch: “System is disabled by an operator — deny everything”
-Rate limits: “Budget-conscious tier gets 100 requests/hour”
-Access tiers: “Budget users cannot access complex-tier models”
Compliance value: These are auditable, testable rules — not scattered if statements in application code. When an auditor asks “how do you enforce cost controls?” you show them routing.rego. The rule IS the documentation. You can unit-test it (opa test), version it in Git, and prove it was active at any point in time.
Access Control Tier
AgentCore Cedar Policies (Tool authorization)
What: “Is this agent allowed to call this tool?”
When: Every tool call through the Gateway, evaluated by AWS (not your code)
What it enforces:
- Only the router agent can call classification tools
-Model invocation is only permitted for provider == “bedrock”
-External providers are forbidden unless explicit consent is provided
-Future: specific users/roles can access specific tools
Compliance value: Even if someone compromises the application code or bypasses OPA, Cedar still blocks unauthorized tool calls at the AWS service level. It’s a second, independent decision point that your code cannot override. This is coverage for ISO 42001 A.9.5 (human oversight) and SOC 2 CC6.1/CC6.2 (access controls).
Let’s Build
I was building on top of my previous project, but my Linux laptop decided to go down for the count, poor old thing. I think I can resuscitate it at some point, but I needed to switch over to a Windows machine for this project. This introduced a large number of learning opportunities and I ended up needing to create some PowerShell versions of existing scripts, and uncovered strange little quirks with line endings and other formatting interpretations. If you look in the current repo, there is a file that will handle some of the cross platform issues I ran into- https://github.com/mgbec/LLM-Router---Deployed-to-AWS-and-SOC2-compliant/blob/main/.gitattributes.
My end result is here: https://github.com/mgbec/LLM-Router---Deployed-to-AWS-and-SOC2-compliant
Changes include:
- added OPA OpenPolicyAgent/Rego for routing and Terraform validation
OPA implemented as a sidecar in the container, port 8181. More details at https://github.com/mgbec/LLM-Router---Deployed-to-AWS-and-SOC2-compliant/blob/main/architecture/opa-policies.md
We have already used AppConfig in this project to control behavior, but OPA is complementary.
- Added Cedar Policy Engine and attached to the existing AgentCore Gateway. Our Cedar Policies control prompt classification, quality feedback recording, and model invocation.
- Added SOC Compliance: this is now the existing state:
Compliance Status
The OPA and Cedar additions move several ISO 42001 controls from “partially covered” to “fully covered” because the enforcement mechanism is now provably correct, not just “we wrote code that should do this.” Our current framework compliance for this project is below: https://github.com/mgbec/LLM-Router---Deployed-to-AWS-and-SOC2-compliant/blob/main/architecture/compliance-comparison.md
Testing
In addition to the tests related to the original functions (testing routing, async, transparency, oversight, guardrails, and error handling), there are additional tests for the Cedar and OPA policies.
test-policies.ps1/test-policies.sh (Terraform validation and routing policies)
test-cedar-policies.ps1/ test-cedar-policies.sh (Cedar/AgentCore Policies)
Together — Defense in Depth
Conftest alone only catches deployment-time issues, but it can’t help at runtime. OPA alone runs in your code, but a bug or bypass in the application could skip it. Cedar alone only knows about tool calls, it won’t understand business rules like budgets.
When we add our policies together they create a system where:
-Bad infrastructure can’t be deployed (Conftest)
-Bad routing decisions can’t be made (OPA)
-Unauthorized tool access can’t happen (Cedar)
Each layer is independently verifiable. An auditor can test each one in isolation, see its logs, and confirm it was active.
Bedrock Guardrails
Bedrock Guardrails add another aspect to our controls. The Guardrails evaluate and filter content or act as a grounding check. They can act on both user inputs (prompts) and model outputs (responses). They operate at a different conceptual layer on a different type of data.
Comparison OPA vs AgentCore Policies vs Guardrails
Observability Details
To get full observability, we need to toggle on tracing in both AgentCore Runtime — Runtime and Identity and also toggle it on in your Gateway. At this point in time, we seem to only be able to do this through the console.
Once that is accomplished, there is a script that can demo the OPA, Cedar, Guardrails stack — https://github.com/mgbec/LLM-Router---Deployed-to-AWS-and-SOC2-compliant/blob/main/scripts/demo-policy-layers.ps1. The bash version is available as well.
Before running the demo, you will probably need to grab a new token using the get-token.ps1. The demo will run through a few actions and then give you the locations where you can look at the logs.
There is plenty of data to dig into and best of all, nice traceable, auditable data.
The Auditor Guide at https://github.com/mgbec/LLM-Router---Deployed-to-AWS-and-SOC2-compliant/blob/main/AUDITOR_GUIDE.md provides the audit team with instructions for assuming the Read Only Auditor role and how to run commands that will give them the information they will be looking for.
Policy as Code (PaC) replaces manual compliance checks with automated, executable logic that evaluates every system change. When a developer submits a request, a policy engine instantly permits, denies, or flags it based on hardcoded rules. This shifts compliance from static documents to a live, continuous control surface embedded directly within the software pipeline. By ensuring repeatable, scalable enforcement in seconds, PaC eliminates late-stage security surprises, strengthens cybersecurity posture, and simplifies regulatory compliance. I’m sure there are many changes to come in both regulations and technology, but I suspect the easier enforcement and auditing that come with Policy as Code will be even more valuable. Thanks for reading!
Top comments (0)