In Part 3, we built a Flask proxy using static regex rules to block prompt injection and redact sensitive database hostnames.
While regex filters are essential for catching obvious keywords, they break down at enterprise scale: - Attackers easily bypass string matching using synonyms, foreign languages, or novel encodings. - Hardcoding rules inside application code violates the separation of duties between security teams and software developers. - Auditing and compliance teams require explainable, version-controlled decision logs, not silent regex drops.
In this concluding installment of our Hands-On AI Security series, we implement Policy-as-Code using Open Policy Agent (OPA) and the Rego query language.
The Policy-as-Code Workflow
Instead of writing security rules in Python, the application queries OPA on port 8181:
sequenceDiagram
autonumber
actor User as Client
participant GW as Flask Gateway
participant LLM as Classifier Model (llama3.2)
participant OPA as Open Policy Agent (:8181)
participant Rules as Policy File (gateway.rego)
User->>GW: Submit Prompt GW->>LLM: Fast Context Classification (Extract Intent) LLM-->>GW: Context: {"role": "guest", "intent": "admin_probe"} GW->>OPA: POST /v1/data/gateway/allow (Payload + Context) OPA->>Rules: Evaluate Rego Policy Rules-->>OPA: Decision: {"allow": false, "reason": "UNAUTHORIZED_ADMIN_PROBE"} OPA-->>GW: Policy Decision Response GW-->>User: 403 Policy Denied: UNAUTHORIZED_ADMIN_PROBE (Audit ID: 9942) ```
---
## Writing the Rego Policy: `gateway.rego`
Here is `policies/gateway.rego` from the lab repository:
```rego
package gateway
default allow = false default reason = "NO_MATCHING_ALLOW_POLICY"
# Allow benign customer banking inquiries
allow {
input.context.intent == "banking_inquiry"
not input.contains_prohibited_terms
}
# Block unauthorized privilege claims
deny[msg] {
input.context.intent == "admin_probe"
input.user.role != "system_admin"
msg := "UNAUTHORIZED_ADMIN_PROBE: Non-admin users cannot trigger diagnostic intent."
}
# Block sensitive data requests
deny[msg] {
input.context.intent == "credential_request"
msg := "PROHIBITED_CREDENTIAL_REQUEST: Model cannot be queried for internal secrets."
}
# Final decision rule
allow {
count(deny) == 0
}
Testing OPA Policy Directly via REST API
You can test OPA policy decisions directly via its HTTP Data API (port 8181) without going through Flask:
curl -s -X POST http://localhost:8181/v1/data/gateway \
-H "Content-Type: application/json" \
-d '{
"input": {
"context": {"intent": "admin_probe"},
"user": {"role": "guest"}
}
}'
Output:
json { "result": { "allow": false, "deny": [ "UNAUTHORIZED_ADMIN_PROBE: Non-admin users cannot trigger diagnostic intent." ] } }
Why Policy-as-Code Wins in Production
- Separation of Concerns: Security teams can update and push new Rego policies to the OPA container in Git without requiring developers to redeploy or restart the Flask web service.
-
Explainability: When a user's prompt is blocked, OPA returns the exact reason string (
UNAUTHORIZED_ADMIN_PROBE). This provides clear auditability in your SIEM without leaking internal logic to the attacker. - Multi-Service Portability: The same OPA server evaluating your LLM chatbot can simultaneously evaluate Kubernetes admission controllers, API gateways, and CI/CD pipelines.
Series Conclusion & Open-Source Materials
Across this 4-part series, we have demonstrated that securing generative AI applications requires: 1. Never relying on system prompts as trust boundaries. 2. Placing an external proxy between the user and the model. 3. Decoupling rules from code using Policy-as-Code (OPA).
Download the complete Docker Compose sandbox, lesson worksheets, and slide decks from the Securing-AI GitHub Repository and watch the full NCyTE Center Fellowship Webinar.
Top comments (0)