DEV Community: Sangyeon Park

Put a Policy Gateway Between Your Coding Agent and the LLM

Sangyeon Park — Thu, 27 Aug 2026 18:03:37 +0000

Your coding agent talks to a model provider over HTTPS. That connection is a straight line: the agent asks, the provider answers, the answer lands in your editor. Nothing in the middle looks at what came back.

For most of what an agent produces, that's fine. For the rest of it — the query built by string concatenation, the API key the model helpfully echoed back into a code sample, the eval() on user input — you find out later, in review, or in a scanner run, or never.

This is a walkthrough of putting a policy layer in that line: a local proxy your agent points at instead of the provider, which inspects the response stream and decides allow, redact, or block before the text reaches you.

I'll use Cencurity Engine because it's the one I build, it's Apache-2.0, and it runs entirely on your machine. The pattern generalises — if you're building your own gateway, the steps below are still the shape of the problem.

What you need first

Go installed (the engine is a Go binary you run from source)
An API key for whatever provider your agent already uses
An agent or IDE that lets you override the API base URL

That last one is the real prerequisite. If your tool hardcodes the provider endpoint, none of this applies to it. Most don't: Roo Code, Continue, Claude Code and Gemini CLI all expose a base URL, and anything reading OPENAI_API_BASE will work too.

Step 1: Start the gateway

Clone the repo, open a terminal in it, and run:

go run ./cmd/cast serve \
  --listen :8080 \
  --upstream https://api.openai.com \
  --policy ./cast.rules.example.json

Three flags, and each one is doing something you should understand before moving on:

--listen is where the gateway accepts traffic. Local only.
--upstream is your real provider base URL. Swap it for https://api.anthropic.com, https://api.deepseek.com, https://api.x.ai — whatever you actually use.
--policy is the rule file. cast.rules.example.json ships in the repo and is a working starter set, not a placeholder.

Note what is not in that command: your API key. The gateway forwards whatever Authorization header your agent sends. The key stays where it already lives, which means adding this layer doesn't create a second place a credential can leak from.

Step 2: Check it before you trust it

go run ./cmd/cast doctor

doctor loads your config and reports the active rule count. Run it now, and run it again every time you edit the policy file. A JSON typo that silently drops half your rules is the exact failure mode this catches — a gateway with zero loaded rules passes everything and looks perfectly healthy from the outside.

The gateway also exposes:

http://localhost:8080/healthz — liveness
http://localhost:8080/metrics — Prometheus-format plaintext

Curl /healthz before you repoint anything. If it doesn't answer, your agent is about to fail every request and you'll waste twenty minutes blaming the agent.

Step 3: Repoint your agent

Change your agent's API base URL from the provider to http://localhost:8080. The paths are passthrough, so the endpoint shape you were already using keeps working:

OpenAI-compatible: http://localhost:8080/v1/chat/completions
Anthropic Messages: http://localhost:8080/v1/messages
Gemini streaming: http://localhost:8080/v1beta/models/{model}:streamGenerateContent

Then use your agent normally. If you skip this step nothing breaks — your traffic simply keeps going straight to the provider and the gateway sits there doing nothing. That's a surprisingly easy state to end up in and believe you're protected, so verify with the tests in the next step rather than assuming.

Step 4: Prove all three actions actually fire

Testing a security control by hoping it never triggers is not testing it. Drive each outcome deliberately. Use curl -N so the SSE stream stays open:

allow — ask for something ordinary, like a function that sums a list. The stream should flow normally and the structured stdout log should carry "action":"allow".

redact — ask the model to print a string shaped like a secret. The stream stays open, the matching token comes through as [REDACTED], and the log shows "action":"redact".

block — ask for Python that uses eval on a string. The stream terminates right after the matching chunk. Downstream receives : blocked by cencurity followed by data: [DONE], and the log shows "action":"block".

That last one is the detail worth internalising. A block is not a clean HTTP error — the connection is already open and streaming when the decision happens. Your agent sees a stream that ends early. If your tooling treats an early [DONE] as a successful empty completion, you'll get silent truncation rather than a visible refusal, and you'll want to know that before you turn enforcement on for a team.

Step 5: Write a rule of your own

The policy file is JSON. Each rule takes six fields:

{
  "id": "cast.custom.internal-hostname",
  "category": "secrets",
  "severity": "medium",
  "action": "redact",
  "pattern": "(?i)\\b[a-z0-9-]+\\.internal\\.example\\.com\\b",
  "enabled": true
}

pattern is a Go regex, and that constraint matters more than it looks. Go's regexp package is RE2: linear-time guaranteed, and therefore no lookahead, no lookbehind, no backreferences. If you're porting patterns from a PCRE-based tool, the ones leaning on (?=...) will not compile. This is a good trade for something sitting in a hot path — RE2 can't catastrophically backtrack and stall your editor — but it does mean some rules have to be rewritten rather than pasted.

Save the file. Rules reload automatically on the next access after the reload interval, which defaults to 3 seconds (CENCURITY_POLICY_RELOAD_MS). No restart. Run doctor again to confirm the count went up.

Start with "action": "redact" or a low severity while you're calibrating. A rule that blocks is a rule that can interrupt someone mid-task, and a pattern-based rule on generated code will produce false positives — that's the nature of the technique, not a bug you can tune away entirely.

Step 6: Compare against the real provider before you roll out

The question that decides whether anyone keeps this turned on is: does routing through the gateway change what I get back?

go run ./cmd/cast shadowtest \
  --upstream https://api.x.ai \
  --model grok-4-0709 \
  --api-key-file ./upstream-api-key.txt \
  --concurrency 1 \
  --iterations 5 \
  --timeout 90s

shadowtest runs the same prompts direct and through the proxy against a real upstream and compares the streams, across four default scenarios: allow-short, allow-long, redact, block. Add --provider anthropic or --provider gemini if auto-detection doesn't pick your upstream correctly.

Run allow-long in particular. Short responses hide streaming bugs; long ones surface them. An inline control layer that subtly mangles a 2,000-token response is worse than no control layer, because you'll spend a week blaming the model.

What you end up with

When a rule fires you get a structured finding rather than a line number:

Field	Example
`language`	`python`
`framework`	`fastapi`
`rule_id`	`cast.fastapi.auth.jwt-verify-disabled`
`severity`	`high`
`confidence`	`high`
`action`	`block`
`evidence`	`eval(user_input)`

And the honest framing of what this is: heuristic, stream-time guardrails. Not a semantic analyser. No type information, no data-flow graph, no way to know whether the value being concatenated is genuinely attacker-controlled. The engine's own README says as much — these are guardrails, not a full semantic SAST engine.

Which is the point. It runs at a moment nothing else covers: while the code is being written, before it's in your file. Your SAST pipeline still runs afterward, and still catches things this can't. Keep both.

I build Cencurity, an open-source policy-driven security gateway for LLM coding agents (Apache-2.0). Writing about LLM security, guardrails, and the gap between generated code and reviewed code.

Your Security Scanner Has a Blind Spot: Streaming

Sangyeon Park — Thu, 27 Aug 2026 17:37:00 +0000

I spent an afternoon convinced my detector was broken.

I was building Cencurity, a local gateway that sits between an IDE and an LLM provider and inspects generated code before it lands in the editor. The rule was simple. If the model produces a SQL query built by concatenating user input, block it. I had a regex. I had a test string. The test passed.

Then I pointed a real coding agent at it, asked for something that should have tripped the rule, and watched the unsafe code arrive in my editor untouched.

The rule was fine. My mental model of the input was wrong.

The thing nobody tells you about inspecting LLM output

Every security tool I had used before this — linters, SAST scanners, pre-commit hooks — operates on a file. A complete, finished, sitting-still-on-disk file. You read it, you parse it, you walk it, you make a decision.

LLM output is not a file. It is a stream.

When an agent asks a provider for code, the response comes back over Server-Sent Events as a sequence of small chunks. Each chunk carries a fragment of text. The fragments are not split on anything meaningful — not on lines, not on tokens you would recognise, certainly not on syntactic boundaries. They are split on whatever the tokeniser and the transport happened to do.

So a query that looks like this when it is finished:

cursor.execute("SELECT * FROM users WHERE id = " + request.args['id'])

might arrive like this:

data: {"choices":[{"delta":{"content":"cursor.execute(\"SELECT * FROM us"}}]}
data: {"choices":[{"delta":{"content":"ers WHERE id = \" + reque"}}]}
data: {"choices":[{"delta":{"content":"st.args['id'])"}}]}

My regex ran against each chunk as it arrived. No single chunk contained the pattern. Every chunk passed. The dangerous line assembled itself in the editor, one safe-looking fragment at a time.

This is the part I want other people building in this space to have for free, because it cost me a day: a detector that is correct on a complete string can be silently useless on a stream, and it will not fail loudly. It will just never fire.

Why you can't simply buffer everything

The obvious fix is to stop being clever. Collect the whole response, wait for the stream to end, then run the detector once against the finished text.

That works, and it is what I would do if the tool ran in CI. It does not work in an editor.

The entire value of streaming is that the developer sees output as it appears. If you buffer the full response before releasing anything, you have converted a responsive agent into a tool that stares blankly for eight seconds and then dumps a wall of text. Users notice this immediately and they turn your tool off. A security control that gets disabled protects nothing.

So the constraint is: inspect a stream, in real time, while still emitting it in real time, and catch patterns that no individual chunk contains.

What actually works: a sliding window with delayed release

The approach that held up is to stop treating chunks as the unit of analysis. Chunks are a transport artefact. They should not be visible to the detector at all.

The gateway keeps a rolling buffer of recently seen text. Each arriving chunk is appended, the detector runs against the tail of the accumulated buffer rather than the chunk alone, and then — this is the part that matters — the gateway releases only the portion of the buffer that can no longer participate in a match.

Concretely: if your longest rule can match across N characters, you always hold back the last N characters. Everything before that is settled. Nothing arriving later can change whether it matched, so it is safe to forward.

chunk arrives
  |
append to buffer
  |
run detectors against buffer tail
  |
match?  --yes-->  allow / redact / block
  |
  no
  |
release buffer[0 : len-N]  -->  forward downstream
hold buffer[len-N : ]

The developer sees output flowing continuously. The delay is bounded by N, not by the length of the response. And the detector sees continuous text, which is what it was written for.

Three details that were not obvious to me at the start:

The window has to be sized off your rules, not guessed. N is a property of your ruleset. Add a rule that can span more characters and your hold-back has to grow with it, or that rule quietly becomes a chunk-boundary lottery.

You have to decide what happens to text you already released. If a match completes at character 900 and you forwarded characters 0 to 850 twenty milliseconds ago, those 850 characters are already in the editor. Blocking now is a partial block. In Cencurity the enforcement actions are allow, redact, and block, and how each behaves mid-stream is a design decision you have to make explicitly rather than discover in production.

Not every chunk is text. SSE frames carry structure, and a naive "append the whole chunk" will happily let JSON scaffolding contaminate the buffer and either create false matches or break real ones. Parsing the frame before appending is not optional.

The honest limits of this

I want to be precise about what this is, because the AI security space is currently full of claims that do not survive contact with a real threat model.

This is a heuristic, pattern-based, stream-time guardrail. It is not a semantic analyser. It has no type information, no data-flow graph, no notion of whether the variable being concatenated is actually attacker-controlled. Cencurity's own README says this in as many words: these are stream-time guardrails, not a full semantic SAST engine.

That means it will produce false positives on code that is fine, and it will miss things a real analyser would catch. What it does that a real analyser cannot is act during authoring — before the code is in your file, before it is in your commit, before a scanner three steps downstream flags it and you have to reconstruct why you accepted it. When someone asked me how this differs from a code review tool, the distinction I landed on was: this is not static analysis of a repository after the fact, it is real-time control of the traffic between the agent and the model.

Both layers are worth having. Neither replaces the other. Anyone telling you a regex over a token stream is a substitute for a SAST pipeline is selling something.

Why I think this generalises

Cencurity is a specific tool with specific choices — it runs as a local gateway on 127.0.0.1:38180, your API key never leaves your IDE, and only policy violations get logged rather than the whole conversation. Those are my choices and you might make different ones.

But the streaming problem is not specific to me. It applies to anything that inspects model output in flight: prompt-injection filters reading tool-call arguments, PII redaction on chat responses, content policy enforcement on generated text. All of them are pattern-matching over a stream. All of them will silently under-fire if they treat the transport's chunk boundaries as real boundaries.

The failure mode is the dangerous part. It does not throw. It does not log an error. Your dashboard shows zero violations and you conclude the traffic is clean.

If you are building anything in this shape, the test to write first is not "does my rule match the bad string." It is "does my rule still match when I split the bad string at every possible offset and feed it through in pieces."

Mine didn't.

Sangyeon Park builds Cencurity, an open-source policy-driven security gateway for LLM coding agents (Apache-2.0). Writing about LLM security, guardrails, and the gap between generated code and reviewed code.