DEV Community: Sangyeon Park

I Shipped a Security Product I Cannot Fully Audit

Sangyeon Park — Fri, 21 Aug 2026 19:32:57 +0000

That is the uncomfortable part. It is also the argument.

I write commercial copy for a living. I have been doing it since 2020, mostly in Korean, mostly for clients who will never know my name. I cannot write a for-loop from memory.

In 2025 I built and shipped Cencurity, a security gateway that proxies traffic between AI coding agents and LLM providers, masking sensitive data on the way out and inspecting generated code on the way back before it reaches the developer's editor. It runs on 127.0.0.1:38180. It is Apache-2.0. It launched on Product Hunt and it installs from the VS Code Marketplace.

Every line of it was written with AI assistance. I directed the architecture. I did not type the implementation, and I cannot fully audit the result.

I want to sit inside that sentence rather than hurry past it, because most writing about AI-assisted building hurries past it, and the hurrying is what makes the genre worthless.

The discourse is calibrated to the wrong artifact

Almost every argument about non-engineers building with AI is conducted over throwaway software. A habit tracker. A landing page. An internal dashboard six people use. In that context the debate resolves easily: the blast radius is small, so who cares.

A security tool is the adversarial case. If Cencurity's masking rules have a bypass, the failure mode is not a broken button. It is a developer who believed their prompts were sanitized and was wrong — someone materially less safe than if they had installed nothing, because they made decisions on a false assumption.

That asymmetry is real and I do not think you can wave it away. The honest version of "AI lets anyone build" has to include "including things they cannot verify," and then say what follows from that.

What actually transferred

Here is what I got wrong before I started: I assumed the scarce input was implementation, and that AI would supply it.

The scarce input was the framing.

The obvious thing to build in this space is a scanner for AI-generated code — that is where the alarming statistics are, and it is what most people mean by AI code security. What I kept noticing instead was that the code was one of four exposures, and not the worst one. Once an agent sits in your loop, you are also sending it repository contents, environment values, terminal output, occasionally a credential. And it acts on what comes back. The failure modes that follow are prompt injection, unsafe tool calls, data leakage on the outbound path, and consequential actions that leave no record.

Traditional application security covers none of those four well. Scanning generated code covers one.

Arriving at that framing was not an engineering achievement. It came from being an unusual user rather than a skilled builder — someone running these tools semi-autonomously and paying attention to the traffic instead of the output.

What else transferred, unexpectedly: the ability to argue with a model. Six years of taking client feedback on copy turns out to be direct training for evaluating a proposed architecture, saying this is wrong and here is why, and holding a position across twenty exchanges without drifting. The Stanford study by Neil Perry and colleagues found that participants who distrusted their AI assistant and rewrote their prompts produced fewer vulnerabilities. Productive skepticism was the differentiator, and skepticism is not a programming skill.

What did not transfer, specifically

I cannot prove the redaction rules have no bypass. I can test the cases I thought of, and I can read the policy definitions, and neither of those is a proof. Someone with real offensive security experience would attack it in ways I have not imagined — that is the entire point of that discipline, and I do not have it. This matters more than it would for most tools, because the rules run on a live token stream rather than on a finished file, and streaming is where edge cases live.

I cannot tell you what the gateway costs at p99 under load. I know it adds a local hop. I do not have the instrumentation instincts to measure that properly, so I have not published a number, and I would rather publish nothing than publish a figure I cannot defend.

I tuned the policy engine's false positives against my own repositories. Sample size of one, one coding style. A DevOps repository full of legitimate subprocess and shell calls will fire alerts it should not. I know that structurally, not from a bug report — zero issues have been filed, and the open-source engine release drew 3 upvotes, which together say more about the size of the install base than about the quality of the tuning.

Those are not modest disclaimers. They are the actual boundary, and two of the three are the kind of thing a buyer should weigh before installing anything I make.

Why open source stopped being a preference

I released Cencurity under Apache-2.0, and I want to be precise about why, because "we believe in open source" is what people say when they have not thought about it.

I open-sourced it because I cannot be the guarantor.

A conventional security vendor's trust story runs through the team: credentialed engineers, audits, a track record. I have none of that and cannot manufacture it. The only trust model available to me is one where verification does not depend on trusting me at all — where anyone who doubts the tool can read it, and where my inability to audit my own work is compensated by everyone else's ability to audit it.

For a builder in my position, open source is not generosity. It is the structural substitute for credentials I do not have. A closed-source Cencurity would have been an unreasonable thing to ask anyone to install.

It also disciplines the product. Publishing the policy definitions means the enforcement logic is legible, and legible rules are the thing that separates a deterministic guardrail from a model you are hoping catches the same problem twice.

The objection I take seriously

The strongest case against people like me is not that our code is bad. It is that we degrade the signal.

If shipping a security tool no longer implies security expertise, buyers lose a heuristic they relied on, and evaluating tools gets more expensive for everyone. My existence imposes a cost on people who had a working shortcut.

I think that is basically correct, and I do not think it settles the question. The heuristic was already unreliable — funded teams of credentialed engineers ship vulnerable security products regularly. What is changing is that the shortcut's failure is becoming visible rather than remaining comfortable. The response that scales is verifiability: readable source, stated limitations, published failure modes. Not gatekeeping the input side.

Which is why the useful question to ask about a tool like mine is never "could the author have written this by hand."

It is: can you check?

For Cencurity, the answer is yes. That is the whole of what I am offering.

Building Cencurity: What Two Reversals Taught Me About Shipping a Security Tool

Sangyeon Park — Fri, 21 Aug 2026 19:25:58 +0000

Problem definition, an architecture I got wrong the first time, and an honest read of a modest launch.

The problem I was actually trying to solve

When I started building agent tooling, the assumption I began with — and the one most writing on this subject still begins with — was that the risk in AI-assisted development is the code the model writes.

That is one of four problems, and not the hardest one.

Once an agent is in your loop, traffic runs in both directions. You are sending it repository contents, environment values, terminal output pasted in during debugging, occasionally a config file with a live credential in it. And it acts on what comes back: writing files, running commands, invoking tools. Traditional application security was not built for any of that. It covers neither prompt injection, nor unsafe tool calls, nor data leakage on the outbound path, nor actions taken with no record that they happened.

That last one is the quiet problem. An agent can do something consequential and leave nothing behind that tells you it did.

So the product question was narrower than "is AI-generated code safe." It was: can you put a checkpoint between the agent and the model, and enforce policy on both directions of that traffic?

The constraint that decided the architecture

I am not a software engineer. I am a commercial writer who has been building with AI assistance since 2024. That constraint eliminated entire categories of solution before I evaluated a single one.

A native plugin per IDE meant four codebases, four release cycles, four extension APIs. Not viable for one person.

A hosted service meant holding other people's prompts and API keys on infrastructure I would then have to secure — an absurd position for a security product.

What was left was a local gateway. It is the least glamorous option and, for a solo builder, the only defensible one. Every agent I cared about already lets you override the provider base URL, and that configuration surface is stable.

IDE / Agent  →  Cencurity Gateway (127.0.0.1:38180)  →  LLM Provider

Requests pass outbound through a policy engine that detects and masks sensitive data. Responses pass inbound through a scanner that inspects generated code before the editor renders it. API keys stay in the IDE — the gateway forwards credentials and never stores them, which closes the first objection in any security review and rules out anything requiring persistent provider-side state.

Reversal one: from audit everything to log almost nothing

The version I launched recorded all traffic as audit logs.

That is the obvious design if you are thinking about compliance, and it is the wrong one if you are thinking about whether anyone will keep the tool installed. Full logging gives you much better policy tuning and a real audit trail. It also means a security tool is now recording every prompt a developer writes — which is a morale problem, and worse, a new exfiltration target. A security product that creates a fresh breach surface has negative value.

Current builds log policy violations and security events only. Normal traffic passes through unrecorded.

I gave up the better telemetry, which is a real cost and the one I am least happy about. I still think it is right. The quiet version is the one that survives.

Reversal two: from Docker to one command

The launch build ran as a self-hosted Docker deployment. Self-hosting was the whole pitch — nothing leaves your machine, you control the deployment.

Getting it running meant: pull and run a Docker image manually, log in with a bootstrap key, then enter the LLM base URL by hand.

I knew the setup barrier was high when I shipped it. I told myself the audience was security-minded developers who would not be put off by a container. Then people told me the same thing directly, and the argument I had been making to myself stopped working.

Developers evaluating a tool do not stand up infrastructure before they have seen it do anything. They install it, watch it work once, and decide.

Version 2 removed all three barriers:

The Docker setup and bootstrap login flow are gone.
Manual LLM configuration is gone. You open the VS Code command palette, select your provider, and the proxy configures itself.
The dashboard opens inside VS Code. No separate browser, no external connection.

No terminal. No config files.

The security model did not change at all. Everything still runs locally, nothing is sent to a third party. What changed was the distance between installing and seeing it work, and that turned out to be the thing that mattered.

In March 2026 I released the engine itself as open source under the name Cencurity Engine — the same inline enforcement, unbundled from the extension. It works on the stream: blocking dangerous constructs like eval and subprocess, redacting secrets such as API keys and credentials, while the model is still producing tokens.

Reading the launch honestly

The first Product Hunt launch landed at 83 upvotes and #19 for the day. The Cencurity Engine open-source release seven months later got 3. On GitHub the repository has 12 stars, 2 forks, and 54 commits on main. Zero issues have been filed.

Those are modest numbers and the second one is worse than the first, which is the opposite of the shape a case study is supposed to have. I am stating them plainly because case studies that round every figure up are the reason nobody believes case studies.

The honest read of that drop: fixing the install path did not fix distribution. Those are two different problems and solving the first one told me nothing about the second. A better product with no audience is still a product with no audience.

What the first launch did produce was signal, and it was concentrated in one question asked two different ways: how is this different from having an LLM check your code, or from a code review tool?

Answering that twice forced the positioning I should have started with. A model reviewing output is a probabilistic check on a probabilistic system — you are asking the thing that produced the vulnerability to notice it, with no guarantee it notices the same one twice. Cencurity operates at the infrastructure layer with policy-based enforcement and structured extraction. Same rule, same verdict, every time, and the rule is readable. Determinism is the product. I had been leading with what it inspects. Everyone who understood it immediately had been led there by the fact that its answers do not vary.

That reframing came out of a comment thread, not out of a month of my own drafting.

What I would do differently

Two things, and the honest version of the second one is uncomfortable.

First, I would have shipped the install path before the policy engine. I knew the Docker barrier was high at launch and shipped anyway, and the fix — an extension and one command — was not technically hard. Knowing about a problem and treating it as a problem are different things, and the gap between them cost me a launch's worth of attention.

Second: zero issues filed is not a sign that the policy engine is well tuned. It is a sign that not enough people have run it against code that is not mine. I tuned false positives against my own repositories, which is a sample size of one with one coding style. A DevOps repository full of legitimate subprocess and shell calls will fire alerts it should not, and I know that structurally rather than from a bug report. Getting real installs against unfamiliar codebases is the next thing that has to happen, and no amount of further building substitutes for it.

Your AI Assistant Ships Insecure Code Almost Half the Time. Catching It in Review Is Too Late.

Sangyeon Park — Fri, 21 Aug 2026 19:24:32 +0000

By Sangyeon Park — creator of Cencurity, an open-source security gateway for LLM coding agents

In March 2026, Veracode published its Spring GenAI Code Security Update. The team ran more than 150 large language models through 80 code-generation tasks in Java, JavaScript, C#, and Python, then tested every output against four common weakness categories.

The syntax was excellent. Over 95% of the generated code compiled and ran.

The security was not. Forty-five percent of it contained a known vulnerability.

The raw number is not what should worry you. What should worry you is that the number has not moved. Veracode's phrasing is unusually blunt for a vendor report: security pass rates "remain stubbornly stuck at approximately 55%" — virtually identical to where they stood two years ago. Two years of longer context windows, better reasoning traces, and steadily climbing SWE-bench scores, and the security line is flat.

The failures are not evenly distributed

Break the results down by weakness class and the shape gets strange:

Weakness class	Security pass rate
Insecure crypto (CWE-327)	86%
SQL injection (CWE-89)	82%
Cross-site scripting (CWE-80)	15%
Log injection (CWE-117)	13%

SQL injection has been the canonical example in every introductory security tutorial written since roughly 2005. Parameterized queries are everywhere in the training corpus. Log injection has almost no pedagogical footprint by comparison — a real weakness that simply never became a teaching example.

The models are not reasoning about security. They are reproducing the security posture of whatever code they saw most of.

Where the corpus was well-taught, output is safe. Where it wasn't, output is not. That is a training-data artifact, and it is not the kind of thing a bigger model fixes on its own — which is precisely what Veracode observed. The reasoning-tuned models did better, reaching 70–72%, and still fell short of anything you would ship.

Language matters too, and not the way most teams assume. Python scored 62%, C# 58%, JavaScript 57%. Java came in at 29%.

The part nobody plans for: developers get more confident, not less

Neil Perry and colleagues at Stanford ran a controlled user study on exactly this question. Participants with access to an AI assistant (OpenAI's codex-davinci-002) "wrote significantly less secure code than those without access."

Then the finding that should reshape how you think about tooling: those same participants "were more likely to believe they wrote secure code."

Worse output, higher confidence. Those two effects compound, and they compound in the place where your last line of defense lives — human review. A reviewer who assumes the generated code is fine reads it faster and more charitably than one who wrote it themselves at 2 a.m. and knows where the shortcuts are.

Perry's team also found the mitigating factor, which complicates the doom narrative: participants who distrusted the assistant and actively rewrote their prompts produced fewer vulnerabilities. Skepticism worked. Skepticism just does not scale to a team of thirty engineers with a sprint deadline.

Vulnerable code is only half the exposure

Focusing only on generated code understates the problem, and this is the part I got wrong when I started building in this space.

When you put an agent in your development loop, you are not just receiving code from a model. You are sending it things. Repository contents, environment values, error output pasted from a terminal, sometimes credentials sitting in a config file the agent decided to read. And the agent acts on what comes back — writing files, running commands, calling tools.

That surface has four distinct failure modes, and traditional application security covers none of them well: prompt injection, unsafe tool calls, data leakage on the outbound path, and actions taken with no record that they happened.

Scanning the code the model produced addresses one of those four.

Shift-left already shifted as far left as it goes

The industry's standard answer to "vulnerabilities are found too late" has been shift-left: move scanning earlier — pre-commit hooks, IDE linters, CI gates. SAST reads source at rest. DAST hits a running application. IAST instruments runtime. All three are good tools, and all three share one assumption that generative AI just broke.

They all assume the code already exists.

With an agent in the loop, there is a window between the model emitting a token stream and that stream landing in your editor — and in agentic setups, between the stream landing and the agent executing it. In a Roo Code or Claude Code session running semi-autonomously, that window can close before any human looks at anything. A subprocess.run(user_input, shell=True) can be written, saved, and executed inside one turn. Your pre-commit hook fires afterward, if it fires at all.

That gap is the problem I built Cencurity to close.

What a security gateway actually does

The architecture is deliberately unremarkable, because unremarkable is what gets adopted:

IDE / Agent  →  Cencurity Gateway (127.0.0.1:38180)  →  LLM Provider

A local gateway binds to loopback. Your agent points its provider base URL at it instead of at api.openai.com. Outbound requests pass through a policy engine that redacts secrets — API keys, credentials — before they leave the machine. Inbound responses are inspected for dangerous constructs like eval and subprocess before the editor renders them. Everything runs locally; nothing is shipped to a third party for analysis.

The part that took the longest to get right is that this happens on the stream. Waiting for a complete response and then scanning it is a much easier problem, and it gives up the thing that makes the approach worth building — enforcement has to land while the model is still producing tokens, or you are back to inspecting code that already exists.

I call the category CAST — Code-Aware Security Transformation. The distinguishing property is not the analysis technique, which is comparatively conventional. It is the moment of intervention: policy is enforced inline, while the model is still writing the code.

Two design decisions carried more weight than the detection logic itself.

API keys never leave the IDE. The gateway forwards credentials; it never stores them. This closes the first objection in any security review, and it rules out an entire class of features — no cross-session budget enforcement, no org-wide key rotation.

Only violations get logged. This one I reversed. Cencurity launched recording all traffic as audit logs, which is the obvious design if you are thinking about compliance, and the wrong one if you are thinking about adoption. Full logging gives you far better policy tuning and a real audit trail. It also means a security tool is now recording every prompt a developer writes, which makes it both a morale problem and a fresh exfiltration target. A security product that creates a new breach surface has negative value. Current builds record policy violations and security events; normal traffic passes through unrecorded. I took the worse telemetry.

Cencurity sits in front of Roo Code, Continue, Claude Code, and Gemini CLI, routing to OpenAI, Anthropic, Gemini, OpenRouter, and OpenAI-compatible endpoints.

The objections, including the ones that land

"How is this different from having an LLM review the generated code?" This is the first question almost everyone asks, and it deserves a real answer rather than a slogan. A model reviewing output is a probabilistic check on a probabilistic system — you are asking the thing that produced the vulnerability to notice it, with no guarantee it notices the same one twice. A gateway does policy-based enforcement with structured extraction at the infrastructure layer. Same rule, same verdict, every time, and you can read the rule. Determinism is the entire argument. Where the model approach wins is semantic nuance, and I would not claim otherwise.

"This is just a proxy. I can write a proxy." Yes. The proxy is the easy half. The policy set, the masking rules, and the false-positive tuning are the part that takes months, and they decide whether the thing survives contact with a real team.

"Another network hop in my inner loop." Fair. Local loopback keeps this small relative to a round trip to a hosted model, but it is not zero, and I have not published p99 numbers under load because I do not yet have measurements I would stand behind.

"False positives will train my developers to ignore it." The strongest objection, and I do not have a complete answer. A gateway that blocks legitimate subprocess calls in a DevOps repository is worse than no gateway. Policy tuning is genuine, ongoing work, and it is the thing most likely to decide whether this approach is viable at all.

Where this leaves you

None of this argues that runtime interception replaces code review, SAST, or the security engineer who actually reads the diff. It argues that a control which fires after generation is structurally the wrong shape for a workflow where generation and execution now happen in the same breath.

Veracode's flat two-year line is the strongest available evidence that this will not resolve itself through model improvement. If the pass rate were climbing, waiting would be a defensible strategy. It isn't, so it isn't.

The gap is real, it is measurable, and it sits in a place where nothing you currently run is watching.

Cencurity is open source under Apache-2.0 on GitHub and launched on Product Hunt.