DEV Community: Manos Saratsis

ISO 42001 for Engineering Teams: What It Actually Asks You to Prove

Manos Saratsis — Mon, 28 Sep 2026 14:39:19 +0000

Originally published on the Dromeas blog.

In short: ISO 42001 is a voluntary, certifiable standard for managing AI systems. For engineering teams it's mostly about evidence: showing how AI-assisted changes are reviewed, tested and approved. It complements the EU AI Act and SOC 2 rather than replacing them.

Framework	Type	Question it answers
ISO 42001	Voluntary certification (3 years, with surveillance audits)	Is your AI governed?
EU AI Act	Law, with penalties	Does your AI meet legal obligations in the EU?
SOC 2	Attestation	Do your security controls work?

ISO 42001 keeps turning up in the same sentence as SOC 2 and the EU AI Act, usually on a slide that implies you need all three by next quarter. I wanted to know what it really asks of the people who write and ship code, so I sat down with it.

Short version: it's less about paperwork than I expected, and a lot more about evidence.

What it is

ISO/IEC 42001 is the international standard for an "AI management system." ISO published it in December 2023, and it's still the current edition. If you've been through ISO 27001, the shape will feel familiar: a set of clauses on how you run the system (the usual plan, do, check, act loop), plus an Annex A with 38 controls across nine areas. You pick the ones that apply and justify your choices in a Statement of Applicability.

The important bit: it's not a spec for how your model should behave. It's about how your company governs the AI it builds, buys or uses. Who's accountable, how you assess risk, where your data comes from, how you watch things once they're live, and what happens when something goes wrong. It applies whether you're building AI products or just letting your engineers use AI tools.

Who's going for it

It's early. BCG announced in January that it was among the first 100 organizations certified worldwide. Pega got certified in February for Pega Cloud and its GenAI features. I couldn't find a reliable global count of certified companies, and I'd be a bit wary of anyone who quotes one.

The trajectory looks a lot like SOC 2 a few years back. AI vendors selling into enterprise are the first to feel it, because it starts showing up in security questionnaires. If that's you, it's worth knowing what it'll ask before a customer asks first.

What an auditor actually wants to see

Certification happens in two stages. The first is a document review: your AI policy, your risk register, your Statement of Applicability. The second is the one that catches people out. The auditor checks that your controls actually run, which usually means interviews, watching processes, and pulling samples. They'll pick a feature or a date and ask you to show them the control working then.

For an engineering team, the Annex A areas that land on your desk are roughly these:

Lifecycle (A.6): design decisions, what you verified before deploying, what changed in each release.
Data (A.7): where your training, fine-tuning or RAG data came from, how you checked its quality, and what you did when problems showed up.
Impact assessment (A.5): done before launch, not written up after an incident.
Use (A.9): evidence the system is used within its intended purpose.
Suppliers (A.10): if you call a foundation model API, who's responsible for what.

The general rule auditors follow will sound familiar if you've done SOC 2. A record that was produced independently, timestamped and tied to a specific decision counts for a lot more than a policy saying the process happens. Policy says what should happen. Evidence shows it did.

How it fits with the EU AI Act and SOC 2

We've written about the EU AI Act before. It's law, with real penalties, and it applies if you serve the EU market. ISO 42001 is voluntary. You get certified by a third party, the certificate lasts three years, and there are surveillance audits in between.

They cover a lot of the same ground (data governance, risk, human oversight, transparency), but one doesn't satisfy the other. If you're building a high-risk system under the Act, a 42001 certificate is useful supporting evidence. It doesn't replace the legal obligation.

SOC 2 is a different question again. It tells a customer your security controls work: access, encryption, incident response. It says nothing about how you govern the AI itself. Some analysts are already describing ISO 42001 plus SOC 2 Type II as the new baseline for AI vendors selling to enterprise. That's one view, not a rule, but it gives you a sense of where things are heading. Three audits, three different questions: is your AI governed, are your security controls working, is your infrastructure secure.

Where Dromeas fits

We don't certify anyone. That's accredited third-party work, and it should stay that way.

What we do produce is the kind of record the Stage 2 auditor goes looking for. Every pull request and trunk commit gets reviewed. Every tagged release gets a verdict across six checks (quality, security, compliance, testing, docs, instrumentation), grounded in the actual diff and stamped with a date. If an auditor asks "show me what was verified before this release shipped," that's already sitting there. More on how we handle frameworks on our compliance page.

— Manos

Sources: ISO/IEC 42001:2023 · BCG, Jan 27, 2026 · Pega via Businesswire, Feb 3, 2026 · Konfirmity, Annex A controls · ISMS.online, ISO 42001 vs EU AI Act · Vanta, EU AI Act & ISO 42001 · Knowlee, ISO 42001 vs SOC 2 vs ISO 27001

Behavioral Bugs Are Still Slipping Through AI Code Review. Here's What Actually Catches Them.

Manos Saratsis — Thu, 17 Sep 2026 15:28:04 +0000

Originally published on the Dromeas blog.

The pull request looks clean, the linter's green, the tests pass — and the code still does the wrong thing the moment someone hands it an input nobody thought about. Not a crash, not a lint warning. Just the wrong answer, quietly.

I've been digging into why that keeps happening even as AI writes more and more of our code, and it turns out there's decent data on it now — not just vibes. So let's go through what the numbers actually say, why the tools most of us already run can't really catch this category of bug, and what I think actually works.

What the 2026 data says

First: people don't fully trust this code, and it's specifically about correctness, not whether it runs. Sonar's 2026 State of Code Developer Survey found 96% of developers don't fully trust that AI-generated code is functionally correct, and 61% agree that "AI often produces code that looks correct but isn't reliable." Only 48% say they always verify AI-assisted code before committing it. Worth sitting with that for a second: the complaint isn't "it doesn't compile." It's "it runs, it looks fine, and I don't actually know if it's right."

Second, when you break down what kind of bugs show up, it lines up with that. A 2026 empirical study ("Debt Behind the AI Boom") looked across five widely used AI coding tools — Copilot, Claude, Cursor, Gemini, Devin — and found code smells made up 89.3% of flagged issues (the stuff any linter catches fine). Correctness issues were a smaller slice, 6.0% — but the single most common one was "undefined variable or reference," almost 24,000 instances, which the researchers describe as code that "may look locally correct, but still fails to stay consistent with the surrounding context." That's a pretty good one-line description of the whole problem. More than 15% of commits from every single tool they studied introduced at least one issue, and 22.7% of the AI-introduced issues they tracked were still sitting in the repo, unnoticed, at the time of the study — some for nine months or longer.

Third, when this reaches production, it's not staying theoretical. New Relic's 2026 State of AI Coding report found 82% of organizations had at least one major production failure caused by AI code in the past six months, and 78% report a measurable spike in incidents tied to AI code overall — with AI-generated code introducing roughly 1.7x more critical runtime issues than human-reviewed code. CloudBees' 2026 State of Code Abundance report found something similar from a different angle: 81% of enterprise leaders report increased production issues tied to AI code, and their own summary of it stuck with me — "writing code is no longer the primary bottleneck, governing it is." Same conclusion, three separate 2026 surveys.

And the backdrop makes all of this harder to catch by hand. GitClear's 2026 research tracked eight quality signals across 623 million code changes from 2023 to 2026, and the trend lines aren't great: duplicated code blocks up 81% since 2023 (highest on record), copy-paste share of changed lines up from 9.4% to 15.7%, actual refactoring down about 70% over the same stretch. GitClear's own read on it: block duplication is the single risk signal most tied to defects and propagated bugs in the research they looked at. So more code is shipping, less of it is getting consolidated or re-examined, and reviewers have less bandwidth per line than ever.

Put simply: it's not that AI writes "bad" code. It clears the bars we're good at checking automatically — style, structure, the obviously dangerous patterns — and it's specifically weaker on whether the logic holds up for a real input, in a way that's now showing up as real production failures, not just review comments. That's worth being precise about, because it tells you exactly why the tools most teams already run don't solve it.

What these bugs actually look like

"Behavioral bug" is the term I keep reaching for, but it's worth knowing it's not the only name people use for this — you'll see the same basic idea called logic bugs or logic errors (the plain-English default), semantic bugs (more of an academic/compiler-literature term — the code is syntactically fine but means the wrong thing), correctness bugs or correctness issues (the term the arxiv paper above uses), business logic bugs or business logic vulnerabilities (the AppSec framing), and functional bugs (QA/testing terminology). Some people just call them silent bugs or silent failures, which honestly might be the most useful name of the bunch — it points at the actual property that makes them dangerous: nothing crashes, nothing throws, nothing shows red in CI.

In practice, most of what falls under that umbrella breaks down into a handful of recurring shapes:

Boolean logic bugs — an inverted condition, the wrong operator, a guard clause that lets through exactly the case it was written to block. Classic version: a permission check written with || where it needed &&, so access gets granted if any one condition is met instead of requiring all of them.

Reference correctness bugs — stale closures, the wrong variable captured, the wrong object mutated once two calls overlap. Classic version: a loop that captures its loop variable by reference in a callback, so every callback ends up pointing at the last value instead of the one that was current when it was created.

Boundary and indexing bugs — the everyday off-by-one: loop bounds, first/last-element handling, pagination math. Classic version: a "load more" offset computed as page * pageSize instead of (page - 1) * pageSize.

Normalization-symmetry bugs — a value gets normalized on write but not on read (case, whitespace, encoding), so a lookup silently misses. Classic version: emails lowercased at signup but not at login.

Nullability bugs — a new code path where a value can legitimately be absent, and nothing downstream accounts for it.

Shared-state concurrency bugs — races between concurrent writes, a rollback to a stale captured value, a cache write that clobbers something fresher.

None of these are exotic. Every engineer has shipped at least one of each at some point. What's changed is the volume and the plausibility — code that reads as confidently correct, generated fast enough that nobody's tracing each new input by hand anymore.

Why static analysis can't really get at this

Tools like SonarQube, Semgrep, and CodeQL work by pattern matching — an AST shape, a taint path from untrusted input to a dangerous sink, something that matches a known CWE. Genuinely useful, and it's why these tools are worth running. But it only works if the bug matches a pattern someone already wrote a rule for.

Most behavioral bugs don't work that way. Gecko Security put this well in a piece on why static analysis struggles with business logic: a lot of these bugs are about something being missing — a missing auth check, a missing validation step — rather than something dangerous being present that a rule can flag. Taint analysis can tell you untrusted data reaches a sensitive spot. It can't tell you whether the authorization logic guarding that spot is actually correct. And a lot of the bugs that cause real incidents span multiple files — their example is a real Cal.com auth-bypass bug that needed three separate issues chained together across different files, each looking fine on its own.

So that's the real limitation: pattern matching can tell you something looks like a shape it's seen before. It can't tell you that a specific input, actually walked through the logic, produces a different result than it should. That requires tracing behavior against intent — and it happens to be exactly where the data above says AI-generated code is weakest.

It's also why I think "detect that AI wrote this, then review it harder" — which a few vendors have shipped, SonarQube's AI Code Assurance being one — is sorting on the wrong axis. It's routing on who wrote the code, when the thing that actually matters is what the code does. The question was never "who wrote this" — it's "does this do the right thing for a real input," and that needs tracing, not detection.

Where this leaves Bug Tracing

This is basically the problem we built Bug Tracing to solve inside Dromeas — trace real inputs through the actual changed code and check what comes out, instead of scanning for patterns. The six categories above are exactly what it scores every change against, and it only reports a finding if it can name the concrete input that triggers it — no reproducible trace, no report.

And because "review a diff" and "audit a whole system" are genuinely different jobs, it runs at whichever scope your team actually works at: on the pull request itself (changed lines plus their blast radius), on a branch or release after merge for teams doing trunk-based development, or as an on-demand sweep across a whole repository when you just want a second opinion on a system nobody's looked at closely in a while.

If this is a problem you're running into, happy to show you how it works — the feature page is at dromeas.ai/bug-tracing.

Shadow AI in Your Codebase: The Governance Gap Most CISOs Haven't Mapped Yet

Manos Saratsis — Sun, 13 Sep 2026 13:27:37 +0000

Your AI code policy covers approved tools. The harder question is whether every change faces the same evidence-backed review, regardless of what wrote it.

Shadow AI is no longer a vague compliance concern. It describes employees using AI systems outside an organization's approved controls. The result can be both data exposure and software entering production without a consistent review record. That second problem matters even when the code looks ordinary.

A note on sourcing: the figures below come from a mix of primary surveys and industry roundups, and each is attributed to the publication that states it. Shadow-AI measurement is still thin, so treat the percentages as directional signals rather than settled numbers.

The visibility gap is larger than most programs assume

Airia's 2026 overview collects findings from named third-party studies: source code represented 35% of sensitive-data events involving AI tools in Cyberhaven's 2024 data; 60% of security teams lacked visibility into which AI tools employees used, per Cisco's 2024 AI Readiness Index; and Gartner reported in 2025 that 43% could not audit or inventory the AI tools in use.

A separate JumpCloud summary reports widespread unsanctioned assistant use. Survey percentages are useful warning signals, but they don't establish which tool authored a given line in your repository. That distinction matters when choosing a control.

Source code creates two different shadow-AI risks

Input risk appears when proprietary code, secrets, customer data, or architecture details are sent to an unapproved service. That requires controls such as approved-tool policy, identity and endpoint management, DLP, vendor review, and data-retention rules.

Output risk appears when generated code is accepted because it looks plausible, or enters through a workflow that never receives equivalent quality, security, and compliance checks. This is the part a repository and release review layer can address directly.

Govern the output without pretending the tool is invisible

Banning unsanctioned tools doesn't guarantee compliance, but governing only the output doesn't solve data leakage either. A durable program needs both: discover and control the tools where possible, then require the same review bar for every software change regardless of authorship.

This avoids a fragile "detect AI code, then review it differently" path. Pull requests, direct commits to the default branch, and pre-PR local diffs can all enter through different routes. The control should follow the change.

What an AI-BOM can, and cannot, tell you

Dromeas generates an AI Bill of Materials when a product actually invokes AI or machine-learning systems: a machine-readable CycloneDX 1.6 ML-BOM plus a human-readable report covering detected AI/ML components, data flows, risk classification, EU AI Act obligations, and source-backed provenance. For multi-repository products, the documentation process assembles evidence across the whole product rather than treating each repository as an isolated system.

That's an inventory of AI/ML components used by the software. It is not a forensic attribution system for which coding assistant wrote each line, and it doesn't calculate a trustworthy percentage of AI-authored code. It complements workforce AI-tool inventory, DLP, endpoint controls, and an AI-use policy; it doesn't replace them.

A practical first step

Inventory the services: use identity, network, endpoint, procurement, and expense data to find unapproved AI tools.
Protect the inputs: define what code and data may be sent to each approved service, and enforce it with technical controls.
Standardize the output gate: review PRs, trunk commits, and local agent diffs against one policy.
Record what shipped: keep review and release evidence, and generate an AI-BOM for software that contains or invokes AI/ML components.

Shadow AI can't be solved by one inventory or one scanner. It becomes governable when tool controls and change-level evidence meet at the release boundary.

Full piece, with more on Dromeas's changed-code and blast-radius review pipeline (independent Security, Quality, Compliance, and Bug Tracing analysts) plus AI-BOM documentation, originally published at dromeas.ai: https://dromeas.ai/blog/shadow-ai-in-your-codebase

Which AI Model Writes the Most Secure Code? What the 2026 Data Actually Shows

Manos Saratsis — Sun, 13 Sep 2026 13:26:40 +0000

Ask five engineering leaders which AI coding model is "safe," and you'll get five confident, contradictory answers. Veracode's 2026 GenAI Code Security Report gives the question a measured answer: across its benchmark tasks, AI-generated code passed security tests 56% of the time and introduced a risky vulnerability in the other 44%. That's effectively flat from the previous report's 55% pass rate, while Veracode estimates AI now authors roughly half of committed code.

Flat security performance at higher volume isn't a wash. It's the same failure rate landing on more production code.

The plateau nobody's marketing slide mentions

A separate CodeRabbit study reported 2.74x more vulnerabilities in AI-co-authored pull requests than in human-only pull requests, from 470 open-source PRs (320 AI-co-authored, 150 human-only), not from Veracode. CodeRabbit notes an important limitation: authorship was inferred from signals rather than confirmed ground truth.

Different datasets point in the same direction: model capability and secure output don't rise on the same curve.

Vulnerability class matters more than the average

Veracode's aggregate result hides a much sharper spread:

SQL injection: 83% pass rate
Cryptographic implementation: 87% pass rate
Cross-site scripting: 15% pass rate
Log injection: 12% pass rate

The hard cases require context: where untrusted data entered, how it moved through calls, where it reached a sink. That's a dataflow problem, not a syntax problem.

Model selection is a security decision, not a security control

GPT-5.5 led Veracode's benchmark at 68%. More than half of tested models clustered at 50-53%. Coding-specialized models averaged 51%, general-purpose models 52%, and reasoning models 56% vs. 51% for non-reasoning variants. Model size showed little correlation with security outcomes.

There's no universal "most secure" model. Even the benchmark leader failed nearly one test in three.

What to do with this data this quarter

Ask for weakness-level evidence, not just an aggregate score.
Review at the change boundary: local diffs, PRs, trunk commits, while context is fresh.
Prioritize reachable risk over raw finding counts.
Record the policy: which models, which checks, who signs off.

The 2026 data doesn't identify a shortcut past review. It makes the case for a review layer that stays useful when the model roster changes.

Full piece, with more on Dromeas's multi-model review pipeline (independent Security, Quality, Compliance, and Bug Tracing analysts), originally published at dromeas.ai: https://dromeas.ai/blog/which-ai-model-writes-most-secure-code

MCP for Code Review: What It Is, and How to Add a Review Layer to Claude Code or Cursor

Manos Saratsis — Wed, 09 Sep 2026 13:24:23 +0000

Originally published on the Dromeas blog.

Your coding agent writes code. MCP is how you give it a reviewer. Here's what the protocol actually does — and a working setup you can copy in minutes.

MCP, in one paragraph

The Model Context Protocol is an open standard that lets an AI client — Claude Code, Cursor, Windsurf, Zed — call external tools through a uniform interface. Instead of pasting output between your terminal and a review dashboard, the agent invokes a tool like review_pull_request directly and receives the result as structured data it can reason about. Tool support, not copy-paste, is what makes an agentic loop possible: the agent can act, observe the verdict, and repair — the loop we described in loop engineering.

Why a review layer belongs in the loop

A coding agent with no review layer grades its own homework. It writes a diff, checks that it compiles, and declares victory. A review layer changes the economics: the same agent submits its diff, gets an independent multi-model verdict — security, quality, compliance — and fixes what it finds before a human spends a minute on it. Self-review before the PR is the single highest-leverage place to insert checking, because the cost of a fix there is one tool call, not a review round-trip.

The setup, step by step

Connect your repositories. Sign up at dromeas.ai and connect GitHub, GitLab or Bitbucket. Dromeas builds a typed code map of your repos so review verdicts come with real context.
Get your MCP endpoint. Open Settings > MCP in the Dromeas app and copy your workspace's MCP server URL and API key. One endpoint covers Claude Code, Cursor, Windsurf, Zed and any other MCP-capable client.
Add the server to your client. In Claude Code run claude mcp add with the URL; in Cursor, add the server under Settings > MCP. The review tools appear in your agent's tool list immediately.
Ask your agent to self-review. Before opening a PR, ask your coding agent to submit the diff for review. It calls review_pull_request or review_local_diff, gets a multi-model verdict, and can fix what it finds before a human ever looks.

What your agent can actually do once connected

The Dromeas MCP server exposes the full review surface as tools: review_pull_request and review_local_diff for verdicts, code_finder_search and the code-map tools for cheap context before edits, get_findings and the fix tools for acting on results, and release tools like describe_release_state for go/no-go decisions. The point is that review isn't a dashboard your agent can't see — it's a function it can call.

Because every tool call is scoped to your workspace and logged, you keep the audit trail that matters when agents start merging on their own. That's the same provenance property the CISO checklist depends on.

Connect a repo, copy your MCP endpoint, and your coding agent gets a six-agent review council it can call from the terminal. Read about loop engineering for the fuller picture of what the loop looks like end to end.

The CISO's Guide to AI-Generated Code at Scale

Manos Saratsis — Wed, 09 Sep 2026 13:23:14 +0000

Originally published on the Dromeas blog.

AI-generated code has 2.74x more vulnerabilities than human-written code. Copilot adoption is near-universal in the Fortune 100. And the EU AI Act's high-risk obligations now land on 2 December 2027. Here is the checklist that actually matters.

The security conversation about AI-generated code has moved past "should we allow it" — your engineers already use it, and most of them use it daily. The question now is operational: how do you govern a codebase whose authorship is increasingly non-human, at a pace no review board can match?

Three numbers frame the problem. AI-generated code carries roughly 2.74x more vulnerabilities than human-written code (Veracode 2025). GitHub Copilot is used by around 90% of the Fortune 100 (GitHub), before you count Claude Code, Cursor, and the rest. And the EU AI Act's obligations for high-risk AI systems now apply from 2 December 2027 — pushed back from August 2026 by the AI Omnibus, which entered into force on 27 July 2026 (systems embedded in regulated products get until 2 August 2028). The extra runway does not change the expectation: documented provenance, an AI Bill of Materials, and evidence of how AI-assisted code was checked. If your governance plan is still "we'll review the PRs carefully," it is already behind the reality on your main branch.

Where the risk actually concentrates

Not all generated code carries the same risk. In practice the exposure clusters in three places, and each one has a different mitigation shape:

Dependency and secret drift. Agents reach for packages and patterns from training data — outdated versions, deprecated libraries, and occasionally hardcoded credentials in example-shaped code. Static dependency scanning catches the known-CVE slice; it misses the "correct-looking but wrong" slice, like an auth helper that works and is subtly unsafe.

The review blind spot. Human reviewers read generated code more charitably than human-written code — it looks confident and idiomatic, so it gets skimmed. That is precisely the code the 2.74x number is describing. The blind spot isn't negligence; it's a predictable cognitive effect, which means it needs a systemic fix, not a reminder.

The provenance gap. When an auditor (or your own incident review) asks "what wrote this line, under which policy, with which model," most teams today have no answer. That gap is what the AI Act's AI-BOM expectation is aimed at: not knowing the answer is becoming a compliance finding, not just an inconvenience.

What a workable governance layer looks like

The teams that get ahead of this do three things, and none of them involve slowing developers down:

One bar, no detection step. Apply the same security and quality checks to every change regardless of authorship. Architectures that first "detect AI code" and then route it somewhere stricter add a failure mode (the detection) and a governance seam (the routing). Sonar is already retiring the auto-detection half of its own feature — we wrote up why that's the wrong default. A single uniform gate can't be skipped by mislabeling.
Post-merge coverage, not just PR gates. Agents increasingly commit directly to trunk. If your security review only exists on the PR, a growing share of your production code passes no gate at all. Every commit — PR or trunk — should get the same automated pass.
Provenance as a byproduct, not a project. Every review run should record what was checked, by which models, against which standard, with which verdict. Do that and the AI-BOM, the SOC 2 evidence, and the EU AI Act documentation all fall out of normal operation instead of becoming a quarterly scramble.

The questions to put in front of your team this quarter

If you do nothing else, get concrete answers to these:

Which agents and assistants are committing code today, to which repos?
What percentage of merges in the last 90 days had no human review?
If an auditor asked for a bill of materials of the AI systems that touched production last month, could you produce one?
Which checks run on trunk commits after merge — and who reads the results?
When a generated-code incident happens, what is the provenance trail you would walk?

Most organizations can answer one or two of these today. December 2027 sounds far away; the codebase you will be audited on is being written this quarter.

PR Review vs. Trunk Review: A Practical Guide to Choosing (or Combining) Both

Manos Saratsis — Wed, 09 Sep 2026 13:21:50 +0000

Originally published on the Dromeas blog.

PR review and trunk-based review solve different problems, and most teams already run a hybrid without naming it. This guide is backed by data from 100,000+ real pull requests and 24,000 trunk commits across 500+ open-source repos.

When we looked at that data, one finding kept surfacing in different forms: the PR-versus-trunk question isn't binary. Teams that think they've picked one are usually already running a hybrid, they just haven't named it. Here's what the data actually supports about when each one earns its keep.

What PR review is for

PR review is built for the moments that benefit from a checkpoint: large or risky changes, anything that needs discussion before it merges. That matters because PR size in the wild is bimodal — plenty of small, quick changes, but a heavy tail of 1,000+ line PRs that are exactly where a pre-merge gate is worth the friction it adds. The PR is also the only place where a change is still cheap to reject: once it merges, every fix is a new change instead of a revision.

What trunk review is for

Trunk review is built for everything else. Direct-to-trunk commits in our sample averaged 63x smaller than PRs — fast, low-friction, and increasingly the shape of how coding agents actually commit. Waiting for a full PR gate on every micro-change doesn't match agent-speed workflows, and it doesn't need to. The mistake is assuming "no PR" means "no review" — trunk commits can be reviewed after the fact against the same bar, without blocking the commit path.

The tail is where the time goes

The part that costs teams the most time isn't the average case in either lane — it's the tail. Rejected PRs took 6x longer to resolve than approved ones in our data. That's where review capacity actually gets consumed: not on the easy approvals, but on the changes that go back and forth. Any review strategy that doesn't have an answer for the tail — smaller initial diffs, automated first-pass review, faster feedback — will feel slow no matter which lane you picked.

A decision framework that matches reality

Instead of "PRs for everything" or "trunk for everything," score each change on four axes and let the lane follow:

Signal	Lean PR gate	Lean trunk + post-merge review
Change size	Hundreds of lines and up; the heavy tail	Tens of lines; single-purpose commits
Blast radius	Touches shared contracts, auth, billing, migrations	Local, reversible, behind a flag
Contributor type	New contributor, unfamiliar area, cross-team change	Owner of the area, or an agent doing a scoped fix
Discussion value	Design decisions others need to weigh in on	Mechanical or already-agreed work

Most teams discover their real policy is already this table, applied informally. Making it explicit does two things: it stops the arguments about whether a given change "deserved" a PR, and it makes the post-merge lane a first-class citizen instead of an unreviewed loophole.

Why most teams will end up running both

The forces pushing the two lanes apart are getting stronger, not weaker. Risky changes are getting larger (more generated code per PR), and routine changes are getting smaller and more frequent (agents committing at agent speed). A single gate tuned for one of those shapes fails the other: too much friction on the small stuff, too little scrutiny on the big stuff.

The workable shape is both lanes with the same rigor on each: a real gate on the PR for the changes that benefit from one, and automatic, same-standard review on every trunk commit for the ones that don't. That's the model Dromeas implements — the same six-agent pipeline and multi-model council runs on PRs and on trunk commits alike, so the lane choice is a workflow decision, not a quality decision.

The data behind this guide

104,968 PRs and 23,964 trunk commits measured across 503 open-source repos — size distributions, review latency, and the slow-motion rejection problem in full.

Read the State of AI Coding 2026 · See the charts

How to Evaluate an AI Code Review Vendor in 2026: A Practical Checklist

Manos Saratsis — Wed, 09 Sep 2026 13:20:29 +0000

Originally published on the Dromeas blog.

Every AI code review vendor's homepage says the same three words: "AI-powered code review." That's not useful information anymore — of course it's AI-powered, it's 2026. The questions that actually separate these tools are more specific, and most comparison content in this category is either a vendor's own battlecard or a listicle nobody fact-checked. Here are six questions worth asking directly, in a demo or a trial, before you sign anything.

1. Does it review one point in the pipeline, or every point?

A bot on the PR is the default shape of this category. Ask what happens to a commit that lands directly on trunk, or a change that ships six weeks after the PR that introduced the underlying risk. If the answer is "we don't look there," that's not disqualifying on its own — but it's a gap you're accepting, and you should know you're accepting it. In our own measurements of 100,000+ PRs across 500+ open-source repos, a meaningful share of changes never went through a PR at all — so "we review PRs" and "we review your code" are materially different claims. The data on that is public.

2. Is a finding one model's opinion, or several checking each other?

Single-model review inherits that one model's blind spots. Ask whether the vendor runs more than one model against the same diff, and whether you can see where they disagreed. "We use GPT-5" is not the same claim as "three models reviewed this independently and here's where they split." The disagreement record is the useful part: a tool that hides it is asking you to trust a single point of failure with a friendly UI. (We wrote up how we built our own multi-model council if you want to see what the transparent version looks like.)

3. What happens after "no findings"?

A clean PR review says nothing about test coverage, stale docs, or missing observability on the code that just shipped. Ask whether the tool's job ends at the diff, or whether it's also checking the five other things that determine whether a release is actually safe. "No findings" should be the start of a release conversation, not the end of a review.

4. Who checks compliance, and how?

If you're regulated — SOC 2, HIPAA, PCI, GDPR — ask directly whether compliance is a first-class check or a slide in the sales deck. Most tools in this category are quality- or security-first and don't cover this at all; that's a fair trade-off if you don't need it, and a real gap if you do. The tell: ask to see a compliance finding in the demo, not a slide about one.

5. Per-seat, or usage-based?

Per-seat pricing was built for a world where humans opened PRs. As agents start committing more often than people do, seat count stops tracking the thing that's actually driving cost or risk. Ask how pricing behaves as commit volume rises independent of headcount — and whether you'll be penalized for the agent-heavy workflow the same vendor's marketing encourages.

6. Can your coding agent actually talk to it?

MCP support is now table stakes to claim, but "we have an MCP server" and "your agent can ask this tool for a verdict and act on it" are different claims. Ask what a coding agent can actually do through the integration — read-only context, or a working verify-and-fix loop. A quick test: can your agent submit its uncommitted diff for review and receive a verdict it can act on, without you leaving the terminal?

Ask everyone — including us

None of these questions require you to already know what you're comparing against. Ask them of any vendor, including us — the answers are the actual differentiator, not the marketing copy above them.

If you want the specific answers for the vendors you're already evaluating, the comparison hub has the full feature-by-feature breakdowns — CodeRabbit, SonarQube, Copilot, Snyk and more, each with sources and a last-verified date.

Run code review and releases from a conversation

Manos Saratsis — Tue, 01 Sep 2026 06:32:59 +0000

We just shipped chat as a first-class interface into Dromeas.

Instead of clicking through dashboards, you can now just ask: "review PR 247 on the payment service," "are we good to release v1.9.0?," "fix the top security finding and open a PR." Dromeas reads the diff, queries the Code Map for blast radius, runs the same quality/security/compliance agents that guard your trunk, and shows the result as a live status card — not a wall of text.

A few things worth calling out for anyone building similar agentic UX:

It's not a separate system. The chat calls the exact same MCP primitives (get_findings, code_map_search, run_finding_fix, approve_pull_request, etc.) that our IDE integrations for Claude, Cursor, and Copilot use. Start a release check in Cursor, see it finish in the chat.
Autonomy is a dial, not a toggle. Every workspace sets a default — manual, observe, assist, or auto — and you can override it per conversation. Manual shows a confirm card before anything ships; auto acts inside caps you set (file budget, severity threshold, model cost) and reports back after.
Structured over conversational-only. Long-running actions (a review, a fix, a doc run) return a live card with step, progress, and a deep link — updating in place instead of dumping another paragraph into the thread.

Video walkthrough: https://youtu.be/ypCt0d8sGis
Try it: https://dromeas.ai/chat

SonarQube Flags AI-Generated Code as a Special Case. That's the Wrong Default.

Manos Saratsis — Tue, 01 Sep 2026 06:31:56 +0000

Originally published on the Dromeas blog.

SonarQube has a feature called AI Code Assurance. When it detects that a project uses GitHub Copilot — checked via the GitHub Copilot Business org settings — it tags the project CONTAINS AI CODE and routes it through a dedicated quality gate built specifically for AI output, instead of the standard one (Sonar). More recently, they shipped a plugin that goes further: inside the GitHub Copilot CLI itself, an agent's generated code now gets run through an automatic verify-fix-reanalyze loop before it ever reaches a pull request (Sonar).

Both are sensible responses to a real problem — AI-generated code has roughly 2.74x more vulnerabilities than human-written code (Veracode 2025), so scrutinizing it more makes sense on paper. But look at the mechanism underneath both features: first you have to detect that a human didn't write this, then you route it somewhere stricter. AI-authored code is the exception you build a special lane for.

That's worth questioning, and not because the detection is badly built. It's because "detect, then route differently" only works as well as the detection does — and detection is inherently one step behind.

What AI Code Assurance actually checks — and what it misses

The detection step is worth being precise about, because its scope defines the size of the hole. SonarQube's autodetect mechanism evaluates Copilot usage patterns and code-contribution data through the GitHub Copilot Business organization API. Two consequences follow.

First, the wire it trips is GitHub Copilot-specific. A team running Claude Code, Cursor, or a local agent alongside (or instead of) Copilot doesn't necessarily trip that wire the same way. The special lane exists, but not every AI-authored line of code is guaranteed to be in it.

Second — and this is the part that makes the architecture question concrete rather than philosophical — Sonar's own docs now flag autodetect as deprecated in SonarQube Server 2026.1 LTA, with removal planned and manual project labeling as the remaining path (docs.sonarsource.com). Read that slowly: the vendor that built "detect AI code, route it to a stricter gate" is retiring the detection half of the design and asking humans to self-declare instead. Manual labeling is detection too — it's just detection delegated to the person least incentivized to do it carefully, at exactly the moment agent-written code is becoming the majority of new lines.

None of this is a knock on the feature's implementation. It's the predictable end-state of any architecture whose first step is "figure out whether a human wrote this."

The CLI plugin is the more interesting move

To be fair to Sonar, the GitHub Copilot CLI plugin (June 2026) is the smarter of the two ideas. It doesn't wait for detection at the project level — it runs an agentic loop right in the terminal: analyze the agent's output, fix what it finds, re-analyze (sonar analyze agentic) before the code reaches a PR. That's the right instinct: meet the agent where it works, and verify before merge rather than audit after.

Why "same rigor, no detection step" scales better

The alternative to a better detector is not needing one.

Every PR and every trunk commit that goes through Dromeas runs the same six-agent pipeline — quality, security, compliance, testing, docs, instrumentation — reviewed by the same multi-model council, regardless of who or what wrote it. There's no CONTAINS AI CODE badge, because there's no separate gate to route into. A junior engineer's Tuesday-afternoon commit and an autonomous agent's 2am commit get the identical bar.

That's not a philosophical stance so much as a practical one: as the share of AI-authored code climbs toward the 65% Sonar's own 2026 developer survey projects for 2027, "detect it, then scrutinize it more" is a rule that has to run correctly on a shrinking minority of code to matter, while "scrutinize everything the same way" doesn't have that failure mode at all.

There's a second-order benefit too. When there is no special lane, there is no lane-splitting argument — no "this was mostly agent-written, so the gate should have caught it" postmortem, and no quiet drift where one class of code quietly gets less review because nobody remembered to label the project. The bar is the bar.

If you're weighing the two approaches side by side, the full feature-by-feature breakdown — including AI Code Assurance and the new CLI plugin — is on the Dromeas vs SonarQube comparison page.

Claude Code's ultrareview vs Dromeas Code Review with LLM council

Manos Saratsis — Thu, 20 Aug 2026 14:51:07 +0000

Originally published at dromeas.ai

We heard about Claude Code's ultrareview and got excited — a cloud-run, multi-agent deep review sounded like exactly the kind of thing worth building a workflow around.

So we pointed it at changes in our own repo and compared it against Dromeas code review: three analyzers (quality, security, compliance) cross-checked by an LLM council. Dromeas held up well in that first pass.

That result was interesting enough that we wanted a harder, more neutral test: a large, real, independently-approved pull request from a codebase neither tool had any stake in. So we picked openclaw/openclaw — a public, actively-developed agentic coding tool — and went looking for its biggest recently-merged, genuinely-reviewed PR. That led us to openclaw#124250, 31 files changed, approved by a human reviewer, and we ran the same head-to-head again.

The PR

"Preserve ClawHub external source identity and expose only supported actions" — merged, approved by a human reviewer (not a bot self-merge), XL size: 31 files changed, +1,064/−116 lines, spanning the Control UI, macOS, iOS, and Android clients plus the backend that serves them.

The bug it fixes: ClawHub's search API returns each result's source under a nested install.reference field, but the client code expected a flat installRef. Every external search result silently fell through to a synthesized @owner/slug reference — quietly pointing installs at a different publisher's skill than the one the operator actually picked. An identity-spoofing bug in a skill-installation flow, fixed across five client surfaces.

What each tool found

ultrareview: 1 finding, nit severity — a duplicate test assertion in an Android test file, unrelated to the identity-spoofing bug the PR exists to fix.

Dromeas's LLM council: 29 candidate findings raised, 17 kept after cross-verification. Three models (Opus 5, DeepSeek V4 Pro, GPT-5.6 Terra) independently analyzed the diff, then a decider cross-checked each finding. All 12 quality findings and all 5 security findings held up; 12 compliance findings were flagged as duplicates of already-caught security issues or dropped outright, with the report explaining why for each.

None of Dromeas's 17 kept findings overlap with ultrareview's one — not because ultrareview did a bad job reading the diff, but because questions like "is this credential field masked" or "does this action get an audit trail" were never in its scope. Full breakdown, cost comparison (~$5 for the full council run vs. $5–25 typical for ultrareview), and the four findings flagged for manual triage are in the full post →

Read the full comparison

Loop Engineering: How to Actually Close the Loop When You're Coding With AI

Manos Saratsis — Mon, 10 Aug 2026 12:09:47 +0000

When we started experimenting with models coding we were looking at the right prompt, later at prompt chaining, then graphs. The higher the autonomy is we see there is the need for loops, both when coding but also when reviewing and releasing code.

Original blog here

Loop diagram with four nodes — act, observe, verify, repair — connected in a cycle
The loop. The interesting engineering lives in the bottom half.
Now multiple models are great on writing code at the act step. What requires further improvement is how the code is self healing, improving performance, take solid architecture decisions. Get real inputs to keep going, better and better each time.

That's the half of the loop Dromeas is built for. Not "write my code for me" — you already have something for that. Verify and repair. We run an amazing experiment: 16 feedback rounds with the same coding agent and a small React/TypeScript repo, building real features with and without the Dromeas MCP help. Btw this was the first time when the actual user is a machine and I have to admit that machines give more structured product feedback allowing a fun iteration until we got something useful.

Below is what actually held up. Including the parts that didn't.

What loop engineering looks like while you're coding
The loop while building a feature is four moves, and only two of them are the fun ones.

Orient. start_task({ repo_full_name, task }) in one call: where the code lives (paths, line ranges, symbols, callers), what's already known-broken in those files, and a token-budgeted read plan.
Act. Read the ranges from disk, make the change, keep the diff scoped.
Verify. Typecheck and lint first — they're free. Then verify_change with the full post-change content of every changed file. It analyses your uncommitted work; nothing needs to be committed or pushed.
Repair. get_findings({ trunk_review_id }) to read the blockers, then preview_fix for a diff or run_finding_fix to push one. Then back to step 2.
The bugs it caught that the agent didn't
This was the most consistent benefit across all 16 rounds, and it's the one worth the money. Not lint noise, not style nits — actual logic errors in code written minutes earlier.

While building a radius-select tool, a new commit() function in useRadiusSelect.ts didn't guard against a null draft. A spurious commit() with draft === null would silently wipe an already-committed selection. get_findings flagged it; the fix was one line — if (!draft) return; — and the agent's own log said it "would probably not have caught that on my own re-read."

Better one: a handlePointerUp click-vs-drag detector whose moved flag was set to true at pointer-down time. Meaning it never measured movement at all, so the comment right above it ("treat as a click if the pointer barely moved") was lying about what the code did. No linter or typechecker catches that. It takes reading code against its own comment — and two independent runs in two different rounds both caught it.

Knowing when to stop searching
Search tools have a failure mode where the agent keeps searching because searching feels like progress. Every code_finder_search and start_task response carries a value_signal, and it's honestly calibrated rather than self-flattering: tested side by side against a 22-symbol repo and a ~3,000-node repo, it rated the small repo's search value low (flat scores, most of the repo returned) and the large one medium. On low signal it returns an empty next_calls list, specifically so the agent doesn't reflexively chain another query.

On the star-map repo, getting_started reported first_move: "read_files" with the reason spelled out: "only 22 indexed symbols — reading the few source files end-to-end beats any search here." Every agent that followed it stopped after one or two orientation calls. That's real credit and context-window savings, from a tool telling you not to use it.

"What else touches this?" in one call
Search results inline the caller/callee graph and blast radius for top hits, not just a path. Searching worldToScreen came back with its file, its line range, its 3 callers (StarMapCanvas, hitTestStar, and the containing file) and a blast radius of 3 — enough to know a signature change ripples into exactly those three places, without opening any of them first. That's a manual grep chase replaced by one response.

Experiment rounds
16
same agent, same repo, with and without the MCP
Real logic bugs
2
caught in freshly written code, pre-commit
Blast radius
1 call
callers + impact inlined with search hits
Findings surfaced at bootstrap
222
60 critical, across two repos
We will cover in a different blog how loop engineering works when reviewing code or releasing products.

New Dromeas Skills
To get started with this we shipped three agent skills you can install straight into your coding tool from Workspace management → Agent instructions & skills

The honest downsides
While our early Loop engineering has been great so far, there are some downsides that you need to be aware.

It costs tokens, and on small codebases the math is worse. Orientation calls, verification payloads and findings all land in the context window. On a 22-symbol repo the agent can just read every file — and, to its credit, our own value_signal and first_move hints say exactly that. The loop earns its keep as coupling and history grow; on a toy repo it's overhead.
It makes tasks take longer. Verification is a real analysis pass, not a lint run — typically 60–120 seconds of polling per loop iteration. If you run it after every micro-edit, you'll feel it. Batch your edits, verify once per meaningful change. We're actively working on cutting that wall-clock time (quick: true already drops compliance for a materially shorter security + quality loop).
It's a loop, which means discipline. The value shows up when you actually read the findings and go back to step 2. An agent that dispatches a verification and then declares victory without reading the verdict has gained nothing. Notably, verdict: "unknown" with analyzed: false, retryable: true is not a pass — and yes, we had to write that in bold in the skill files.
Should I get started?
Actually, yes — especially if you build something robust and you have a sizeable codebase you will get real code context around dependencies, code scope and issues identified. Then your agent will self-heal your code every time it touches components with issues, always with some cost on time&token per task. Just add the new skills and get going.