DEV Community: Codacy

How to Build a Remediation Agent That Closes Its Own Issues

Codacy — Tue, 29 Sep 2026 16:51:53 +0000

TL;DR

A remediation loop needs only three parts: a scripted harness, one coding agent, and a human who approves the merge.
Codacy's Jira integration creates the tickets, a Jira automation on merge or the harness alling the Jira API closes them.
Codacy's Smart False Positive Triage should remove false positives before ranking anything, since ranking noisy data only produces a well-sorted list of non-issues.
The loop's value shows up as a lower incident rate, not just a faster clock, because unresolved findings are the risk being removed.

This article lays out a reference architecture based on real remediation agents that a European retail giant with 1,500 software engineers has built on top of Codacy, kept anonymous, alongside practices worth adopting even if you’re starting from zero.

The problem it tackles is familiar: remediation has to compete with feature work for the same engineering hours, so known findings can sit unresolved while new ones keep coming in. Someone still needs to pull the findings, triage them, decide what matters, create the work, write the fix, test it, and close the ticket.

An agentic remediation loop turns those steps into a repeatable cycle that keeps running in the background. A planned harness routes the actionable work, eliminates false positives, and pulls the results. One coding agent takes the fixes through pull requests, Codacy checks the changes, and a human owns the approval and merge.

Once the fix lands, the ticket closes and the loop returns to the backlog to work through whatever remains.

Section 1: What the loop is

The loop is a repeatable seven-stage cycle where a harness handles the mechanical stages, a single coding agent handles the fix, and a human owns the merge decision, returning to Stage 1 each run against a shrinking backlog.

That return trip is what separates a loop from a one-time cleanup project: the same script that pulled 400 open findings in week one pulls whatever's left in week eight, and the number keeps going down because the fixes from prior runs have already merged.

Stage	What happens	Codacy component
1. Source	The harness pulls open issues and security findings on a schedule	Codacy Cloud CLI
2. Triage	Remove false positives, then rank the rest by severity	Cloud CLI ignore + filters, API
3. Suppress	Record each ignore with a reason; exclude irrelevant paths	Cloud CLI ignore; `.codacy.yaml`
4. Route	Create detailed tickets, grouped by team, assigned to the fix agent	Jira integration
5. Fix	The fix agent writes the fix and opens a pull request	One coding agent + repository
6. Gate & merge	Codacy checks the PR; a human approves and merges	AI Reviewer; merge gate
7. Close	Close the ticket when the fix merges	Jira automation / the harness

The harness runs stages 1 through 4 and stage 7. The fix agent runs stage 5. Codacy and a human jointly run stage 6, which is the one safety mechanism in the entire system that a script cannot pass on its own.

Two security metrics matter here, and only one of them should sit on a leadership dashboard:

Incident rate, meaning how often production actually breaks, is the primary outcome metric because it shows whether the reduction in unresolved findings is translating into fewer incidents

Time to remediate high-severity findings is the operational metric, useful for tuning the loop's throughput, but it's a means to the first number rather than an end in itself.

Section 2: Source the findings

Sourcing means pulling a structured and reliable list of open findings on a fixed schedule rather than whenever someone remembers to check the dashboard.

The harness calls the Codacy Cloud CLI to list open issues and security findings, filtered by repository, severity, category, and status, with JSON output enabled so the next stage can parse it without a human reading a report first.

Authentication runs through an API token so the CLI executes non-interactively, which matters because this whole system is meant to run unattended.

A GitHub Actions workflow with an on: schedule cron trigger runs the pull against the default branch on whatever timetable the team sets, and a plain cron job or Kubernetes CronJob does the identical job outside GitHub's runners.

Installing the Codacy Cloud CLI Skill puts the commands and options in front of the fix-agent without hard-coding glue code that breaks the next time an API changes. What comes out of this stage is a structured list of open findings, ready for triage rather than for a human to skim.

Section 3: Triage the findings

Triage happens in two passes: strip out what isn't real, then rank what's left. Running severity sorting on a list still full of false positives just produces a well-organized list of noise, so the order matters more than it looks.

The first pass ignores findings that aren't real issues, each with a recorded reason such as FalsePositive or NotExploitable, entered through the Cloud CLI so the decision is auditable and doesn't silently resurface on the next scan.

Codacy's Smart False Positive Triage uses broader context from the surrounding code to help assess whether findings are relevant, which matters specifically for teams processing a high volume of AI-generated commits where static analysis alone tends to overflag.

The second pass sorts what remains by severity, category, and status, and for dependency findings, the SCA data carries CVE identifiers and affected-function detail that should drive which fix lands first. What's left after both passes is an ordered and actionable list, not just a shorter one.

Section 4: Record ignores and exclude irrelevant paths

Once the finding is identified as a false positive (and therefore not actionable), it can be dismissed as such. For entire files or directories that should be excluded moving forward, such as a test folder full of intentionally insecure fixtures, those paths can be excluded with an exclude_paths entry in the repository's .codacy.yaml, committed through a pull request so the change is version-controlled and reviewed like any other code change, rather than toggled in a settings panel nobody remembers exists.

Codacy uses that configuration when analyzing the repository, so the exclusion remains in effect until someone deliberately changes the configuration. Keeping the ruleset tuned matters just as much as the exclusion itself; the Configure Codacy skill turns on the tools and patterns that actually apply to a given project and turns off the ones that don't, which is what stops irrelevant findings from reappearing every time the scanner runs.

Section 5: Route work to the fix agent

Routing converts an actionable finding into a ticket that the fix agent can actually act on, and the difference between a usable ticket and a useless one is context.

Codacy's native Jira integration creates a basic ticket, but a title and severity label without a code location or remediation guidance forces the fix agent to go rediscover information the scanner already had.

The harness closes that gap by writing detailed tickets, pulling file, line, severity, description, and remediation guidance straight from the Codacy API, grouped by team and project, so no single owner ends up buried under work meant for three different squads.

Each ticket gets assigned to the fix agent directly, turning the queue into a worklist rather than a backlog someone has to triage twice. What comes out is a queue of detailed, assigned tickets that a coding agent can pick up without a human translating scanner output into a task first.

Section 6: Fix and gate the change

The fix agent (whichever coding agent the team already prefers) takes one ticket, writes the fix, and opens a pull request scoped to that single finding, because a PR covering five unrelated fixes is a PR nobody reviews carefully.

Every pull request runs through Codacy’s AI PR Reviewer, with a merge gate configured to fail the PR if it introduces new issues or if its diff coverage falls below the configured threshold. Making things easier, it’s possible to set up coverage tracking with the setup-coverage skill rather than doing it manually.

When the gate fails, the fix agent gets the chance to revise the same PR; if it can't resolve the failure on its own, it escalates to a human rather than looping indefinitely on a fix that isn't converging. A human reviewer still approves and merges every change.

The gate checks the PR but never merges it itself, and that approval is the single point in the entire loop where a person, not a script, accepts the risk of the change going live.

Section 7: Close the ticket, and optionally verify

Closing occurs after a human has approved and merged the fix. Two approaches work here, and which one a team picks depends on how much control they want over the closing logic itself.

The Jira-native approach has the fix agent put the ticket key, something like PROJ-123, in the branch name or PR title; Jira's own GitHub integration links the PR to the ticket automatically, and a Jira Automation rule transitions the ticket to Done the moment the PR merges, with no custom code required on Codacy's side.

The harness-driven approach keeps the mapping between finding, ticket, and PR inside the harness itself, which watches for the merge event and closes the ticket through the Jira API directly, useful when a team wants the close event under its own logic rather than delegated to a Jira rule.

Verification is optional but valuable for security work specifically: the harness re-runs Codacy analysis after the merge and checks through the Cloud CLI that the finding no longer appears, confirming the fix actually removed the problem rather than just closing the paperwork around it.

Section 8: Keep humans in the loop

The loop runs unattended between checkpoints, but the checkpoints themselves stay human, and that distinction is what keeps an automated backlog cleanup from becoming an unmonitored one. A few of these checkpoints matter more than others.

Merge approval stays the one non-negotiable gate: a person owns the final decision on every change that ships, regardless of how confidently the gate passed it.
Suppression review happens periodically, checking that the ignores accumulated over weeks of runs were actually the right calls and not a pattern quietly hiding something real.
Scope and prioritization decisions, meaning which repositories go first and what counts as "done" for a given vertical, stay with people who understand the business context a scanner can't see.
Guardrails on automated ignores cap how many findings a single run can dismiss, so a misconfigured rule can't mass-ignore a category of real vulnerabilities before anyone notices.

Section 9: Roll it out in stages

Rolling out the loop works best as a staged expansion rather than an org-wide flip, because proving the architecture on a small surface catches configuration mistakes before they touch every team's backlog at once.

Run the loop end-to-end on a single repository, watching every stage manually the first few times through.
Expand one team or vertical at a time, tuning the ticket routing and gate thresholds as new codebases surface edge cases the pilot didn't.
Reach a steady state, where the loop processes new findings as they appear so the backlog stops rebuilding between runs.

The target worth setting is zero open critical findings within a targeted vertical, worked through at scale rather than chased finding by finding.

Closing

Once sourcing, triage, fixing, and verification run as one continuous loop instead of four separate manual steps, a backlog that used to get pushed to next sprint becomes something a small team clears on a schedule.

The result that matters to leadership is a lower incident rate, because the findings sitting in that backlog were the risk all along, and a loop that closes them continuously helps keep that risk from compounding faster than a team can review it by hand, especially as NIST reported that CVE submissions surged 263% from 2020 to 2025.

AI Code Review Tools Compared (2026): Why Most Can't Safely Block a Merge

Codacy — Fri, 25 Sep 2026 10:17:36 +0000

TL;DR: AI code review tools compared at a glance
Comparison methodology: three questions we measure every AI code review tool against
Top AI code review tools in 2026
The three enforcement tiers and where each AI code review tool sits
Can the AI code review tool produce a merge-blocking check?
How does the AI code review tool behave when the analysis is incomplete or fails?
How to choose the right AI code review tool for your team
Where Codacy fits: deterministic enforcement as a governance layer

Most AI code review tools sitting on pull requests today cannot safely block a merge. Of the 14 tools compared here, three have no native merge-blocking mechanism at all.

That leaves engineering leaders with a familiar bad trade. Make the reviewer strict, and developers spend their week fighting a bot that fails intermittently on the same diff. Leave it advisory, and the review everyone assumed was happening at the pull request never actually enforced anything.

This article compares 14 AI code review tools, shows where each is strong and weak, and explains why AI code governance, meaning consistent enforcement at the point of change rather than a single non-deterministic check, is the model holding up in 2026.

TL;DR: AI code review tools compared at a glance

Tool	Verdict Mechanism	Can Emit a Failing Check?	Source Control Management Enforcement	Failure Behavior
Codacy	Rule/threshold	Yes	Required	Fails closed (∅ coverage fails)
SonarQube	Rule/threshold	Yes	Required	Not documented
DeepSource	Hybrid (rules + AI Review)	Yes	Required	Configurable (fails on missing data)
Semgrep	Hybrid (rules + AI-assisted)	Yes	Required	Fails closed by default
Snyk Code	Rule/threshold (AI-assisted engine)	Yes	Required	Distinct exit code (execution failure ≠ finding)
Qlty	Rule/threshold	Yes	Required	Not documented
CodeRabbit	LLM-judgment	Conditional	Required (error mode + Request Changes)	Non-blocking (inconclusive)
Greptile	LLM-judgment	Yes	Required	Not documented
Qodo	Hybrid (AI review + rule system)	Conditional	Depends on integration	Not documented
Cursor Bugbot	LLM-judgment	Conditional	Org-gated	Non-blocking (neutral)
OpenAI Codex	LLM-judgment / Hybrid	Depends on feature	Custom CI required	Distinct exit code (Security only)
GitHub Copilot code review	Advisory	No	No native mechanism	N/A
Claude Code Review	Advisory	No	No native mechanism	Non-blocking (neutral)
Gemini Code Assist Enterprise (GitHub)	Advisory	No	No native mechanism	N/A

Comparison methodology: three questions we measure every AI code review tool against

Every tool in this comparison is judged against the same three questions, asked in the same order, regardless of vendor claims:

Does it run before a commit exists, or only after the PR opens? Catching an issue pre-commit avoids the round trip of opening a PR, failing a check, and pushing a fix.
Is the verdict deterministic? The identical diff has to produce the identical result on a second run. A verdict that shifts between runs can’t be trusted to fail closed without risking false blocks.
Does it fail closed when uncertain, or let the PR through anyway? This is the difference between a control a team can actually offload responsibility to and a suggestion box that happens to live in the PR.

Top AI code review tools in 2026

There are 14 tools worth highlighting in the market. At first glance, they might seem to be solving the same problem, but how they do it matters most.

Let’s go through each tool and how they fit into your AI-assisted SDLC:

Codacy

Codacy is a code quality, application security, test coverage, and compliance platform that includes AI-assisted review alongside deterministic guardrails enforced across coding agents, editors, and pull requests.

Key capabilities:

Six configurable quality gate rules under one organization-level policy.
All six rules roll up into a single check a repository can require before merge.
The Diff Coverage rule fails closed when coverage is missing or below threshold.
The Analysis CLI and an auto-installing MCP server let agents run the same analysis locally and pre-commit.
Codacy Verity (beta): checks an agent’s output against the prompt it was given, and blocks the commit on a mismatch.

Codacy’s gate is rule-and-threshold-based rather than an LLM’s judgment call. The same diff produces the same verdict on every run — a guarantee the LLM-judgment and advisory-only tools in this comparison can’t make.

Its Diff Coverage rule also fails closed on missing coverage by default, not a setting a team has to remember to enable.

Verity, Codacy’s beta review layer for AI coding agents, is also the only mechanism in this list that blocks on prompt conformance rather than reviewing existing code, a category none of the other 13 tools occupy.

SonarQube

SonarQube is a static analysis platform that runs thousands of rules across more than 40 languages and reports a quality-gate verdict CI can enforce.

Key capabilities:

Quality Gates evaluate configurable conditions and return a pass/fail verdict that a repository can require before merge.
SonarQube for IDE performs local analysis using hundreds of language-specific rules before code is pushed.
A documented pre-commit hook blocks on detected secrets specifically, not the full rule set.
The MCP server exposes SonarQube for IDE’s local engine, letting agents invoke the same analysis directly.

SonarQube’s Quality Gate produces a repeatable, rule-based verdict. The same diff returns the same result every run. It’s one of six tools in this comparison’s rule-and-threshold tier.

The native pre-commit hook only catches secrets. Full rule-set analysis before a commit requires the separate SonarQube for IDE product, which the MCP server then exposes to agents.

DeepSource

DeepSource is a static analysis platform that pairs polyglot rule-based scanning with Autofix, one-click fixes for supported issues, and a newer AI Review layer for semantic feedback.

Key capabilities:

Issue and metric gates evaluate configurable thresholds and can be required before merge.
Can be configured to fail a check when expected analysis data does not arrive, rather than passing silently.
Autofix generates a fix for a supported issue, which a developer can turn into a pull request or commit with one click.
AI Review adds an LLM-based layer alongside the deterministic gates, rather than replacing them.
The CLI and MCP server retrieve and act on cloud-generated analysis rather than scanning a local working tree directly.

DeepSource’s core gate is deterministic: issue and metric thresholds evaluate the same way on the same diff every time. AI Review sits alongside it as a separate, non-deterministic layer, not a replacement for the gate itself.

Its CLI and MCP server work against cloud-generated analysis rather than scanning a local, uncommitted working tree directly, so an agent invoking DeepSource mid-task is retrieving a prior cloud run, not triggering a fresh local scan.

Semgrep

Semgrep is a static analysis (SAST) tool that scans code locally via its CLI, combining pattern-matching rules with Semgrep Multimodal, an AI-reasoning layer on top of its conventional static and dataflow analyses.

Key capabilities:

Semgrep Multimodal layers AI reasoning on top of the conventional pattern-matching and dataflow engine for deeper, cross-context findings.
Runs locally through the CLI and via Guardian’s MCP server, hooks, and agent skills, so issues are caught before code reaches version control.
Block mode returns a failing exit code a CI pipeline can enforce.
Fails closed on internal errors by default (exit code 2); a team can opt in to --suppress-errors to pass them through instead.

Semgrep’s core rule engine is deterministic, but Multimodal’s AI-reasoning layer isn’t, so whether a specific verdict repeats on the same diff depends on which engine produced it, not on Semgrep as a whole.

Semgrep is also one of the more thoroughly documented tools here for pre-PR use: Guardian’s MCP server, hooks, and agent skills are purpose-built to scan code before it ever reaches version control, not just to report on a prior run.

Snyk Code

Snyk Code is an AI-assisted, semantic SAST engine that scans for vulnerabilities locally and through source control management (SCM) integrations, returning a distinct result for every outcome rather than a single pass/fail signal.

Key capabilities:

Returns four distinct exit codes — no vulnerabilities found, vulnerabilities found, execution failure, and unsupported project — so a failed scan is never mistaken for a clean one.
Supports local analysis and agent invocation through Snyk’s CLI and MCP integration.
Can be enforced through SCM branch protection once a Snyk check is configured as required.

Snyk Code sits in the rule-and-threshold tier for gating purposes. A scan returns one of four defined exit codes, but the detection engine itself is AI-based and semantic, not pattern-matching, which is why we didn’t classify its verdict mechanism as purely deterministic.

That separation carries through to failure behavior: execution failure (exit code 2) is a distinct, documented outcome from both “no vulnerabilities found” (0) and “vulnerabilities found” (1), so a broken scan can’t quietly register as a clean pass.

Qlty

Qlty is a CLI-driven code quality platform that runs the same checks locally, in Git hooks, and in CI, without a separate server or agent integration layer.

Key capabilities:

Formats code at pre-commit and checks quality standards at pre-push, installed with a single git-hooks command.
A Qlty Gate, Coverage, or Diff Coverage status can be required before merge.
The CLI reads the local filesystem and writes to standard output, which Qlty says “avoids the need for a Model Context Protocol (MCP) server or API integration.”

Qlty doesn’t reject agent integrations: its CLI is designed so the same local-filesystem, stdout-based interface does the job an MCP server would, without the extra layer.

CodeRabbit

CodeRabbit is an LLM-based code review tool that reviews pull requests conversationally, plus a CLI that runs the same review technology locally before a PR exists.

Key capabilities:

Defaults to a warning posture; blocking only activates once a check is set to error mode and paired with the Request Changes workflow.
The CLI reviews uncommitted code with the same underlying review technology, though CodeRabbit states local and PR results can differ.
Agent-invocable through Claude Code and Codex plugins.

CodeRabbit’s failure behavior is one of the more clearly documented in the LLM-judgment tier: an inconclusive result doesn’t block, so a team relying on the default warning posture is trusting the review to catch issues, not to gate the merge.

Local and PR reviews use the same underlying technology but aren’t guaranteed to agree. CodeRabbit itself notes the CLI is tuned for fast developer feedback while the PR review draws on broader repository context, so a clean CLI pass isn’t a guarantee of a clean PR review.

Greptile

Greptile is an LLM-judgment code review tool that indexes a full codebase into a graph for cross-file reasoning, plus a CLI built for agent-driven review before a PR exists.

Key capabilities:

Indexes the entire codebase into a graph so a review can reason across files, not just the diff.
The CLI’s agent-oriented review workflow is built specifically to let a coding agent review its own work before opening a PR.
Can emit a status a repository can require before merge, though the exact conditions that trigger a failing status aren’t fully documented.

Independent user reviews often describe Greptile’s false-positive rate as a real cost, especially for large pull requests, though several note that it improves as the platform adapts to a team’s preferences over time.

Greptile can emit a status a repository requires before merge, but the exact conditions that flip that status to failing aren’t spelled out in the sources we reviewed.

Qodo

Qodo is a hybrid code review platform that combines AI-based review with a Rule System to define and enforce team coding standards.

Key capabilities:

The Rule System defines coding standards once and enforces them consistently across developers, reviewers, and AI agents.
Agent Skills extend that Rule System to multiple coding agent platforms, including Cursor and Windsurf.
Whether a review or rule actually blocks a merge depends on which specific feature and repository integration a team configures, rather than one fixed gate.

Qodo’s verdict mechanism is genuinely hybrid: AI-based review sits alongside configurable rules, and only some of those rules behave deterministically.

Cursor Bugbot

Cursor Bugbot is an LLM-judgment review tool built into Cursor and connected SCMs (GitHub, GitLab, Bitbucket), triggered automatically on each PR update or manually via a comment or slash command.

Key capabilities:

Runs automatically on every PR update once enabled, or on demand via a “bugbot run” comment or the /review and /review-bugbot commands.
Returns one of three conclusions: success (no issues), neutral (issues found, run canceled, or an internal error), or failure (issues found with fail-on-unresolved enabled).
Can review a branch's committed and uncommitted changes via /review-bugbot before pushing, though there's no native pre-commit hook that blocks.

Bugbot’s neutral status is doing double duty: the same conclusion covers a canceled run, a genuine internal error, and unresolved findings, so requiring the check in branch protection isn’t enough on its own to block a merge.

Getting from a required status to an actual gate takes two separate steps: making Bugbot’s check required, and then enabling fail-on-unresolved-issues where an organization’s plan allows it.

OpenAI Codex

OpenAI Codex splits code review into three separate surfaces: a PR review feature, a GitHub Action for CI, and Codex Security, a CLI-based scanner with its own severity policy and exit codes.

Key capabilities:

Codex PR review posts AI-judgment comments on a pull request but doesn’t block a merge on its own.
The Codex GitHub Action can be wired into CI to gate a pipeline on Codex’s findings.
Codex Security’s --working-tree flag scans staged and unstaged changes against a base, so it can run before a commit exists.

Codex Security’s failure behavior is precise: exit code 2 covers both a broken scan and one that merely has partial or unknown coverage, so an incomplete run can’t be mistaken for a clean pass.

That precision is scoped to Codex Security alone, though. PR review never blocks on its own, and turning any of this into an actual merge gate means wiring the GitHub Action or Codex Security into CI — none of it happens by default.

GitHub Copilot code review

GitHub Copilot code review is an AI reviewer built into GitHub that comments on a pull request automatically or on request, tuned for lightweight natural-language feedback rather than a policy gate.

Key capabilities:

Runs automatically on eligible repositories once enabled, or on demand by requesting Copilot as a reviewer on any pull request.
Leaves comments rather than a formal approval or request-changes review.
No dedicated pre-commit hook identified for GitHub’s PR reviewer.

GitHub Copilot code review sits in the same advisory tier as Claude Code Review and Gemini Code Assist Enterprise: comments and severity signals, with no native path to a required check.

Claude Code Review

Claude Code Review is Anthropic’s managed PR review service: a fleet of specialized agents analyzes a pull request against the full codebase, tags findings by severity, and posts them as inline comments.

Key capabilities:

Runs when a PR opens, on every push, or on demand via “@claude review,” depending on how a repository is configured.
Findings are tagged Important, Nit, or Pre-existing, with a verification step that checks each candidate against actual code behavior before it’s reported.
A separate local /code-review command reviews a branch’s own commits and working-tree changes, but doesn’t read a repository’s REVIEW.md the way the managed PR review does.

Claude Code Review’s check run always completes with a neutral conclusion, even after an internal error or timeout.

A team that wants Claude’s findings to actually block a merge has to parse the check run’s machine-readable severity data in its own CI. The managed review itself doesn’t gate anything.

Gemini Code Assist Enterprise on GitHub

Gemini Code Assist on GitHub is Google’s AI reviewer for pull requests, posting reviews and severity-ranked comments directly on the PR.

Key capabilities:

The consumer version of Gemini Code Assist on GitHub was discontinued on July 17, 2026; only the Enterprise tier continues.
Documents no native merge-gating status comparable to a conventional required CI check.
Has no pre-PR mechanism identified: no local, CLI, or IDE analysis surface documented for this comparison.

The three enforcement tiers and where each AI code review tool sits

Several products now combine static rules and AI reasoning into a single review flow, making the old scanner-versus-judge split harder to draw.

Every tool in this comparison still falls into one of three tiers, based on what ultimately determines whether a check passes or fails:

Tier A. Rule and threshold-based enforcement.

A deterministic rule or a numerical threshold makes the actual gate decision, even where a tool layers AI reasoning on top for parts of the broader review.

Codacy
SonarQube
DeepSource
Semgrep
Snyk Code
Qlty

Tier B. LLM-judgment review.

An AI reasons semantically across the diff and surrounding files, which is genuinely deeper analysis, but the verdict itself isn’t guaranteed to repeat on the same diff.

CodeRabbit
Greptile
Qodo
Cursor Bugbot
OpenAI Codex

Tier C. Advisory only.

The tool leaves comments and severity signals but doesn’t natively act as a blocking status.

GitHub Copilot code review
Claude Code Review
Gemini Code Assist Enterprise on GitHub

Can the AI code review tool produce a merge-blocking check?

Whether a tool can emit a merge-blocking check at all decides if its findings are enforceable or just advisory.

The tools in this comparison fall into four levels:

Can emit a status a repository can require before merge, unconditionally: Codacy, SonarQube, DeepSource, Snyk Code, Qlty, Semgrep, and Greptile.
Can block, but only once a team explicitly turns on error mode or org-level gating: CodeRabbit and Cursor Bugbot.
Depends on which specific feature or integration is wired in: OpenAI Codex and Qodo.
Have no native path to a required check: GitHub Copilot code review, Claude Code Review, and Gemini Code Assist Enterprise.

How does the AI code review tool behave when the analysis is incomplete or fails?

Behavior	Tools
Fails closed on missing data	Codacy (Diff Coverage rule fails on missing or below-threshold coverage)
Configurable to fail on missing data	DeepSource
Failure represented distinctly from a finding	Snyk Code (distinct exit codes)
Documented failure exit code	OpenAI Codex Security, Semgrep
Non-blocking on uncertainty	CodeRabbit (inconclusive results don't block), Cursor Bugbot (neutral on internal errors), Claude Code Review (errors and timeouts resolve to neutral)
Not sufficiently documented	SonarQube, Qlty, Greptile

Codacy's Diff Coverage rule is the clearest example of the behavior that makes a check safe to offload responsibility to: in a category where most tools treat an uncertain state as a pass, it fails the pull request outright when coverage is missing or below threshold.

How to choose the right AI code review tool for your team

Work through these checks before picking a tool.

Decide what you’re actually offloading. Catching subtle bugs and design flaws favors a deeper LLM-judgment reviewer. Guaranteeing a consistent standard at every merge favors a deterministic gate you can fail closed on.
Check failure behavior before strengths. A reviewer that passes when it errors out can’t be your only line of defense, regardless of how good its comments read.
Confirm where enforcement actually happens. If a tool only emits a status, budget time to configure the branch protection rule that makes it required — the tool never does that on its own.
Favor tools that can act before the commit exists if your teams work with coding agents, since that avoids the failed-PR round trip entirely.

Where Codacy fits: deterministic enforcement as a governance layer

AI is generating more code, with less scrutiny applied per change. A single non-deterministic check sitting alone at the pull request isn’t something a team can safely hand its standards over to.

Governance, in this context, means consistent enforcement at the point of change, applied identically across every repository.

It’s backed by a verdict a team can fail closed on, without second-guessing whether the next run agrees with the last.

Codacy occupies that deterministic tier. Its role is to guarantee the specific outcome a team decides matters, on every diff, without exception.

You can run Codacy alongside LLM-reviewing tools like CodeRabbit rather than competing with it. While CodeRabbit’s review stays advisory, Codacy enforces the rule that decides whether the merge actually goes through.

Fill the governance gap your AI code reviewer alone can't.

Codacy tracks quality, security, and coverage trends across every repository your team owns, so the next question your leadership team gets asked has an answer already sitting in a dashboard.

Scan your repository for free →

Agentic SDLC Loop Engineering: How Black Box Runs PR Review Gates at Scale (2026)

Codacy — Wed, 09 Sep 2026 14:23:22 +0000

TL;DR

A production-grade agentic loop routes work by task type, not by preference, and reviews plans with a model that didn't write them.
Five independent PR reviewers plus a manual merge step turn a fast loop into a governed one.
Every recurring mistake should write itself back into a rule, a lint check, or an agent instruction file, or the loop never gets safer.
Pushing feedback earlier, into hooks and local checks, is cheaper than catching it at PR review, where every round burns tokens and wall-clock time.
Git hygiene and workspace sprawl break before the models do once agent count climbs past a handful.
Verity adds an in-loop, adversarial review layer that detects and fixes quality, security and intent gaps introduced by coding agents, on every run

It's 2 a.m. and a bug fix lands in the queue, gets picked up by Codex, gets reviewed by Claude because Claude didn’t write it, gets flagged by three more automated reviewers, and sits waiting for one human to click merge. Nobody wrote a ticket. Nobody assigned a reviewer. The system just did what it was built to do.

That is a very different software development lifecycle from the one most engineering teams are used to. And once agents are responsible for more than writing code, the questions change. Who checks the checkers? What happens when several agents are working at once? And how do you know the loop is getting better instead of worse?

Now imagine having multiple agents running this loop every day. That’s where Erik Jost, Chief Digital & AI Strategist at Black Box, is operating today: roughly eight agents in the development loop, with five independent reviewers checking every pull request before Jost merges it by hand.

This is the story of how Black Box got there, what Jost has learned along the way, what breaks when you scale it past a handful of agents, and what to put in place before you try it yourself.

The Loop, End to End: How Black Box Runs the SDLC as One System

Running the SDLC as one loop means a backlog agent, a planning model, an implementation model, and a stack of reviewers all operate on the same queue without waiting for a human to hand off work between stages.

At Black Box, that queue restacks itself twice a day, bugs and features route to different models by design, and every pull request clears five reviewers before Jost merges it personally.

IntelliPact, the platform Jost runs at enterprise scale across Azure and AWS, started as forward-deployed, fast-built software and matured into this governed loop as the agents working on it became more autonomous.

The underlying belief driving the setup is straightforward: with AI code generation and the right guardrails, a small team can deliver outcomes that used to require a much larger one.

As Jost put it while walking through the setup, "I like the defensive nature of having a different layer looking at it," a line that shows up again once the review stack is broken down.

CI/CD runs on GitHub, development happens primarily in Claude Code and OpenAI Codex, and hosting sits in enterprise Azure deployments. The backlog moved from GitHub Issues to Linear because it proved more agent-friendly, and roughly eight agents run inside the loop at any given time.

How Black Box Splits Work Between Claude and Codex

Black Box routes work by task type rather than by preference, sending feature design and net-new capability work to Claude and sending bug fixes, tests, and regression work to Codex.

The split exists because the two harnesses perform differently depending on the shape of the task, and Black Box has found Claude more robust at reasoning through feature development, while Codex performs better inside a bug-fixing and testing harness.

Jost frames the choice the same way he'd frame staffing a team: you wouldn't hire a web front-end developer to build something in Rust on Apple Metal.

Delegation between models works on the same logic. Once the backlog agent marks an issue "ready for plan," it gets routed to whichever model fits the work, not whichever model happens to be open.

Why Does the Model That Didn't Write the Plan Review It?

The model that reviews a plan is never the one that wrote it, adding an independent layer of review before implementation. If Codex plans a change, Claude reviews it, and if Claude plans it, Codex reviews it, using open-source planning tooling to keep the cycle structured.

This adversarial setup puts Jost’s idea of defensive layering into practice: the model reviewing the plan is deliberately different from the one that wrote it.

Blockers that need a human judgment call, like site access or a product decision, get flagged for the team rather than pushed forward on a guess. Once the plan clears review with no blockers, the issue moves to "ready to implement," and the assigned agent writes the code and opens the PR.

The Backlog-Grooming Agent That Restacks Priorities Twice a Day

A backlog-grooming agent reviews the queue on a set cadence, aligns priorities, sets dependencies, and looks at the actual codebase before marking anything ready for a planning model to touch.

This step matters because it's what makes the Claude/Codex delegation reliable in the first place: nothing downstream works if the queue feeding it is stale or contradictory.

Jost describes the agent going in, looking at priorities, aligning things, setting dependencies, and checking the codebase before anything gets flagged ready. Without that grooming pass, agents would burn cycles planning work against outdated context.

What Do Five Stacked PR Gates Look Like Before a Human Merges?

Every pull request at Black Box passes through five independent reviewers plus standard GitHub enforcement before Jost merges it by hand.

The reviewers include Claude, Codex, GitHub Copilot, Codacy, and CodeRabbit, and GitHub's own "all conversations must be resolved" rule sits on top as a hard gate.

Reviewer	What it primarily catches
Claude / Codex (cross-model)	Logic and intent gaps the writing model missed
GitHub Copilot	General code review coverage
Codacy	Security, quality, and coding-standard violations
CodeRabbit	Additional review pass and summary
GitHub branch rules	Unresolved conversations, required checks

Agent-generated code introduces more technical debt than human-written code, which makes layered review increasingly important.

As one industry analysis of loop design put it, a probabilistic model check "should not act as the final gate," and a deterministic verification tier is what turns an open-ended loop into a bounded one.

That's the same logic Black Box applies by keeping a human as the literal last click on every merge, even inside a highly autonomous system. It's also the layered-control answer boards are starting to ask for, alongside the usual question of how much faster the team is shipping.

The Stacked PR Gates Before a Human Merges by Hand

Every repeatable issue found in a PR gets written back into the system as a new rule, whether that means a lint check, a CLI validation, or an update to the Agent.md and Claude.md files that brief every agent session.

Two meta agents, one running daily and one weekly, review the skills built across all agents and fold lessons learned back into the shared instruction set.

This is the detail that separates a durable platform from a pilot: a mistake that only gets fixed once, in one PR, teaches the system nothing.

Jost describes it directly: every mistake carries a hardening step with it. Increasingly, external tooling reviews those instruction files and prompts against best practices, creating a recursive loop where the system that governs the agents also gets governed.

Why Layered and Specialized Beats a Single Pass
For Jost, defense in depth means “multiple levels of feedback and specialized feedback.” Security, quality, and performance reviews each provide a different lens on the same change. When agents are producing most of the code, that layered approach gives the loop more than one chance to catch a problem before the final merge.

That is the governance side of the equation: how much faster agents can ship, and how many independent controls remain over what they ship.

Token Economics: Seven Rounds of PR Feedback Gets Expensive

Every round of agent-generated PR feedback burns tokens, and a change that takes seven or eight rounds to land gets expensive fast, both in spend and in wall-clock time spent waiting on reviews, pulling them down, and re-triggering hooks on every edit.

Jost is direct about it: running that many rounds "gets very expensive."

The fix is pushing feedback earlier in the cycle, into post-tool-use hooks, pre-commit hooks, and pre-push hooks that pull signal out of the CLI before a change ever reaches a PR.

Real-world numbers back up why this matters: Anthropic says Claude Code Review averages $15 to $25 per pull request, with cost scaling based on PR size, codebase complexity, and the number of issues requiring verification, and reported Uber spend on Claude Code ranged from $150 to $250 per engineer monthly, with power users reaching $500 to $2,000.

The operating principle is simple: the more corrections happen at build time, the less correction has to happen at review time. A useful health metric here is the accepted-change rate rather than raw token spend.

Two Layers of Defense: PR Review vs. Production-Log Monitoring

PR review and production-log monitoring form two distinct layers of defense once agents write most of the code, catching different failure classes at different points in the pipeline.

Layer one is the multi-lens PR review already described; layer two is an agent reading error logs across dev, test, and demo environments, feeding warnings and performance issues back into the loop.

At SaaS scale, aggregating logs across tenants surfaces macro trends that feed straight back into how the loop is engineered.

Security judgment, cloud architecture, and database design still aren't fully offloaded to agents at Black Box, and the most under-discussed weakness in loop design generally is operations during an actual outage, which is why pressure-testing and chaos engineering inside these loops deserve more attention than they currently get.

What Breaks First When Black Box Scales Past a Handful of Agents

Git and GitHub hygiene are among the first things to break as the agent count climbs.

Merge queues, rebasing, and general repository cleanliness degrade under concurrent agent load faster than most teams expect going in. Jost has run cleanup after roughly 65 abandoned local work trees accumulated, left behind by agents that finished a task and never cleaned up after themselves.

Token cost is the second pressure point, since prices climb even as usage grows, making cost optimization for the speed you want to move at a real engineering concern rather than a finance afterthought.

This visibility gap is broader than any single team: Codacy's own scanning has found traces of different coding assistants across more than 6,700 repositories from over 800 organizations, meaning most engineering leaders don't actually know how many agents are touching their codebase until they go looking.

Where Should Engineering Leaders Start with Loop Engineering?

Start by building a small, self-improving agent loop personally before applying any of it at team scale, because AI amplifies what's already there: the lessons from a personal loop transfer directly, and the mistakes are cheaper.

Jost's clearest advice on this is blunt: "You can't manage what you don't measure," which is why telemetry sits at the center of every skill turn in his system, with self-reflection and meta-reflection built in from the start.

The same loop-engineering discipline extends past code into go-to-market work, sales ops, and collateral creation, and it's worth applying there once the code loop is stable.

Harnesses move fast, so tracking release notes and new features is itself an engineering practice, not a side task, and there's no single right combination of model, tool, and harness for every team.

Where Does Codacy Verity Fit Inside the Loop?

Verity is an in-loop, adversarial review layer inside Claude Code that sits inside the loop Black Box’s kind of setup depends on, pairing deterministic checks with an independent model review to catch and fix security, quality, and intent gaps after every agent turn.

Crucially, it compounds a git-tracked, markdown-based knowledge graph of decisions, giving subsequent turns access to the context of what came before. Teams can also see the economics of agentic development with Verity's estimations of token consumption and cost per run, task, and project.

Jost turned to it specifically because it kept pace with his agents. As he put it, “I pulled CodeRabbit out of my pre-push hook because it kept timing out. The Verity responses were a lot faster, and the knowledge graph approach is huge for multi-agent workflows.” Agents running five to twenty concurrent work trees across multiple machines don’t tolerate that kind of lag.

Verity keeps its knowledge graph available both locally and in the cloud, which fits a workflow where dozens of work trees are running in parallel and each one needs the same context. The exact kind of close-to-implementation catch that gets more expensive the later it happens.

How to secure AI generated code from prompt to pentest

Codacy — Wed, 05 Aug 2026 11:04:55 +0000

We ran a session with Jordan Constantine, Head of Offensive Security at WorkNest Secure. Codacy CTO Kendrick Curtis covered what goes wrong while the code is being written; Jordan covered what he finds when he's paid to attack it afterwards.

4 vulnerability classes in AI-assisted development:

Insecure dependencies and malware. Agents are insecure by default on versions: stale training data means they pull outdated packages, and the corpus over-represents older versions because that's what people wrote examples against. Ask the LLM to remediate and it swings to bleeding edge instead, which is its own risk. Add slopsquatting to that — typosquatting, except the model makes the typo, at scale.

The single highest-return fix in the whole session: set a minimum age in your .npmrc. Most malicious packages get flagged and pulled within hours, so 3 days of insulation removes the large majority of bleeding-edge dependency risk. One config line. For the other end — known-vulnerable older versions — you need a version database, which is the part we do; Verity runs our CLI inside the agent and corrects the version before it lands.

Malicious MCP servers. An MCP server is a wrapper around an API, which means it's a middleman in your code path on the developer machine and in production. Same threat model as a malicious package: exfiltrate what's on the machine and post it out. The fix isn't banning them, it's a curated allowlist committed somewhere developers can actually find, a process for adding to it, and scoped tokens per server so the blast radius is contained when 1 turns out to be hostile.
Prompt injection. You can now hack computers in English. On the dev machine it doesn't even need executable code — a text file inside a dependency instructing the agent to read your env vars and POST them somewhere is enough, because agents can't separate instructions from data. Containment is the answer: sandbox the agent, control what crosses the boundary, keep keys in a vault and only short-lived ones in env vars.
Unbounded agent permissions. Agents execute as you, with your permissions, including dropping to a terminal. The weekly "the AI deleted my production database" post is a permissions failure, not an AI failure. Read-only if it must have prod at all, or hand it a clone and review the script it writes.

Now from the attacking side:

How guardrails actually get bypassed. Not with zero-days in the safety logic. Role-play and pretexting, indirect injection hidden inside documents, task decomposition into a chain of individually harmless steps, and spacing/encoding tricks that reassemble server-side after a grammar pass. Structurally identical to XSS and SQLi filter evasion, different surface. The bypass goes around the guardrail, not through it.

Walkthrough 1: chatbot to password hashes. Well-configured web app, almost nothing else found. They asked the customer-facing chatbot which database tables it could reach and it answered dbo.Users. It gave a count of 60,000 but withheld the rows, so they asked what parameters the backend expected, learned it wanted a user ID, supplied their own test account's ID, and got the full record — including an MD5 password hash. With user enumeration also present, they could cycle accounts and pull hashes. No payload, no exploit.
*Walkthrough 2 *: LLM document ingestion to AWS credentials. Ingestion service on AWS, so SSRF against the EC2 metadata endpoint was the obvious target. Direct requests to localhost and the metadata IP were blocked. So they pointed it at a permitted external URL that redirected to the metadata endpoint, and the LLM followed. The response never came back directly — it got vectorized into the LLM's own document store — so they asked the chatbot what it had recently ingested, and it read the AWS credentials back out, reasoning that it could only answer from data in its context. High-privilege credentials.
Treat AI like infrastructure. Jordan's summary of what most teams get wrong: AI gets the access level of a service but the governance of a feature, usually because narrowing scope slows development down. Least privilege on the agent's token, human in the loop on a defined list of actions rather than everything, and incident response that works in hours — which means accepting false positives and deciding where you sit on usability versus security before it's an incident.
One from the Q&A worth repeating. When asked which AI-generated vuln is hardest to catch in review, Kendrick's answer was missing authorization on API endpoints - no token check, or no scoping of results to the requesting user. Scanners are good at things that are there and backed by a pattern or a database. Humans and tools are both bad at spotting the absence of something that should exist.

Best CodeRabbit Alternatives for AI Code Review & Code Quality (2026)

Codacy — Tue, 04 Aug 2026 20:37:17 +0000

The best CodeRabbit alternatives for AI code review and code security fall into three categories:

Dedicated PR reviewers like Greptile and Cursor Bugbot;
Coding assistants with review capabilities like GitHub Copilot, Gemini Code Assist, and Claude Code;
Enforcement platforms like Codacy that add codebase-wide security scanning, coverage gates, and compliance evidence.

CodeRabbit, Codacy, GitHub Copilot, Gemini Code Assist, Claude Code, Greptile, and Cursor Bugbot all help developers catch issues earlier in the change. But catching issues in an open pull request is a different job from enforcing quality and security across every repository, every branch, and the code that already shipped (like Codacy does).

I'll walk you through how to compare these tools by workflow coverage, review scope, security depth, and policy enforcement.

But first, how should engineering leaders compare AI code review tools?

The most useful way to compare AI code review tools is by operating model, not feature count, because nearly every tool on the market can summarize a diff and leave a comment. What separates them is whether that feedback holds consistently across repositories, branches, the IDE, and code that was written before the tool was ever installed.

Review quality: Does it catch real issues without burying developers in low-value comments?
Workflow coverage: Does it work in the IDE, CLI, PR, Git provider, and CI/CD?
Review scope: Does it evaluate only new changes, or the entire codebase continuously?
Security depth: Does it include SAST, SCA, secrets detection, IaC, DAST, and SBOM export?
Policy enforcement: Are rules centralized at the org level, or configured per repository?
Reporting: Does it produce trend data and compliance evidence, or only PR comments?

What are the main CodeRabbit alternatives?

The main CodeRabbit alternatives split into dedicated AI reviewers, coding assistants with review features, and unified enforcement platforms, and each serves a different role in the delivery workflow. This section covers Codacy, GitHub Copilot, Gemini Code Assist, Claude Code, Greptile, and Cursor Bugbot, all of which overlap with CodeRabbit somewhere in the review cycle but diverge sharply once you look past the pull request.

Codacy

Key capabilities:

AI-assisted pull request reviews and summaries
Repository-wide static analysis and code quality scanning
SAST, SCA, secrets, IaC, DAST, and container scanning
Test coverage tracking and quality gates
Organization-wide policies and compliance reporting
AI Inventory Detection of AI models, tools and MCPs

Where CodeRabbit focuses on reviewing developer-selected code changes across the IDE, CLI, CI/CD, and pull requests, Codacy extends into continuous repository-wide scanning, so a repository that has never had a pull request touched still gets scanned for vulnerabilities, complexity, and dependency risk.

That distinction becomes most valuable over time. Pull request review evaluates code at the moment it changes, but software risk doesn’t stand still. New CVEs, vulnerable dependencies, and policy violations can emerge long after code is merged. Continuous repository scanning keeps evaluating existing repositories as those risks evolve, even when there are no active pull requests.

Codacy connects to the repository, runs static analysis, security scanning, and coverage checks against the existing branch, and surfaces what has been sitting there unaddressed, which is precisely the scenario engineering leaders describe when they inherit acquired codebases or onboard a new service team.

Codacy also adds AI-powered PR summaries, fix suggestions, and local IDE scanning with agent handoff, so the developer-facing review experience is present, but it sits on top of a static analysis and security engine rather than replacing one.

GitHub Copilot

GitHub Copilot is primarily an AI coding assistant built for code generation, editing, and explanation, with AI-assisted pull request review as an added capability rather than its central purpose. Reviewing PRs is one feature inside a broader coding assistant rather than its primary focus.

Key capabilities:

AI code generation and editing
Chat and code explanation
AI-assisted pull request review
IDE integrations across Visual Studio Code, Visual Studio, JetBrains, and Neovim
GitHub-native development workflows

Organization-wide application security and policy enforcement live in separate GitHub products such as GitHub Advanced Security, not in Copilot itself, so teams evaluating Copilot for review purposes are really evaluating a convenience feature bundled with a developer assistant they likely already pay for.

Gemini Code Assist and Gemini CLI

Gemini Code Assist combines code generation, chat, and AI-assisted pull request review, making it Google's developer assistant rather than a dedicated code quality governance platform.

Key capabilities:

AI code generation and completion
Chat-based development assistance
AI pull request reviews
Gemini CLI for terminal-based workflows
Integration with Google Cloud and GitHub

Gemini CLI participates in agentic development loops, and CodeRabbit's own CLI integrates seamlessly with AI coding agents like Claude Code, Cursor CLI, and Gemini to review code as it's generated, before it ever reaches a pull request.

That framing is the right way to think about Gemini's role here: it matters wherever a team is using an agent to produce or modify code and needs an independent review step around that output, not as a standalone AppSec or governance layer.

Claude Code and Claude Code Review

Anthropic's Claude Code is an agentic coding assistant that runs from the terminal, designed to help developers generate, modify, and understand code directly in that environment.

Key capabilities:

Terminal-based coding agent
Local /code-review workflow
Multi-agent pull request review (Claude Code Review)
Repository-aware code understanding
Integration with CodeRabbit review workflows

Developers can run a local /code-review command before opening a pull request, and CodeRabbit's plugin for Claude Code creates autonomous AI development workflows where Claude Code can trigger CodeRabbit reviews directly through simple commands.

Claude Code Review extends that local workflow to GitHub pull requests using multiple specialized agents that analyze proposed changes against the surrounding codebase, prioritizing production-impacting issues like logic errors and security vulnerabilities over style nitpicks.

That combination makes Claude Code a genuine review alternative to CodeRabbit rather than only a generation tool, though its focus stays on the change in front of it rather than enforcing quality or security policy across every repository a team owns.

Greptile

Greptile is a dedicated AI code review agent focused on pull request analysis rather than code generation. Parallel agents review the changed code and post inline comments, and the tool adjusts what it flags over time as engineers approve or reject its suggestions, applying team-specific rules to shape what gets surfaced on future PRs.

Key capabilities:

AI pull request reviews
Repository indexing for broader context
Parallel review agents
Learns from developer feedback and reactions
Team-specific review rules

Cursor Bugbot

Cursor Bugbot is a dedicated AI reviewer built by the team behind the Cursor editor, aimed narrowly at catching real bugs and security issues inside a pull request rather than summarizing the change.

Key capabilities:

AI bug-focused pull request reviews
Inline GitHub comments
Automatic re-review on every push
Optional Autofix workflow
Designed for Cursor-based development teams

It posts inline comments directly on the diff in the Git host and re-checks the PR on every push, with an optional Autofix step that runs cloud agents to test changes and propose fixes on the PR itself.

It fits tightest for teams already living inside the Cursor ecosystem, but like Greptile, it finds and helps fix problems in the change without providing codebase-wide AppSec, coverage, or org-wide policy.

Tool	Primary role	Codebase-wide scanning	Security depth	Platform reach
CodeRabbit	AI PR reviewer	No (PR-focused)	AI review + integrated static analyzers	GitHub, GitLab, Bitbucket, Azure DevOps
Codacy	Quality, security, coverage, and AI governance platform	Yes	SAST, SCA, secrets, DAST, IaC, containers	GitHub, GitLab, Bitbucket
GitHub Copilot	Code generation + basic PR review	No	None dedicated	GitHub
Gemini Code Assist	Code generation, chat, PR review	No	None dedicated	GitHub
Claude Code / Code Review	Coding agent + multi-agent PR reviewer	No	None dedicated	GitHub
Greptile	AI PR reviewer with codebase indexing	Partial (indexed for context)	None dedicated	GitHub, GitLab
Cursor Bugbot	AI PR bug hunter	No	None dedicated	GitHub

Skill to unblock Pull Requests with one prompt (Tutorial)

Codacy — Mon, 27 Jul 2026 20:21:11 +0000

Now that coding agents have multiplied how much code gets written, code review has become the bottleneck for teams adopting agentic workflows.

With Codacy Skills, there's a new way to let coding agents like Claude handle that gruntwork to unblock pull requests faster, configure Codacy rules and settings, and even perform the code analysis locally pre-commit.

At its core, Codacy Skills teach coding agents to use the Codacy Cloud CLI and Analysis CLI to address a range of powerful (dare I say ‘revolutionary’?) use cases.

Installing the Codacy Cloud CLI and Skills

The Codacy Cloud CLI (@codacy/codacy-cloud-cli) brings Codacy data to the terminal: issues, security vulnerabilities, pull request analysis results, coverage metrics, configured tools and patterns, across GitHub, GitLab, and Bitbucket. The output is a table by default, or JSON when you want to pipe it somewhere.

It’s open-source and can be installed via npm. Once installed, log in to connect your codacy.com account.

npm install -g @codacy/codacy-cloud-cli

codacy login

Here's what you'll see in the terminal:

The Codacy Cloud CLI Skill instructs coding agents how to use the Codacy Cloud CLI. They work with Claude Code, OpenAI Codex, GitHub Copilot, and Gemini CLI through the Agent Skills standard.

Note: You can use the Codacy Cloud CLI with or without Codacy Skills installed. If you want to use the Cloud CLI manually, see our documentation for detailed instructions, commands and workflow examples. You can also embed the Codacy Cloud CLI as part of your CI environment for advanced workflow automations.

Use the snippet below to add the marketplace and install the Codacy Skills plugin for Claude (see instructions for other agents here):

claude plugin marketplace add codacy/codacy-skills

claude plugin install codacy-skills@codacy

Once installed successfully, you will see this:

Both the Codacy Cloud CLI and Codacy Skills are free to install on every Codacy plan. What the agent can act on still follows the Codacy features available on your plan (see our pricing page for more details)

You can now perform basic operations like adding your repositories to Codacy.

Now, let's get to the interesting bit. Below are our three favorite ways to use the codacy-cloud-cli skill like a pro.

Use case 1: Clearing a blocked PR in one prompt

Here is the case we built this for: a PR is failing the merge check. You can set up to six criteria to trigger your Codacy PR gate:

number of new issues introduced
number of security issues introduced
hitting the complexity threshold
hitting the duplication threshold
insufficient diff coverage (percentage of changed lines of code that are covered by tests)
overall test coverage drops.

Whichever is triggered, the PR comes back red. Normally you would open each finding, fix it, write the missing test, and re-run the analysis.

With the codacy-cloud-cli skill installed, you can hand the whole thing to Claude Code in one prompt:

Pull request 42 is failing the Codacy gate. Fix what's real, add the tests it needs, ignore the false positives with a reason, then re-run the scan.

The agent pulls the analysis and reads back everything that's blocking the gate, using the pull-request subcommand to return the annotated diff with new issues inline and uncovered lines marked.

codacy pull-request gh my-org my-repo 42 --diff

From there the agent fixes the genuine issues (applying Codacy's suggested fix where there is one, writing the rest itself), writes missing tests, refactors any duplicated blocks, dismisses the confirmed false positives with a logged reason, and re-runs the analysis to confirm the gate is green.

Quick disclaimer: The PR check may not go green on the first pass, but instead of scattering your code review and remediation efforts between agent, terminal, IDE and browser tabs, you get to triage your scan results quickly from a single place, in bulk, and against your existing coding standards.

Use case 2: Do a security sweep across the repo

The same pattern scales past a single pull request. Point the agent at the backlog:

Fix the critical and high security findings in this repo.

Your agent runs the lookup and reads the findings with their severity and CVE context, using:

codacy findings gh my-org my-repo --severities Critical,High

Findings can also be filtered by scan type (SAST, Secrets, SCA, IaC), status (Overdue, Due soon, On track). This allows you to pull detailed vulnerability reports and instant, scoped-out security audits.

Use case 3: Triage false positives, in bulk

False positives are where teams reviewing high volumes of AI-generated code lose the most time. Instead of dismissing them one by one in the UI, the agent clears them in a single command, with a reason attached:

Ignore all issues that are flagged as false positives and tag each with an ignore reason

Codacy flags which findings are likely false positives, so the agent acts on Codacy’s data rather than guessing. Here's the Cloud CLI command it uses:

codacy pull-request gh my-org my-repo 42

--ignore-all-false-positives

The reasons it attaches are logged, and become a record of why each call was made, which is useful feedback the next time you tune the repo's rule configuration (we’ll talk more about configuring Codacy rules in Part 2 of our Skills series, so stay tuned).

In the example below, this one prompt helped us reduce the issue density from 18.07 to 12.21 issues/kLOC.

One practical consequence (for your wallet)

Let me be precise about the division of labor: Codacy does not edit your code. Your agent does, using Codacy as the source of truth for what needs attention and as the check that the change actually worked.

One practical consequence: the analysis itself does not consume AI tokens.

Codacy's pull request review, including the AI Reviewer layer, is already included in every paid plan at a flat per-seat price. Your agent only consumes tokens for the edits you ask for, and nothing on the analysis underneath.

To get started, install the CLI and the skills from the Codacy Cloud CLI documentation, or read the source on GitHub.

Applications Open: A Fellowship Program for Open Source Developers

Heloisa Moraes — Wed, 20 Sep 2023 10:14:48 +0000

Here at Codacy, we recognize the importance of the open-source software (OSS) community and are dedicated to nurturing and supporting it in any way we can.

Throughout the history of our company and product, OSS has been at the heart of what we do and how we provide value. lifts the whole world and flattens mountains of knowledge disparity.

That’s why we’ve launched Codacy Pioneers—a fellowship program to fund, promote, and mentor innovative and creative OSS projects worldwide.

Supporting builders is part of our DNA. Through our platform, we help hundreds of thousands of developers with software insights. With Pioneers, we wanted to find new ways to help people build the future.

Projects selected to participate in the program will receive a year-long monthly stipend, free access to the tools they need to work, widespread promotion, and mentorship from some of the brightest minds of today’s OSS community.

We’ve assembled an incredible roster of open-source superstars to mentor and guide our Pioneers on their journey, including:

Vue framework creator Evan You
Enix co-founder Jérôme Petazzoni
Prisma founder Johannes Schickling
CHAOSS community lead Ruth Ikegah
Nakazawa Tech CEO Christoph Nakazawa

With Pioneers, we want to inspire talented builders and tell their stories. We want to remove friction from the process of inventing the future.

Do you have an open-source project you’re incredibly proud of? If so, we’d love to hear from you!

Applications for the Pioneers program are open as of today and close on September 29.

Pioneers will be selected no later than October 16, 2023, by our mentors and engineers.

To apply for the program, head to our Pioneers page and fill out a quick questionnaire to inform us about your project.

This type of initiative is very new to us. And while we don’t know what to expect, we know that helping builders is the right thing to do.

Leave us a comment if you have any questions.

Best of luck to all potential Pioneers!

Code reviews in large-scale projects: best practices for managers

Heloisa Moraes — Tue, 31 Jan 2023 13:29:11 +0000

Managing code reviews for large-scale projects can be challenging, as the volume and complexity of the code might seem overwhelming. However, code reviews are essential for guaranteeing the quality and maintainability of the codebase, as well as identifying and addressing performance bottlenecks.

Today we’ll cover some best practices your team should follow when performing code reviews on large-scale projects.

Divide and conquer

Break the codebase into smaller, manageable chunks and assign specific sections to different team members for review. This will make the process more manageable and ensure you don’t overlook areas of the code. There are several ways you can divide the codebase into manageable chunks:

Functionality-based: Divide the codebase based on the different functionalities or features of the project. For example, you can divide a codebase into the frontend, backend, and database sections.
Modularity-based: Divide the codebase based on the different modules or components of the project. This approach can be especially useful for projects with complex codebases.
Team-based: Assign different sections of the codebase to different teams or team members. This way, they’ll view all areas of the codebase, and no one team member will be overwhelmed.

Prioritize high-risk areas

Identify the areas of the codebase that are most critical to the project's success and focus on these areas first. This will ensure you identify and address potential issues as soon as possible, reducing the risk of delays or setbacks. Here are some ways to identify high-risk areas in a codebase:

Identify areas of the codebase that are most frequently changed or updated. These areas may be more prone to errors and should be reviewed more frequently.
Look for areas of the codebase that are critical to the functionality or performance of the project. For example, you should consider high-risk the code responsible for handling user authentication or sensitive data.
Identify areas of the codebase that are new or untested. New code is more prone to errors, and untested code may contain hidden bugs.
Analyze the codebase using automated tools like Codacy Quality for static analysis, which can help identify areas of the codebase that are most likely to contain errors or bugs.

Once you have identified high-risk areas, prioritize these areas for review, focusing on them before moving on to lower-risk areas. However, prioritizing high-risk areas does not mean neglecting the rest of the codebase but rather focusing on the most critical areas first.

Automate where possible

Automating certain aspects of the review process can make the process more efficient and effective. For example, automated tools can help to identify potential issues in the codebase and standardize the review process, making it more consistent and objective.

With Codacy Quality, your team can make sure the codebase adherence to coding conventions, such as style guidelines. They can also identify potential issues such as bugs, security vulnerabilities, or performance bottlenecks, guarantee code coverage and tackle technical debt.

By automating certain aspects of the code review process, you can make the process more efficient and effective and also standardize the review process.

Communicate effectively

Effective communication can foster a collaborative and productive work environment and make the code review process smoother. Here are some ways to effectively communicate when conducting code reviews:

Set clear guidelines: Establish guidelines for what your team should review and how they should report and track issues. This can help your team understand the expectations and participate effectively.
Make sure everyone understands their roles: Clearly communicate the roles and responsibilities of all team members involved in the code review process. Make sure everyone understands their role and how it contributes to the overall process.
Encourage open communication: Create a culture of transparency and feedback by encouraging open communication between team members. This ensures that any issues or concerns are identified and addressed quickly and can also help to promote team learning and development.
Use code review tools: Your code review tools should support communication and collaboration, like comments and annotations. This will allow your team to have an organized way to communicate and track the progress of the review process.
Provide feedback and recognition: Regularly provide feedback and recognition for the team member’s efforts and contributions, and help them understand their work's impact on the project.

Review regularly

Schedule regular code reviews throughout the development process rather than waiting until the end. This will help identify issues early on and allow for timely course correction, reducing the risk of delays. Here are some ways to conduct regular code reviews:

Establish a schedule: Set up a schedule for code reviews, such as daily, weekly, or after each Sprint. This will help ensure all code is reviewed promptly and any issues are identified and addressed as soon as possible.
Make it a part of the development process: Incorporate code reviews into the development process, such as having developers review their own code before submitting it.
Use a pull-based review process: With a Pull Request review process, developers submit their code for review before they merge it into the main branch. This allows multiple sets of eyes to review the code and can help to identify issues early on.
Encourage CI/CD practices: Automating the testing and deployment of code change, along with regular reviews, allows your team to catch issues early and fix them quickly before they become bigger problems.

Keep in mind that reviewing code regularly throughout the development process also helps to promote a culture of quality and accountability.

Rotate the reviewers

Rotating the reviewers allows all the team members to be familiar with the different parts of the codebase and can provide valuable feedback. It also avoids bias in the review process, as different reviewers may have different perspectives and can identify issues others may have missed. Here are some ways to rotate the reviewers:

Assign different reviewers for different sections of the code: This way, all team members are familiar with different parts of the codebase.
Rotate reviewers for different projects: All team members will be familiar with the codebase of different projects and can provide valuable feedback.
Rotate reviewers for different team members' code: By doing this, all team members will be familiar with the code written by their colleagues and understand their coding style and best practices.

Focus on the learning

Code reviews are not only about finding bugs but also an opportunity for the team members to learn from each other. It's an excellent way to learn new coding practices, best practices, and problem-solving approaches. Here are some ways to focus on the learning during code reviews:

Encourage sharing knowledge: Encourage your team to share their knowledge and best practices during the review process. This can help to promote learning and development among team members.
Use code reviews as a mentoring opportunity: Code reviews are excellent for mentoring junior developers and providing constructive feedback on improving their code.
Promote collaboration: Encourage your team to work together during the review process and share their thoughts and ideas.
Use code reviews to promote best practices: Encourage your team to write readable and maintainable code and to follow best practices.
Use code reviews as a way to learn new technologies: Use code reviews to learn new technologies and approaches and encourage your team to experiment with new tools and technologies.

🎥 [Webinar] Level Up Your Team’s Code Reviews

Join our Engineering Manager Kendrick Curtis and our Lead Account Executive Mark Raihlin in a discussion about helping your team perform their best code reviews.

Join us in our Webinar Level Up Your Team’s Code Reviews. See you there!

👉 When: February 2nd, 2023, 4 pm UTC / 10 am CDT
👉 Where: online – registration needed

Save me a seat

What NOT to do as a developer

Heloisa Moraes — Wed, 25 Jan 2023 10:40:00 +0000

The following views were expressed by a member of our Codacy Community

Before I came to the industry side of the world, I was an academic. I was focused on computer science research (with a special love for software quality) and on teaching students at the university.

From my years in academia, I was able to determine patterns in students when they were writing code. I saw the same patterns from junior to senior developers at different companies. Some of those patterns might be hindering your performance, so let’s take a look at what not to do as a developer:

Jumping into programming right away. Before writing your first lines of code, think about the problem and the proper solution. It might help brainstorm on a piece of paper!
Not updating your knowledge. Programming languages, frameworks, technologies, and techniques change every day. So you need to keep yourself updated and never stop learning.
Forgetting edge cases. Even if your code seems to be working and you added some tests, don’t forget to check for edge cases. It depends on the solution you are working on, but it might be negative numbers, empty strings, different input types, and everything else in between.
Not checking performance and complexity. Most of the time, there are better and simpler ways to go about a particular solution and piece of code. So always check the complexity of your code and if you can improve it. I discussed the Big-O notation in a previous post, so give it a read.
Not reviewing your code. Always review your code, both manually and automatically. With code reviews, your team can find issues early on, share knowledge, distribute ownership, and standardize development practices, among many other benefits. Plus, static code analysis tools like Codacy Quality can effectively reduce heavy lifting and tedious parts of the code review process.

Free to share other things developers should not do 👇

The true impact of technical debt

Heloisa Moraes — Thu, 05 Jan 2023 17:15:12 +0000

Technical debt happens in all software projects, regardless of the programming languages, frameworks, or methodologies. It is a common metaphor for software’s accumulated backlog (like bugs, missing documentation, and legacy code).

There are different types of technical debt, and while some are beneficial to achieve a goal, you’ll need to pay that debt off sooner or later. To help in that process, we shared ways your team can start tackling technical debt while delivering value at the end of each Sprint.

If not addressed, technical debt will compound and cause greater issues in your software. It will impact not only the customer experience but also your team and your business. Today, we’ll dive deeper into the true impact of technical debt.

What is the true impact of technical debt?

The impact of accumulated technical debt extends broadly across different parts of an organization’s ecosystem. It can have serious consequences, like losing customers, making a product more vulnerable to security risks, increasing development costs, and negatively impacting your team’s morale.

Technical debt impacts your code

Increased number of bugs: Defects, issues, and bugs are typical consequences of technical debt. These bugs also impact the product, raising maintenance costs and affecting the customer experience.
Increased complexity: When developers rush to meet a deadline, they may not develop clean and well-organized code. This will make the code more complex than it needs to be, making it more challenging to maintain and more prone to bugs and other issues.
Decreased code quality: Poor design and development practices might allow developers to finish their work in time for tight deadlines. However, they also reduce code quality and create long-run issues. Low code quality will also make it difficult for developers to maintain the codebase, directly impacting the product.

Technical debt impacts your product

Decreased usability and functionality: Some features might fail to work correctly due to technical debt, impacting usability, customer satisfaction, and reputation. It can also increase the downtime of your product.
Increased development cycles: Technical debt can also lead to delays in the development process, as developers may need to spend more time fixing issues and addressing other aspects of the codebase. This can impact the ability to meet deadlines, slow down the build process, and delay the release of new features and updates to the market. In addition, the delays impact customers and teams relying on delivery, like marketing and sales.
Increased security problems: Poorly written code is often poorly secured, which can introduce unexpected vulnerabilities and increase the risk of serious events, such as data security breaches.

Technical debt impacts your team

Decreased development speed: Since developers need to spend time on technical debt, working on and delivering new features takes longer.

Decreased team morale: As technical debt accumulates, it becomes more difficult to understand the codebase and the amount of work needed to add a new feature. As such, technical debt can also lower your team’s productivity considerably. Plus, the team’s morale decreases as the team spends more time fixing preventable problems.
Decreased retention: Unhappy or demotivated developers are more likely to resign, impacting your team’s organization, workload, and overall performance.

Technical debt impacts your business

Decreased customer satisfaction: Technical debt almost always negatively correlates with customer satisfaction due to poor user experience. This translates into an increased expense due to more tickets and customer support and loss of sales and renewals due to lower client retention.
Decreased innovation: When technical debt is severe, developers must continue servicing the issues instead of devoting time to building innovative new features with business value. The team will not be able to adapt quickly to opportunities or changes in the market.
Increased costs: Technical debt can increase the cost of maintaining and improving the codebase over time. This can include the cost of fixing issues and improving performance and the cost of lost productivity due to delays. There are also customer support costs related to clients not being satisfied with the product.
Decreased revenue: Customer dissatisfaction due to technical debt means less client retention and inefficient marketing spending.
Brand and reputation: Ultimately, if technical debt is severe, it can affect the brand and reputation and undermine the business's survival.

Codacy Quality: the best solution for tackling technical debt

We run a DevOps Intelligence Platform that helps thousands of developers ship billions of lines of code per day by automating and standardizing code reviews. We’ve built a suite of products that help developers quantify and act on their software quality, engineering performance, and security.

Codacy Quality is the best-in-class solution for static code analysis. Integrating seamlessly into your workflows, Codacy Quality helps engineering teams save time in code reviews and tackle technical debt. Quality supports 40+ programming languages and is available in free open-source and enterprise versions.

If you’re looking for a static analysis tool that allows you to check your code quality and keep track of your technical debt, start a free 14-day trial of Codacy Quality today.

Dashboard of Codacy Quality

3 popular Python style guides that will help your team write better code

Heloisa Moraes — Wed, 28 Dec 2022 17:04:14 +0000

A code style guide is a set of rules, standards, or best practices that outline how your team should write, format, and organize the source code. In an ideal world, your team’s source code should look like it was written by a single person, even if hundreds of developers collaborated on it.

Being consistent with a style guide makes your code easier to read, debug, and maintain. It is also smoother to add new features or update legacy code, and new developers have an easier integration with the team.

Many programming languages have more than one recommended style guide, and Python is no exception. To help you get started, here are 3 of the most popular style guides for Python that will help your team write code in a consistent way and improve your coding standards.

PEP 8 – Style Guide for Python Code

Python has an excellent style guide called PEP 8 that covers most situations you and your team will find while writing Python. You can use it with PEP 257, which focuses on semantics and conventions associated with Python docstrings.

PEP 8 presents guidelines on naming Python objects, how to structure your code, when to include comments and whitespaces, and some general programming recommendations. This style guide will allow you to have consistency in your Python code, making it easier for everyone in your teams to understand and contribute to the codebase.

However, PEP 8 is a generic Python guideline rather than strict rules that you should follow since it allows different approaches to achieve similar goals. So you’ll need to make sure that everyone in your team follows the same approach in the first place.

The Hitchhiker’s Guide to Python

If you are looking for a community-driven style guide, The Hitchhiker’s Guide to Python is the one for you. However, remember that it might not always be updated and accurate since it depends on the community who writes it.

This style guide aims to give novice and expert Python developers a best practice handbook for installing, configuring, and using Python. It includes recommendations for structuring your project and guidelines for code style, documentation, testing, logging, and more.

Google Python Style Guide

The Google Python Style Guide defines the coding standards used at Google. This style guide has two parts, one focusing on language rules (conventions and coding standards) and the other on style rules (aesthetic formatting issues). It’s a great guide if you have a big team with many developers actively working on a large codebase.

There is also a formatter for Python files called yapf that your team can use to avoid arguing over formatting conventions. Plus, Google also provides a settings file for Vim, noting that the default settings should be enough if you're using Emacs.

Conclusion

When you’re working in a large team with other developers, everyone will have their style of writing code because no two developers write code the same way. If you’ve got different developers writing code differently, your codebase will be almost impossible to understand. Also, onboarding new developers might be harder, as they won’t know the best approach to use.

This is where style guides come to the rescue. A style guide is a set of rules that developers follow when writing their code to ensure consistency among developers. In this article, we’ve covered 3 Python style guides your team can use in your Python projects.

Share your favorite Python style guide with us, and let’s keep the conversation going in our community.

7 best practices for writing great software tests

Heloisa Moraes — Tue, 27 Dec 2022 11:33:25 +0000

An important metric of code quality is how much of your codebase is covered by tests, as we saw in a previous article about code coverage. However, having a good code coverage score isn’t enough.

The quality of your tests influences their effectiveness and can significantly affect the quality of your product. Today, we’ll cover 7 best practices for writing great software tests. Those practices help you get the most out of your testing strategy, so keep reading.

What makes a great software test?

Simple and short: you should be able to understand and execute the tests;
Goal-oriented: focusing on a single output gives clear purpose and meaning to the test;
Consistent: all the tests should be consistent regarding the terms used;
Valid: you should align the tests with the acceptance criteria defined for the requirements;
Maintainable: tests should be easy to maintain and update when requirements change;
Discoverable: you should be able to search and find a test using multiple criteria easily;
Fast: you should be able to execute a test as quickly as possible to reduce effort;

Best practices for writing great tests

#1 – Treat your tests with the same importance as your code

Your tests require the same care and attention you give the rest of your codebase. Like any other part of your code, tests should be regularly evaluated, maintained, and improved.

You should also use comments in your tests and have a defined code review process. Tests take development time, so they should be planned, and you should design them to support and enhance your business' critical infrastructure.

Finally, it would help if you kept your tests organized, as it allows avoiding repetitions. Get yourself and your team familiar with patterns for modularizing test cases.

#2 – Make your tests independent of each other

Tests should never depend on each other. For example, if your tests have to be run in a specific order because some tests depend on others, you need to change your tests to avoid those dependencies.

If, in your system, it’s unavoidable to have some tests dependent on others, try to isolate them as much as possible through pre-conditions. However, this is just in particular cases, and you should avoid it as much as possible.

#3 – Your tests should work individually or in parallel

It’s only natural that your test suite grows along with your codebase. When your test suite gets larger, writing tests you can run individually or in parallel on multiple machines really pays off.

This approach allows you to test only the parts of your code that you or your team changed. You can also better distribute testing resources across multiple machines, which helps you to get results faster. Finally, tests that can work individually or in parallel also mean that you can refactor them without fearing that change will affect other tests.

#4 – Refactor your tests regularly

As with any part of your codebase, you should regularly review and refactor your tests. As such, plan to survey your test suite from time to time to ensure your tests remain relevant and valuable.

When you change, add, or remove functions to your code, verify if the tests still make sense. Usually, with these changes in the code, the data passed into functions also changes, and your tests need to reflect this modification.

#5 – Minimize waits as much as possible

Tests that require “waiting” will likely cause more problems as services interact differently on various machines. In addition, if you run tests often, adding waits to a test suite can make it very slow.

If you’re testing asynchronous code, it sometimes appears to require wait or sleep steps. However, it would be best if you tried to incorporate a library designed to handle asynchronous waits in your programming language.

#6 – Test across the boundaries of your system

Your test suite should always test both sides of a given boundary. Imagine the piece of code you will test as a rectangle. You’ll need to test what happens to points inside and outside the rectangle and at the boundaries. These boundaries are where your code might fail.

If your code depends on external services, don’t forget to prevent problems by testing the boundary layers. This anticipation will allow you to have some control over code you might not have written and have no access to.

#7 – Keep track of your code coverage

Tools that keep track of code coverage, like Codacy Quality, can tell you exactly which parts of your code are covered by tests. This will help you direct your efforts toward pieces of code that lack tests.

To better enforce tests, you can also define thresholds where Pull Requests are not accepted if code coverage is below a specific percentage. Code coverage tools also allow you to track coverage over time, so you can report on and measure the results of your efforts.

[Live Talk] Take Your Code Coverage to the Next Level

If you missed our webinar, watch the recording to learn how you can leverage your code coverage. See how our engineers apply it in their day-to-day.