Verification, QA and closing the loop for better outcomes in Agentic Software Engineering

We're all doing agentic software engineering now—or at least that's what the cool kids keep telling me.

But how do we make it effective? How do we give agents enough access to plan, execute, and verify their work safely?

Let's look at how we can close the loop:

Plan → implement → test → inspect failures → fix → verify in CI → monitor the result.

Quality gates

Quality gates are more important than ever because LLMs never have complete knowledge of a codebase.

During a session, an agent reads the code it considers relevant. Recent models are very capable of finding useful context, but they cannot inspect everything. Context windows are finite, so gathering information before implementing a change is necessarily a best-effort process.

As engineers, we have used quality gates and continuous integration for decades. We should apply the same standards to work produced by agents. This will be a long list, but bear with me:

• Formatting, linting, type checking, and production builds
• Static analysis for code quality and security, using tools such as Sonar
• Unit, integration, and browser-based end-to-end tests
• Dependency and license-compliance checks
• Architecture-rule enforcement

During fast agent iterations, not every check needs to run locally after every change. The right balance depends on the repository and workflow.

However, all required checks should run in CI and block pull requests when they fail. More importantly, the agent should remain in the loop: wait for the results, inspect failures, fix them, and repeat until the checks pass.

A CI pipeline that reports failures only to a human has not fully closed the loop.

Agent QA

Agents should also be able to perform exploratory QA on a pull request.

Modern agents can operate browsers effectively. Good accessibility and semantic markup make frontend testing considerably easier. Highly complex interfaces may still require human judgment, but agents can cover a meaningful portion of common user flows.

You can let agents reuse an active Chrome session through Chrome DevTools MCP. In Chrome 144 or later, enable remote debugging at chrome://inspect/#remote-debugging and configure the MCP server with --autoConnect. Chrome asks for permission whenever the server requests a debugging session.

Browser testing is only one layer. Giving agents access to an application at the API level can be even more valuable.

If an application exposes an OpenAPI specification, tools such as mcp2cli can turn that API into a CLI the agent can use. You can then create a project-specific skill that explains how to authenticate, start the local application, exercise endpoints, and verify the results.

The same principle applies to other parts of the local stack:

• Message queues
• SQL databases
• Background workers
• Object storage
• Email-testing services
• Local observability tools

This access should be scoped and should use sanitized, non-sensitive data. Done safely, it gives agents far more ability to test implementations and debug failures across system boundaries.

I don't even want to get into how many use cases this unlocks—or how much we handicap our agents when we don't provide this access. Today's models are incredibly capable of testing and debugging across the stack when they have the right tools and context.

Logs and deployment visibility

Make life easier for both yourself and your agents by giving them access to logs.

Many logging and cloud platforms can be connected through APIs, CLIs, custom tools, or MCP servers. Read-only access is usually enough for an agent to inspect:

• Application and infrastructure logs
• Deployment status
• Build failures
• Function errors
• Traces and metrics
• Recent configuration changes

For example, an agent working with AWS could assume a narrowly scoped, read-only IAM role.

The important principle is least privilege: provide read-only access by default, restrict it to the relevant environments, avoid exposing production secrets, and maintain an audit trail where possible.

Without logs, an agent can only guess why a deployment failed. With logs, it can investigate.

Automated PR review and security scanning

Pull requests should receive automated review from an agent with enough repository context and a model capable of understanding the code.

Model quality matters. A weak or poorly configured reviewer may produce too many false positives, stylistic preferences, and non-blocking nice-to-haves. That creates noise and can trap the workflow in an unproductive loop.

Automated review works best when:

• The reviewer understands repository-specific conventions
• Findings are tied to concrete code and observable risks
• Blocking and non-blocking feedback are clearly separated
• Security scanners complement, rather than replace, reasoning-based review
• The implementation agent can receive and address actionable findings

The goal is not to maximize the number of comments. It is to find issues that materially improve the change.

MCP gateways and agent plugins

Agents become more useful when they can reach the development tools already used by the team.

Connections can be packaged as Codex or Claude Code plugins, exposed through MCP servers, or consolidated behind an MCP gateway.

Services such as Composio can provide access to multiple tools through one integration layer.

Whatever mechanism you choose, access should be deliberate:

• Grant only the permissions required for the task
• Prefer read-only access until write access is necessary
• Separate development, staging, and production credentials
• Require approval for consequential actions
• Log what the agent does

The objective is not to give an agent unrestricted access. It is to give it the smallest useful set of capabilities needed to complete and verify its work.

Automated loops

The next stage is creating recurring automation loops.

These can be set up through Codex automations, Claude routines, Hermes Agent cron jobs, CI workflows, or other schedulers. I plan to cover implementation patterns and use cases in a follow-up post.

Some useful examples include:

• Daily pull-request brief: summarizes open PRs, blockers, requested reviews, and failing checks
• PR babysitting: monitors CI and review feedback, then proposes or applies fixes
• Dependency updates: validates Dependabot PRs and merges low-risk updates when all checks and policies pass
• Flaky-test detection: identifies recurring failures, reproduces them, and proposes or implements a fix
• Feature-flag cleanup: finds flags that have been fully enabled in production and creates PRs to remove them when they are no longer needed
• Code-quality maintenance: finds duplicated, overly complex, or dead code and creates small refactoring PRs
• Documentation-drift detection: notices when code and documentation disagree and proposes corrections

These loops are most useful when their success conditions are explicit. Each loop needs a trigger, bounded permissions, quality gates, an escalation path, and a clear definition of "done."

Conclusion

Agents can do a lot of useful work when we give them the tools to inspect and verify it.

That does not mean giving AI unrestricted access to every system. Permissions should reflect the sensitivity of the data, the environment, and the consequences of each action. Be smart about it and adapt the approach to your context and constraints. 😇

Human attention is too valuable to spend on every repetitive engineering task. Agents can handle much of the boilerplate and routine maintenance—provided that we give them safe access, strong quality gates, and a feedback loop they are responsible for closing.

The goal is not simply to generate more code.

It is to produce better outcomes with less manual coordination.