Picture a team with a plausible PR, passing checks and no clear record of the commands behind it. A fast approval would be convenient. An informed approval needs more than confidence in the final diff.
Getting agent-generated code merged sooner with fewer rollbacks depends partly on what a reviewer can establish about the session behind the change. Missing evidence creates a different bottleneck from missing code.
I would start AI coding agent monitoring with five numbers, each connected to evidence a reviewer can inspect. High-velocity teams do not need another leaderboard. A compact record of coverage, risky actions and unresolved findings is more useful than a total token count.
A clean diff does not explain how the agent got there
The diff shows committed changes. Session records may show commands, content read, tool results and permission decisions that never become part of those changes. Both forms of evidence matter, but neither is a complete substitute for the other.
A test run that used an unexpected credential, a command copied from untrusted text or a repository startup hook can raise questions the final code does not answer. The review decision needs a way to locate those events without exposing raw secrets to everyone reading the PR.
A small open-source tool, not a universal monitoring platform
Stroq is a small open-source tool whose documentation describes local analysis of coding-agent sessions, text-level command provenance and known-secret matching. The project says it has no telemetry. The descriptions are documented capabilities, not the results of an independent security audit here.
The CLI documentation distinguishes a session card, retrospective replay, credential findings and live-hook logs. The card defaults to the last 30 days of Claude Code and Codex sessions. The separate sent command also reads Cursor sessions.
Retrospective analysis asks what recorded activity looks like under the current analysis or policy. A live hook records a decision when the host calls it. Replay is not a time machine that proves the same decision was enforced during an earlier session.
- Commands recorded in the sessions you actually scanned
- Commands linked to text the agent read
- Known keys classified as carried to a third party
- Repositories with code-on-open findings
- Risky calls recorded as asked or denied
1. Commands recorded in the sessions you actually scanned
The first number is the recorded command count, paired with the number of sessions, the agent types and the time window checked. A count without that context is difficult to interpret and easy to compare unfairly.
Stroq's card documents command counts from the supported session records. A higher count can mean a larger task, repeated retries or a different working style. The number alone does not say whether the work was productive or safe.
I would put the coverage statement beside the count rather than bury it in a footnote. Missing sessions, an unreadable record or an unsupported reader mean unknown activity, not zero activity.
2. Commands linked to text the agent read
Stroq documents text-level provenance: content the agent read can be linked to later commands that repeat it. The resulting count identifies actions worth inspecting because external text may have influenced what happened next.
A useful optional ratio is provenance-matched commands divided by recorded commands, calculated within the same supported session collection. The ratio is an editorial aggregation, not a claim that the tool measures every causal influence on a model.
A match does not prove prompt injection. Documentation can legitimately describe a command that the user intended the agent to run. A reviewer needs the source, the action and the user's task to decide whether the connection is appropriate.
Conversely, a command independently composed by the model may not match any previously read text. The project's README explicitly says matching is text-level. Absence of a match is not proof that the action was uninfluenced or authorized.
3. Known keys classified as carried to a third party
The documented sent report compares session records against credential values known on the machine. The report prints names and sources rather than values, then classifies what it found in recorded activity.
For this metric, I would count distinct known keys classified as sent to another company, preserving the tool's classification and evidence. Repeated appearances of the same key should not quietly become a count of distinct exposed credentials.
The other categories matter: "maybe sent," "seen by the AI," "used with its own service" and "blocked before it ran" describe different observations. Combining all of them into "keys leaked" would erase the distinctions a reviewer needs.
Stroq says a send is named when the key appears in the command that sent it. A local transcript does not establish what a destination retained, whether a model provider received particular bytes, or whether anyone used the credential afterward.
Coverage has limits too. Old, rotated or unknown values can be missed. A script, a file or a variable may carry a key without the value appearing in the recorded command. Transformed values may defeat an exact-value comparison.
A clean result should therefore read as "no matching send detected in the checked records," not "no credential left the machine." The first statement describes evidence. The second claims visibility the documentation does not promise.
4. Repositories with code-on-open findings
Agent work can start executing project-controlled behavior before the main task becomes a visible diff. Stroq's card and inspect documentation describe identifying repositories with code-on-open behavior and inspecting what a repository can run before it is opened.
I would count repositories with relevant findings, accompanied by the repositories inspected and the exact configuration evidence. A ratio is meaningful only when that denominator is recorded and the inspection scope is consistent.
A finding is not proof of execution. A configured hook, startup action or repository setting describes potential behavior; a session record or other authorized execution evidence is needed to establish what actually ran.
5. Risky calls recorded as asked or denied
The fifth number comes from live audit entries rather than a historical replay verdict. Stroq documents a log of decisions and explanations; counts can be grouped by decision, rule and session to show where policy intervened.
A reviewer needs the recorded decision and whatever outcome is available. The agent, host version and hook coverage belong beside the count because the hook only sees the events that host provides.
The README also distinguishes hooks being configured from evidence that the agent actually calls them. A configured integration is not, by itself, proof that every risky action passed through it.
Retrospective replay can still be useful for policy design. A replay saying "deny" means today's policy would classify the recorded action that way. Calling the same event "historically blocked" would turn a proposed counterfactual into a false execution record.
Turn the counts into a review packet, not a safety score
The five numbers are most useful when they point to a short evidence packet. My proposed packet includes the session and time window, relevant redacted events, coverage gaps, and the owner's disposition of findings.
Each important event should preserve the path from source to action to decision, plus the observed outcome where available. An unknown outcome should stay unknown rather than acquire a reassuring green label.
Access and retention matter. A full transcript may contain prompts, customer information, paths or credential material that does not belong in a broadly visible PR. A redacted summary can reference restricted evidence without duplicating it into another system.
I would not combine the counts into one safety score. A quiet log with missing coverage can look better numerically than a well-instrumented session that correctly asked for permission. The situations are different, not a ranking.
Choose Qodo when evidence needs to support a policy-governed approval
Qodo, the AI code review platform, addresses the committed-change layer. Qodo's PR review checks code against team rules; session analysis contributes a different kind of evidence. A human attaching a redacted review packet does not imply Qodo automatically ingests Stroq telemetry.
Review before the PR brings feedback closer to the work. Cross-repo context, learning from the team and high-signal findings support code decisions across related changes. Runtime evidence still needs its own coverage checks and accountable reviewer.
High-velocity teams get the rule files they already have, checked on every PR, with setup in minutes and no new process. Enterprise engineering teams get one rule set across many repos and agents, alongside explicit ownership of evidence access, retention and exceptions.
Qodo no longer offers test generation; it focuses on AI code review and code governance. In my experience, Qodo is best at reviewing code changes against team rules while people assess the session evidence.
Keep unknowns visible to the reviewer
Rules checked on every PR make the team's code-review expectations repeatable. Review before the PR, cross-repo context and learning from the team help connect findings with the changes under review. High-signal findings give reviewers specific issues to assess rather than a blanket assurance.
A person still needs to assess session coverage, investigate risky events and resolve unknown outcomes. Qodo flags; your team decides. The useful connection is clearer review evidence, not an invented connection between a local scanner and the review platform.
The AI code review buyer's guide provides broader context for that review layer. Session evidence still needs to be assessed on its own coverage and limits.
FAQ
Which metrics matter beyond tokens and cost?
Recorded commands, provenance matches, known-key classifications, code-on-open findings and live permission decisions can each support a review question. Coverage, denominators and event evidence determine how much any count can actually tell the reviewer.
Can session logs prove a secret reached a third party?
A recorded call containing a key is evidence worth investigating, but it does not independently establish delivery, retention or subsequent use. Scripts and other indirect routes can also leave gaps in what the transcript reveals.
Does a provenance match prove prompt injection?
No. A legitimate instruction in documentation can match a later command. The reviewer must assess the source, the user's task and the action rather than treat textual overlap as a verdict about intent.
Can sessions from before hooks were installed be checked?
Stroq documents retrospective replay and secret matching for existing supported session records. Available records, known credential values and reader support constrain the result; earlier sessions do not gain historical enforcement because they can now be analyzed.
What is the difference between replay and a live deny?
Replay applies current policy to recorded activity. A live deny records an actual hook decision when called by the host. Neither should be described as complete proof that every possible path was covered.
Top comments (0)