DEV Community

mars70s
mars70s

Posted on

Six Git Checks After an AI-Assisted Change — And What They Cannot Guarantee

A coding assistant changes the two files you asked it to edit. The diff looks reasonable. Is the next Git operation safe to proceed?

Not necessarily. You might be in the wrong repository. An extra file might already be staged. The branch or configured push destination might have changed. Or the secret scanner might have failed to run at all.

These are not necessarily mistakes in the generated code. They are mismatches between the workflow you authorized and the Git state that actually exists.

I built a small, public reference Git Safety Harness to check selected mismatches before a later operation. It is a PowerShell-based, fail-closed workflow control—not a sandbox, not an independent enforcement boundary, and not a guarantee of safe AI-generated code.

A two-file example

Suppose the authorized work is:

  • Edit docs/article.md and public/article.html in a particular repository.
  • Stay on main.
  • For the next operation, stage only docs/article.md.
  • Check the configured push URL for origin against an explicitly approved URL.
  • Run a staged-diff secret scan before any separately authorized push.

The harness does not invent these expectations. A human or an authorized workflow must define them first. The harness then compares selected observed Git state against them.

Six checks, six different questions

Check Question before proceeding Example of a reason to stop
Repository Root Is this the intended Git repository? The command is running in another valid checkout.
Write Set Are observed tracked changes and non-ignored untracked paths within the allowed set? README.md changed even though it was not authorized.
Stage Set Does the actual Git index contain only explicitly allowed staged paths? An unrelated file was staged earlier.
Branch Is the current symbolic branch the expected branch? HEAD is detached or the branch differs.
Push URL Does the selected remote have exactly one configured push URL, matching the approved URL? origin points somewhere else, or has multiple push URLs.
Secret Scan Can Gitleaks run and accept the staged diff? Gitleaks is missing, cannot execute, or exits nonzero.

Each question has a different failure mode. A PASS from one check does not substitute for the others.

Why Write Set and Stage Set are separate

The Write Set asks which paths are allowed to change. The Stage Set asks a different question: which paths are allowed in the Git index for the next operation?

The reference implementation reads the actual index with git diff --cached --no-renames --name-only and compares staged paths with a separate allow-list. That is a separate comparison from the Write Set, which covers staged and unstaged tracked changes as well as non-ignored untracked paths, using its own allow-list.

Example: an allowed change, but an unapproved staged file

The approved Write Set contains both docs/article.md and public/article.html. The approved Stage Set contains only docs/article.md. Now imagine the following hypothetical output, not a captured test transcript:

$ git diff --cached --no-renames --name-only
docs/article.md
public/article.html
Enter fullscreen mode Exit fullscreen mode

Both paths are allowed to change, so their presence alone does not violate the Write Set. But public/article.html is not allowed in the index for this operation. The reference implementation treats this as a Stage Set mismatch and reports GSH_STOP_STAGE_SET instead of accepting that staged state.

What would happen next? Consider two hypothetical paths from this same Git state:

  • Without an effective Stage Set check: If the workflow now runs git commit without changing the index, both docs/article.md and public/article.html are included in that commit. The second file was permitted to be edited, but was not authorized for this commit. If that commit is later pushed under a separate authorization, the unintended change can reach the remote repository. Neither a commit nor a push is claimed to have occurred in this example.
  • With the harness invoked and its exit respected: The Stage Set mismatch triggers GSH_STOP_STAGE_SET and a nonzero result. The calling workflow stops before its later commit or push action, and the staged files remain for inspection—the harness does not silently unstage or repair them. A human can then decide whether to correct the staging, change the authorization, or stop the work.

This illustrates a workflow boundary, not a measured incident or a new test result. It also does not mean that the harness itself blocks direct Git commands: if an actor bypasses the checked path, the local Layer-2 harness cannot enforce that stop.

Likewise, checking a remote's fetch address is not enough to establish its push destination. The harness inspects push URLs and requires one exact match.

What fail-closed means here

If a required state cannot be confirmed, the normal harness path stops rather than treating missing evidence as success.

That includes a repository root that cannot be resolved, an unexpected branch, an unavailable Gitleaks executable, and a scanner execution failure. The validated PowerShell wrapper also treats invocation errors, a missing child exit status, and nonzero exits as failures.

STOP is not an instruction for the assistant to silently repair the situation and continue. It is a point for inspecting the observed state and obtaining any necessary human decision.

What was actually tested?

The commit-pinned Validation Record reports results for the identified implementation in documented Windows reference environments:

  • 28/28 fail-closed cases passed: 14 under PowerShell 7 and 14 under Windows PowerShell 5.1.
  • 6/6 success-path cases passed: three under each shell, including a non-empty permitted staged diff scanned with real Gitleaks.
  • The record identifies the source and test files using SHA-256 hashes and the versions used: Git 2.54.0.windows.1, PowerShell 7.6.5, Windows PowerShell 5.1.26100.9444, and Gitleaks 8.30.1. A later preserved revalidation used the same source/test identities with PowerShell 7.6.6.

The tests used synthetic local remotes and recorded bounded results for the identified implementation and environments. They did not establish universal compatibility, unbypassability, or the safety of a later real-world push or deployment.

The Evidence Pack from that later revalidation preserves execution output, case results, exit codes, and a SHA-256 manifest. These are records of the later run, not the original run’s raw logs.

What the harness cannot guarantee

Three limits matter especially in AI-assisted workflows:

  1. Allowed does not mean correct. A change inside docs/article.md can still be technically or semantically wrong.
  2. A local check can be bypassed. If a person or agent can invoke raw Git, shell commands, or APIs outside the harness, the harness does not itself prohibit those paths. Independent restrictions belong outside this local workflow control (Layer 2), in a separate enforcement layer.
  3. The check is point-in-time. State can change after verification. The harness checks a configured push URL but does not perform or authorize the later git push. Its Gitleaks check covers the staged diff examined by the scanner, not every file or every secret.

Those limitations are reasons to keep human approval separate from independent enforcement, such as repository-side controls. Human approval is a workflow decision, not independent enforcement; the local harness replaces neither.

The useful boundary

The goal is not to prove that an AI assistant is safe. It is narrower: before a selected Git workflow continues, make certain mismatches observable and stop when required checks cannot be accepted.

The full English design article on OSIIX explains the checks in detail. For the current implementation—including its separate AllowedStagedFiles parameter—use the commit-pinned source, Getting Started guide, and limitations, rather than treating every illustrative snippet in the earlier article as the latest tested code.

If you use AI-assisted Git automation, which of these checks is enforced outside the agent-controlled execution path, and which is only a convention the agent is expected to follow?

Top comments (0)