Preventing AI Coding Mistakes: Proofs vs Planning

Blocking AI Mistakes: Proofs vs Planning

When a code generation system outputs a 500-line React component with a subtle state-synchronization bug, the standard developer response is to feed the error log back into the model. This feedback loop is an expensive, non-deterministic way to write software. It treats code generation as an optimization problem to be solved after the fact, rather than a system design problem to be constrained before execution.

The Core Problem of Post-Hoc Debugging

The core issue in preventing AI coding mistakes is the phase at which constraints are applied. In traditional software engineering, we have two primary paradigms for ensuring correctness: formal verification (proofs) and structured planning (specifications). When using generative models to write code, developers often skip both. They rely on the model to infer the specification from a loose prompt, leading to hallucinated APIs, broken state transitions, and architectural drift.

To build reliable software with generative systems, we must shift our focus from post-hoc debugging to pre-generation constraint definition. This article compares the utility of mathematical proofs with structured planning, demonstrating why front-loading the planning phase is the most viable path to preventing AI coding mistakes.

The Limits of Formal Proofs in Generative Workflows

Formal verification, using tools like TLA+, allows developers to mathematically prove that a system design meets specific correctness properties. In theory, one could generate code and then run it through a formal proof solver to guarantee its correctness.

In practice, this approach fails for generative workflows due to three primary bottlenecks:

1. Specification Complexity: Writing a formal specification that is complete enough to prove a program correct is often more difficult than writing the program itself.
2. State Space Explosion: For typical business applications with database interactions, network calls, and user interfaces, the state space is too large for practical proof solvers to evaluate in real time.
3. Lack of Flexibility: Business requirements change rapidly. A formal proof must be rewritten with every minor change to the user flow, creating a massive maintenance bottleneck.

Relying solely on post-generation verification, such as running unit tests or static analysis, only catches syntax errors or basic type mismatches. It does not catch logical drift or architectural misalignment. If the system generates a function that perfectly passes its unit tests but implements the wrong business logic, the test suite is green, but the application is broken.

Structured Planning as a Constraint Engine

Instead of verifying code after it is written, we must constrain the decision space before the system generates a single line of code. This is the core of the planning as execution methodology. By narrowing the decision space early, we eliminate the vast majority of potential generation errors.

A structured plan for a generative system consists of three core pillars:

1. Clear User Stories: These define the state transitions and business rules.
2. Strict Data Schemas: These define the boundaries of data flow and database interactions.
3. Explicit UX Flows: These define the interaction model and user interface states.

When these three pillars are defined as structured data, they act as a constraint engine. The planner no longer has to guess the database schema or the state management library; those decisions are locked down. The generation task is reduced from writing a checkout feature to implementing a function that maps state A to state B using schema C.

A Concrete Workflow Sketch

To illustrate this approach, let us look at a methodology sketch. We will define a structured plan in JSON format and show a simple Python runner that validates this plan before code generation.

Here is a pseudo-code representation of a planning manifest for a checkout feature:

{
  "feature": "Discount Code Application",
  "state_transitions": [
    {
      "from": "CartEmpty",
      "to": "CartWithItems",
      "trigger": "ADD_ITEM"
    },
    {
      "from": "CartWithItems",
      "to": "DiscountApplied",
      "trigger": "APPLY_DISCOUNT",
      "constraints": [
        "discount_code_must_exist",
        "discount_not_expired"
      ]
    }
  ],
  "schema": {
    "Cart": {
      "items": "array",
      "subtotal": "decimal",
      "discount_total": "decimal",
      "grand_total": "decimal"
    },
    "Discount": {
      "code": "string",
      "percentage": "decimal",
      "is_active": "boolean"
    }
  }
}

Now, let us look at a Python script that acts as the validator. This script ensures that the plan is logically consistent before we pass it to the code generation system.

# validator.py
# A simple runner to validate the planning manifest before code generation

import json

class PlanValidator:
    def __init__(self, plan_json):
        self.plan = json.loads(plan_json)

    def validate_transitions(self):
        states = set()
        for transition in self.plan.get("state_transitions", []):
            states.add(transition["from"])
            states.add(transition["to"])

        # Ensure we have a starting state and no disconnected states
        if "CartEmpty" not in states:
            raise ValueError("Missing initial state: CartEmpty")
        return True

    def validate_schema(self):
        schema = self.plan.get("schema", {})
        if "Cart" not in schema or "Discount" not in schema:
            raise ValueError("Required schemas are missing")
        return True

    def run_all(self):
        if self.validate_transitions() and self.validate_schema():
            print("Plan is valid. Proceeding to code generation.")
            return True
        return False

# Example usage
manifest = """
{
  "feature": "Discount Code Application",
  "state_transitions": [
    {\"from\": \"CartEmpty\", \"to\": \"CartWithItems\", \"trigger\": \"ADD_ITEM\"},
    {\"from\": \"CartWithItems\", \"to\": \"DiscountApplied\", \"trigger\": \"APPLY_DISCOUNT\"}
  ],
  "schema": {
    "Cart": {\"items\": \"array\", \"subtotal\": \"decimal\"},
    "Discount": {\"code\": \"string\", \"is_active\": \"boolean\"}
  }
}
"""

validator = PlanValidator(manifest)
validator.run_all()

By running this validation step, we ensure that the system is working with a logically sound specification. If the plan fails validation, we halt the process before generating any code, saving compute resources and preventing the introduction of broken code into the repository.

Trade-offs and Adversarial Review

Front-loading the planning phase requires a shift in developer behavior. It is often tempting to write a quick prompt and let the system generate code immediately. However, this approach leads to a high rate of downstream errors that require manual debugging.

The trade-offs of structured planning include:

• Upfront Time Investment: Developers must spend time defining schemas and state transitions before code is written.
• Tooling Requirements: Teams must adopt or build tools that can parse and validate planning manifests.
• Cognitive Load: Developers must think deeply about system design before execution.

Despite these trade-offs, structured planning significantly reduces the non-deterministic search space of the generator. By treating the plan as the primary artifact, we can perform adversarial reviews on the design itself. We can ask the system to find edge cases in the state transitions before any code is generated. This is far more efficient than trying to find edge cases in thousands of lines of generated JavaScript or Python.

Conclusion

Preventing AI coding mistakes is not about building better compilers or faster feedback loops. It is about narrowing the decision space before code generation begins. By shifting our focus from post-hoc debugging to structured planning, we can build more reliable, maintainable software with generative systems.

Read the complete article on preventing AI coding mistakes on the Bridge blog.