DEV Community

Hagicode
Hagicode

Posted on • Originally published at docs.hagicode.com

The Hallucination Problem of AI Programming Assistants: How to Implement Specification-Driven Development with OpenSpec

#ai #openspec #hagicode

The Hallucination Problem of AI Programming Assistants: How to Implement Specification-Driven Development with OpenSpec

AI programming assistants are powerful, but they often generate code that doesn't meet actual requirements or violates project specifications. This article shares how the HagiCode project implements "specification-driven development" through the OpenSpec process, significantly reducing AI hallucination risks with a structured proposal mechanism.

Background

Anyone who has used GitHub Copilot or ChatGPT to write code has likely experienced this: the AI-generated code looks beautiful, but it's full of problems when actually used. It might use a component from the project incorrectly, ignore the team's coding standards, or write large chunks of logic based on non-existent assumptions.

This is the so-called "AI hallucination" problem—in the programming domain, it manifests as generating code that appears reasonable but doesn't align with the actual situation of the project.

Actually, this situation is quite frustrating. As AI programming assistants become more widespread, this problem becomes increasingly serious. After all, AI lacks understanding of project history, architectural decisions, and coding standards, while excessive freedom easily leads it to "creatively" generate code that doesn't match reality. This is quite similar to writing articles—without proper structure, it's easy to write something unfocused, when that's not actually the case.

To address these pain points, we made a bold decision: instead of trying to make AI smarter, we put it in a "specification" cage. The changes brought by this decision may be greater than you imagine—I'll elaborate shortly.

About HagiCode

The solution shared in this article comes from our practical experience in the HagiCode project. HagiCode is an open-source AI code assistant project dedicated to solving real-world problems in AI programming through structured engineering practices.

The Root Causes of AI Hallucination

Before diving into the solution, let's first look at where the problem actually lies. After all, know yourself and know your enemy, and you can fight a hundred battles with no danger of defeat—though applying this saying to AI is also somewhat interesting.

Missing Context

AI models are trained on public codebases, but your project has its own history, conventions, and architectural decisions. This "tacit knowledge" cannot be directly accessed by AI, so the generated code often disconnects from the actual project situation.

This can't be entirely blamed on AI—after all, it hasn't worked on your project, so how would it know your unwritten rules? It's like a new intern who doesn't understand the rules—that's normal, but the cost might be somewhat high.

Excessive Freedom

When you ask AI to "help me implement a user authentication feature," it might generate code in any form. Without clear constraints, AI will implement according to what it "thinks" is reasonable, rather than according to your project's requirements.

This is like letting someone who has never learned your project specifications freely express themselves—can problems be avoided? Actually, it's not that it's irresponsible, it just doesn't know what responsibility is.

Lack of Verification

After AI generates code, if there's no structured review process, code based on wrong assumptions will directly enter the codebase. By the time problems are discovered in testing or even production environments, the cost is too high.

This is no different from locking the barn after the horse is stolen—except the horse has already run away, so what's the use of locking the barn? Everyone understands this principle, but actually doing it always feels troublesome. After all, before things go bad, who wants to spend extra time?

OpenSpec: The Answer to Specification-Driven Development

HagiCode chose OpenSpec as the solution, with the core idea being: all code changes must go through a structured proposal process, transforming abstract ideas into executable implementation plans.

This sounds quite grand, but it essentially means having AI write the requirements document before writing the code. After all, preparedness ensures success, unpreparedness spells failure—the ancients didn't deceive us.

What is OpenSpec

OpenSpec is an npm-based command-line tool (@fission-ai/openspec) that defines a standard proposal file structure and validation mechanism. Simply put, it requires AI to "write the requirements document first" before writing code.

Three-Step Process to Prevent Hallucination

OpenSpec ensures proposal quality through a three-step process:

Step 1: Initialize proposal - Set session state to Openspecing
Step 2: Intermediate processing - Maintain Openspecing state, gradually refine artifacts
Step 3: Complete proposal - Transition to Reviewing state

This design has a clever detail: the first step uses the ProposalGenerationStart type, and completion doesn't trigger a state transition. This ensures the entire multi-step process won't prematurely enter the review phase before completion.

Actually, this detail is quite interesting—it's like cooking. If you lift the lid before the cooking time is reached, you definitely won't produce a good dish. Only by being patient, taking it step by step, can you finally create a good dish.

// Implementation in the HagiCode project
public enum MessageAssociationType
{
    ProposalGeneration = 2,
    ProposalExecution = 3,

    /// <summary>
    /// Marks the start of the three-step proposal generation process
    /// Does not transition to Reviewing state upon completion
    /// </summary>
    ProposalGenerationStart = 5
}
Enter fullscreen mode Exit fullscreen mode

Standardized File Structure

Every OpenSpec proposal follows the same directory structure:

openspec/
├── changes/                    # Active and archived changes
│   ├── {change-name}/
│   │   ├── proposal.md        # Proposal description
│   │   ├── design.md          # Design document
│   │   ├── specs/             # Technical specifications
│   │   └── tasks.md           # Executable task list
│   └── archive/               # Archived changes
└── specs/                     # Independent specification library
Enter fullscreen mode Exit fullscreen mode

According to statistics from the HagiCode project, there are currently 4000+ archived changes and 150,000+ lines of specification files. This historical accumulation not only gives AI rules to follow but also provides a valuable knowledge base for the team.

This is like the classics left by ancients—read more and you'll naturally gain some insight. It's just that these classics aren't written on bamboo slips, but stored in files.

Multi-Layer Validation Mechanism

The system implements multi-layer validation to ensure proposal quality:

// Validate required files exist
ValidateProposalFiles()

// Validate execution prerequisites
ValidateExecuteAsync()

// Validate startup conditions
ValidateStartAsync()

// Validate archive conditions
ValidateArchiveAsync()

// Proposal name format validation (kebab-case)
ValidateNameFormat()
Enter fullscreen mode Exit fullscreen mode

These validations are like gatekeepers with layers of checks—only truly qualified proposals can pass. Although it seems cumbersome, it's better than letting terrible code enter the codebase, right?

Prompt Template Constraints

AI execution in HagiCode uses predefined Handlebars templates that include clear step-by-step guidance and safeguards. For example:

  • Prohibit continuing without understanding user intent
  • Prohibit generating unverified code
  • Require re-providing when names are invalid
  • If a change already exists, recommend using the continue command rather than recreating

This approach of "dancing with shackles" actually makes AI more focused on understanding requirements and generating code that complies with specifications. Actually, constraints aren't necessarily a bad thing—after all, excessive freedom easily leads to chaos.

Practice: How to Use OpenSpec in Projects

Installation and Initialization 

npm install -g @fission-ai/openspec@1
openspec --version  # Verify installation
Enter fullscreen mode Exit fullscreen mode

The openspec/ folder structure will be automatically created in the project root directory.

This step doesn't need much explanation—installing tools, everyone understands. Just remember to use the @fission-ai/openspec@1 version, as new versions might have pitfalls—after all, stability comes first.

Creating Proposals

In HagiCode's conversation interface, use the shortcut command:

/opsx:new
Enter fullscreen mode Exit fullscreen mode

Or specify the change name and target repository:

/opsx:new "add-user-auth" --repos "repos/web"
Enter fullscreen mode Exit fullscreen mode

Creating proposals is like outlining an article—once you have an outline, the rest is easier to write. It's just that many people like to start writing directly, only to realize halfway through that their thinking doesn't flow—that's truly a headache.

Generating Artifacts

Use /opsx:continue to gradually generate the required artifacts:

proposal.md - Describes the purpose and scope of the change

# Proposal: Add User Authentication

## Why
The current system lacks user authentication functionality and cannot protect sensitive APIs.

## What Changes
- Add JWT authentication middleware
- Implement login/registration API
- Update frontend integration
Enter fullscreen mode Exit fullscreen mode

design.md - Detailed technical design

# Design: Add User Authentication

## Context
Currently using public APIs accessible to anyone...

## Decisions
1. Choose JWT over Session...
2. Use HS256 algorithm...

## Risks
- Token leakage risk...
- Mitigation measures...
Enter fullscreen mode Exit fullscreen mode

specs/ - Technical specifications and test scenarios

# user-auth Specification

## Requirements

### Requirement: JWT Token Generation
The system SHALL use the HS256 algorithm to generate JWT tokens.

#### Scenario: Valid login
- WHEN user provides valid credentials
- THEN the system SHALL return a valid JWT token
Enter fullscreen mode Exit fullscreen mode

tasks.md - Executable task list

# Tasks: Add User Authentication

## 1. Backend Changes
- [ ] 1.1 Create AuthController
- [ ] 1.2 Implement JWT middleware
- [ ] 1.3 Add unit tests
Enter fullscreen mode Exit fullscreen mode

These artifacts are actually like drafts for writing an article—once the draft is written, the main text naturally flows smoothly. It's just that many people don't like writing drafts, thinking it wastes time, but actually drafts are where you can best clarify your thinking.

Review and Application

After completing all artifacts:

/opsx:apply
Enter fullscreen mode Exit fullscreen mode

AI will read all context files and execute tasks step by step according to the checklist in tasks.md. At this point, because there are clear specifications, the quality of generated code will be much higher.

Actually, by this step, things are already halfway done. With a clear task list, the rest is just methodical execution. It's just that many people skip the previous steps and come directly here, so quality naturally can't be guaranteed.

Archiving

After the change is complete:

/opsx:archive
Enter fullscreen mode Exit fullscreen mode

Move completed changes to the archive/ directory for future reference and reuse.

Archiving is quite important—like properly storing a finished article. When you encounter similar problems in the future, flipping through previous records might give you answers. It's just that many people are too lazy to do it, thinking it's troublesome, but actually these accumulations are the most precious wealth.

Notes and Best Practices

Proposal Naming Convention

Use kebab-case format, starting with a letter, containing only lowercase letters, numbers, and hyphens:

  • ✅ add-user-auth
  • ❌ AddUserAuth
  • ❌ add--user-auth

Naming conventions aren't a big deal, but being unified is generally good. After all, in code, consistency is important—it's just that many people don't pay attention to it.

Avoiding Common Errors

  1. Using the wrong type in step 1 of the three-step process - Will transition state prematurely
  2. Forgetting to trigger state transition in the final step - Will get stuck in Openspecing state
  3. Skipping review to execute directly - Should first verify all artifacts are complete

These errors are easily made by beginners—experienced people naturally know how to avoid them. It's just that beginners will eventually become experienced, and taking some detours is fine, I just hope they don't take too many.

Managing Multiple Changes

OpenSpec supports managing multiple proposals simultaneously, which is particularly useful when handling large features:

# View all active changes
openspec list

# Switch to a specific change
openspec apply "add-user-auth"

# View change status
openspec status --change "add-user-auth"
Enter fullscreen mode Exit fullscreen mode

Managing multiple changes is like writing several articles simultaneously—it requires some skill and patience. But you'll get used to it—after all, people can always adapt.

Understanding the Session State Machine

Understanding state transitions helps troubleshoot problems:

Init → Drafting → Openspecing → Reviewing → Executing → ExecutionCompleted → Completed → Archived
Enter fullscreen mode Exit fullscreen mode
  • Openspecing: Planning in progress
  • Reviewing: Under review (can repeatedly modify artifacts)
  • Executing: Executing (applying tasks.md)

State machines are essentially a set of rules. Rules can be annoying sometimes, but more often they're useful. After all, nothing can be accomplished without norms—as the ancients said long ago.

Summary

Through the OpenSpec process, the HagiCode project has achieved significant results in addressing AI hallucination:

  1. Reduce hallucinations - AI must follow structured specifications and cannot generate code arbitrarily
  2. Improve quality - Multi-layer validation ensures changes meet project standards
  3. Accelerate collaboration - Archived changes provide reference for future development
  4. Traceability - Each change has complete proposal, design, specification, and task records

This solution doesn't make AI smarter, but puts it in a "specification" cage. Practice has shown that dancing with shackles actually produces better dance.

Actually, this principle is simple—constraints aren't necessarily a bad thing. It's like writing articles—with format constraints, it's actually easier to write good things. It's just that many people don't like constraints, thinking they limit their creativity, but actually creativity also needs soil to bloom and bear fruit.

If you're also using AI programming assistants and have encountered similar problems, why not try OpenSpec? Specification-driven development may seem to add some steps, but this upfront investment will pay back many times over in code quality and maintenance efficiency.

After all, sometimes being slower is actually being faster. It's just that many people don't understand this principle...

References

If this article helps you, feel free to give a Star on GitHub. HagiCode public beta has begun—install now to participate in the experience.

This article is pretty much done. There's nothing particularly profound, just a summary of some practical experience. Hope it's useful to everyone—after all, with sharing, you learn and let others learn too—it's a win-win, so why not?

But articles are just articles—what's truly useful is practice. After all, what you get from paper is shallow—you must personally practice to truly understand, as the ancients said...

Original Article & License

Thanks for reading. If this article helped, consider liking, bookmarking, or sharing it.
This article was created with AI assistance and reviewed by the author before publication.

Top comments (0)