I Built an OSS to Stop AI from Writing Inconsistent Code Every Time (Claude Code / Cursor / Kiro)

yunbow — Sun, 03 May 2026 04:07:06 +0000

"Write a function to fetch the list of users." — same prompt, same codebase.

Yesterday: getUsers(). Today: fetchUserList(). Tomorrow: loadAllUsers().

Six months of AI-assisted coding and I kept hitting this wall. My initial reaction was "maybe I need to write better prompts." I wrote better prompts. The functions got slightly better. New inconsistencies appeared elsewhere.

The problem wasn't the AI's capability. It was that I had never given it my team's unwritten rules.

That realization led me to build AI Dev OS — an open-source framework that converts implicit developer knowledge into explicit, enforceable rules for AI coding assistants. It supports Claude Code, Cursor, and Kiro (Amazon's AI-native IDE).

GitHub: github.com/yunbow/ai-dev-os

The Root Problem: "Almost Correct" Is the Most Expensive Output

When an AI produces obviously wrong code, you catch it immediately and discard it. Not a big deal.

When an AI produces almost correct code — code that compiles, passes basic tests, looks reasonable — you merge it. Six months later, your codebase has three error handling patterns, four naming styles, and no one is sure which is canonical.

That's the real cost. Not obvious errors. Plausible drift.

Here's why this happens structurally: your AI assistant has no access to your team's conventions by default. It doesn't know that you use ActionResult<T> for Server Actions, that Zod schemas live in lib/schemas/ and not colocated with components, that you've banned useEffect for data fetching. These conventions exist in your engineers' heads and in unwritten PR review culture — nowhere an AI can read.

Switching to a more capable model doesn't fix this. A more capable model without your conventions just produces more fluent drift.

The Solution: A 4-Layer Rule Framework

The core insight of AI Dev OS: rules have different lifespans, and mixing them into one file is what causes fragility.

┌─────────────────────────────────────────────────────────┐
│  L1: Design Philosophy          (2–5 year lifespan)     │
│      "We optimize for correctness and observability     │
│       over developer convenience"                       │
│                                                         │
│  L2: Technology Decisions       (1–3 years)             │
│      "Next.js App Router, Prisma, Zod, NextAuth"        │
│      "We use Tailwind — no CSS-in-JS"                   │
│                                                         │
│  L3: Concrete Coding Rules      (6–12 months)           │
│      Naming conventions, error handling patterns,       │
│      security invariants, test structure                │
│                                                         │
│  L4: Tool-Specific Config       (2–4 months)            │
│      CLAUDE.md, .cursor/.mdc files, Kiro Steering Rules │
└─────────────────────────────────────────────────────────┘

Why does this matter? When you update your AI tool's configuration (L4), you shouldn't risk destabilizing your design philosophy (L1). When you adopt a new library (L2), you shouldn't have to rewrite your security rules (L3). Separating by lifespan makes updates surgical.

The 7-Repository Structure

AI Dev OS is distributed across 7 repositories, each with a clear responsibility:

ai-dev-os                     ← Core specification and theory     [available]
ai-dev-os-rules-typescript    ← L1–L3 rules for TypeScript / Next.js  [available]
ai-dev-os-rules-python        ← L1–L3 rules for Python / FastAPI  [available]
ai-dev-os-plugin-claude-code  ← L4: Skills, Agents, Hooks for Claude Code  [available]
ai-dev-os-plugin-cursor       ← L4: .mdc rules for Cursor         [available]
ai-dev-os-plugin-kiro         ← L4: Steering Rules for Kiro       [available]
ai-dev-os-cli                 ← init / update / doctor commands   [available]

You don't need all of them. If you're using Claude Code with TypeScript, you need ai-dev-os-rules-typescript and ai-dev-os-plugin-claude-code. Pick what fits your stack.

How to Get Started

Option A: CLI (10 minutes)

npx ai-dev-os init --rules typescript --plugin claude-code

This creates:

CLAUDE.md — guidelines index referencing your rule files
docs/ai-dev-os/ — rule submodule (L1–L3)
.claude/plugins/ai-dev-os/ — Skills, Agents, Hooks (L4)
Pre-commit hooks for automated rule verification

Verify the setup:

npx ai-dev-os doctor

Option B: Manual (20 minutes, more control)

Step 1: Add the rule submodule and copy the CLAUDE.md template.

git submodule add https://github.com/yunbow/ai-dev-os-rules-typescript.git docs/ai-dev-os
cp docs/ai-dev-os/templates/nextjs/CLAUDE.md.template ./CLAUDE.md

CLAUDE.md is an index file — 3 lines of directives plus references to the guideline files in the submodule. Add your project-specific rules (3 files recommended) in the ## Project-Specific Guidelines section:

## Project-Specific Guidelines
- docs/guidelines/your-business-rule.md
- docs/guidelines/your-external-api.md
- docs/guidelines/your-compliance.md

Step 2: Add the Claude Code plugin for Skills, Agents, and Hooks:

git submodule add https://github.com/yunbow/ai-dev-os-plugin-claude-code.git .claude/plugins/ai-dev-os

Claude Code loads CLAUDE.md automatically on every interaction. The guideline files referenced within it are read on demand.

Before / After: What This Actually Changes

API Route

Before (no guidelines):

// app/api/users/route.ts
export async function GET(req: Request) {
  const users = await db.user.findMany()
  return Response.json(users)
}

Problems: no auth check, exposes all user fields including sensitive ones, no error handling.

After (with AI Dev OS):

// app/api/users/route.ts
import { auth } from '@/lib/auth'
import { db } from '@/lib/db'

export async function GET(): Promise<Response> {
  const session = await auth()
  if (!session) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 })
  }

  const users = await db.user.findMany({
    select: { id: true, name: true, email: true, createdAt: true },
  })

  return Response.json({ data: users })
}

Auth check. Field selection. Consistent response shape. The AI generated this without being asked because the rules made it the default.

Server Action

Before:

'use server'
export async function updateUser(formData: FormData) {
  const name = formData.get('name')
  await db.user.update({ where: { id: '...' }, data: { name } })
}

Problems: no validation, no error handling, name is typed as FormDataEntryValue | null, hardcoded ID.

After:

'use server'
import { z } from 'zod'
import type { ActionResult } from '@/types/action'
import { auth } from '@/lib/auth'
import { db } from '@/lib/db'

const UpdateUserSchema = z.object({
  name: z.string().min(1).max(100),
})

export async function updateUser(
  formData: FormData
): Promise<ActionResult<void>> {
  const session = await auth()
  if (!session) return { success: false, error: 'Unauthorized' }

  const parsed = UpdateUserSchema.safeParse({ name: formData.get('name') })
  if (!parsed.success) return { success: false, error: parsed.error.message }

  await db.user.update({
    where: { id: session.user.id },
    data: parsed.data,
  })

  return { success: true }
}

Every Server Action in the codebase now follows this pattern — not because individual developers remember to, but because the AI enforces it by default.

The Effect Over Time

The value compounds. After 2 weeks with AI Dev OS:

Every new API route has auth checks and field selection
Every Server Action validates input and returns ActionResult<T>
PR reviews stop being about convention enforcement and start being about logic

After 2 months, a new engineer joining the project reads the guideline files and understands the codebase's patterns — because those patterns are now consistently applied everywhere, including the AI-generated portions.

What This Isn't

AI Dev OS won't catch logical bugs. It won't validate your business rules. It won't replace code review.

What it does: ensure the AI produces code that matches your codebase's established patterns. The output is still yours to review — but you're reviewing logic, not convention drift.

Try It

# Check out the core spec
git clone https://github.com/yunbow/ai-dev-os

# Or start with the TypeScript rules
git clone https://github.com/yunbow/ai-dev-os-rules-typescript

If you've been hitting the inconsistency wall and want to talk through how to adapt the rules for your stack — drop a comment. What conventions does your team enforce that your AI assistant keeps ignoring?

DEV Community: yunbow

AI Dev OS