DEV Community

GrahamduesCN
GrahamduesCN

Posted on

I Built a 'Production-Ready' AI Agent Framework. It Was a Lie. So I Fixed It.

#ai #webdev #opensource #typescript

I Built a "Production-Ready" AI Agent Framework. It Was a Lie. So I Fixed It.

A story about AI-generated repos, fake READMEs, and what I did about it.

Two weeks ago, I built an automation pipeline that generates open-source projects on GitHub. The idea: market-research trending tech, scaffold a repo, fill it with code, add docs, and push — fully automated. 14 agents working together. 65 seconds end-to-end. I was proud of it.

Then I actually looked at the generated code.

The Ugly Truth

The pipeline had generated three repos:

  • ai-agent-starter: "Production-ready AI agent framework with multi-provider support"
  • dev-cli-kit: "Modern CLI toolkit with scaffolding, code generation, and git automation"
  • nextjs-saas-starter: "Next.js SaaS starter with auth, payments, and AI integration"

Sounds great, right? Here's what was actually inside:

Repo Source Code Tests Could It Run?
ai-agent-starter 183 lines of broken code 0 No
dev-cli-kit 1 Handlebars template file (not even .ts) 0 No
nextjs-saas-starter Zero lines of code 0 No

The nextjs-saas-starter didn't even have a src/ directory. Just a package.json, a tsconfig.json, and a README claiming it had "authentication, Stripe payments, and an AI chat interface." None of those existed. Not even a single component.

The READMEs were AI-generated marketing fluff. Every claim was false. And somehow, each one had donation links (PayPal, Buy Me a Coffee, GitHub Sponsors) — asking for money before delivering a single working line of code.

I had built a lying machine.

What I Did

Instead of deleting the repos (GitHub URLs and star history have value), I decided to make every claim true.

I rewrote all three from scratch, and built a fourth:

1. ai-agent-starter (v0.2.0)

What it claimed → What it now is:

  • "Multi-provider support (OpenAI, Anthropic, Ollama)" → Actually implemented all three, with streaming
  • "Built-in tool execution engine" → Real function calling via OpenAI/Anthropic native tool APIs, not regex string matching
  • "Conversation memory" → Token-aware memory manager with auto-pruning
  • "CLI and programmatic API" → npx ai-agent-starter chat, npx ai-agent-starter run "prompt", and importable package

650+ lines of TypeScript, 24 tests, zero build errors.

2. dev-cli-kit (v0.2.0)

The old version's only source file was cli.ts.hbs — a Handlebars template with // TODO: Implement init logic as its core functionality.

Now: real project scaffolding from 3 built-in templates (Node/TS, React/Vite, CLI tool), code generation (component/hook/util/api), and git workflow helpers. 480+ lines, 8 tests.

3. nextjs-saas-starter (v0.2.0)

From zero lines to a complete Next.js 14 app: landing page (hero, features grid, pricing), dashboard with sidebar + stats, login/register pages, billing page, settings page, API route, Tailwind CSS with dark mode. 520+ lines across 14 files.

4. ai-chat-saas (NEW)

Built on top of the first three — a complete AI chat application. Multi-provider streaming chat with server-side API key protection. 763 lines. Ready to deploy on Vercel in 60 seconds.

What I Learned

  1. AI-generated code isn't the problem — unverified AI-generated code is. My pipeline agents were generating template stubs and calling them "production-ready." The fix was embedding real, tested code as the generation template.

  2. READMEs should describe what EXISTS, not what you WISH existed. Every line in the new READMEs corresponds to actual code you can run.

  3. Donation links on empty repos are a red flag. If a project has zero working code and asks for money, it damages trust in the entire open-source ecosystem. I removed all donation links and added them back only where appropriate.

  4. "Production-ready" means something. It means tests pass, builds succeed, and someone can clone your repo, run npm install && npm run dev, and see a working product. Not a TODO comment.

What's Next

The pipeline is fixed. The code agents now generate real, working code instead of stubs. Cron runs daily at 16:00 Beijing time.

I'm also launching a paid template — ai-chat-saas — on Gumroad for $19. It's the chat application with all three providers, streaming, dark mode, and server-side API protection. Clone, set your API keys, deploy.

The moral: if you're using AI to generate code, verify it. Actually run npm install. Actually run npm test. Actually try to use it. Otherwise you're just building a very fast bullshit machine.

All repos: github.com/GrahamduesCN

Top comments (0)