DEV Community

Cover image for Stop letting AI rubber-stamp your broken code. I built an unforgiving QA system to fix it.
Mohamed Saleh
Mohamed Saleh

Posted on Edited on

Stop letting AI rubber-stamp your broken code. I built an unforgiving QA system to fix it.

We've all been there. You paste a complex chunk of code into ChatGPT or Claude, ask for a deep security review, and 10 seconds later you get:

"Looks great! Your code is well-structured. Good job!"

Then you push to staging, and immediately hit a massive CORS error, a memory leak, or a broken state mutation.

LLMs are incredible tools, but fundamentally, they are "people-pleasers." They are inherently lazy. If you don't force them into a corner, they will hallucinate a clean bill of health just to finish the prompt quickly.

I got completely fed up with this. I didn't want an AI cheerleader; I wanted an unforgiving, paranoid Senior QA Engineer.

So, I spent the last few weeks architecting a strict, open-source workflow to fix this behavior. I built a comprehensive AI Testing Suite (Prompts & Workflows) that physically forces LLMs to stop rubber-stamping code and actually audit it.

šŸ›”ļø How it fixes AI laziness (The 10-Layer System):

šŸ›‘ Citation Or It Didn't Happen: The AI is forbidden from making claims without citing the exact file path and line number. If it can't cite the line, the review is invalid.
🚧 Mandatory Phase Gates: It forces the AI through up to 20 strict, methodical gates. It physically cannot skip to phase 2 without printing the completion checklist for phase 1.
šŸ”“ Automated CLI Offensive Testing: The AI is forced to actually run Red Team tools (Nuclei, Trivy, MobSF, ZAP) via terminal commands. Fabricating text output = cheating.
šŸ”„ Mandatory Database Freshness: The AI is strictly required to update vulnerability databases (e.g., nuclei -update-templates) before scanning so it doesn't test 2026 code with 2023 knowledge.
🚨 Real-World Sentry Validation: It doesn't just guess. The workflow cross-references the AI's static analysis with actual runtime errors from Sentry before deployment.
šŸ•µļø The Hacker Mindset (Rule 6): When the AI proposes a security fix, the prompt forces it to act as a Red Team hacker and attempt to bypass its own newly written code.
šŸ‘€ Fresh-Eyes Re-Analysis: Forces a mandatory second-pass audit (acting as a "different" reviewer) to catch what it missed the first time.


šŸš€ The Open Source Suite

I built four specialized versions depending on your stack. You just point your AI agent (Cursor, Antigravity, GitHub Copilot) to the SKILL.md file in these repos, and watch the tone and strictness of the AI completely change.

ā˜ļø Backend & Cloud Auditor Skill
20 Phases covering Zero Trust, Automated Offensive Testing (Nuclei/Trivy), OWASP API Top 10, Chaos Engineering, and advanced forensics.

🌐 Web App QA & Security Auditor Skill
14 Phases covering Core Web Vitals, XSS/CSRF/CSP, Dynamic attacks (OWASP ZAP), state mutations, and SEO readiness.

šŸ“± Mobile App QA Tester Skill
14 Phases covering iOS/Android specific crashes, memory leaks, CLI Red Teaming (MobSF/apkleaks), and local SQLite integrity.

šŸ–„ļø Desktop App QA Tester Skill
14 Phases covering IPC security (Electron/Tauri), DLL injection prevention, vulnerability databases, and distribution readiness.


Stop letting AI blindly approve your pull requests. Force it to prove its work.

I’d love for you guys to try it out on your next refactor. Fork it, break it, and let me know if your AI still manages to miss bugs!

šŸ¤ Let's connect on LinkedIn: Mohamed Saleh

Top comments (0)