The latest articles on DEV Community by fallen-leaves (@fallenleaves089). https://dev.to/fallenleaves089 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4237233%2Fbf156eda-bc49-475c-83b2-068b0f042457.jpg https://dev.to/fallenleaves089 en fallen-leaves Tue, 10 Nov 2026 13:55:03 +0000 https://dev.to/fallenleaves089/how-i-stopped-ai-agents-from-hallucinating-and-bypassing-security-open-source-59c7 https://dev.to/fallenleaves089/how-i-stopped-ai-agents-from-hallucinating-and-bypassing-security-open-source-59c7 <p>Hi everyone! 👋</p> <p>I'm a 17-year-old developer. Recently, while using AI agents to review my code, I got extremely frustrated with two things:</p> <ol> <li>AI often hallucinates success by swallowing error exit codes. It tells me "All tests passed" while the script is actually broken.</li> <li>AI can be easily tricked by prompt injections hidden in READMEs, and it might modify my uncommitted code without permission.</li> </ol> <p>So, I decided to build a solution. </p> <p>I am excited to share <strong><code>project-health-macos</code></strong>, a deep health-check skill strictly for native Apple Silicon (M1 and later). It audits existing projects and generates P0-P3 findings with evidence.</p> <h3> ✨ Key Features </h3> <ul> <li> <strong>Read-only by default</strong>: It finds issues (P0-P3 with <code>file:line</code> evidence) and reports them without touching your code.</li> <li> <strong>Prompt Injection Defense</strong>: It treats all text in the target repo as untrusted evidence, never as instructions.</li> <li> <strong>Zero Trust CWD</strong>: Fails closed if the working directory fingerprint is wrong (preventing malicious <code>docker-compose</code> execution).</li> <li> <strong>Native arm64 Gate</strong>: Rejects Rosetta terminals and Intel Macs via a hard <code>sysctl</code> check.</li> <li> <strong>Supply Chain Security</strong>: The release zip is signed with OpenSSH Ed25519. You can verify it using the provided <code>allowed_signers</code> and <code>SHA256SUMS.sig</code>.</li> </ul> <p>It's fully open-sourced under <strong>Apache-2.0</strong>. CI is green across Python 3.9 and 3.12 on GitHub's macOS runners.</p> <h3> 🔗 Links </h3> <ul> <li> <strong>GitHub Repository</strong>: <a href="https://github.com/Fallen-leaves089/project-health-macos" rel="noopener noreferrer">https://github.com/Fallen-leaves089/project-health-macos</a> </li> </ul> <p>Since I only tested it via GitHub Actions (macOS arm64), I would love to hear feedback from real Mac users. If you have a Mac, feel free to try it out on a disposable project. Let me know what you think!</p> ai security python opensource