An observation from building software with AI assistance—and what research tells us about trusting generated code.
AI coding assistants can generate APIs, write database queries, create tests, and explain unfamiliar code in seconds.
That's impressive.
But while working on a software project involving APIs, authentication, database operations, and automated testing, I noticed something important:
Generating code is often easier than verifying whether it is actually correct.
A function can compile, return the expected response, and still contain a security or logical flaw.
A simple example
Imagine asking an AI assistant:
Create an API endpoint to fetch an invoice using its ID.
It might generate a database query that retrieves the invoice successfully.
But there's another question:
Does the requesting user actually have permission to access that invoice?
Without an ownership or tenant-level authorization check, one user might access another user's data simply by changing the invoice ID.
The endpoint works. The security requirement doesn't.
This is the difference between code that runs and software that can be trusted.
What does research say?
This concern isn't just an observation from development.
A 2025 study published in ACM Transactions on Software Engineering and Methodology, titled Security Weaknesses of Copilot-Generated Code in GitHub Projects, analyzed 733 code snippets and identified potential security weaknesses in:
- 29.5% of the Python snippets.
- 24.2% of the JavaScript snippets.
These figures relate to the study's dataset, not all AI-generated code.
Another study published in Communications of the ACM examined 1,689 programs generated by GitHub Copilot across 89 security scenarios. Approximately 40% were found to be vulnerable in that experimental setting.
These findings highlight an important point: generated code still needs independent review.
What should developers verify?
Before accepting AI-generated code, ask:
- Correctness: Does it actually satisfy the requirement?
- Security: Are authentication, authorization, and input validation handled?
- Edge cases: What happens when inputs are missing, invalid, or duplicated?
- Performance: Will it work with larger datasets and concurrent requests?
- Testing: Do the tests cover failure scenarios, not just successful execution?
AI can help write tests, but passing tests do not automatically prove that an implementation is correct.
The requirements and test cases need independent review too.
My takeaway
I don't think the question should be whether developers should use AI to write code.
The more important question is whether developers understand the code they accept.
AI can accelerate implementation, but it cannot replace engineering judgment.
The ability to generate code is becoming easier. The ability to evaluate it is becoming more important.
What is your approach to AI-generated code? Do you review every change, rely on automated tests, or use a combination of both?
I'd be interested in hearing how other developers handle this.
Tags: #ai #softwareengineering #programming #security
Top comments (1)
Aloha. Apenas estamos en el gran y sorprendente 2026, antes de 2030 ya se aclara todo, mientras tanto auditar es bueno Claude. El problema mayor que veo es que estan restringidas las cotas de uso de los LLMs y que los modelos compententes para codigo son de pago, que la mayoria de la comunidad open source no estamos dispuestos a pagar. Entonces hasta que se equilibre y estabilice todo ese campo, audita con el LLM mas competente y economico, para tareas menores puede hacerlo en agente con un LLM free (pero con cautela) xd... Aloha, feliz semana. :) Gracias.
Aloha. We are barely in the great and surprising 2026, before 2030 everything is cleared up, meanwhile auditing is good Claude. The biggest problem I see is that the usage levels of LLMs are restricted and that the competent models for code are paid, that most of the open source community are not willing to pay. Then until all that field is balanced and stabilized, audit with the most competent and economical LLM, for minor tasks you can do it in agent with a free (but with caution) lLM xd... Aloha, happy week. :) Thank you.