Over the past few months, I ran AI-powered code reviews on 100 open source repositories across GitHub — everything from weekend side projects to repos with thousands of stars. The goal was simple: find out what patterns emerge when you let AI analyze real-world codebases at scale.
The results surprised me. Not because the code was bad — most of it was functional and well-intentioned. But consistent patterns of neglect showed up again and again, regardless of language, framework, or project size. Here are the five most common issues I found, what they cost teams in practice, and how you can catch them before they compound.
1. Inconsistent or Missing Error Handling
This was the single most common issue — present in roughly 73 out of 100 repos.
Developers handle errors carefully in some parts of the codebase and then completely ignore them in others. You'll find a beautifully structured try-catch in one module and a raw, unhandled promise rejection two files over. The AI flagged patterns like empty catch blocks, swallowed exceptions, and missing error boundaries in React components.
Why it matters: Inconsistent error handling is the number one source of those "works in development, crashes in production" bugs. When a function silently fails, the error surfaces somewhere completely unrelated, and debugging becomes a nightmare.
The fix: Establish a project-wide error handling strategy and enforce it. Use a shared error class, standardize logging, and make sure every async operation has explicit failure handling.
2. Hardcoded Secrets and Configuration Values
I expected this in small projects. I did not expect it in repos with 2,000+ stars.
The AI flagged API keys, database connection strings, and third-party service tokens committed directly into source code in 41 repos. Some had .env.example files but still had real credentials scattered in config files or test fixtures. A few had AWS keys sitting in plain text inside Docker Compose files.
Why it matters: Hardcoded secrets in public repos get scraped by automated bots within minutes of being pushed. Even if you rotate the key immediately, the damage window is real. And in private repos, hardcoded config makes deployments fragile and environment-specific.
The fix: Use environment variables. Add secret-scanning tools like Gitleaks or GitHub's built-in secret scanning to your CI pipeline. Review .gitignore before your first commit, not after.
3. Dead Code and Unused Dependencies
About 58 repos carried significant dead weight — commented-out blocks, unused imports, deprecated functions that nobody removed, and package.json files bloated with dependencies that nothing in the codebase actually references.
One React project had 14 unused npm packages. A Python repo had three entire modules that were imported nowhere. The AI caught these instantly, but a human reviewer scanning a 200-file PR would likely miss them.
Why it matters: Dead code isn't harmless. Unused dependencies increase your attack surface and slow down installs and builds. Commented-out code creates confusion about what's intentional and what's abandoned. It also inflates your bundle size, which directly impacts performance for frontend projects.
The fix: Run tree-shaking analysis and dead code detection as part of your CI. Tools like depcheck (Node.js) or vulture (Python) automate this. Make removing dead code a regular maintenance habit, not something you do once a year.
4. Missing or Outdated Documentation
This one is almost universal. 67 repos had README files that no longer matched the actual setup process. Functions with complex logic had zero comments. API endpoints lacked any description of expected inputs and outputs.
The AI identified functions where the docstring described parameters that no longer existed, or where return types had changed but the documentation still referenced the old structure.
Why it matters: Poor documentation kills contributor onboarding. The GitHub blog recently highlighted "AI slop" as a growing problem in open source — low-quality contributions from developers who don't understand the codebase they're modifying. Outdated docs make this worse because even well-intentioned contributors start from a wrong mental model.
The fix: Treat documentation as code. Include doc checks in your review process. At minimum, ensure your README covers installation, usage, and contribution guidelines accurately. Flag docstrings that reference parameters or return values that no longer exist.
5. No Test Coverage for Critical Paths
This was the most dangerous pattern. 52 repos had some test coverage, but the tests covered the easy, predictable paths — the happy path through a function, the default case in a switch statement.
Edge cases, error paths, and boundary conditions were almost universally untested. The AI flagged functions handling financial calculations with zero tests, authentication flows tested only for the success case, and data validation functions that never tested malformed input.
Why it matters: Tests that only cover the happy path give you false confidence. Your CI goes green, you merge with confidence, and then a user submits an empty string where you expected a UUID and your application crashes. The value of testing is directly proportional to how well your tests cover the cases that actually break things.
The fix: Focus test coverage on failure modes, not success modes. Require tests for error branches, boundary values, and edge cases before merging. Use mutation testing to check whether your existing tests actually catch real bugs.
What I Learned from All 100 Reviews
The most important takeaway isn't any single issue — it's that these problems are systematic, not random. Every codebase had at least two of the five issues above. The projects with fewer issues weren't written by better developers; they were projects with better review processes.
Manual code review catches business logic errors and architectural concerns well. But it consistently misses the repetitive, pattern-based issues listed here because human reviewers have limited attention and time. This is exactly where AI code review earns its keep — it doesn't get tired, doesn't skip files, and doesn't forget to check the error handling in that utility function buried three directories deep.
How I Ran These Reviews
I used GetCodeReviews, an AI-powered code review tool I built specifically for this kind of analysis. It connects to GitHub repos and uses Claude to analyze code for quality issues, security risks, and maintainability concerns.
If you want to try it on your own repos, you can start with 10 free reviews per month on the free plan. The Pro plan at $29/month gives you 200 reviews, which is enough to cover most active projects. For teams, there's a $99/month plan with shared reviews across up to 5 members.
The tool doesn't replace human reviewers — it handles the mechanical checks so your team can focus on architecture, logic, and design decisions that actually require human judgment.
Final Thoughts
Code quality in open source isn't declining — it's just not keeping pace with the speed at which code is being written. AI-generated code has increased PR volumes significantly across major projects, but review capacity hasn't scaled with it. The result is a growing gap between what gets shipped and what gets properly verified.
Automated code review won't close that gap entirely, but it catches the patterns that humans reliably miss. If you maintain an open source project or contribute to one, run an AI review on it. The results might surprise you the same way they surprised me.
Top comments (0)