Digital Craft Workshop

Posted on Sep 2 Originally published at Medium

Don't Trust AI-Generated Code Without Testing It First

I let an AI refactor a database query in my .NET project. The code looked clean. I glanced at it, nodded, committed, and pushed. Then QA pinged me with a stack trace.

The error: InvalidOperationException: Lambda expression used inside Include is not valid.

The AI had used a filtered .Include() with a .Where() clause inside. That syntax shipped in Entity Framework Core 5.0. Our project runs on EF Core 3.1. The AI did not check the version, and on a busy afternoon I did not either.

That moment, staring at the error in the QA environment and knowing I had rubber-stamped AI output without ever running the code, was one of the most embarrassing moments in my recent career. The bug was not complex. I had every tool to catch it and I let it through anyway.

The Task Was Simple

I was fixing an export feature for our Knowledge Base. The original query worked but had a suboptimal structure. It loaded data through ArticleFeedbacks and joined articles on the side. I wanted to flip the approach: start from ArticlesContents, include the related data, and filter feedbacks inline.

The AI rewrote the query into a single LINQ expression with filtered includes:

return await this.database.ArticlesContents
    .Include(a => a.Article).ThenInclude(a => a.Category)
    .Include(a => a.Article).ThenInclude(a => a.ArticlesViewsSimple)
    .Include(a => a.ArticleFeedbacks
        .Where(f =>
            (string.IsNullOrEmpty(source) || f.Source == source) &&
            (!dateFrom.HasValue || f.CreatedAt >= dateFrom.Value) &&
            (!dateTo.HasValue || f.CreatedAt < dateToEndOfDay.Value)))
    .Where(a => a.Article.ProjectId == projectId && !a.IsDeleted && !a.Article.IsDeleted)
    .OrderByDescending(a => a.CreatedAt)
    .ToListAsync();

The filtered Include syntax, where you pass a .Where() lambda inside .Include(), was one of the headline features of EF Core 5.0. It shows up everywhere in modern .NET tutorials and Stack Overflow answers from the last few years.

We are on EF Core 3.1. In 3.1, the only valid argument to .Include() is a simple navigation property expression. No filtering. No logic. Just a => a.ArticleFeedbacks, and the query plan flows from there.

The AI Hallucinated a Newer Framework

AI coding assistants are trained on a lot of code. They have seen millions of EF Core queries, most of them from newer versions. The filtered Include pattern is everywhere in modern .NET content, from official samples to weekend blog posts.

The AI does not open my .csproj file to read the package version. It does not know I have a project pinned to a legacy framework version. It generates what looks correct based on the most common patterns it has learned across the corpus, and most of that corpus assumes a newer EF Core than the one I actually ship against.

The query would compile against a project a few framework versions ahead of mine. In mine, it crashed at runtime as soon as the export endpoint was hit.

I have seen this pattern repeatedly with AI-generated code:

It uses APIs that exist in newer versions of your framework
It imports packages you do not have installed
It uses syntax from a language version your compiler does not support
It assumes configuration or infrastructure that does not exist in your environment

The AI optimizes for code that matches the bulk of its training data. The bulk of that training data sits on newer framework versions than the legacy projects most working developers actually maintain. That mismatch shows up at runtime rather than at compile time, often inside the very request that hits the real database for the first time after the change.

How a version-blind suggestion lands in a legacy project. | Generated with Claude

If you'd rather have the setup that keeps my AI assistant from repeating these mistakes, I put it into The Claude Code Memory Starter — a free email series.

The Real Problem: My Review Process

The AI making a mistake is not where I want to focus. The story is that I did not catch it.

I looked at the code. It made logical sense. The query structure was clean. The filtering was correct. I thought "yeah, that is better than what we had" and moved on.

I did not:

Run the application locally
Hit the export endpoint even once
Write or run a single test against it
Check if filtered Include was supported in our EF Core version

I treated the AI output like code from a senior colleague I trust completely. The AI is no senior colleague. It generates plausible code without knowing my project constraints, and on a quiet afternoon it will hand me a snippet that compiles in a different version of the world.

When QA pinged me, my first reaction was confusion. The query looked fine in the diff. I pulled the branch, ran the export endpoint locally, and got the same stack trace on the first request. The fix took less time than the conversation about the bug.

Load all feedbacks via a plain .Include(a => a.ArticleFeedbacks) and filter them in memory after materialization:

var results = await this.database.ArticlesContents
    .Include(a => a.Article).ThenInclude(a => a.Category)
    .Include(a => a.Article).ThenInclude(a => a.ArticlesViewsSimple)
    .Include(a => a.ArticleFeedbacks)
    .Where(a => a.Article.ProjectId == projectId
        && !a.IsDeleted && !a.Article.IsDeleted)
    .OrderByDescending(a => a.CreatedAt)
    .ToListAsync();

foreach (var content in results)
{
    content.ArticleFeedbacks = content.ArticleFeedbacks
        .Where(f =>
            (string.IsNullOrEmpty(source) || f.Source == source) &&
            (!dateFrom.HasValue || f.CreatedAt >= dateFrom.Value) &&
            (!dateTo.HasValue || f.CreatedAt < dateToEndOfDay.Value))
        .ToList();
}

A 30-second fix for a bug that should never have shipped. Running the code once would have caught it, and one local hit on the export endpoint was the difference between a silent commit and a QA ticket.

This is the same instinct I now put behind every git push. I wrote about wiring a code-reviewer subagent in front of every push to main for exactly this reason. The hook reads the diff before the push actually goes out and flags the kinds of things I would normally only notice during a slow PR review, which gives me a second pair of eyes on every commit even when I am working alone.

The Embarrassment Factor

Let me be honest about why this stung. I have been writing C# and working with Entity Framework for years. If a junior developer on my team had submitted this code in a pull request, I would have caught the filtered Include issue immediately and left a comment on the line.

Because the AI generated it, I treated it differently. There is something about AI-generated code that short-circuits the normal review instinct. The structure is professional. The variable names are sane and the indentation is consistent, so the eye glides over it without ever landing on the lambda inside the Include.

So I let my guard down.

Then QA finds a crash, and I am sitting there explaining that I pushed code I never ran because "the AI wrote it and it looked fine."

That is not a good look for anyone.

It is also not the first time I have had to rethink how I work with AI. I went from treating AI like a faster Google to actually pairing with it on real changes. As my trust grew, the verification gap got bigger instead of smaller. The well-structured looking diffs are the ones I now have to remind myself to actually read, because the polish is what makes them feel safe to skim.

The same diff in my editor and in QA. | Generated with Claude

The AI-as-a-tool workflow behind posts like this is a short email series I run — AI as a Solo Founder’s Tool.

Five Rules I Follow Now

After this incident I changed how I work with AI coding assistants. These are not theoretical guidelines. They are rules I follow because I got burned in QA in front of the whole team and I would prefer not to repeat the experience.

Rule 1: Always run it. No exceptions. If the AI changes a query, I hit that endpoint locally before committing. If it modifies a component, I open the browser and click through the happy path at least once. The minimum bar is that the code runs end-to-end on real data without throwing.

Rule 2: Check version-sensitive APIs. When the AI uses a method or pattern I am not certain about for my current framework version, I verify. A quick search for "EF Core 3.1 filtered include" would have saved me in 30 seconds. The same is true for anything tied to a package version, an SDK version, or a runtime that the AI cannot see from the prompt.

Rule 3: Treat AI output like a PR from a new hire. Smart, probably correct, but worth a second look. I read every line and question the assumptions baked into the diff. If I would push back on a teammate's PR for missing a test on a behavior change, I push back on the AI's output the same way.

Rule 4: Don't let clean code fool you. The most dangerous AI-generated bugs are in code that looks perfect. Syntax errors are easy to catch in review. Version incompatibilities in well-structured code slip through because the reviewer brain says "this looks like what a senior would write" and quietly stops working.

Rule 5: Own the code. The moment I commit it, it is mine. "The AI wrote it" is not an excuse anyone wants to hear in a postmortem. I review the diff like I wrote it, because once it is in the repo, I did.

I treat these like a seatbelt: boring, repeatable, applied even when I am tired and the laptop fan is loud. My AI workflow also changed how I write comments, but these five rules are the ones that actually live in muscle memory now. I do not always remember them at the start of a task. I remember them at the moment my finger hovers over git push, which turns out to be enough.

Faster Is Only Faster If It Ships Clean

AI coding assistants are genuinely useful. I use them every day. They save time on the boilerplate I would otherwise type by hand, and they let me move through a sprint at a pace I could not sustain alongside a side project and a small daughter at home.

Moving faster only counts if I am moving in the right direction. Shipping broken code faster is a liability dressed up as a productivity gain.

The developer community is going through an awkward phase with AI tools. We are past the "wow, it can write code" excitement and not yet at the point where we have built reliable habits for verifying AI output. The tools are good enough to be dangerous, and the workflows around them have not fully caught up to that reality.

Every minute I save by skipping a test of AI-generated code, I pay back with interest when something breaks at runtime, because the AI does not know my project the way I do.

When I weigh the cost of running the code locally against the cost of a stack trace landing in QA the next morning, the local run is always cheaper. So I run it. Then I push.

\→ Get it free: AI as a Solo Founder’s Tool, a 5-part email series.

External Sources

EF Core eager loading docs — official Microsoft docs on what is valid inside .Include() per version.
EF Core 5.0 announcement — Microsoft blog post listing filtered Include as a headline feature.
dotnet/efcore #1833 — the long-running GitHub issue that became filtered Include in EF Core 5.
ASP.NET Core integration tests — official guide to the kind of test that would have caught the bug in one run.

Daniel Rusnok is a solo builder shipping content, SaaS, and games.

If you want that habit as a few short emails instead of a hard afternoon, The Claude Code Memory Starter walks you through it — free.

I build small tools and kits for solo creators. You can find them here: https://danielrusnok.gumroad.com

DEV Community