DEV Community: Jaskaran Singh

AI Agents Are Shipping Features Without You. Now What?

Jaskaran Singh — Wed, 22 Apr 2026 00:12:59 +0000

Jaskaran Singh — Senior Software Engineer, AI Trainer

A few weeks ago I watched an agent open a GitHub issue, write the fix, run the tests, and open a pull request. No human typed a line of code. The PR passed review.

I didn't find this inspiring. I found it genuinely disorienting. I say that as someone who trains AI models for a living and is currently building an agent of my own.

If you're a software engineer in 2026 and you haven't had that moment yet, you will. Agentic AI is being called the third seismic shift in software engineering this century, after open source and DevOps. That framing might be overblown. It might not be. Either way, something real is happening and it's worth thinking clearly about instead of panicking or dismissing it.

The Numbers Stopped Being Theoretical

Source: Unsplash — Luke Chesser

A survey of nearly 1,000 engineers published in early 2026 found that 95% use AI tools at least weekly, 75% use AI for half or more of their engineering work, and 55% regularly use AI agents. That last number is the one that matters. Copilots have been mainstream for two years. Agents are different.

A copilot suggests. An agent acts. It reads your codebase, decides what to do, does it, checks whether it worked, and tries again if it didn't. The feedback loop is closed without you in it.

In 2025, coding agents moved from experimental tools to production systems shipping real features to real customers. In 2026, single agents are becoming coordinated teams of agents.

I've been watching this from an unusual angle. My job involves evaluating AI-generated code for quality: finding the failure modes, writing the rubrics, doing the multi-turn reviews. At the same time I'm building a Python agent that monitors the OINP immigration portal and pushes Telegram alerts whenever a new Masters Graduate stream draw drops. Two different relationships with the same technology, and both have given me a clearer picture than I'd have from either side alone.

What Agents Are Actually Good At

Agents handle implementation tasks well when the problem is well-scoped and verifiable. "Add pagination to this endpoint." "Write tests for this module." "Refactor this class to use dependency injection." Tasks with clear success criteria: the code runs, the tests pass, the interface contract is unchanged. The agent can verify its own work.

Quality still varies. My evaluation work confirms what engineers describe: intuitions for delegation develop over time. People hand off tasks that are easily verifiable or low-stakes. That intuition is real and it matters. Knowing what to delegate is itself a skill now.

Where agents fall apart is anything requiring judgment about what the right problem even is. An agent given an ambiguous brief will confidently solve the wrong version of it. I've seen this pattern repeatedly, not as an occasional edge case but as a consistent failure mode when the task specification has gaps. The agent doesn't ask for clarification. It infers, fills in, and proceeds. Sometimes the inference is right. When it's wrong, it's wrong in ways that are coherent and hard to catch. That's the part that should make you nervous.

The Shift That's Actually Happening to Engineering Teams

Gartner predicts 80% of organizations will evolve large software engineering teams into smaller, AI-augmented teams by 2030. The trajectory is already visible. Teams that used to need eight engineers to maintain a product are running it with four. Not because the other four got fired, but because agent-assisted output per engineer went up enough that the headcount math changed.

The pattern emerging in 2026: software development is moving toward human expertise focused on defining problems worth solving while AI handles the tactical implementation work.

That framing is mostly right but it undersells something. "Defining problems worth solving" sounds clean and strategic. In practice it means writing a spec precise enough that an agent doesn't go off the rails, reviewing agent output at a level that catches subtle correctness issues, and making architecture decisions that hold up when the agent starts filling in implementations you didn't anticipate.

Those are all hard skills. They're also different from the skills that got most of us into engineering. We learned by writing the implementation ourselves. The feedback loop of "I wrote this, it broke, I understand why" is how you build the mental models that make good judgment possible. Whether that judgment transfers cleanly to directing agents at tasks you've never done yourself is an open question. I don't think anyone knows yet.

What This Means If You're Mid-Career

I'm five years in. I've shipped production Android apps, done fintech work, and I'm now working at the AI training layer. The people who seem least threatened by this shift share one thing: they understand systems, not just syntax.

A developer who knows Kotlin and can write Jetpack Compose components is in a different position than one who understands why coroutine cancellation works the way it does, when a ViewModel scope is the wrong choice, and what the architectural consequences of a particular state management approach are three features down the road. The first kind of knowledge is increasingly delegatable. The second is what you need to review what the agent produces.

This is not a comfortable message. It basically says the work that builds deep knowledge is being automated before you've had a chance to accumulate it through repetition. That's a real problem for junior developers and I don't have a clean answer to it. Engineers who actively seek out the "why" behind every pattern they use, even when an agent handed them that pattern, will pull ahead of those who treat agent output as a black box. That's my best guess.

The Security Problem Nobody Talks About

Source: Unsplash — Lewis Kang'ethe Ngugi

Agentic coding is changing security in two directions. As models get more capable, building security into products gets easier. The same capabilities that help defenders help attackers too.

There's a third direction worth adding from my evaluation work: agents introduce security risks through confident implementation of insecure patterns. An agent writing a data pipeline reaches for the most direct path to working code. Input sanitization, parameterized queries, credential management, error handling that doesn't leak internals: these require deliberate thought. Agents do them inconsistently.

The more autonomous the coding pipeline, the more critical it is to have security review that isn't the same agent that wrote the code. I've flagged SQL injection vulnerabilities in agent-generated Python and credential handling issues in agent-generated Kotlin. The code was functionally correct. It would have passed a cursory review. It shouldn't have shipped.

Why I'm Still Building Agents

None of this made me stop building the OINP monitoring bot. It made me more deliberate about it.

The thing I'm building isn't trying to do something clever. It checks a government webpage on a schedule, parses the draw results, compares against the last known state, and fires a Telegram message if something changed. The agent part is the parsing logic: handling inconsistencies in how the page is structured, dealing with cases where the data format shifts slightly. That's a good fit for what these tools are actually good at.

The immigration system in Canada is opaque in ways that are genuinely stressful for people on it. If a monitoring tool reduces that stress even slightly, it's worth the weekend. The judgment about what's worth building and why is still entirely mine.

That's probably the honest answer to "now what." The judgment work is still yours. The implementation is increasingly negotiable.

Jaskaran Singh is a Senior Software Engineer working in AI training and evaluation, with production experience in Android development using Kotlin and Flutter. Currently building a Python-based OINP immigration monitoring agent.

LinkedIn · Portfolio

I Grade AI Code for a Living. Here's What Nobody Talks About.

Jaskaran Singh — Tue, 21 Apr 2026 23:56:05 +0000

Jaskaran Singh — Senior Software Engineer, AI Trainer

I've spent the last year doing something most engineers haven't: reading AI-generated code all day and deciding whether it's actually good.

Not "does it compile." Not "did the tests pass." Good as in, would I be comfortable shipping this to production at 2am on a Friday if something went wrong.

The answer, more often than people want to admit, is no.

I use LLMs myself. But after evaluating enough AI-generated code across Python, Java, Kotlin, and C/C++, I know the failure modes aren't random. They follow patterns. And once you see them, you can't unsee them in AI code or your own.

The Job Nobody Has a Good Title For

My official role is AI Trainer. What that actually means: I'm a human in the RLHF loop.

Reinforcement Learning from Human Feedback works by having engineers like me evaluate model outputs against structured rubrics, then rank and rewrite them so the model learns what "better" looks like. I write adversarial prompts to expose failure modes. I do multi-turn code reviews, meaning I follow an entire back-and-forth between a user and a model across five or ten turns, and assess whether the reasoning held up or quietly drifted off the rails somewhere in the middle.

Less "AI whisperer." More "very opinionated senior reviewer who never runs out of things to flag."

The Pattern That Bothers Me Most

There's a category of bug I call "confident and wrong." The code compiles. It's readable. The variable names are sensible. It even has a comment explaining what it does. And it's still wrong. Not obviously wrong, but wrong in the way that only shows up under load, or with a specific input type, or after three other things happen first.

Here's a real example. Prompt was something like: "Write a function to fetch user details and cache the result."

The model produced:

object UserCache {
    private val cache = HashMap<String, User>()

    fun getUser(userId: String, fetchFn: () -> User): User {
        return cache.getOrPut(userId) { fetchFn() }
    }
}

Clean. Concise. Totally broken in a concurrent environment.

HashMap isn't thread-safe. Two coroutines calling getOrPut simultaneously on the same key can corrupt the map. The model didn't add a mutex, didn't suggest ConcurrentHashMap, didn't even mention the assumption that this runs single-threaded. It just wrote code that works in the demo and fails in production.

The correct version uses ConcurrentHashMap or wraps access with a Mutex if you need atomic get-or-fetch semantics:

object UserCache {
    private val cache = ConcurrentHashMap<String, User>()
    private val mutex = Mutex()

    suspend fun getUser(userId: String, fetchFn: suspend () -> User): User {
        cache[userId]?.let { return it }
        return mutex.withLock {
            // double-checked after acquiring lock
            cache.getOrPut(userId) { fetchFn() }
        }
    }
}

The model's version would pass code review at most places. That's what worries me.

The Edge Case Problem is Structural, Not Random

After a few hundred evaluations, I stopped thinking of missed edge cases as oversights. They're structural. LLMs optimize for the problem as stated. If the prompt doesn't mention null inputs, concurrent access, or network timeouts, the model won't think about them either.

Good engineers treat those as implied. You don't wait to be asked "what if this list is empty." You just handle it.

Here are the categories where models fail most consistently:

Concurrency. Single-threaded assumptions that explode under real-world load. The HashMap example above is the most common flavor.

Failure state propagation. Functions that catch exceptions and return null or false, then callers that don't check the return value, and the whole chain silently fails. The model gets each function right in isolation. It gets the composition wrong.

Resource cleanup. Network connections, file handles, database cursors left open because the happy path worked and nobody wrote the finally block or used the right scoping construct.

Behavioral drift across turns. In turn 1, the model sets up a class a certain way. By turn 4, after a few "can you refactor this" prompts, it has made changes that contradict the original design without acknowledging it. The code still runs. The architecture is now inconsistent in ways that will cause problems in six months.

What I Actually Look For in a Code Review

My rubric has eight criteria. The ones that surface the most issues:

Correctness under adversarial input. Not "does it work with the example." Does it work when the input is empty, null, malformed, enormous, or concurrent? I'll trace through a model's code in my head with the worst inputs I can think of before scoring it.

Explicitness of assumptions. Code that works is not the same as code that communicates its constraints. If a function assumes its input is sorted, that needs to be in a comment, a precondition check, or the function name. The model almost never does this unprompted.

Error handling that means something. There's a specific anti-pattern I call "error theater":

// This is not error handling. This is error cosplay.
try {
    val result = riskyOperation()
    return result
} catch (e: Exception) {
    Log.e("TAG", "Something went wrong")
    return null
}

It looks like error handling. It isn't. The caller has no information. The system has no way to recover. The log message gets ignored. Good error handling changes what the caller can do. It doesn't just muffle the crash.

Security surface. SQL construction via string interpolation, credentials in code comments, user input passed to shell commands without sanitization. These come up. Not constantly, but often enough that I check every time.

The Skill That Transferred Back

I didn't expect this job to change how I write code. It did.

Spending eight hours a day articulating why something is wrong, not just flagging it but writing a clear explanation that a model can actually learn from, builds a habit of internal interrogation that's hard to turn off.

Now, before I submit a PR, I run my own rubric. Is this thread-safe? What happens on retry? Who owns cleanup? Does this function do what its name says, or has it quietly acquired a second responsibility?

That last one is underrated. Functions that do two things are where bugs live. The AI writes them constantly because function names get generated from the prompt context, and prompts often have two goals. "Fetch and validate" is two functions pretending to be one.

Where AI Code Actually Shines

I've been critical, so to be fair.

AI-generated code is genuinely good at boilerplate. Serialization logic, configuration parsing, test scaffolding, adapters between interfaces that differ only in naming. Tedious work that models handle well. If I ask for a Room database entity with a DAO and a repository, the output is usually solid and saves thirty minutes.

// This kind of scaffolding? Models nail it.
@Entity(tableName = "users")
data class UserEntity(
    @PrimaryKey val id: String,
    val name: String,
    val email: String,
    val createdAt: Long = System.currentTimeMillis()
)

@Dao
interface UserDao {
    @Query("SELECT * FROM users WHERE id = :userId")
    suspend fun getUserById(userId: String): UserEntity?

    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertUser(user: UserEntity)
}

Models are also good at surfacing options I'd forgotten about. Not because they know my codebase, but because they've seen enough code to suggest a StateFlow where I was reaching for LiveData, or use runCatching in a context where it genuinely fits.

The mistake is treating it as something that reasons about your system. It doesn't know your system. It knows patterns. Those overlap most of the time and fail in ways that aren't obvious the other times.

Why I Wrote This

A few months ago I started noticing that engineers I respect were shipping AI-generated code without reviewing it seriously. Not because they're lazy. Because the code looked fine. That's the problem. It's calibrated to look fine.

The engineers who work well with AI tooling treat it the way experienced engineers treat a junior developer: capable, useful, not fully trusted without review, and prone to specific failure patterns you learn over time.

That framing changed how I work with it. I think it'll change how you do too.

Jaskaran Singh is a Senior Software Engineer working in AI training and evaluation. Previously built Android fintech apps at Comviva Technologies and Talentica Software. Currently building a Python-based OINP immigration monitoring bot on the side, because immigration status shouldn't require manually refreshing government websites.

Find me on LinkedIn or at my portfolio.