The Narrow Sidewalk: Where Developers Stand When Agents Write the Code

When coding agents such as Claude Code and Codex became good enough to take whole tasks, the first reaction in our industry was very loud: developers would be replaced, and soon. A year later the picture is more interesting than that. Agents write a large share of the code now, and that is not going back. But it is also becoming clear that someone still has to know what correct code looks like, and that this someone is a developer with a critical mindset. In my view we are close to the point where the balance between agents and humans becomes visible, and the honest description of that balance is a narrow sidewalk.

On one side of the sidewalk is the developer who keeps working as before, writing every line by hand and treating agents as a toy. On the other side is the developer who hands everything to the agent and stops learning, because the code compiles and the tests are green. Both of them fall off. The path in the middle is narrow, and in this article I want to describe what walking on it actually requires, with real code.

What the Numbers Say About Both Sides

METR ran a randomized controlled trial in early 2025 with 16 experienced open-source developers working on 246 real issues in their own repositories. With AI tools they took 19% longer. The more interesting part is the perception: before the study they expected AI to make them 24% faster, and even after the slowdown they still believed it had made them 20% faster. So our feeling about how much the agent helps is not a reliable measurement.

In February 2026 METR reported that it had to change the design of its follow-up study, mainly because many developers no longer wanted to work without AI at all, and that developers are likely faster now than in early 2025. Taken together, these two results describe the sidewalk very well. Going back to the old way of working is not realistic, but trusting our own impression of the agent's output is also not safe.

Green Checks, Wrong Meaning

When coding agents work on real systems, one failure shows up again and again. The code type-checks, the tests pass, the build is green, and the behaviour is still wrong, because the mistake is in the meaning and not in the syntax. A check can only verify what somebody told it to verify, and the agent writes both the code and, very often, the checks.

Let us take a concrete example. An endpoint asks an external access service which permissions a user has, and refuses the request when the user has none. A typical generated version looks like this:

public interface IPermissionClient
{
    Task<IReadOnlySet<string>> GetPermissionsAsync(string userId, CancellationToken ct);
}

public sealed class NaiveAccessCheck(IPermissionClient client)
{
    public async Task<IResult> CheckAsync(string userId, CancellationToken ct)
    {
        IReadOnlySet<string> permissions;
        try
        {
            permissions = await client.GetPermissionsAsync(userId, ct);
        }
        catch (HttpRequestException)
        {
            permissions = new HashSet<string>(); // fail closed
        }

        return permissions.Count == 0
            ? Results.StatusCode(StatusCodes.Status403Forbidden)
            : Results.Ok();
    }
}

It compiles, it reads naturally, and it even carries a comment about failing closed, which is the correct security instinct. A test that gives the stub an empty permission set and expects 403 passes. The problem is that an outage of the access service also produces an empty set, so during an outage every user gets a 403 and the UI tells them they have no access. They contact support about permissions, and nobody looks at the access service, which is the thing that is actually broken.

Seeing this requires knowing that "the user is refused" and "the authority could not be asked" are two different answers, and that they lead to different status codes, different messages and different alerts. No type and no test inside this class encodes that knowledge. The fix is to make the difference impossible to lose:

using System.Diagnostics;

public abstract record AccessDecision
{
    public sealed record Allowed(IReadOnlySet<string> Permissions) : AccessDecision;
    public sealed record Denied(string Reason) : AccessDecision;
    public sealed record Unavailable(Exception Cause) : AccessDecision;
}

public sealed class AccessCheck(IPermissionClient client)
{
    public async Task<AccessDecision> CheckAsync(string userId, CancellationToken ct)
    {
        try
        {
            var permissions = await client.GetPermissionsAsync(userId, ct);
            return permissions.Count == 0
                ? new AccessDecision.Denied("No permissions granted")
                : new AccessDecision.Allowed(permissions);
        }
        catch (HttpRequestException ex)
        {
            return new AccessDecision.Unavailable(ex);
        }
    }
}

public static class AccessResults
{
    public static IResult ToHttpResult(AccessDecision decision) => decision switch
    {
        AccessDecision.Allowed => Results.Ok(),
        AccessDecision.Denied d => Results.Problem(d.Reason, statusCode: StatusCodes.Status403Forbidden),
        AccessDecision.Unavailable => Results.Problem("Access service unavailable",
            statusCode: StatusCodes.Status503ServiceUnavailable),
        _ => throw new UnreachableException()
    };
}

Now a refusal and an outage are different types, and the switch makes the caller handle each one. An outage becomes a 503, which a monitoring system will page on, instead of a 403, which nobody pages on.

Fail-Open Defaults Hide Behind Healthy Deployments

The second pattern is a check that passes because it was never active. Verification code is often written so that it is skipped when its settings are missing, which is convenient on a developer machine:

using System.ComponentModel.DataAnnotations;
using Microsoft.Extensions.Options;

public sealed class TokenOptions
{
    [Required] public string Issuer { get; set; } = "";
}

public sealed class NaiveTokenCheck(IOptions<TokenOptions> options)
{
    public bool IsTrusted(string? issuer)
    {
        if (string.IsNullOrEmpty(options.Value.Issuer))
            return true; // verification not configured yet

        return issuer == options.Value.Issuer;
    }
}

If the deployment never passes the issuer setting, every token is trusted, the service starts, health checks are green and nothing in the logs looks wrong. Only a negative test against the deployed service, a request that should be refused, shows that nothing is checked. In .NET the better default is to refuse to start at all when security settings are missing:

public static class TokenOptionsSetup
{
    public static IServiceCollection AddTokenOptions(this IServiceCollection services)
    {
        services.AddOptions<TokenOptions>()
            .BindConfiguration("Token")
            .ValidateDataAnnotations()
            .ValidateOnStart();
        return services;
    }
}

ValidateOnStart runs the data annotation validation when the host starts, so a missing issuer becomes an OptionsValidationException at startup instead of a silent hole in production.

Test Both Directions

Both fixes are only worth something if the tests prove the negative case as carefully as the positive one. These are the tests that matter here, and the outage test is the one a generated suite usually leaves out:

using Microsoft.AspNetCore.Http.HttpResults;
using Microsoft.Extensions.Options;

sealed class StubClient : IPermissionClient
{
    private readonly IReadOnlySet<string>? _permissions;
    private readonly Exception? _error;

    public StubClient(IReadOnlySet<string> permissions) => _permissions = permissions;
    public StubClient(Exception error) => _error = error;

    public Task<IReadOnlySet<string>> GetPermissionsAsync(string userId, CancellationToken ct) =>
        _error is null
            ? Task.FromResult(_permissions!)
            : Task.FromException<IReadOnlySet<string>>(_error);
}

public class AccessCheckTests
{
    [Fact]
    public async Task Refused_user_gets_403()
    {
        var check = new AccessCheck(new StubClient(new HashSet<string>()));
        var result = AccessResults.ToHttpResult(await check.CheckAsync("u1", default));
        Assert.Equal(403, ((IStatusCodeHttpResult)result).StatusCode);
    }

    [Fact]
    public async Task Outage_gets_503_not_403()
    {
        var check = new AccessCheck(new StubClient(new HttpRequestException("down")));
        var result = AccessResults.ToHttpResult(await check.CheckAsync("u1", default));
        Assert.Equal(503, ((IStatusCodeHttpResult)result).StatusCode);
    }

    [Fact]
    public async Task Missing_token_settings_fail_at_startup()
    {
        var builder = Host.CreateApplicationBuilder();
        builder.Services.AddTokenOptions();
        using var host = builder.Build();
        await Assert.ThrowsAsync<OptionsValidationException>(() => host.StartAsync());
    }
}

All the code in this article was run on .NET 10 before publishing, including the naive versions, to confirm that the first one really answers 403 during an outage and the second one really trusts any issuer when it is not configured.

The Same Patterns, Again and Again

The two examples above are not special. In practice, the mistakes that survive into production with agent-written code tend to follow a small number of shapes:

• Tests mock exactly the boundary where the bug lives. Each side of an integration is tested against a stub of the other side, so a mismatch between them, for example 401 where 403 was meant, is invisible to both suites.
• The premise is wrong, and every check agrees with it. A feature flag read with the wrong polarity, or a coordinate system assumed to point the other way, produces code and tests that are perfectly consistent with each other and wrong about the world.
• A green check measures less than its name. A release step joined with ; instead of && ignores the CI result it was supposed to wait for. A test that discovers components with a type check can silently skip the ones that do not match, so coverage shrinks while every run stays green.
• The producer is verified instead of the consumer. Checking a value in the data you serve proves nothing if the consumer reads a different copy of it.
• Readable and proportionate are not testable. A page can be correct and fully tested and still show thirty nearly identical rows for one file.
• A guardrail stops the agent. When a policy blocks an action, the right move is to give the decision back to a human, not to find another route around the block.

What these have in common is that none of them is caught by more automation alone. They are caught by someone who knows what the system is supposed to mean: the domain, the architecture, and who reads what.

Walking the Sidewalk

So what does staying on the sidewalk look like in practice? In my view it has two parts, and both of them need continuous learning.

The first part is to keep learning how code works. Not because we will write most of it by hand again, but because the knowledge of HTTP semantics, failure modes, type design and the platform we work on is exactly what lets us recognise when a plausible answer is wrong. A developer who stops learning here can no longer review the agent, and then the agent is not supervised at all.

The second part is to learn how to work with the agent: how to give it the context and constraints it cannot infer, how to put checks around it that test the negative case, and how to critique its output instead of accepting a green build as proof. That is a skill of its own, and it is still very young.

The developers who will do well in the next years are, I think, the ones who keep both of these going at the same time. The sidewalk is narrow, but it is the only path that does not end in either irrelevance or a production incident that nobody can explain.