DEV Community: Virendra Vyas

My API Was Logging People's National Insurance Numbers. I Never Told It To.

Virendra Vyas — Tue, 25 Aug 2026 10:00:00 +0000

I never wrote a single line of code that logged anyone's National Insurance Number. That's what made this one uncomfortable to find.

GovBridge sits in front of several UK government APIs, and some of those endpoints take genuinely sensitive identifiers as part of the request - a NINO for an Income Tax lookup, a VAT registration number, a Unique Taxpayer Reference. Reasonable enough. That's what the underlying government APIs actually require.

The problem wasn't the request. It was everything that happened after the request, without me ever asking it to.

Where it was actually leaking

Most of these identifiers weren't sitting quietly in a request body, where sensible logging conventions tend to leave things alone. They were in the URL path -

/individuals/{nino}/income-tax/...

because that's how the government's own APIs are structured, and a gateway sitting in front of them naturally mirrors that shape.

And request paths get logged. By default. Almost everywhere. Your web server logs them. Your framework's request logging middleware logs them. Any APM or monitoring tool you've wired in logs them. None of that logging was written with "this specific path segment happens to be someone's tax identifier" in mind, because generic logging infrastructure has no way to know that.

So without a single deliberate Log.Info(nino) anywhere in my codebase, a real person's National Insurance Number was sitting in plain text in application logs, purely as a side effect of URL structure.

Why this is so easy to miss

Nobody sets out to log PII. It happens because logging infrastructure and API design are built by different parts of your brain, on different days, and they never actually talk to each other. Your route design is driven by REST conventions and what the upstream API expects. Your logging setup is driven by "I want visibility into what's happening in production." Neither decision, on its own, looks wrong. The collision between them is invisible until you specifically go looking for it.

And this isn't a government-API-specific problem. Any system where a sensitive identifier - an account number, a medical record ID, anything regulators would call out, ends up as a path segment or query parameter has the same exposure, completely independent of what industry you're in.

The fix

The real fix isn't "stop putting sensitive data in URLs" sometimes you don't control that, especially when you're mirroring an upstream API's own routing structure, the way GovBridge does.

The fix is treating log redaction as its own explicit layer, not an afterthought bolted onto whatever logging framework you're using. Concretely, that meant building a dedicated redaction component - a SensitiveDataRedactor - that knows the shape of the identifiers that matter (NINO, VRN, UTR, and anything added to that list going forward) and scrubs them from anything about to be written to a log, regardless of whether they showed up in a path, a query string, or a header.

The important design choice: this lives centrally, in the shared core of the application, not scattered across individual controllers or endpoints. A rule like "don't log this identifier" is exactly the kind of thing that's useless if you have to remember to apply it every time you add a new endpoint. It only works if it's structurally impossible to forget.

Why I'm writing this up

Because I suspect a meaningful number of people reading this have some version of this exact gap sitting in their own systems right now, undetected, for the same reason mine was: it doesn't look like a bug from either side. The routing looks fine. The logging looks fine. It's only wrong when you look at both at once.

If you've got sensitive identifiers anywhere in your URL structure, and if you're integrating with any regulated API, you probably do, it's worth a specific, deliberate check: open your actual logs, not your code, and see what's really sitting in them.

I Rate-Limited My Own API. Then I Found a Way to Bypass It Myself.

Virendra Vyas — Sun, 23 Aug 2026 22:39:53 +0000

Rate limiting feels like a solved problem. You pick a partition key, set a window, wire up the middleware, and move on. I did exactly that on GovBridge - a gateway sitting in front of several UK government APIs, and for a while I genuinely thought it was done.

Then I went looking for edge cases in my own system, the way you're supposed to before you trust something with production traffic. And I found one.

The bug

My rate limiter was partitioning requests by the raw x-api-key header value - a completely reasonable-sounding default. Different key, different bucket, different limit. Simple.

Except: nothing was actually checking whether that key was valid before the rate limiter ran. Authentication happened later in the pipeline. Which meant the rate limiter was perfectly happy to hand out a fresh limit bucket to any string you put in that header - real key or not.

If you wanted to bypass the limit entirely, you didn't need to break anything clever. You just needed to change the header value on every request. Each "new" fake key got its own untouched allowance. The limiter was doing exactly what I told it to do - it just wasn't protecting what I actually wanted protected.

This is the kind of bug that doesn't show up in normal testing, because normal testing uses real keys. It only shows up when you deliberately try to misuse your own system the way someone with bad intentions eventually would.

Why this is an easy trap to fall into

The instinct to partition by "whatever's in the header" feels natural, because it's the first thing available and it looks like identity. But a header value isn't identity — it's just a string someone sent you. Identity is what your authentication layer confirms after checking that string against something real.

Rate limiting on the wrong side of that line protects a piece of text, not a user.

The fix

Two changes, and they had to happen together:

Authentication moved ahead of rate limiting in the middle-ware pipeline - so by the time the limiter runs, the request has already been through validation.
The rate limiter's partition key changed from the raw header to the authenticated claim - the identity that validation actually confirmed, not the string that arrived on the wire.

Now an invalid key doesn't get a fresh bucket to exploit, it gets rejected by authentication before rate limiting ever sees it. And a real, valid key can't be rotated to dodge its own limit, because the limiter is now counting the thing being limited, not an arbitrary label attached to the request.

Why I'm writing this up

Not because the fix is clever, it's genuinely simple, in hindsight almost embarrassingly so. I'm writing it up because "partition by header, not by verified identity" is an easy default to reach for, and I don't think I'm the only person who's reached for it without noticing the gap.

If you've built rate limiting or throttling into anything - an API, a login flow, a signup form — it's worth a specific check: is the thing you're limiting actually confirmed to exist, or are you just limiting a string that showed up?

Cheaper to find that gap yourself than to have someone else find it for you.

GovBridge UK: Why I Built a Unified API Gateway for UK Government Data (HMRC, Companies House, DVLA, Land Registry)

Virendra Vyas — Fri, 14 Aug 2026 15:59:54 +0000

If you've ever integrated with more than one UK government API, you already know the pain: each one has its own auth flow, its own error format, its own rate limits, its own quirks in sandbox vs production. HMRC alone has a dozen+ API families under Making Tax Digital, each slightly different from the last.

I got tired of solving this from scratch on every project, so I built GovBridge UK — a single gateway that normalizes HMRC MTD, Companies House, DVLA, and Land Registry behind one consistent interface.

What it handles:

A single error envelope across all four government APIs, instead of learning four different failure formats OAuth token management and refresh handled once, not per-integration 40+ working HMRC endpoints already wired up and sandbox-tested (Income Tax MTD, VAT, CIS, PAYE, Property/Savings income, and more) A developer portal (React) so you can explore and test endpoints without reading four separate sets of gov.uk docs

Stack: .NET 8/10, ASP.NET Core, PostgreSQL, Redis for token/session caching, React/TypeScript for the portal.

It's still evolving - security hardening and the CT600/GovTalk integration are in progress. If you've built anything against HMRC's MTD APIs, Companies House, or DVLA, I'd genuinely like to hear how you handled the integration overhead - did you roll your own, or is there something out there I've missed?

Swagger UI Is Showing the Wrong Auth Requirement on Every Endpoint — Here's the Fix

Virendra Vyas — Tue, 11 Aug 2026 23:57:15 +0000

Edit:Updated after a sharp comment from a reader — the original version used reflection over attributes, which misses RequireAuthorization() on minimal APIs and fallback policies, and collapsed multi-scheme requirements incorrectly. Fixed below.

If you've wired up Swashbuckle with AddSecurityDefinition and AddSecurityRequirement following the standard tutorial, you've probably hit this: every single endpoint in Swagger UI shows the padlock icon and demands the same auth scheme - even endpoints that don't need auth at all, or that need a different scheme entirely. This gets worse fast once you're running more than one JWT scheme in the same API.

Why this happens

The common tutorial snippet looks like this:

builder.Services.AddSwaggerGen(options =>
{
    options.AddSecurityDefinition("Bearer", new OpenApiSecurityScheme
    {
        Name = "Authorization",
        Type = SecuritySchemeType.Http,
        Scheme = "Bearer",
        BearerFormat = "JWT",
        In = ParameterLocation.Header,
    });

    options.AddSecurityRequirement(new OpenApiSecurityRequirement
    {
        {
            new OpenApiSecurityScheme
            {
                Reference = new OpenApiReference
                {
                    Type = ReferenceType.SecurityScheme,
                    Id = "Bearer"
                }
            },
            Array.Empty<string>()
        }
    });
});

AddSecurityRequirement at this level applies globally - to every operation in the generated spec, regardless of whether the underlying endpoint actually requires that scheme, or requires a different one, or requires no auth at all. Swashbuckle has no way to know your intent unless you tell it per-operation.

The first attempt (and why it wasn't enough)

My first fix was an IOperationFilter that reflected over AuthorizeAttribute and AllowAnonymousAttribute on the controller/action. That works fine for typical MVC controllers — but a reader pointed out two real gaps:

Minimal APIs and fallback policies don't show up via reflection. RequireAuthorization() on a minimal API endpoint, or a global AuthorizationOptions.FallbackPolicy, protect the endpoint at runtime without any AuthorizeAttribute being declared. Reflection-based detection reports these as anonymous, which is wrong.
Multi-scheme requirements were being collapsed incorrectly. In OpenAPI, schemes inside one requirement object mean AND (all required together); separate requirement objects mean OR (any one satisfies it). The original code took schemeNames.First(), silently dropping any additional required schemes from the spec.
The fix: read runtime metadata, not attributes

The more reliable source is ApiDescription.ActionDescriptor.EndpointMetadata, which reflects what's actually enforced at runtime — including minimal API policies and fallback policies — not just what's declared via attributes:

public class SecurityRequirementsOperationFilter : IOperationFilter
{
    public void Apply(OpenApiOperation operation, OperationFilterContext context)
    {
        var endpointMetadata = context.ApiDescription.ActionDescriptor.EndpointMetadata;

        var authorizeData = endpointMetadata.OfType<IAuthorizeData>().ToList();
        var hasAnonymous = endpointMetadata.OfType<IAllowAnonymous>().Any();

        if (hasAnonymous)
        {
            return; // explicitly anonymous, no padlock
        }

        if (!authorizeData.Any())
        {
            // No explicit [Authorize] - check if a fallback policy applies
            // (inject IAuthorizationPolicyProvider or pass AuthorizationOptions
            // in to resolve FallbackPolicy here if one is configured)
            return;
        }

        // Group scheme names per AuthorizeData entry - each entry's schemes
        // are its own AND group; multiple entries are OR'd as separate
        // requirement objects, matching OpenAPI semantics.
        var requirementGroups = authorizeData
            .Select(a => (a.AuthenticationSchemes ?? "Bearer")
                .Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
            .Where(schemes => schemes.Length > 0)
            .Distinct(new StringArrayComparer())
            .ToList();

        if (!requirementGroups.Any())
        {
            requirementGroups.Add(new[] { "Bearer" });
        }

        operation.Security = requirementGroups
            .Select(schemes =>
            {
                var requirement = new OpenApiSecurityRequirement();
                foreach (var scheme in schemes)
                {
                    requirement[new OpenApiSecurityScheme
                    {
                        Reference = new OpenApiReference
                        {
                            Type = ReferenceType.SecurityScheme,
                            Id = scheme
                        }
                    }] = Array.Empty<string>();
                }
                return requirement;
            })
            .ToList();
    }

    private class StringArrayComparer : IEqualityComparer<string[]>
    {
        public bool Equals(string[]? x, string[]? y) =>
            x != null && y != null && x.OrderBy(s => s).SequenceEqual(y.OrderBy(s => s));

        public int GetHashCode(string[] obj) =>
            obj.OrderBy(s => s).Aggregate(17, (hash, s) => hash * 31 + s.GetHashCode());
    }
}

builder.Services.AddSwaggerGen(options =>
{
    // ... AddSecurityDefinition calls for each scheme (e.g. "MainScheme", "UserTokenScheme")

    options.OperationFilter<SecurityRequirementsOperationFilter>();
});

Now operation.Security correctly represents:

Anonymous endpoints ([AllowAnonymous] or no auth metadata at all) → no padlock
Single-scheme endpoints → one requirement object, one scheme
Multi-scheme AND ([Authorize(AuthenticationSchemes = "A,B")]) → one requirement object containing both schemes
Multi-scheme OR (stacked [Authorize] attributes with different schemes) → separate requirement objects, one per attribute

This is closer to the actual OpenAPI spec semantics, not just a padlock that happens to look right for the common case.

Fallback policies still need explicit handling

If you use a global FallbackPolicy (protects any endpoint without explicit authorization metadata), the filter above deliberately does nothing for those endpoints rather than guessing — you'll want to resolve the policy's scheme requirements yourself (via IAuthorizationPolicyProvider) and apply them the same way as an explicit [Authorize]. I've left this as a TODO above rather than baking in an assumption about your policy setup, since fallback policies vary a lot between projects.

Why this matters more with multiple schemes

If you're only running a single JWT scheme, the blanket AddSecurityRequirement approach is merely imprecise — annoying, but not actively misleading. Once you're running two schemes side by side (see my earlier post on dual JWT Bearer schemes), the global approach becomes actively wrong: it tells every consumer of your Swagger UI to authenticate with the wrong token on some endpoints, or hides a required second token entirely, which is exactly the kind of thing that sends other developers down the wrong debugging path.

Checklist
Does Swagger UI show a padlock only on endpoints that actually require auth?
If you have [AllowAnonymous] endpoints, do they correctly show no lock?
If you're running multiple schemes, does each protected endpoint show the correct scheme(s) — not just "a" scheme?
Do multi-scheme AND requirements show as one requirement object with both schemes, not just the first one?
Do you have any minimal API endpoints or a fallback policy, and if so, are they handled explicitly rather than assumed anonymous?
Have you tested the Authorize button end-to-end — does the token you enter actually get attached to requests for the endpoint you're testing?

This is a small amount of extra setup for a much more honest Swagger UI — and it stops other developers (or future you) from trusting a spec that's quietly lying about what each endpoint needs.

Two JWT Bearer Schemes, One API: A Real ASP.NET Core Setup

Virendra Vyas — Tue, 11 Aug 2026 23:47:09 +0000

Most ASP.NET Core auth tutorials assume a single JWT scheme: one token, one identity, one [Authorize] and you're done. Real systems don't always fit that shape. On a production system I worked on, we needed a second, independently-issued JWT for a narrower purpose alongside the main session token — and getting the two schemes to coexist cleanly (without silently breaking claim lookups) took longer than it should have.

Why two schemes

The main API surface authenticates with a standard Authorization: Bearer token issued at login. Separately, a specific set of endpoints needed to authenticate a secondary JWT — issued for a narrower, scoped context — carried in a custom header rather than the standard Authorization header. Trying to force both use cases through a single scheme meant either overloading one token with responsibilities it shouldn't have, or writing brittle conditional logic inside a single JwtBearerEvents handler. Registering two named schemes turned out to be far cleaner.

Registering both schemes

services.AddAuthentication()
    .AddJwtBearer("MainScheme", options =>
    {
        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidateIssuer = true,
            ValidateAudience = true,
            ValidateLifetime = true,
            ValidateIssuerSigningKey = true,
            // issuer/audience/key config here
        };
    })
    .AddJwtBearer("UserTokenScheme", options =>
    {
        options.MapInboundClaims = false; // more on this below
        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidateIssuer = true,
            ValidateAudience = true,
            ValidateLifetime = true,
            ValidateIssuerSigningKey = true,
        };
        options.Events = new JwtBearerEvents
        {
            OnMessageReceived = context =>
            {
                context.Token = context.Request.Headers["X-User-Token"];
                return Task.CompletedTask;
            }
        };
    });

Each scheme gets a distinct name, and [Authorize(AuthenticationSchemes = "UserTokenScheme")] on a controller or action routes that specific endpoint to check the right token, extracted from the right place.

The MapInboundClaims gotcha

This is the part that actually cost the time. By default, JwtBearerOptions.MapInboundClaims is true, which means ASP.NET Core silently remaps standard JWT claim types on the way in — sub becomes ClaimTypes.NameIdentifier, email becomes ClaimTypes.Email, and so on, using the legacy .NET claim type URIs.

If your code reads claims by their raw JWT names (context.User.FindFirst("sub")), this remapping means the claim you're looking for silently isn't there under that name anymore — no exception, no error, just a null where you expected a value. It's a quiet failure mode that's easy to burn an afternoon on, because everything looks correct: the token is valid, the request is authenticated, but a specific claim lookup keeps failing.

The fix is one line:

options.MapInboundClaims = false;

With this set, HttpContext.User.Claims reflects the JWT's claim types exactly as they were issued — no remapping. Worth deciding deliberately per-scheme rather than hitting it by accident, especially if one scheme's tokens were issued by code that assumes the raw claim names and the other assumes the mapped ClaimTypes.* values.

Keeping the frontend agnostic

On the client side, permission and identity checks shouldn't need to know or care which scheme authenticated a given request. A thin context/helper layer that reads claims generically (rather than hardcoding assumptions about which scheme produced them) keeps this detail contained to the backend, where it belongs.

Checklist

[ ] Are your schemes named explicitly, with [Authorize(AuthenticationSchemes = "...")] used per endpoint rather than relying on a default?
[ ] Have you checked MapInboundClaims on each scheme if claim lookups aren't behaving as expected?
[ ] Is the second token's source (header, cookie, query string) documented clearly for other developers touching the code?
[ ] Are validation parameters (issuer, audience, lifetime, signing key) configured independently per scheme, not assumed to be shared?

Running two JWT schemes side by side isn't exotic once it's set up, but the defaults in ASP.NET Core's JwtBearerOptions — particularly MapInboundClaims — are exactly the kind of thing that looks fine until it silently isn't.

What HMRC's Fraud Prevention Headers Actually Require (That the Docs Don't Tell You)

Virendra Vyas — Tue, 11 Aug 2026 22:58:27 +0000

If you've built against HMRC's Making Tax Digital APIs, you've seen the phrase "Fraud Prevention Headers" in the docs and probably assumed it was a checkbox exercise — a handful of headers, a quick lookup table, done. It isn't. Get these wrong and HMRC either silently downgrades your submission's trust score or rejects it outright with an error message that tells you almost nothing about which header caused it. This post covers what actually trips people up when implementing this in a real .NET backend.

Why this exists

HMRC requires every call to their MTD APIs (VAT, ITSA, and others) to carry a set of Gov-Client-* and Gov-Vendor-* headers. The stated purpose is fraud prevention — HMRC wants a fingerprint of the originating device and software, not just the taxpayer's credentials. This is mandatory, not optional, and HMRC does run automated conformance checks against your header data.

The gotcha nobody's docs make clear: server-side vs browser-side

This is the part that catches out backend-only implementations. A chunk of the required headers describe the end user's device and browser, not your server:

Gov-Client-Device-ID
Gov-Client-Screens
Gov-Client-Window-Size
Gov-Client-Browser-JS-User-Agent
Gov-Client-Timezone

If your architecture is a typical SPA-calls-API-calls-HMRC setup, these values have to be captured client-side, passed to your backend, and then relayed onward — you can't reconstruct a meaningful device fingerprint from inside an ASP.NET Core service sitting in Azure. Header values like Gov-Client-Device-ID need to be a stable per-device identifier generated and persisted in the browser (typically in local storage), not a GUID your server invents per session.

The headers that genuinely are yours to generate server-side are the Gov-Vendor-* set — things like Gov-Vendor-Version, Gov-Vendor-Product-Name, Gov-Vendor-License-IDs — since those describe your software, not the user.

Structuring it in .NET

A clean way to handle this is a small pipeline: capture on the client, forward via a custom header on your own API, then a DelegatingHandler that assembles the full HMRC header set before the outbound HttpClient call.

public class HmrcFraudHeaderHandler : DelegatingHandler
{
    private readonly IHttpContextAccessor _httpContextAccessor;

    public HmrcFraudHeaderHandler(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var ctx = _httpContextAccessor.HttpContext;

        // Client-supplied values, forwarded from the frontend
        CopyHeaderIfPresent(ctx, request, "Gov-Client-Device-ID");
        CopyHeaderIfPresent(ctx, request, "Gov-Client-Screens");
        CopyHeaderIfPresent(ctx, request, "Gov-Client-Window-Size");
        CopyHeaderIfPresent(ctx, request, "Gov-Client-Timezone");

        // Server-generated values, describing your software
        request.Headers.Add("Gov-Client-Connection-Method", "WEB_APP_VIA_SERVER");
        request.Headers.Add("Gov-Vendor-Version", "yourapp=1.4.0");
        request.Headers.Add("Gov-Vendor-Product-Name", "YourApp");

        return base.SendAsync(request, cancellationToken);
    }

    private static void CopyHeaderIfPresent(
        HttpContext? ctx, HttpRequestMessage request, string headerName)
    {
        if (ctx != null && ctx.Request.Headers.TryGetValue(headerName, out var value))
        {
            request.Headers.TryAddWithoutValidation(headerName, value.ToString());
        }
    }
}

The important design decision here is the split: your frontend collects device data once per session and sends it as ordinary headers to your API, and this handler is the only place that knows how to translate that into HMRC's exact header contract. That keeps your controllers and services blissfully unaware of HMRC's header spec.

What you actually see when it's wrong

HMRC doesn't always give you a clean 400. In sandbox testing, malformed or missing fraud prevention headers typically show up as:

A 400 with a vague INVALID_HEADER or MISSING_HEADER code but no indication of which header
Submissions that succeed but get flagged for manual review on HMRC's side (invisible to you at integration time)
Inconsistent behavior between sandbox and production — sandbox is more forgiving

The practical fix is to test against HMRC's official Test Fraud Prevention Headers validation endpoint before you ever point at the live API — it will tell you precisely which header failed and why, which the production API won't.

Checklist

[ ] Are Gov-Client-* device/browser headers actually coming from the client, not fabricated server-side?
[ ] Is Gov-Client-Device-ID persisted per-device (not regenerated every session)?
[ ] Are Gov-Vendor-* headers static and versioned correctly with each release?
[ ] Have you run your header set through HMRC's validation endpoint, not just the live sandbox?
[ ] Does your timezone header format match IANA format exactly (e.g. Europe/London), not an offset?

Fraud Prevention Headers are one of those HMRC requirements that look like paperwork until you realize they're a genuine architectural decision — where in your stack does "the browser" end and "your server" begin, and how do you carry that boundary through cleanly. Get the split right early and it stops being a recurring headache.

What building against HMRC's MTD APIs actually looks like in 2026 (the undocumented gotchas nobody warns you about)

Virendra Vyas — Tue, 04 Aug 2026 13:32:02 +0000

I spent a recent session wiring up close to 20 HMRC Making Tax Digital endpoints — VAT, Income Tax MTD, CIS, Property Business, Partner Income, and a chunk of the legacy PAYE family — against the sandbox environment. The official docs are decent once you're on the right page. Getting to the right page is the hard part.

Here are the specific things that cost real debugging time, in case they save someone else the same hours.

"Charges" isn't a flat resource — it's charges/pensions

If you're using the generic pattern most of HMRC's "Individuals X Income" APIs follow — GET /individuals/{category}/{nino}/{taxYear} — it's tempting to assume every category slots into that same shape. Most do. Charges doesn't.

The Individuals Charges (MTD) API is specifically about pension charges, and the real path has an extra segment:

GET /individuals/charges/pensions/{nino}/{taxYear}

Not /individuals/charges/{nino}/{taxYear}. That extra /pensions/ isn't mentioned anywhere obvious — you find it by actually reading the endpoint list in the Developer Hub docs page for that specific API, not by pattern-matching from a working example elsewhere.

Also worth knowing: this endpoint is versioned separately from most of the income-category family — it needs Accept: application/vnd.hmrc.3.0+json, not the 2.0 that dividends/pensions/foreign income use.

Reliefs isn't one API — it's five

Same trap, worse. There's no single "reliefs" resource at all. The Individuals Reliefs API splits into five genuinely separate sub-resources, each with its own path:

GET /individuals/reliefs/investment/{nino}/{taxYear}
GET /individuals/reliefs/other/{nino}/{taxYear}
GET /individuals/reliefs/foreign/{nino}/{taxYear}
GET /individuals/reliefs/pensions/{nino}/{taxYear}
GET /individuals/reliefs/charitable-giving/{nino}/{taxYear}

If you've built a generic "fetch by category name" tool the way I had, none of the obvious category names (reliefs, relief) will work — you need the actual sub-resource name, which again only shows up if you read the specific API's endpoint list rather than inferring it.

A 404 doesn't always mean "wrong path" — sometimes it means "empty test user"

This one's a genuine trap for anyone testing in sandbox: if you get a 404 on a category you already know works (say, dividends-income, which you tested five minutes ago), and you're now testing it against a different NINO, the 404 might just mean that test user has no seeded data for that category — not that your integration broke.

The fix is boring but important: before debugging a path, re-test a known-good category against the same NINO. If that also fails, it's the test user, not your code.

Sandbox subscriptions are separate from API existence

Several endpoints that returned 404 turned out to be a subscription gap, not a path problem — the app registered in the Developer Hub simply wasn't subscribed to that specific API version. HMRC's sandbox doesn't always distinguish "not subscribed" from "genuinely not found" in its error responses, which makes this one of the more time-costly failure modes to diagnose, since the fix (go subscribe in the Developer Hub) doesn't look anything like what the error message suggests.

Some APIs are gated by a future tax year, and the error doesn't say why

The Individuals Partner Income API — a partner's share of partnership profit, reported through their own Self Assessment — only supports tax years from 2026-27 onward. Test it with 2024-25 (the reasonable default for most other MTD APIs) and you get:

{"code":"RULE_TAX_YEAR_NOT_SUPPORTED","message":"The tax year specified does not lie within the supported range"}

No indication of which years are actually supported. The only way to know is to check the example value HMRC shows in the docs (2026-27) and infer that's the floor — the API itself won't tell you.

Stateful sandbox testing doesn't always require a "real" registered record

One of the more useful discoveries: some stateful APIs (Property Business's period-summary create/fetch cycle, for example) will accept a synthetic ID matching the expected format, even if that ID doesn't correspond to a business that's actually registered anywhere. The sandbox stores it fresh on create and retrieves it on fetch, purely self-contained. Others — CIS Deductions, notably — behave inconsistently: create succeeds, but read/amend/delete against the exact same ID return MATCHING_RESOURCE_NOT_FOUND, which looks like a genuine sandbox bug rather than expected behavior.

The lesson: don't assume all "stateful" APIs behave identically. Test the full create-then-read cycle for each one individually before trusting it.

Quick reference — categories and versions that actually work together

dividends-income, pensions-income, foreign-income, other-income, insurance-policies-income, employments-income/other, deductions/other — version 2.0
expenses/other — version 3.0
charges/pensions — version 3.0
disclosures, state-benefits — version 2.0
reliefs/investment, reliefs/pensions, reliefs/foreign, reliefs/charitable-giving, reliefs/other — version 3.0

None of this is criticism of HMRC's APIs — the documentation is genuinely more thorough than a lot of government API docs, and the sandbox stateful-testing model is a good idea when it works consistently. It's just that the path from "read the docs" to "working integration" has more hidden turns than the docs alone suggest, and I didn't find a single place that laid all of this out together. Hopefully this saves someone a few hours.

If you're building against these APIs and hitting your own version of this, I'd genuinely like to compare notes — feel free to reach out.