DEV Community: Krishna Yadav

Spring Boot 4.1's SSRF Mitigation Is Opt-In: How to Actually Close the Hole

Krishna Yadav — Fri, 23 Oct 2026 04:35:31 +0000

Open the webhook sender in almost any payments service and you will find a method that reads a URL out of a database row and calls it. A merchant registered the address during onboarding. A partner bank supplied a callback host. A KYC vendor returned a link to a scanned document. The call is a single line, restClient.get().uri(url).retrieve(), and it is also the line where an attacker can make your service fetch http://169.254.169.254/latest/meta-data/iam/security-credentials/ for them.

Spring Boot 4.1 ships a first-party defence for this. I went through the 4.1.0 RC1 release notes and then looked at how the feature behaves in a service with hand-built HTTP clients. Two things are easy to miss. The filter does nothing until a bean exists for it, and it only covers the clients Boot itself builds.

Below I cover both gaps, show a resolver-level guard for clients Boot cannot see, and write the tests that show the hole is shut. I also treat the code filter as the last of several layers, since egress control belongs underneath it. Then I turn to the other 4.1 change that matters for microservice teams: gRPC server and client auto-configuration now lives inside Boot, which makes mutual TLS between internal services a configuration exercise.

Why payment backends get hit

A service in a private network can reach what the public internet cannot: the cloud instance metadata endpoint, the Kubernetes API server, an admin port bound to the pod network, a Redis instance left without a password because only cluster traffic reaches it. If an attacker controls where one outbound request goes, the service's network position becomes theirs.

Payment systems have more of these attacker-influenced destinations than most. In the services I have worked on, they come from four places:

Webhook targets that merchants type into a dashboard.
Callback and token endpoints belonging to partner banks and processors, stored in configuration or onboarding records.
Document links (statements, logos, KYC scans) submitted through forms.
Location headers on responses from third parties, if the client follows them.

On a node where the metadata service still accepts unauthenticated GETs, the first request to 169.254.169.254 can return temporary cloud credentials, and everything after that depends on how broad the instance role is.

My time on the OpenBanking PSD2 side of fintech shaped how I read partner-supplied hosts. The whole model there is calling other organisations' endpoints, and a hostname being on the partner's onboarding sheet says who signed a contract. It says nothing about where the name resolves today. Partner infrastructure gets compromised, lapsed domains get registered by someone new, and a name that pointed at a public address last month can be repointed at 10.0.4.17 by whoever controls its DNS zone.

What 4.1 gives you

The new type is InetAddressFilter. It is evaluated after DNS resolution, against the InetAddress the client is about to connect to. That placement is the reason it works better than URL inspection. A string check for 169.254 in the host loses to a decimal-encoded IP, an IPv4-mapped IPv6 literal, a trailing dot, or simply a domain the attacker owns whose A record is the metadata address. Looking at the address that will actually receive the SYN makes the spelling of the URL irrelevant.

InetAddressFilter.externalAddresses() is the built-in factory. As I read the release notes, it refuses loopback, private-range, multicast and other special-purpose addresses, including the link-local block that holds the metadata endpoint. Check the exact set against the Javadoc for your patch version rather than relying on my summary. Once a filter bean exists, Boot attaches it to the clients it auto-configures: RestClient, RestTemplate and WebClient.

The opt-in is one bean:

@Configuration(proxyBeanMethods = false)
class OutboundSecurityConfig {

    // No bean, no filtering. Boot 4.1 does not install this on its own.
    @Bean
    InetAddressFilter outboundAddressFilter() {
        return InetAddressFilter.externalAddresses();
    }
}

Most write-ups on the feature stop at this snippet. The next two sections are about what it leaves open.

Gap one: no bean, no protection

Take a project on 4.1 with no InetAddressFilter bean and point a test at a WireMock instance on 127.0.0.1. My prediction, which follows directly from the opt-in design and which you can confirm in five minutes, is that the call succeeds. Nothing is installed, so nothing blocks.

Here is the version of this that worries me. A platform team sees "Spring Boot 4.1 mitigates SSRF" in the release notes, raises the parent POM version across its services, and closes the ticket. I am describing a scenario, not a specific company. The upgrade changed no behaviour for SSRF. Every service is as exposed as before, and the tracker says otherwise.

What I do about it is put the bean in a shared internal starter and add a startup check beside it. If a service has a RestClient.Builder bean but no InetAddressFilter, it refuses to boot outside local and test profiles:

@Component
@Profile("!local & !test")
class OutboundFilterGuard implements SmartInitializingSingleton {

    private final ObjectProvider<InetAddressFilter> filter;

    OutboundFilterGuard(ObjectProvider<InetAddressFilter> filter) {
        this.filter = filter;
    }

    @Override
    public void afterSingletonsInstantiated() {
        if (filter.getIfAvailable() == null) {
            throw new IllegalStateException(
                "No InetAddressFilter bean declared, so outbound SSRF protection is OFF. "
                + "Import OutboundSecurityConfig, or record an explicit exception for this service.");
        }
    }
}

The !local & !test profile expression is a trade-off. Local runs and integration tests talk to WireMock, Testcontainers and embedded servers, all of which live on loopback, and an enforcing filter would break them. The cost is that the guard never runs in those profiles, so the tests described later in this post, which exercise the real resolver explicitly, are what keep the protection covered in CI. If you want the filter active in test profiles too, you will need an allow exception for loopback that exists only there, and that exception must never ship in a production profile.

Gap two: clients you build yourself

Boot's filter reaches clients through its own auto-configuration. Define your own HttpClient, CloseableHttpClient or OkHttpClient bean and wrap it in a request factory, and Boot's customisation never touches it. As far as I can tell nothing logs a warning either, so you find out by testing, not by reading startup output. If you have verified different behaviour on a specific 4.1.x build, put your finding next to mine: I haven't tested every client type.

Real payment code ends up here constantly, because partner integrations ask for things the defaults can't do:

Mutual TLS with a keystore dedicated to one partner.
A pool sized to a processor's cap on concurrent connections.
Timeouts that differ per acquirer, so a slow one cannot starve the others.
An OkHttp interceptor that produces the detached JWS signature a bank requires.

Each requirement leads to a hand-built client, and a hand-built client is outside Boot's reach. The remedy is to attach the address check to the client yourself. For Apache HttpClient 5 the narrowest point is the DnsResolver, because every connection attempt passes through it. This version uses only JDK address classification, so it does not depend on Boot internals, and it takes its delegate as a constructor argument so tests can feed it canned answers:

public final class PublicOnlyDnsResolver implements DnsResolver {

    private final DnsResolver delegate;

    public PublicOnlyDnsResolver() {
        this(SystemDefaultDnsResolver.INSTANCE);
    }

    PublicOnlyDnsResolver(DnsResolver delegate) {
        this.delegate = delegate;
    }

    @Override
    public InetAddress[] resolve(String host) throws UnknownHostException {
        InetAddress[] resolved = delegate.resolve(host);
        for (InetAddress address : resolved) {
            if (isForbidden(address)) {
                // Host and address only. The full URL can carry tokens.
                throw new BlockedDestinationException(
                    "Blocked outbound connection: " + host + " resolved to " + address.getHostAddress());
            }
        }
        return resolved;
    }

    @Override
    public String resolveCanonicalHostname(String host) throws UnknownHostException {
        return delegate.resolveCanonicalHostname(host);
    }

    static boolean isForbidden(InetAddress address) {
        if (address.isLoopbackAddress() || address.isAnyLocalAddress()
                || address.isLinkLocalAddress() || address.isSiteLocalAddress()
                || address.isMulticastAddress()) {
            return true;
        }
        byte[] raw = address.getAddress();
        if (raw.length == 4) {
            int first = raw[0] & 0xFF;
            int second = raw[1] & 0xFF;
            // 100.64.0.0/10 (carrier-grade NAT) and 198.18.0.0/15 (benchmarking)
            // are not covered by the JDK helpers above.
            return (first == 100 && (second & 0xC0) == 0x40)
                    || (first == 198 && (second & 0xFE) == 18);
        }
        // fc00::/7, IPv6 unique-local
        return raw.length == 16 && (raw[0] & 0xFE) == 0xFC;
    }

    /** UnknownHostException subtype so retry policies can tell policy from network failure. */
    public static final class BlockedDestinationException extends UnknownHostException {
        public BlockedDestinationException(String message) {
            super(message);
        }
    }
}

On IPv6: InetAddress hands back an Inet4Address for IPv4-mapped literals such as ::ffff:10.0.0.1, so the checks above see the embedded IPv4 value. Other transition formats (NAT64 prefixes, 6to4) are not normalised that way, which is one more reason to run this behind an egress policy that does not care how an address is spelled.

Wire it into a pooled connection manager and give the client to a request factory:

@Bean
RestClient acquirerRestClient(PartnerTlsProperties tls) {
    var connectionManager = PoolingHttpClientConnectionManagerBuilder.create()
        .setDnsResolver(new PublicOnlyDnsResolver())
        .setSSLSocketFactory(tls.mutualTlsSocketFactory())
        .setMaxConnPerRoute(20)
        .build();

    CloseableHttpClient httpClient = HttpClients.custom()
        .setConnectionManager(connectionManager)
        .disableRedirectHandling()   // a redirect is a new destination chosen by someone else
        .build();

    return RestClient.builder()
        .baseUrl(tls.acquirerBaseUrl())
        .requestFactory(new HttpComponentsClientHttpRequestFactory(httpClient))
        .build();
}

Redirects

I disable automatic redirect handling on any client whose target a tenant or partner can influence. A host you are entitled to call can reply 302 Location: http://169.254.169.254/.... With the resolver installed, the redirected hop goes through the same pooled manager and gets checked, so the guard would likely catch it. Even so, I would rather a human-written branch decide whether a redirect is acceptable than let the client follow one silently. Payment APIs seldom need redirects (a 3-D Secure challenge URL is sent to the browser, not followed by your server). In the few cases that do, write explicit code that validates the new target against the same rules and caps the hop count.

The resolver and DNS rebinding

Checking at resolve time, inside the same call that returns the addresses the connection manager then uses, avoids the classic time-of-check/time-of-use trap where a validator resolves a name once and the client resolves it again and gets a different answer. That holds only because the check and the connect use the same resolver result. If you add a separate pre-flight "is this URL safe" step that does its own lookup, you have reintroduced the race.

For OkHttp the equivalent hook is OkHttpClient.Builder.dns(Dns), and isForbidden can be reused unchanged. I keep that method in one shared library class so a fix to a range lands in every client together.

Allowlists, and the first support ticket

An address filter is a denylist by category. Where destinations form a closed set, use an allowlist instead: the hosts of your partner banks are known, so store them, compare against them, and treat everything else as an onboarding exception needing review. Hold the address filter for the open-ended cases such as merchant webhooks.

Expect friction once merchants hit it. The first ticket is predictable: a merchant has a legitimate endpoint on a private address, perhaps a VPN-fronted ERP or a staging box on an internal range, and every delivery to it is now blocked. Decide the policy in advance. Mine is that there is no per-tenant bypass in the shared library. A merchant that needs private connectivity gets a dedicated delivery path, with its own network route and its own review, and not a flag in the filter.

Proving it with tests

Security code I haven't watched reject something is code I do not trust. Three tests cover the guard. The first two need no network; they check classification and resolution using canned answers:

class PublicOnlyDnsResolverTest {

    private static DnsResolver resolvingTo(String... literals) {
        return new DnsResolver() {
            @Override
            public InetAddress[] resolve(String host) throws UnknownHostException {
                InetAddress[] out = new InetAddress[literals.length];
                for (int i = 0; i < literals.length; i++) {
                    out[i] = InetAddress.getByName(literals[i]);
                }
                return out;
            }

            @Override
            public String resolveCanonicalHostname(String host) {
                return host;
            }
        };
    }

    @ParameterizedTest
    @ValueSource(strings = {
        "169.254.169.254",   // cloud metadata
        "127.0.0.1",
        "10.0.4.17",         // a "public-looking" name pointing at a private range
        "172.16.8.9",
        "192.168.1.20",
        "100.64.3.4",        // carrier-grade NAT
        "::1",
        "fd00::1",           // IPv6 unique-local
        "::ffff:10.0.0.1"    // IPv4-mapped IPv6
    })
    void rejectsInternalAddressesNoMatterWhatTheNameLooksLike(String literal) {
        var resolver = new PublicOnlyDnsResolver(resolvingTo(literal));

        assertThatThrownBy(() -> resolver.resolve("partner.example.com"))
            .isInstanceOf(PublicOnlyDnsResolver.BlockedDestinationException.class)
            .hasMessageContaining("Blocked outbound connection");
    }

    @Test
    void rejectsWhenOnlyOneOfSeveralRecordsIsInternal() {
        // Round-robin DNS can mix addresses; one bad record must poison the lookup.
        var resolver = new PublicOnlyDnsResolver(resolvingTo("93.184.216.34", "10.0.0.8"));

        assertThatThrownBy(() -> resolver.resolve("mixed.example.com"))
            .isInstanceOf(PublicOnlyDnsResolver.BlockedDestinationException.class);
    }

    @Test
    void allowsOrdinaryPublicAddresses() throws Exception {
        var resolver = new PublicOnlyDnsResolver(resolvingTo("93.184.216.34"));

        assertThat(resolver.resolve("partner.example.com")).hasSize(1);
    }
}

The third goes through the real client bean, so it exercises the wiring and not only the helper. It relies on a wildcard DNS service resolving to loopback, which means it needs outbound DNS from your CI runners; if yours do not have that, replace the hostname with one your internal test resolver maps to 127.0.0.1:

@Test
void clientBeanRefusesNameThatResolvesToLoopback() throws Exception {
    try (var server = new MockWebServer()) {
        server.enqueue(new MockResponse().setBody("secrets"));
        server.start(InetAddress.getLoopbackAddress(), 0);

        // localtest.me resolves to 127.0.0.1: the name looks external, the address is not.
        String url = "http://localtest.me:" + server.getPort() + "/latest/meta-data/";

        assertThatThrownBy(() -> acquirerRestClient.get().uri(url).retrieve().body(String.class))
            .hasRootCauseInstanceOf(PublicOnlyDnsResolver.BlockedDestinationException.class);

        assertThat(server.getRequestCount()).isZero();
    }
}

The last assertion carries the weight. A request count of zero on the listener shows that the connection was refused before any bytes left the process, which is a stronger statement than "an exception was thrown". Write one test like this per client bean you own. If the codebase has eleven client beans and the test class has six cases, the five missing ones are your list of unprotected paths.

I have not pasted run output here because the numbers would depend on your build and Boot patch release. Run these against your own services and attach the result to the change that introduces the guard; that is the artefact a reviewer or auditor will ask for.

Defence in depth: the code filter is the last layer

Everything above lives in the application. The application is also the layer that gets forgotten, misconfigured and bypassed, so treat it as the final wall and not the only one:

Egress at the network layer. A Kubernetes NetworkPolicy, security group or egress firewall that denies the link-local range and RFC 1918 space for pods that only need to reach the internet is the real fix. If application code fails, the packet still goes nowhere. For services that must call arbitrary tenant URLs, route them through an egress proxy or a small webhook-dispatcher service in an isolated network segment, so only that one component holds the risk.
IMDSv2 with a hop limit of 1 on AWS. Version 2 of the metadata service wants a session token obtained with a PUT, which the usual blind-GET SSRF primitive cannot produce, and a hop limit of one keeps container-originated traffic from reaching it. For the metadata attack in particular, this is the highest-value infrastructure change I know of.
Allowlists wherever the destination set is closed, as described above.
Short timeouts and response caps. SSRF probing often relies on timing differences to map a network. A two-second connect timeout and a one-megabyte body limit remove most of that signal.
Egress observability. Tag outbound spans with the resolved address if you already trace requests. I wrote about wiring that up in distributed tracing with OpenTelemetry on Spring Boot. A counter of blocked connections per tenant is a cheap detector, and a tenant whose count suddenly climbs deserves a look.

The other half: gRPC between your own services

Closing the outbound door covers traffic leaving your estate. Traffic between your own services needs the same attention, and Boot 4.1 changes the setup cost here too. The Spring gRPC project (version 1.1.0 at the time of writing) has been brought into Boot, so server and channel configuration arrive through ordinary auto-configuration and properties and not through a separately versioned starter.

For a ledger, risk and settlement split, gRPC gives typed contracts and low overhead. It also gives you one obvious place to enforce transport security, which plain HTTP between pods rarely gets in practice. I count plaintext inside the cluster as a finding.

A ledger service exposing an endpoint looks like this:

@Service
class LedgerGrpcService extends LedgerServiceGrpc.LedgerServiceImplBase {

    private final PostingEngine postingEngine;

    LedgerGrpcService(PostingEngine postingEngine) {
        this.postingEngine = postingEngine;
    }

    @Override
    public void postEntry(PostEntryRequest request, StreamObserver<PostEntryReply> reply) {
        // The idempotency key travels in the message body, not in transport metadata,
        // so a retry from any hop deduplicates against the same value.
        PostingResult result = postingEngine.post(request.getIdempotencyKey(),
                request.getDebitAccount(), request.getCreditAccount(), request.getAmountMinor());

        reply.onNext(PostEntryReply.newBuilder().setPostingId(result.id()).build());
        reply.onCompleted();
    }
}

With auto-configuration, that BindableService bean is discovered and registered without hand-assembling a ServerBuilder. Transport security then becomes configuration. The property names below follow the Spring gRPC reference as I understand it, so confirm them against the 4.1.x documentation for your exact patch version before copying:

spring:
  ssl:
    bundle:
      pem:
        ledger-server:
          keystore:
            certificate: file:/etc/tls/ledger/tls.crt
            private-key: file:/etc/tls/ledger/tls.key
          truststore:
            certificate: file:/etc/tls/ca/internal-ca.crt
  grpc:
    server:
      port: 9090
      ssl:
        bundle: ledger-server
        client-auth: need   # mutual TLS: a caller without a certificate never reaches your code

client-auth: need is the line that matters. A caller without a certificate chained to your internal CA fails during the handshake, before any of your code executes. With cert-manager issuing short-lived workload certificates, the day-to-day cost is small.

The calling side reuses the SSL bundle mechanism, and the channel factory resolves named channels from configuration:

@Configuration(proxyBeanMethods = false)
class LedgerClientConfig {

    @Bean
    LedgerServiceGrpc.LedgerServiceBlockingStub ledgerStub(GrpcChannelFactory channels) {
        // "ledger" maps to spring.grpc.client.channels.ledger.* in application.yml,
        // including its SSL bundle and target address.
        ManagedChannel channel = channels.createChannel("ledger");
        return LedgerServiceGrpc.newBlockingStub(channel)
            .withDeadlineAfter(800, TimeUnit.MILLISECONDS);
    }
}

The two halves cover different trust boundaries. Channels to ledger.payments.svc.cluster.local are internal and authenticated by mutual TLS. The InetAddressFilter and the custom DnsResolver belong to the HTTP clients that reach out to the world. Keep them visibly apart in configuration. One line is worth adding to the code review checklist: channel targets come from configuration, never from request data. If anyone writes ManagedChannelBuilder.forTarget(valueFromARequest), they have rebuilt the SSRF hole one protocol over, and the HTTP-layer controls will not see it because gRPC does not travel through those clients.

Behaviour under failure

A hardened path still has to fail sensibly. Two changes are worth making before the first load test, not after:

Blocked calls must not retry. Many resilience configurations treat UnknownHostException as transient and retry it with backoff. A blocked address is a policy decision, so retrying it three times only triples the log volume and delays the alert. That is why BlockedDestinationException is its own subtype above: exclude it from the retry predicate.
Webhook delivery needs a terminal state for rejected destinations. When the filter rejects a merchant's endpoint, the event cannot silently disappear. Park it and flag the merchant record for review. If you deliver through an outbox, as in the transactional outbox and idempotent consumers post, a blocked destination becomes a terminal status on the outbox row and not an endless retry loop.

Also plan for false positives from things that legitimately call loopback or private addresses: health checks, sidecar proxies, Actuator scrapes and calls to a local agent. Those should not use the clients you are guarding. If they do, keep them on a separate client bean with a narrow, documented purpose and no tenant-controlled input.

Order of work for an existing estate

If I were given a fintech microservice estate on 4.1 and a quarter to make it safer, this is the order I would go in:

Ship the shared starter that declares InetAddressFilter.externalAddresses(), along with the startup guard that fails a deploy when the bean is missing.
Search for HttpClient, CloseableHttpClient, OkHttpClient and Feign builders. Every hit is a client the Boot filter cannot see. Give each the resolution-time guard and a test like the ones above.
Turn off automatic redirects on anything that calls a tenant-controlled URL.
Enforce IMDSv2 with hop limit one on every node, and add egress policy that blocks the link-local and private ranges for workloads that have no reason to reach them.
Move internal gRPC to mutual TLS through SSL bundles, and ban request-derived channel targets in review.

None of this is exotic. It removes a whole class of incident from the risk register, and the framework now gives you a starting point where there used to be a blank file.

If you suspect your outbound clients are less covered than the dashboards claim, my background across airline-scale traffic and fintech platforms is a reasonable place to begin a conversation. Send the list of HTTP client beans in your busiest service through the contact section, and I will tell you which of them the Boot filter never touches.

OpenBanking PSD2 API Development: the partner-integration side of fintech, where every outbound call needs a threat model.
Distributed Tracing with OpenTelemetry on Spring Boot in Production: how to make blocked and suspicious egress visible.
Transactional Outbox and Idempotent Consumers for Spring Boot Payment Systems: reliable webhook delivery, including what to do when a destination is rejected.

From Startup to Acquisition: Lessons from Building and Selling an eCommerce Platform

Krishna Yadav — Mon, 05 Oct 2026 17:02:26 +0000

In February 2014 I was twenty-two, halfway through an M.Tech, and convinced I could build a marketplace that would change how local vendors sold online. Three years later the platform was serving eighty-plus vendors, processing real money, and I signed the papers that handed it to someone else.

This is the short version of that story, and the seven things it taught me.

The Setup

Small vendors in our region had no affordable way to sell online. Big marketplaces charged steep commissions and buried small sellers. We wanted to build a multi-vendor platform where each shop got its own storefront with shared logistics, payments, and traffic.

My co-founder handled business. I handled everything that touched a terminal. We bootstrapped — no angel round, no incubator. Every hour of engineering time came directly out of the hours I was supposed to spend on coursework.

Building on a Twelve-Dollar VPS

I picked Java and Spring for the backend, MySQL for persistence, jQuery for the frontend. Not trendy, but I trusted it wouldn't collapse at midnight when I was the only person on call.

The MVP took ten weeks of nights and weekends. One Spring monolith, one database, one Tomcat server on a VPS that cost twelve dollars a month. It was ugly. It worked.

The first real challenge was payment integration. Two Indian payment gateways, each with its own callback format, retry logic, and definition of "success." I wrote an adapter layer that normalized both into common internal events — PAYMENT_CAPTURED, PAYMENT_FAILED, PAYMENT_REFUND_INITIATED — so the rest of the order pipeline didn't care which gateway handled a transaction. That pattern ended up being the most reusable code in the entire system.

Scaling and Cracking

Each new vendor exposed a new edge case. Weight-based items. Variant pricing. Bundle deals. Every feature request was a negotiation between "this helps one vendor" and "this complicates things for everyone."

Around vendor forty, the monolith started cracking. Page loads crept past three seconds. I extracted the inventory service into its own process with its own database. Not a full microservices migration — one pragmatic extraction that relieved the biggest bottleneck.

Meanwhile I was attending lectures from 9 AM to 1 PM, building the platform from 2 PM to midnight, and squeezing thesis research into whatever gaps remained. The saving grace: every distributed systems concept from the M.Tech curriculum — consistency models, fault tolerance, CAP theorem — I was seeing in production the same week I read about them in papers. The gap between theory and practice wasn't a gap. It was the same Tuesday.

The Exit

By early 2017 the platform was profitable but growth had plateaued. A regional eCommerce company approached us. They wanted our vendor network and our technology — specifically the vendor management system and the payment adapter layer, which they said would save them six to eight months of development.

The negotiation took two months. We closed in May 2017. The terms were fair. The vendors kept their storefronts, the customers kept their accounts. I walked out with no regrets and a clear picture of what I wanted next: building at larger scale, which eventually led me to Oracle Financial Services, Kotak Securities, and AirAsia where I now work on systems processing 130 million daily transactions.

Seven Lessons for Technical Founders

1. Ship the ugly version first. Our MVP was architecturally embarrassing. It was also live, collecting payments, and teaching us what users actually needed. The beautiful rewrite can happen after you've proven the business.

2. Pick boring technology. Java and Spring in 2014 won zero hackathons. But when our payment integration broke at midnight, I could find a Stack Overflow answer in under a minute. Boring technology compounds reliability.

3. Solve the class, not the instance. When vendor thirty-seven asks for a feature, ask what general problem it represents. Build the general solution or don't build it at all. Every special case is a maintenance liability that outlives whoever requested it.

4. Wrap every external dependency in an adapter. Payment gateways, shipping APIs, SMS providers — they will change their interfaces, go down, or get replaced. Your codebase should never know or care which vendor sits behind the adapter.

5. Cofounder alignment beats cofounder skills. We never argued about direction because we agreed on the fundamentals before we started: bootstrap, stay profitable, build something vendors actually use. Technical skills can be hired. Strategic alignment cannot.

6. Know your exit conditions before you need them. We didn't have a formal trigger for when to sell, and that made the decision more emotional than necessary. Write down the conditions under which you'd sell, shut down, or raise money — before you launch.

7. Burnout is not a badge of honor. I pushed through it because I was young and stubborn. One full day off per week would not have killed the company and would have made me a better engineer the other six days.

That startup gave me scars and skills in roughly equal proportion. If you're building something right now — side project, startup, internal tool at a company that doesn't appreciate it — the work compounds. The code you're embarrassed by today teaches you to write something better tomorrow.

Originally published on krishnakky.com — the full version has more detail on the technical architecture, payment integration patterns, and the acquisition negotiation.

AI-Driven Development: Running Autonomous Agents Across a Microservices Codebase

Krishna Yadav — Mon, 05 Oct 2026 16:57:07 +0000

At AirAsia, I own production microservices on the Manage My Booking platform — flight changes, fare summaries, price-slash, post-booking ancillaries. Over a dozen services, each with its own repo, pipeline, and accumulated quirks. The platform processes 130 million requests a day.

My problem was never writing code. It was the context-switching overhead. Bouncing between a fare-calculation refactor, a circuit breaker fix three repos over, and a schema migration for the ancillary service — every switch cost 20–30 minutes of mental reload. Entire days evaporated into friction.

I started using Claude Code not to write code for me, but to keep multiple workstreams moving in parallel. It operates directly in the terminal — reads the codebase, runs commands, writes files, stages changes. An agent, not an autocomplete engine.

Permission Scopes: The Non-Negotiable First Step

You cannot hand an AI agent unrestricted access to a production codebase. That is how you get a force-push to main at 2 AM.

I set up scoped permissions per project in .claude/settings.json:

{
  "permissions": {
    "allow": [
      "Read",
      "Edit",
      "Write",
      "Bash(npm run test*)",
      "Bash(npm run lint*)",
      "Bash(mvn test*)",
      "Bash(git diff*)",
      "Bash(git status*)",
      "Bash(git log*)",
      "Bash(git add -p*)"
    ],
    "deny": [
      "Bash(git push*)",
      "Bash(git checkout main*)",
      "Bash(rm -rf*)",
      "Bash(docker*)"
    ]
  }
}

The agent can read, write, edit, run tests and linters, view diffs, and stage changes. It cannot push, switch to main, delete directory trees, or touch Docker. Every push and merge goes through me. Payment-adjacent services get tighter restrictions. Internal tooling repos are looser.

The Three-Phase Model

I restructured feature delivery into three phases:

Inception (human only). I define the scope, draw sequence diagrams, identify affected services, and write interface contracts. This produces a spec document — the agent's input. Thirty to sixty minutes here, and the quality directly determines how useful the agent is downstream.

Construction (agents + review). Each agent gets a spec, a repo, and constraints. I run parallel terminal sessions — one agent per service:

Terminal 1: Agent on booking-service, implementing the new endpoint
Terminal 2: Agent on pricing-engine, adding fare calculation logic
Terminal 3: Agent on ancillary-catalog, updating schema and data layer
Me: Reviewing diffs, running integration tests, handling cross-service architecture decisions

The agents don't talk to each other. I am the coordination layer. When the pricing agent finishes, I review it and feed interface changes to the booking agent as context.

Operations (human only). Post-merge monitoring, production validation, performance profiling. I watch dashboards, check distributed traces in Zipkin, validate circuit breaker behavior. AI agents have no business in production operations.

Results After Six Months

Feature delivery time dropped ~40%. Cross-service features that took 4–5 days now take 2–3. The reduction comes from parallel Construction — three repos at once instead of sequential.

Unit test coverage increased. Writing tests is mechanical work agents handle well. I used to skip edge-case tests when tired. The agent does not get tired.

Code review quality improved. When I write code, I review it with the same mental model — and miss my own blind spots. Agent-written code gets genuinely fresh-eyed review. The separation between author and reviewer becomes real.

Commit history became cleaner. Selective staging and single-purpose commits made git bisect actually useful.

Not everything parallelizes. Tightly coupled changes where Service B depends on Service A's final interface still bottleneck. Those still see gains, but more like 15–20%.

Pitfalls Worth Knowing

The agent optimizes locally. It writes correct code for the service it sees while introducing interface mismatches with services it cannot see. Cross-service contracts must come from a human holding the full system model.

Spec quality is the bottleneck. "Add error handling to the booking endpoint" gets generic try-catch blocks. "Return HTTP 409 with BOOKING_CONFLICT when the fare class is unavailable, include available fare classes in the response body, emit a Kafka event to pricing-update" gets exactly what you need.

Don't let agents refactor while implementing. Mixed diffs — new functionality tangled with cleanup — are impossible to review. Implement against the existing structure; refactor in a separate PR.

Check in every 15–20 minutes. Not because the agent will break something catastrophic (permission scopes prevent that), but because a five-minute course correction at the 20-minute mark beats throwing away 90 minutes of wrong-direction implementation.

The Bigger Picture

This is not about AI replacing developers. It is about restructuring the workflow so human attention goes where it matters: architecture, system-level reasoning, production reliability. The mechanical act of translating a well-defined spec into working code with tests — that is what agents handle well. At the scale we operate at AirAsia, that mechanical work was eating a disproportionate amount of my time.

The spec-writing phase now takes longer than it used to. That tradeoff is worth it.

Originally published on krishnakky.com. Read the full version with detailed Git integration workflow, war stories, and what I'm experimenting with next.

Building Microservices at 130 Million Requests Per Day

Krishna Yadav — Mon, 05 Oct 2026 16:56:59 +0000

I've spent the last three-plus years building and operating the microservices behind AirAsia Move — the travel super-app that handles flights, hotels, and ancillaries for millions of travelers across Southeast Asia. The system processes over 130 million requests per day, roughly 1,500 requests per second sustained, with spikes well above that during flash sales.

I own services on the Manage My Booking platform: flight changes, fare summaries, price-slash features, post-booking ancillaries. A single user action like changing a flight can fan out to eight or nine downstream calls. At this scale, the engineering problems aren't about whether your code compiles. They're about what happens when one of those downstream services is 200ms slower than usual.

Circuit Breakers with Resilience4j

The pattern that proved its weight in gold was the circuit breaker. We use Resilience4j across all Spring Boot services — if a downstream service starts failing, stop calling it instead of piling up timeouts that cascade through the system.

@CircuitBreaker(name = "pricingService", fallbackMethod = "cachedFareFallback")
public FareSummary getFareSummary(String bookingId) {
    return pricingClient.fetchFare(bookingId);
}

private FareSummary cachedFareFallback(String bookingId, Throwable t) {
    log.warn("Pricing service unavailable for booking {}, using cache", bookingId);
    return fareCache.getLastKnown(bookingId)
        .map(fare -> fare.withStaleFlag(true))
        .orElseThrow(() -> new ServiceUnavailableException("No cached fare available"));
}

We pair breakers with fallbacks. If the pricing service is down, the user still sees a fare — it might be a few minutes stale, but that beats a blank screen or a 500.

We tuned thresholds through load testing with JMeter. The defaults were too aggressive and tripped breakers during normal latency variance. A 50% failure rate over a sliding window of 20 calls, with a 30-second open-state wait, matched our actual traffic pattern.

Distributed Tracing

When a request crosses ten services, debugging without distributed tracing is a lost cause. We use Spring Cloud Sleuth for trace propagation and Zipkin for visualization. We sample 10% of traces in production — at 1,500 RPS, that's 150 traces per second, more than enough to spot patterns.

The biggest tracing win wasn't debugging individual slow requests. We noticed every Tuesday between 2–4 AM UTC, latency spiked on the inventory service. A batch job was running full table scans on the same database the API was reading from. Moved the job to a read replica, spikes gone.

The Cascading Timeout

During a Diwali sale, the payment gateway started responding 300ms slower than usual. Not enough to fail — just enough to back up our thread pool. We had a fixed pool of 200 threads with a 5-second payment timeout and a bounded queue of 500. When every thread was blocked on slow payment calls, the queue filled. Upstream services calling us started timing out. Within 90 seconds, three services were effectively down.

The root cause wasn't the payment gateway. It was our thread pool and timeout configuration.

We made three changes:

Cut the payment timeout from 5s to 2s. If it hasn't responded in 2 seconds during peak load, it's not going to.
Added a bulkhead — isolated the payment call to its own thread pool so one slow dependency can't starve everything else.
Switched the circuit breaker to a time-based sliding window so it reacts faster during traffic spikes.

The Memory Leak

Our flight-change service was getting OOMKilled by Kubernetes every 48 hours. Two days of profiling with JVisualVM found a ConcurrentHashMap used as an in-memory cache with no eviction. Every unique booking ID got cached and never expired. Under sustained 1,500 RPS, it grew until the JVM ran out of heap.

Five lines of code, two days of investigation:

Cache<String, FareSnapshot> fareCache = Caffeine.newBuilder()
    .maximumSize(50_000)
    .expireAfterWrite(Duration.ofMinutes(10))
    .build();

What I'd Do Differently

Contract testing from day one. When Service A renames fare_amount to fareAmount, both services' unit tests pass. Staging explodes. Consumer-driven contract tests catch this at build time.

Structured logging earlier. Searching Kibana for a booking ID across 10 services when half log bookingId=ABC123 and the other half log Processing booking ABC123 for user xyz is painful. Consistent field names should be a service template requirement on day one.

More aggressive load shedding. At 90% capacity, return 429s for low-priority endpoints and keep booking and payment paths fully served, instead of letting everything degrade equally.

The Takeaway

Building microservices at this scale is about understanding failure modes. Every pattern — circuit breakers, tracing, async messaging, bulkheads — exists because something broke in production and we needed it to not break the same way again. The system processes 130 million requests a day not because we got the design right on the first try, but because we built the instrumentation to see what was breaking and the patterns to contain the blast radius.

Originally published on krishnakky.com — the full version has additional code examples, Kafka event-driven architecture details, and a third production war story about consumer group rebalances.