At twenty past eight on a Monday every call to our logistics partner began failing with the same Java message: unable to find valid certification path to requested target. Not some calls. All of them, from every instance, starting at a moment that matched nothing on our side. No deploy, no config change, no restart.
Their first line support told us, politely and correctly, that their API was healthy, that their certificate had been renewed on Saturday, that it was valid until next year, and that their own uptime monitoring was green. They sent a screenshot of the padlock in a browser.
Everything in that reply was true. The certificate was valid. What had changed was the chain.
A server is supposed to send its own certificate together with the intermediate certificates that connect it to a root your client already trusts. Their renewal had installed the new leaf and left the intermediate out of the bundle. A browser papers over that without telling anyone. It keeps intermediates it has seen before from other sites, and if it still cannot build a path it fetches the missing one from a URL printed inside the certificate itself. Our service does neither. It validates against what was sent plus what is in its truststore, and when the path has a hole in it, it stops. The same certificate is therefore genuinely valid for anything with a repair mechanism and genuinely invalid for anything without one, and nobody on either side of the call had ever had a reason to know the difference.
They corrected the bundle in four hours. We spent the first of those putting the intermediate into our own truststore, which worked and which I removed a week later, because carrying somebody else's chain around is a fix with a shelf life.
What stayed is a daily job that opens each partner endpoint from a container with an empty certificate cache, reads back the chain exactly as sent, and verifies it standalone. It alerts on an incomplete chain and on anything expiring within three weeks. Our integration smoke tests now use the same HTTP client and truststore the service ships with, rather than a command line tool that is cleverer than our code.
A padlock tells you the browser coped. It says nothing about what the server actually sent.
– Sergey Shinder
Top comments (0)