Two providers changed the rules, and my app went back to the hangar
Showtime on the Balcony (Gone Wrong)
"I like planes too. I wish I had an app that showed me the planes nearby and their details. That'd be so cool."
One of my buddy's buddy's buddies said this at a party, while we were out on the balcony getting some fresh air. Well, I was getting fresh air. He was smoking the whole time.
That was my moment to shine. He was describing, almost word for word, the app I'd won my first hackathon with. So I thought to myself: "It's showtime!"
"I have just the thing for ya"
I said with a slight smirk, pulled out my phone, typed metalbirdswatch.pilotronica.com into the browser... only to get this:
My smirk turned into a "what the hell is this bull shirt" face in about half a second. Obviously I wasn't going to show anyone my app in that state.
"Well... umm... yeah... some kind of problem, probably browser-related. I'll show you later."
It wasn't browser-related.
I knew that for sure. I figured CARTO, the service that draws my map, had changed something in their API or made it paid. Either way it'd be a quick fix, and that'd be it. Easy.
"I'll fix it tomorrow and send you the link so you can check it out"
I said.
Little did I know I was about to find out that the planes were gone too. Metal Birds Watch without the metal birds? Doesn't sound great, does it?
A Quick Layover
But let's not get ahead of ourselves. What's the point of explaining anything without proper storytelling? Storytelling on a technical write-up is like a good ragù on tagliatelle. Sure, you can eat the tagliatelle plain, but the sauce makes it so much better.
Otherwise I'd just drop a link to the release notes and call it a day. Easy, simple. But let's not do that.
Why This One Is Personal
By the way, this is me proudly holding a Boeing 737-800 model while recording the project video.
...and yes, the plane was more camera-friendly than I was.
I built Metal Birds Watch for the GitHub Copilot CLI Challenge on DEV. It was the first hackathon of my life, and I started it on hard mode: the competition was brutal, with over 400 participants.
But I won.
While building it I had every obstacle in the world, including back, neck and arm pain from shoveling my driveway for hours non-stop. (Pro tip: don't shovel an entire driveway in one go right before a hackathon.)
I could barely sit up straight while writing the whole thing. My back was killing me.
So yeah, as you can imagine, this app is pretty special to me. It's my baby.
Engine #1: The Map That Said No
The morning after the party. I poured a cup of dark coffee, sat down at my desk, put on some music (RHCP, as usual), and got to work. Time to fix this thing.
the first job was to figure out why on earth the map looked like that.
It turned out CARTO had changed its policy around 28 August 2026. Their free basemaps now require an API key, and requests without one get tiles with "API KEY REQUIRED" stamped diagonally across them.
I didn't have a CARTO account, so they had no email address to warn me, and I'm sure plenty of other users were in the same position. Their best way to tell us was to cover our maps with the message. Fair enough.
A Quick Patch on the Tarmac
This was the easy part. I requested a free key, which allows 5 million tile requests per month, and added it to the tile URL. My map URLs live in CSS custom properties, one per theme:
/* variables.css, before */
--map-tiles: 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png';
/* after */
--map-tiles: 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png?key=YOUR_KEY';
That was the whole change: not a single line of JavaScript. The map reads its tile URL from CSS on startup and again on every theme switch, so both picked up the key automatically:
const tileUrl = getComputedStyle(document.documentElement)
.getPropertyValue('--map-tiles')
.trim()
.replace(/['"]/g, '');
tileLayer = L.tileLayer(tileUrl, { /* ... */ }).addTo(map);
Past me, thank you for that architecture decision.
Pre-Flight Checklist
There were a few details to think through, though.
1. Should the key be committed to a public repo? My first instinct was "never commit keys." But a map tile key can't be secret: the browser sends it with every tile request, and anyone can see it in DevTools. What protects it is a domain restriction set in CARTO's dashboard, so the key only works on my production domain. Committing it was fine.
2. A CSP trap. CARTO's current docs show the apex domain, basemaps.cartocdn.com. My Content Security Policy only allows https://*.basemaps.cartocdn.com, and that wildcard doesn't match the apex domain. If I'd copied the URL from the docs, the tiles would have been silently blocked by my own CSP. Keeping the {s}. subdomain form avoided a second outage in the same fix.
3. The domain restriction backfired locally. Because the key only works on production, every tile on localhost returned 403, and my local map went completely grey. I later added a small helper that swaps the key on localhost for a personal development key stored in the browser, or removes it entirely:
function getTileUrl() {
const url = getComputedStyle(document.documentElement)
.getPropertyValue('--map-tiles').trim().replace(/['"]/g, '');
if (!isLocalhost) return url; // production: unchanged
const devKey = localStorage.getItem('carto-dev-key');
return devKey
? url.replace(/key=[^&]+/, `key=${devKey}`) // your own dev key
: url.replace(/[?&]key=[^&]+/, ''); // no key: watermark, but it works
}
My development key lives in my browser, not in a file, so it can never be committed by accident.
I added the key, tested it, and everything was back to normal. I was so happy to see my favorite app, my baby, running smoothly again.
Time to deploy.
Engine #2: Where Did All the Birds Go?
I ran the pipeline, and it deployed. Alright, let me check production.
OK, whoa whoa whoa whoa. What do you mean, Failed to fetch plane data: API error: 503 Service Unavailable?
The map was clean, and it was also completely empty. No planes. The status indicator said OFFLINE, and an error toast showed up every 20 seconds or so like clockwork: the frontend asked for planes, waited about 10 seconds, got a 503, then tried again.
The Instrument That Lied
The backend logs were very clear about the problem:
Authentication failed - check OpenSky credentials
Makes sense. My flight data comes from OpenSky Network, and the backend authenticates with OAuth client credentials before fetching aircraft. Maybe the credentials had expired, or been revoked.
So I rotated them. It didn't help.
So I rotated them again. It didn't help either.
Two credential rotations and several hours later, I finally looked closely at the code that produced the message:
tokenRefreshPromise = (async () => {
try {
return await getAccessToken();
} catch (error) {
console.error('Failed to get OAuth token:', error.message);
throw new Error('Authentication failed - check OpenSky credentials');
}
// ...
})();
Can you see it? Every failure became "Authentication failed": network timeouts, DNS errors, HTTP errors, and genuine credential problems. The code swallowed the real error and replaced it with a confident, specific and completely wrong diagnosis.
The error message had been lying to me all along, and I'd written it myself. That hurt.
Opening Up the Engine
Enough guessing. If the error message wasn't going to tell me the truth, I'd make the error tell me itself.
Step 1: log everything. Instead of trusting my own summary, I logged every detail of the raw error: its code, syscall, address, port, and most importantly its cause. That's how you tell a DNS failure from a TCP failure, a TLS failure or an actual HTTP error:
console.error('OpenSky auth error details:', {
name: error.name,
code: error.code,
cause: error.cause && {
name: error.cause.name,
code: error.cause.code,
message: error.cause.message
}
});
I deployed, waited, and the logs finally told the truth. It wasn't an authentication failure at all. It was a TCP connect timeout to OpenSky's auth server. My backend never got as far as sending the credentials I'd rotated twice, because it couldn't even open a connection.
Step 2: be patient. A timeout? Maybe OpenSky's server was just slow to respond from Railway's network. So I increased the auth request timeout to 30 seconds:
signal: AbortSignal.timeout(30000)
I deployed, waited, and got exactly the same error, still after 10 seconds.
Wait, what? I'd just set it to 30.
Step 3: find out where the 10 seconds came from. Node's fetch is built on undici, and undici has its own separate 10-second limit for opening a TCP connection. That limit applies before any request timeout you set yourself. So my 30-second timeout never got a chance to run; undici gave up first.
To change it, you pass fetch a custom undici Agent:
const { Agent } = require('undici');
const authDispatcher = new Agent({
connect: { timeout: 30000 } // give the TCP handshake 30s instead of 10s
});
await fetch(OPENSKY_AUTH_URL, {
// ...
dispatcher: authDispatcher,
signal: AbortSignal.timeout(30000)
});
My theory was that the TCP handshake from Railway to OpenSky was just slow and would get there eventually if I gave it enough time.
I deployed, waited, and waited some more.
Reader, it did not get there eventually. I gave it 30 seconds. I could have given it 30 minutes. A connection that isn't slow but ignored will never complete, however long you wait.
Silence on the Frequency
This was the turning point, so it's worth a short technical detour.
When you try to connect to a server, there are roughly two ways it can go wrong:
- Connection refused: the server's machine answers and says "nobody's listening on that port". It's fast and polite.
- Connection timeout: nothing answers at all. Your packets leave, and nothing comes back.
A timeout on a server that's otherwise up and working for everyone else usually means one thing: something is dropping your packets on purpose, typically a firewall. And the only unusual thing about my backend's packets was where they came from: a cloud provider's IP range.
Tower, Do You Copy?
So I asked OpenSky directly:
Do you block hosting and cloud provider IP ranges?
"Yes."Can a specific address be allowlisted?
"No."What's the recommended approach for hosted apps?
They're considering a paid option to handle the requests coming from cloud providers.
So that was that. The blocking is deliberate, exceptions aren't possible, and a paid option might come someday. There's currently no supported way to use OpenSky from a hosted backend.
That's a bummer. Real bummer. I really liked OpenSky's service.
To be fair to OpenSky, I get it. They run a free, community-driven service, and they're being flooded by automated traffic from cloud servers.
Back to the Hangar
So it was time to land the plane. I fixed one engine mid-air, but I couldn't do it for the second one. So I had to land and go back into the hangar.
Adding an API key is easy, especially a public one. Researching and integrating a new flight data provider is not something you do in a few minutes, or even a few hours.
So I put the app into maintenance mode. It'll take a while to find the right provider, but hey, great things take time.
Ten Seconds of Doubt
That was my first idea, and it's a trap. The backend takes about 10 seconds to return that 503. A banner that appears after the failed request leaves users staring at an empty map for 10 seconds first. That's exactly when they decide the app is broken, or that there just aren't any planes overhead.
So the maintenance page is unconditional. It appears immediately and doesn't wait for anything to fail. One flag controls it:
MAINTENANCE: {
ACTIVE: true, // Set to false once the new data provider is live
ID: 'opensky-2026-09',
SINCE: '2026-09-20', // Used for the "day N" counter
ISSUE_URL: 'https://github.com/georgekobaidze/metal-birds-watch/issues/98'
}
When it's on, the map doesn't load, the browser doesn't ask for your location, and polling never starts:
function startPolling() {
// Skip polling during maintenance - the backend can't reach the data provider,
// so every request would time out and show an error toast
if (CONFIG.MAINTENANCE.ACTIVE) {
return;
}
// ...
}
No more doomed requests, no more error toasts every 20 seconds, and no more "is it me?"
Speaking Fluent Aviation
Since this is an aviation app, the maintenance page talks like one:
- A NOTAM-style notice. A NOTAM ("Notice to Air Missions") is how pilots are told about closures and outages. The notice sits over a full-screen photo of a plane being repaired in a hangar.
- A radio call: "All stations, all stations: Metal Birds Watch is out of service for unscheduled maintenance until further notice."
- A counter: "Grounded since 20 Sep 2026 · day N", updated automatically.
- Ask the Tower: a small panel that answers the questions people actually have, such as Is it something on my end? ("Negative."), Is my logbook safe? ("Affirmative. It lives in your own browser.") and When will it be fixed?
There's no AI in the Tower, which felt right given the circumstances. It's a list of pre-written questions and answers with a typing indicator for effect:
logbook: {
question: 'Is my logbook safe?',
answer: 'Affirmative. Your logbook lives in your own browser, not on our servers, so the ' +
'outage can\'t touch it. Open it anytime with the 📖 button up top.',
followUps: ['stillWorks']
}
Looking for a New Engine
Replacing the data provider is tracked in issue #98.
The good news is that my OpenSky code is already an isolated adapter. It converts OpenSky's response into my own data format, and nothing above it knows where the data comes from. A new provider is one new service file behind the same interface. The routes, the cache and the entire frontend stay the same.
Reading the Black Box
If I had to summarize the flight recorder:
- Silent failures are the worst failures. An HTTP 200 containing a watermark and a 503 nobody was watching both went unnoticed for about three weeks. I found out at a party, from a phone, while trying to show off. Some basic monitoring would have found them for me.
- Your error messages are part of your debugging tools. A misleading one costs hours. Mine cost two credential rotations and an afternoon.
- A timeout and a refusal tell you different things. A refusal means nobody is listening. A timeout, when the server works for everyone else, usually means someone chose not to answer you.
- Free services can change their rules at any time. That's not a complaint. They don't owe anyone anything. But design for it: isolate your providers behind adapters, and have a plan for the day one disappears.
- When you can't fix it right away, say so clearly and immediately. An honest maintenance page is better than an empty map that makes people think the app is broken.
Final Approach
As for my buddy's buddy's buddy: I still owe him that link. It's coming, I promise, with planes on it.
Until then, Metal Birds Watch is in the hangar, the mechanics are at work, and the new engine is on order.
We promise we'll be flying again together soon. 🛫
Follow the progress in issue #98, or read the v1.0.2 release notes.
Enjoyed this write-up? Let's stay connected!
I share more software engineering insights, projects, and experiments across these platforms:
Top comments (0)