upordown.online

Posted on Nov 9

If I set up our status page again, here's what I'd do differently

#sre #devops #incidentmanagement #webdev

A status page is one of those things every team sets up in an afternoon. Pick a tool, list your components, link it from the footer. Done.

Then the first real outage arrives, and you find out what the afternoon didn't cover. The page is green while customers are shouting. Nobody remembers the login. The update you finally post says "We are investigating an issue" and nothing else for 90 minutes.

Here's what I'd do differently if I were setting one up again.

1. Host it somewhere that doesn't share your failure modes

The first status page I worked with lived on a subdomain served by the same infrastructure as the product. When the load balancer went down, so did the page explaining that the load balancer was down.

A status page should be independent of everything it reports on: different hosting provider, different DNS provider if you can manage it, and no dependency on your auth system, CDN configuration or main database. Hosted status page services exist largely for this reason. If you self-host, put it somewhere genuinely separate, and test it by imagining your main provider disappearing entirely.

Don't forget DNS. If status.example.com is served by the same DNS provider as example.com, a DNS outage takes both. Some teams use a completely separate domain for exactly this reason.

2. List components the way customers think, not the way we deploy

My first component list was an architecture diagram: "API gateway", "worker cluster", "Postgres primary", "Redis". Customers don't know what any of that means. They want to know whether they can log in, whether checkout works and whether their emails are going out.

Next time, I'd list user-facing capabilities:

Website and dashboard
Login and sign-up
Payments and checkout
API
Email notifications
Integrations (named, if there are a few important ones)

Keep the list short. Ten components with three statuses each is already more than most people will read.

3. Decide who can update it, and make sure they can at 3am

During one incident, the only people with status page access were asleep or on a plane. The engineers fixing the problem couldn't tell anyone.

I'd make sure:

Everyone on call can post updates. Not just managers, not just support.
Logins work without the systems that might be down. If status page access goes through your company's single sign-on, and single sign-on is affected, you're locked out. Keep a tested break-glass account.
Access is tested quarterly, as part of an incident drill.

4. Automate the "something's wrong" signal, not the explanation

There's a tempting idea: connect monitoring directly to the status page so it turns red automatically. I'd do it partially.

Automating the first signal is good. If an external check sees the login page failing from multiple regions for a few minutes, showing "Investigating possible issues with login" quickly is better than silence.

But fully automated status updates have two problems: false alarms erode trust, and automation can't explain anything. I'd automate detection and a holding message, then require a human to confirm, update and resolve.

5. Write message templates before you need them

The worst status updates are written by stressed people at speed. Prepared templates make the difference:

Investigating: "We're seeing errors affecting [capability] for some customers. We're investigating and will update by [time]."
Identified: "We've found the cause: [short plain description]. We're working on a fix. Next update by [time]."
Monitoring: "A fix is in place and [capability] is recovering. We're watching closely."
Resolved: "[Capability] is working normally. Between [start] and [end], [what customers experienced]. We'll publish a summary by [date]."

The most important part of every template is "next update by [time]". It tells customers when to come back, and commits you to updating even when there's no news.

6. Say what customers experienced, not what broke internally

"Elevated error rates on the primary database replica" is accurate and useless. "Some customers can't save changes to their projects; existing data is safe" is what people need.

I'd add a line to every template asking: what can't customers do right now, and is their data safe? Those two answers cover most of what people want to know.

7. Show history honestly

A status page with a perfect history isn't reassuring. It's suspicious. People know systems fail.

I'd keep the incident history visible, including short incidents and partial degradations, and link postmortems or summaries for the significant ones. Honest history builds more trust than an unbroken row of green.

It also helps your own team. A visible record of incidents makes patterns obvious: the same component failing every month is hard to ignore when it's on a public page.

8. Let people subscribe, and respect their inboxes

Customers who care will want notifications by email, RSS or webhook. Offer them. But I'd be thoughtful about volume: let people subscribe to specific components, and avoid sending a notification for every minor edit. Batch small updates into the next scheduled one.

What I'd keep

Some things I'd keep exactly the same:

A link from the product itself, for example in the help menu and on error pages, not just the website footer.
Plain language throughout. If a customer needs a glossary to read it, rewrite it.
Times shown with time zones, or in the reader's local time.

The short version

Host it independently. List what customers use. Make sure on-call can update it. Automate detection, not explanations. Prepare templates with a "next update by". Describe impact, not internals. Keep history honest. Let people subscribe.

External checks from outside your infrastructure are what tell you, and your status page, that something's wrong before customers do. That's what upordown.online is built for.

DEV Community