Short answer: use one external probe per customer region for the checkout health endpoint, one deadline-based heartbeat for every critical cron job, and make deployment rollback depend on both signals rather than on process uptime alone.
For a small fintech SaaS, the least complex useful design is two independent tests. The first asks whether a customer in the EU or US can reach a shallow checkout endpoint. The second asks whether the reconciliation job checked in before a fixed deadline. Keep those signals outside the application process, or the component being measured can fail at the same time as its evidence.
Don't start with a large telemetry stack. Start with failure coverage.
What should simple uptime monitoring check for checkout health and missed cron runs?
Consider a bounded failure exercise. A checkout deployment starts, the process stays alive, and /health/live continues to return 200; however, the new release cannot reach a required dependency, while the hourly reconciliation cron never completes. A process-only monitor calls the deployment healthy. The useful invariant is stricter: a release is eligible to remain deployed only while a regional synthetic check can complete the chosen dependency path and every critical background job finishes inside its declared time budget.
That distinction matters because a health endpoint and a cron heartbeat answer opposite questions. The endpoint is pulled on a schedule: "Can I use this path now?" The job pushes evidence after success: "Did this work finish on time?" Combining them into one green badge hides which clock stopped. It also creates bad rollback behavior — an endpoint can recover while a missed financial reconciliation still requires intervention.
Define three outcomes before choosing a service:
- Liveness says the process can serve requests. It should be shallow and cheap.
- Readiness says the instance can perform the dependency path needed for checkout traffic.
- Job freshness says a named run completed before its deadline, with late and missing runs treated separately.
An HTTP status is only the first layer. Return a small machine-readable body with a stable check name, state, and timestamp, then record latency at the probing side. The metric contract in this example uses the Prometheus naming rules: a base unit, one logical meaning per metric, and _total for counters. Names such as checkout_health_probe_duration_seconds and checkout_reconcile_runs_total leave fewer traps than a mixed-unit checkout_health_time or a counter without a suffix.
Build a narrow health contract
The endpoint contract should be boring. A load balancer can use liveness, while the external monitor exercises readiness. Don't expose account data, dependency credentials, exception text, or an unbounded dependency graph. A checkout readiness probe that recursively checks every internal service expands the alert surface until operators can't tell whether customers are blocked.
This Go server keeps the transport contract small. Replace checkoutReady with a bounded dependency check that represents the minimum path required to accept checkout traffic; give that call a shorter timeout than the external probe interval.
package main
import (
"context"
"encoding/json"
"log"
"net/http"
"time"
)
type health struct {
Check string `json:"check"`
Status string `json:"status"`
CheckedAt time.Time `json:"checked_at"`
}
func writeHealth(w http.ResponseWriter, code int, check, status string) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
_ = json.NewEncoder(w).Encode(health{
Check: check, Status: status, CheckedAt: time.Now().UTC(),
})
}
func checkoutReady(ctx context.Context) bool {
// Put one bounded check for the checkout-critical dependency path here.
select {
case <-time.After(25 * time.Millisecond):
return true
case <-ctx.Done():
return false
}
}
func main() {
mux := http.NewServeMux()
mux.HandleFunc("GET /health/live", func(w http.ResponseWriter, _ *http.Request) {
writeHealth(w, http.StatusOK, "checkout_liveness", "ok")
})
mux.HandleFunc("GET /health/ready", func(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 750*time.Millisecond)
defer cancel()
if !checkoutReady(ctx) {
writeHealth(w, http.StatusServiceUnavailable, "checkout_readiness", "blocked")
return
}
writeHealth(w, http.StatusOK, "checkout_readiness", "ok")
})
server := &http.Server{
Addr: ":8080",
Handler: mux,
ReadHeaderTimeout: 2 * time.Second,
}
log.Fatal(server.ListenAndServe())
}
The numbers above are design inputs, not universal recommendations. The 750 ms timeout only makes sense if it fits inside the checkout latency objective and leaves room for the external monitor's own network budget. Your mileage may vary; measure the dependency's tail latency, then set the timeout and alert window from the SLO rather than copying a round number.
Probe the same contract from at least one EU location and one US location when customers depend on both paths. Alert on each region independently, but page only on customer-impacting combinations that match the service objective. A single regional failure may justify investigation; two consecutive failures from multiple regions may justify rollback. Those are example policies, and a team should derive the actual counts from its error budget and tolerance for false positives.
Make the cron job prove completion
A scheduler saying "started" is weak evidence. Send the heartbeat after the reconciliation transaction and durable result recording have succeeded. If the process exits early, the absence of that success signal becomes the alert; no special crash hook is needed.
Here is a small heartbeat sender that uses only the Go standard library. The URL is configuration because a managed monitor and a self-hosted receiver should be interchangeable at this boundary.
package heartbeat
import (
"context"
"fmt"
"io"
"net/http"
"time"
)
func Success(ctx context.Context, client *http.Client, url string) error {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, nil)
if err != nil {
return err
}
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
_, _ = io.Copy(io.Discard, resp.Body)
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("heartbeat returned status %d", resp.StatusCode)
}
return nil
}
func Client() *http.Client {
return &http.Client{Timeout: 3 * time.Second}
}
Don't put a customer identifier, payment reference, or settlement amount in the heartbeat URL. Give the monitor a pseudonymous job name and keep detailed diagnostics in controlled logs. RFC 5424 defines severity levels from Emergency through Debug; use those semantics consistently so a late run can be classified differently from a confirmed failed reconciliation, instead of mapping every event to the loudest level.
The deadline needs capacity headroom. If a job runs every hour, model its normal runtime, retry budget, upstream maintenance window, and the latest completion time the business can tolerate. A deadline equal to the schedule interval leaves no space for retries. I would document all four values beside the monitor because the person changing the cron expression six months later otherwise has no reason to update the alert.
Tie alerts to rollback safety
Rollback automation should consume explicit deployment evidence, not a generic uptime percentage. Record the release identifier with probe results inside the deployment system, keep the public health response release-agnostic, and evaluate a short canary window before increasing traffic. The safe action is deterministic: stop promotion when checkout readiness breaches its canary threshold; roll back when the breach is attributable to the candidate and rollback itself remains within the operational budget; page without an automatic rollback when the missed cron run belongs to work that an older binary cannot safely repeat.
That last branch is easy to miss. A payment reconciliation job may have side effects, so deploying yesterday's code doesn't necessarily undo today's partial work. Test idempotency with duplicate run identifiers, interrupted transactions, and delayed heartbeat delivery. A synthetic 503 from readiness is useful deployment evidence. A missing heartbeat is temporal evidence. They should meet in the incident view, but they shouldn't trigger identical automation.
Capacity planning belongs here too. Estimate probes per minute as regions multiplied by endpoints multiplied by frequency, then add retry traffic. Estimate heartbeat cardinality as jobs multiplied by environments, not customers; per-customer monitors can turn a small operational control into an unbounded series. Before rollout, load-test the endpoint above expected probe concurrency and confirm that monitoring traffic cannot exhaust the same connection pool checkout needs.
Use a rollback drill to validate the chain: deploy a canary whose readiness dependency check is deliberately closed, confirm that it receives no broad traffic, observe the external regional failures, and verify that promotion stops. Separately, run the reconciliation job in a test environment without sending its final heartbeat, wait past the deadline, and check that the alert identifies the job, environment, expected deadline, and runbook. No production outage is required to test either path.
Choose the operating model, then the tool
Healthchecks.io, StatusCake, and Better Stack are reasonable names to include in an initial market scan because they appear in the reader's candidate set, but a shortlist isn't a decision. Verify current regional probe locations, heartbeat semantics, notification paths, retention, data residency, export behavior, and contract terms in each provider's own documentation before selecting one. I'm not sure which will best fit a given team's residency and on-call constraints without those current answers, and product details can change.
The durable comparison is buy versus build:
| Decision area | Managed monitoring | Self-hosted receiver and probes |
|---|---|---|
| Rollback independence | Useful when probes run outside the deployment failure domain | Useful only if the control plane is isolated from the application |
| On-call load | Provider operates the monitoring control plane | Team owns upgrades, storage, regional agents, and alert delivery |
| Data control | Requires review of residency, retention, and export terms | Team chooses placement and retention, then operates both |
| Integration | Faster when HTTP probes, heartbeats, and existing paging fit | Flexible when internal policy needs a custom event contract |
| Exit cost | Lower when raw events and alert rules can be exported | Lower vendor dependency, with ongoing maintenance cost |
The catch is that simple external monitoring is not suitable when checkout correctness depends on traces across many asynchronous services, or when compliance requires evidence that a managed offering cannot place in an approved region. Add tracing and metric correlation for the first case. Stick with an isolated self-hosted design for the second, provided the team accepts its on-call and capacity burden. Conversely, don't self-host merely to avoid a subscription if two engineers would then own a second control plane; the service's error budget must include the monitor itself.
My decision rule is plain: choose the smallest operating model that can prove regional reachability and job freshness from outside the release, export its evidence, and survive the failure domain that triggers rollback. Revisit the choice when job count, regions, compliance boundaries, or on-call staffing changes.
References
- Prometheus, "Metric and label naming": https://prometheus.io/docs/practices/naming/
- IETF, "RFC 5424: The Syslog Protocol": https://datatracker.ietf.org/doc/html/rfc5424
Top comments (0)