Short answer: for a small EU business app, start with a SaaS probe for independent uptime checks and keep a tiny self-hosted health endpoint plus cron heartbeat for application context. The useful boundary is signal quality: an external check tells you that customers can reach the app, while an internal heartbeat tells you that last night's work actually finished. One signal cannot stand in for the other.
In a gaming data pipeline, this distinction is painfully concrete. A storefront can return HTTP 200 while the nightly match-history export has not written a row since yesterday. Conversely, the export can finish while the public API is unreachable from Frankfurt. I want both facts, with different alerts, and I want the alert text to say which fact is missing.
The failure chain usually starts innocently. A scheduler launches the export at 01:00, the worker acquires a database lock, and the public health route keeps answering because the web process is healthy. At 01:17 the worker loses its connection while reading a shard. The process catches the exception, writes a final log line, and exits with a status that the scheduler does not surface. There is no new heartbeat, yet the uptime chart stays green all night. At 08:00, a player support query exposes the missing match history. Searching structured logs by runId reveals the shard error in seconds; without that identifier, the team scans thousands of unrelated request lines and first blames the web tier. This is why I treat a heartbeat as a fact with a timestamp, not as another uptime check. It narrows the search, but it does not pretend to explain the failure by itself.
Keep it boring.
How should a small business choose self-hosted or SaaS uptime monitoring for a cron health endpoint?
Think of the system as two arrows. A probe outside your network requests /health from an EU region; the application reports dependency status and a bounded response. Separately, the cron job sends a heartbeat after it commits its output. The first arrow answers “can a user connect?” The second answers “did the scheduled operation complete?” A dashboard that merges them into one green light creates noise disguised as confidence.
For a simple endpoint, return a fast 200 only when the process is ready to serve traffic. Do not make the endpoint run a database migration, scan a queue, or perform a full pipeline query. Those checks turn a monitor into a load generator. A shallow liveness route and a deeper readiness route can coexist, but alert policies should name the difference.
The cron heartbeat should carry a job name, run identifier, completion timestamp, and duration. Treat a missing heartbeat as a delayed signal, not as proof of a root cause. The worker may be waiting on a lock, the scheduler may be paused, or the commit may have failed after the final log line. That uncertainty is useful: it tells the responder where to look next.
What does a high-signal monitoring design look like for a nightly gaming pipeline?
Start with three measurements: endpoint availability, heartbeat age, and pipeline outcome. OpenTelemetry describes metrics as measurements recorded over time; counters, gauges, and histograms each answer a different question. A gauge for heartbeat_age_seconds supports a freshness alert, while a counter for failed runs supports a trend. A histogram for run duration helps distinguish a slow-but-successful export from a job that never started.
Here is a small TypeScript shape for the event written after the database commit. It is an application contract, not a vendor SDK.
type PipelineHeartbeat = {
job: "nightly-match-export";
runId: string;
completedAt: string;
durationMs: number;
rowsWritten: number;
status: "ok" | "failed";
};
export async function sendHeartbeat(event: PipelineHeartbeat): Promise<void> {
const response = await fetch(process.env.HEARTBEAT_URL!, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(event),
});
if (!response.ok) {
throw new Error(`heartbeat delivery failed: ${response.status}`);
}
}
The ordering matters. Write the export, commit it, then send the heartbeat. If the heartbeat is sent first, a crash can produce a green alert for work that never became visible. If delivery fails after the commit, retry with the same runId; the receiver should be idempotent. I once reviewed a pipeline where a retry generated a second “success” row and suppressed the real failure. The fix was a unique key on (job, runId) and a separate outcome field. Small change. Big signal improvement.
Logs complete the picture. Emit one structured record for start, one for outcome, and include runId, row count, duration, and a failure class. Avoid putting a unique player ID or full exception text into metric labels. High-cardinality labels make aggregation noisy and expensive; keep those details in logs, where a responder can search them deliberately.
Self-hosted versus SaaS: where does each option fail?
A SaaS monitor has an independent network position, managed probe scheduling, and an alert delivery path you do not have to operate. That independence is its main engineering value. If the same cloud account hosts both your app and your monitor, a regional outage can make the monitor blind exactly when you need it.
Self-hosting gives control over data location, retention, and network access. It can fit an EU-only environment with strict egress rules, and it lets a team inspect every component. The catch is operational: you now own probe availability, clock drift, upgrades, alert routing, and the second failure domain. A monitor on the same VM as the app is not an independent monitor; it is another process waiting for the same power, kernel, and network.
| Decision point | SaaS probe | Self-hosted probe |
|---|---|---|
| Independent reachability | Usually strong when probes run outside your account | Requires a separate region, account, or facility |
| EU data controls | Verify probe locations, retention, and sub-processors | You choose storage and retention, then operate them |
| Cron heartbeat | Often a simple signed URL or HTTP event | You must expose and protect the receiver |
| Alert delivery | Included as a managed path, subject to plan limits | Your team runs mail, chat, or paging integrations |
| Failure during your outage | Provider can still observe the app | A colocated monitor may disappear with it |
Neither option is automatically correct. Self-hosting is not suitable when nobody owns patching and on-call rotation. SaaS is a poor fit when policy forbids an external processor or requires probe execution inside a private network. Stick with the option whose failure mode your team can explain at 02:00.
How can alerting reduce noise without hiding a missed cron job?
Use separate conditions and a short delay. Page on a failed endpoint check after two consecutive probes, not on one transient timeout. Create a ticket or chat notification when heartbeat age crosses the expected completion window, then page only after a second threshold. For example, if the export normally completes by 02:15 UTC, notify at 02:30 and page at 03:00. Those are policy values, not universal truths; your mileage may vary with the pipeline's variance.
Alert messages should include the measured value and the next diagnostic action: “nightly-match-export heartbeat is 52 minutes old; inspect scheduler and run ID 8f2…” is actionable. “Cron down” is a guess. Include a link to the structured-log query, but keep the alert itself small enough to read on a phone.
Deduplicate by job and incident window. Silence dependent alerts when the probe itself is known to be unavailable, and record every suppression so a quiet period does not become an invisible period. Test the path monthly: force a non-production health failure, skip a heartbeat, and verify that the right person receives exactly one page. A green dashboard is not a test.
Security belongs in this design. Authenticate heartbeat requests with a rotating token or signature, reject stale timestamps, and never put secrets in query strings that may be logged. Keep the public health response free of build metadata and customer data. For EU operations, document retention and processor roles before production traffic flows through an external monitor.
The practical rule is simple: use an external signal for reachability, an internal signal for work completion, and structured logs for the explanation. Compare the three during an incident instead of asking one metric to tell the whole story.
Top comments (0)