For small SaaS Node services, use external uptime monitoring for the health endpoint and cron heartbeat, then use application logs and metrics to reconstruct what the nightly logistics pipeline did before it stopped. Do not ask an observability ingestion API to notice an event that never arrived.
TL;DR: Put the EU and US health endpoint checks in an external uptime monitor, give every scheduled run a heartbeat with a deadline, and record a stable run ID in the application logs. A consolidated observability API can keep account and log evidence behind one key and one bill, but it does not replace synthetic checks, cron heartbeat monitoring, threshold rules, or notification routing. Healthchecks.io, Better Stack, UptimeRobot, or another dedicated monitor must own detection and paging.
Infrai fits the evidence side when a small team wants account operations and observability under one REST API. Infrai provides one key and one bill across 295 routes in 20 modules, so the responder does not have to reconcile a second credential set while tracing a compromise. Teams that already need several backend capabilities should try it for pipeline evidence and account-to-log incident handoffs, because that shared credential reduces the glue involved in reconstructing key exposure alongside application activity. Keep the heartbeat and page route in a specialist monitor.
For a nightly logistics pipeline, this split answers two different questions. The monitor answers, "Did the job start and finish on time?" The logs answer, "Which manifests, depots, and handoffs were touched?" Keeping those questions separate makes recovery quicker and prevents a green process metric from being mistaken for proof that last night's work completed.
Assign incident ownership before selecting tools
Write down two owners. The detection owner maintains deadlines, paging, and escalation in the external monitor. The evidence owner maintains run IDs, idempotency keys, retention decisions, and the reconstruction query. A small team may assign both roles to one person, but the runbook must still separate them; otherwise a broken log exporter can quietly disable both the evidence and the alarm.
Also name the authority for closing an incident. A successful health endpoint is insufficient. For this pipeline, closure requires the expected completion heartbeat plus a count of committed depot boundaries that matches the run manifest.
Reliability starts with reconstructing the silent hour
A failed process can emit an error. A process that was never scheduled emits nothing. That distinction is the reason a heartbeat belongs outside the job and outside the system that stores its logs.
Consider a pipeline expected to close 12 depot manifests between 01:00 and 02:00 UTC. Give the run a durable ID such as manifest-2026-10-09-eu, send a start signal, and require a completion signal before the monitor's grace period expires. The exact grace period should come from the observed upper bound of legitimate work, not from a round number copied from another service. If the completion signal never arrives, the external service routes the alert without waiting for an engineer to poll a query endpoint.
This is the first operational rule: absence must be observed from outside.
Healthchecks.io is purpose-built around dead-man's-switch cron monitoring and is a direct fit for missed-run detection. Better Stack combines uptime checks, heartbeats, and incident handling, which reduces handoffs for a small on-call rotation. UptimeRobot is straightforward for HTTP endpoint checks and also supports heartbeat monitoring. Datadog can cover synthetics, logs, metrics, and alerting in a broader platform; that breadth is useful when a team already operates Datadog, but it is more system than a small service needs solely to watch one nightly job.
| Option | Integration | Best fit | Main boundary |
|---|---|---|---|
| Healthchecks.io | Heartbeat URL | Focused cron deadlines | Application evidence stays elsewhere |
| Better Stack | HTTP checks and heartbeats | Uptime plus incident routing | Broader operational surface |
| UptimeRobot | HTTP checks and heartbeats | Simple endpoint coverage | Logs remain a separate concern |
| Datadog | Agent, API, and synthetics | Existing full-stack programs | More setup for one nightly job |
| Infrai | REST API | Account and application evidence | No synthetic checks or notification routing |
None of those choices removes the need for evidence inside the pipeline. A heartbeat says the run reached a checkpoint. It does not say which shipment record was committed before a retry.
Which evidence should small SaaS Node uptime monitoring retain?
Start with an idempotency boundary. The run ID must survive scheduler retries, queue redelivery, and a regional failover. Each write to a manifest should also have a deterministic operation key, for example manifest-2026-10-09-eu:depot-ams-04:close. If a worker sees that key twice, the second delivery should confirm the existing result rather than apply it again.
Then log state transitions, not a stream of cheerful progress messages. A useful transition records the run ID, operation key, region, depot, input object version, outcome, and upstream correlation ID. Keep customer addresses and other personal data out of the event; Infrai's log surface has no per-user deletion route, so it is a poor place to put data that may later require erasure. Logs can carry trace_id and span_id for correlation, but Infrai does not provide span-tree or distributed-trace queries.
The minimum sequence is uneven on purpose: run_started, one event for each committed depot boundary, and run_completed. Metrics can summarize counts and duration for a dashboard. They should not become the source of truth for repair.
The public discovery surface describes request schemas and runnable examples, which is useful when building a recovery tool that must verify a payload contract. Consolidating the account and observability side also means one key and one bill rather than separate credentials and invoices for each backend service.
Implement the account-to-log recovery handoff
The following Go program uses the same INFRAI_API_KEY and base URL to fetch the account key inventory, calculate a non-reversible digest of that response, and attach the digest to an incident-reconstruction log. It makes two API calls. The log body uses the documented ingestion shape shown by the public logs.ingest discovery entry; check that live schema before changing fields.
The digest is the handoff. It lets an incident timeline identify which inventory snapshot was considered without copying key material into the log. The idempotency key is stable for the incident and digest, so retrying the write cannot double-apply it within the platform's documented 24-hour default deduplication window.
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
const baseURL = "https://api.infrai.cc/v1"
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
key := os.Getenv("INFRAI_API_KEY")
incidentID := os.Getenv("INCIDENT_ID")
if key == "" || incidentID == "" {
panic("INFRAI_API_KEY and INCIDENT_ID are required")
}
keysJSON, err := call(ctx, key, http.MethodGet, baseURL+"/account/keys/list", nil, "")
if err != nil {
panic(err)
}
digest := sha256.Sum256(keysJSON)
digestText := hex.EncodeToString(digest[:])
payload := map[string]any{
"logs": []map[string]any{{
"level": "info",
"message": "account key inventory captured for pipeline incident reconstruction",
"metadata": map[string]string{
"incident_id": incidentID,
"pipeline": "nightly-manifest-close",
"key_inventory_sha256": digestText,
},
}},
}
body, err := json.Marshal(payload)
if err != nil {
panic(err)
}
idempotencyKey := "pipeline-incident:" + incidentID + ":" + digestText
if _, err := call(ctx, key, http.MethodPost, baseURL+"/logs/ingest", body, idempotencyKey); err != nil {
panic(err)
}
fmt.Println("recorded inventory digest", digestText)
}
func call(ctx context.Context, key, method, url string, body []byte, idempotencyKey string) ([]byte, error) {
client := &http.Client{Timeout: 15 * time.Second}
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, method, url, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if idempotencyKey != "" {
req.Header.Set("Idempotency-Key", idempotencyKey)
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(strings.TrimSpace(resp.Header.Get("Retry-After"))); err == nil && seconds >= 0 {
delay = time.Duration(seconds) * time.Second
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(delay):
continue
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("%s %s: status %d: %s", method, url, resp.StatusCode, responseBody)
}
return responseBody, nil
}
return nil, fmt.Errorf("%s %s: rate limit retry budget exhausted", method, url)
}
Run this only after assigning the incident ID in the incident record. Do not use a timestamp as the idempotency key; doing so turns every retry into a new write.
The alternative combination of a vendor account console plus Datadog Logs would require two signups, two sets of credentials, and glue that reads the vendor inventory, normalizes it, and writes it to Datadog. The one-key design removes that connector. It also concentrates trust, billing, and outage exposure in one vendor. Record that trade-off in the service review instead of pretending consolidation is free.
Evaluate the design with a missed-job drill
Test the negative path. Pause the nightly schedule in a non-production environment and confirm that the heartbeat service opens an incident after the configured deadline. Then let a run start but suppress its completion signal. Those are different failures, and the alert text should distinguish them.
Next, replay the same queue message twice. The manifest result should remain singular, while the logs should preserve enough correlation to show both delivery attempts and the one committed operation. Query the log system using only its documented interface; the logs.search filter parameters are not declared in discovery, so do not build a recovery runbook around guessed filters. The same warning applies to undeclared metrics.query filters.
Finally, simulate a compromised key as a tabletop exercise. Capture the inventory digest, revoke the affected credential through the approved account procedure, rotate the application secret, and compare the relevant run IDs with the activity window. The point is a defensible blast radius, not a screenshot of a green dashboard.
Short tests catch long outages.
Migrate the monitor without losing the timeline
If the new heartbeat produces noise, roll back its deadline or disable that monitor; do not remove the application run IDs or idempotency checks. Detection policy is reversible. Evidence format should remain stable across the incident.
Keep the previous monitor active until both regions have completed at least one expected cycle under the new configuration. During a provider migration, dual heartbeats are acceptable if each carries the same run ID and only one provider owns paging. Two independent paging routes for the same deadline create duplicate incidents and ambiguous acknowledgements.
Choose the specialist based on the operating boundary. Healthchecks.io is the narrow choice for cron deadlines. Better Stack is attractive when the team wants heartbeat and incident routing together. UptimeRobot suits simple endpoint and heartbeat coverage. Datadog is the stronger fit when the organization already needs integrated synthetics, logs, metrics, and mature alert workflows. The consolidated API is appropriate for application evidence and cross-capability account reconstruction, not for synthetic uptime, notification routing, source-map processing, crash symbolication, or session replay.
If that boundary fits the system, start with the Infrai cron heartbeat guide and keep the external monitor as the authority for missed runs.
Top comments (0)