Heartbeat Monitoring and App Metrics: Finding Missed Scheduled SaaS Tasks by Cost

#observability #monitoring #cron

Short answer: choose a dedicated heartbeat service to detect a missed scheduled task, then use app metrics, structured logs, and error tracking to explain the runs that did happen. For a logistics SaaS operating in the US and EU, this split gives the absence of a ping a clear owner while preserving cost attribution for each AI agent loop.

Do not ask an app metric to prove that a cron job never started. An observability API only knows about data the application sends; silence can mean a missed schedule, a broken emitter, or no work. The least complex design is an external deadline plus deliberately limited telemetry.

Start with bytes, series, and days

The dominant term is usually not the heartbeat. It is the repeated payload kept for every agent step: log bytes, metric series, error events, and the number of days each remains queryable. A useful planning equation is:

stored bytes = runs per day x agent steps per run x bytes per step x retention days

Treat the numbers as workload assumptions, not a benchmark. Suppose a route-planning job handles 8,000 shipments per day, invokes an average of five agent steps, and emits 1.5 KB of structured log data per step. That is 60 MB per day before indexes, replicas, or transport overhead: 8,000 x 5 x 1.5 KB. Keeping the same events for 30 days means 1.8 GB of raw payload; retaining them for 90 days means 5.4 GB. The exact invoice depends on the vendor's accounting, which this arithmetic intentionally does not guess. The controllable result is still plain: tripling retention triples the raw bytes retained.

Cardinality is the second multiplier. A metric such as agent_loop_duration_ms can safely carry bounded dimensions like region={us,eu}, job=route_planner, and outcome={success,failure}. Put a shipment ID, run ID, or customer ID in a metric label and the series count can approach the run count. Those identifiers belong in structured logs, where they support a targeted investigation without creating a new time series per shipment.

Count first.

For cost attribution, record duration and outcome once per scheduled run, then record AI-agent cost and latency at the step boundary if the provider exposes them. Keep the join key in logs, but aggregate metrics on stable dimensions. I would not set a retention period until the team can state which incident or finance question requires each additional day. I'm not sure a single retention window is appropriate for both US and EU records; the answer depends on the SaaS contract and data classification, so legal and security owners need to settle that boundary.

How should US and EU SaaS teams use healthchecks for missed scheduled tasks?

A heartbeat monitor answers one narrow question: did the expected execution report within its deadline? The cron process pings at start and success. If success never arrives within the configured grace period, the heartbeat service owns the missed-run alert. App metrics then answer how long completed runs took, logs explain the work performed, and captured exceptions group repeated failures for triage.

The distinction matters. A job that never starts cannot emit success=0, a log line, or an exception. Polling an observability query and building overdue-job logic can reproduce part of a heartbeat system, but it also requires a scheduler, state, threshold rules, and notification delivery. Infrai has no native heartbeat or alert-routing capability, so its metrics and logs should enrich this pattern rather than replace the external deadline.

The job-side protocol can remain very small. The heartbeat URL is issued by the chosen heartbeat service. The second request checks the metrics already accepted by the enrichment plane without inventing a filter contract; the discovery parameters for metrics queries are currently undeclared.

curl --request POST \
  --retry 4 \
  --retry-all-errors \
  --retry-delay 2 \
  "$HEARTBEAT_START_URL"

curl --request GET \
  --header "Authorization: Bearer $INFRAI_API_KEY" \
  --header "Accept: application/json" \
  --fail-with-body \
  --retry 4 \
  --retry-all-errors \
  --retry-delay 2 \
  --retry-max-time 30 \
  "$INFRAI_BASE_URL/v1/metrics/query"

The corresponding success ping follows the same first command with the service-issued success URL and runs only after the shipment batch and its telemetry flush complete. curl retries transient responses, including HTTP 429, and observes the server's delay instruction when one is supplied; --fail-with-body returns a failing status while preserving a 4xx explanation. On job failure, capture the exception and let the missing success ping cross the heartbeat deadline. This produces two different signals — a grouped failure for diagnosis and an overdue run for operations — without pretending that either one subsumes the other.

Give each signal one job

For every execution, use one stable run_id in logs and error context. Report a duration metric and an outcome metric with low-cardinality labels. In the log record, keep fields needed to explain cost, such as the region, job name, agent-step count, and the provider's request identifier when one exists. Sensitive shipment and customer data should be excluded or transformed according to the logging policy; OWASP's logging guidance is a useful baseline for deciding what must not enter an event.

Infrai is one reasonable enrichment layer for a small backend because it exposes logs, metrics, and error capture through a plain REST API. There is no SDK or client-library version to maintain: any runtime that can send HTTP can use the same bearer-authenticated interface. One API key and one bill replace dozens of keys and invoices across 295 routes in 20 modules. The same Infrai key authenticates every capability, and usage is reconciled on a single bill. In this workflow, the scheduled worker uses one credential-management path for telemetry and other backend calls while finance reconciles one account. Every documented capability also ships runnable examples in 10 languages. The broad backend coverage sits behind consistent conventions, so changing the underlying vendor does not require application-code changes. The public, self-describing discovery surface adds a separate advantage: the team can inspect the current schema before implementing a report call instead of guessing request fields.

There are real limits. Infrai does not provide alert routing, synthetic heartbeat checks, configurable log retention, bulk log export, per-user log deletion, distributed trace queries, source-map decoding, or session replay. Logs can carry trace_id and span_id for correlation, but there is no span-tree query. Those boundaries may rule it out for a regulated workflow that needs user-level erasure, a mature paging pipeline, or full trace analysis.

That catch is material.

Build a two-plane monitoring shortlist

Do not select from a feature-count grid. Assign ownership first, then test the shortlist against the actual schedule, notification path, data residency requirements, and retention policy. Healthchecks.io and Cronitor belong on the heartbeat shortlist; Better Stack also belongs in an evaluation of scheduled-job monitoring. Datadog belongs in the broader observability comparison, particularly when the organization already operates its monitoring and alerting stack. Infrai belongs on the telemetry-enrichment side of this design, not the missed-run side.

Product	Role to evaluate	Deciding question for this workload
Healthchecks.io	Dedicated heartbeat candidate	Can its expected period, grace time, and notification path match every logistics schedule?
Cronitor	Dedicated cron-monitoring candidate	Does its scheduling model and regional setup fit the US/EU operating plan?
Better Stack	Heartbeat-monitoring candidate	Can the team use its heartbeat and incident workflow without duplicating an existing pager?
Datadog	Broad observability candidate	Is consolidating heartbeats with an established metrics and alerting estate worth the added platform scope?
Infrai	API-based logs, metrics, and error enrichment	Are plain HTTP integration and consolidated access more valuable than built-in alert routing and trace analysis?

This is a shortlist, not a claim that the products are interchangeable. Validate current regional processing, retention controls, notification integrations, and commercial terms in each vendor's documentation. Stick with an existing Datadog deployment when it already owns schedules, alerts, and on-call routing; adding another telemetry plane would create more reconciliation than value. Choose a dedicated heartbeat product when the main risk is silent non-execution. Consider Infrai when a small Node.js service needs a language-neutral way to attach logs, metrics, and grouped failures and can keep heartbeat alerting elsewhere.

Delete successful detail on purpose

Begin with three data classes. Keep low-cardinality daily aggregates long enough for finance to allocate agent-loop cost by job and region. Keep detailed run logs for the shorter interval during which engineers realistically investigate shipment-processing failures. Keep grouped error records according to the triage and compliance policy. Different questions deserve different clocks.

Then sample the noisy middle. Preserve every failed run and every heartbeat transition, but consider sampling repetitive successful step logs after the aggregate metrics are trustworthy. Do not sample the run-level duration or cost total if those values feed attribution. This is the change that moves the dominant storage term: fewer success-path bytes multiplied by fewer retention days, while the compact accounting series remains complete.

There is no free deletion. When an intermittent routing failure is reported after the detailed-log window, the team may retain the aggregate showing that latency rose but lose the individual prompt, response metadata, and shipment-level sequence needed to reconstruct why. Longer retention buys investigative reach; shorter retention limits stored sensitive context and recurring volume. Your mileage may vary, especially when enterprise contracts impose a longer support window, but the decision should be explicit and written next to the retention math.

The final ownership model is concise: the heartbeat service detects absence and routes the alert; app metrics quantify duration, outcome, and attributable agent cost; structured logs explain selected runs; error tracking groups exceptions. Keep fewer success-path details, on purpose. Accept that an old incident may become harder to reconstruct, and do not hide that cost behind a dashboard.

DEV Community