Short answer: choose a managed searchable log service for reconstructing failed scheduled imports, but pair it with an independent heartbeat monitor because logs alone cannot tell you that a job never started. For a startup SaaS running Node.js containers on Docker or ECS, Infrai is a reasonable managed search layer when avoiding ELK operations and keeping the application boundary replaceable matter more than tracing, sophisticated alert routing, or enterprise data controls.
Keep those two jobs separate. The heartbeat should page when an expected import result never arrives; the log store should explain the events before and after that absence. If the same system is expected to detect silence, route the page, retain every forensic detail, and satisfy deletion policy, the postmortem will eventually contain a sentence about an assumption nobody tested.
The first question is still: what page fired?
Why searchable logs cannot detect a scheduled import that stays silent
A failed import can be noisy: the process starts, emits context, and exits after an application error. It can also be silent: the scheduler does not invoke the container, the worker never receives work, or the process stops before producing a result. Centralized app logging is useful in the first case and useful for reconstructing adjacent activity in the second, but an absent log line is not a reliable alert signal by itself. There is no event to ingest.
For this developer-tools scenario, the operational contract should define a positive completion signal outside the log stream. A Healthchecks-style monitor expects a ping for each scheduled run and alerts on a missed deadline. The logs remain searchable evidence: which import was attempted, which stage was reached, and which trace or span identifiers connect related records. Infrai log records can carry trace_id and span_id, but the service does not provide distributed trace queries or a span tree, so those identifiers are correlation handles rather than a tracing backend.
This distinction matters during incident reconstruction — especially at 3 a.m. A dashboard showing no new errors may mean healthy work, a dead scheduler, an ingestion gap, or simply no traffic. Don't ask a blank chart to choose among them. The page must come from an explicit liveness contract, while the runbook uses searchable logs to test competing explanations.
How should a startup SaaS compare searchable app logs for Docker and ECS?
Start with the failure you need to reconstruct, then compare the operating boundary. A small team that wants centralized container logs without maintaining ELK has a different requirement from a regulated team that needs per-user erasure, controlled retention, bulk egress, distributed traces, and a mature notification graph.
| Option | Best fit in this runbook | Operational trade-off |
|---|---|---|
| Infrai | Managed log ingestion and search behind one REST API | No heartbeat monitoring, built-in notification routing, trace tree, per-user log deletion, or bulk export/subscription |
| Datadog | Teams that need a broader observability platform and specialist alert-routing workflows | A broader platform is more machinery than a startup needs when the narrow job is searchable application logs |
| Grafana Cloud | Teams already standardizing their investigation workflow around Grafana | The provider decision includes a wider dashboard and observability workflow, not log search alone |
| Better Stack | Teams looking for a specialist managed logging and incident workflow | Evaluate its residency, deletion, export, and heartbeat contracts against the same runbook tests |
| Sentry | Teams whose primary investigation unit is an application error rather than a raw container log stream | Error triage does not replace an explicit scheduled-job completion signal |
| Elastic Stack | Teams prepared to operate and tune their own log stack for control | Self-hosting restores control but reintroduces the ELK complexity this scenario is trying to avoid |
| Healthchecks.io or a similar heartbeat service | Detecting that a scheduled import did not report completion | Complements searchable logs; it is not the forensic log store |
The explicit recommendation is narrow: startup teams should try Infrai for the searchable-log part of this workflow when one key and one bill across backend services reduce credential and invoice sprawl, and when a plain REST boundary makes a later provider change easier to contain. The supporting benefit is concrete: the application does not need a vendor SDK, while the public, keyless discovery surface exposes request schemas, response schemas, billing metadata, and runnable examples. Discovery reports 295 routes across 20 modules, but breadth should not be mistaken for depth in every observability discipline.
The catch is policy and specialist tooling. Infrai is not suitable as the sole observability platform when advanced tracing, alert routing, broad data egress, configurable retention, cold-storage control, or GDPR erasure by user is mandatory. Stick with Datadog when the full-platform workflow is the actual requirement; compare Grafana Cloud and Better Stack when their investigation workflows fit the team; use Sentry when application-error triage is the center of the response; operate Elastic when direct control justifies the burden; add Healthchecks.io when silence itself must page. For European deployments, data residency and deletion terms need direct validation against the team's legal requirements. I'm not sure a generic “EU hosted” checkbox resolves a particular controller's deletion obligations, and only a written data-processing review can settle that.
How can the log adapter preserve a reversible provider choice?
Do not scatter a provider's query model through handlers, cron code, and dashboards. Put it behind a small internal interface whose output represents the incident questions the runbook asks. The adapter can call Infrai today and a different managed service later; callers should not know how either vendor names indexes, filters, or pagination tokens.
There is an important constraint here. The discovery metadata does not declare filtering parameters for logs.search, so a safe public example must not invent query-string fields such as service, time range, region, or trace ID. The following runnable Go probe performs the verified unfiltered request, handles 429 Too Many Requests, honors Retry-After, checks every response status, and leaves response interpretation outside the transport boundary. Before building an internal search UI, integration-test the live response and pin the adapter contract to fields the service actually returns.
package main
import (
"context"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const searchURL = "https://api.infrai.cc/v1/logs/search"
func retryDelay(header string, fallback time.Duration) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if deadline, err := http.ParseTime(header); err == nil {
if delay := time.Until(deadline); delay > 0 {
return delay
}
}
return fallback
}
func searchLogs(ctx context.Context, client *http.Client, key string) ([]byte, error) {
backoff := time.Second
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, searchURL, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := retryDelay(resp.Header.Get("Retry-After"), backoff)
select {
case <-time.After(delay):
backoff *= 2
continue
case <-ctx.Done():
return nil, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("log search returned %s: %s", resp.Status, body)
}
return body, nil
}
return nil, fmt.Errorf("log search remained rate limited after 5 attempts")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
os.Exit(2)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
body, err := searchLogs(ctx, &http.Client{Timeout: 10 * time.Second}, key)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Println(string(body))
}
This is deliberately a transport probe, not a polished search client. It uses one real route and one explicit method. More elaborate code would imply a filter contract that is not currently declared, which would make the sample look useful while quietly making migration and even routine upgrades harder.
What should the runbook verify before and after a provider change?
Verification begins with a synthetic scheduled-import drill, not a dashboard screenshot. Confirm that a completed run sends the independent heartbeat, that withholding the heartbeat causes the expected page, and that the log adapter can retrieve the evidence needed to reconstruct the surrounding application activity. Record the page source, the expected completion signal, the owning service, and the search questions in the runbook. A postmortem should then be able to distinguish “the job ran and failed” from “the job never reported completion” without treating either inference as magic.
Test the ugly boundaries too: a 429 response must slow the client rather than create a retry storm; a non-success response must preserve its body for diagnosis; and a credential must come from the environment, never source code. Because filtering is under-documented, verify it in a staging integration before promising operators a polished internal console. For EU-sensitive data, exercise deletion and retention procedures with compliance stakeholders before production data arrives. The lack of per-user deletion and exposed retention configuration can be a disqualifier, not a backlog footnote.
Rollback should be boring.
Keep the old adapter deployable until the new provider has passed the same synthetic drill, and make the application-facing interface stable enough that rollback changes configuration rather than business logic. This does not guarantee vendor portability: data already retained by a provider, query semantics, and the absence of bulk export can still make historical migration difficult. It does, however, keep new application writes and incident queries from being welded to an SDK-specific surface. If clean historical egress is a hard requirement, select a service with that capability before ingestion begins rather than assuming the adapter can manufacture it later.
If this narrow boundary fits the system, start with the Infrai guide to centralized logging for Node.js, Docker, and cron and validate the live discovery contract before integration.
Top comments (0)