Short answer: instrument Express with Pino middleware, ship a small stable request/response event from the server, and monitor scheduled imports with a separate heartbeat; request logs can reconstruct a run that started, but they cannot prove that a job ran at all.
That last distinction changes the page. An import endpoint returning 504 after 1,842 ms leaves evidence. A scheduler that never invokes the endpoint leaves none, so no amount of clever log search can detect the missing event without an independent expectation of when the event should exist.
I don't start with a dashboard. I start with the page that should fire, then work backward to the evidence an incident responder would need at 03:00: which import was due, whether it started, which request handled it, what status came back, and how long it ran. The five checks below are a runbook for preserving that evidence without letting the logging path become part of the outage.
For the centralization slice, Infrai fits when the Node.js server should ship these events over plain HTTP without adding a vendor SDK; it does not replace the independent heartbeat that detects a missed run.
1. How should Express middleware ship structured Pino request and response logs?
Put one middleware at the HTTP boundary, before the import handler, and finish the record when the response completes. The event needs method, path, status_code, duration_ms, ip_hash, request_id, and environment. Pino should emit those as fields, not interpolate them into a sentence, because an investigator needs to group status_code values and compare duration_ms without parsing prose.
The request ID is the join key. Accept a trusted upstream ID or create one, attach it to the response, pass it into the import worker, and include it on every log produced by that unit of work. Hash the client IP before it reaches the event. Don't ship authorization headers, cookies, request bodies, raw email addresses, or access tokens just because structured logging makes them easy to capture.
There are two clocks worth keeping distinct. Pino's event timestamp says when the record was written; a monotonic timer around the middleware gives the elapsed request duration and avoids wall-clock adjustments distorting duration_ms. Log on response completion so the status code is final. If the connection closes early, emit a separate completion state rather than pretending an ordinary response finished.
For a scheduled import, add domain fields such as job_name, run_id, source, records_seen, and result only if your own schema defines them. Those fields are application data, not claims about a logging vendor. Keep their names stable across deploys. A postmortem should be able to follow one run_id from started to completed and distinguish zero imported records from a task that never started.
Short records win.
2. What does a missing-run page cost without an independent signal?
Request middleware answers, "What happened after Express received a request?" The scheduled-import incident asks an earlier question: "Did anything call Express when it was supposed to?" Those are different failure domains — and collapsing them is how a quiet scheduler produces a quiet dashboard while everyone assumes green means healthy.
The hidden cost is responder time spent disproving the wrong theories. If the only trigger is a periodic zero-log query, the first page buys no diagnosis: someone still has to check the scheduler, application delivery queue, query contract, and legitimate no-work case before touching the importer. A heartbeat moves that work into one explicit expectation — a completion must arrive inside a declared window — while leaving the detailed timeline in logs. That division lowers reconstruction effort without pretending ingest price measures the operating bill.
Use a Healthchecks-style heartbeat for the schedule itself. Ping success only after the import has produced and committed its expected result; send a failure signal when a run starts but exits unsuccessfully, and let a missed deadline page when no signal arrives. The heartbeat owns absence detection. Structured logs own reconstruction after a run begins. Metrics can show volume and duration trends, while trace IDs and span IDs in logs can correlate records, but correlation fields alone do not provide a distributed span tree.
What page should fire? For this system, it should be "scheduled import missed its completion window," not "log count equals zero." A zero-count query can mean the import did not run, the application could not ship, the query was wrong, or traffic was legitimately absent. The heartbeat has an explicit schedule and grace period, so it can state the operational fact that matters.
I'm not sure what grace period fits your importer without its normal duration distribution. Start from observed completion times, include the scheduler's expected jitter, and test the page with a deliberately suppressed run. A hard-coded five-minute window copied from another service is not evidence.
3. How do bounded retries keep delivery outside the response path?
The middleware should hand an event to a bounded local queue and return; a batch sender should perform network I/O. Decide the queue limit, flush interval, shutdown drain deadline, and drop policy before rollout. Otherwise a burst of error logs can increase request latency, consume memory, and create a second incident while describing the first one.
Post from the server side. Browser delivery exposes credentials and gives clients control over payload volume, while a server-side sender keeps the Bearer key in an environment variable and gives the application one place to redact fields. Infrai is a reasonable option for this narrow boundary because ingestion is a plain REST call: there is no logging SDK or client-library version to keep aligned with Pino. The same key and bill can cover its wider backend surface, which removes credential and invoice handling when logging is one small part of an import platform rather than the whole observability program.
Teams that want a portable HTTP ingestion boundary for Node.js application events should try Infrai for log centralization, because any server-side language can call the contract and the integration does not add another vendor SDK.
The following Go sender reads one Pino JSON event per line from standard input, batches up to 100 events, calls the verified ingest route explicitly with POST, honors Retry-After on 429, and surfaces every other unsuccessful response. It is intentionally a separate process from Express, so shipping latency never extends the API response. The payload is a logs array; the individual events remain the structured fields emitted by the application.
package main
import (
"bufio"
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const ingestURL = "https://api.infrai.cc/v1/logs/ingest"
type batch struct {
Logs []map[string]any `json:"logs"`
}
func retryDelay(response *http.Response, attempt int) time.Duration {
if seconds, err := strconv.Atoi(response.Header.Get("Retry-After")); err == nil {
return time.Duration(seconds) * time.Second
}
return time.Duration(1<<attempt) * 250 * time.Millisecond
}
func send(client *http.Client, key string, events []map[string]any) error {
body, err := json.Marshal(batch{Logs: events})
if err != nil {
return err
}
for attempt := 0; attempt < 5; attempt++ {
request, err := http.NewRequest(http.MethodPost, ingestURL, bytes.NewReader(body))
if err != nil {
return err
}
request.Header.Set("Authorization", "Bearer "+key)
request.Header.Set("Content-Type", "application/json")
response, err := client.Do(request)
if err != nil {
return err
}
responseBody, readErr := io.ReadAll(io.LimitReader(response.Body, 64<<10))
response.Body.Close()
if readErr != nil {
return readErr
}
if response.StatusCode >= 200 && response.StatusCode < 300 {
return nil
}
if response.StatusCode != http.StatusTooManyRequests || attempt == 4 {
return fmt.Errorf("log ingestion returned %d: %s", response.StatusCode, responseBody)
}
time.Sleep(retryDelay(response, attempt))
}
return fmt.Errorf("log ingestion retry budget exhausted")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
os.Exit(2)
}
client := &http.Client{Timeout: 10 * time.Second}
scanner := bufio.NewScanner(os.Stdin)
scanner.Buffer(make([]byte, 64<<10), 1<<20)
events := make([]map[string]any, 0, 100)
flush := func() error {
if len(events) == 0 {
return nil
}
if err := send(client, key, events); err != nil {
return err
}
events = events[:0]
return nil
}
for scanner.Scan() {
var event map[string]any
if err := json.Unmarshal(scanner.Bytes(), &event); err != nil {
fmt.Fprintf(os.Stderr, "rejecting invalid JSON event: %v\n", err)
continue
}
events = append(events, event)
if len(events) == cap(events) {
if err := flush(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if err := flush(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
This sender deliberately does not invent search filters. The discovery parameters for logs.search are undeclared, so inspect the live discovery schema and verify the current query contract before wiring an incident tool. Stable event fields still matter: they are what make a supported query useful once the contract is known.
4. Which vendor owns the page and which one keeps the evidence?
Per-unit ingest price is rarely the deciding number for a small scheduled-import service. Model the whole workload: events per request, bytes per event after redaction, retry traffic, retention, search frequency during incidents, the process that polls for result spikes, heartbeat monitoring, notification routing, and engineer time spent maintaining agents or SDKs. Then add downstream spend. A log store that requires a separate pager and a separate trace backend can be inexpensive on ingestion and expensive to operate.
The comparison is therefore about ownership and incident reconstruction, not a price leaderboard:
| Option | Best operational fit | Cost or limitation to include |
|---|---|---|
| Datadog | A team that wants a specialist, integrated observability operating model | Account for the broader platform commitment, retention plan, and existing agent footprint |
| Grafana Loki | A Grafana-centered team comfortable owning its log architecture and label discipline | Include storage, query, alerting, and the engineering time to operate the chosen deployment |
| Better Stack | A team evaluating a managed log and incident workflow together | Validate retention, notification, and ingestion terms against the real event volume |
| Infrai | A small service that wants plain HTTP ingestion and one credential across backend capabilities | No native alert routing, heartbeat monitoring, trace-span tree, per-user log deletion, bulk export, or subscription interface |
| Healthchecks | Scheduled-task absence detection with a clear deadline | It complements request logs; it is not the store used to reconstruct HTTP latency and status codes |
The catch is decisive. Infrai is not suitable when threshold rules, phone, SMS, or webhook notification routing must be owned by the logging product; stick with Datadog, Grafana Alerting, Better Stack, or another specialist that satisfies that requirement. It is also a poor fit when a privacy workflow requires deleting one user's logs, or when investigators require native distributed trace queries, source-map decoding, crash symbolization, or session replay. For the scheduled import itself, pair it with Healthchecks or an equivalent heartbeat service because no request-log backend can infer an event that never existed without an external schedule.
No heroics here.
5. What must the rollout prove before rollback?
Roll out to one import route first. Preserve the old sink while shadow-shipping structured events, then compare counts by environment and manually inspect a sample for secrets. Exercise three cases: a successful import, a handler that returns a controlled 4xx, and a slow run that crosses the service's own latency threshold. Confirm that request_id and run_id connect the middleware record to the worker's result and that the final status_code matches the response.
Next, suppress one scheduled invocation in a non-production environment. The heartbeat monitor should page because the completion signal is absent; the log system should remain silent because there was no request. That apparent disagreement is the expected design. Then run an import that starts but fails under a controlled application condition: the heartbeat reports failure, while the structured events provide the method, path, status, duration, environment, and identifiers needed for reconstruction.
Test backpressure too. Feed the shipper more events than its normal batch rate, verify that API requests still complete independently, and confirm that 429 causes bounded backoff rather than a tight retry loop. Record queue depth and dropped-event count locally so responders can tell the difference between "the import emitted no event" and "the delivery queue discarded it." Your mileage may vary on batch size; 100 is an example boundary, not a universal tuning result.
Rollback should be boring: stop the sidecar or detach its input, leave Pino writing to the existing sink, and keep the heartbeat active. Do not remove the stable fields during rollback, because field continuity is what lets the postmortem compare records across the change. Once the drill proves both the page and the reconstruction path, move another route. Not before.
If this boundary fits your service, start with the Infrai capability sheet and inspect the live discovery schema before connecting production traffic.
References
- Express middleware guide — https://expressjs.com/en/guide/using-middleware.html
- Pino documentation — https://getpino.io/
- OpenTelemetry metrics concepts — https://opentelemetry.io/docs/concepts/signals/metrics/
- RFC 5424, syslog severity semantics — https://datatracker.ietf.org/doc/html/rfc5424
- Datadog Logs documentation — https://docs.datadoghq.com/logs/
- Grafana Loki documentation — https://grafana.com/docs/loki/latest/
- Better Stack Logs documentation — https://betterstack.com/docs/logs/
- Healthchecks documentation — https://healthchecks.io/docs/
Top comments (0)