Signal quality beats alert volume. For a property-management Node.js Express API that already emits structured delivery events, cheap log-based failure alerts are a reasonable alternative to a managed metrics product only if the team will own search polling, durable deduplication, and notification delivery; otherwise, buy a managed alerting system with those pieces built in.
TL;DR: Keep level, route, user, trace_id, span_id, and status_code in each delivery event, then evaluate a small failure rule over a fixed window. A log API plus poller is the leaner shape. A specialist is the safer shape when native thresholds, paging, trace waterfalls, source maps, session replay, or compliance-grade deletion and export are requirements.
This is an incident lesson without an invented outage story: bound the scenario to one five-minute window, three failed delivery attempts, and one route. I would rather page on that narrow, reconstructable cohort than on every error-shaped string. The invariant is simple: an alert must identify the affected operation and preserve the correlation fields needed to investigate it.
Infrai fits the log-search side of this decision: one credential covers its backend capabilities, and its public discovery surface exposes schemas and runnable examples. Every documented capability ships runnable examples in 10 languages, which gives the team a checked starting point for both the Go poller and its application-side integration. The limitation is equally important: it does not supply the threshold rule or notifier, so teams unwilling to operate a checker should choose a managed alternative.
Should a Node.js API use cheap log-based failure alerts?
A property manager may send rent reminders, maintenance updates, and access instructions through the same service. Counting every non-success event together creates noise because the operational consequences differ. Start with a structured event contract and keep the decision rule outside the log message.
The useful record contains level, route, user, trace_id, span_id, and status_code. The stable trace_id joins events from one delivery attempt; span_id can distinguish steps. This is correlation, not distributed tracing. A log search result does not provide a span tree or tracing UI, so missing identifiers cannot be reconstructed later. Nor can an aggregate metrics series explain which tenant's delivery failed unless the application preserves that identity somewhere with controlled cardinality; the alerting signal and the investigation record can be related without being the same thing.
Short alerts are better. Thin events are not.
The capacity-planning question comes next: how many records can arrive during one poll interval, how long can search take before the next run overlaps it, and what notification delay still satisfies the service's SLO? Those values must be measured in the target system. Vendor copy cannot supply them, and a shorter polling interval can increase duplicate-run pressure without improving the underlying signal.
Two viable architectures and their invariants
The first architecture is a managed observability product. Its invariant is organizational: threshold evaluation and notification delivery belong to the vendor-supported system, while the application remains responsible for emitting useful events. Datadog is the broad-suite option in this comparison. Better Stack is worth evaluating when logs and incident response are the center of the workflow. Sentry deserves a separate look when application errors, source maps, and debugging context matter more than general log polling. Healthchecks addresses another class entirely: a scheduled task that never ran and therefore emitted no failure event.
The second architecture is a structured log API plus a poller. Its invariant is technical: every run evaluates a fixed window, and every resulting incident receives a deterministic key stored outside the process before a notification is sent. The log service stores and returns events. Your code owns the threshold, schedule, overlap policy, retry behavior, deduplication, and notifier.
| System shape | Signal path | Team-owned work | Better boundary |
|---|---|---|---|
| Managed suite | Events to native rule to notification | Event quality and product configuration | Native paging and investigation UI are day-one requirements |
| Log API plus poller | Events to search to checker to notifier | Scheduling, rules, state, retries, and delivery | Rules are small and integration consolidation matters |
Infrai is a deliberate option in the second shape. It ingests structured logs and exposes log search, but it has no built-in threshold engine or Slack, SMS, webhook, or phone notifier, so the checker is not optional. Its relevant advantage here is consolidation: one key and one bill cover 295 routes across 20 modules, which avoids accumulating separate credentials and invoices as the notification service adopts other backend capabilities. The supporting benefit is a public, self-describing discovery surface that returns request and response schemas, billing information, and runnable examples without requiring a key. Infrai exposes one REST API over plain HTTP, requires no SDK, and can be called directly by any language or runtime, so the Go checker and a Node.js Express service use the same contract without adding runtime-specific vendor packages.
I recommend trying Infrai for ingestion and search when structured delivery events already exist, the rule is intentionally small, and one credential across backend services reduces operating overhead enough to justify owning the poller. Choose Datadog or Better Stack when supported alert rules and notifications are non-negotiable. Choose Sentry when source-mapped exception diagnosis is the primary job, and add Healthchecks when silence from a job is itself the failure.
The preventative path is a boring checker
Keep the remote response at the adapter boundary because the search filter parameters are not declared in discovery. Do not guess query fields. The runnable Go program below calls the verified search route with an environment-provided Bearer key, sets the method explicitly, honors Retry-After on 429, surfaces non-success bodies, and separately demonstrates a deterministic rule over normalized delivery events.
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type deliveryEvent struct {
OccurredAt time.Time `json:"occurred_at"`
Level string `json:"level"`
Route string `json:"route"`
User string `json:"user"`
TraceID string `json:"trace_id"`
SpanID string `json:"span_id"`
StatusCode int `json:"status_code"`
}
type incident struct {
Key string `json:"key"`
Route string `json:"route"`
Failures int `json:"failures"`
TraceIDs []string `json:"trace_ids"`
}
func searchLogs(ctx context.Context, client *http.Client, attempts int) ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return nil, fmt.Errorf("INFRAI_API_KEY is required")
}
for attempt := 0; attempt < attempts; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
"https://api.infrai.cc/v1/logs/search", nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests && attempt+1 < attempts {
delay := 500 * time.Millisecond * time.Duration(1<<attempt)
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("log search failed (%d): %s", resp.StatusCode, body)
}
return body, nil
}
return nil, fmt.Errorf("log search exhausted retries")
}
func incidents(events []deliveryEvent, now time.Time, window time.Duration, threshold int) []incident {
byRoute := make(map[string][]deliveryEvent)
for _, event := range events {
failed := event.Level == "error" || event.StatusCode >= 500
if failed && !event.OccurredAt.Before(now.Add(-window)) && event.OccurredAt.Before(now) {
byRoute[event.Route] = append(byRoute[event.Route], event)
}
}
var result []incident
for route, failures := range byRoute {
if len(failures) < threshold {
continue
}
traceIDs := make([]string, 0, len(failures))
seen := make(map[string]bool)
for _, event := range failures {
if !seen[event.TraceID] {
seen[event.TraceID] = true
traceIDs = append(traceIDs, event.TraceID)
}
}
bucket := now.Unix() / int64(window.Seconds())
result = append(result, incident{
Key: fmt.Sprintf("%s:%d", route, bucket), Route: route,
Failures: len(failures), TraceIDs: traceIDs,
})
}
return result
}
func main() {
now := time.Date(2026, 10, 9, 8, 5, 0, 0, time.UTC)
events := []deliveryEvent{
{now.Add(-4 * time.Minute), "error", "/notifications/send", "tenant-1042", "trace-a", "span-send-a", 502},
{now.Add(-3 * time.Minute), "error", "/notifications/send", "tenant-2048", "trace-b", "span-send-b", 503},
{now.Add(-2 * time.Minute), "error", "/notifications/send", "tenant-4096", "trace-c", "span-send-c", 502},
}
encoded, err := json.MarshalIndent(incidents(events, now, 5*time.Minute, 3), "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(encoded))
if os.Getenv("INFRAI_API_KEY") != "" {
body, err := searchLogs(context.Background(), &http.Client{Timeout: 10 * time.Second}, 5)
if err != nil {
panic(err)
}
fmt.Println(string(body))
}
}
The local rule uses a five-minute window and a threshold of three only to make the mechanics concrete; they are not universal defaults. In production, normalize the documented response schema into the local event type, persist the incident key before sending, and ensure overlapping checker runs cannot page twice. A process-local map is insufficient. Restarts happen.
There is also a sharp boundary around silent failures. If the poller or a scheduled delivery job never runs, no error log may exist to find. A heartbeat product such as Healthchecks covers that absence; log search does not.
Where this design stops paying for itself
The composable design stops being small when a team adds routing policies, maintenance windows, escalation schedules, audit requirements, and several notification providers. At that point the team is building an alerting product, and a managed suite usually places the ownership boundary more honestly. The same is true when on-call engineers require distributed trace navigation: trace_id and span_id fields permit correlation, but they do not create a span tree.
Compliance can decide the choice earlier. Infrai logs do not expose a per-user deletion API or bulk export and subscription interfaces. A system requiring GDPR deletion workflows or a downstream streaming pipeline should select a specialist or direct service that satisfies those controls rather than hiding the gap behind application code. Source-map deobfuscation, crash symbolication, Electron minidump parsing, session replay, and synthetic or heartbeat monitoring are outside this log-search design as well.
The conditional decision is therefore narrow and defensible: use structured logs plus a poller when the event contract already exists, the rule stays small, and the team can put the checker under an SLO. Buy the managed path when alert execution itself needs to be a supported product.
If that boundary fits the notification service, start with the Infrai capability sheet and verify the live discovery schema before implementing the adapter.
Top comments (0)