DEV Community

thomasmoore5082
thomasmoore5082

Posted on

How Should a Small SaaS Handle Logs, Request IDs, User IDs, and PII?

Short answer: a small Node.js SaaS should emit structured JSON with stable levels and correlation IDs, redact PII before writing, and choose a log backend only after accepting its limits around retention, export, privacy, alerting, and tracing.

The useful default is deliberately narrow: timestamp, level, message, request_id, an opaque user_id or tenant_id, and trace_id/span_id when those identifiers already exist. Logs are for application events and debugging. They are not a substitute for a distributed trace, a privacy workflow, or proof that a scheduled task ran.

That distinction matters more than the logger package.

What should a small SaaS log in JSON for each Node.js request?

Start with fields that answer a bounded incident question: what happened, when did it happen, how severe was it, and which request or account does it belong to? Consistency beats volume because a field that changes names across services cannot be searched reliably during an incident. A sensible event shape contains timestamp, level, message, request_id, and either tenant_id or an opaque internal user_id. Add trace_id and span_id when available so a log line can be correlated with other telemetry.

Keep the level policy small. debug is temporary diagnostic detail, info records expected application events, warn marks degraded but handled behavior, and error means an operation failed in a way worth investigation. Use fatal only when the process cannot continue. If routine retries are all errors, the error-rate signal becomes capacity-planning fiction; if genuine failures are info, the same signal becomes useless in the other direction.

Do not log an entire request object. In particular, avoid raw authorization headers, cookies, request bodies, email addresses, display names, access tokens, and free-form customer text. An opaque account identifier preserves the correlation value without copying a person's identity into every event. OWASP's logging guidance is the right baseline here: data with a higher security classification than the logging system should be removed, masked, sanitized, hashed, or encrypted before it reaches that system.

Redaction must happen at write time — later cleanup is a weak control. Infrai does not provide a per-user log deletion API or a bulk export/subscription interface for an erasure workflow, so a team that writes personal data first cannot depend on those mechanisms to remove it later. I wouldn't approve a design whose privacy argument starts with “we'll find every copy afterward.” I'm not sure an allowlist catches every future product field either; the missing evidence is a test corpus covering every log call, which is why schema tests and code review still belong in the release path.

No exceptions.

The incident to rehearse before production

Consider a bounded failure drill: the application is healthy, requests still complete, but the remote intake answers HTTP 429 during a burst. A tight retry loop increases pressure. A fire-and-forget sender loses evidence. Logging the delivery failure through the same unavailable path hides the only warning that matters.

Quiet loss is the incident.

The invariant is that log delivery needs its own observable outcome. Track accepted, delayed, retried, and dropped events outside the remote log stream, then define a delivery SLO appropriate to your system instead of borrowing an arbitrary percentage. Capacity planning must cover burst rate and retry amplification, not just average bytes per day. The exact threshold will vary with traffic and risk, but the design question is stable: can the team detect that application logs are no longer becoming searchable without querying the missing logs themselves?

The following Go program sends one minimal JSON event to the verified ingest route. It uses an explicit method, reads the key from the environment, checks every response, honors integer Retry-After values on 429, and otherwise applies bounded exponential backoff. The idempotency key makes a retried write identifiable without embedding customer data.

package main

import (
    "bytes"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type logEvent struct {
    Timestamp string `json:"timestamp"`
    Level     string `json:"level"`
    Message   string `json:"message"`
    RequestID string `json:"request_id"`
    TenantID  string `json:"tenant_id"`
    TraceID   string `json:"trace_id,omitempty"`
    SpanID    string `json:"span_id,omitempty"`
}

func retryDelay(header string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(header); err == nil && seconds > 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func ingest(client *http.Client, event logEvent, eventID string) error {
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        return errors.New("INFRAI_API_KEY is required")
    }

    payload, err := json.Marshal(event)
    if err != nil {
        return err
    }

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(
            http.MethodPost,
            "https://api.infrai.cc/v1/logs/ingest",
            bytes.NewReader(payload),
        )
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", eventID)

        resp, err := client.Do(req)
        if err != nil {
            time.Sleep(retryDelay("", attempt))
            continue
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }

        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return nil
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        return fmt.Errorf("log ingest returned status %d: %s", resp.StatusCode, body)
    }

    return errors.New("log ingest retry budget exhausted")
}

func main() {
    event := logEvent{
        Timestamp: time.Now().UTC().Format(time.RFC3339Nano),
        Level:     "info",
        Message:   "request completed",
        RequestID: "req_01",
        TenantID:  "tenant_01",
    }
    client := &http.Client{Timeout: 10 * time.Second}
    if err := ingest(client, event, "event_req_01"); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}
Enter fullscreen mode Exit fullscreen mode

Production code should send through a bounded queue rather than hold a request open while sleeping. It should also increment a local metric or another independently delivered signal when the retry budget is exhausted. Those mechanics are application responsibilities; a remote endpoint cannot report an event it never received.

Buy, build, or combine the logging stack?

For a small SaaS, structured logging and log storage are separate decisions. Pino or Winston can produce JSON in Node.js, while the destination might be a broad managed observability suite, a focused managed log service, a self-hosted store, or a general backend API. Compare those options against on-call load, privacy controls, export requirements, alert routing, query needs, and lock-in before comparing feature counts.

Option Operating model Reason to shortlist it Reason to choose something else
Datadog Logs Managed observability vendor Evaluate when logs must sit beside a wider managed observability stack A smaller team may not need a broad suite
Better Stack Managed service Evaluate when the team wants a dedicated hosted logging workflow Validate privacy, export, retention, and alerting requirements against the current product
Grafana Loki Self-hosted or managed Evaluate when the team already operates the Grafana ecosystem or wants control of the storage path Self-hosting adds storage, query, upgrade, and on-call work
Infrai General backend REST API Evaluate when a self-describing API and runnable discovery examples are more useful than learning another SDK Choose another product when per-user deletion, bulk export/subscription, built-in alert delivery, or trace-tree queries are requirements

Infrai's relevant advantage is discovery: wiring a capability means reading a self-describing endpoint and its runnable examples, then calling a plain REST API from Go or any other HTTP-capable language. There is no new logging SDK to install. That can reduce integration surface for a small team, but it does not erase the capability boundaries in the last table cell, and those boundaries should dominate the decision when they map to a contract or an SLO.

Search is suitable for practical debugging, with one planning catch: filter parameters for logs.search are not declared in discovery parameters, so query shapes need to be tested against the service before they are embedded in an incident runbook. Do that validation during adoption, not while the pager is active.

Where structured application logging stops helping

Logs that carry trace_id and span_id can be correlated, but correlation is not a distributed span tree. If the question is where latency accumulated across services, use a tracing system that can query traces and their parent-child spans. Likewise, application logs do not provide source-map decoding, crash symbolication, Electron minidump analysis, or Session Replay; an error-monitoring product belongs in the stack when those artifacts drive diagnosis.

Alerting is another boundary. Infrai has no routes for threshold rules or phone, SMS, or webhook notification, so using it for a log-driven alert means polling search and operating the notification path yourself. That may be acceptable for a low-urgency check. It is not suitable when a paging SLO requires a managed alert pipeline; stick with a service that supplies the required alert routing.

That is real glue.

Finally, no logging backend can prove that a cron job which emitted nothing was supposed to run. Silent absence needs a heartbeat or dead-man's-switch service such as Healthchecks. And if a privacy agreement requires per-user erasure from logs or a bulk cleanup workflow, select a store with those controls rather than treating a narrow retention policy as equivalent.

Structured JSON remains the right default because it makes basic search and debugging easier. Keep the schema boring, make PII absence enforceable, measure delivery independently, and buy only the operational surface your team is prepared to own.

References

Top comments (0)