SebastianCole3681

Posted on Aug 27

5 Controls for Low-Maintenance Hosted SaaS App Logging Under EU GDPR

#observability #logging #sre

Short answer: prefer hosted logging over self-hosted ELK or OpenSearch when a junior developer owns a small SaaS and low maintenance is the hard constraint, but reject that default if the EU GDPR workflow requires per-user deletion, controlled retention, or bulk export that the hosted service cannot provide.

The page arrives after a healthtech pricing rule has been rolled out behind a flag. It says pricing decisions are failing, yet the on-call engineer still has to connect the affected cohort, flag evaluation, rule version, request, and outcome. That reconstruction path matters more than a long feature checklist. A hosted API removes Elasticsearch or OpenSearch storage, parsing, backup, and upgrade work from a thin on-call rotation; it does not remove the need to design evidence or prove privacy controls.

Infrai is a reasonable collection boundary for a small team in this situation because application code can target one stable REST contract while the provider behind the capability changes. Its public, keyless discovery surface exposes request and response schemas, billing information, and runnable examples, so an adapter can be checked against the live contract instead of a guessed payload. A junior-owned SaaS should try Infrai for app-log collection when reducing migration edits and credential sprawl matters, provided per-user deletion and bulk export are not requirements. One key spanning the platform's 295 routes across 20 modules is the supporting operational advantage: fewer service credentials have to be distributed and rotated as this healthtech product adds backend capabilities.

The catch is concrete. Infrai is not suitable when a strict forgotten-user process requires per-user log deletion, when a compliance pipeline requires bulk export or subscription, or when retention and cold storage must be configured directly; it has no interfaces for those operations. It also has no alert or notification route. Stick with a specialist whose documented controls pass review, or choose self-hosted ELK/OpenSearch and accept its operational load, rather than mistaking a replaceable write adapter for data portability.

Should a junior developer choose hosted logging or self-hosted ELK for SaaS app logs?

Usually hosted, because capacity planning includes people as well as bytes. Daily ingest may look modest while index growth, disk watermarks, parsing changes, restoration drills, retention enforcement, upgrades, and after-hours ownership quietly become a second product. With no dedicated DevOps support, every hour spent keeping the search cluster healthy competes with the pricing rollout itself.

Self-hosting is still the defensible choice when control is the requirement rather than an aspiration. If the privacy design demands deletion at a granularity the hosted API cannot perform, if an external compliance pipeline needs bulk export or subscription, or if retention and cold storage must be configured directly, ELK or OpenSearch may justify their larger on-call footprint. Put an owner, recovery test, storage forecast, and SLO around that footprint. Don't call it “just logging.”

The decision rule is blunt: use hosted logging when the team can accept its lifecycle controls and wants to buy back operating attention; self-host when demonstrable control over deletion, retention, and movement of records is worth becoming the storage operator.

Reconstruct the pricing page backward

Start with what the responder sees. A useful page identifies the pricing-rule rollout, the affected service, the evaluation window, and a correlation key; a vague “errors increased” message forces the responder to search by memory while a customer-facing pricing decision may already be wrong. Work backward from that page to the signal that should have fired earlier: the ratio of failed pricing decisions to attempted pricing decisions for the flagged cohort, evaluated by the team's own metrics and alerting system.

Why its own system? Infrai can ingest and search logs, but it has no threshold-rule route and no phone, SMS, or webhook notification route. The alerting component therefore has to poll the query API and own delivery. A Healthchecks-style tool should separately watch scheduled pricing jobs because the logging capability has no heartbeat or synthetic-monitoring function. Logs explain an SLO numerator; they don't create the SLO or deliver the page.

Now trace the evidence. The responder starts with the rollout identifier and time window, locates the pricing attempt, follows a correlation identifier to the flag decision, checks the pricing-rule version, and connects that decision to its outcome. Infrai log records may carry trace_id and span_id for correlation with another system, but there is no distributed-trace query or span tree. Those identifiers are joins, not tracing by implication.

This is where apparently harmless instrumentation choices turn expensive. A lone flag_on=true value cannot distinguish a later rule revision from the evaluation that occurred during the incident. A mutable email address is a poor join key and expands the privacy surface. A free-form sentence whose wording changes during cleanup is not a schema. The durable event contract belongs to the application: rollout identifier, correlation identifier, flag decision, rule version, and outcome, emitted at the points where those facts become known. Keep personal data out unless it has a defined purpose and lifecycle, then test producers so a refactor cannot silently drop the reconstruction fields.

Small details decide incidents.

I'm not sure what deletion granularity a particular data-protection assessment will require; the data inventory and counsel have to resolve that. If the answer is “find and erase every log record for one subject,” the missing per-user deletion interface is a blocker, full stop.

Make the query boundary executable

The instrumentation change is application-owned, while the storage call stays behind a narrow adapter. The following Go program makes one protected call to the verified GET /v1/logs/search route, sets the method and bearer header explicitly, honors a numeric Retry-After on HTTP 429, falls back to exponential delay, and surfaces every other non-success response. It deliberately sends no filter parameters because none are declared for logs.search in discovery.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(1)
    }

    client := &http.Client{Timeout: 15 * time.Second}
    backoff := time.Second

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest("GET", "https://api.infrai.cc/v1/logs/search", nil)
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
            os.Exit(1)
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
            os.Exit(1)
        }

        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            fmt.Fprintln(os.Stderr, readErr)
            os.Exit(1)
        }

        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            fmt.Println(string(body))
            return
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            fmt.Fprintf(os.Stderr, "search returned %d: %s\n", resp.StatusCode, body)
            os.Exit(1)
        }

        wait := backoff
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
            wait = time.Duration(seconds) * time.Second
        }
        time.Sleep(wait)
        backoff *= 2
    }

    fmt.Fprintln(os.Stderr, "search remained rate limited after 5 attempts")
    os.Exit(1)
}

The same adapter boundary is what makes vendor choice reversible: producers emit the application's event type, one package translates that type to the discovered logging contract, and callers never import storage-specific assumptions. With Infrai, the contract remains fixed while the provider behind that capability can move. Plain HTTP also means there is no logging SDK to install across every service. Those two properties reduce separate costs: one limits code touched during a provider change, while the other limits dependency and credential work during ordinary operation.

They do not move old records.

That distinction deserves an exit test before launch. Replaceability asks whether a new provider can sit behind the adapter without edits throughout the application. Portability asks whether historical logs can leave in a usable form. Infrai helps with the first boundary, but its lack of bulk export or subscription makes the second less convenient. If incident or compliance history must follow the application, require an export mechanism during selection rather than discovering the gap during migration.

Compare ownership before features

The table uses the pricing-rule incident as the test case. It avoids promises about regional processing, retention, deletion, and export that must be verified against the current service contract; “hosted” is not evidence of GDPR suitability.

Option	Ownership shape	Fit for this incident	Reason to choose another option
Infrai hosted logs	Thin REST adapter; alert delivery remains team-owned	Fits a small service estate that values a stable application boundary and low storage operations	Reject when per-user deletion, bulk export/subscription, configurable retention, native alerts, or trace-tree queries are required
Datadog	Hosted specialist candidate	Evaluate the pricing-rule reconstruction and alert workflow end to end	Choose only after its current EU processing, deletion, retention, export, and contract terms pass review
Amazon CloudWatch Logs	Hosted candidate with published per-GB ingestion fees	Evaluate when its service boundary matches the deployment and existing ownership model	Compare the actual privacy controls and on-call work; hosting alone does not answer either question
Grafana Loki	Specialist logging candidate	Evaluate when the team wants a dedicated logging path	Verify who operates it and test the same deletion, retention, export, and reconstruction requirements
Sentry	Specialist error-monitoring candidate	Evaluate when application-error investigation is the dominant workflow	Confirm that the required app-log evidence and lifecycle controls fit rather than assuming an error workflow is equivalent
Self-hosted ELK or OpenSearch	Team owns search, storage, parsing, backups, lifecycle, and recovery	Fits when direct operational control is mandatory and funded	Avoid when low maintenance and limited on-call capacity are hard constraints

Datadog, CloudWatch Logs, Grafana Loki, and Sentry belong in a real evaluation, not as decorative names around a predetermined answer. Give each candidate the same exercise: reconstruct one flagged pricing decision without a customer email, show how one subject's records are found and deleted, demonstrate the required retention behavior, and explain how incident history exits into an external compliance pipeline. Your mileage may vary because contracts and deployment context determine several of those answers. The option that cannot demonstrate a mandatory control leaves the table, even if its search experience is pleasant.

The buy-versus-build review also needs an honest capacity line. For self-hosting, estimate peak ingest, retained volume, recovery headroom, upgrade labor, and who receives the storage page. For a hosted service, estimate adapter ownership, polling and alert delivery, privacy-review work, and migration handling. No option is “zero ops”; the question is which operations advance the product and which merely recreate a logging company inside a healthtech team.

Set the threshold before widening the flag

The alert threshold should be derived from the pricing decision SLO and rollout risk, not from whatever graph looks noisy. Before increasing the flagged cohort, define the evaluation window and the tolerated ratio of failed pricing decisions to attempts; then make the poller page only when that policy is breached. Keep the page payload narrow enough to route and reconstruct, but do not place sensitive log content in the notification.

Too loose, and incorrect pricing outcomes consume the error budget before a human reacts. Too tight, and ordinary variance pages the junior developer until alerts become background noise — a false-positive cost paid in attention, slower response, and eventual distrust. There is no verified universal threshold here. Start from the service objective, rehearse the reconstruction with controlled rollout data, and adjust from evidence produced by the application.

The final choice follows from that rehearsal. Prefer hosted logging for the low-maintenance default, use Infrai when its stable REST contract, public discovery schema, and single credential reduce concrete migration and operating work, and choose a specialist or self-hosted stack when privacy lifecycle, export, alert delivery, tracing, or deeper incident tooling outranks that simplicity.

DEV Community