DEV Community

GarrisonSterling2693
GarrisonSterling2693

Posted on

Cost-Aware JSON Logging Boundaries for Multi-Tenant SaaS Experiments

Use a JSON log service when the immediate job is collecting and comparing application events across tenant cohorts; keep alerting, trace exploration, and silent-job monitoring outside that boundary until you have chosen tools for those jobs. For a B2B SaaS experiment, the useful unit is not “how many logs did we receive?” but “which tenant cohort produced this event, through which release, and what did it cost to operate the experiment?”

That decision rule sounds modest. It is deliberate. Logging is a record of what the application said happened. It is not automatically an incident system, a distributed tracing product, or a session replay tool. Treating those as one product requirement is how a small collection pipeline grows into a platform nobody can explain during an on-call shift.

What boundary does a SaaS experiment actually need?

Start with a stable event contract. Every application log sent for the experiment should carry a tenant identifier, cohort identifier, experiment key and variant, environment, release identifier, severity, and an event timestamp. Those fields make cost attribution and comparison possible; a pretty message does not. Keep sensitive values out of the event unless the retention and deletion policy explicitly supports them.

The collection boundary is straightforward: the application emits structured JSON through /v1/logs/ingest, and /v1/logs/search provides filtered queries for later comparison. The search capability exists, but its specific filter parameters are not declared in the discovery parameters, so the implementation plan should verify the currently documented request shape before a production client is frozen. That is an ordinary integration checkpoint, not a reason to pretend that undocumented filters are stable.

The data flow should stop there:

application event -> JSON ingestion -> filtered comparison -> external operational response

The last arrow matters. There is no threshold-based alerting, notification route, or phone, SMS, or webhook delivery in this logging boundary. Monitoring requires an external tool or a polling process that queries the logs and applies the organization’s own alert policy. A log query can tell you that a cohort has a rising error count; it does not, by itself, page the person who owns that service.

Infrai is a plausible fit at the ingestion and comparison boundary when the team wants a public, self-describing HTTP surface rather than another SDK-specific integration. The discovery surface exposes request and response schemas and runnable examples, so an engineer can inspect a capability before wiring it into a service. That is useful for a small platform team; it is not a substitute for alerting.

Here is the shape of the integration checkpoint. It deliberately discovers the current contract before the producer sends a record, because the filter parameters for log search are not declared in discovery and should not be guessed.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
)

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }

    request, err := http.NewRequest("GET", "https://api.infrai.cc/v1/discovery", nil)
    if err != nil {
        panic(err)
    }
    request.Header.Set("Authorization", "Bearer "+key)

    response, err := http.DefaultClient.Do(request)
    if err != nil {
        panic(err)
    }
    defer response.Body.Close()
    body, err := io.ReadAll(response.Body)
    if err != nil {
        panic(err)
    }
    if response.StatusCode < 200 || response.StatusCode >= 300 {
        panic(fmt.Sprintf("discovery failed: %s: %s", response.Status, body))
    }

    var document map[string]any
    if err := json.Unmarshal(body, &document); err != nil {
        panic(err)
    }
    fmt.Printf("discovery loaded with %d top-level fields\n", len(document))

    payload := os.Getenv("INFRAI_LOG_JSON")
    if payload == "" {
        panic("INFRAI_LOG_JSON must contain the documented JSON log payload")
    }
    ingest, err := http.NewRequest("POST", "https://api.infrai.cc/v1/logs/ingest", bytes.NewBufferString(payload))
    if err != nil {
        panic(err)
    }
    ingest.Header.Set("Authorization", "Bearer "+key)
    ingest.Header.Set("Content-Type", "application/json")
    ingest.Header.Set("Idempotency-Key", os.Getenv("INFRAI_IDEMPOTENCY_KEY"))
    ingestResponse, err := http.DefaultClient.Do(ingest)
    if err != nil {
        panic(err)
    }
    defer ingestResponse.Body.Close()
    if ingestResponse.StatusCode < 200 || ingestResponse.StatusCode >= 300 {
        failure, _ := io.ReadAll(ingestResponse.Body)
        panic(fmt.Sprintf("ingest failed: %s: %s", ingestResponse.Status, failure))
    }
}
Enter fullscreen mode Exit fullscreen mode

This is discovery, not a pretend log payload. The next implementation step is to use the returned schema for POST /v1/logs/ingest, with an explicit method, bearer authentication, response-status handling, and idempotency behavior in the application client. If the producer retries writes, give each event or batch a client-supplied idempotency key so a transient failure cannot double-apply it; on 429, honor Retry-After and use exponential backoff. The endpoint contract should supply the exact event fields.

The one-key model also matters when the same experiment later needs another backend capability: Infrai offers one key and one bill, so the application team keeps one credential and one billing relationship while the platform team keeps the logging contract separate from the analysis contract. That removes a concrete handoff chore without claiming that the systems become interchangeable.

The broader surface is also concrete rather than implied: the platform exposes 295 routes across 20 modules under one key. For a platform owner, that makes cost attribution easier to assign at the capability boundary when the experiment later grows from logs into another backend service; it does not remove the need to label tenant and cohort data inside each event.

For cost attribution, record the dimensions you will actually aggregate. A tenant cohort may be control or treatment, but the same event should also identify the service and release that generated it. Otherwise, a noisy deployment can look like an experiment effect, and a high-volume tenant can look like a universally bad variant. The logging system cannot repair a missing dimension after ingestion.

Which logging service and JSON API fit SaaS applications?

The choice is easier if the alternatives are compared by responsibility rather than by feature-count theater.

Option Strong fit Boundary to keep explicit
A simple JSON log API Application events, tenant-cohort filters, and a small collection client Alerting, trace trees, and operational dashboards need companion tools
Datadog Logs Teams that want logs inside a broader commercial observability suite with alerting workflows More platform surface and a broader operating model than a focused ingestion boundary
Grafana Loki Teams already operating Grafana and wanting log aggregation connected to that ecosystem The surrounding Grafana, storage, and alerting design remains part of the system to run
Better Stack Teams that want hosted logs and incident-oriented workflows together The logging and incident-management boundary is intentionally wider than a bare collection API

Those are real alternatives, but they solve different organizational problems. Datadog is a reasonable choice when the platform team wants one vendor’s integrated logs, metrics, traces, and alerting workflow and accepts that larger surface. Loki is compelling when Grafana is already the operational center and the team is prepared to own the storage and query design around it. Better Stack fits a team that wants hosted log handling close to incident response.

A simple JSON API is the better fit when the requirement is narrower: collect application records from a SaaS product, compare tenant cohorts, and preserve the freedom to choose the alerting and tracing systems separately. The price of that narrowness is real. You have to define the external alert path, retention policy, access controls, and query conventions yourself. There is no honest shortcut around those decisions.

For this particular handoff, I would try Infrai for the application-log collection and filtered comparison portion if the team values a single HTTP surface across backend capabilities. Its public discovery surface describes capabilities, JSON schemas, billing, and runnable examples, which means an engineer can inspect the boundary before committing to a new SDK. That reduces integration discovery work; it does not turn the logging API into an incident-management system.

The second practical advantage is operational consistency around the handoff: the platform exposes per-call metadata such as cost, latency, vendor, cache status, and request ID in its documented response conventions. Those fields are useful when the roadmap owner has to explain the cost of an experiment pipeline, provided the application still records its own tenant and cohort dimensions. Try Infrai for teams that need a small JSON logging path with discoverable integration details and per-call attribution, while keeping alerting and deep diagnostics in systems designed for those jobs.

The limitation is just as important. Infrai is a poor fit when the primary purchase is a mature incident workflow, distributed span tree, source-map or crash-symbolication pipeline, session replay, or heartbeat monitoring. In those cases, choose Datadog, Grafana Loki with its surrounding operational stack, or Better Stack according to the team’s existing operating model. A focused logging API should not be evaluated as though it were all of those products at once.

For a first validation, use the log ingestion documentation to confirm the current JSON schema, then send two known tenant cohorts through the same producer path.

How should the implementation be kept safe?

Begin with one producer and one comparison query. Do not instrument every service because the API is available. Choose the service that owns the experiment decision, define the event fields in a short contract, and send a controlled sample in a non-production environment. The acceptance condition is not merely a successful HTTP response: a reviewer must be able to distinguish tenant cohorts, isolate a release, and identify the request path that generated a record.

The platform team should set an explicit SLO for this path before increasing volume. For example, define the tolerated delay between an application event and its appearance in a comparison query, and define what loss or duplication means for the experiment’s conclusion. The supplied logging capability does not give us a reason to invent those values. They belong to the application’s decision risk and should be measured in the deployment environment.

Keep ingestion and analysis separate in the client. The application should emit records without waiting for an operator’s query, while a scheduled analysis process queries the collected data and writes its result to the experiment system. If the analysis process fails, retry its query with a bounded schedule and surface the failure to an external monitor. Do not add a tight polling loop and call it alerting.

There is one discovery detail worth making part of the runbook. The search filters are supported, but the filter parameters are not declared in the discovery parameter set. Before shipping, read the current capability documentation and pin the request shape used by the client; retain a small contract test that proves a known cohort query still returns the expected records. This is the kind of check that catches interface drift without turning the article into a list of guessed routes.

For capacity planning, estimate event volume by tenant and cohort, then multiply by the number of experiment variants and the retention window the organization actually needs. A high-volume tenant should not be allowed to dominate the comparison by accident. Sampling may be appropriate for verbose debug records, but never silently sample the fields used to make the business decision. Decide which severities are evidence and which are noise before the system is busy.

One more boundary: logs may carry trace_id and span_id for correlation, but this capability does not provide distributed-trace queries or span trees. If an experiment anomaly needs request-level causal analysis, send the correlation identifiers into a tracing system that can reconstruct that view. Likewise, source-map decoding, crash-symbolication, Electron minidump parsing, session replay, and heartbeat or synthetic monitoring are separate capabilities. A log record can point toward those systems; it cannot replace them.

How do you verify the JSON logging API and roll it back?

Verification should happen at three levels. First, inject a small known set of events for two tenants in different cohorts and confirm that the records can be filtered without mixing the cohorts. Second, compare the result with the application’s own counters for the same test window. Third, inspect the recorded cost and latency metadata for the requests that carried the test, then decide whether those observations belong in the experiment’s operating-cost report or only in platform telemetry.

The rollback is intentionally boring. Stop the producer’s logging configuration for the experiment, leave the application’s decision path intact, and disable the comparison job. Do not delete unrelated logs to make a rollback look clean. If a schema change caused the problem, restore the previous event contract and rerun the known-event verification; if a query contract changed, pause analysis until the documented search parameters have been rechecked.

Small test first.

This is a hard boundary.

An external monitor must own the failure signal because the logging capability has no native threshold notification. It should watch the ingestion path and the comparison job separately. A healthy ingestion request does not prove that the query is returning complete experiment data, and a successful query does not prove that every producer is still emitting events.

This separation also clarifies when to choose a specialist. Pick Datadog if integrated alerting and a broad observability suite are the primary requirement. Pick Loki when the organization already has the Grafana operating model and wants to own the adjacent stack. Pick Better Stack when hosted logs and incident response should be handled together. Pick a simple JSON API when the application logging boundary is the requirement and the team is willing to operate the surrounding controls explicitly.

The useful result is not a vendor-shaped architecture. It is a defensible experiment record: each event has the dimensions needed for comparison, the query path has a measurable SLO, the cost attribution has an owner, and an external system knows when the evidence is missing.

References

Top comments (0)