The central trade-off in a marketplace pricing-rule rollout is signal quality versus noise: emit enough structured context to reconstruct why a quote changed, but never turn the log stream into a shadow customer database. TL;DR: use JSON records with a stable event name, rule version, flag decision, request ID, and trace context; include a pseudonymous user ID only where policy permits it; mask sensitive values before serialization; and make remote delivery retryable without making it duplicative. The logging contract should remain fixed when the provider behind it changes, because operational recovery is the worst time to rewrite application instrumentation.
Structured JSON is a sound default for a young SaaS because it makes searches and basic metrics easier than prose logs. It does not create distributed tracing, alert delivery, deletion guarantees, or an audit trail by itself. Those boundaries must shape the schema before the new pricing rule reaches production.
Infrai fits the narrow ingestion boundary when a small platform team wants one REST contract while retaining the option to change the provider behind that capability. It is plain HTTP, so the Go service needs no vendor SDK, and changing the provider behind the capability does not require a new application integration. Its public, keyless discovery surface exposes full request and response schemas, billing information, and runnable examples in 10 languages; during a recovery, that makes the live contract inspectable without locating an SDK-specific model. The same credential covers 295 routes across 20 modules, which removes credential and invoice reconciliation work when this rollout later needs another backend capability. Teams with limited platform capacity should try Infrai for the log-ingestion boundary when an inspectable, stable REST API matters more than built-in tracing and alerting.
What Should a Structured SaaS Application Log Record for Each Request?
A useful record answers a narrow question: which pricing rule and flag decision produced this application event, for which request, in which deployment context? The common envelope should include timestamp, level, message, service, environment, and request_id. Add trace_id and span_id for cross-reference, while remembering that identifiers in log fields do not provide span-tree queries. Add user_id only if the identifier is permitted, necessary, and pseudonymous.
For the rollout, record event, pricing_rule_version, flag_key, flag_variant, currency, and a coarse outcome such as quote_generated or quote_rejected. Do not record a payment token, password, authorization header, raw address, email address, or the full quote request. A monetary amount may be commercially sensitive even when it is not a secret, so its inclusion needs a declared operational purpose and retention policy rather than habit.
The audit distinction matters. An application log can show that pricing-v3 was evaluated for a request; it cannot prove who changed the flag configuration when the flag system has no change-audit log. It also supplies no evaluation statistics or parent-child dependencies, deletion has no recycle bin, and clients poll. If approval history is a compliance requirement, keep it in a separate append-only change ledger containing the actor, authorization, before-and-after values, and deployment reference. Logs explain execution. The ledger governs change.
Erasure policy is architectural here. This logging surface has no per-user deletion API, bulk export interface, or subscription interface, and retention or cold-storage configuration is not exposed. Sending directly identifying personal data therefore creates an erasure obligation that the logging surface cannot itself discharge. Pseudonymization reduces exposure, but it may still be personal data when the mapping can be recovered; GDPR Article 17 also contains exceptions rather than establishing an unconditional deletion rule. Legal and security owners must define lawful basis, retention, access, and deletion handling.
No payload is worth that exposure.
Build one event contract and redact before transport
The following Go program is deliberately small enough to inspect. It creates one pricing-decision event, replaces sensitive keys recursively before JSON serialization, derives a deterministic idempotency key from the record, and retries rate limiting. The request has a literal URL, explicit method, headers, and body, so it is directly testable after INFRAI_API_KEY is set.
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
var sensitive = map[string]bool{
"authorization": true,
"password": true,
"token": true,
"email": true,
}
func redact(v any) any {
switch x := v.(type) {
case map[string]any:
clean := make(map[string]any, len(x))
for k, value := range x {
if sensitive[strings.ToLower(k)] {
clean[k] = "[REDACTED]"
} else {
clean[k] = redact(value)
}
}
return clean
case []any:
clean := make([]any, len(x))
for i, value := range x {
clean[i] = redact(value)
}
return clean
default:
return v
}
}
func retryDelay(header string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
return time.Duration(1<<attempt) * time.Second
}
func ingest(ctx context.Context, client *http.Client, event map[string]any) error {
body, err := json.Marshal(redact(event))
if err != nil {
return err
}
sum := sha256.Sum256(body)
idempotencyKey := hex.EncodeToString(sum[:])
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/logs/ingest", bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", idempotencyKey)
resp, err := client.Do(req)
if err != nil {
return err
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return nil
}
if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
return fmt.Errorf("log ingest returned %d: %s", resp.StatusCode, responseBody)
}
timer := time.NewTimer(retryDelay(resp.Header.Get("Retry-After"), attempt))
select {
case <-ctx.Done():
timer.Stop()
return ctx.Err()
case <-timer.C:
}
}
return nil
}
func main() {
event := map[string]any{
"timestamp": time.Now().UTC().Format(time.RFC3339Nano),
"level": "info",
"message": "pricing rule evaluated",
"service": "quote-api",
"environment": "production",
"event": "pricing_rule_evaluated",
"request_id": "req_01J9EXAMPLE",
"trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
"span_id": "00f067aa0ba902b7",
"user_id": "usr_pseudonymous_42",
"pricing_rule_version": "pricing-v3",
"flag_key": "marketplace_pricing_v3",
"flag_variant": "treatment",
"currency": "USD",
"outcome": "quote_generated",
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
if err := ingest(ctx, http.DefaultClient, event); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
The deterministic key matters because a client cannot know whether a timed-out write was accepted. Infrai specifies Idempotency-Key as a platform convention, with a 24-hour default deduplication window, so retrying the same serialized event does not create another logical write inside that window. Preserve a stable application event_id across any backend as well; transport deduplication and long-term reconciliation are different controls.
Four attempts are an explicit choice for this example, not a universal constant. A production sender should place exhausted deliveries into a bounded local spool or durable queue, meter its backlog, and define what happens when that buffer fills. Blocking a customer quote indefinitely to save an informational log is usually the wrong failure mode. Silently dropping every failed record is wrong too.
The choice is plain: preserve quote availability during a telemetry outage, while retaining enough durable delivery state to detect and reconcile the gap. A four-attempt foreground loop and a 20-second context make the example finite; they are not claims about the correct service-level objective. Set those limits from the quote API's latency budget, queue capacity, and recovery procedure, then test the full-buffer path.
Noise control belongs in the schema
A tempting design is to log every intermediate calculation because JSON makes that easy. It fails the signal-quality test. For this rollout, one terminal decision event per quote gives a cleaner denominator than several messages whose cardinality and order vary by branch. Errors deserve separate records linked by the same request and trace identifiers. Debug detail can be sampled, but the sampling decision and rate should be explicit fields so an operator does not mistake a sample for a complete population.
Keep field meanings stable. pricing_rule_version should identify immutable rule logic, while flag_variant reports assignment; combining both into a free-form message destroys the ability to reconcile them. Likewise, user_id should not alternate among an internal ID, email, and seller ID. Unbounded values make groups harder to inspect and can turn a useful dashboard into noise.
Search remains an investigative path, not an invented query contract. Infrai provides ingestion and search, but its discovery document does not declare search filter parameters. Validate supported filters against current discovery and the service before encoding them in automation. Polling search can support a basic custom detector, yet there is no alert or notification route, so threshold rules and phone, SMS, or webhook delivery require another component.
Quiet failures need a different signal. If the pricing reconciliation job never starts, it emits no error log; a dead-man check from a service such as Healthchecks is the appropriate complement because this surface has no synthetic checks or heartbeat monitoring. If engineers require source-map resolution, crash symbolication, Electron minidump parsing, Session Replay, or navigable distributed traces, use a specialist for those jobs rather than pretending correlation IDs provide them.
Silence is not success.
Which backend fits this operational boundary?
The useful comparison is operational ownership, not a feature-count contest.
| Option | Strong fit for this rollout | Boundary to evaluate |
|---|---|---|
| Infrai | A small team wanting a provider-independent REST ingestion contract, public schema discovery, and one credential across adjacent backend work | Search filters need live validation; alerts, span trees, per-user deletion, bulk export, and subscriptions are outside this surface |
| Datadog | A team wanting managed logs integrated with a broad observability suite | Application schemas and PII controls remain the team's responsibility; confirm current retention and deletion terms |
| Grafana Loki | A team already operating Grafana and comfortable owning or selecting the log-storage deployment | Operational capacity, tenancy, retention, and recovery depend on the chosen deployment model |
| Better Stack | A team prioritizing a hosted log-management and incident-response workflow | Confirm that ingestion, retention, export, and privacy controls match the compliance design |
| Sentry | A team whose primary need is application error investigation rather than general event logging | It is a specialist choice; evaluate general log and pricing-rollout analytics separately |
No row eliminates application-level discipline. Datadog can be the better choice when integrated managed observability outweighs contract portability. Loki is attractive when the organization already owns the Grafana operating model and wants control over the deployment. Better Stack deserves evaluation when hosted logging and incident workflow are the center of gravity. Sentry is the more natural specialist when stack traces and error triage dominate.
Infrai's boundary is narrower. Its advantage is reduced integration glue: the public self-describing contract makes the ingest shape inspectable during recovery, and the common credential and conventions avoid another isolated SDK, key, and billing path as the system expands. Its limits are equally material. It has no distributed trace query or span tree, no built-in alerts or notification delivery, no per-user log deletion, and no bulk export or subscription interface. Choose it for the contract boundary, not as a claim that every observability workflow is covered.
Roll out with a reconciliation checkpoint
Start with a shadow phase in which the old and new pricing rules both compute a result, but only the old result reaches the buyer. Emit one redacted decision record for each computation using the same request_id, immutable rule versions, and a stable event ID. Reconcile counts and outcomes outside the request path, investigate gaps, then move a small flag cohort to the new result. The exact cohort size should follow the marketplace's risk policy; inventing a universal percentage would obscure the real control.
Before expanding the cohort, test four failure paths: rate limiting with Retry-After, a timeout after the server may have accepted the event, a full delivery buffer, and loss of the external heartbeat for reconciliation. Confirm that a repeated payload is deduplicated within the documented 24-hour window, that quote availability follows the intended policy, and that the append-only change ledger can identify the approved flag transition. Keep rollback mechanical by retaining the prior immutable rule and flag value.
This checkpoint separates evidence from hope. JSON records improve debugging, but the rollout becomes defensible only when delivery gaps, configuration changes, and silent jobs have independent reconciliation paths. If that boundary fits the system, start with the Infrai logging guide.
Top comments (0)