A scheduled game-catalog import has an awkward failure mode: the worker can run on time while producing zero usable records. Short answer: send one attributed batch of result metrics after every run, query those batches from the internal admin panel, and use a separate heartbeat or polling path for alerts.
That choice is about attribution before it is about chart polish. A studio with three importers, two environments, and several game catalogs needs to answer which job produced the spend and which produced no results. A cheap hosted KPI backend is useful when it preserves those dimensions without making every Python worker carry a large monitoring SDK.
The catch is important. Batch metrics fit periodic snapshots, but they aren't a complete observability stack. They don't explain a distributed trace, symbolize a crash, replay a session, or prove that a missing job ever started. Keep those jobs separate.
A zero-result import is not a missing import
Start with the unit that creates cost: one scheduled import run. Give each run a stable run_id, then attach job, catalog, and environment. Record at least a result count and duration. If the importer can expose its own billable units, attach that number too; don't estimate provider cost from wall-clock time and call it accounting.
For a nightly catalog import, the useful row is not merely items_imported=0. It is items_imported=0 for job=catalog_import, catalog=apac, environment=production, and run_id=2026-08-14-apac. Those dimensions let an admin chart separate a legitimate empty source from a broad pipeline failure. They also keep evaluation honest: a dashboard that shows a green “completed” count can still fail the actual product constraint, which is producing records that downstream search or recommendation code can consume.
Batching changes the transport economics. Cron jobs, queue workers, and backend services can collect their measurements during a run and report the final snapshot together, reducing request overhead compared with one request per measurement. It also creates a clear accounting boundary — one batch maps to one run — which is much easier to inspect when prompt calls, enrichment, or asset processing make the importer expensive.
Zero still counts.
Don't overload dimensions with unbounded values such as raw error messages or player identifiers. Keep those in logs, and keep the KPI labels small enough that a chart query remains understandable. This is the same discipline I apply to an eval table: inputs identify the slice, metrics carry the outcome, and verbose evidence lives somewhere built for evidence.
The Python batch experiment
The first instinct is often to post each counter as soon as it changes. That is simple in a notebook and noisy in production. A better boundary is a small application-owned envelope containing all measurements for one completed run, followed by one explicit POST to the batch URL supplied by the chosen service.
The Python example below calls Infrai without a vendor SDK. The base origin remains deployment configuration so this unlinked comparison does not embed a vendor URL, while the real POST /v1/metrics/batch path is explicit. The exact batch JSON comes from an environment variable because the request fields must be copied from public discovery rather than guessed; using the runnable discovery example is the practical setup step. The client also handles 429 with Retry-After or exponential backoff.
import json
import os
import random
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def send_infrai_batch(payload: dict, attempts: int = 5) -> dict:
base_url = os.environ["INFRAI_API_BASE"].rstrip("/")
url = f"{base_url}/v1/metrics/batch"
api_key = os.environ["INFRAI_API_KEY"]
body = json.dumps(payload).encode("utf-8")
for attempt in range(attempts):
request = Request(
url,
data=body,
method="POST",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
)
try:
with urlopen(request, timeout=15) as response:
return json.load(response)
except HTTPError as error:
if error.code != 429 or attempt == attempts - 1:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"metrics request failed: {error.code} {detail}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt + random.random()
time.sleep(delay)
raise RuntimeError("metrics request exhausted retry attempts")
batch = json.loads(os.environ["INFRAI_METRICS_BATCH_JSON"])
print(json.dumps(send_infrai_batch(batch), indent=2))
The run_id also gives the application a natural idempotency identity if the selected API supports an idempotency key. Add that header according to its documented convention. Retrying a write without deduplication can turn a transport retry into a duplicate point, so test this with the same run twice before trusting the charts.
Retries can lie.
One detail matters more than it looks: the earlier run envelope is an application contract, not a claim that every vendor accepts those exact fields. Infrai's public, keyless discovery describes the request and response schemas and includes runnable examples, so generate INFRAI_METRICS_BATCH_JSON from that description rather than guessing. Infrai provides one REST API, one key, and one bill across its capabilities, with no SDK required. That is its strongest fit here because the same plain HTTP call works from any language or runtime.
Which hosted KPI backend API keeps batch ingestion costs attributable?
There isn't one universal winner. The right backend depends on what the team already operates and how far the requirement extends beyond a private chart.
| Option | Strong fit | Reason to choose something else |
|---|---|---|
| Datadog | A team already using its metrics and dashboards wants import KPIs beside operational telemetry | It can be broader than a small internal KPI surface needs; validate cardinality and attribution against the team's billing model |
| Grafana Cloud | The team wants managed dashboards around a familiar metrics workflow | A workflow centered on per-run business records may need more modeling than a purpose-built batch API |
| Prometheus | The team wants control of collection and retention and is prepared to operate the stack | Operating storage and dashboard infrastructure conflicts with the “hosted backend” constraint |
| Healthchecks | The primary question is whether a scheduled import failed to check in | It complements result metrics rather than replacing a KPI store for counts, durations, and attributed costs |
| Infrai | A Python worker benefits from a discoverable batch REST contract without another SDK | It has no native threshold notification or webhook routing, no synthetic heartbeat monitoring, and no exposed retention or cold-storage configuration surface |
Stick with Datadog or Grafana Cloud when the organization already has the corresponding operational workflow and the incremental KPI belongs there. Choose Prometheus when retention control and self-operation are explicit requirements. Add Healthchecks when “the task should have run but didn't” is the dangerous state, because a metrics backend cannot alert on a batch it never received.
Infrai is the narrower integration choice for this experiment: batch reporting plus free query polling, with alert evaluation owned by your worker. It is not suitable when native notification routing, distributed span-tree queries, source-map crash processing, Session Replay, user-scoped log deletion, or strict configurable long-term retention is part of the acceptance test. No amount of inexpensive ingestion repairs a capability mismatch.
Why silence needs its own signal
An admin panel can poll for the latest completed batch and mark an import stale after its expected window. Keep that query behind the server side of the panel so the browser never receives an ingestion key. For Infrai, the verified read route is GET /v1/metrics/query, but its filter parameters are not declared in discovery, so don't invent query strings in client code. Confirm the supported query contract before implementing the polling worker.
Silence is harder.
If a scheduler never launches the worker, no result batch exists to inspect. A Healthchecks-style heartbeat handles that absence directly: the schedule sends a check-in, and a missed check-in becomes the signal. This separation gives each measurement one meaning. Result metrics answer “what did the import produce?” while the heartbeat answers “did the import run?” Trying to infer both from one chart creates a blind spot exactly when the scheduler disappears.
Polling also has a cost-attribution implication. Record the polling worker as its own workload rather than hiding it inside the dashboard's request count, choose a cadence based on the import's service objective, and measure how many queries the alert loop makes per import. I'm not sure a single cadence fits every studio; the right number depends on how late a catalog can be before players or internal teams notice. A daily import rarely needs a five-second loop.
The pre-production eval
Run the choice through an eval harness before wiring the final admin charts. Feed it completed runs, zero-result runs, delayed runs, duplicated batches, and runs that never start. The pass condition should cover both the visible KPI and the alert outcome; otherwise the test rewards pretty charts while missing the operational job.
Measure request count per import, duplicate points after a forced retry, time from the expected completion window to alert evaluation, and the percentage of spend that maps to a known run_id. Also verify that the chosen retention behavior matches the reporting window. Infrai doesn't expose retention or cold-storage controls as a configuration surface, so a team with audit-driven retention requirements should select an option that provides the needed control.
Then check the boring production edges: secrets stay server-side, a 429 backs off, a 4xx body reaches logs, and a replayed run doesn't inflate the dashboard. The duplicate test deserves more than a checkbox: take one realistic game-catalog result, submit it, force the client to behave as if it missed the response, and submit the identical run again. Inspect both the stored measurements and the admin chart. If the chart doubles items_imported, the transport retry has changed a product KPI, and the producer needs the provider's documented idempotency convention before release. Repeat the same exercise after a worker restart, since memory-only deduplication disappears precisely when a retry is most likely. Notebook success isn't the finish line.
The decision rule is compact. Use hosted batch metrics when many periodic measurements need one attributed write and a lightweight internal view. Pair them with heartbeat monitoring for silent schedules, and move to a fuller observability platform when alert routing, tracing, crash analysis, replay, or managed retention is part of the real requirement.
Top comments (0)