Timestamp, Level, Service: Debugging Node.js Health Check Log Ingest 400s

Short answer: treat a 400 Bad Request from health-check log ingest as a payload-contract failure, capture the exact rejected bytes, and compare one record against the receiver's schema before changing retries or transport code.

Start with framing, then field types, then required fields. For a structured logging record, timestamp, level, and service may look obvious while still having the wrong type or spelling for that particular receiver. A valid JSON value can fail schema validation. Malformed JSON is only one branch of the diagnosis.

This distinction changed the shape of my debugging code. I care about time-to-first-call, so I want one tiny encoder, one captured response body, and no config maze. The deciding constraint is that a health probe can be healthy while its result record is rejected; application health and telemetry acceptance are separate signals.

Don't retry the same body yet.

What should a Node.js health check log include before JSON ingest?

It should include only fields that have an explicit contract with the ingest side. A practical local model might contain a timestamp, severity level, service identifier, check name, outcome, duration, and message. Those names are choices, not universal law. If the remote schema calls the duration latency_ms, sending durationMs is still wrong. If it declares level as a string, sending a numeric severity is wrong even though the number is meaningful to your logger.

Write the contract down as a table before touching the sender. This is dull work — good. It prevents a vague “the JSON looks fine” review from replacing an actual type comparison.

Field	Local decision	Rejection to look for
timestamp	UTC string produced by one clock helper	wrong type, invalid format, or missing value
level	closed string union	number or unknown label
service	non-empty startup value	absent or empty identifier
check	stable, low-variation name	dynamic URL or request ID used as a name
status	pass or fail	boolean or unexpected state label
durationMs	finite non-negative number	string, null, or negative number
message	short JSON-safe text	missing value or an oversized payload

Then inspect the body that actually crossed the network. The object printed before serialization isn't enough. Middleware can wrap it, a batcher can turn it into an array, and a line-oriented transport can add separators. Capture a redacted copy of the final bytes (and its exact framing) plus the response status and response text. Never capture credentials.

There are two independent questions: does JSON.parse accept the body, and does the parsed value satisfy the agreed schema? Keep them independent in the test output. A parser failure points to encoding or framing. A schema failure points to names, types, required values, or bounds. If both are collapsed into “bad JSON,” engineers tend to edit the serializer while the real mismatch sits in level: 30.

The transport contract matters too. A receiver expecting one JSON document should get one document. A receiver expecting line-delimited records should get exactly that documented framing. I'm not sure which contract your endpoint uses; its API definition and a captured accepted request resolve that uncertainty. Guessing from the URL doesn't.

The smallest working TypeScript implementation

I prefer to make invalid states hard to emit, but I don't pretend a TypeScript type validates runtime input. The type helps callers. The encoder below checks the few runtime values that can drift, serializes exactly once, and returns the final string that the transport sends.

type Level = "debug" | "info" | "warn" | "error";
type CheckStatus = "pass" | "fail";

interface HealthLog {
  timestamp: string;
  level: Level;
  service: string;
  check: string;
  status: CheckStatus;
  durationMs: number;
  message: string;
}

interface HealthResult {
  check: string;
  ok: boolean;
  durationMs: number;
  message: string;
}

function encodeHealthLog(service: string, result: HealthResult): string {
  if (service.trim() === "") throw new Error("service is required");
  if (result.check.trim() === "") throw new Error("check is required");
  if (!Number.isFinite(result.durationMs) || result.durationMs < 0) {
    throw new Error("durationMs must be a finite non-negative number");
  }

  const record: HealthLog = {
    timestamp: new Date().toISOString(),
    level: result.ok ? "info" : "error",
    service,
    check: result.check,
    status: result.ok ? "pass" : "fail",
    durationMs: result.durationMs,
    message: result.message,
  };

  return JSON.stringify(record);
}

interface RejectedPayload {
  status: number;
  responseText: string;
  body: string;
}

async function sendHealthLog(
  endpoint: string,
  body: string,
  authHeaders: Record<string, string>,
): Promise<RejectedPayload | undefined> {
  const response = await fetch(endpoint, {
    method: "POST",
    headers: {
      ...authHeaders,
      "content-type": "application/json",
    },
    body,
  });

  if (response.ok) return undefined;

  return {
    status: response.status,
    responseText: await response.text(),
    body,
  };
}

The endpoint remains an argument on purpose. So do the caller-supplied authentication headers. A generic example should not smuggle in a made-up route, and secrets don't belong in the record or its diagnostic copy.

For the first call, send one record. Batching hides the failing item and introduces a second contract about array or line framing. Once a single payload is accepted, add batching as a separately tested layer. Small steps win.

The function returns a rejection record rather than throwing away the response body. Persist that diagnostic through your existing secure error path, with the authorization header excluded. On a 400, compare body byte for byte with the fixture that your contract test accepts. Do not turn the retry count from three to ten; unchanged input usually reproduces an input rejection and creates noise around the useful evidence.

Consider a concrete debugging pass where the response text identifies $.level and the captured body contains "level":30. First prove that the body parses, which rules out malformed JSON for this fixture. Next compare the top-level shape with one known accepted record. Then compare the named field against the contract: the receiver wants a string, while the emitted value is a number. Change the mapping at the encoder, add the rejected value as a contract fixture, and send one corrected record. Do not rename unrelated fields, change the timestamp, wrap the object in an array, and add retries in the same edit; four simultaneous changes destroy the evidence about which contract mismatch produced the 400. This narrow loop is fast because every step removes one branch from the decision tree.

One field. One fix.

How can schema troubleshooting separate malformed JSON from a valid rejected record?

Build a local classification test. It needs fixtures for syntax, top-level shape, missing fields, and wrong field types. Keep the error categories boring and stable so an alert can say “schema: level” instead of “ingest failed.”

type Diagnosis =
  | { kind: "malformed-json"; detail: string }
  | { kind: "wrong-shape"; detail: string }
  | { kind: "schema"; detail: string }
  | { kind: "candidate"; value: Record<string, unknown> };

function diagnose(body: string): Diagnosis {
  let value: unknown;

  try {
    value = JSON.parse(body);
  } catch (error) {
    return {
      kind: "malformed-json",
      detail: error instanceof Error ? error.message : "JSON parse failed",
    };
  }

  if (typeof value !== "object" || value === null || Array.isArray(value)) {
    return { kind: "wrong-shape", detail: "expected one JSON object" };
  }

  const record = value as Record<string, unknown>;
  const requiredStrings = ["timestamp", "level", "service", "check", "status"];

  for (const field of requiredStrings) {
    if (typeof record[field] !== "string" || record[field] === "") {
      return { kind: "schema", detail: `${field} must be a non-empty string` };
    }
  }

  if (typeof record.durationMs !== "number" || !Number.isFinite(record.durationMs)) {
    return { kind: "schema", detail: "durationMs must be a finite number" };
  }

  return { kind: "candidate", value: record };
}

const fixtures = [
  "{",
  "[]",
  JSON.stringify({ timestamp: new Date(0).toISOString(), level: 30 }),
  JSON.stringify({
    timestamp: new Date(0).toISOString(),
    level: "info",
    service: "probe-worker",
    check: "database",
    status: "pass",
    durationMs: 12,
  }),
];

for (const fixture of fixtures) {
  console.log(diagnose(fixture));
}

This validator intentionally mirrors only the local example. In production, use the receiver's published schema or generate both sides from one schema artifact. Otherwise the local check can approve a record the receiver rejects. That false confidence is worse than having no validator because it sends the investigation toward the network.

Also test the HTTP wrapper. Assert the method, content-type, raw request body, and handling of the response text. A mock that asserts only “fetch was called” misses nearly every useful failure mode. I benchmark this path by time to isolate the bad field, not requests per second: logging code rarely needs an elaborate framework, but it does need an answer a tired operator can act on.

One more trap is the health check itself. If successful probes emit continuously, a perfectly accepted log stream can become high-volume background noise. Measure records per interval and bytes per record before choosing retention or sampling. Keep failures visible. Put temporary verbose fields behind a short-lived feature toggle so diagnosis doesn't permanently expand the steady-state payload; Fowler's feature-toggle discussion is useful here because it treats toggle lifetime and ownership as design concerns, not magic switches.

What changes at scale, and when is this approach not suitable?

At a few Node.js services, a shared encoder and contract fixture are easy to understand. At a mixed-language fleet, move the normalization boundary toward a collector or shared schema artifact. A Java service may use an appender while Node.js uses a transport function; the output contract should still agree. The Logback appender documentation is a useful primary reference for that extension point, but an appender does not remove the need to define the emitted record.

The catch is duplicated validation. If the schema changes often, hand-maintained checks in every emitter will drift. Generate validators from the contract, validate at a common collection boundary, or do both with a single versioned source. Stick with emitter validation when immediate feedback and a small service count outweigh that maintenance cost. Prefer a collection boundary when multiple languages, buffering, or centralized policy make per-service glue the larger risk.

This example is not suitable for audit records if rejection means silently dropping evidence. Route rejected payloads to controlled storage with retention and access rules, then alert on the count. It is also a poor fit for a receiver that requires batch framing: use its documented batch shape after proving one record and add a test that identifies which item caused rejection. Your mileage may vary with the ingest contract, but the diagnostic order should stay fixed: final bytes, JSON syntax, top-level shape, schema fields, then batching and transport.

Keep the operation visible. Track accepted records, rejected records by reason, and the age of the oldest unsent record. A health dashboard that only shows the target service as green can miss a broken observability path; a separate ingest signal closes that gap without pretending logs and uptime are the same system.

References

• Martin Fowler, “Feature Toggles”: https://martinfowler.com/articles/feature-toggles.html
• Logback Manual, “Appenders”: https://logback.qos.ch/manual/appenders.html