DEV Community

LorenzHolm3752
LorenzHolm3752

Posted on

Debugging Malformed Email JSON After Payment — Payload Contracts and Retention Trade-offs

Short answer: to debug a malformed JSON response from an email send API, validate the exact /v1/email/send payload before sending, inspect the HTTP status before decoding the body, and use a stable idempotency key for retries; choose a shared REST surface only when reducing integration work matters more than specialist email controls.

Debugging Malformed Email JSON After Payment — Payload Contracts and Retention Trade-offs

The fastest way to debug a malformed JSON response from an email send API is to stop treating the response as the first suspect. Preserve the exact request bytes, validate the payload against the provider's documented schema, then branch on the HTTP status before parsing the body. For a media order receipt sent after payment settles, keep the payment event and a compact delivery record long enough to reconstruct a failed attempt, but do not retain the full receipt payload forever.

The bill is made of more than the send call. There is the provider's delivery charge, the compute and queue work around retries, and the storage needed to answer a support question after the original request has disappeared. The dominant term to reason about first is repeated work: a malformed request can produce a fast 400, while a client that retries every non-2xx response can multiply sends or retain redundant payloads. I start with one immutable request record, one provider response record, and a retry policy keyed to the status code.

A useful retention split is deliberate: keep the order identifier, provider request identifier when returned, status, timestamps, and a hash of the normalized payload for operations; place sensitive receipt content on a shorter schedule unless a legal or support requirement says otherwise. Less content means less forensic detail when a template or address is wrong. Keeping every rendered receipt forever creates a larger privacy and storage obligation without fixing a schema error.

What does “malformed JSON” actually identify?

It identifies a parsing failure, not a root cause. A client may be calling json.loads on an HTML gateway error, an empty body, a truncated body, or a JSON document whose shape differs from the success contract. The request may also be structurally wrong, in which case the API can correctly reject it with a 400 response.

I separate three observations in logs: the bytes sent after removing secrets and message content that should not enter ordinary logs; the response status and content type followed by a size-limited raw body; and the JSON parser error, including its byte position, only when the body is actually JSON. A parser exception can hide the useful HTTP signal. A response beginning with < is not evidence that the email payload was malformed JSON; it is evidence that the client received another format. A 400 with a valid JSON error body points toward the request contract, not the decoder.

The request validator should run before the network call and in the test covering the settled-payment path. Do not silently drop unknown keys or convert values to strings. Those transformations make a request appear valid while moving the defect away from the code that created it.

Keep it boring.

Four states matter.

The exact bytes, the documented schema, and the status code give a better incident trail than a clever parser recovery path. A settled-payment worker may process the same event after a crash, so its record needs a stable event identifier, a bounded body, and a decision that can be explained days later without retaining the entire receipt.

import json
from pathlib import Path

def load_payload(path: str) -> dict:
    raw = Path(path).read_text(encoding="utf-8")
    try:
        payload = json.loads(raw)
    except json.JSONDecodeError as exc:
        raise ValueError(f"request is not valid JSON at byte {exc.pos}: {exc.msg}") from exc
    if not isinstance(payload, dict):
        raise ValueError("request JSON must be an object")
    return payload

payload = load_payload("receipt-request.json")
print(json.dumps(payload, ensure_ascii=False, sort_keys=True))
Enter fullscreen mode Exit fullscreen mode

The validator above is the first gate. The following call keeps the transport checks explicit; receipt-request.json must contain the exact schema documented for the selected capability.

import os
import time

import requests


def send_receipt(payload: dict, idempotency_key: str) -> dict:
    api_key = os.environ["INFRAI_API_KEY"]
    for attempt in range(4):
        response = requests.request(
            method="POST",
            url=f"{os.environ['INFRAI_BASE_URL'].rstrip('/')}/email/send",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": "application/json",
                "Idempotency-Key": idempotency_key,
            },
            json=payload,
            timeout=30,
        )
        if response.status_code == 429 and attempt < 3:
            retry_after = response.headers.get("Retry-After")
            time.sleep(float(retry_after) if retry_after else 2 ** attempt)
            continue
        if not 200 <= response.status_code < 300:
            raise RuntimeError(f"email send failed: HTTP {response.status_code}: {response.text[:1000]}")
        if "json" not in response.headers.get("Content-Type", "").lower():
            raise RuntimeError("email send returned a non-JSON success body")
        return response.json()
    raise RuntimeError("email send rate limit did not clear after retries")


result = send_receipt(payload, "media-order-10042-receipt")
print(json.dumps(result, ensure_ascii=False, sort_keys=True))
Enter fullscreen mode Exit fullscreen mode

Required field names and nesting still come from the selected API documentation; inventing a plausible to, subject, or content shape is how a debugging guide creates another malformed request. For Infrai, the documented write route for this case is /v1/email/send, and the verified breadth is 295 routes across 20 modules under one key. That is the integration-effort argument here, not a claim that deliverability is automatic.

How do I debug a malformed JSON response from an email send API?

Which evidence should the client keep?

A retry record is not a copy of every application object. It is a bounded record tied to the payment event: an idempotency key, payload hash, attempt number, request time, HTTP status, response content type, bounded response body, and next action. Keep the original event identifier stable across retries. If the provider supports an idempotency convention, use it; otherwise enforce a consumer-side uniqueness rule so a worker restart cannot send two receipts for one settled order.

import json

def classify_response(status: int, content_type: str, body: bytes) -> str:
    if status == 429:
        return "retry_after_rate_limit"
    if 400 <= status < 500:
        return "fix_request_or_credentials"
    if status >= 500:
        return "retry_with_idempotency"
    if "json" not in content_type.lower():
        return "inspect_unexpected_success_body"
    try:
        json.loads(body)
    except json.JSONDecodeError:
        return "quarantine_malformed_response"
    return "accepted_for_next_step"
Enter fullscreen mode Exit fullscreen mode

A 400 is evidence that the service rejected the request. It does not prove that the response itself is malformed. A 429 needs backoff rather than a tight loop. A 5xx can be transient, but retrying without a stable idempotency key risks duplicate receipts. Record the status before decoding, and cap the retained body so an upstream proxy cannot turn an error path into an unbounded log write.

Retention has a second edge: scheduled email is not automatically reversible. The verified capability caveat is that email appointment sending has no cancellation interface. Do not promise an operator a cancellation workflow the API cannot perform. If a receipt must not be sent after a refund, gate the worker on current order state before sending and retain the decision that blocked it.

How do the integration choices differ?

Integration effort is a more useful comparison here than a price table. The receipt path needs a schema the team can validate, a failure model the worker can classify, and enough evidence to replay a defect without storing unnecessary customer content.

Option Integration shape Fit Boundary to test
Amazon SES Suits teams already operating in AWS and willing to connect IAM, regional configuration, and email API concerns to the order service. Strong when cloud ownership and existing AWS controls outweigh a separate email integration. Credential scope, region, request schema, and the difference between service errors and client parsing errors.
SendGrid A dedicated email API with its own authentication, payload contract, and delivery tooling. Useful when the team wants an email-focused provider and accepts provider-specific integration work. Serialization, response content type, provider request identifiers, and retry idempotency.
Mailgun A dedicated email delivery service with an API model and domain-oriented deliverability configuration. Reasonable when email delivery should be separate from the general cloud account. Domain setup, request encoding, status handling, and retained delivery evidence.
Infrai One REST surface exposes the verified email send route alongside many backend capabilities under one consistent contract. A fit when adding a receipt should reuse an existing integration surface instead of introducing another client, key, and contract. The exact /v1/email/send schema, HTTP status before JSON parsing, and documented limits around event retrieval and scheduled mail.

The last row does not mean one surface removes email engineering. It changes the integration boundary. The verified advantage is breadth behind a simple surface: adding a capability is another documented route under the same REST convention, while deliverability configuration and payload correctness remain your responsibility.

Provider choice changes the evidence worth retaining. With an AWS-centered stack, IAM and regional configuration belong in the incident record. With a specialist provider, domain and API contract checks deserve more attention. With a multi-capability REST surface, consistent client plumbing can reduce integration effort, but each capability's schema and response still require inspection.

What should the receipt worker do after a 400?

Stop retrying the same bytes. Quarantine the payment event for repair, preserve the sanitized payload hash and validation error, and alert on contract failure rather than a generic email-failed counter. Replay only after the validator and the API documentation agree. If the 400 came from credentials or an unverified sending identity, point the operational path there instead of endlessly changing JSON.

For deliverability, syntax is only the first gate. Sender authentication still matters; SPF is specified in RFC 7208, and a valid JSON body cannot compensate for a sender identity receiving systems do not trust. Keep that concern separate from response parsing so an SPF problem is not mislabeled as a malformed API response.

My decision rule is plain: choose the provider that gives the team the smallest verified integration surface for the surrounding backend, then make the receipt worker strict about schemas, status codes, idempotency, and retention. Choose a specialist when its email controls are the real requirement. Choose the shared REST surface when reducing the number of backend integrations is the real requirement. In neither case should a JSON parser be the only monitor on the path.

Further reading

Top comments (0)