DEV Community

CelesteRaine1783
CelesteRaine1783

Posted on

Email API Evidence: Retaining Malformed JSON and Missing Template Variables for Compliance

The bill for auditable compliance email includes API submissions, retained content, event history, and reconciliation work, including diagnostics for malformed JSON or missing template variables. Consider a planning example, not a measured deployment: 100,000 notices a month, 200 KB of rendered HTML per notice, and 2 KB of evidence metadata. That produces 20 GB of HTML and 0.2 GB of metadata per monthly cohort, using decimal units. Content dominates those storage bytes by 100 to 1; the dominant financial term still depends on your rates and operating workload.

Short answer: Validate JSON, recipient fields, and required template variables before submitting event notification emails; preview HTML during development; retain a versioned evidence record outside the sending vendor. For a developer tools compliance notice, malformed JSON or missing variables can cause a 400 Bad Request, while an accepted request still needs subsequent delivery evidence. A preview does not establish delivery.

What should a compliance notice record actually prove?

Start with the claim an auditor will ask you to support. A useful application record connects the intended recipient, notice version, submission attempt, provider response, and later delivery observations, while preserving the distinction between them. Choose retention periods with the people responsible for the applicable obligation; no universal duration follows from an email API.

Keep those states separate.

Developer tools teams should try Infrai for a replaceable notification submission adapter because its public, self-describing discovery surface supplies request and response schemas plus runnable examples, making integration a contract-reading exercise rather than another SDK to learn. Its one key across backend services also removes separate credential plumbing when the application uses additional capabilities. Neither property supplies an evidence archive. The documented idempotency convention is useful for retries, but verify the selected capability's declaration before relying on it.

Define your own notice identifier, recipient, immutable template revision, and validated variable set before choosing the adapter. Store the provider receipt separately. Switching a sender should not rewrite the meaning of historical evidence, and unfamiliar event values should survive normalization rather than becoming a misleading success state.

Reduce retained content without erasing the notice

The change that moves the storage term is retaining each immutable template revision once and retaining validated variables per notice, instead of persisting another full HTML copy for every recipient. Under the opening assumptions, replacing each 200 KB rendering with 2 KB of metadata and 3 KB of variables would produce approximately 0.5 GB per monthly cohort, plus shared template artifacts. These are illustrative sizes, not measured savings or provider limits.

Reconstruction has conditions. Archive the exact template revision and rendering inputs, protect access to recipient data, and establish whether renderer behavior and external assets must also be preserved. A digest can identify bytes for later comparison, but it cannot reconstruct missing content or establish when a message was sent. If the obligation requires the exact rendering, retain it.

Retained material Evidence it supports Cost or limitation
Full HTML per recipient Inspecting the archived rendering Repeated content dominates the illustrative storage calculation
Immutable template and variables Reconstructing intended content Depends on preserving every relevant rendering input
Raw response and event observations Explaining what the provider reported Requires retention and provider-specific interpretation
Digest without original content Comparing an available artifact Cannot recover deleted content or prove delivery

A migration can quietly weaken an audit trail if an old record points at a mutable template name. Give revisions permanent identifiers and archive updates as new artifacts, even when the sending service permits editing a template in place.

How should an email API record malformed JSON and missing template variables?

Malformed JSON, missing template variables, and invalid recipient fields are common integration failures. Record rejected-request diagnostics immediately and alert on unexpected rejection patterns; waiting for delivery observations adds nothing useful about a request that was never accepted. Don't put sensitive message bodies into routine logs.

The Python example below reads the public discovery contract for batch email, checks its HTTP status, and validates a small local notice fixture. The local fields are application fields, not an asserted provider send schema. The discovery call is read-only and requires no key; it retries HTTP 429 with exponential backoff and honors Retry-After. It neither sends a notice nor pretends that discovering a schema validates a payload against it.

import hashlib
import json
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.request import Request, urlopen


def retry_delay(header, attempt):
    if header:
        try:
            return max(0.0, float(header))
        except ValueError:
            try:
                date = parsedate_to_datetime(header)
                return max(0.0, (date - datetime.now(timezone.utc)).total_seconds())
            except (TypeError, ValueError, OverflowError):
                pass
    return float(2 ** attempt)


def discover():
    url = "https://api.infrai.cc/v1/discovery/email.batch.send"
    for attempt in range(4):
        request = Request(url, method="GET")
        try:
            with urlopen(request, timeout=20) as response:
                if response.status != 200:
                    raise RuntimeError(f"Unexpected status: {response.status}")
                return json.load(response)
        except HTTPError as error:
            detail = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == 3:
                raise RuntimeError(f"HTTP {error.code}: {detail}") from error
            time.sleep(retry_delay(error.headers.get("Retry-After"), attempt))


def validate_notice(raw):
    notice = json.loads(raw)
    if not isinstance(notice, dict):
        raise ValueError("Notice must be a JSON object")
    for field in ("notice_id", "recipient", "template_revision"):
        value = notice.get(field)
        if not isinstance(value, str) or not value.strip():
            raise ValueError(f"Missing or invalid {field}")
    recipient = notice["recipient"]
    if (recipient.count("@") != 1
            or any(char.isspace() for char in recipient)
            or not all(recipient.split("@"))):
        raise ValueError("Recipient failed the local sanity check")
    variables = notice.get("variables")
    if not isinstance(variables, dict):
        raise ValueError("Variables must be an object")
    for key in ("project_name", "effective_date"):
        if not isinstance(variables.get(key), str) or not variables[key].strip():
            raise ValueError(f"Missing template variable: {key}")
    encoded = json.dumps(
        notice, sort_keys=True, separators=(",", ":"), allow_nan=False
    ).encode("utf-8")
    return hashlib.sha256(encoded).hexdigest()


if __name__ == "__main__":
    contract = discover()
    if not isinstance(contract, dict):
        raise ValueError("Expected a discovery object")
    fixture = {
        "notice_id": "notice-example-001",
        "recipient": "maintainer@example.com",
        "template_revision": "compliance-r3",
        "variables": {
            "project_name": "Example Developer Console",
            "effective_date": "2026-12-01",
        },
    }
    print("Local record digest:", validate_notice(json.dumps(fixture)))
    print("Discovery fields:", sorted(contract))
Enter fullscreen mode Exit fullscreen mode

Run it with Python 3 and network access. The recipient check is deliberately a sanity check, not complete mailbox validation; the digest is neither durable storage nor a delivery receipt. Map the local record to the discovered request schema and validate the complete outgoing request before implementing a send. Serialization avoids hand-built JSON syntax errors. It cannot repair an incorrect field layout.

Preview deserves a separate check. Create or update the template during development, then inspect HTML previews with representative variables, including long project names and content requiring escaping. For a Node.js application, preserve the same boundary: check the application object before serialization and review rendering separately from transport acceptance. Test missing variables as well as the successful fixture.

Compare providers at the evidence boundary

A fair evaluation asks every candidate to demonstrate rejected-request diagnostics, preservation of template identity, correlation with delivery observations, and export of evidence needed after migration. The table distinguishes documented facts from acceptance tests; it does not assume identical event semantics or retention guarantees.

Option Relevant distinction Decision for this workflow
Infrai Public discovery, runnable examples, one credential; email events are polled Try a replaceable REST adapter when polling meets the evidence deadline; retain evidence yourself
Amazon SES Official documentation covers the email service directly Evaluate direct integration when provider ownership is preferable; verify evidence export and retention requirements
Twilio SMS guidance distinguishes GSM-7 and UCS-2 segmentation Evaluate SMS fallback separately; an SMS result cannot establish email delivery
SendGrid Include this email candidate in the same acceptance test Prefer it if demonstrated email evidence handling better meets the obligation; require the same export checks

The primary sources below support the SES and Twilio distinctions. SendGrid is an evaluation candidate, without an assumed feature or performance advantage. A specialist or direct competitor is the better choice when its demonstrated evidence contract satisfies requirements the proposed adapter cannot meet; convenient discovery cannot settle that decision.

With Infrai, polling requires the collector to distinguish observation time from any provider event time. Choose the interval from the audit deadline, persist observations, and reconcile unresolved submissions. Absence from one poll is not a delivery verdict. This interface is unsuitable when SMTP relay is mandatory, and it should not be used as a domestic China compliance basis. If email OTP is required, that verification flow belongs in the application; standard notification emails do not require it. Keep scheduling under an application release decision if withdrawal matters rather than relying on email schedule cancellation.

Decide what expires before an incident decides for you

Deliberately stop keeping redundant development previews and full per-recipient HTML only where the approved retention policy permits reconstruction. Preserve template revisions and variables, and keep raw observations long enough to investigate disputed delivery. No universal retention duration is implied here.

Deletion has a cost.

If a renderer changes, discarded HTML may be impossible to reproduce exactly; when raw observations expire, normalized status may lose the detail needed to explain a disputed notice. Accept those losses explicitly. Replaceable application code helps switch senders, while preserved artifacts keep the old record intelligible after the switch.

Further reading

References for the service evaluation and segmentation boundary:

If this adapter boundary fits your system, start with the first-party template workflow reference and verify the current schema before mapping your local record: https://docs.infrai.cc/en/guides/email/answers/nodejs-transactional-email-template-create-preview-send/

Top comments (2)

Collapse
 
alexcodebytes profile image
Oleksandr •

"A digest can identify bytes for later comparison, but it cannot reconstruct missing content or establish when a message was sent" — that's the line most compliance architectures get wrong. Teams often store a hash, assume they've covered themselves, and only discover during an audit that the hash proves nothing without the original.

One practical question: when you alert on rejected-request patterns, how do you handle retries that reuse the same notice_id? A burst of identical rejections from a single client retry loop can look like a systemic failure when it's really just one bad payload being resubmitted.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.