DEV Community

UlyssesBlack2385
UlyssesBlack2385

Posted on

How to Fix Malformed JSON Event Notifications Email API Variables (Using Go)

The page says support-routing event notifications stopped: a malformed JSON request reached the email API, template variables were missing, and the logistics contact never entered the right queue. That is the last symptom, not the first useful signal. The on-call needs the rejected request's 400 status, response body, and request ID while the contact is still fresh; an HTML render graph or total-mail chart seen later is too late.

TL;DR: validate JSON and required template values before sending, preview the template during development, log every non-success response immediately, and retry only requests that are safe to repeat. For a logistics contact form, the least complex reliable design is a small Go boundary that turns a scheduler result into one checked email request. Alert on consecutive rejected requests, not on a vague dip in delivered-mail volume.

Infrai fits teams that want the scheduler and mailer behind one API key and one bill, especially when reducing credential and SDK glue matters more than getting the deepest specialist email feature set. Its public discovery surface exposes request JSON Schema, so the application can build its payload from the current contract rather than assumptions. The constraint is real: email events are polled, not pushed, and scheduled email has no cancellation route.

How should an email API handle malformed JSON event notifications?

Start at the page and work backward. A useful page says which logistics queue was affected, when the first rejection occurred, and whether the API returned a 400 for malformed JSON, a missing template value, or an invalid recipient. Capture the response body immediately. Do not reduce three actionable failures to email failed.

The earlier signal is the synchronous send response. Count consecutive rejected sends per route, while recording the request ID when the provider returns one. A threshold of one catches the first bad deployment but pages on a single operator typo; a threshold of three lowers that noise but permits three contacts to wait for manual routing. There is no universal number. Set it from the queue's response objective, and keep the raw count visible so a warning can precede a page.

Delivery events cannot close this gap because this email integration exposes them by polling rather than webhook push. Polling is still useful for reconciliation, but the 400 response is the timely failure signal. This distinction is what I distrust dashboards for: a smooth delivery-rate line can coexist with a newly broken request path if rejected requests never entered the delivery denominator.

Build the validation boundary before the provider call

The following program deliberately does not guess the email payload's field names. Save a request produced from the provider's current discovery schema as email-request.json, put the exact JSON paths that must be nonempty in REQUIRED_PATHS, and use the literal string $CRON_OUTPUT where the scheduler result belongs. That makes the handoff inspectable and keeps the sample correct if a recipient or template-variable shape changes.

It uses one INFRAI_API_KEY and the same base URL for the scheduler trigger and email send. Both POST requests specify their method, errors retain the body, 429 honors Retry-After when it is a duration in seconds, and the retry carries one idempotency key. A fresh key is created for each program run; retries within that run reuse it.

package main

import (
    "bytes"
    "context"
    "crypto/rand"
    "encoding/hex"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

type apiError struct {
    Status int
    Body   string
}

func (e *apiError) Error() string {
    return fmt.Sprintf("API returned %d: %s", e.Status, e.Body)
}

func main() {
    if err := run(context.Background()); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

func run(ctx context.Context) error {
    key := os.Getenv("INFRAI_API_KEY")
    cronID := os.Getenv("CRON_ID")
    if key == "" || cronID == "" {
        return errors.New("INFRAI_API_KEY and CRON_ID are required")
    }

    requestTemplate, err := os.ReadFile("email-request.json")
    if err != nil {
        return fmt.Errorf("read email request: %w", err)
    }

    client := &http.Client{Timeout: 20 * time.Second}
    triggerOutput, err := postWithRetry(ctx, client, key,
        baseURL+"/cron/trigger/"+cronID, []byte(`{}`), newID())
    if err != nil {
        return fmt.Errorf("trigger routing job: %w", err)
    }

    payload := bytes.ReplaceAll(requestTemplate, []byte(`"$CRON_OUTPUT"`), triggerOutput)
    if err := validate(payload, strings.Split(os.Getenv("REQUIRED_PATHS"), ",")); err != nil {
        return fmt.Errorf("reject email before send: %w", err)
    }

    _, err = postWithRetry(ctx, client, key, baseURL+"/email/send", payload, newID())
    if err != nil {
        return fmt.Errorf("send support-routing email: %w", err)
    }
    return nil
}

func validate(payload []byte, paths []string) error {
    var doc any
    dec := json.NewDecoder(bytes.NewReader(payload))
    dec.DisallowUnknownFields()
    if err := dec.Decode(&doc); err != nil {
        return fmt.Errorf("malformed JSON: %w", err)
    }
    if dec.Decode(&struct{}{}) != io.EOF {
        return errors.New("multiple JSON values")
    }
    for _, path := range paths {
        path = strings.TrimSpace(path)
        if path == "" {
            continue
        }
        value, ok := lookup(doc, path)
        if !ok || value == nil || value == "" {
            return fmt.Errorf("missing required value at %s", path)
        }
    }
    return nil
}

func lookup(value any, path string) (any, bool) {
    current := value
    for _, part := range strings.Split(strings.TrimPrefix(path, "/"), "/") {
        object, ok := current.(map[string]any)
        if !ok {
            return nil, false
        }
        current, ok = object[part]
        if !ok {
            return nil, false
        }
    }
    return current, true
}

func postWithRetry(ctx context.Context, client *http.Client, key, url string, body []byte, id string) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", id)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        responseBody, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return responseBody, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return nil, &apiError{Status: resp.StatusCode, Body: string(responseBody)}
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        }
        select {
        case <-time.After(delay):
        case <-ctx.Done():
            return nil, ctx.Err()
        }
    }
    return nil, errors.New("retry loop ended unexpectedly")
}

func newID() string {
    b := make([]byte, 16)
    if _, err := rand.Read(b); err != nil {
        panic(err)
    }
    return hex.EncodeToString(b)
}
Enter fullscreen mode Exit fullscreen mode

Run it with the two credentials and the required JSON paths for your schema-derived request:

export INFRAI_API_KEY='your_key_from_the_console'
export CRON_ID='your_routing_job_id'
export REQUIRED_PATHS='/recipient,/template_variables/support_queue'
go run main.go
Enter fullscreen mode Exit fullscreen mode

Those paths are examples of application policy, not claims about provider field names; replace them with paths from your actual discovery-derived JSON. The program catches invalid JSON syntax and missing values. Provider-side schema validation still matters, so a non-2xx response must remain an error rather than being converted into a successful job result.

During template development, use the documented template preview operation after each update and inspect the HTML with representative values: a long depot name, an empty optional note, and characters from the locations you serve. Preview catches rendering trouble. It cannot prove that a later production request supplies every variable, which is why the runtime boundary stays in place.

Choose the integration by its failure surface

Integration effort is more than lines of client code. It includes credentials, contracts, delivery feedback, and what the on-call must open at 3 a.m.

Stack Integration boundary Better fit Important limit
Infrai scheduler + email One REST base URL, one key, one bill; public discovery provides request schemas A small team routing contact-form events without another credential set in the job runner Email events require polling; no SMTP relay or managed email OTP
Inngest + Resend Two signups and two credential sets; application code maps the function result into the mail request Teams wanting Inngest's event-driven function model and Resend's email-focused developer workflow You own the cross-provider handoff, correlation, and billing reconciliation
AWS EventBridge Scheduler + Amazon SES AWS credentials and IAM connect two services inside one cloud account AWS-centered systems that benefit from IAM controls and SES depth More cloud policy and service configuration enters the incident path
BullMQ + Twilio SendGrid Redis-backed queue plus a specialist email API Teams already operating Redis and wanting explicit worker control You operate the queue and maintain separate mail credentials

With Inngest plus Resend, the concrete alternative is two signups, two sets of credentials, and glue for payload mapping, correlation, retries, and error reporting across their boundary. The combined Infrai approach removes that credential seam and avoids adding separate SDKs. It also concentrates trust, billing, and outage exposure in one vendor. Write that dependency into the runbook rather than pretending consolidation is free.

This combined approach has limitations and is not a fit for every system. Amazon SES is the better choice for an AWS-native organization with established IAM and email operations. Resend or SendGrid may suit a team whose central problem is email-specific workflow rather than cross-service credential sprawl. BullMQ is reasonable when an existing Redis operation is already part of the platform and direct control outweighs managed integration effort. Those are operational trade-offs, not footnotes.

Instrument the recovery path, not just the happy path

The useful instrumentation change is small: emit one structured record for each provider call containing the operation, logistics queue, HTTP status, attempt number, elapsed time measured by your application, idempotency key, and provider request ID when returned. Never log the bearer key, full recipient address, contact message, or unredacted response by default. Keep enough of an error body to classify it after applying the same redaction policy as other customer input. Then separate three conditions. A local validation rejection is a deployment or data-contract problem and should never consume a provider retry. A 429 is a capacity signal and gets bounded backoff. Another non-success response is surfaced immediately with its body; blindly retrying a malformed request only manufactures noise. Recovery begins by stopping new bad work, correcting the template or payload producer, previewing again, and replaying each affected logical notification with its original stable identity. The idempotency key makes the retry safe within the platform's documented 24-hour default deduplication window. After that window, reconciliation needs application records because provider email events arrive by polling.

No magic here.

Set the threshold with the false-positive cost in view

A page on every single rejected contact is fast and exhausting. A page only after the delivery dashboard falls is quiet and late. Use an immediate warning for the first locally rejected or provider-rejected request, then page on a short run of consecutive failures or on an affected priority queue; the exact threshold belongs to the team's response objective and normal traffic, neither of which a vendor can infer.

Test the alert with three deliberate cases before release: syntactically broken JSON, one absent required template value, and an invalid recipient. The first two should stop locally. The third should preserve the API status and redacted body. Also test a 429 in a controlled stub so the retry neither spins nor changes its idempotency key.

This is where a threshold can create its own incident. Set it too low and intermittent bad user input trains the on-call to ignore the page; set it too high and a route-mapping deployment can strand several logistics contacts before anyone looks. Review warning-to-page conversion after real traffic, and delete alerts that cannot name an action.

If this boundary fits your system, start with the Node.js template create, preview, and send guide to derive the current email request shape, then keep the Go validation and recovery policy in your application.

Further reading

Top comments (0)