Sales Call Automation: Testing In-App Chatbot API Context and JSON Mode

#ai #backend #go

The least complex reliable design is a server-side adapter that accepts one versioned action schema, validates every model response before touching the CRM, and records enough evidence to replay the decision. Pick an AI API only after the same sales-call fixtures pass that contract; advertised pricing, context-window size, and JSON mode are inputs to the test, not the verdict.

Short answer: for an in-app chatbot that turns sales calls into CRM actions, the best API is the one that produces valid, grounded actions under your actual transcripts and retry policy at an acceptable measured cost. No provider label answers that question by itself.

This ordering matters because a fluent summary is reversible, while a fabricated follow-up date or an incorrectly assigned owner can escape into a system of record. The runtime therefore needs the temperament of a ledger: deterministic validation, idempotent writes, explicit state transitions, and an audit trail that distinguishes what the model proposed from what the application committed.

How should a Node.js in-app chatbot test API pricing, context windows, and JSON mode?

Start with a frozen evaluation set, even if the production caller happens to be Node.js. The language of the client library has little bearing on semantic correctness; what matters is that every candidate receives equivalent instructions, transcript content, schema, and retry treatment. Include ordinary calls, long monologues, corrections made late in a call, several speakers with similar names, and adversarial phrases such as "ignore the CRM rules." Redact sensitive material before the fixtures enter a development workflow, and retain only what the organization's compliance policy permits.

Define success at three layers. Transport success means the request completed and, when streaming is used, the event stream ended cleanly. Contract success means the assembled response parses as JSON and satisfies the exact schema. Business success means each proposed action is supported by transcript evidence and passes authorization rules. A native JSON mode may improve contract success, but it cannot prove that an amount, owner, or deadline came from the call.

Measure pricing against the whole workload rather than a displayed token rate. For each fixture, record input tokens, output tokens, retries, failed validations, and any context-reduction pass. Then calculate expected cost from the candidate's current published rates outside the application code. I wouldn't freeze those rates in a long-lived architectural decision record because they can change; store the dated rate card beside the evaluation output instead.

The context window deserves the same treatment. A larger window is useful only if the model can reliably identify the evidence that matters within it, and a long transcript plus system instructions, schema, prior turns, and output allowance all consume capacity. One big request can also make deletion, regional processing, and retention obligations harder to reason about. The practical comparison is the longest transcript your policy allows, not the largest number on a model page.

The failure boundary belongs before the CRM

A sales assistant should produce a proposal, never an immediate side effect. The proposal can contain a call summary and a bounded set of commands such as creating a follow-up task or suggesting a stage change, but a deterministic service must validate those commands, resolve CRM identifiers, enforce tenant authorization, and decide whether human approval is required.

Validate first.

Keep three records with different meanings: the normalized input envelope, the raw or policy-redacted model response, and the validated action proposal. Add a schema version, prompt version, model configuration identifier, request ID, and a digest of the relevant transcript. This is the minimum shape needed to explain why an action appeared and to replay the inference path after a prompt or provider migration. It is also where compliance limits bite: an audit trail is not permission to retain raw calls indefinitely, so retention and access controls must be set from the applicable contractual and regulatory obligations rather than copied from an observability default.

Exactly once is a business invariant here, not a property an AI API supplies. Assign each proposed action an idempotency key derived from the tenant, call, schema version, and stable action identity; insert that key into a local outbox under a uniqueness constraint; and let a worker apply the CRM mutation. A retry may repeat inference or delivery, but it must not create a second task.

That's the line.

Use explicit application errors. A malformed or schema-invalid proposal can become 422 CONTRACT_VIOLATION; missing transcript evidence can become 422 UNGROUNDED_ACTION; a repeated commit with the same idempotency key should return the previously recorded result. These are design examples, not claims about any vendor's response codes, and keeping them in your own adapter prevents external error taxonomies from leaking into product behavior.

Make the action contract narrower than the conversation

The chat transcript may be open-ended; the write path should not be. A discriminated action set, strict field limits, and transcript evidence offsets turn an ambiguous completion into a reviewable proposal. Unknown fields should fail validation because silently accepting them makes schema evolution impossible to audit.

package actions

import (
    "crypto/sha256"
    "encoding/hex"
    "errors"
    "fmt"
)

type Evidence struct {
    Start int `json:"start"`
    End   int `json:"end"`
}

type Action struct {
    Kind     string     `json:"kind"`
    Subject  string     `json:"subject"`
    Evidence []Evidence `json:"evidence"`
}

type Proposal struct {
    SchemaVersion string   `json:"schema_version"`
    Summary       string   `json:"summary"`
    Actions       []Action `json:"actions"`
}

func (p Proposal) Validate(transcriptBytes int) error {
    if p.SchemaVersion != "crm-actions.v1" {
        return errors.New("unsupported schema version")
    }
    for i, action := range p.Actions {
        if action.Kind != "create_follow_up" && action.Kind != "suggest_stage_change" {
            return fmt.Errorf("action %d has an unsupported kind", i)
        }
        if action.Subject == "" || len(action.Evidence) == 0 {
            return fmt.Errorf("action %d lacks a subject or evidence", i)
        }
        for _, span := range action.Evidence {
            if span.Start < 0 || span.End <= span.Start || span.End > transcriptBytes {
                return fmt.Errorf("action %d has an invalid evidence span", i)
            }
        }
    }
    return nil
}

func IdempotencyKey(tenantID, callID, actionIdentity string) string {
    material := tenantID + "\x00" + callID + "\x00crm-actions.v1\x00" + actionIdentity
    sum := sha256.Sum256([]byte(material))
    return hex.EncodeToString(sum[:])
}

This code deliberately stops short of calling a model or CRM. Provider adapters should translate the common request into each external API and translate the result back into Proposal; the validator and outbox should remain independent. Keep raw decoding strict as well, using json.Decoder.DisallowUnknownFields in Go, and reject trailing values after the first JSON document.

Evidence offsets need a declared coordinate system. Byte offsets work conveniently with stored UTF-8 payloads, while user interfaces often count Unicode code points or UTF-16 units; choose one, version it, and convert only at boundaries. Otherwise an apparently correct citation can highlight the wrong phrase for non-ASCII customer names, weakening the audit record precisely when a reviewer needs it.

Streaming complicates assembly but should not change the contract. Server-Sent Events provide a one-way server-to-client channel with text/event-stream; the browser-facing endpoint can use that mechanism for progress or assistant text, while the backend buffers the structured action payload and validates it only after the terminal event. Don't execute a CRM action from a partial token stream.

Compare behavior, not provider names

A useful scorecard separates hard gates from optimization metrics. Valid JSON is a hard gate. Supported action kinds, evidence bounds, tenant authorization, and idempotent commit behavior are hard gates too. Median latency, tail latency, token use, retry frequency, and evaluated cost are optimization metrics after correctness passes.

Decision area	Test artifact	Pass rule	Trade-off to inspect
Structured output	Versioned schema plus strict decoder	Every accepted response validates	JSON controls cannot establish grounding
Grounding	Labeled evidence spans	Every action maps to transcript evidence	Stricter rules may increase human review
Context	Long and noisy transcripts	Required facts survive in approved input	Larger windows can increase exposure and cost
Operations	Forced timeouts and duplicate deliveries	Retries never duplicate CRM effects	Durable state adds operational work
Economics	Dated run with token and retry totals	Budget holds at expected volume	Low rates can be offset by retries
Portability	Common adapter conformance suite	A second adapter passes unchanged	A gateway adds another control plane

Direct vendor integration is reasonable when one API meets the contract, the team values provider-specific controls, and migration is unlikely enough to accept adapter work later. A self-hosted gateway is another architectural option when a team needs a common interface across multiple model providers, but the catch is ownership — the team must deploy, secure, observe, and upgrade an additional service. It is not suitable when operational simplicity outweighs portability. Stick with a direct adapter when the gateway would be the least understood component in the request path.

I'm not sure a universal weighting for latency versus review rate would be defensible; the answer depends on whether the CRM action is merely suggested, queued for approval, or allowed to commit automatically. Resolve that uncertainty by assigning error costs with sales operations and compliance stakeholders before scoring candidates. A wrong automatic stage change should carry a different penalty from a summary that arrives two seconds later.

Roll out through replay and shadow traffic

Begin with an offline conformance suite and pin every variable that the external API permits: prompt text, schema version, decoding controls, and adapter version. Record results by fixture rather than collapsing them immediately into one average, because a candidate that handles short English calls perfectly but fails long multilingual corrections has a deployment boundary worth seeing.

Next, run shadow inference on policy-approved production inputs without exposing responses or writing CRM state. Compare proposed actions with the active path, investigate disagreements, and promote only after the new adapter clears the hard gates. Roll out by tenant cohort, retain a kill switch at the adapter boundary, and reconcile the outbox against CRM receipts.

Then commit.

Migration should be boring.

The domain service keeps emitting the same inference request and consuming the same validated proposal; only the adapter changes. Re-run old fixtures whenever a model, prompt, schema, rate card, or gateway version changes, because a provider choice is a revisable operational decision rather than a permanent property of the product.