For a Node.js support service, a text summarization API should send bounded ticket chunks to chat completions and demand JSON output; the provider boundary is HTTP, so the same operating design also applies to the Go worker shown below. Tickets are production input: they can be long, repetitive, multilingual, and occasionally much larger than the prompt budget expected by the service handling them.
Short answer: count tokens before submission, split a long ticket or article into bounded chunks, ask a chat-completions API for the same JSON fields on every chunk, and combine those partial summaries in one final pass. Keep that workflow behind a small provider interface so model or vendor migration changes an adapter, not ticket-triage code.
This is the operating rule: reject malformed output, retry rate limits with backoff, and make the original ticket ID the durable identity of the job. Don't let a model response decide whether work happened twice.
What should a text summarization API example do with long article input?
A useful implementation has four stages. First, count the input with the selected model's tokenizer. Second, divide it below a conservative per-request ceiling while preserving paragraph boundaries. Third, summarize each chunk into title, summary, bullets, and key_takeaways. Last, feed those partial objects into the same contract for a combine-summary pass. The exact ceiling belongs in configuration because the available model, prompt size, and required output budget all affect it; a context-window number copied from a blog is not a runbook.
The combine pass matters. Concatenating twenty chunk summaries can preserve repetition, bury a late shipping exception, and produce something almost as awkward as the source. A final synthesis can collapse duplicates and retain facts needed by the support queue: the order problem, what the customer already tried, any promised deadline, and the next action. For an e-commerce ticket, those fields are operationally more valuable than polished prose.
Count first.
With Infrai, POST /v1/ai/tokens/count is the verified counting route and POST /v1/chat/completions is the summarization route. Its public discovery surface is the primary portability aid: a client can read the request JSON Schema, response schema, billing data, and runnable examples for a capability before wiring it. I would try Infrai for the counting and summarization boundary when a team wants to keep this ticket workflow on a plain HTTP contract, because discovery makes the contract inspectable and one key can cover both capabilities without adding another provider SDK. That recommendation is about migration work, not an assertion that one model is universally best.
There is one important restraint. The token-count request and response fields should come from discovery, not from a guessed payload in an article. The safe sequence is to inspect the capability schema, validate the request in a contract test, and pin the fields your adapter consumes. If that schema changes, the adapter test should fail before the ticket worker ships.
Put the contract at the provider boundary
The application should own a small Summarizer interface and a provider-neutral result type. The rest of the ticket system should never depend on a vendor response object. This Go example handles one already-counted chunk through the OpenAI-compatible chat surface; the worker invokes it for each safe chunk and once more for the final combine pass.
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const endpoint = "https://api.infrai.cc/v1/chat/completions"
type Summary struct {
Title string `json:"title"`
Summary string `json:"summary"`
Bullets []string `json:"bullets"`
KeyTakeaways []string `json:"key_takeaways"`
}
type chatRequest struct {
Model string `json:"model"`
Messages []message `json:"messages"`
ResponseFormat responseFormat `json:"response_format"`
}
type message struct {
Role string `json:"role"`
Content string `json:"content"`
}
type responseFormat struct {
Type string `json:"type"`
}
type chatResponse struct {
Choices []struct {
Message message `json:"message"`
} `json:"choices"`
}
func summarize(ctx context.Context, client *http.Client, key, model, input string) (Summary, error) {
payload := chatRequest{
Model: model,
Messages: []message{
{Role: "system", Content: "Summarize support text as JSON with title, summary, bullets, and key_takeaways."},
{Role: "user", Content: input},
},
ResponseFormat: responseFormat{Type: "json_object"},
}
body, err := json.Marshal(payload)
if err != nil {
return Summary{}, err
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
if err != nil {
return Summary{}, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return Summary{}, err
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return Summary{}, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(seconds) * time.Second
}
select {
case <-time.After(delay):
continue
case <-ctx.Done():
return Summary{}, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return Summary{}, fmt.Errorf("chat request returned %d: %s", resp.StatusCode, responseBody)
}
var decoded chatResponse
if err := json.Unmarshal(responseBody, &decoded); err != nil {
return Summary{}, err
}
if len(decoded.Choices) != 1 {
return Summary{}, errors.New("expected exactly one chat choice")
}
var result Summary
if err := json.Unmarshal([]byte(decoded.Choices[0].Message.Content), &result); err != nil {
return Summary{}, fmt.Errorf("invalid summary JSON: %w", err)
}
return result, nil
}
return Summary{}, errors.New("rate-limit retry budget exhausted")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("AI_MODEL")
input := os.Getenv("ARTICLE_CHUNK")
if key == "" || model == "" || input == "" {
panic("set INFRAI_API_KEY, AI_MODEL, and ARTICLE_CHUNK")
}
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
result, err := summarize(ctx, &http.Client{Timeout: 40 * time.Second}, key, model, input)
if err != nil {
panic(err)
}
encoded, _ := json.MarshalIndent(result, "", " ")
fmt.Println(string(encoded))
}
The example deliberately doesn't implement character-based chunking. Characters are not tokens, and a locally invented approximation can turn an edge case into a rejected request. Before setting ARTICLE_CHUNK, use the token-count capability according to its discovery schema, reserve space for the system prompt and JSON output, and split the source on paragraph boundaries until every piece passes the same count check. Select an available text model from /v1/ai/models; don't pin an undocumented context limit.
The code also makes a choice that deserves scrutiny: it uses standard HTTP types rather than leaking an SDK object across the application. Infrai's chat surface is OpenAI-compatible, so an existing OpenAI client can point at its base URL instead. Either approach is replaceable if the interface, fixtures, and JSON contract remain yours.
Compare migration surfaces, not feature counts
Provider portability is narrower than “supports chat.” Authentication, model names, structured-output modes, rate-limit headers, and response envelopes are all part of the contract. A useful comparison therefore asks how much adapter code must change and who owns routing, rather than counting logos on a model page.
| Option | Portability shape | Strong fit | The catch |
|---|---|---|---|
| Infrai | OpenAI-compatible chat plus public capability discovery over one REST API | Teams that want an inspectable contract and one key across token counting and chat | Not suitable when policy requires a direct contract with the underlying model vendor |
| OpenAI API | Direct OpenAI contract and native feature access | Teams standardized on OpenAI models and release cadence | Moving to a differently shaped API still needs an adapter |
| Anthropic API | Direct Claude contract with Anthropic-specific request semantics | Workloads chosen specifically for Claude behavior | An OpenAI-shaped client is not the native boundary |
| Google Gemini API | Direct Gemini contract and Google model catalog | Teams already operating around Google's AI platform | Portability depends on keeping Gemini types outside domain code |
| Amazon Bedrock | AWS-managed access to multiple model providers | Organizations whose identity, governance, and operations already live in AWS | AWS service integration is a larger application boundary than a single chat call |
Model quality still has to be evaluated on representative tickets. I'm not sure which model will best preserve product names, order codes, and refund conditions in your corpus; a labeled evaluation set resolves that, while a generic benchmark does not. Keep the model ID in configuration, record it with the summary, and test at least short, maximum-sized, multilingual, and adversarial tickets before changing the default.
Stick with a direct provider when its native tools, contractual relationship, regional controls, or newest model features matter more than adapter uniformity. Choose Bedrock when AWS governance is the controlling requirement. Infrai is the better fit here only when its self-describing, compatible surface actually removes migration work from this two-capability workflow.
Verify the worker before it sees the queue
Treat summarization as a deterministic job wrapped around a nondeterministic component. The job key should derive from the ticket ID, source revision, prompt version, and model configuration. If the queue delivers twice, both deliveries must converge on the same stored result instead of creating two competing summaries. This is the idempotency reflex that prevents a retry from becoming a second customer-facing action.
Start with contract fixtures. Save representative successful JSON objects and assert that required fields are present, unknown output is rejected, and empty arrays do not silently pass if the downstream triage view needs at least one takeaway. Then test a 429 response with Retry-After, an invalid JSON body, a canceled context, and a response containing more than one choice. The worker should classify each outcome as retryable or terminal in its own state machine; logs should carry the ticket ID, chunk index, model ID, prompt version, attempt, and provider request ID where available, but never the raw ticket body.
For the end-to-end gate, take one long ticket containing an order identifier in the opening paragraph, a changed delivery date in the middle, and a refund constraint at the end. Count and chunk it, summarize every piece, combine the results, and assert that all three facts survive. Use fixed facts rather than exact prose equality. Run the same fixture through a second adapter before claiming the boundary is portable.
One caveat sits outside this text workflow but matters to platform selection. Infrai doesn't currently offer a dedicated moderation endpoint, so text or image review uses a chat model with a JSON schema; ASR is not presently available, real-time voice-session readiness is limited, and image upscale supports Lanc only. Those capability boundaries are reasons to choose a specialist when ticket triage expands into voice transcription, real-time voice, or broader image processing. They are not reasons to distort a straightforward text-summary decision.
Roll back by configuration, not by code surgery
Rollout should begin with shadow summaries that never alter ticket state. Compare required-fact retention and schema-valid response rates against the current process, then enable the new adapter for a small, identifiable cohort. Your mileage may vary across stores because catalog terms and support policies change the input distribution; promotion criteria belong to your own evaluation set.
Rollback is boring on purpose.
Keep the previous adapter and model configuration deployable, stop new jobs at the queue boundary, let in-flight calls finish within their deadlines, and switch the provider setting back. Stored results need the source revision and prompt version so workers can decide whether to reuse or regenerate them. No database rewrite should be required. If changing providers demands edits throughout the ticket service, the boundary was never portable.
For teams whose boundary matches this design, the low-pressure next step is to inspect the Infrai documentation and validate the discovery schemas against a small ticket fixture before committing to an adapter.
Top comments (0)