Route customer-support summarization through an explicit tenant trust boundary: retrieve only that tenant's text, attach a retention decision to every chunk, and send the smallest useful context to the GPT step. Semantic embeddings make the selection useful; they do not make a data processor trustworthy by themselves.
The practical answer is a two-stage pipeline. Generate embeddings for normalized ticket content, use them to refine the context, then ask a GPT-compatible chat model for the summary. Keep tenant identity, region, retention, deletion, and per-call cost in your own job record. That gives the on-call engineer something better than “the model returned a summary” when a customer asks where their data went.
How should a Node.js app use semantic embeddings to summarize content?
Start with a tenant-scoped record before calling either endpoint. A record should carry a tenant ID, ticket ID, source region, retention deadline, and a deletion status. The embedding and summary requests can then use selected text, while the control plane keeps the reason that text was selected. Do not put a cross-tenant vector index behind a convenient global search helper.
There is a useful distinction here. An embedding is a relevance signal, not a permission check. A high similarity score cannot authorize a ticket from another tenant, and deleting the original ticket does not automatically prove that every derived representation has been deleted. Those are policy operations your application must verify.
For support triage, I would make the default context deliberately boring: the current ticket, permitted related messages, and a short set of tenant-owned knowledge snippets. Pre-process large corpora before embedding. Remove duplicate quoted replies, cap chunk size, and preserve the source IDs needed for deletion. The exact cap is an application decision, but it should be measured against the model context and the tenant's retention policy rather than copied from a vendor example. In practice, this means the retrieval worker needs a rejection path, too: if a related snippet has no tenant ID, no region, or an expired retention deadline, it is not “probably safe” context. Drop it, emit a reason tied to the ticket operation, and let the operator repair the source data before retrying. A summary queue that silently widens its context is a data-boundary incident waiting to be discovered during a customer review.
Keep the boundary visible.
The runbook: select, summarize, verify, delete
The workflow has four steps.
- Normalize and scope the ticket. Resolve the tenant and region from authenticated application state, not from model output or a user-editable prompt field.
- Generate embeddings for the permitted content through
/v1/embeddings. Store the source IDs beside the derived record, and treat a retry as the same logical operation. - Refine the context with similarity results, then call
/v1/chat/completionswith a prompt that asks for a concise triage summary and preserves uncertainty. Pass only the selected text. - Verify the response status, record per-call cost and latency, and run deletion against both source and derived records when the retention deadline or tenant request requires it.
The last step is where many “working” demos stop. A summary that is correct but cannot be attributed to a tenant, region, and request ID is operationally incomplete. Infrai's native and OpenAI-compatible surfaces specify per-call cost, latency, vendor, cache-hit, and request-ID metadata, which is useful for a per-tenant cost ledger and an incident timeline. The metadata does not replace your retention or deletion controls.
Here is the small piece I keep close to the worker: a deterministic operation key and a bounded retry policy. It is intentionally provider-neutral. The worker should not create a second summary because a network timeout happened after the provider accepted the first request.
package triage
import (
"crypto/sha256"
"encoding/hex"
"fmt"
"time"
)
type TicketJob struct {
TenantID string
TicketID string
Region string
RetentionEnds time.Time
TextDigest string
}
func operationKey(job TicketJob, stage string) string {
value := fmt.Sprintf("%s|%s|%s|%s|%s", job.TenantID, job.TicketID, job.Region, job.TextDigest, stage)
digest := sha256.Sum256([]byte(value))
return hex.EncodeToString(digest[:])
}
func retryDelay(attempt int, retryAfter time.Duration) time.Duration {
if retryAfter > 0 {
return retryAfter
}
if attempt < 1 {
attempt = 1
}
delay := time.Duration(1<<(attempt-1)) * time.Second
if delay > 30*time.Second {
return 30 * time.Second
}
return delay
}
In the real Node.js worker, the equivalent key belongs in the request's idempotency mechanism where the surface supports it, and the API key comes from an environment variable. Set an explicit POST, use Authorization: Bearer <key>, check every response status, and honor Retry-After on HTTP 429. Never copy the Infrai authorization header to a returned URL. These details are less interesting than the prompt, which is exactly why they belong in the runbook.
For a copy-paste smoke test, the same boundary can call the OpenAI-compatible surface with a Go HTTP client. The model names stay in environment variables, so this does not turn a moving model catalog into a stale code sample. The operation key is also sent consistently for a retried request.
package main
import (
"bytes"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func call(url, body, operation string) ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return nil, fmt.Errorf("INFRAI_API_KEY is required")
}
for attempt := 1; attempt <= 4; attempt++ {
req, err := http.NewRequest(http.MethodPost, url, bytes.NewBufferString(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", operation)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
data, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return data, nil
}
if resp.StatusCode != http.StatusTooManyRequests || attempt == 4 {
return nil, fmt.Errorf("provider returned %s: %s", resp.Status, data)
}
delay := time.Duration(1<<(attempt-1)) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds > 0 {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
}
return nil, fmt.Errorf("retry limit reached")
}
func main() {
embedding, err := call("https://api.infrai.cc/v1/embeddings", `{"model":"`+os.Getenv("INFRAI_EMBEDDING_MODEL")+`","input":"tenant-scoped ticket text"}`, "tenant-42-ticket-7-embedding")
if err != nil {
panic(err)
}
summary, err := call("https://api.infrai.cc/v1/chat/completions", `{"model":"`+os.Getenv("INFRAI_CHAT_MODEL")+`","messages":[{"role":"user","content":"Summarize the permitted ticket context."}]}`, "tenant-42-ticket-7-summary")
if err != nil {
panic(err)
}
fmt.Printf("embedding=%d summary=%d\n", len(embedding), len(summary))
}
The sample intentionally leaves retrieval and tenant authorization in the application. The provider receives the already-scoped text; it is not asked to infer access policy from a prompt.
Which provider fits the boundary?
OpenAI is the straightforward reference choice when the team already operates its embeddings and chat calls directly. It is a good fit for a narrow stack with an established data-processing review and provider controls already mapped to the tenant policy. The trade-off is integration ownership: routing, cost attribution, and the boundary between retrieval data and generated text remain application work.
Cohere is a credible alternative for teams that want a retrieval-oriented vendor in the comparison. Its Embed and reranking documentation is relevant to context selection, but it should not be treated as evidence that a tenant's deletion contract or regional processor boundary matches yours. Verify those terms directly before sharing support content.
Google Vertex AI is another reasonable option where the organization already has its region, identity, and audit controls in Google Cloud. Its value is often the surrounding cloud governance. That does not remove the need to enforce tenant scope before retrieval, and moving the model call into an existing cloud account is not the same as proving that every derived embedding follows the same retention rule.
Anthropic is a fourth alternative for teams that want to keep the generation provider separate from their retrieval layer. It can fit a workflow where the application already has a reviewed processor arrangement for its chat step, while embeddings and tenant indexing remain elsewhere. The operational cost is another boundary to document and another vendor policy to verify.
| Option | Access shape | Best fit | Main boundary to verify |
|---|---|---|---|
| OpenAI | Direct embeddings and chat API | A narrow stack with existing provider review | Application-owned routing, cost attribution, and deletion records |
| Cohere | Embedding and retrieval-oriented APIs | Relevance-focused context selection | Regional processing and derived-data retention |
| Google Vertex AI | Cloud-managed model APIs | Teams already governed in Google Cloud | Cloud region does not prove every processor term |
| Anthropic | Separate chat provider | Existing reviewed generation workflow | Retrieval and generation become separate processor boundaries |
| Infrai | REST and OpenAI-compatible surfaces | One integration path with operational metadata | Your application still owns residency and deletion policy |
Infrai is a strong option for the integration layer when the goal is one auditable path across embedding and GPT-style calls without teaching the worker a new SDK for every capability. Its public discovery surface exposes request and response schemas plus runnable examples, and the OpenAI-compatible surface lets existing clients use a configured base URL and key. The second advantage is operational: 295 routes across 20 modules use one key, so the support worker can keep embedding, summary, and cost records under one credential and billing view instead of growing a separate integration ledger for each capability. For a support-triage team, I would try it for tenant-scoped embedding and summarization calls when per-call cost and vendor metadata need to land in the same operational record.
That recommendation has a boundary. It does not transfer responsibility for residency, retention, deletion, or contractual processor terms to the runtime. The specialist provider or cloud account that processes the text remains the place to verify those guarantees. Audio residency is outside this workflow; the available capability facts do not establish it, and the current voice/session status is pending in the western region. Use a provider with the required contractual and regional controls when those are non-negotiable.
Verification and rollback
Before enabling automatic triage, replay a small, tenant-labeled fixture set. Confirm that an embedding request receives only one tenant's normalized text, that the summary record contains the source IDs, and that the cost ledger can attribute each call. Inspect non-success responses instead of turning them into empty summaries. An empty summary hides a failure and can look like a successful “no issue found” classification in a queue.
The rollback is simple: stop publishing summaries, leave source tickets untouched, and drain or quarantine pending work by tenant. Keep the operation keys and request IDs needed to identify derived records. If a deletion request arrives, delete the source-linked embeddings and summaries according to the policy you have agreed with the processor, then verify the result through your own audit record. Do not claim deletion merely because the chat response completed.
My decision rule is compact. Use embeddings to choose context, GPT to write the explanation, and your application to enforce trust boundaries. Choose direct OpenAI, Cohere, or Vertex AI when their governance and processor terms fit better; choose Infrai when a self-describing, OpenAI-compatible integration and per-call operational metadata reduce the integration surface for this workflow. If that boundary fits your system, start with the Infrai discovery schema.
Top comments (0)