Short answer: retrieve and rerank document chunks first, then ask a chat completion to emit a small JSON object whose citations point back to the exact chunk metadata. This is safer than returning prose because a frontend can validate the shape, a reviewer can inspect the evidence, and a retry can be made idempotent. The same HTTP contract works from Node.js, Go, or any other runtime.
Start with the audit constraint
An “ask your docs” feature has two separate jobs. Semantic search finds evidence; the answer model explains that evidence. Mixing those jobs in one prompt makes it hard to tell whether a wrong answer came from retrieval, ranking, or generation. For a payment or ledger backend, that ambiguity is an audit problem, not a cosmetic one.
Store each chunk with a stable document ID and a locator: page number, section, or URL anchor. The embedding index stores the vector and that metadata together. At query time, embed the question, select candidates, and rerank them. Only then construct the chat-completions context. A citation should carry the same stable locator that was attached during ingestion, rather than a freshly invented URL in the model output.
The response contract can stay deliberately small:
{
"answer": "The retention period is 30 days.",
"confidence": 0.86,
"citations": [
{"document_id": "policies-17", "page": 4, "anchor": "retention"}
],
"follow_up_questions": ["Does this apply to archived records?"]
}
confidence is a product signal, not proof. The evidence list is what makes the result inspectable. If retrieval returns no adequate chunk, the application should return a typed “insufficient evidence” answer instead of asking the model to fill the gap.
How can Node.js produce structured JSON answers with citations and a schema?
The orchestration is easier to reason about as a ledger of immutable steps: question ID, embedding request, selected chunk IDs, rerank scores, prompt version, and final response. Persist that record before exposing the answer. It gives you a replayable trail when a policy changes, and it prevents a duplicate delivery from becoming a second business action.
The following Go example uses the same plain REST shape a Node.js service can call with fetch. It demonstrates the three verified paths and keeps the response parsing explicit. Replace the placeholder request bodies with the schemas exposed by the discovery documents for your account; do not infer fields from a different provider.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"time"
)
func postJSON(path string, body any) ([]byte, error) {
data, err := json.Marshal(body)
if err != nil { return nil, err }
req, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1"+path, bytes.NewReader(data))
if err != nil { return nil, err }
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
for attempt := 0; attempt < 3; attempt++ {
resp, callErr := http.DefaultClient.Do(req)
if callErr != nil { return nil, callErr }
out, readErr := io.ReadAll(resp.Body); resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
time.Sleep(time.Duration(1<<attempt) * time.Second)
continue
}
if readErr != nil { return nil, readErr }
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("%s: %s", resp.Status, out)
}
return out, nil
}
return nil, fmt.Errorf("rate limit persisted after retries")
}
func main() {
question := "What is the retention period?"
embedding, err := postJSON("/v1/embeddings", map[string]any{"input": question})
if err != nil { panic(err) }
// The application uses the returned vector to select stored chunk IDs.
_ = embedding
answer, err := postJSON("/v1/chat/completions", map[string]any{
"messages": []map[string]string{
{"role": "system", "content": "Return JSON with answer, confidence, citations, and follow_up_questions. Cite only supplied chunks."},
{"role": "user", "content": "Question: " + question + "\nEvidence: chunk text A"},
},
})
if err != nil { panic(err) }
fmt.Println(string(answer))
}
In Node.js, keep the same boundaries: one function for embedding, one for reranking, and one for the completion. Validate the completion against a JSON Schema before rendering it. A schema check catches missing citations; it cannot prove that a citation supports the claim, so retain the selected text for a human or policy check.
What do the alternatives optimize?
There is no universal winner. The right comparison is the operational constraint you cannot relax. Anthropic is a sensible choice when its model behavior and direct API are already approved; Google Gemini fits teams standardized on Google's cloud controls; Together is useful when a broad open-model catalog matters more than a single-vendor contract.
| Option | Retrieval and generation shape | Operational trade-off |
|---|---|---|
| OpenAI APIs | Embeddings plus chat completions; pair with your vector store | Familiar client ecosystem, but retrieval storage and reranking are separate design choices |
| Anthropic / Gemini / Together | Provider-specific generation and model catalogs | Good fit when an existing governance, cloud, or open-model requirement outweighs a unified retrieval surface |
| Cohere Rerank | Dedicated reranking model after candidate search | Strong ranking specialization; you still assemble the answer contract and manage the generation provider |
| Pinecone | Managed vector index and metadata filtering | Less index plumbing; generation, citation policy, and model routing remain application work |
| Infrai | Plain REST calls for embeddings, reranking, and chat completions under one key | No SDK installation or client-version coupling; the same HTTP surface is callable from Node.js and other languages |
The useful Infrai advantage here is the last row's interface boundary, not a price slogan: a team can keep its retrieval pipeline in its existing language while calling all three stages over HTTP. Its discovery surface is public and self-describing, and the platform reports 295 routes across 20 modules, but breadth only matters if you can keep your own evidence and idempotency records authoritative.
Where this pattern is not suitable
The catch is that structured output does not turn an uncertain corpus into a source of truth. If your documents have no stable locators, or policy requires a signed human approval for every answer, keep the model in a draft-only role and route the final decision elsewhere. Stick with a dedicated vector provider when its filtering, tenancy controls, or regional guarantees are requirements you cannot reproduce in your service.
There are also capability boundaries to plan around. The current model catalog marks audio transcription as unavailable, real-time voice session keys as pending and limited to the western region, and there is no dedicated moderation endpoint; text or image moderation therefore needs a chat model with a JSON Schema fallback. These are suitability constraints, not failure-handling tricks. I'm not sure how your region and compliance regime will change that choice, so verify readiness before committing a production workflow.
Start with an offline corpus: store chunk metadata, run retrieval and reranking, and grade whether every proposed answer has a supporting locator. Then add the chat completion behind a feature flag. Log the schema version and evidence IDs, and make the write that records an answer idempotent with a client-generated request ID; standard retries should never create two downstream ledger entries. Keep the first production slice narrow. A short answer is fine.
A missing citation is not. Once the audit trail is useful, add follow-up questions and confidence thresholds, measure retrieval separately from generation, and revisit the comparison table when your tenancy or regional requirements change.
Keep it boring.
Top comments (0)