KnutBerg8412

Posted on Oct 18

Node.js Embed Query for Top-K Vector Search with Metadata Filter

#node #rag #sre

In Node.js, embed the help-center query first, then run vector search with top k and a metadata filter for the tenant in the same request. A healthtech page changes at 02:13, but the alert presented to the on-call engineer should identify the changed passage, the affected tenant, and the prior text that made the change meaningful. A page-level hash can say that something moved; it cannot say whether the edit changes a medication-support workflow or merely corrects punctuation.

TL;DR: embed the incoming query with the same embedding model used for the stored page chunks, then send that vector, the collection, a top-k limit, and the tenant constraint in one vector-search call. Do not fetch globally and filter afterward. For change detection, keep chunk identity stable and attach freshness metadata so the alert can distinguish a new policy passage from an old near-duplicate.

This is where the Node.js boundary should stay boring: the application owns chunking, model consistency, tenant selection, and alert policy; the vector service owns nearest-neighbor retrieval. If the provider behind that capability changes while the contract stays fixed, the alerting code should not need a rewrite. Infrai is one option for that boundary, alongside Pinecone, Weaviate, Qdrant, and PostgreSQL with pgvector.

How should Node.js embed a query for vector search with metadata?

Work backward from the notification. By the time a generic page changed event reaches an on-call channel, the useful evidence has already been discarded. The earlier signal should have been a semantic difference between the newly scraped chunks and the tenant's previously indexed chunks, evaluated only after the new content is available and tagged with its source URL, tenant, crawl time, and stable chunk key.

The retrieval sequence is short. Normalize the user's help-center query. Compute its embedding with the same model and model version used for document chunks. Submit that vector to the relevant collection with the tenant constraint in the same request, asking for top k. Then compare the returned passages with the fresh page chunks before deciding whether an alert is actionable. The API does not compute the query vector during vector query; the caller supplies it.

Model symmetry is non-negotiable. Vectors produced by different embedding models do not share a meaningful distance space, even when their dimensions happen to match. Treat the embedding model identifier as index metadata and reject a query when it differs from the collection's recorded model. That guard belongs before the network call because a successful response with meaningless rankings is worse than a loud failure.

Model drift breaks retrieval.

Tenant isolation is equally strict. Applying tenant_id after retrieval spends the top-k budget on records the caller cannot use and can expose cross-tenant metadata to application code. The filter travels with the vector query. Always.

Filter before ranking.

Instrument the retrieval path, not just the scraper

A scrape-success counter proves only that bytes arrived. The alert path needs enough telemetry to explain why a change was or was not surfaced: collection name, embedding model identifier, tenant identifier in a privacy-safe representation, requested k, returned result count, source crawl timestamp, and the age of the newest indexed chunk. Record correlation identifiers across scrape, chunk, embed, index, query, diff, and notify stages. Do not record raw health content in logs.

Freshness is observable.

I would define two SLOs separately. The freshness SLO covers the time from a source page becoming available to its chunks becoming queryable. The alert SLO covers the time from queryable evidence to notification. Combining them into one number hides whether the crawler, embedding step, index, or notifier consumed the budget.

Capacity planning starts with chunk churn, not page count. A single frequently edited page can produce more embedding and upsert work than a large stable help center, and overlap multiplies that work. Track chunks created per crawl, unchanged-chunk reuse, embedding calls, indexed bytes, and query volume per tenant. Those measurements tell you when a smaller chunk improves retrieval but makes freshness harder to sustain.

Page count misleads.

No invented threshold belongs here. Establish the freshness objective from the clinical and operational impact of a missed change, then load-test at the expected peak edit rate and observe the actual lag distribution. The evidence should set the alert threshold.

Chunking determines whether the diff is useful

Page-sized vectors are cheap to reason about and poor at localization. Tiny fixed windows localize edits, but they sever headings from the procedures they qualify and create a noisy set of near-duplicates. For a help center, start with structure-aware chunks: preserve a heading with its following paragraphs, keep lists intact, and assign a deterministic key derived from the page identity and structural position. Re-embed only changed chunks.

There is a trap in aggressive overlap. It can make retrieval look better in a small relevance review because the same sentence appears in several candidates. In production, those duplicates occupy top-k slots and can turn one harmless wording edit into several apparent changes. Deduplicate by stable chunk lineage before alert scoring, while retaining enough overlap to keep a warning next to the instruction it modifies.

Freshness metadata should answer operational questions rather than decorate the record: when was this source fetched, which content revision produced the chunk, and which embedding model produced the vector? Metadata filtering is useful for hard boundaries such as tenant scope. Freshness ranking or diff policy remains application logic unless the selected service explicitly documents otherwise.

The false-negative risk grows when chunks are too broad; the false-positive load grows when chunks are too narrow or heavily overlapped. That is the real tuning axis.

Measure both.

Buy or build the vector boundary?

The products below can all participate in this design, but their operating models are different. The fair comparison is not a feature-count contest; it is a decision about control, on-call ownership, portability, and how directly each option expresses tenant-filtered top-k retrieval.

Option	Operating model	Fit for this alert path	Boundary to examine
Pinecone	Managed vector database	Teams that want a managed service with metadata filtering	Validate namespace and metadata-filter semantics against the tenant model
Weaviate	Managed cloud or self-managed	Teams that want vector search with a broader database surface	Decide whether its schema and operational surface are acceptable dependencies
Qdrant	Managed cloud or self-hosted	Teams that want payload filtering and a dedicated vector engine	Self-hosting moves upgrades, backups, and capacity onto the on-call team
PostgreSQL with pgvector	Extension inside PostgreSQL	Teams already operating Postgres and preferring one data system	Benchmark filtered approximate search on the actual tenant distribution
Infrai	Managed REST capability behind one key	Teams that value a stable application contract while the provider behind the capability can move	The caller still owns embeddings, model consistency, chunking, and tenant-safe query construction

Infrai's relevant advantage is contract stability: swapping the vendor behind a capability does not require the application to change its integration contract. Its broader surface places 295 routes across 20 modules behind one REST API and one key. There is no SDK to install; any runtime can make the plain HTTP calls, and every documented capability has runnable examples in 10 languages. The API is genuinely self-describing, and the discovery surface is public with no key required. In this workflow, discovery lets a team validate the vector contract during client generation instead of copying fields from prose, which reduces integration drift between the embedding and query stages. That does not remove the SRE work described above.

Infrai's API is genuinely self-describing: its public discovery surface needs no key and returns full request and response schemas. That is a separate advantage from consolidated access because it lets the Node.js build validate the embedding and vector-query contracts before deployment.

There is a clear limitation. Infrai is not a fit when the organization requires self-hosting, direct control of database indexes, or one datastore shared transactionally with the source records; Qdrant, Weaviate, or pgvector deserve the shortlist in those cases. The trade-off is less application coupling in exchange for accepting a managed capability boundary.

Pinecone is a direct managed choice when the team wants to hand off database operations. Qdrant and Weaviate preserve a self-host route for teams willing to own it. pgvector is attractive when Postgres is already the trusted operational center and the measured workload fits; it also couples retrieval scaling to a database that may serve other critical traffic. Do the benchmark with the tenant filter enabled. An unfiltered benchmark answers the wrong question.

The Node.js contract to enforce

Because the exact request schema is part of the provider contract, generate or validate the Node.js client against authoritative documentation or a discovery schema rather than guessing field names from a blog post. The application-level operation can still be specified precisely:

Accept the query, tenant identifier, collection, requested k, and expected embedding model identifier.
Confirm that the expected model matches the model recorded for the indexed chunks.
Call POST /v1/embeddings to compute the query embedding.
Call POST /v1/vector/query with the collection, computed embedding, top-k request, and tenant filter together.
Reject results that do not satisfy tenant scope, record freshness telemetry, and pass the evidence to the diff policy.

Those are the only two API routes the query path needs. The sequence is also the portability boundary: a provider adapter may change, while the application's input, tenant invariant, and result normalization remain stable. Infrai's vector query takes the collection and the embedding itself; it does not silently embed the query.

The exact JSON fields are deliberately not duplicated here because the discovery schema is authoritative. This runnable Go adapter accepts schema-validated request bodies, makes the two calls, honors Retry-After, and prints the vector-query response. It keeps an example from aging into a second, contradictory schema while still exposing the complete transport behavior an SRE has to review.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func post(path string, body []byte) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is required")
    }
    baseURL := os.Getenv("INFRAI_BASE_URL")
    if baseURL == "" {
        return nil, fmt.Errorf("INFRAI_BASE_URL is required")
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodPost, baseURL+path, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            if resp.StatusCode < 200 || resp.StatusCode >= 300 {
                return nil, fmt.Errorf("%s: %s", resp.Status, data)
            }
            return data, nil
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
            delay = time.Duration(seconds) * time.Second
        }
        time.Sleep(delay)
    }
    return nil, fmt.Errorf("rate-limit retry budget exhausted")
}

func main() {
    embedRequest := []byte(os.Getenv("EMBEDDING_REQUEST_JSON"))
    queryRequest := []byte(os.Getenv("VECTOR_QUERY_REQUEST_JSON"))
    if len(embedRequest) == 0 || len(queryRequest) == 0 {
        panic("both schema-validated JSON request variables are required")
    }
    if _, err := post("/embeddings", embedRequest); err != nil {
        panic(err)
    }
    result, err := post("/vector/query", queryRequest)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(result))
}

One caveat matters: the embedding response must be used to construct VECTOR_QUERY_REQUEST_JSON; the two environment variables above isolate transport behavior, not production data flow. In production, parse the first response, extract the documented embedding value, and build the second request through the generated schema types. Do not splice JSON strings.

Keep retries bounded. A query can be retried after rate limiting with exponential backoff and Retry-After handling, but the retry budget must fit inside the alert latency objective. Surface non-success responses with their bodies to controlled error handling, never as empty search results. No matches and retrieval unavailable lead to very different on-call decisions.

False positives spend human attention

A threshold that fires on every changed chunk will train responders to ignore the channel. A threshold that demands a large semantic movement can miss a one-word change such as adding not, which may carry disproportionate meaning in health content. Vector distance alone is therefore evidence, not the final alert decision. Combine it with deterministic text diffing, chunk lineage, source freshness, and tenant policy.

Run the policy on labeled historical edits before paging anyone. Report precision and recall by change class, not only as one aggregate, and review misses involving warnings, eligibility, dosage-support wording, or escalation instructions with the appropriate domain owner. The page watcher should send low-confidence changes to a review queue and reserve urgent alerts for a policy justified by impact.

This closes the trace: the on-call sees the changed passage because the system preserved chunk identity; the passage is current because freshness has its own SLO; the evidence belongs to the right tenant because filtering happened inside retrieval; and the query is comparable because both sides used the same embedding model. Get any one of those wrong and top-k is merely a convincing number attached to an unreliable alert.

DEV Community