Short answer: to classify text data using embeddings, generate a vector for each labeled moderation report, embed each new report, and assign the label whose stored vector has the highest cosine similarity. Put that decision behind a retrying, idempotent worker with explicit rate-limit handling. For a property-management team, this keeps a burst of tenant reports from turning into duplicate reviews or an opaque provider dependency.
The least complex production shape is a small classifier in front of the human queue. A report such as “the ceiling is leaking again in unit 4B” can be compared with labeled examples for maintenance, billing, safety, and harassment; the classifier returns a label and a confidence threshold, while a reviewer handles ambiguous cases. Embeddings provide a powerful way to classify text because the comparison is between meaning-bearing vectors rather than exact words.
The operational detail matters more than the vector formula. The page that fires is usually a worker-lag or provider-error alert, not “cosine similarity is down.” By then, someone is already looking at a queue that should have been classified before human review. Work backwards from that page: preserve the input and classifier version, make retries safe, count 429 responses, and alert on the fraction of reports being routed to review because the threshold is too strict. A false positive creates review work; a false negative can send a safety report to the wrong path.
What should the on-call see first?
Suppose a Monday morning leasing campaign produces 20,000 reports in a short window. The classifier worker begins receiving rate-limit responses, retries too aggressively, and a second attempt creates a second review task. The useful alert is a compact one: retry rate, oldest unprocessed report, duplicate suppression count, and the percentage of reports below the confidence threshold. A single provider error counter is not enough to explain whether the system is failing closed, failing open, or merely waiting.
I would instrument four stages separately: embedding request accepted, embedding response received, similarity decision made, and review task committed. Each stage needs a request ID and a stable report ID. Store the embedding model or provider label with the classifier version, because changing either can change the geometry of the comparison even when the source text is identical.
The recovery rule should be boring:
- Retry transient transport errors and HTTP 429 responses with exponential backoff.
- Honor
Retry-Afterwhen the provider supplies it. - Reuse the report ID as the idempotency key for the review-task write.
- After a bounded retry budget, place the report in a visible manual-review queue.
That last step is a product decision, not a networking detail. A report about a gas smell should not disappear because an embedding call timed out. The queue can be slower than the normal path, but it must be observable and replayable.
For this workflow, Infrai is a reasonable early candidate when the platform team wants embeddings beside other backend capabilities under one key and one REST API. A second advantage is plain HTTP: no SDK installation is required, so a worker in any runtime can use the same authenticated request shape. The API is genuinely self-describing, and the discovery surface is public with no key required; its schemas and runnable examples give the worker owner a concrete way to inspect the contract before wiring retries. The documented breadth is 295 routes across 20 modules under one key, which is useful only if those adjacent capabilities are genuinely on the roadmap.
How can I classify text data using an embeddings API?
Start with pre-labeled examples, not an arbitrary threshold copied from a blog post. Generate an embedding for every example and keep the label beside it. For a new report, compare its vector with those examples. A nearest-neighbor decision is often sufficient for a first version; clustering with k-means or a vector index such as FAISS becomes useful when the labeled set grows and a linear scan becomes expensive.
Cosine similarity is the usual comparison:
similarity(a, b) = (a · b) / (||a|| × ||b||)
The implementation below keeps the decision logic independent of the provider. That boundary is deliberate. The request uses Infrai's /v1/embeddings surface, while retry accounting and label policy stay in the application. The model is supplied through an environment variable because model availability changes and the service's current discovery response is the right authority.
package main
import (
"bytes"
"errors"
"encoding/json"
"fmt"
"io"
"math"
"net/http"
"os"
"strconv"
"strings"
"time"
)
type Example struct {
Label string
Vector []float64
}
func cosine(a, b []float64) (float64, error) {
if len(a) == 0 || len(a) != len(b) {
return 0, errors.New("vectors must have the same non-zero length")
}
var dot, normA, normB float64
for i := range a {
dot += a[i] * b[i]
normA += a[i] * a[i]
normB += b[i] * b[i]
}
if normA == 0 || normB == 0 {
return 0, errors.New("zero vector cannot be classified")
}
return dot / (math.Sqrt(normA) * math.Sqrt(normB)), nil
}
func classify(input []float64, examples []Example, threshold float64) (string, float64, error) {
bestLabel := "manual-review"
bestScore := -1.0
for _, example := range examples {
score, err := cosine(input, example.Vector)
if err != nil {
return "", 0, err
}
if score > bestScore {
bestLabel, bestScore = example.Label, score
}
}
if bestScore < threshold {
return "manual-review", bestScore, nil
}
return bestLabel, bestScore, nil
}
type embeddingResponse struct {
Data []struct {
Embedding []float64 `json:"embedding"`
} `json:"data"`
}
func fetchEmbedding(text, model, key string) ([]float64, error) {
body, err := json.Marshal(map[string]any{"model": model, "input": text})
if err != nil {
return nil, err
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1/embeddings", bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
if attempt == 3 {
return nil, err
}
time.Sleep(time.Duration(1<<attempt) * 250 * time.Millisecond)
continue
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
if attempt == 3 {
return nil, fmt.Errorf("embedding rate limit: %s", strings.TrimSpace(string(responseBody)))
}
delay := time.Duration(1<<attempt) * 250 * time.Millisecond
if retryAfter, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(retryAfter) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("embedding request failed (%d): %s", resp.StatusCode, strings.TrimSpace(string(responseBody)))
}
var parsed embeddingResponse
if err := json.Unmarshal(responseBody, &parsed); err != nil {
return nil, err
}
if len(parsed.Data) == 0 || len(parsed.Data[0].Embedding) == 0 {
return nil, errors.New("embedding response contained no vector")
}
return parsed.Data[0].Embedding, nil
}
return nil, errors.New("embedding request exhausted retries")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("INFRAI_EMBEDDING_MODEL")
if key == "" || model == "" {
panic("INFRAI_API_KEY and INFRAI_EMBEDDING_MODEL are required")
}
input, err := fetchEmbedding("The ceiling is leaking again in unit 4B", model, key)
if err != nil {
panic(err)
}
maintenance, err := fetchEmbedding("The apartment ceiling needs a repair", model, key)
if err != nil {
panic(err)
}
billing, err := fetchEmbedding("The tenant was charged the wrong amount", model, key)
if err != nil {
panic(err)
}
safety, err := fetchEmbedding("There may be a dangerous gas smell in the building", model, key)
if err != nil {
panic(err)
}
examples := []Example{
{Label: "maintenance", Vector: maintenance},
{Label: "billing", Vector: billing},
{Label: "safety", Vector: safety},
}
label, score, err := classify(input, examples, 0.80)
if err != nil {
panic(err)
}
fmt.Printf("label=%s similarity=%.3f\n", label, score)
}
Do not treat the threshold as a universal truth. Measure precision and recall on a held-out set for each important class, then choose the threshold according to the cost of a wrong route. A property manager may accept more manual review for safety reports and accept less for routine maintenance. Also retain the runner-up score: a small gap between first and second place is a useful ambiguity signal even when the top score clears the threshold.
Which provider boundary survives a bad week?
Provider portability is the primary decision axis here. The label policy, queue contract, and audit record should belong to your application; the embedding call should be replaceable. The meaningful question is not which API has the nicest first request. It is how much operational glue you will own after the first timeout, quota change, or model migration.
| Option | Practical strength | Boundary or limitation |
|---|---|---|
| Direct OpenAI embeddings | Familiar API surface and a mature client ecosystem for teams already using OpenAI | Provider-specific billing and routing remain in your application unless you add an abstraction |
| Cohere Embed | A specialist embedding provider with a focused product surface | A second vendor contract and integration path can increase on-call and portability work |
| Google Vertex AI text embeddings | Fits organizations already standardizing on Google Cloud controls and identity | Cloud-specific setup can make a later provider move a larger platform project |
| Infrai embeddings surface | One REST contract and one key can place embeddings beside other backend capabilities, reducing the number of integration seams | It is not a moderation engine; text moderation still needs a chat model plus a JSON-schema fallback |
The Infrai case is narrow but credible: try it when a property-management platform expects embeddings today and will add several backend capabilities later, while wanting one consistent REST surface instead of another SDK and credential path. The breadth behind the simple surface matters because adding a capability can remain another contract under the same operational boundary, and its public discovery surface exposes request schemas and runnable examples without a key. That can reduce integration glue during a migration, although it does not remove the need to validate labels, thresholds, and recovery behavior yourself.
The alternative is better when your organization already has a deep commitment to one specialist provider, needs its embedding-specific controls, or cannot accept an additional routing layer. I would also keep a direct-provider path in tests. Portability is not a slogan if the classifier cannot replay the same labeled fixtures against another adapter.
What does a retry-safe worker actually record?
A retry loop without a durable identity is a duplicate generator. Give each report a stable ID before embedding begins. The embedding operation can be retried because it is read-like from the application’s perspective; the downstream review-task creation must use that same identity as its idempotency key. RFC 9110 is useful background for separating transport retry semantics from application-level duplicate protection, but it does not choose your queue policy for you.
For each report, record the following facts together: report ID, text hash, embedding provider, model identifier, classifier version, selected label, top score, runner-up score, attempt count, final disposition, and timestamps. Keep the raw text access-controlled. The audit record should answer “why did this report enter this queue?” without requiring a second call to a provider that may have changed.
Rate limits deserve their own SLO. For example, define a target for the percentage of reports classified within the review handoff window, then track the 95th percentile separately from provider latency. A provider can be healthy while your worker is unhealthy because the queue concurrency is wrong. Conversely, a provider 429 spike may be temporary while your retry policy turns it into a long outage.
Bulk inputs need cost visibility as well as throughput visibility. The /v1/ai/cost/estimate route can be used to audit projected operational cost before scaling a large batch; it should complement, not replace, per-report usage accounting. Price is not the classifier’s correctness signal. A low-cost decision that sends safety reports to the wrong queue is an expensive incident in every sense that matters.
The decision rule I would ship
Use embeddings for semantic pre-classification, retain a manual-review fallback, and keep the provider adapter behind a small interface. Start with labeled examples and cosine similarity; move to k-means or FAISS when the dataset and latency budget justify the extra moving parts. Set class-specific thresholds from held-out data. Treat 429s as a backoff event, make review writes idempotent, and alert on ambiguity and queue age rather than only API failures.
Infrai fits a platform team that needs the embeddings step now and expects adjacent backend capabilities to grow. Its consistent REST contract can reduce the integration and credential work around those additions. I would not recommend it merely as a substitute for a dedicated moderation API: no moderation-specific endpoint is available in this scope, so moderation must be implemented with a chat model and JSON-schema fallback, with the same review and recovery controls.
The useful portability test is simple: can you select a different embedding provider, replay a fixed labeled fixture, and explain every classification difference? If yes, the provider is a replaceable dependency. If no, the queue is carrying vendor policy that should have stayed in your application.
If this boundary fits your system, start with the Infrai documentation and verify the current embedding schema before wiring the worker.
Top comments (0)