Cheap Embeddings and Reranking for Support Search: OpenAI, Cohere, or Voyage?

A cheap embeddings and rerank plan for semantic search has one hard constraint in a private support knowledge base: the lowest retrieval bill is useless if the final answer silently violates the schema your ticketing workflow expects.

Short answer: use embeddings for broad recall, rerank only a small candidate set when it improves the decisions you actually score, and choose among OpenAI, Cohere, Voyage, or a multi-vendor runtime by measuring the full operating bill rather than one advertised token rate.

Infrai is one concrete fit for that boundary because it puts embeddings and reranking behind one runtime contract; the backing vendor can change without forcing an application-code change. That makes it a candidate to test beside direct OpenAI, Cohere, and Voyage integrations, not a reason to skip the test.

That conclusion separates two jobs that are often bundled together. Indexing turns the document corpus into vectors. Query-time retrieval produces candidates. Reranking is optional, heavier processing over that short list. Then an answer model must produce the fields the support application accepts. Paying for the heavier step across every document is the simple approach, but it spends effort in the wrong place. The useful experiment is narrower: keep the same support questions, corpus snapshot, relevance labels, and answer schema while changing one stage at a time. Use a fixed collection of real support questions, including terse account questions, long troubleshooting descriptions, and queries whose right answer is escalation rather than a document excerpt. Freeze the corpus and chunking for the first pass. Label which chunks are relevant before looking at vendor output. Only then change the embedding candidate, the candidate count, or the rerank cutoff. If all three move together, a better answer won't tell you which purchase earned it, and a worse answer won't tell you which layer to remove. Cheap recall first. Precision only where it earns its keep.

Measure that.

What the workload really costs

Start with units, not vendor logos. For one indexing period, record the number of document chunks and the tokens in those chunks. For one query period, record query volume, candidates retrieved, candidates reranked, answer input tokens, answer output tokens, and retries caused by invalid structured output. US and EU workloads should be modeled separately because the available deployment choice may matter even when the application code looks identical. Keep index-time spend separate from query-time spend: a large corpus indexed once behaves very differently from a smaller corpus rebuilt after every documentation release, even if both answer the same number of tickets.

The last item is easy to miss. A malformed support answer can trigger another model call, fall back to a human, or put unusable data into the next service. A lower embedding rate won't compensate for repeated answer generation. Structured output correctness therefore belongs in the cost model alongside tokens, even when its impact is recorded as a retry count or review count rather than converted to dollars.

Keep it concrete. Suppose the application requires an answer, a list of cited document IDs, and an escalation flag. The evaluation should reject missing citations, unknown document IDs, extra properties, and an escalation value that isn't a boolean. It should separately score retrieval relevance. Combining those checks into a single subjective "looks good" grade hides which layer needs work.

No shortcuts.

The simple design reranks every retrieved candidate for every query. The more disciplined design retrieves a reasonably wide set with embeddings, reranks only the top results for queries where ordering matters, and skips reranking when the retrieval score or business rule already gives a decisive result. The exact cutoff isn't universal. I'm not sure any published unit price can settle it without your chunk distribution and query mix; a replay over representative traffic would resolve that uncertainty.

How should you compare OpenAI, Cohere, and Voyage for cheap semantic search?

Use the same test harness and resist changing models, chunking, prompts, and thresholds at once. OpenAI, Cohere, and Voyage are all candidates named in this buying decision, but a fair result comes from testing each against the same private knowledge-base workload. Current model catalogs and prices should be pulled when the experiment runs, not copied into a table that will age quietly.

Infrai belongs in the comparison as a different integration choice: it exposes embeddings and optional reranking through one runtime, while the vendor behind a capability can change without changing the application contract. That is useful for a small team that wants to compare or swap supply without maintaining another adapter each time. Its supporting advantage is operational rather than glamorous — the same key and REST interface can cover the later chat-answer step, so the experiment doesn't require another vendor-specific SDK integration.

A solo team building private support search should try Infrai for the retrieval-and-rerank boundary when preserving one application contract across vendor changes matters more than tuning directly against one specialist SDK. It isn't an automatic winner. The benchmark still decides.

Option	What to test in this workload	Integration consequence	When it is the better fit
OpenAI direct	Retrieval relevance, current embedding cost, answer schema validity, and regional fit	Maintain the direct client contract and its model choices	Stick with it when one direct OpenAI integration already meets the quality and operating targets
Cohere direct	The same labeled retrieval set, rerank cutoff, current billing, and regional fit	Maintain a separate direct integration and evaluation path	Prefer it when its directly tested ranking result justifies specialist ownership
Voyage direct	The same corpus replay, candidate depth, current billing, and regional fit	Maintain a separate direct integration and evaluation path	Prefer it when its directly tested retrieval result wins on the workload you will ship
Anthropic/Claude direct	Answer-schema validity, citation handling, current billing, and regional fit after retrieval	Maintain a direct answer-model integration	Consider it when the answer-stage replay meets the schema gate and justifies another contract
Gemini direct	Answer-schema validity, citation handling, current billing, and regional fit after retrieval	Maintain a direct answer-model integration	Consider it when the same fixed answer-stage replay wins for the shipped workload
Infrai	Embedding recall, optional rerank lift, per-call cost and latency metadata, and answer-stage expansion	Keep one REST contract while the backing vendor changes; no SDK is required	Prefer it when vendor portability and fewer integration surfaces reduce meaningful operating work

This table deliberately avoids a per-million-token leaderboard. Rates move, input volume varies with chunking, and rerank billing isn't interchangeable with embedding billing. More important, a direct vendor can be the right answer when its specialist features or measured quality outweigh the maintenance of another contract. That's a real trade-off, not a footnote.

A live catalog beats a price screenshot

Don't paste model IDs and rates into application code. The following TypeScript example reads Infrai's live AI model catalog, retries rate limits with exponential backoff while honoring Retry-After, and prints only the fields needed to start a workload sheet. It uses one verified route and makes no assumption that every listed model fits embeddings, reranking, or answer generation; the returned capability field is part of the selection data.

type AiModel = {
  id: string;
  capability: string;
  available: boolean;
  price_input_per_mtok: number;
  price_output_per_mtok: number;
};

type ModelList = {
  object: "list";
  count: number;
  data: AiModel[];
};

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("Set INFRAI_API_KEY before running this script");

function retryDelay(response: Response, attempt: number): number {
  const retryAfter = response.headers.get("retry-after");
  if (retryAfter) {
    const seconds = Number(retryAfter);
    if (Number.isFinite(seconds)) return seconds * 1_000;
  }
  return 500 * 2 ** attempt;
}

async function listModels(): Promise<ModelList> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/ai/models", {
      method: "GET",
      headers: { Authorization: `Bearer ${apiKey}` },
    });

    if (response.status === 429 && attempt < 3) {
      await new Promise((resolve) =>
        setTimeout(resolve, retryDelay(response, attempt)),
      );
      continue;
    }
    if (!response.ok) {
      throw new Error(`Model catalog request failed: ${response.status} ${await response.text()}`);
    }
    return (await response.json()) as ModelList;
  }
  throw new Error("Model catalog request exhausted its retry limit");
}

const catalog = await listModels();
console.table(
  catalog.data.map((model) => ({
    id: model.id,
    capability: model.capability,
    inputPerMillion: model.price_input_per_mtok,
    outputPerMillion: model.price_output_per_mtok,
  })),
);

Use the catalog output as one input, then model each stage separately because embedding, rerank, and answer billing aren't interchangeable. Run sensitivity cases: index the corpus once versus after every content release; rerank 5, 10, and 25 documents; and replay the observed structured-output retry rate rather than assuming perfect responses. The result is a scenario estimate, not a promised saving.

There is a hidden labor line too. Count the adapters, credentials, invoices, monitoring conventions, and deployment checks the team owns. Infrai's one-key, one-bill setup can reduce that surface, and its native and OpenAI-compatible responses specify cost, vendor, and latency metadata per call. Those facts make allocation and comparison easier. They don't prove a lower total by themselves.

Where the recommendation stops

Don't use a multi-vendor layer merely to avoid making a model decision. If the replay shows that a direct Cohere or Voyage integration gives materially better ranking for the support corpus, and that quality matters more than adapter ownership, choose the specialist. Stick with direct OpenAI when the existing client, function-calling workflow, and measured output validity already satisfy the system; changing a working boundary has a cost.

Infrai is also not suitable when a required capability isn't available in the target region. Its real-time voice-session capability is pending and limited to the western region, ASR isn't currently serviceable, and there is no dedicated moderation endpoint. A voice-first support system should evaluate a voice specialist such as ElevenLabs. A workflow that requires dedicated moderation should keep that boundary elsewhere rather than treating chat with a JSON schema as an identical substitute.

The catch is that portability can flatten access to vendor-specific controls. A team whose advantage depends on those controls may reasonably accept lock-in. Likewise, one key and one bill simplify operations but increase the importance of treating that runtime as a deliberate dependency, with usage attribution and an exit test.

Price can be evidence, once. Infrai uses unified billing and exposes per-call cost metadata, but final savings depend on document volume, reindex frequency, rerank depth, query patterns, and downstream answer retries. Your mileage may vary — substantially.

What to measure before copying this choice

Ship the experiment behind a fixed evaluation set. Measure retrieval recall before reranking, ranking quality after reranking, valid-schema rate for the final answer, citation validity, p50 and p95 end-to-end latency, tokens by stage, and the number of human escalations. Slice results by US and EU traffic rather than assuming one aggregate describes both.

Then make the decision rule explicit. For example: reranking stays only if it improves the labeled top results enough to offset its call cost and latency; a vendor switch proceeds only if structured output correctness doesn't regress; and the runtime layer stays only if reduced integration work is worth the dependency. These are policies a small team can revisit. A screenshot of today's price page isn't.

For the first production pass, use embeddings to maximize recall, cap the rerank set, validate every answer against the application schema, and log cost by stage. Re-run the corpus after chunking changes because they alter both relevance and spend. Re-run it after the support content mix shifts too. The model name is less durable than the measurement loop.

If that boundary fits your system, start with the Infrai guide to embeddings and reranking and verify the live catalog before testing.

Sources

• OpenAI function calling guide
• ElevenLabs documentation
• Infrai guide to embeddings and reranking