Every number on this page is derived from inputs you substitute. The prices are placeholders, deliberately, because they move; the arithmetic and the shape of the answer do not.
Inventory: what actually has to be regenerated
The estimate is wrong before it starts if the inventory is wrong, and the usual mistake is counting only the chunk vectors. The test for whether an artefact is invalidated is simple: is it a vector, or was it computed from vectors? If either, it is dead. Working through a typical retrieval stack:
- Chunk embeddings. The obvious one, and usually the cheapest line on this list.
- Document-level and summary embeddings. If you embed a generated summary or a set of hypothetical questions alongside the chunk, the generated text survives — it is text — but its vectors do not. Do not regenerate the summaries; do re-embed them. Getting this distinction right often halves the estimate.
- Anything computed from a similarity. Cluster assignments, topic labels derived from clusters, deduplication decisions taken at a threshold, precomputed related-document tables, recommendation lists. All of these were answers to questions asked in the old space.
- The semantic cache. If you cache generations against an embedding of the request, every entry becomes unreachable at cutover. This has a second-order cost worth naming: with hit rate
hagainst a daily generation spendG, a cold cache costs roughlyh × Gper day until it refills, and that is usually larger than the entire embedding bill. - Fine-tuned adapters and rerankers. An adapter trained on top of the old base model is worthless against the new one. If you have one, its retraining is the real cost of the migration and belongs in the estimate as its own line.
- Evaluation baselines. Your recall and precision numbers were measured on the old index. They are not a comparison point until re-measured, which means the retrieval set has to exist before the migration, not after.
From corpus size to tokens
Four inputs, all of which you can measure rather than guess. Label them as assumptions and keep them visible in whatever you write down:
D = documents in the corpus assumption: 400,000
C = mean chunks per document assumption: 8
T = mean tokens per chunk assumption: 350
P = price per million input tokens assumption: substitute your own
chunks = D × C = 400,000 × 8 = 3,200,000
tokens = D × C × T = 3,200,000 × 350 = 1,120,000,000
= 1,120 million tokens
Measure T rather than assuming it. Chunk size is usually configured in characters or in a tokenizer’s tokens, and the embedding model’s tokenizer is not necessarily the one you configured with — see what a tokenizer mismatch does to a count. Take a thousand random chunks, embed them, and read the token count off the response’s usage field; that is the only count that matches the invoice.
The bill, with the price left as a variable
The bill is one multiplication:
cost = (tokens ÷ 1,000,000) × P
= 1,120 × P
If P = $0.02 per million (placeholder — substitute): $22.40
If P = $0.10 per million (placeholder — substitute): $112.00
If P = $0.50 per million (placeholder — substitute): $560.00
The finding survives whichever number you put in, and it is the point of the page: for a corpus of a few hundred thousand documents, embedding is not the expensive part of a re-embed. Embedding models are small relative to generation models and are priced accordingly. If you have been putting off a migration because it sounds costly, run this line first — there is a good chance the answer is a rounding error against your monthly generation spend, and the real costs are engineering time, the storage below, and the duration after that.
Per-token prices change without notice and vary by model, region and contract. Nothing here quotes a real price; substitute the current figure from your provider’s pricing page on the day you build the estimate, and write that date next to it.
One adjustment is worth checking before you commit: several providers offer an asynchronous batch tier at a discount for work that can wait hours rather than seconds, which describes a backfill exactly. The discount is provider-specific and is a percentage off the same per-token price, so it multiplies the line above. See how a batch inference API is shaped and what the batch discount is actually for.
The embedding line is easy to price because it is one model and one call shape. The number that is hard to hold is the total across a migration that also runs two generation providers side by side for a fortnight. If your calls already go through one gateway, that total is a query rather than three invoices reconciled by hand — which is the case Multigrid’s unified cost tracking exists for.
Storage, which is the number that surprises people
Storage is the line people under-count, because a single vector is small and three million of them are not. A float32 vector of d dimensions is 4d bytes:
bytes = chunks × d × 4
d = 1,536: 3,200,000 × 1,536 × 4 = 19,660,800,000 ≈ 19.7 GB
d = 3,072: 3,200,000 × 3,072 × 4 = 39,321,600,000 ≈ 39.3 GB
During the migration you hold both: 19.7 + 39.3 ≈ 59.0 GB
Plus the approximate index structure, which is additional
and, for a graph index, is not small.
Two consequences. First, moving from a 1,536-dimension model to a 3,072-dimension one doubles a bill that recurs monthly, unlike the embedding call which is paid once — over a year the storage delta can easily exceed the entire re-embedding cost. Second, the peak is during the migration, not after it, so a managed store with a size limit needs headroom provisioned before the backfill starts rather than discovered halfway through. Half-precision storage or quantisation changes the multiplier in that first line directly; the storage-cost breakdown works through the trade.
Duration: which rate limit binds
Duration decides whether this is an afternoon or a fortnight, and it is set by whichever published limit binds first. Two candidates, using the same corpus:
Tokens per minute limit, assumption: L = 1,000,000 TPM
1,120,000,000 ÷ 1,000,000 = 1,120 minutes ≈ 18.7 hours
Requests per minute limit, assumption: R = 3,000 RPM
batching B = 100 chunks per request
3,200,000 ÷ 100 = 32,000 requests
32,000 ÷ 3,000 ≈ 10.7 minutes
The token limit binds by two orders of magnitude, so tuning your concurrency is pointless and raising the batch size only helps until the per-request token count itself hits a cap. Do this arithmetic before writing the worker, because it tells you which knob matters. It also tells you the honest schedule: 18.7 hours is the floor with perfect utilisation, and real backfills lose time to retries after 429s and to the tail of a partition, so plan on a multiple. Link the retry behaviour to backoff on 429 rather than inventing a fresh one.
What the estimate is most sensitive to
Every quantity above is linear in the token count, so the estimate is only ever as good as T and the chunk count. Two things move it more than people expect. Overlap first: if chunks of T tokens overlap by o, the corpus is embedded roughly T / (T − o) times over. At 350-token chunks with 50 tokens of overlap that is a factor of 1.17; at 200-token chunks with 50 tokens of overlap it is 1.33. Halving the chunk size while holding overlap fixed therefore raises the bill by more than the chunk count suggests.
Second, the multiplicity of what you embed. A stack that embeds the chunk, a summary of the chunk and three hypothetical questions per chunk is embedding four to five times the corpus, and the estimate has to count each one. Multiply the token line by the number of vectors per chunk, not by one.
What barely moves it: the model’s dimension count, which affects storage but not usually price, and the exact chunk-count estimate, which is linear and forgiving. If your inputs are within twenty per cent, the answer is within twenty per cent, and for a decision about whether to migrate that is a comfortable margin. Record the assumptions next to the answer so that when someone questions the number six weeks later, the disagreement can be about an input rather than about the conclusion.
Top comments (0)