A generation query in BigQuery is billed twice: once by BigQuery for the bytes it scanned, and once by Vertex AI for the tokens the model processed. The two numbers are not close to each other, and knowing which one dominates changes what you optimise.
Two meters, one query
Google’s generative AI overview for BigQuery states the split directly: the bytes processed by BigQuery are billed under standard BigQuery pricing, and remote models make calls to Vertex AI, so queries against remote models also incur Vertex AI charges. Both land on the same project, on the same invoice, under two different service names.
That is worth internalising before looking at any figure, because it explains a common surprise: a team watches its BigQuery spend after adopting generation functions, sees it barely move, and then finds the real number under Vertex AI where nobody was looking.
The BigQuery side
BigQuery charges for the bytes the query reads. For a generation query that means the prompt column and any other columns the subquery selects — not the generated output, which BigQuery did not read from storage.
Google’s BigQuery pricing page lists on-demand analysis at $6.25 per TiB scanned, with the first 1 TiB per month free, and a minimum of 10 MB billed per table referenced. Under a capacity-based edition you are buying slot-hours instead and this component is absorbed into that commitment.
The $6.25 per TiB rate and the 1 TiB free tier are the published on-demand figures for BigQuery at the time of writing, and they vary by region and by edition. Read the current rate off Google’s pricing page before using this arithmetic on a real budget.
The Vertex AI side
The model call is billed as any other Vertex AI generative call: per input token and per output token, at the rate for the model your ENDPOINT names. Google’s Vertex AI generative AI pricing page lists Gemini 2.5 Flash at $0.30 per million input tokens for text, image and video, and $2.50 per million output tokens; Gemini 2.5 Flash-Lite at $0.10 and $0.40 respectively. Neither of those two is tiered by prompt length, which some larger models are.
Model prices move, and models are retired. These are the published Vertex AI rates at the time of writing for two specific model names; confirm both the rate and the model’s availability before committing to a figure.
If the endpoint is one of your own rather than a shared Gemini model, this side is not token-priced at all — it is node-hours for the machine type the endpoint runs on, charged whether or not a query is running. That is a completely different cost shape and is the argument in autoscaling a Vertex AI endpoint by traffic.
A worked per-row cost
Assumptions, stated so you can substitute your own: 100,000 rows; a prompt column averaging 400 bytes per row and no other columns read; around 250 input tokens and 150 output tokens per row; Gemini 2.5 Flash; on-demand BigQuery pricing with the free tier already consumed.
BigQuery side
bytes scanned = 100,000 rows x 400 B = 40,000,000 B
= 40,000,000 / 1,099,511,627,776 = 0.0000364 TiB
cost = 0.0000364 TiB x $6.25/TiB = $0.00023
Vertex AI side
input tokens = 100,000 x 250 = 25,000,000
input cost = 25 M x $0.30 per 1 M = $7.50
output tokens = 100,000 x 150 = 15,000,000
output cost = 15 M x $2.50 per 1 M = $37.50
subtotal = $45.00
Total = $45.00023
Per row = $0.00045
BigQuery's share = 0.0005% of the bill
The conclusion generalises well beyond these assumptions. For text generation over reasonably sized prompt columns, the BigQuery charge is a rounding error and the token charge is the bill. You would have to scan gigabytes per thousand rows before the scan cost became visible next to the tokens.
Which tells you where the levers are, and they are all on the Vertex side. Output tokens are $2.50 against $0.30 for input on this model, so the same eight-fold ratio that governs any generation cost governs this one: a max_output_tokens of 16 for a one-word label instead of a default in the hundreds is the largest single saving available. Choosing Flash-Lite over Flash cuts both rates by roughly two thirds and a factor of six respectively, at whatever quality cost your task incurs. And the number of rows is a WHERE clause: filtering before generating, rather than generating and filtering after, is free on the BigQuery meter and saves the entire token cost of the excluded rows.
One saving that does not apply here is worth naming. Prompt caching discounts a repeated prefix, and Google lists cached input tokens at $0.03 per million for Flash. Whether a per-row query benefits depends on the shared prefix being long enough and reused closely enough in time to be cached at all, which for a long instruction block repeated over a million rows is plausible and for a short one is not. Do not build a forecast on it without checking your own token counts.
There is a trap in the usual pre-flight check that follows directly from the two-meter split. A dry run — bq query --dry_run, or the estimate the console shows before you press run — reports the bytes BigQuery will process. On a generation query that number is the cheap half, and the estimate is silent about the expensive one. A dry run reporting 40 MB looks like a free query and can be a forty-five dollar one. The only pre-flight that means anything here is row count multiplied by your own measured tokens per row, which is why running the small LIMIT-ed version first and reading the actual token usage off the billing export is the step to take before the full run.
Attributing the Vertex charge back to the query
The obvious follow-up is how you tell which BigQuery job caused which Vertex AI charge, and Google’s documentation gives a real answer: filter Cloud Billing for the Vertex AI service and use the bigquery_job_id_prefix label, which ties the Vertex usage back to the originating job. That label is the difference between “our Vertex spend went up” and “this scheduled query is the reason”.
-- export billing to BigQuery first, then:
SELECT
(SELECT value FROM UNNEST(labels)
WHERE key = 'bigquery_job_id_prefix') AS job_prefix,
service.description AS service,
SUM(cost) AS cost
FROM `PROJECT_ID.billing_export.gcp_billing_export_v1_XXXXXX`
WHERE service.description LIKE '%Vertex%'
AND usage_start_time >= TIMESTAMP('2026-08-01')
GROUP BY job_prefix, service
ORDER BY cost DESC
LIMIT 20;
Setting a budget alert on the Vertex AI portion specifically is a separate mechanism, covered in setting a budget alert for Vertex AI spend.
Two meters for one logical operation is the general shape of this problem, not a BigQuery quirk: any setup that reaches more than one model provider ends up reconciling usage records that use different units, different rounding and different reporting delays before it can answer “what did this feature cost”. Multigrid is an LLM gateway, so it records tokens and cost per request at the point of the call, tagged with whatever key or team made it. The general version of the problem — and how to solve it without a gateway — is in attributing LLM cost to teams and features.
Top comments (0)