The CREATE MODEL statement is one line. Everything that goes wrong happens in the two steps before it, in a connection object and an IAM grant on a service account that BigQuery creates for you and does not mention again.
What a remote model is
A BigQuery ML remote model is not a model. It is a named reference, stored in a dataset, that binds three things: an endpoint identifier, a connection, and a set of default options. Nothing is trained, nothing is copied, and creating one costs nothing. When a query calls a generation function against it, BigQuery uses the connection’s identity to call Vertex AI on your behalf, once per row.
The reason it exists rather than letting you name a model inline is authorisation. A BigQuery query runs as the querying user, but the call out to Vertex AI has to be made by something with roles/aiplatform.user, and you do not want every analyst who can run a query to hold that role directly. The connection is the indirection: it holds the identity, an administrator grants that identity the role once, and the model object is what analysts are given access to. That is the design, and it explains why the connection cannot be skipped.
The connection, and its hidden service account
You need a CLOUD_RESOURCE connection — the same type used for remote functions and BigLake tables. It must live in a location compatible with the dataset you will create the model in.
bq mk --connection \
--location=US \
--project_id=PROJECT_ID \
--connection_type=CLOUD_RESOURCE \
vertex-conn
Creating the connection creates a service account with it. You do not name it and it does not appear where you would look for it; you read it off the connection:
bq show --format=prettyjson --connection PROJECT_ID.US.vertex-conn
# the address is at .cloudResource.serviceAccountId, e.g.
# bqcx-123456789012-ab3d@gcp-sa-bigquery-condel.iam.gserviceaccount.com
Then grant that account the role, per Google’s text-generation documentation, which specifies roles/aiplatform.user for the connection’s service account:
SA=$(bq show --format=json --connection PROJECT_ID.US.vertex-conn \
| python3 -c 'import json,sys; print(json.load(sys.stdin)["cloudResource"]["serviceAccountId"])')
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:${SA}" \
--role="roles/aiplatform.user"
IAM propagation is not instant. A CREATE MODEL run within a minute or so of the grant can fail with a permission error and then succeed unchanged on a retry, which is one of the few times “wait and try again” is the correct diagnosis rather than a dodge.
CREATE MODEL
CREATE OR REPLACE MODEL `PROJECT_ID.analytics.gemini_flash`
REMOTE WITH CONNECTION `PROJECT_ID.US.vertex-conn`
OPTIONS (ENDPOINT = 'gemini-2.5-flash');
Three details in that statement decide whether it works. The connection reference is project.location.connection_id, and the location part is the connection’s location, not the dataset’s region string — for a multi-region dataset it is US or EU, and for a regional one it matches the region. The dataset in the model path must already exist. And CREATE OR REPLACE is safe here in a way it is not for a trained model, because there is nothing to lose: replacing a remote model rewrites a pointer.
What goes in ENDPOINT
ENDPOINT accepts a model name such as gemini-2.5-flash, or a full endpoint URL for a model you have deployed yourself on a Vertex AI endpoint. The first form is what almost everyone wants and the second is what you use when the model is your own — a tuned model, or something from Model Garden you deployed to a dedicated endpoint, as in deploying a model to a Vertex AI endpoint.
Two things follow from the second form being possible. The model does not have to be a Google one — anything reachable at a Vertex AI endpoint in the same project works, including open-weight models you host. And the cost profile changes completely: a shared Gemini endpoint is billed per token, and your own endpoint is billed for the nodes it runs on whether or not a query is running, which is the subject of autoscaling a Vertex AI endpoint.
The model name in ENDPOINT is a live reference, not a snapshot. When a model version is retired by Google, the remote model does not break at creation time — it breaks the next time somebody runs a query against it, potentially months later, in a scheduled job nobody is watching. Pin to a specific version string rather than an alias where the alias would move under you, and put the model name somewhere your deployment process can grep.
Using it, and where it lives
The model object shows up in the dataset alongside tables, and INFORMATION_SCHEMA knows about it, which is how you audit what exists:
SELECT model_name, model_type, creation_time
FROM `PROJECT_ID.analytics.INFORMATION_SCHEMA.MODELS`
ORDER BY creation_time DESC;
-- and the options, including the endpoint it points at
SELECT model_name, option_name, option_value
FROM `PROJECT_ID.analytics.INFORMATION_SCHEMA.MODEL_OPTIONS`
WHERE option_name = 'endpoint';
Access to the model is dataset-level BigQuery IAM. A user with roles/bigquery.dataViewer on the dataset and the ability to run jobs can call the model, and in doing so spends Vertex AI money through the connection’s identity rather than their own. That is the intended design and it is also the thing to think about before granting the dataset widely: the blast radius of the connection is every user who can query the dataset it is referenced from.
Running an actual generation query against it — the arguments, the output columns, the error column you must check — is the ML.GENERATE_TEXT walkthrough.
What a remote model does not give you
It does not give you a queue. A query over a million rows issues calls at whatever rate BigQuery decides, against your project’s Vertex AI quota, and a large enough query will exhaust it. There is no per-model rate limit you can set on the BigQuery side.
It does not give you freedom about location. Three locations have to be compatible — the dataset holding the model, the connection, and the region the model is served from — and the constraint is the first thing to check when a CREATE MODEL that looks correct is rejected. A connection in US pairs with a dataset in the US multi-region; a connection in us-central1 pairs with a dataset in us-central1. Mixing a multi-region dataset with a regional connection fails, and so does a query that joins a table in one region to a model in another, because BigQuery does not move data across regions to satisfy a query. Decide the location once, at the dataset, and create everything else to match.
It does not give you retries you control. Transient failures surface as a populated error column on the affected rows rather than as a failed query, which is better than failing the whole job but means a successful query can contain thousands of rows that produced nothing. A query is not done because it returned.
And it does not give you one bill. The BigQuery side and the Vertex AI side are charged separately to the same project, which is worth understanding before the first large run rather than after — what a remote-model query costs works it through per row.
Top comments (0)