If you still export Cloud Trace to BigQuery through a sink, that pipeline is already dead. Per the Trace release notes, trace sinks were deprecated on February 18, 2026, and since August 31, 2026 you can no longer create or edit one. The replacement is Observability Analytics: a linked dataset that exposes the trace storage to BigQuery with no copy of the data.
The same mechanism exists for Cloud Logging, and that is where it gets confusing. Both features are called "linked dataset", both produce a BigQuery dataset holding a view, and both are configured with Terraform. Almost everything else about them differs. I wired up both in one project, and this is what actually had to be true.
Two bucket systems that look like one
The single biggest time sink: _Trace is not a Cloud Logging log bucket.
$ gcloud logging buckets list --project=my-project
LOCATION BUCKET_ID RETENTION_DAYS LIFECYCLE_STATE
global _Default 30 ACTIVE
global _Required 400 ACTIVE
No _Trace. It is not hidden, it is not a permissions problem — it is in a different system entirely:
$ gcloud observability buckets list --location=us --project=my-project
name: projects/my-project/locations/us/buckets/_Trace
description: Bucket for storing spans from Cloud Trace.
Note the location. Log buckets default to global; observability buckets follow the project's observability default storage location, which here is us. Any blog post or LLM answer telling you to point google_logging_project_bucket_config at _Trace in global is wrong on both counts.
| Cloud Trace | Cloud Logging | |
|---|---|---|
| Storage | observability bucket _Trace, dataset Spans
|
log bucket _Default
|
| Location | project's observability default (e.g. us) |
global |
| CLI | gcloud observability buckets |
gcloud logging buckets |
| Enable step | none — the dataset already exists | enable_analytics = true |
| Terraform | google_observability_link |
google_logging_project_bucket_config + google_logging_linked_dataset
|
| View | _AllSpans |
_AllLogs |
While we are listing things that do not exist: there is no google_cloud_trace_sink resource. Ask an LLM for Terraform to export Cloud Trace to BigQuery and there is a good chance it invents one, because the API-level concept existed right up until it was turned off.
Cloud Trace
Terraform support for observability buckets and links landed on September 22, 2026, in provider 8.2.0. The link itself is four fields:
data "google_project" "this" {
project_id = var.project_id
}
resource "google_project_iam_member" "observability_bigquery" {
project = var.project_id
role = "roles/bigquery.admin"
member = "serviceAccount:service-${data.google_project.this.number}@gcp-sa-observability.iam.gserviceaccount.com"
}
# The grant needs time to propagate before the link is created.
resource "time_sleep" "wait_observability_iam" {
create_duration = "90s"
depends_on = [google_project_iam_member.observability_bigquery]
}
resource "google_observability_link" "trace" {
project = var.project_id
location = "us" # must match the _Trace bucket, NOT global
bucket = "_Trace"
dataset = "Spans"
link_id = "trace_linked_dataset" # becomes the BigQuery dataset name
depends_on = [time_sleep.wait_observability_iam]
}
The IAM grant is the part people skip. The dataset on the BigQuery side is created and managed by the Observability service agent, so without roles/bigquery.admin on service-<PROJECT_NUMBER>@gcp-sa-observability.iam.gserviceaccount.com the link creation fails. The service agent already exists if trace data has ever landed in the project; confirm with:
gcloud observability settings describe --location=global --project=my-project
# serviceAccountId: service-123456789012@gcp-sa-observability.iam.gserviceaccount.com
link_id only accepts letters, digits and underscores — no hyphens, which is not the usual GCP naming convention and will bite you if you mirror your other dataset names.
Cloud Logging
Log buckets need Log Analytics switched on first, and _Default already exists, so this resource acquires it rather than creating it:
resource "google_logging_project_bucket_config" "default" {
project = var.project_id
location = "global"
bucket_id = "_Default"
retention_days = 30 # match the current value or you will see a diff
enable_analytics = true
}
resource "google_logging_linked_dataset" "default" {
parent = "projects/${var.project_id}" # not `project`
location = "global"
bucket = google_logging_project_bucket_config.default.bucket_id
link_id = "logging_linked_dataset"
}
Three traps packed into ten lines:
-
enable_analyticscannot be disabled once enabled. There is no undo on this one. Decide before you apply. -
_Defaultand_Requiredare auto-created and cannot be deleted. Terraform takes over the existing bucket on create, andterraform destroyonly drops it from state — the bucket and its logs stay. Harmless, but it meansretention_daysis a live setting on a bucket that already has your logs in it, not a greenfield value. -
google_logging_linked_datasettakesparent, notproject. It is the odd one out among Logging resources, and the error is a flatAn argument named "project" is not expected here.
The 403 that creates the resource anyway
First apply, with Application Default Credentials from gcloud auth application-default login:
Error: Error waiting to create Link: googleapi: Error 403: Your application is
authenticating by using local Application Default Credentials. The
observability.googleapis.com API requires a quota project, which is not set by
default.
...
"consumer": "projects/32555940559",
"reason": "SERVICE_DISABLED"
That consumer project number is not yours — it is Google's shared gcloud OAuth client. The Observability API refuses to bill the call to it and demands an explicit quota project. The Terraform-native fix is on the provider, not in your shell:
provider "google" {
project = var.project_id
billing_project = var.project_id
user_project_override = true
}
Here is the nasty part. The POST succeeded; only the long-running-operation poll got the 403. So the link exists in GCP while Terraform recorded a failure and wrote nothing to state:
$ gcloud observability buckets datasets links list \
--bucket=_Trace --dataset=Spans --location=us --project=my-project
name: projects/my-project/locations/us/buckets/_Trace/datasets/Spans/links/trace_linked_dataset
Re-running apply would just fail on "already exists". Import it:
terraform import google_observability_link.trace \
projects/my-project/locations/us/buckets/_Trace/datasets/Spans/links/trace_linked_dataset
Then apply converges to zero changes. Whenever an operation-based GCP resource fails while waiting, check the API before assuming nothing happened.
Querying: --location=US, even for global
Both links produce an ordinary three-part BigQuery path. If you have seen a five-part project.global._Trace.Spans._AllSpans floating around, that is not a thing:
$ bq --project_id=my-project ls
datasetId
-----------------------
logging_linked_dataset
trace_linked_dataset
The counterintuitive bit is the location. bq --location is the BigQuery job location and has to match the dataset — it has nothing to do with the bucket the data came from. Both links here land in the US multi-region, including the one attached to a global log bucket:
$ bq show --format=prettyjson trace_linked_dataset | grep location # "US" (bucket: us)
$ bq show --format=prettyjson logging_linked_dataset | grep location # "US" (bucket: global)
bq resolves the location on its own if you omit the flag, so this is less about remembering to pass US and more about not passing the bucket's location out of habit:
$ bq query --location=global ...
BigQuery error in query operation: Location global does not support this operation.
$ bq query --location=asia-northeast1 ...
Not found: Dataset my-project:trace_linked_dataset was not found in location asia-northeast1
global is not a BigQuery location at all. Read the dataset with bq show instead of assuming the bucket location carries over.
Trace spans, note the OTel-native column names:
SELECT trace_id, span_id, name, start_time,
duration_nano / 1e6 AS duration_ms,
status.code AS status_code
FROM `my-project.trace_linked_dataset._AllSpans`
WHERE end_time > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR)
ORDER BY start_time DESC
LIMIT 100
It is name, not display_name; attributes is a JSON column, not a key-value RECORD; duration is duration_nano. The Cloud Trace v2 API schema that most documentation describes is not what you get in BigQuery — spans arrive in the OpenTelemetry-shaped schema, and a query written against the API docs returns Unrecognized name: display_name.
Logs:
SELECT timestamp, severity, log_name, resource.type AS resource_type,
COALESCE(text_payload, TO_JSON_STRING(json_payload)) AS payload
FROM `my-project.logging_linked_dataset._AllLogs`
WHERE timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR)
ORDER BY timestamp DESC
LIMIT 100
Joining traces to logs
This is the payoff, and the two views disagree on how to spell a trace ID. _AllSpans.trace_id is the bare 32-hex string; _AllLogs.trace is the full resource name projects/<id>/traces/<trace_id>:
SELECT
s.trace_id,
s.name AS span_name,
s.duration_nano / 1e6 AS duration_ms,
COUNT(l.insert_id) AS log_count
FROM `my-project.trace_linked_dataset._AllSpans` AS s
LEFT JOIN `my-project.logging_linked_dataset._AllLogs` AS l
ON REGEXP_EXTRACT(l.trace, r"/traces/(.+)$") = s.trace_id
WHERE s.end_time > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
GROUP BY 1, 2, 3
ORDER BY duration_ms DESC
Two separate linked datasets, one query, no ETL between them.
What you are actually buying
The cost argument against the old Logging sink is simple: a sink writes a second physical copy into BigQuery, so you pay log bucket storage and BigQuery storage forever. A linked dataset is zero-copy — _AllSpans and _AllLogs are views over the existing bucket, so BigQuery storage cost is zero and you only pay for bytes scanned.
What it does not buy you is retention. Spans in _Trace are kept 30 days, _Default logs 30 days, and the link does not extend either by a single hour. If you need year-over-year analysis, you still need a scheduled query copying into a real BigQuery table — the linked dataset is just a much better source for it than a sink was.
Verified on hashicorp/google 8.11.0, Terraform 1.9.5, gcloud 569.0.0 (gcloud beta observability on that version; it went GA on September 23, 2026).
Top comments (0)