DEV Community

Bryan Sazon
Bryan Sazon

Posted on • Edited on

4 1

Visualize Google Cloud Billing data in Grafana with BigQuery

Alt Text

Wouldn't it be nice to have your infrastructure GCP billing along with with your infrastructure monitoring dashboard?

Let's get started!

Export GCP Billing to BigQuery

Reference: https://cloud.google.com/billing/docs/how-to/export-data-bigquery

Summarized Steps

  • Create a BigQuery data set. GCP will save the Billing data here.

  • Go to Billing console and configure the Billing export.

BigQuery

First, let's ensure that the billing data are already exported.

Let's say we want to know how much cost was incurred per service from April 1, 2020. The SQL query would be this.

SELECT service.description as service, currency, sum(cost) AS cost
FROM `DATASET_NAME.gcp_billing_export_v1_*`
WHERE usage_start_time >= "2020-04-01 00:00:00 UTC"
GROUP BY service, currency
ORDER BY cost DESC

Example result

Alt Text

Grafana

Finally, use the example configuration to visualize the query in Grafana.

Alt Text

This is the equivalent query.

SELECT $__timeGroupAlias(usage_start_time, 1d),
  service.description AS metric,
  sum(cost) AS cost
FROM `DATASET_NAME.gcp_billing_export_v1_*`
WHERE $__timeFilter(usage_start_time)
GROUP BY 1, 2
ORDER BY 1, 2
  • $__timeGroupAlias and $__timeFilter are Grafana SQL macros.
  • The column that is aliased as metric will be the labels to be shown and grouped in the graph.

The End

Any question? Let me know in the comments section!

Thanks for reading!

Billboard image

Deploy and scale your apps on AWS and GCP with a world class developer experience

Coherence makes it easy to set up and maintain cloud infrastructure. Harness the extensibility, compliance and cost efficiency of the cloud.

Learn more

Top comments (2)

Collapse
 
cgavrila profile image
Cristian Gavrila

This article saved me another 10 hours of trying to get this to work.

Bryan, you are a beautiful soul. I hope you live up to 160, in the body of a 20-year-old, in a permanent nirvana state.

Collapse
 
santima10 profile image
Santiago M.A.

Great post! Thanks!

Billboard image

Try REST API Generation for Snowflake

DevOps for Private APIs. Automate the building, securing, and documenting of internal/private REST APIs with built-in enterprise security on bare-metal, VMs, or containers.

  • Auto-generated live APIs mapped from Snowflake database schema
  • Interactive Swagger API documentation
  • Scripting engine to customize your API
  • Built-in role-based access control

Learn more

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay