Pablo Reyes

Posted on Nov 6 Edited on Nov 6 AI-assisted

How to connect to a private GCP database from your terminal, a coding agent or GitHub Actions

#googlecloud #devops #showdev #tutorial

How do you connect to your private databases on GCP?

When your databases on Google Cloud have only a private IP, you can't reach them from your laptop, coding agent or a GitHub Actions runner because they are outside the VPC.

The usual fixes are a public IP, a VPN or a bastion VM that runs all the time. At my company, Pilef, we built bqto, an open source CLI that connects you to a private database without a public IP, without a VPN and without a VM left running forever.

First I'll show how it works, and then how to use it.

How bqto works

It checks that you are logged in to gcloud and that you have the IAP permission in the project.
It asks the service's API for the private address of the instance you want to reach.
It checks that the firewall rule lets Identity-Aware Proxy (IAP) reach port 22.
It picks a zone and a subnet for the VM, and creates it.
It opens an SSH connection to the VM through an IAP tunnel and forwards a local port (for example 15432) to the instance's private IP. Your database client or your app then connects to 127.0.0.1:15432.
When you press Ctrl-C, it deletes the VM and tells you. If it can't, the VM has a deadline (30 minutes by default) and Google deletes it when the deadline passes.

Access goes through your own IAM roles, so there are no SSH keys to share.

What you need before starting

You need gcloud 550.0.0 or later, an instance with a private IP, and a firewall rule that lets IAP (35.235.240.0/20) reach port 22. The README lists the APIs and roles.

Install bqto

Pick one:

curl -fsSL https://raw.githubusercontent.com/pilef-co/bqto/main/install.sh | sh
brew install pilef-co/tap/bqto
go install github.com/pilef-co/bqto/cmd/bqto@latest

Log in to gcloud

gcloud auth login
gcloud auth application-default login

Connect from your terminal

bqto connect --project your-project --service cloudsql --instance your-db

When bqto prints ready: 127.0.0.1:15432 --> ..., point your database client or your app to that address. When you are done, press Ctrl-C and bqto deletes the VM.

For AlloyDB use --service alloydb --instance region/your-cluster/your-primary.

For Memorystore, use --service redis or --service valkey with --instance region/your-cache.

Connect from a coding agent

Install the skill

npx skills add pilef-co/bqto -g

Then ask Claude Code, Codex or your favourite agent something about your data, like "Count the orders in orders-db. It's the Cloud SQL instance in your-project". The agent runs the query through a bqto session, and bqto deletes the VM when the query ends. You can also ask it to keep the session open for several queries. If the agent doesn't have the project or the database password, it asks you.

Connect from GitHub Actions

Google's auth action signs the job in with Workload Identity Federation (WIF), so you don't store a service account key. Then the bqto action runs your command through a session:

- uses: google-github-actions/auth@v3
  with:
    workload_identity_provider: ${{ vars.WIF_PROVIDER }}
    service_account: migrations@your-project.iam.gserviceaccount.com
- uses: google-github-actions/setup-gcloud@v3
- uses: pilef-co/bqto-action@v1
  env:
    PGPASSWORD: ${{ secrets.DB_PASSWORD }}
  with:
    project: your-project
    instance: your-db
    run: ./migrate up

The action's README has the full workflow and the commands to set up WIF.

Limits

macOS and Linux only.
Each session creates a new VM, so it takes a minute or two before bqto is ready.
It's not meant for bulk transfers, because IAP isn't built for them.

bqto is open source (Apache-2.0): https://github.com/pilef-co/bqto

bqto Pro, now in beta, also adds Cloud Run, Managed Kafka, Memorystore clusters and a Mac menu bar app: https://bqto.dev/#pro.

Questions and feedback are welcome in the comments. I hope bqto helps!

DEV Community