DEV Community: Nirmal Chhodvadiya

One manifest, two clouds: multi-cloud infrastructure with stackql-deploy

Nirmal Chhodvadiya — Mon, 28 Sep 2026 15:44:16 +0000

Tutorial 3 introduced stackql-deploy against a single Google Cloud project. One manifest, one provider, one VPC. The framework proved its shape: declarative resources described as SQL-anchored files, an exists-create-statecheck-exports loop that runs idempotently against live cloud APIs, no state file to maintain.

This tutorial takes the same framework and puts it to work across two clouds at once. One manifest declares both a GCP VPC and an AWS VPC. One build command provisions both in the same run. The manifest becomes a single source of truth for infrastructure that spans providers, without introducing a second tool or a second DSL.

Along the way, the AWS side of the walkthrough shows features that the minimal Google example does not need: per-environment values, tag-based resource identification, resource composability through exports, and provider-specific query helpers. The interesting part is not just deploying to two clouds, but managing both through the same manifest and lifecycle.

What multi-cloud in one manifest looks like

A multi-cloud stackql-deploy project is structurally the same as a single-cloud one, with two additions: multiple providers in the providers block, and one .iql file per resource under either cloud. Scaffold two starter projects first, then combine:

stackql-deploy init multi-cloud-stack --provider google
stackql-deploy init aws-stack --provider aws

Merge the resulting manifests and copy both resource files into a single project. The combined manifest structure:

version: 1
name: "multi-cloud-stack"
description: "GCP VPC + AWS VPC in one deployment"
providers:
  - google
  - awscc
globals:
  - name: project
    value: "{{ MY_PROJECT_NAME }}"
  - name: region_aws
    value: "{{ AWS_REGION }}"
  - name: global_tags
    value:
      - Key: 'stackql:stack-name'
        Value: "{{ stack_name }}"
      - Key: 'stackql:stack-env'
        Value: "{{ stack_env }}"
resources:
  - name: gcp_vpc
    props:
      - name: vpc_name
        value: "{{ stack_name }}-{{ stack_env }}-gcp-vpc"
    exports:
      - vpc_name
      - vpc_link
  - name: aws_vpc
    props:
      - name: vpc_cidr_block
        values:
          prd: {value: "10.0.0.0/16"}
          sit: {value: "10.1.0.0/16"}
          dev: {value: "10.2.0.0/16"}
      - name: vpc_tags
        value:
          - Key: Name
            Value: "{{ stack_name }}-{{ stack_env }}-aws-vpc"
        merge: ['global_tags']
    exports:
      - vpc_id
      - vpc_cidr_block

There are two details to notice in the manifest. First, the provider list carries two entries, and the framework loads both before processing the resources. Second, the AWS resource declares per-environment CIDRs through a values block, while the GCP resource uses a simpler single-value pattern. Using an AWS-specific variable name (region_aws) also avoids collisions when provider-specific settings share generic names such as region.

The GCP side

The GCP resource file (resources/gcp_vpc.iql) follows the same shape covered in Tutorial 3. The exists check queries google.compute.networks by name. The create anchor INSERTs into the same view. For this provider method, the request-body fields use the data__ prefix while project stays plain because it maps to the URL path. Statecheck verifies autoCreateSubnetworks, and delete removes the network.

For readers who want the full anchor breakdown, Tutorial 3 walks through each block. Here the point is that adding a GCP resource to a multi-cloud stack takes exactly one .iql file. No coordination code, no shared state, no cross-provider glue.

The AWS side

The AWS resource file (resources/aws_vpc.iql) uses the awscc provider, which wraps AWS Cloud Control API. The anchor structure is the same as GCP, but the queries have real differences worth walking through.

The exists check for AWS uses tags as the resource identifier, not names:

/*+ exists */
WITH tagged_resources AS
(
  SELECT split_part(ResourceARN, '/', 2) as vpc_id
  FROM awscc.tagging.tagged_resources
  WHERE region = '{{ region_aws }}'
  AND TagFilters = '{{ global_tags | to_aws_tag_filters }}'
  AND ResourceTypeFilters = '["ec2:vpc"]'
),
vpcs AS
(
  SELECT vpc_id
  FROM awscc.ec2.vpcs_list_only
  WHERE region = '{{ region_aws }}'
)
SELECT r.vpc_id
FROM vpcs r
INNER JOIN tagged_resources tr
ON r.vpc_id = tr.vpc_id;

This is a JOIN between two AWS API surfaces. The tagged_resources view returns anything with matching stackql tags. The vpcs_list_only view returns VPCs in the region. The inner join returns only VPCs that match both, which is how the framework asks "has our stack already put a VPC here?" without needing to remember an ID from a previous run. The stack tags identify the VPC, so the framework does not need to persist the VPC ID from a previous run.

The to_aws_tag_filters filter is a stackql helper that turns the manifest's tag list into the JSON shape the AWS tagging API expects. Provider-specific helpers like this keep the manifest declarative without leaking wire-format details into the resource files.

The create INSERT uses direct column names (no data__ prefix) for this awscc method:

/*+ create */
INSERT INTO awscc.ec2.vpcs (
  CidrBlock,
  Tags,
  EnableDnsSupport,
  EnableDnsHostnames,
  region
)
SELECT
  '{{ vpc_cidr_block }}',
  '{{ vpc_tags }}',
  true,
  true,
  '{{ region_aws }}'
RETURNING *;

The AWS resource also demonstrates two useful features. The vpc_cidr_block value comes from the per-environment values block in the manifest, so this same create runs different CIDRs in dev, sit, and prd. The RETURNING * clause captures the created resource's fields into the framework's context, which is how downstream anchors can reference {{ this.vpc_id }} without another query.

Statecheck uses a custom awscc function to compare tag policies:

/*+ statecheck, retries=5, retry_delay=5 */
SELECT COUNT(*) as count FROM
(
  SELECT AWS_POLICY_EQUAL(tags, '{{ vpc_tags }}') as test_tags
  FROM awscc.ec2.vpcs
  WHERE Identifier = '{{ this.vpc_id }}'
  AND region = '{{ region_aws }}'
  AND cidr_block = '{{ vpc_cidr_block }}'
) t
WHERE test_tags = 1;

AWS_POLICY_EQUAL is a provider-specific helper used here to compare the returned tags with the desired tag policy. It keeps the statecheck compact instead of expanding the tag comparison into a much larger SQL expression.

Preview with a dry run

With both AWS and GCP credentials available in your environment, render the full multi-cloud plan first. The dry run resolves variables and prints the provider-specific SQL without creating resources.

stackql-deploy build multi-cloud-stack dev --dry-run \
  -e MY_PROJECT_NAME=your-gcp-project \
  -e AWS_REGION=ap-south-1

In the validated run, stackql-deploy loaded both google and awscc, rendered the GCP VPC queries with the method-specific data__ fields, rendered the AWS tag-based lookup and VPC INSERT for ap-south-1, and finished with dry-run build complete.

Dry run, part 1: both providers load and the GCP resource renders without creating anything.

Dry run, part 2: AWS queries render with ap-south-1, then the run ends with dry-run build complete.

Run the real multi-cloud build

After the dry run looks right, run the same stack without --dry-run:

stackql-deploy build multi-cloud-stack dev \
  -e MY_PROJECT_NAME=your-gcp-project \
  -e AWS_REGION=ap-south-1

The first validated build created both VPCs in one deployment. GCP completed the exists → create → statecheck → exports sequence, followed by AWS. The AWS exists query retried while Cloud Control finished provisioning, then captured the VPC ID. The full multi-cloud build completed successfully in 13.39 seconds in the captured run.

First real build: both gcp_vpc and aws_vpc are created, verified, exported, and the build completes.

Run it again: verify idempotence

Run the same build command again without changing the manifest. On the second validated run, both VPCs already existed, both statechecks passed, and there was no create step for either resource. The same AWS VPC ID was reused and the build completed in 4.69 seconds.

Second build: both resources already exist, statechecks pass, and nothing is recreated.

Tear down both clouds

The same project can clean up both resources with one teardown command:

stackql-deploy teardown multi-cloud-stack dev \
  -e MY_PROJECT_NAME=your-gcp-project \
  -e AWS_REGION=ap-south-1

The validated teardown collected the resource exports, deleted the AWS VPC first, then deleted the GCP VPC, and used post-delete checks to confirm that both resources were gone. The run finished with teardown complete.

Teardown: both cloud resources are deleted and confirmed absent.

When Cloud Control operations run async

AWS Cloud Control operations can complete asynchronously. In this example, the create request returns before the VPC is immediately discoverable through the tag-based exists query, so the resource definition retries the check while the API finishes provisioning.

The stackql-deploy framework handles that timing through retry settings on the exists and state checks. In this AWS resource, retries=5 with retry_delay=5 causes repeated checks at five-second intervals. Slower resources may need larger retry windows based on their normal provisioning time.

If the exists check exhausts its retries after a create, the framework reports the resource as failed. When that happens, the create might still be running in the background. To check the actual status of a stuck Cloud Control operation, query it directly through AWS CLI:

aws cloudcontrol list-resource-requests \
  --region ap-south-1 \
  --resource-request-status-filter "OperationStatuses=FAILED,Operations=CREATE" \
  --output table

The output shows recent failed CREATE requests with their StatusMessage and ErrorCode. That is useful when the build does not complete because the underlying Cloud Control operation was rejected or is still progressing. Common causes include quota limits, IAM permission gaps, and parameter validation errors.

What this enables

This run demonstrates that AWS and GCP resources can use the same deployment lifecycle while keeping their provider-specific SQL separate.

Environment values, tags, exports, state checks, and teardown remain in one project, while each .iql file reflects the API semantics of its provider. The second build demonstrated the idempotent path: both VPCs already existed, their statechecks passed, and neither resource was recreated.

Instead of maintaining separate provisioning scripts for each cloud, the project keeps the resource definitions, environment-specific values, tags, and exports in one manifest-driven workflow.

The same pattern can be extended to additional stackql providers, subject to the capabilities and method contracts of each provider. Provider-specific SQL stays in each resource file while the lifecycle remains consistent.

Where this fits in the series

Tutorial 1 introduced stackql for querying live cloud state across multiple clouds with SQL. Tutorial 2 showed how the same query engine lets agents read live state before mutating anything, one query at a time. The Actions guide put the audit pattern in CI. Tutorial 3 introduced stackql-deploy as the framework that operationalizes the query-before-mutation pattern for a full cloud stack. This tutorial takes that framework across two clouds at once, showing what a real multi-cloud manifest looks like and how the framework handles the different provider semantics under the same lifecycle.

If the pattern makes sense to you, star the stackql repository so more developers find it. Full documentation, more manifest examples, and provider-specific guides are at stackql-deploy.io.

This closes the series for now. Cloud infrastructure exposes structured APIs that can be queried and changed declaratively. Across the four tutorials, the same query layer was used to audit live cloud state, gate mutations with SQL checks, run continuous audit checks in CI, and declare a tested AWS + GCP stack through manifest-plus-anchored queries.

Declarative cloud stacks in SQL: an intro to stackql-deploy

Nirmal Chhodvadiya — Thu, 17 Sep 2026 04:03:49 +0000

Tutorial 2 in this series showed the query-before-mutation pattern by hand. Query live state, apply a policy gate, mutate what's out of policy, verify convergence. Every step was a SQL statement typed into a shell. The pattern held, but the shape was low-level. One resource at a time. One session at a time. No structure for repeating it across environments or holding it right over time.

stackql-deploy takes that pattern and turns it into a framework. Define your resources once as SQL-anchored files, declare them in a manifest, and let the framework run the exists check, the create-or-update, the state check, and the exports as a single build. Same underlying pattern, now something a team can operate at scale.

This tutorial walks through the tool end to end. What it is, how a project is shaped, how a single resource is defined, and how the whole thing runs against a real Google Cloud project.

What stackql-deploy is

stackql-deploy is a model-driven deployment framework built on top of stackql. You describe cloud resources as SQL query files (each with a few named blocks called anchors), collect them in a manifest, and run a single command to converge your cloud state to what the manifest declares. It's inspired by dbt in shape: a project directory, models as files, environments as configurations, and a small CLI that operates on the whole thing.

Three properties are worth naming up front.

Declarative. You describe what you want to exist. The tool figures out whether to create, update, or leave alone based on live queries against the cloud API, not a cached state file.

Idempotent by design. Every resource has an exists check and an optional state check. The tool skips resources that already match, updates what has drifted, and creates what is missing. Running build twice in a row produces the same converged state, but only the first run actually mutates anything.

Provider-agnostic through the stackql query layer. Because everything under the hood is stackql SQL against real cloud APIs, the same tool can deploy AWS, GCP, Azure, Snowflake, Databricks, or any other provider stackql supports. One framework, one query grammar, many clouds.

How a project is shaped

A stackql-deploy project is a directory with a manifest at the root and one .iql file per resource. Scaffolding a new project uses the init command:

stackql-deploy init example-stack --provider google

This creates a directory shaped like this:

example-stack/
├── stackql_manifest.yml
└── resources/
    └── example_vpc.iql

The manifest declares the providers, the global variables, and the resources the stack contains. Here is a minimal one for a single VPC network in GCP:

version: 1
name: "example-stack"
description: description for "example-stack"
providers:
  - google
globals:
  - name: project
    value: "{{ MY_PROJECT_NAME }}"
  - name: region
    value: australia-southeast1
resources:
  - name: example_vpc
    props:
      - name: vpc_name
        value: "{{ stack_name }}-{{ stack_env }}-vpc"
    exports:
      - vpc_name
      - vpc_link

The globals block holds values that apply across every resource. The template syntax reads environment variables or values passed at build time. The resources block lists each thing to deploy, with per-resource properties that get substituted into the SQL below.

How a resource is defined

Each .iql file holds one resource. Inside the file, named blocks (anchors) declare the different query shapes the framework runs at different points in the lifecycle:

Read the anchors in order. exists is a live query that answers whether the resource is already there. create is the mutation to run if it isn't. statecheck runs after create to confirm the resource actually reached the desired configuration, with retries and delays for slow-to-propagate cloud APIs. exports captures values to reuse in other resources within the same stack. delete is the reverse mutation for teardown.

A note on the INSERT syntax. The columns split between path parameters (project stays plain since it maps to the URL path) and body fields (data__name, data__autoCreateSubnetworks, data__routingConfig get the data__ prefix). This is method-specific rather than universal across providers, and you can confirm the exact contract for any resource with:

DESCRIBE METHOD google.compute.networks.insert;

This is the same query-before-mutation loop Tutorial 2 walked through by hand, formalized into a resource-file structure. The framework runs it for every resource in the manifest, in dependency order, with proper handling for the states each resource can be in.

Running a dry-run build

Install stackql-deploy via cargo:

cargo install stackql-deploy

Then, from the parent directory of your stack, run build in dry-run mode with a stack name, an environment name, and your project ID:

stackql-deploy build example-stack dev --dry-run -e MY_PROJECT_NAME=your-project-id

The framework loads the manifest, resolves the globals with your environment variables, then walks each resource in order. In dry-run mode, it renders every query it would execute against the cloud API and reports what it would do, without actually mutating anything. All the template variables are resolved: {{ project }} becomes your project ID, {{ vpc_name }} becomes example-stack-dev-vpc, and the routingConfig JSON is inlined into the INSERT.

Dry-run is the first thing to reach for. It gives you the full execution plan without touching your cloud, which makes it the natural pattern for pull-request checks and pre-deploy previews.

Running for real

Once the dry-run output looks right, drop the flag to execute:

stackql-deploy build example-stack dev -e MY_PROJECT_NAME=your-project-id

The framework runs the same sequence, but this time the exists check hits the cloud API, and if the resource is missing, the create INSERT is sent for real. statecheck retries until GCP reports the resource has converged to the desired configuration, then exports capture the live values:

Read that output top to bottom. exists returned no match, so create ran. Right after create, the framework re-ran the exists check and confirmed the VPC was there. statecheck ran once and passed. exports captured vpc_name and vpc_link. Total time: under 4 seconds for a single-resource stack. The same pattern scales to a stack with dozens of resources across multiple providers.

For test-only runs (checks without mutations, useful in CI):

stackql-deploy test example-stack dev -e MY_PROJECT_NAME=your-project-id

And to reverse the whole stack:

stackql-deploy teardown example-stack dev -e MY_PROJECT_NAME=your-project-id

What this enables

The obvious use is provisioning: describe a stack once, deploy it across dev, sit, and prod environments by changing one flag. But the deeper value is what the anchor structure enables.

Because every resource has an exists check and a state check, running build against an already-deployed stack is a policy convergence run. If someone changed a firewall rule out of band, the next build detects the drift via statecheck and re-applies the correct configuration. Nothing else on the stack gets touched. That's continuous convergence without needing a separate drift-detection tool.

Because everything is SQL and the query grammar is the same across every provider, extending a stack from GCP-only to multi-cloud is a matter of adding new .iql files and provider entries in the manifest. No new tool to learn, no separate DSL, no provider-specific templating conventions.

And because the framework is CI-friendly (test for validation without mutation, dry-run for previews, exit codes for pass/fail), it slots into whatever pipeline you already use to ship infrastructure changes.

Where this fits in the series

Tutorial 1 introduced stackql for querying live cloud state across multiple clouds with SQL. Tutorial 2 showed how the same query engine lets agents read live state before mutating anything, one query at a time. The Actions guide put the audit pattern in CI. This tutorial shows the framework that operationalizes the whole pattern: not one query at a time, but a manifest-driven declaration of an entire cloud stack, with idempotence, environment scoping, and continuous convergence built in.

If the pattern makes sense to you, star the stackql repository so more developers find it. Full documentation, more manifest examples, and provider-specific guides are at stackql-deploy.io.

Next in this series: authoring your own multi-cloud stack from scratch, using stackql-deploy against AWS and GCP together.

Add continuous cloud audit to your CI/CD in ten minutes

Nirmal Chhodvadiya — Tue, 08 Sep 2026 14:48:48 +0000

Cloud security reviews are usually quarterly. Configuration drift happens daily. Between review cycles, buckets get created, firewall rules get loosened, service accounts get reused, and a bucket that was locked down last quarter might not be locked down today. Nothing about the review cadence catches any of it.

Tutorial 1 in this series walked through auditing three clouds with stackql from your terminal. Tutorial 2 showed how the same query engine lets an agent read live state before mutating anything. This guide takes the same audit pattern and puts it in your CI, so the check runs every day, or on every pull request, or every hour, without anyone remembering to run it.

The tool is stackql-audit-action, a GitHub Action that runs an opinionated set of security checks against your cloud accounts and renders findings inline on the workflow run page. Setup is copy-paste. First results in under two minutes.

What it checks

The action ships with seven built-in checks for GCP:

SSH (22/tcp) open to the internet — HIGH
RDP (3389/tcp) open to the internet — HIGH
Cloud SQL instances reachable on a public IP — HIGH
Compute instances with public IPs — MEDIUM
Compute instances using the default service account — MEDIUM
Storage buckets without uniform bucket-level access — MEDIUM
Default VPC network exists — MEDIUM

Similar built-in check sets exist for AWS and Azure. A provider only runs if its credentials are supplied, so you can start with GCP and add the others by dropping in their secrets. This walkthrough covers GCP end to end.

Setup

You need three things:

A GitHub repository
A GCP project you're authorized to audit
A Google Cloud service account JSON key with read access to Compute, Cloud SQL, Storage, and IAM

Store the service account key as a GitHub secret. In your repository, go to Settings → Secrets and variables → Actions → New repository secret. Name the secret GCP_SA_JSON and paste the full JSON key as the value.

Never commit credentials to the repo. Never share a key JSON on Slack or email. Rotate the key if you ever paste one somewhere by accident.

Add the workflow

Create .github/workflows/cloud-audit.yml with this content:

name: GCP Audit

on:
  workflow_dispatch:
    inputs:
      project-id:
        description: GCP project ID to audit
        required: true
      fail-on-severity:
        description: Severity threshold to fail on (CRITICAL/HIGH/MEDIUM/LOW/NONE)
        required: false
        default: HIGH

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: stackql/stackql-audit-action@v0.1
        with:
          project-id: ${{ inputs.project-id }}
          gcp-credentials: ${{ secrets.GCP_SA_JSON }}
          fail-on-severity: ${{ inputs.fail-on-severity }}

Two things worth noting. The action is pinned to @v0.1 for reproducibility. If you use @main, tomorrow's audit runs against tomorrow's version, which can change without warning. Pin to a real tag or a commit SHA. The fail-on-severity input controls when the workflow fails. For a first run, set it to NONE so the workflow completes even if findings surface. Once you've reviewed the output and confirmed the checks match your expectations, raise it to HIGH or MEDIUM as an enforcement gate.

Run the audit

Commit and push. Then in your repository, go to Actions → GCP Audit → Run workflow. Fill in the project ID and start the run. First run takes about 20-30 seconds.

When the run completes, click into it and scroll down. The audit summary renders inline as a markdown table on the workflow run page:

Every check reports its status. Green checkmarks mean the check ran and found nothing. Yellow (MEDIUM) or red (HIGH) markers mean findings surfaced, with the affected resource names listed in a table and a short remediation note underneath.

In this test run, the audit surfaced nine medium-severity findings across three categories: one Compute instance using the default service account, one Compute instance with a public IP, and seven buckets without uniform bucket-level access. The other four checks ran and returned no findings. That distinction matters. A check running cleanly is signal too.

Making it continuous

Manual triggers are for testing. For continuous audit, change the trigger to run on a schedule or on pull requests:

on:
  schedule:
    - cron: '0 9 * * *'   # every day at 09:00 UTC
  pull_request:
    branches: [main]
  workflow_dispatch:      # keep manual trigger as a fallback
    inputs:
      project-id:
        description: GCP project ID to audit
        required: true

The scheduled run catches drift that happens outside your PR flow. The pull-request run catches policy violations before they merge. Together they cover both the intentional and the accidental changes that can widen your attack surface between security reviews.

Extending it with your own checks

The seven built-in checks are a starting point, not the full set your organization cares about. The action lets you point at your own directory of check definitions, one YAML file per check:

id: my-org-firewall-check
name: Allow only known source ranges
severity: HIGH
description: Catch firewall rules with sourceRanges outside our corp CIDRs.
remediation: Restrict to known CIDRs or migrate to IAP.
query: |
  SELECT name, network, sourceRanges
  FROM google.compute.firewalls
  WHERE project = '${PROJECT_ID}'
    AND direction = 'INGRESS'
columns: [name, network, sourceRanges]

A query returning zero rows means no findings. Any rows returned become finding rows. Every check is a small YAML file, so extending the audit to cover your organization's specific policies is one file per rule. See the queries/ directory in the action repository for the built-in checks as reference implementations.

Where this fits

Tutorial 1 in this series introduced the audit pattern manually: SELECT across three clouds, spot the risky resources, know what to fix. Tutorial 2 introduced query-before-mutation: read live state before an agent acts, apply a policy gate, converge safely. This guide connects the two. The audit runs continuously in CI. When something drifts, the query-before-mutation pattern is how an agent or an operator brings it back into policy.

If the pattern makes sense to you, star the stackql-audit-action repository so more developers can find it. And if you have suggestions for checks that belong in the built-in set, the queries/ directory takes pull requests.

Query before you mutate: how agents should touch your infrastructure

Nirmal Chhodvadiya — Tue, 01 Sep 2026 09:53:58 +0000

The problem with plan, review, apply

Infrastructure-as-code was designed around human-speed changes: write a plan, review it, apply it, and use state snapshots to know what was managed before. That works best when writers are few and changes are infrequent.

Agents change those assumptions. They run continuously and multiple actors can touch the same resources at the same time. Drift is no longer exceptional. It is unavoidable and relentless.

Canonical IaC tools like Terraform still matter because they represent intent. What's weakened is their ability to reflect current reality. Agents rarely own the whole picture. They usually operate in narrow scopes, working on a slice of infrastructure that other agents, other tools, or humans are also touching. In that world, an agent will regularly encounter infrastructure that was mutated out of band, outside whatever IaC system nominally manages it.

Query-before-mutation handles this case. Read live cloud state, compare it with policy, apply a bounded policy gate, mutate only what is out of policy, then verify. Each run starts from reality rather than a cached view.

The demo takes about ten minutes and uses Google Cloud Storage bucket encryption, but the pattern is provider-agnostic.

Query before mutation

Correctness comes from querying live state before acting. Safety comes from something separate: policy gates that bound what a single run is allowed to do. Location locks, resource caps, budget ceilings, allowlists. Two patterns, two jobs. Agentic query-before-mutation complements centralised IaC approaches leveraging snapshots of state. Policy gates replace the review meeting.

The combined loop is correct by default because it reads reality, and safe by default because it cannot do more than the gate allows. Agents can run it continuously. Humans can run it manually. The output converges either way.

stackql implements this pattern. Any resource on any supported cloud can be read with SQL and mutated with SQL where the provider exposes mutation methods. Kubernetes solved reconciliation inside the cluster, but nothing reconciles across your cloud. That gap is where this pattern belongs. The demo uses Google Cloud Storage because the mutation surface is clean, but the shape is the same for AWS S3, Vault secrets, GitHub org settings, or anything else an agent needs to reason about.

Setup

You need three things:

Docker (to run stackql without installing anything else)
A Google Cloud project with credentials that can read buckets and update encryption config, plus an existing Cloud KMS key you want to enforce
About ten minutes

Clone the tutorial folder from the stackql repo:

git clone https://github.com/stackql/stackql.git
cd stackql/docs/tutorials/query-before-mutation-01

Copy the sample environment file and add your service account credentials:

cp .env.sample .env
# Then edit .env to set:
# GOOGLE_CREDENTIALS=<service-account-json-on-one-line>

Bring up an interactive stackql shell:

docker compose run --rm --entrypoint bash stackql
# then inside the container:
stackql shell

Pull the Google provider from the registry. This only needs to happen once per shell session:

REGISTRY PULL google v26.07.00432;

You're ready.

Query the live state

Ask the GCS API which buckets exist and what encryption is currently applied to each.

SELECT name, location, encryption 
FROM google.storage.buckets 
WHERE project = 'your-project-id';

Expected output on a project with a mix of encryption configurations:

| name                       | location | encryption                                    |
| demo-app-bucket1           | US       | null                                          |
| demo-app-bucket2           | US       | null                                          |
| stackql-demo-src-bucket    | US       | null                                          |
| stackql-encrypted-bucket-1 | US       | {"defaultKmsKeyName":"projects/.../keys/..."} |

Three buckets show encryption as null, meaning Google-managed encryption keys (GMEK), the default. One already uses a customer-managed key. Under a policy that requires customer-managed keys, three buckets are out of policy.

A state snapshot can tell you what Terraform previously configured. This query tells you what GCS reports now, regardless of who created or changed the bucket. If ten agents and two humans are all touching this project at various points during the day, the query still returns the current truth every time you run it. That's the property the pattern is built on.

Apply a policy gate

Three buckets don't match policy. Before mutating, the agent needs to prove it's allowed to. That's what the policy gate does.

A gate is anything that bounds what a single run of the agent can do. Scope by project or location, cap the number of resources changed, or enforce an explicit allowlist. The shape depends on how much freedom the agent has and how much you're willing to lose in one bad run.

For this demo, two gates cover most of the risk. The first is a location lock. The agent should only ever act on buckets in the location it was configured for. This is a WHERE clause on every query and mutation:

WHERE location = 'US'

Trivial when the project's buckets are in one location. In production, agents move between locations based on task, and touching a bucket in the wrong region can mean acting on resources in an environment the agent has no business in. One clause, but it's the difference between a bounded action and an unbounded one.

The second gate is a count cap. Before running a mutation across multiple buckets, ask the API how many the mutation would touch:

SELECT COUNT(*) 
FROM google.storage.buckets 
WHERE project = 'your-project-id' 
AND location = 'US' 
AND encryption IS NULL;

If the query returns 20 buckets and the agent's policy expects to be operating in a project with 3 to 5 non-compliant buckets, something has changed. Maybe an account merger, maybe a script created buckets out of band, maybe the credentials point to the wrong project entirely. The agent shouldn't proceed. It should log the discrepancy and wait for a human to look.

Gates like these aren't clever. They're routine, boring, and easy to skip. They're also the reason a query-before-mutation loop can run without a human review meeting attached to every apply. The gate carries the weight the meeting used to.

Mutate to converge

The gates passed. For each bucket without a customer-managed key, apply the policy:

UPDATE google.storage.buckets 
SET data__encryption = '{"defaultKmsKeyName":"projects/your-project-id/locations/us/keyRings/your-ring/cryptoKeys/your-key"}' 
WHERE bucket = 'demo-app-bucket1';

Two things worth noting. The data__encryption prefix is how stackql passes fields into the request body, distinguishing them from URL parameters. The WHERE clause uses bucket rather than name, because bucket is the identifier the underlying patch method requires.

Expected output:

The operation was despatched successfully

In an agent loop, this runs as a bounded iteration over the non-compliant buckets returned by the earlier query, applying the same UPDATE to each one.

Verify convergence

A successful mutation is not enough. Query the bucket again to verify:

SELECT name, encryption 
FROM google.storage.buckets 
WHERE bucket = 'demo-app-bucket1';

Expected output:

| name             | encryption                                                                    |
| demo-app-bucket1 | {"defaultKmsKeyName":"projects/your-project-id/locations/us/keyRings/..."}    |

The bucket converged from GMEK to the required customer-managed key.

Run the loop again and something interesting happens: the mutation is a no-op. The SELECT finds no drift, the count check passes trivially, no UPDATE fires. That's idempotence by design. The agent can run every minute, every hour, every day, and it does the same right thing whether the target is already compliant or has just drifted. No state snapshot needed to remember what it did last time. Every run starts from live reality.

The same pattern from an agent

Everything above is you typing SQL into a shell. The same pattern maps directly to an agent through stackql's MCP server, listed in the Claude directory at claude.ai/directory/ant.dir.gh.stackql.stackql.

Install it, add your GCP credentials, and ask Claude in plain English:

List all GCS buckets in the project stackql-demo, and for each 
show whether encryption is a customer-managed key or the 
Google-managed default.

Claude uses the stackql MCP server to run the same underlying SELECT you saw earlier, then formats the result into a readable table. The query, gate, mutate, and verify loop runs the same way an agent runs it, just as reasoning steps in a conversation instead of shell commands.

Notice what the agent adds that the raw shell output doesn't: it separates CMEK buckets from Google-managed default ones, calls out which KMS key each CMEK bucket uses, and flags a caveat you didn't ask about (the difference between bucket-default encryption and per-object encryption). That's the value of running the pattern through an agent rather than a script. Same query, same result, plus the reasoning layer on top.

The shell walkthrough above was for you to see the pattern happen line by line. The MCP integration is how the pattern runs end to end in an actual agent's workflow.

Where this goes next

Query before mutation is one instance of a broader pattern. Six patterns, really, that hold together as a working model for how agents should operate on infrastructure. Query before mutation is one. Idempotent assertions, bi-temporal views, agents as event sources, universal interfaces, and policy gates are the other five. Together they replace the assumptions the plan-review-apply model was built on.

The full argument is in Jeff Aven's talk from AI Engineer Melbourne, "Treating Infrastructure as Data: Building an AI-Native Control Plane." If the pattern in this tutorial made sense to you, the talk is the map for what else fits with it.

If this tutorial was useful, star the stackql repo. Every star helps other developers and agents find the pattern, and helps the project climb the MCP registry where more agents can discover it.

Next in this series: idempotent assertions with stackql-deploy. Same shape, different pattern, applied to declarative infrastructure that runs continuously without breaking on repeat.

Auditing three clouds without writing three scripts

Nirmal Chhodvadiya — Tue, 18 Aug 2026 12:49:44 +0000

If you've audited buckets in more than one cloud, you already know what this is like. Three consoles. Three CLIs. Three auth patterns. And a bit of glue code to pull the outputs together every time someone asks a question that spans all three.

The question is usually simple. "Which buckets are public across all our clouds?" You end up running three separate scripts, then reconciling three different output shapes just to give one answer.

This tutorial doesn't make that faster. It replaces it. One query, one view, all three clouds.

What StackQL is, in three sentences

StackQL is SQL for cloud APIs. You write a normal SQL query, and it makes the API calls to AWS, GCP, Azure, or dozens of other providers to get the answer back as rows and columns.

The important part for this tutorial: it queries the live provider APIs on every run. There's no local database, no cache, no sync job to schedule. What you see is the current state of your cloud, right now.

It exists because someone got tired of writing the same audit logic three times.

What you'll need

You'll need Docker installed and read-only credentials for whichever clouds you want to audit. All three are optional. If you leave a cloud's credentials blank, it gets skipped.

For AWS, GCP, and Azure, the audit needs a service account or role that can list storage buckets and read their configuration. The audit-action repo has the exact permissions per provider here.

Run the audit

The tutorial folder has three files: a docker-compose file, an .env.audit.example template, and a README. Grab the folder from docs/tutorials/preview-bucket-01/ in the stackql/stackql repo.

1. Copy the env template

cp ./audit/.env.audit.example ./audit/.env.audit

Then open .env.audit and fill in credentials for whichever clouds you want. The variables are named for what they are: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, GOOGLE_CREDENTIALS (the service account JSON, on one line), and GOOGLE_ORG_ID.

If you're only auditing one or two clouds, leave the others blank.

2. Pull the StackQL Docker image

docker compose -f docker-compose.bucket.audit.yaml pull

The tutorial pins to stackql/stackql:v0.10.601. The audit uses views under stackql_preview.* which are still evolving, so pinning matters for reproducibility.

3. Run the audit

docker compose -f docker-compose.bucket.audit.yaml run --rm stackql

On a modest account this takes about 30 seconds. Output goes straight to your terminal as a table.

What the output tells you

When the audit finishes, you get a single table printed to your terminal. Something like this (bucket names are placeholders):

Example output from the audit. Bucket configuration across three cloud providers in one normalised table.

The point isn't the specific findings. It's the shape.

Three different resource types (S3 buckets, GCS buckets, and Azure Storage Accounts) normalised into one table with the same columns. Encryption class, public flag, HTTPS enforcement. Side by side, for every cloud you provided credentials for.

A few patterns to notice in the table:

Public buckets across providers. Filter by public = true and you have the answer to your security team's question, in every cloud, in one place.
Encryption side by side. The encryption_class column shows which buckets use provider-managed keys versus customer-managed. Different providers have different naming conventions internally; StackQL normalises them to the same two categories.
HTTPS enforcement in one column. Whether it's S3 bucket policies, GCS uniform access, or Azure secure transfer, same column, same values, no translation needed.

This is what SQL for cloud APIs actually delivers. Not just SQL as a query language, but a uniform data model across providers, so you can ask one question and get one answer.

Performance and scope

StackQL runs each provider's query in parallel using the credentials you provided. There's no local database being populated and no background sync; every run hits the live APIs and returns what's there right now.

On the test account this took about 30 seconds. The query is scoped to one AWS region and one GCP organization, which keeps the API footprint bounded and reflects the current preview default. Streaming output and broader audits (across regions, deeper across resource types) are in flight from the StackQL team.

What this covers today

Today, this audit covers storage buckets across AWS, GCP, and Azure. One region at a time for AWS, one organization at a time for GCP. Entitlements, IAM, and other resource types are on the roadmap as separate audits under stackql_preview.* and will get their own tutorials as they land.

Where to go next

github.com/stackql/stackql is the flagship repo. If this tutorial was worth reading, give it a star. It's genuinely the thing that tells the team to build more tutorials like this one, and it helps other engineers with the same three-console problem find the project.