Deeper into Dataform 2: Other API features

In part 1 we looked at the Dataform CompilationResult and WorkflowInvocation objects. Let's follow up with a quick look at some other API endpoints that I find useful.

Workspace management - delete_workspace and create_workspace

delete_workspace is essential for Dataform instances used by large teams as it allows workspaces to be automatically deleted post-merge, saving the bi-annual "can everyone please let me know which workspaces they're finished with?" message. This can be integrated into CI/CD tools such as GitHub Actions.

from google.cloud import dataform_v1

client = dataform_v1.DataformClient()

workspace_path = client.workspace_path(
    PROJECT_ID,
    REGION,
    REPOSITORY_ID,
    WORKSPACE_ID,
)

try:
    operation = client.delete_workspace(name=workspace_path)

There's also create_workspace - very useful for automatically creating a workspace if a code change is made on a new branch outside of the Dataform UI.

Git operations - push_git_changes and pull_git_changes

Another set of useful endpoints that allow code changes to be automatically applied outside of the Dataform UI:

from google.cloud import dataform_v1

client = dataform_v1.DataformClient()

workspace_path = client.workspace_path(
    project_id,
    region,
    repository_id,
    workspace_id,
)

# ------------------------------------------------------------
# 1. Pull latest changes from Git into the workspace
# ------------------------------------------------------------
print("Pulling latest Git changes into workspace...")

pull_request = dataform_v1.PullGitChangesRequest(
    name=workspace_path
)

pull_operation = client.pull_git_changes(request=pull_request)
print("Pull initiated:", pull_operation.operation.name)

# Optional: wait for completion (simplified polling)
pull_result = pull_operation.result()
print("Pull completed successfully")

# ------------------------------------------------------------
# (Optional) 2. Make programmatic changes here
# ------------------------------------------------------------
# e.g. update config blocks, inject variables, generate models, format SQL, etc.

print("Applying automated workspace changes...")

# ------------------------------------------------------------
# 3. Push workspace changes back to Git
# ------------------------------------------------------------
print("Pushing workspace changes to Git...")

push_request = dataform_v1.PushGitChangesRequest(
    name=workspace_path
)

push_operation = client.push_git_changes(request=push_request)
push_result = push_operation.result()
print("Push completed successfully")
print("Workspace successfully synced with Git")