DEV Community

Aja for Google Cloud

Posted on • Originally published at thagomizer.com on

2 1

Cloud Build Dependency Graph

A couple of months ago, my coworker Mark Mandel asked if anyone had a way of visualizing dependencies in Cloud Build. As you’d expect, several of us jumped at the problem. I wrote my solution up in Ruby using the graph gem.

Cloud Build configuration files are in yaml. Each build step lists the container to run and some configuration options like arguments to pass and directories to use. Each step also has a waitFor field to specify dependencies between the steps. This is all the information you need to parse and understand to build the dependency graph.

Since I’d used it before I started with Zenspider’s graph gem. The graph gem uses a DSL to output the dot file format used by Graphviz. Using the graph gem, you define edges with the syntax edge "A", "B". An edge is just an arrow. In this case, the arrow is between A and B. If either A or B doesn’t exist yet, the gem automatically adds it.

The DSL makes creating the dependency graph pretty straightforward. All I had to do was parse the build config file and then use the waitFor field to draw edges between dependent steps. To make the graph easy to read, I chose to use the ID as the node label. If there was no ID field, I used the name. The only tricky bit is that Cloud Build config files don’t have to state all their dependencies using waitFor. If no dependency is given for a step, it is assumed to depend on all previous steps. This meant I had to keep track of the previous steps to get the edges correct. Putting it all together resulted in this code:

require 'graph'
require 'psych'

steps = Psych.load(File.read(ARGV[0]))["steps"]

previous_steps = []

digraph do
  steps.each do |step|
    id = step["id"] || step["name"]

    deps = step["waitFor"] || previous_steps
    deps.each do |dep|
      break if dep == "-"
      edge dep, id
    end

    previous_steps << id

  end

  save "buildgraph", "png"

end
Enter fullscreen mode Exit fullscreen mode

Gist here

Running that code against the build file Mark shared with the team leads to this dependency graph:

Agones Build Dependencies

All told it took about 30 minutes at hack night to code this up. Most of that time was reading about the Cloud Build configuration file format. If you’re curious, the dependency graph shown above is for Agones an open source game server scaling and orchestration platform built on top of Kubernetes.

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more →

Top comments (0)

Billboard image

Create up to 10 Postgres Databases on Neon's free plan.

If you're starting a new project, Neon has got your databases covered. No credit cards. No trials. No getting in your way.

Try Neon for Free →

AWS GenAI Live!

GenAI LIVE! is a dynamic live-streamed show exploring how AWS and our partners are helping organizations unlock real value with generative AI.

Tune in to the full event

DEV is partnering to bring live events to the community. Join us or dismiss this billboard if you're not interested. ❤️