DEV Community

Spacelift team for Spacelift

Posted on

How to Evaluate an IaC Platform

An infrastructure as code (IaC) platform sits above Terraform, OpenTofu, and the other tools your team writes infrastructure in. It runs the plans, enforces the policies, tracks the state, and coordinates changes across teams. Provisioning is no longer the hard part. Coordination, governance, and review are.

That makes IaC platform selection an architecture decision rather than a tooling preference. This guide covers six criteria, the scenario that tests each one, and what to confirm before you sign.

Do you need an IaC platform?

Not every team does. If you run a handful of environments with Terraform or OpenTofu in a CI pipeline with remote state, that setup works.

You outgrow it when coordination replaces provisioning as the main problem. The signal is not headcount. It is how much engineering time goes to questions like who can deploy what, how changes get reviewed, what happens when two teams modify the same shared service, where policy is enforced, and why a run failed. When those questions consume time every week, you have an operations problem, and an IaC platform is what addresses it.

1. The execution layer

Most IaC platforms present a similar dashboard. The differences appear in how they execute.

Ask each vendor how plans run, whether execution scales during busy periods, whether workers run inside your own network, how credentials are stored, and what happens when dozens of changes queue at once. Confirm the platform runs every IaC tool your team already uses, including Terraform, OpenTofu, CloudFormation, Pulumi, and Kubernetes, from one control plane. A platform that covers one tool moves the coordination problem instead of removing it.

Then test it rather than watching a demo. Trigger concurrent plans against a workload that resembles your production environment and watch what queues.

2. Dependency management between stacks

Infrastructure changes rarely happen in isolation. Networking affects clusters, shared services affect application environments, and identity changes affect everything. If the platform does not model those relationships, engineers coordinate deployment order manually.

Test this directly. Change a shared component, trigger the dependent stacks, and check whether the platform resolves the execution order without help. At a dozen stacks, dependency management looks like a minor feature. At several hundred, it determines whether the platform coordinates releases or your team does.

3. Where policy enforcement runs

Policy as code is on every vendor's feature list. Where the check actually runs is the part they leave vague, and it is the part that decides whether a bad change ships.

Confirm that the platform can restrict who deploys, require approvals, inspect a plan before execution, enforce organizational standards, and block risky changes automatically. A policy check that runs in CI after the fact produces a report. A policy that reads the plan and stops the apply prevents the change.

Spacelift enforces policy with Open Policy Agent at the point of execution. Plan policies read the plan before the apply, approval policies gate deployments, and push policies control what a commit starts. Spaces and role-based access control handle who reaches which stacks. The rules apply to every deployment rather than only the pipelines someone configured by hand.

4. Drift detection and remediation

Someone will change a resource in the console. The question is how long the platform takes to notice and what it does next.

Ask whether detection runs on a schedule or continuously, whether it covers every stack by default or only the ones a team opted into, and whether remediation happens automatically or waits for approval. Test it by making a manual change outside the platform and timing the response. A platform that reports drift in a dashboard nobody opens is a platform that does not detect drift.

5. Usability outside the platform team

IaC platforms fail operationally more often than technically. A platform engineer builds a workflow that works, adoption stalls, and infrastructure requests return to the same person. The bottleneck stays where it was, now with a license fee attached.

During the evaluation, give an application developer a real task such as provisioning a new environment. If they cannot determine where to start, documentation will not resolve it. Look for self-service that produces Golden Paths, where developers get what they need and your policies still apply.

6. Failure recovery and support quality

An IaC platform becomes a production system once rollout finishes. Interrupt a run partway through, watch the recovery, and check what the state file looks like afterward. Then find out who helps when that goes badly.

Diagnosing a failed deployment requires an engineer who understands runner architecture, cloud providers, and networking. Share an architecture diagram, describe a failure you have experienced, and judge the quality of the answer rather than the response time. You are evaluating the engineering organization as much as the product.

Run the six scenarios

Feature matrices describe what a vendor built. Scenarios show how the platform behaves. Give every vendor the same six tasks, one for each criterion above:

  1. Process concurrent infrastructure changes

  2. Model dependent deployments

  3. Enforce an organizational policy

  4. Detect infrastructure drift

  5. Provision a new environment as a developer

  6. Recover from a failed deployment

Use the same workload for every vendor. Differences that matter show up under load, not in a sandbox.

What to confirm before you sign

Two questions do not show up in a trial, and both get expensive later.

Pricing

The billing metric affects behavior more than the list price does.

Per-resource pricing, also called resources under management, bills every managed resource in state whether or not it changes, metered against your hourly peak. Resources declared individually each count, so a security group whose nine rules are separate rule resources bills as 10. The model charges for infrastructure inventory and works against consolidation and lean module design.

Per-run pricing bills activity, but the billable unit varies: some vendors exclude drift checks and pre-plan policy rejections, others bill every pull request plan. Get the exclusion list in writing.

Concurrency pricing, which Spacelift uses, bills per private worker with unlimited users. The invoice does not change when state grows, when engineers join, or when drift detection runs more often. The trade-off is that concurrency is the thing you have to size correctly: buy too little and runs queue at your busiest hour, which is exactly the failure the first scenario tests for. Size against peak rather than average, and confirm what adding a worker costs mid-term.

Whichever model you choose, get the billable unit defined in writing, along with the list price of every tier you might grow into. Unpublished tiers are where renewal increases appear.

AI-driven changes

AI already generates infrastructure code. The question for platform selection is how much responsibility the AI holds after the code exists. Determine whether an agent can open a pull request or apply a change, whose identity it acts under, which resources it can reach, and what the audit trail records. Without an answer, you approve every AI-generated change by hand.

Spacelift Intelligence covers this through Intent and Infra Assistant. Developers reach Intent over the Spacelift MCP server from the AI client they already use, such as Claude Code, Cursor, or VS Code. Infra Assistant provides the same capability inside the Spacelift UI and also explains failed runs. Access is scoped at authentication, so a read-only session inspects your account without changing anything, and every write is checked against the Intent policies attached to the project. IaC remains the system of record for production.

What to measure after rollout

Six months in, three outcomes indicate the evaluation worked. Developers change infrastructure without waiting on the platform team. Platform engineers spend their time building rather than coordinating deployments. Security gets consistent governance without becoming the next bottleneck.

Infrastructure keeps growing. The manual process around it should not. Every scenario in your evaluation should trace back to one of those three outcomes.

Book a demo to run your evaluation scenarios against Spacelift.

Top comments (0)