DEV Community

quixoticmonk
quixoticmonk

Posted on • Edited on

Exploring terraform provider capabilities with schema analysis

#hashicorp #provider #aws #schemas

Terraform has become the de facto name when it comes to tools which help you model infrastructure in a declarative language, the idea of which we call as Infrastructure as Code. At a very high level, Terraform is a Go binary which knows how to do some things well. And providers are that much needed context which terraform needs to work with a platform. The platform can be a cloud service provider or an artifactory platform like jFrog and so on. Most developers only scratch the surface of what Terraform providers offer. While everyone knows about basic resources and datasources available within a provider, anything beyond that is still not easily discovered ( unless the documentation surfaces them up well).

You're deploying infrastructure with Terraform and need to invalidate a CloudFront cache after your S3 bucket updates (Like with this blog). You've heard Terraform has a new concept called "actions" for this, but how do you discover what's available? In this post, we'll explore how to use the terraform providers subcommand to identify some of the core capabilities which might exist with the provider you normally interact with.

Note: With the availability of Terraform MCP servers, these might be easier than how it was earlier.

Pre-requisites

The sections below lists commands which can provide you more details around the core capabilities( for lack of better terms) that exist with aws provider. You may use the provider that you use in your daily life.

  • jq for maneuvering around the json schema file
  • terraform binary :1.14.0-beta versions to gather some of the attributes
  • The latest version of your provider.

providers schema command

If you have ever run the terraform providers command, you will see the list of options available; out of which we are going to dive into the schema subcommand. Terraform's providers schema command exposes every capability your provider supports in a structured JSON format.

❯ terraform providers --help

Options:

  -test-directory=path  Set the Terraform test directory, defaults to "tests".

Subcommands:
    lock      Write out dependency locks for the configured providers
    mirror    Save local copies of all required provider plugins
    schema    Show schemas for the providers used in the configuration
Enter fullscreen mode Exit fullscreen mode

Warning: If you run the command terraform provider schema -json on your terminal expect a ball of text to fill your terminal screen.

Quick Setup

Let's start with a minimal Terraform configuration to explore the AWS & AWSCC provider:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
    awscc = {
      source = "hashicorp/awscc"
    }
  }
}

provider "aws" {
  region = "us-west-2"
}

provider "awscc" {
  region = "us-west-2"
}
Enter fullscreen mode Exit fullscreen mode

After running terraform init, we can extract the complete provider schema using the below command. I am going to pipe the results out to a json file so that I can perform my analysis on the available file.

terraform init
terraform providers schema -json > aws_schema.json
Enter fullscreen mode Exit fullscreen mode

Provider Capability Categories

First, let's see what capability types are available. Over the course of time, Terraform introduced many schema capabilities/types that providers adopted. To get the list of capabilities, all we need to do is find the keys inside provider_schemas.

At the time of writing this, the AWS provider supports 8 distinct capability types:

❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"] | keys' aws_schema.json

[
  "action_schemas",
  "data_source_schemas",
  "ephemeral_resource_schemas",
  "functions",
  "list_resource_schemas",
  "provider",
  "resource_identity_schemas",
  "resource_schemas"
]
Enter fullscreen mode Exit fullscreen mode

What about the AWSCC provider ? It supports 5 at the moment.

❯ jq '.provider_schemas["registry.terraform.io/hashicorp/awscc"] | keys' aws_schema.json
[
  "data_source_schemas",
  "list_resource_schemas",
  "provider",
  "resource_identity_schemas",
  "resource_schemas"
]
Enter fullscreen mode Exit fullscreen mode

Let's get some numbers

Now let's find how many resources exist in each of the provider.


❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].resource_schemas | keys | length' aws_schema.json
1544


❯ jq '.provider_schemas["registry.terraform.io/hashicorp/awscc"].resource_schemas | keys | length' aws_schema.json
1214
Enter fullscreen mode Exit fullscreen mode

There are 1544 & 1214 resources in the AWS and AWSCC providers respectively at this point. The key thing to note here is that you can replace the capability type you want to review and gather the specific detaills of those types.


# Count data sources  
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].data_source_schemas | keys | length' aws_schema.json

# Count functions
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].functions | keys | length' aws_schema.json

# Count action schemas
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].action_schemas | keys | length' aws_schema.json

# Count ephemeral resources
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].ephemeral_resource_schemas | keys | length' aws_schema.json

# Count resource identity schemas
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].resource_identity_schemas | keys | length' aws_schema.json

# Count list resources
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].list_resource_schemas | keys | length' aws_schema.json
Enter fullscreen mode Exit fullscreen mode

For ease of associating them, I have just mapped them to the capability type. You can probably do this with jq as well.

[
  "action_schemas",           // 8 actions
  "data_source_schemas",      // 622 data sources  
  "ephemeral_resource_schemas", // 7 ephemeral resources
  "functions",                // 3 functions
  "list_resource_schemas",    // 4 list resources
  "provider",                 // Provider configuration
  "resource_identity_schemas", // 212 identity schemas
  "resource_schemas"          // 1,544 resources
]
Enter fullscreen mode Exit fullscreen mode

Dive deep into a type

So all these commands are great at presenting you the details in a provider schema. Why do I need it again ? HashiConf '25 introduced two core capabilities with provider support.

  • Terraform search : Query a list of resources of certain type(s) and provide a means to import them to your state file , if you so desire.
  • Terraform actions : A HCL native way of managing pre/post- operations on resources during the lifecycle management of resources. In short, the idea is to give an alternative to some of the bespoke scripts and local-exec provisioners running in the modules you know of. This blog itself uses a cache invalidation step with CloudFront once the deployment is complete.

One of my biggest pain points with the provider documentation currently is with how these types are listed. Or rather not listed. AWS provider groups capability types by the resource types like S3 bucket and so there is no easy way to identify the number of actions or list resources supported across the provider. Once I know that a resource type exists, I can search for that one and identify the attributes to use.

Can we get that information from the providers schema? Yes, you can. Let's start with the action schema.

Question: What action resources are supported in AWS provider ?

❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].action_schemas | keys' aws_schema.json

[
  "aws_cloudfront_create_invalidation",
  "aws_codebuild_start_build", 
  "aws_ec2_stop_instance",
  "aws_events_put_events",
  "aws_lambda_invoke",
  "aws_ses_send_email",
  "aws_sfn_start_execution",
  "aws_sns_publish"
]
Enter fullscreen mode Exit fullscreen mode

Question: What about list resources ?

❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].list_resource_schemas | keys' aws_schema.json
[
  "aws_batch_job_queue",
  "aws_cloudwatch_log_group",
  "aws_iam_role",
  "aws_instance"
]
Enter fullscreen mode Exit fullscreen mode

So what if I want to know the supported actions or list resources across other providers ? You can easily change the provider configuration in the providers.tf file I had above to include your provider details and run the same commands. OR, you can take a look at my GitHub repo here. Yes, it is named provider actions before I realized I could do this for all types ( actions, list, functions, ephemeral and so on)

To drive the point home:

Discover available provider functions: 

# List all functions
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].functions | keys' aws_schema.json

[
  "arn_build",      // Construct ARNs programmatically
  "arn_parse",      // Parse ARN components
  "trim_iam_role_path"  // Clean IAM role paths
]
Enter fullscreen mode Exit fullscreen mode

Ephemeral Resources: Temporary Data

Discover ephemeral resources:

# List all ephemeral resources
❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].ephemeral_resource_schemas | keys' aws_schema.json


[
  "aws_cognito_identity_openid_token_for_developer_identity",
  "aws_eks_cluster_auth",
  "aws_kms_secrets", 
  "aws_lambda_invocation",
  "aws_secretsmanager_random_password",
  "aws_secretsmanager_secret_version",
  "aws_ssm_parameter"
]
Enter fullscreen mode Exit fullscreen mode

Review attributes in a specific resource

It doesn't stop there. You have the ability to dive deeper into a resource in a provider and understand the list of attributes expected and if needed pull the required attributes alone. How will you do it ?

Let's check how many attributes are marked as required on aws_s3_bucket resource.


❯ jq '.provider_schemas["registry.terraform.io/hashicorp/aws"].resource_schemas.aws_s3_bucket.block.attributes | to_entries | map(select(.value.required == true)) | map(.key)' aws_schema.json

[]. # None
Enter fullscreen mode Exit fullscreen mode

Conclusion

The next time you look at a provider and want to identify what it offers, I hope you will give this a try.

Note: The Terraform MCP server is adding the actions and list resources discovery into the latest iteration from the looks of it. Hopefully that makes this simpler if you use a Gen AI agent in your work.

To explore your provider schemas; replace the with yours once the provider block is updated.

# Initialize your Terraform configuration
terraform init

# Extract complete schema
terraform providers schema -json ❯ provider_schema.json

# Explore !!!
jq '.provider_schemas | keys' provider_schema.json
jq '.provider_schemas["registry.terraform.io/hashicorp/<provider_name>"] | keys' provider_schema.json
Enter fullscreen mode Exit fullscreen mode

Top comments (0)