Straight Forward Tofu

Chris Gradwohl — Thu, 19 Sep 2024 14:27:35 +0000

In this entry level blog post, we will learn the basics of provisioning infrastructure with OpenTofu by building an application logging service using AWS EventBridge and AWS CloudWatch.

Here is a high level architecture diagram of what we will build in this blog post. Let's dive right in!

Repo Link

If you want to skip to the code, checkout the repo here: https://github.com/cloudspark-io/tofu_log_service

Prerequisites

To follow along in this post, you will need an AWS Account and the AWS CLI configured.

Let's Go!

Head over to https://opentofu.org/docs/intro/install/ and install OpenTofu for your operating system.

For Mac:



brew update

brew install opentofu

tofu --version

Let's setup a repo for our project.



mkdir tofu_log_service &&

cd tofu_log_service &&

touch main.tf &&

git init &&

git add . &&

git commit -m "hello tofu!"

Configure the Provider

OpenTofu, like Terraform, relies on software plugins called providers to interact with cloud providers, SaaS providers, and other APIs. Each provider adds a set of resource and data blocks that we can use to provision our infrastructure.

The first thing we need to do is configure Tofu to use the AWS provider. To do that, we still need to use the terraform block and reference the hashicorp/aws provider, even though we are using the OpenTofu language. OpenTofu has kept these naming conventions so that it can remain backward compatible with existing Terraform projects. The provider source code is actually owned and hosted on the OpenTofu registry, making OpenTofu truly backwards compatible and open source.

Open main.tf and add the following to configure Tofu to use the AWS provider.



terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  required_version = ">= 1.8.0"
}

provider "aws" {
  region = "us-west-1"
}

Next let's learn how to add infrastructure resources to our Tofu configuration.

The `resource` Block

To create and configure an infrastructure resources we use the resource block. We write a collection of resource blocks to describe one or more infrastructure objects, such as a Lambda function, an EC2 instance, or a higher-level module such an EKS Cluster.

All resource declarations use the following syntax:



resource "resource_type" "local_name" {

  # Configuration block

}

resource_type: Specifies the type of resource you want to manage (e.g., aws_instance, aws_s3_bucket). It defined by the AWS provider we configured a moment ago in the the ‘terraform’ block.
local_name: A unique identifier within the Tofu configuration for this resource. The local_name is used to refer to this resource from elsewhere in the same Tofu project, but has no significance outside that module's scope.
The resource_type and local_name together serve as an identifier for a given resource and so must be unique within a module.
The Configuration block defines the specific resources settings and properties. It contains key-value pairs that specify the desired state or attributes of the resource being created.

Creating EventBridge Resources

Let's create an EventBridge rule for our logging service. This resource will define the API of our service. Add the following to main.tf



resource "aws_cloudwatch_event_rule" "service_log_rule" {  
  name        = "service-log-rule"  
  description = "EventBridge rule to capture logs from any application service and send to CloudWatch Log Group."  

  event_pattern = jsonencode({  
    "detail" = {  
      "env"     = [{ "wildcard" : "*" }],  
      "level"   = ["info", "warn", "error"],  
      "message" = [{ "wildcard" : "*" }],  
      "service" = [{ "wildcard" : "*" }]  
    },  
    "detail-type" = ["service.log"],  
    "source"      = [{ "wildcard" : "*" }]  
  })  
}

This resource block defines an aws_cloudwatch_event_rule which defines the properties of the log events that our service will be listening for. All events emitted to EventBridge that match this event schema will be forwarded to our CloudWatch log group.

To learn more about how EventBridge filters events check out this documentation: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-events.html

Creating CloudWatch Resources

Next, let's create our CloudWatch resources, which will serve as the destination for our log events.

Add the following to main.tf



resource "aws_cloudwatch_log_group" "log_service_cw_log_group" {
  name = "/aws/events/log-service"
}


resource "aws_cloudwatch_event_target" "services_event_target" {
  rule = aws_cloudwatch_event_rule.service_log_rule.name
  arn  = aws_cloudwatch_log_group.log_service_cw_log_group.arn
}

The first resource block defines an aws_cloudwatch_log_group to store our incoming logs.

The second resource block defines an aws_cloudwatch_event_target which maps the EventBridge rule we defined earlier to the CloudWatch Group.

Note how we use the unique identifier of our EventBridge rule, consisting of the resource_name and local_name, to pass the name value to the CloudWatch configuration.

The last thing we need to do is to give EventBridge permissions to write into our new log group. One way we can do that is to create a resource based policy for our CloudWatch Group. Let's review the data block in OpenTofu and see how it can simplify writing this IAM policy.

## The `data` Block

Data sources in OpenTofu allow us to reference or create information from external resources that aren’t available to reference otherwise.

All data block declarations use the following syntax:



data "source_type" "local_name" {

  # Configuration block

}

source_type: Specifies the type of data you want to create or reference from the data source. Just like **resource_types**, it is defined by the AWS provider from OpenTofu.
local_name: A unique identifier within the Tofu configuration for this data source. The local_name is used to refer to this data source from elsewhere in the Tofu project.
The source_type and local_name together serve as an identifier for a given data source and so must be unique within the Tofu project.
The Configuration block specifies identifier information for the data lookup or creation. It contains key-value pairs that specify the desired state or attributes of the resource being managed.

Now let’s get back to that IAM policy.

Creating IAM Resources

Add the following to your main.tf file.



data "aws_caller_identity" "current" {}

data "aws_iam_policy_document" "cw_log_group_policy" {  
  statement {  
    actions = [  
      "logs:CreateLogStream",  
      "logs:PutLogEvents",  
    ]  
    resources = [  
      "${aws_cloudwatch_log_group.log_service_cw_log_group.arn}:*"  
    ]  
    principals { # the identity of the principal that is enabled to put logs to this account.  
      identifiers = ["events.amazonaws.com"]  
      type        = "Service"  
    }  
  }  
}

The first data block defines an aws_caller_identity and allows us to reference information about the AWS Account that Tofu is currently configured in. We can fetch information like the Account ID, as well as User IDs and ARNs.

The second data block defines an aws_iam_policy_document and allows us to create an IAM policy document in JSON format.

With the help of these data blocks, it now becomes very easy to create our CloudWatch resource policy. Add the following resource block to your main.tf file after the two data blocks.



resource "aws_cloudwatch_log_resource_policy" "eventbridge_log_policy" {  
  policy_document = data.aws_iam_policy_document.cw_log_group_policy.json
  policy_name     = "eventbridge-log-policy"  
}

Note we could have manually written the resource policy, but the aws_iam_policy_document data block makes it easier to create a well formatted JSON object that the policy expects. Nice!

Initialize Tofu Backend

So far our Tofu project has been configured to use the AWS provider, and we have created all the necessary resources for our logging service.

Now let's learn how to initialize and deploy the Tofu project.

Run tofu init and you should see the following output:



Initializing the backend...

Initializing provider plugins...

OpenTofu has been successfully initialized!

You may now begin working with OpenTofu. Try running "tofu plan" to see
any changes that are required for your infrastructure. All OpenTofu commands
should now work.

If you ever set or change modules or backend configuration for OpenTofu,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

You should also see a .terraform/ folder and .terraform.lock.hcl file now in your repository. These assets hold and describe all the Tofu dependencies required for the AWS provider packages.

After initializing the Tofu project with tofu init we are ready to deploy the infrastructure to our AWS account.

Tofu Validate, Plan and Apply

First let's validate our configuration to make sure we don't have any syntax errors. Run tofu validate. You should see the following output



❯ tofu validate
Success! The configuration is valid.

Before actually deploying the infrastructure to our account, we can view the deployment plan. The deployment plan, gives us an overview about what has changed in our current configuration state, such as what resources have been added, mutated or removed.

Run tofu plan to see the deployment plan.



❯ tofu plan
data.aws_caller_identity.current: Reading...
data.aws_caller_identity.current: Read complete after 0s [id=956613775090]

OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
 <= read (data resources)

OpenTofu will perform the following actions:

  # data.aws_iam_policy_document.cw_log_group_policy will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "cw_log_group_policy" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions   = [
              + "logs:CreateLogStream",
              + "logs:PutLogEvents",
            ]
          + resources = [
              + (known after apply),
            ]

          + principals {
              + identifiers = [
                  + "events.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # aws_cloudwatch_event_rule.service_log_rule will be created
  + resource "aws_cloudwatch_event_rule" "service_log_rule" {
      + arn            = (known after apply)
      + description    = "EventBridge rule to capture logs from any application service and send to CloudWatch Log Group."
      + event_bus_name = "default"
      + event_pattern  = jsonencode(
            {
              + detail      = {
                  + env     = [
                      + {
                          + wildcard = "*"
                        },
                    ]
                  + level   = [
                      + "info",
                      + "warn",
                      + "error",
                    ]
                  + message = [
                      + {
                          + wildcard = "*"
                        },
                    ]
                  + service = [
                      + {
                          + wildcard = "*"
                        },
                    ]
                }
              + detail-type = [
                  + "service.log",
                ]
              + source      = [
                  + {
                      + wildcard = "*"
                    },
                ]
            }
        )
      + force_destroy  = false
      + id             = (known after apply)
      + name           = "service-log-rule"
      + name_prefix    = (known after apply)
      + tags_all       = (known after apply)
    }

  # aws_cloudwatch_event_target.services_event_target will be created
  + resource "aws_cloudwatch_event_target" "services_event_target" {
      + arn            = (known after apply)
      + event_bus_name = "default"
      + force_destroy  = false
      + id             = (known after apply)
      + rule           = "service-log-rule"
      + target_id      = (known after apply)
    }

  # aws_cloudwatch_log_group.log_service_cw_log_group will be created
  + resource "aws_cloudwatch_log_group" "log_service_cw_log_group" {
      + arn               = (known after apply)
      + id                = (known after apply)
      + log_group_class   = (known after apply)
      + name              = "/aws/events/log-service"
      + name_prefix       = (known after apply)
      + retention_in_days = 0
      + skip_destroy      = false
      + tags_all          = (known after apply)
    }

  # aws_cloudwatch_log_resource_policy.eventbridge_log_policy will be created
  + resource "aws_cloudwatch_log_resource_policy" "eventbridge_log_policy" {
      + id              = (known after apply)
      + policy_document = (known after apply)
      + policy_name     = "eventbridge-log-policy"
    }

Plan: 4 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so OpenTofu can't guarantee to take exactly these actions if you run "tofu apply" now.

As you can see, the deployment plan reveals a tree like structure, illustrating the new resources we intend to create as well as summary which states: Plan: 4 to add, 0 to change, 0 to destroy..

That looks correct to me! Let's deploy our logging service!

To deploy your infrastructure to your AWS account run tofu apply. You should see the same deployment plan report we saw earlier. Enter yes when prompted to start the deployment process.



Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

If you can see a similar message in your terminal, then congrats! Our logging service is now live and ready to use.

Verify The Infrastructure

Let's run a quick manual test to ensure that things are working as expected.

We can use the AWS CLI to mimic one of our service's that will emit events to our logging service.



aws events put-events  --region us-west-1  --entries '[
    {
      "Source": "service.logging",
      "DetailType": "service.log",
      "Detail": "{\"service\": \"example-service\", \"env\": \"production\", \"level\": \"info\", \"message\": \"This is a test log message.\"}",
      "EventBusName": "default"
    }
  ]'

Finally navigate to the AWS CloudWatch console. Click on log groups and find the log group named /aws/events/log-service. Here we can to see that our service messages have been logged!

Variables

Let's refactor our configuration to reduce duplication. We have hard coded the region, which makes it difficult to use this service in a different region.

Variables in OpenTofu allow us to pass dynamic values to our configuration and always use the following syntax.



variable "variable_name" {

  # Configuration block

}

variable_name: Is the name for the variable, which must be unique among all variables in the same configuration. This name is used to assign a value to the variable from outside and to reference the variable's value from within the module
The Configuration block specifies the type and description of the variable being declared.

Let's add a dynamic variable for our region.

Add the following to your main.tf file:



variable "region" {
  description = "The aws region to deploy the infrasctructure to."
  type = string
}

Now refactor the provider block to use the variable.



provider "aws" {
  region = var.region # reference the 'region' variable
}

Let's deploy our refactored code and pass in a value for the region.

Run the following command in your terminal.



tofu apply --var "region=us-west-1"

You should see the following output.



No changes. Your infrastructure matches the configuration.

Notice that nothing changed in our deployment plan! This is what we expect, since we are deploy essentially the same infrastructure configuration, just refactored for maintainability.

Destroy The Infrastructure

When you are ready, we can easily remove all the infrastructure resources we created by running tofu destroy

You should see the following output.



Destroy complete! Resources: 4 destroyed.

You should also see a deployment plan, similar to the ones we saw when running tofu plan and tofu apply.

Conclusion

In this post we reviewed the basics of OpenTofu and how to get started provisioning AWS infrastructure. We reviewed providers, resources, data sources and variables. Hopefully this summary will help those who are completely new OpenTofu and its predecessor Terraform.

For those who are already familiar with Terraform or have at least heard of it before, you may be wondering why we would even use Open Tofu in the first place.

Not So Straight Forward Terraform

While Terraform is an incredible tool, its parent company Hashicorp recently changed Terraform's product license from the Mozilla Public License (v2.0) to a Business Source License (v1.1), effectively close sourcing all new versions of Terraform. In addition, Hashicorp also changed their terms of service for their (provider registry)[https://github.com/opentofu/roadmap/issues/24#issuecomment-1699535216], making it a significant legal risk to provision AWS infrastructure with Terraform.

Overnight, thousands of organizations were required to purchase a Hashicorp license if they wanted to keep using Terraform to manage and provision their infrastructure.

But then, there was Tofu.

Within (one month)[https://opentofu.org/blog/the-opentofu-fork-is-now-available/] of Hashicorp's license change, the good people behind OpenTofu, successfully forked Terraform v1.5 and made it open source and completely backwards compatible with existing Terraform code bases. In addition, OpenTofu has recently launched an open source provider registry, making it a complete and truly open source replacement for Terraform. Amazing!

OpenTofu is an incredible achievement for the open source community and demonstrates the beauty and power of open source software. I am excited to support the project and help the project grow. I hope you will join me!

DEV Community: Chris Gradwohl