How to Reference Terraform Cloud State from ecspresso and ecschedule

Photo by Ian Taylor on Unsplash

---

Introduction: ecspresso, ecschedule, and Terraform

ecspresso, ecschedule, and Terraform are powerful tools for achieving Infrastructure as Code (IaC) and managing resources in AWS. Let's take a closer look at each tool.

ecspresso

ecspresso is a tool designed for managing the definition of ECS services and tasks as code. It enables you to define and manage your ECS infrastructure using familiar coding practices.

You can find more information about ecspresso on the official GitHub repository: ecspresso GitHub.

kayac / ecspresso

ecspresso is a deployment tool for Amazon ECS

ecspresso

ecspresso is a deployment tool for Amazon ECS.

(pronounced same as "espresso")

Documents

• Differences of v1 between v2.
• ecspresso Advent Calendar 2020 (Japanese)
• ecspresso handbook (Japanese)
• Command Reference (Japanese)

Install

Homebrew (macOS and Linux)

$ brew install kayac/tap/ecspresso

asdf (macOS and Linux)

$ asdf plugin add ecspresso
# or
$ asdf plugin add ecspresso https://github.com/kayac/asdf-ecspresso.git
$ asdf install ecspresso 2.0.0
$ asdf global ecspresso 2.0.0

Binary packages

Releases

CircleCI Orbs

https://circleci.com/orbs/registry/orb/fujiwara/ecspresso

version: 2.1
orbs:
  ecspresso: fujiwara/ecspresso@2.0.3
jobs:
  install:
    steps:
      - checkout
      - ecspresso/install:
          version: v2.0.0 # or latest
          # version-file: .ecspresso-version
      - run:
          command: |
            ecspresso version

version: latest installs different versions of ecspresso for each Orb version.

• fujiwara/ecspresso@0.0.15
  • The latest release version (v2 or later)
• fujiwara/ecspresso@1.0.0
  • The latest version of v1.x
• fujiwara/ecspresso@2.0.3
  • The latest version of v2.x

version: latest is not recommended because it may cause unexpected…

View on GitHub

ecschedule

ecschedule is another valuable tool in the IaC ecosystem, specifically built for managing scheduled tasks on ECS. With ecschedule, you can define and manage tasks that need to be executed on a schedule, similar to cron jobs.

To learn more about ecschedule, visit the GitHub repository: ecschedule GitHub.

Songmu / ecschedule

ecschedule is a tool to manage ECS Scheduled Tasks.

ecschedule

ecschedule is a tool to manage ECS Scheduled Tasks.

Synopsis

% ecschedule [dump|apply|run|diff] -conf ecschedule.yaml -rule $ruleName

Description

The ecschedule manages ECS Schedule tasks using a configuration file (YAML, JSON or Jsonnet format) like following.

region: us-east-1
cluster: clusterName
rules:
- name: taskName1
  description: task 1
  scheduleExpression: cron(30 15 ? * * *)
  taskDefinition: taskDefName
  containerOverrides:
  - name: containerName
    command: [subcommand1, arg]
    environment:
      HOGE: foo
      FUGA: {{ must_env `APP_FUGA` }}
- name: taskName2
  description: task2
  scheduleExpression: cron(30 16 ? * * *)
  taskDefinition: taskDefName2
  containerOverrides:
  - name: containerName2
    command: [subcommand2, arg]

Installation

% brew install Songmu/tap/ecschedule
# or
% go install github.com/Songmu/ecschedule/cmd/ecschedule@latest

GitHub Actions

Action Songmu/ecschedule@main installs ecschedule binary for Linux into /usr/local/bin. This action runs install only.

jobs
  deploy:
    runs-on

…

View on GitHub

Terraform

Terraform is a widely adopted IaC tool that allows you to define and manage infrastructure resources in AWS, such as VPCs, RDS instances, and ElastiCache clusters. It provides a declarative approach to infrastructure management, enabling you to define your desired state and automatically provision and manage resources accordingly.

The Issue of Managing ECS Task Definitions with Terraform

ECS task definitions often include elements that are tightly coupled with the application's implementation, such as environment variable settings and command parameter configurations. When managing task definitions within Terraform, applying changes in Terraform and then reflecting them on the application side becomes a necessary step.

While this approach works well for occasional changes, frequent updates can introduce the potential for mistakes and inefficiencies.

To address this issue, it is recommended to decouple ECS services and task definitions from Terraform. By leveraging ecspresso and ecschedule, I can manage the application code and related ECS components together. This approach simplifies the management process and streamlines development workflows.

Leveraging Terraform Cloud State with ecspresso and ecschedule

One of the powerful features offered by ecspresso and ecschedule is the tfstate plugin. This plugin allows you to reference the configuration values of infrastructure resources managed by Terraform using the Mustache syntax.

Using the tfstate plugin, you can incorporate the following example of referencing values using the Mustache syntax:

{{ tfstate `path.to.resource` }}

This syntax enables you to access specific resource configurations managed by Terraform and utilize them within your ecspresso and ecschedule configurations seamlessly.

While the README documentation provides examples of referencing local paths and S3 bucket URLs, it does not explicitly cover referencing state managed by Terraform Cloud. However, by exploring the ecspresso source code, I discovered the usage of tfstate-lookup internally.

To reference Terraform Cloud state, I can utilize the following URL format: remote://app.terraform.io/{ORGANIZATION}/{WORKSPACE}. This format allows me to access the state of a specific workspace within my Terraform Cloud organization.

To make this work, I need to generate an API token for Terraform Cloud and set it in the TFE_TOKEN environment variable. The API token should have appropriate permissions based on my operational requirements.

Once the API token is set, I can configure ecspresso and ecschedule to reference the Terraform Cloud state. For example, in the config.yaml file, I can add the following configuration:

plugins:
  - name: tfstate
    config:
      url: remote://app.terraform.io/{ORGANIZATION}/{WORKSPACE}

With this configuration in place, I can now read and reference the configuration values of my Terraform-managed resources within ecspresso and ecschedule.

Before deploying changes, it is recommended to run commands such as ecspresso render or ecschedule diff to confirm that the configuration reflects the values from the Terraform Cloud state.

Conclusion

• It is easier to manage ECS services, task definitions, and scheduled tasks as Infrastructure as Code (IaC) using ecspresso and ecschedule, rather than Terraform.
• Set the Terraform Cloud API Token in the TFE_TOKEN environment variable.
• The URL format to reference Terraform Cloud state is remote://app.terraform.io/{ORGANIZATION}/{WORKSPACE}.

The tfstate plugin provided by ecspresso and ecschedule allows me to reference and leverage the configuration values of my Terraform-managed resources. By utilizing this feature, along with proper configuration and the API token for Terraform Cloud, I can seamlessly integrate Terraform Cloud into my ecspresso and ecschedule workflows.

I hope this article provides valuable insights and guides you towards optimizing your infrastructure management practices.