DEV Community

Cover image for 34.Copy Data to S3 Using Terraform
Thu Kha Kyawe
Thu Kha Kyawe

Posted on

34.Copy Data to S3 Using Terraform

Lab Information

The Nautilus DevOps team is presently immersed in data migrations, transferring data from on-premise storage systems to AWS S3 buckets. They have recently received some data that they intend to copy to one of the S3 buckets.

S3 bucket named datacenter-cp-13069 already exists. Copy the file /tmp/datacenter.txt to s3 bucket datacenter-cp-13069 using Terraform. The Terraform working directory is /home/bob/terraform. Update the main.tf file (do not create a separate .tf file) to accomplish this task.

Note: Right-click under the EXPLORER section in VS Code and select Open in Integrated Terminal to launch the terminal.

Lab Solutions

Step 1: Update Main Terraform Configuration

# main.tf

resource "aws_s3_bucket" "my_bucket" {
  bucket = "datacenter-cp-13069"
  acl    = "private"

  tags = {
    Name        = "datacenter-cp-13069"
  }
}

# Upload file to S3 bucket
resource "aws_s3_object" "upload_file" {
  bucket = aws_s3_bucket.my_bucket.bucket
  key    = "datacenter.txt"
  source = "/tmp/datacenter.txt"
  etag   = filemd5("/tmp/datacenter.txt")
}
Enter fullscreen mode Exit fullscreen mode

Check Existing Resources

ls -l /tmp/datacenter.txt 
Enter fullscreen mode Exit fullscreen mode

Output

bob@iac-server ~/terraform via πŸ’  default ➜  ls -l /tmp/datacenter.txt 
-rw-rw-r-- 1 bob bob 27 Nov  8 15:11 /tmp/datacenter.txt
Enter fullscreen mode Exit fullscreen mode

Step2: To deploy this configuration

Navigate to the Terraform directory:

cd /home/bob/terraform
Enter fullscreen mode Exit fullscreen mode

Initialize Terraform:

terraform init
Enter fullscreen mode Exit fullscreen mode

Output

bob@iac-server ~/terraform via πŸ’  default ➜  terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.91.0"...
- Installing hashicorp/aws v5.91.0...
- Installed hashicorp/aws v5.91.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Enter fullscreen mode Exit fullscreen mode

Plan the configuration:

terraform plan
Enter fullscreen mode Exit fullscreen mode

Output

bob@iac-server ~/terraform via πŸ’  default βœ– terraform plan
aws_s3_bucket.my_bucket: Refreshing state... [id=datacenter-cp-13069]

Terraform used the selected providers to generate the following execution plan. Resource
actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_s3_object.upload_file will be created
  + resource "aws_s3_object" "upload_file" {
      + acl                    = (known after apply)
      + arn                    = (known after apply)
      + bucket                 = "datacenter-cp-13069"
      + bucket_key_enabled     = (known after apply)
      + checksum_crc32         = (known after apply)
      + checksum_crc32c        = (known after apply)
      + checksum_crc64nvme     = (known after apply)
      + checksum_sha1          = (known after apply)
      + checksum_sha256        = (known after apply)
      + content_type           = (known after apply)
      + etag                   = "628f77ec27c0e6eb1e0c6543cc3dd865"
      + force_destroy          = false
      + id                     = (known after apply)
      + key                    = "datacenter.txt"
      + kms_key_id             = (known after apply)
      + server_side_encryption = (known after apply)
      + source                 = "/tmp/datacenter.txt"
      + storage_class          = (known after apply)
      + tags_all               = (known after apply)
      + version_id             = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
β•·
β”‚ Warning: Argument is deprecated
β”‚ 
β”‚   with aws_s3_bucket.my_bucket,
β”‚   on main.tf line 3, in resource "aws_s3_bucket" "my_bucket":
β”‚    3:   acl    = "private"
β”‚ 
β”‚ acl is deprecated. Use the aws_s3_bucket_acl resource instead.
β”‚ 
β”‚ (and one more similar warning elsewhere)
β•΅

─────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to
take exactly these actions if you run "terraform apply" now.
Enter fullscreen mode Exit fullscreen mode

Plan the configuration:

terraform apply
Enter fullscreen mode Exit fullscreen mode

Then type yes when prompted to confirm the creation of the snapshot.

Output

bob@iac-server ~/terraform via πŸ’  default ➜  terraform apply
aws_s3_bucket.my_bucket: Refreshing state... [id=datacenter-cp-13069]

Terraform used the selected providers to generate the following execution plan. Resource
actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_s3_object.upload_file will be created
  + resource "aws_s3_object" "upload_file" {
      + acl                    = (known after apply)
      + arn                    = (known after apply)
      + bucket                 = "datacenter-cp-13069"
      + bucket_key_enabled     = (known after apply)
      + checksum_crc32         = (known after apply)
      + checksum_crc32c        = (known after apply)
      + checksum_crc64nvme     = (known after apply)
      + checksum_sha1          = (known after apply)
      + checksum_sha256        = (known after apply)
      + content_type           = (known after apply)
      + etag                   = "628f77ec27c0e6eb1e0c6543cc3dd865"
      + force_destroy          = false
      + id                     = (known after apply)
      + key                    = "datacenter.txt"
      + kms_key_id             = (known after apply)
      + server_side_encryption = (known after apply)
      + source                 = "/tmp/datacenter.txt"
      + storage_class          = (known after apply)
      + tags_all               = (known after apply)
      + version_id             = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
β•·
β”‚ Warning: Argument is deprecated
β”‚ 
β”‚   with aws_s3_bucket.my_bucket,
β”‚   on main.tf line 3, in resource "aws_s3_bucket" "my_bucket":
β”‚    3:   acl    = "private"
β”‚ 
β”‚ acl is deprecated. Use the aws_s3_bucket_acl resource instead.
β”‚ 
β”‚ (and one more similar warning elsewhere)
β•΅

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_s3_object.upload_file: Creating...
aws_s3_object.upload_file: Creation complete after 0s [id=datacenter.txt]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Enter fullscreen mode Exit fullscreen mode

Resources & Next Steps
πŸ“¦ Full Code Repository: KodeKloud Learning Labs
πŸ“– More Deep Dives: Whispering Cloud Insights - Read other technical articles
πŸ’¬ Join Discussion: DEV Community - Share your thoughts and questions
πŸ’Ό Let's Connect: LinkedIn - I'd love to connect with you

Credits
β€’ All labs are from: KodeKloud
β€’ I sincerely appreciate your provision of these valuable resources.

Top comments (0)