DEV Community

Cover image for AWS Data Lake with Terraform - Part 3 of 6
Augusto Valdivia for AWS Community Builders

Posted on

AWS Data Lake with Terraform - Part 3 of 6

Amazon Redshift

Amazon Redshift is a massive AWS parallel processing data warehouse designed for large scale data sets. A very useful feature that Redshift offers is Amazon Redshift Spectrum. In this section I will give a high level summary of why it is so powerful.

Redshift Parallel Processing
Redshift

Let’s start by reiterating that Amazon Redshift Spectrum is a serverless and there's nothing to provision or manage. You just pay for the resources you consume for the duration of your Redshift Spectrum.

Redshift Spectrum is a query processing engine that would allow you to analyze data that is stored in Amazon S3 using standard Structure Query Language(SQL) without ETL processing.

spectrum

What do I mean by analyzing data that is sitting in Amazon S3?

I mean that you do not need to think about using Amazon Redshift storage for any data storage as the data will always live on Amazon S3. You can pull, aggregate and filter all sorts of data using Amazon Redshift Spectrum. Remember Amazon Redshift Spectrum is serverless and so you do not need to worry about anything else other than your code.

Redshift clusters

DB

Redshift Terraform

resource "aws_redshift_cluster" "data_logs_db" {
  cluster_identifier = "data-logs-cluster"
  database_name      = "data_logs_db"
  master_username    = "admin"
  master_password    = "Mustbe8+ch@r@cters"
  node_type          = "dc1.large"
  cluster_type       = "single-node"
}
Enter fullscreen mode Exit fullscreen mode

Diagram version 3: Data lake
Diagram

Diagram final version: Data lake
Diagram

Please stay tune for part 4

Top comments (0)