I am new to terraform; I've been involving in a project and trying to make a CICD pipeline in order to work on data through multiple tiers and of course proper way to automate moving data between dev, staging and prod environment. I have some questions:
- What could be the best practice to set up terraform structure?
- Dividing modules into specific submodules is recommended?
- Is it good to use terraform for invoking Lambda or serverless is the better solution?
Well, I need your #help and Looking forward to hearing from you.
Top comments (8)
See here for some good guidelines:
In general you need to think hard on the structure of Terraform, I propose you do these 3 things;
This could look like this:
You can check out my public repo for an example:
rpstreef / openapi-tf-example
Example of how you can use OpenAPI with AWS API Gateway, Also includes integrations with AWSLambda, AWS Cognito, AWS SNS and CloudWatch logs
OpenAPI with AWS API Gateway, Lambda, Cognito, SNS and CloudWatch logs
This repo only deploys the infrastructure via Terraform. The source code, this repo, will be deployed automatically via AWS CodePipeline. By default, it's configured to automatically deploy at every push to the master branch.
Check the companion articles series 'OpenAPI' on dev.to.
To get the API running
If you meet all the pre-requisites, do the following
Thank you for answer the questions, was really great help to make my mind clear!
If you could also know your opinion on this as well?
which one is better to create a bunch of s3 buckets for multiple environment? write a module to create them once or use simple module and apply multiple times?
Do they have the same functionality across these environments (like development, test etc) then you would have 1 S3 module and in your application directory you create the single bucket. It will be named slightly different per environment (include the name environment name, dev, test,prod).
Then in your environment directory you supply the correct naming for each environment, then you have to deploy each of them separate with their respective
tfstatefile per environment.
That is probably the easiest way to do it and then make sure you store the Terraform state in each environment where it belongs to as well.
To target each of them, you can create profiles in your ./.aws/config file and reference them in the Terraform tfstate and your remote state configuration.
I hope that helps
Well, there are five buckets at least which will repeat in dev, stage and prod! but as far as they contain their environment name in their name, should be change per environment.
I highly recommend getting some structured training to become familiar with the fundamentals first. Hands-on courses like Using Terraform to Manage Applications and Infrastructure are likely to answer many of your questions.
There is a lot of free content on these topics around, but IMO paid services like Linux Academy and Cloud Guru provide a higher quality learning experience even if you're not working towards a certificaiton.
Thanks, just registered to complete the course.
Terraform should deal with declaring and provisioning your required infrastructure only. You can invoke Lambda functions in terraform to fetch data from that lambda function to use in your terraform modules however but it shouldn't be done to perform actions that may change infrastructure or data unknowingly.
If you need to call your lambda functions for another reason it might be good to use something like Ansible to script this action and call this as a separate step in your pipeline.
One of my favorite of the terraform Structure is from Charity. She create the article here: charity.wtf/2016/03/30/terraform-v...
the way how she explains the problem, the process, the way how she looks at that article gives me some idea of how I create Terraform projects.
Dividing modules into specific submodules is recommended?
Depends on how big your projects and how granular you want. If you only work for small services/small projects maybe not too much submodule will be better. Since too much modules/submodules will be more hard maintaining it later
Is it good to use terraform for invoking Lambda or serverless is the better solution?
Depends again. If your terraform is part of big projects for maintaining your whole environments, I think it a bit of risk if you want to use terraform for invoke Lambda. Since one error/problem will risk your state.