Pipeline for Archetype and Infra as Code using AWS CloudFormation

Archetype and Infrastructure as Code, both are required for application development and serves important operational aspects. Here's my try to create a sample pipeline (a very high level and basic) for each.

Infrastructure as code, also referred to as IaC, is an IT practice that codifies and manages underlying IT infrastructure as software. The purpose of infrastructure as code is to enable developers or operations teams to automatically manage, monitor and provision resources, rather than manually configure discrete hardware devices and operating systems. Infrastructure as code is sometimes referred to as programmable or software-defined infrastructure.

Archetype is a project templating toolkit. An archetype is defined as an original pattern or model from which all other things of the same kind are made.

Pipeline detail

Services: GitHub (on-premise), CloudFormation, IAM, Code Pipeline, Code Build, Code Deploy, S3

Archetype pipeline:

1. User interface with a list of project types like a. Spring boot b. Python service c. Node JS etc.
2. Create scripts (archetype) for above sample project creation and keep in GitHub
3. Developer selects a project and clicks create, or a change made in scripts by DevOps engineer
4. Jenkins’s job triggered or AWS CodePipeline is triggered (depends on your choice of pipeline). Here's a blog I had written for creating a CI/CD pipeline using CodePipeline link
5. Job runs script to create project
6. Commit project in GitHub
7. Selects CloudFormation template based on project selection and executes through AWS CLI to create all infrastructure
8. Sample project is built and deployed to AWS development environment
9. Mail notification is sent to developer

Infrastructure as Code pipeline:

1. Manually create initial CloudFormation template and push those scripts in GitHub
2. Create a sample AWS Code Pipeline to run it
3. Every time a change is needed in infrastructure just update CloudFormation script
4. Upon code check-in pipeline will be triggered
5. Resources will be created and tested
6. Once tested, resources will be deleted

I used CloudFormation to create a sample high availability deployment infrastructure in AWS.

I found it to be one of the best tools in Infrastructure as Code (IAC), which eases the process of provisioning IT resources in the cloud. This way we can use code to automate the process of setting up anything, e.g. a virtual machine, container, entire pipeline, security etc.

CloudFormation uses templates, configuration files defined in JSON or YAML syntax, that are human readable. It reads those templates and generates a stack, a set of resources ready to use on AWS. Stack can be modified, partially deleted, listen to the changes and eliminated completely if needed. Best part is CloudFormation is free, and it can help in saving strategy (e.g. all the resources can be deleted at 5 pm and recreated at 8 am safely on Dev environment to save money). The code can be versioned, and I am looking into ‘Pipeline for Infrastructure as Code’ as well.

I made a design diagram of the deployment architecture created using CloudFormation using the below code.

I written a sample yaml file to create this infra. Let me explain the architecture, my aim here is to create a high available architecture that spans multiple availability zone. A VPC configured with public and private subnets, to provide organizations own virtual network. An Internet gateway to allow access to the Internet. Managed NAT gateways to allow outbound Internet access for resources in the private subnets.

Now I will add a portion of my yaml file here

Resources:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: !Sub '10.${x}.0.0/16' 
      EnableDnsSupport: true
      EnableDnsHostnames: true
      InstanceTenancy: default
      Tags:
      - Key: Name
        Value: !Sub '10.${x}.0.0/16'
  VPCCidrBlock:
    Type: 'AWS::EC2::VPCCidrBlock'
    Properties:
      AmazonProvidedIpv6CidrBlock: true
      VpcId: !Ref VPC
  InternetGateway:
    Type: 'AWS::EC2::InternetGateway'
    Properties:
      Tags:
      - Key: Name
        Value: !Sub '10.${x}.0.0/16'
  EgressOnlyInternetGateway:
    Type: 'AWS::EC2::EgressOnlyInternetGateway'
    Properties:
      VpcId: !Ref VPC
  VPCGatewayAttachment:
    Type: 'AWS::EC2::VPCGatewayAttachment'
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway

Here's a pictorial representation of the code for better understanding:

Explaining couple of lines. This way we can create n number of resources.

Below is a screenshot of AWS console which shows some of the created resources

It has a lot of other features and functionalities. Let me know if you have any question/thought.