DEV Community: Anuj Bansal

Introduction to AWS CloudFormation 📜

Anuj Bansal — Mon, 30 Aug 2021 13:56:09 +0000

Deploying more than just a handful of services on AWS and setting each one up manually can get quite tedious. Having to configure each AWS resource by hand also leaves you at a much higher risk of making errors or introducing inconsistencies.

Enter AWS CloudFormation.

CloudFormation is a service that provides an easy way to create and manage a collection of AWS resources in a controlled and predictable way.

How does AWS CloudFormation work?

The key concepts in CloudFormation are templates, stacks, and change sets.

Template: A template is a description of your infrastructure, written in JSON or YAML, that can be interpreted by CloudFormation.
Stack: Infrastructure created by CloudFormation using a template is known as a stack.
Change Set: A summary of proposed changes to your stack that will allow you to see how those changes might impact your existing resources before implementing them

The first thing you do is to create a CloudFormation template in JSON or YAML format.

You then deploy the template using the CloudFormation console, API, or AWS CLI. CloudFormation provisions and configures resources by making calls to the AWS services that are described in your template.

After all the resources have been created, CloudFormation reports that your stack has been created. If stack creation fails, CloudFormation rolls back your changes by deleting the resources that it created.

To update a stack, create a change set by submitting a modified version of the original stack template, different input parameter values, or both. CloudFormation compares the modified template with the original template and generates a change set. You can then view the changes that will be deployed before you decide to execute the change.

Template Basics

The template can consist of the following sections:

Format Version: The AWS CloudFormation template version that the template conforms to.
Description: A text string that describes the template.
Metadata: Arbitrary JSON or YAML objects that provide additional information about the template.
Parameters: Values to pass to your template at runtime.
Rules: Validates a parameter or a combination of parameters passed to a template during stack creation or stack update.
Mappings: A mapping of keys and associated values that you can use to specify conditional parameter values, similar to a lookup table.
Conditions: Conditions that control whether certain resources are created or whether certain resource properties are assigned a value during stack creation or update.
Transform: Set of macros to process your template. Basically creates a processed version of your template.
Resources: AWS resources that you want to include in your stack.
Outputs: Declares output values that you can import into other stacks, return in response (to describe stack calls), or view in the console.

This is what a basic template looks like 👇

Let's go a bit more in-depth on some of the sections.

Parameters

Parameters are used to customize a template with values. They give us the flexibility to change settings without having to modify the template code.

Resources

This is the bulk of the template. Here is where we define and configure the resources that CloudFormation will manage for us. When defining resources, you need to know about the type and that type’s properties.

Outputs

These are like return values for the template. We use them to make it easy to find some of the resources that CloudFormation will create for us.

Intrinsic Functions

CloudFormation has several intrinsic functions that you can use to refer to other resources and their properties. You can use them to assign values to properties that are not available until runtime.

Fn::Base64 - returns the Base64 representation of the input string
Fn::Cidr - returns an array of CIDR address blocks
Condition functions - Used to conditionally create stack resources. These are Fn::And, Fn::Equals, Fn::If, Fn::Not and Fn::Or.
Fn::FindInMap - returns the value corresponding to keys in a two-level map that's declared in the Mappings section
Fn::GetAtt - returns the value of an attribute from a resource in the template
Fn::GetAZs - returns an array that lists Availability Zones for a specified region in alphabetical order
Fn::ImportValue - returns the value of an output exported by another stack
Fn::Join - appends a set of values into a single value, separated by the specified delimiter
Fn::Select - returns a single object from a list of objects by index
Fn::Split - split a string into a list of string values using a delimiter
Fn::Sub - substitutes variables in an input string with values that you specify
Fn::Transform - specifies a macro to perform custom processing on part of a stack template
Ref - returns the value of the specified parameter or resource

CloudFormation allows you to either use the full function name (Fn::Sub) or the short form (!Sub).

This is what a sample template would look like with intrinsic functions 👇

AWSTemplateFormatVersion: '2010-09-09'
Description: 'Creating an EC2 instance using a security group that has already been created or a new security group'

Parameters:
  ExistingSecurityGroup: 
    Description: An existing security group ID (optional).
    Default: NONE
    Type: String

Conditions:
  CreateNewSecurityGroup: !Equals [!Ref ExistingSecurityGroup, NONE]

Resources:
  MyInstance:
    Type: "AWS::EC2::Instance"
    Properties: 
      ImageId: "ami-0ff8a91507f77f867"
      SecurityGroups: !If [CreateNewSecurityGroup, !Ref NewSecurityGroup, !Ref ExistingSecurityGroup]
            Tags:
              - Key: Name
                Value: !Sub '${AWS::StackName}-webserver'

  NewSecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Condition: CreateNewSecurityGroup
    Properties:
      GroupDescription: Enable HTTP access via port 80
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0

Outputs:
  SecurityGroupId:
    Description: The group ID of the security group used.
    Value: !If [CreateNewSecurityGroup, !Ref NewSecurityGroup, !Ref ExistingSecurityGroup]

Conclusion

By leveraging an IaC tool such as CloudFormation, you can streamline the AWS deployment process. You can define your resource configurations once, then deploy them as many times as you need. You can manage anything from a single Amazon EC2 instance to a complex multi-tier multi-region application.

Feel free to reach out to me on Twitter! 😊

Git internal architecture 🏛

Anuj Bansal — Sun, 19 Apr 2020 20:51:34 +0000

Git is a really simple and elegant solution to a complex problem. I think it’s important that we understand what’s going on behind the scenes and the engineering decisions made to fully grasp it's simplicity and power.

How does Git store files? What happens when we run the various Git commands? How is everything linked? What is the data structure used?

We are going to answer all of the questions below 👇

Initialising the repo

When you run git init in a directory, Git creates the .git directory, which is where almost everything that Git stores and manipulates is located.

👆 Output of tree .git when we run the git init command

It contains a few different types of files and directories:

Configuration: the .git/config, .git/description and .git/info/exclude files essentially help configure the local repository.
Hooks: the .git/hooks directory contains scripts that can be run on certain lifecycle events of the repository.
Staging Area: the .git/index file (which is not yet present in our tree listing above) will provide a staging area for our working directory.
Object Database: the .git/objects directory is the default Git object database, which contains all content or pointers to local content.
References: the .git/refs directory is the default location for storing reference pointers for both local and remote branches, tags and heads. A reference is a pointer to an object, usually of type tag or commit. References are managed outside of the Object Database to allow the references to change where they point to as the repository evolves. Special cases of references may point to other references, e.g. HEAD.

Staging a file

When we run the command git add . Git adds all the changes from the working directory to the staging area and creates blob files in the .git/objects sub-directory.

Each objects file has a 40-char SHA-1 hash as its filename. Git uses the first 2 chars to organise the objects in directories.

This blob object has the contents of the file. All objects are immutable once created. Making changes to a file and staging it will result in an entirely new object getting created.

Let's verify and check the contents of this blob file. When we run the command

cat .git/objects/55/7db03de997c86a4a028e1ebd3a1ceb225be238

we don't get what we expected.

Well, Git actually compresses each and every object using zlib and therefore what you see is the compressed content.

To see the actual file content use the command git cat-file -p <SHA-1>

Commits

Running the git commit command creates two more objects in the objects sub-directory. One is a tree object and the other is a commit object.

Tree Objects

A single tree object contains one or more entries, each of which is the SHA-1 hash of a blob or subtree with its associated mode, type, and filename.

Commit Objects

The format for a commit object is simple: it specifies the top-level tree for the snapshot of the project at that point, the parent commits if any, the author/committer information and a commit message.

The commit id that we commonly use is the SHA-1 hash of the contents of the commit object.

Git stores all the content as a directed acyclic graph using these different types of objects. Here is what the data structure would look like at this point of time.

Branches

A branch in Git is simply a lightweight movable pointer to one of the commit objects. Because of this reason, creating new branches in Git is "cheap".

Every time you commit, the branch pointer moves forward automatically.

How does Git know what branch you’re currently on? It keeps a special pointer called HEAD. HEAD is nothing but a special pointer which points towards a branch. The branch that you are currently working on.

When we run the command git checkout -b <NAME> Git creates a new file in the refs/heads directory with the branch name. The file contains the pointer to the latest commit.

In conclusion, Git commands are an abstraction over the data storage. Hashes, file based key-value storage and tree data structure, these are the key things behind Git.

Feel free to reach out to me on Twitter.

Let me know what other architectural deep dives would interest you. Comment below. 👇

Top Mac Apps for becoming more productive 💪

Anuj Bansal — Sat, 11 Apr 2020 16:52:04 +0000

Thousands of apps have been made for mac. I am going to list a few that I use on a day-to-day basis to help you become more productive and save some time.

Rectangle

Move and resize windows in macOS using keyboard shortcuts or snap areas

HazeOver

Too many windows to manage? Large display? Or sometimes getting lost in multiple monitors? HazeOver is for you! This app automatically highlights the front window by fading out all the background windows.

Recordit

Recordit is a screencast tool for Mac and Windows operating systems. Recordit is a GIF-based screencast tool that provides desktop clients for both Windows and Mac. It allows users to record desktop screen for as long as five minutes.

Flycut

Clean and simple clipboard manager for developers. Every time you copy a code piece, Flycut stores it in history. Later, you can paste it using Shift-Command-V even if you have something different in your current clipboard. You can change the hotkey and other settings in preferences.

Itsycal

It is weird that Mac doesn't have a calendar in the menu bar if you want to have a quick look at the dates. If you want to have a quick look for the date next Friday, you would have to open up the calendar application on Mac, which is time consuming.

Itsycal is a tiny menu bar calendar. If you want, it will display your events as a companion to the Mac Calendar app.

These were some of the apps that I use regularly.
What other apps do you think should be on the list? Comment below 👇