DEV Community: Harris Geo 👨🏻‍💻

Hexagonal Architecture: A High Level Overview

Harris Geo 👨🏻‍💻 — Sun, 27 Feb 2022 20:33:45 +0000

Hexagonal architecture is a great way to build structure to your system and split it into different layers each of which serves a specific purpose.

Do not let the name trick you into thinking that it contains 6 pieces of logic. It is more of a representation of the multiple sides a hexagon has and makes it ideal for apps that have multiple connections with external systems. The hexagon is also a common component to use in UML diagrams.

Now let’s talk about the 3 layers that make Hexagonal architecture.

Adapters
Ports
Domain

Adapters

The way I like to think of adapters is like the I/O of our app. How data reaches into our app and then where does this data go?

That might be a HTTP endpoint that invokes our app, or an EventBridge event that our app is listening to. Then on the opposite end, once the app executes its business logic it has to do something with that data.

A very common scenario is to store that data in a Database like DynamoDB, or MongoDB, or send a notification to the customer. Adapters can be anything that allows our app to have an inbound or outbound communication with the outside world.

Domain

When the data received within the app needs to be processed and execute some business logic, stuff like calculations, data reshaping and other internal to the app processes. This is the domain layer.

Isolating the domain logic is a great practice for building resilient systems that not only can scale but also are easy to work with and modify. More of the latter later.

Ports

The ports layer in my opinion is the part that causes the biggest amount of confusion in the whole concept of this architectural pattern. Let’s see if we can make some sense out of it.

As we've already said, one of the selling points of Hexagonal architecture is the fact that it can make our app domain agnostic. What that means is that our business logic should be decoupled from the specific tools and infrastructure that we use. In other words, our domain should not be dependant of the specifics of the Database we use.

Similarly, the domain should not know that we’re sending it data via an SQS queue. The port is the bridge that connects the domain with the adapter and holds the logic that decides what information should be passed from one to the other. In typed languages a port is usually an interface that specifies the shape of the data that adapters have to pass to the domain and vice versa.

A use case

Let’s take a classic example where our application is a RESTful API which receives data via a HTTP POST endpoint and stores it to MongoDB. The journey would look like that

Our HTTP adapter will process the HTTP POST request and send the data to the port which will communicate that to the domain. This is where our internal logic will be executed. Stuff like internal calculations, reshaping data etc.

Then we need to follow the same logic in reverse. The domain has some data, wants to store it in the DB and has to send them to the repository port which will then send it to the Database adapter.

Adapter (input)	HTTP handler
Port (to domain)	HTTPHandler.retrieveData
Domain	Process data and send data to repository
Port (to adapter)	Repository.storeData
Adapter (output)	Connection with MongoDB

Benefits of Hexagonal architecture

You’re probably wondering “yeah that’s cool but why would we go through all that trouble?”

1. Flexibility meets structure

To put Hexagonal architecture into a business perspective, it is painless when we want to introduce new features due to the loosely coupled way of structuring our code. We can change parts of our app without causing major disruption.

In addition to that, our future selves will really thank us when it comes to debugging an error, as we will immediately know where to look.

Did the app return the wrong data? That sounds like an issue in the domain layer. Was there a network issue during that request? Sounds like an adapter issue.

2. Isolated testing

One of my favourite parts of Hexagonal architecture is that testing our code becomes much simpler. We all have experienced codebases that are really difficult to test due to their lack of boundaries where all of the implementation is just thrown into a function / method / class / whatever you want to name it, that is 100+ lines long.

With Hexagonal architecture each layer is a separate module we can test in isolation. This can be done by mocking its communication with other layers, which gives us the flexibility to have smaller tests that are easier to write and faster to execute. Bonus point, that can then result with higher testing coverage.

3. Domain agnostic app

The whole concept of “plug and play” adapters is great because it ensures our business logic does not rely the tools we use.

The more specific our business logic is to a certain infrastructure, the more difficult it will be for us in the future to move away from this infrastructure.

How many times have we had to spend days if not weeks trying to find out how to switch from Database A to Database B because our code is too tighly coupled to Database A. This tools logic leakage inside our domain is something we need to be careful about.

Hexagonal architecture guides us on how to have clear boundaries between the tools and our business logic. Then once we decide to move away from a tool, it should be as simple as adding a new adapter.

Obviously I am not saying that migrating away from tools is going to be a piece of cake, but the transition within our app, will probably be the smallest of our concerns.

Conclusion

For those of us working with Serverless apps, it is a known problem that we very have our entire logic inside the handler. Then slowly once our application starts getting bigger and bigger, we either end up with gigantic handlers or some weird structure which looks like that infrastructure logic is mixed within the business logic. This is where we need to introduce some boundaries and Hexagonal architecture can help us with it.

I have to admit that the first time I tried to write some code using this pattern, it felt really weird. I think the biggest issue was not really understanding what kind of problem Hexagonal architecture is trying to solve. With time though it started making a lot more sense, and since then it has been the number one choise of structuring projects I've been working on.

AWS Learn In Public Week 9, CloudFormation And CloudFront

Harris Geo 👨🏻‍💻 — Mon, 26 Apr 2021 08:05:44 +0000

This week we're going to talk about CloudFormation which is what we talked about in last week's Elastic Beanstalk blog that is used under the hood. After that, I also have a small introduction to CloudFront. Before I spoil anything else, let's get started.

CloudFormation

This week we're going to talk about CloudFormation in AWS.

What is CloudFormation? It way for declaring what AWS infrastructure you want provision in a template. We can create, configure and delete AWS components and also reference them with each other.

The format is AWS::Lambda::Function or AWS::EC2::Instance

CloudFormation supports most AWS services and the full list can be found here

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html

CloudFormation Parameters and more

Let's talk about AWS CloudFormation parameters

We can specify inputs insure our templates which are great for reusing templates and values of services that we want to use after they are created
That way we won't have to re-upload templates all the time.

Referencing

We can use referencing for using params anywhere within the template

The API name is Fn:Ref and in the yaml config it is shortened to !Ref

Pseudo params

We can also use pseudo params for AWS related values that we do not want to store in our code and again use them at any time. These can be like the AWS account id with AWS::AccountId, the region with AWS::Region and more.

Mappings

We also have mappings which are fixed variables useful when adding sets of hardcoded data in our code. An example is FN:FindInMap which allows us to search within maps.

Outputs

Then we have outputs which are optional but work really well when we want to use the value of a service that was just created in order to reference it to another resource.

Cross Reference

Then we have the cross stack reference where we create another template that uses a security group. We can reference that with Fn::ImportValue. Once a stack is referenced in another template, all the references need to be deleted first before deleting the first stack.

Conditions

We can control the creation of resources based on conditions. Such conditions are environment stage, AWS region etc

Conditions can reference other conditions, parameter values or mappings
We have intrinsic function like and Fn:And, equals Fn:Equals, if Fn:If etc

CloudFormation Rollbacks

Let's talk about AWS CloudFormation rollbacks

If a stack creation fails, by default all underlying resources get deleted
We also have an option to disable that and troubleshoot the error
If a stack update fails it automatically rolls back to the previous state that was working
Same as when creating, we have the ability to see in the logs and debug what exactly went wrong

CloudFront

What is AWS CloudFront? It is a CDN (Content Delivery Network) formed of distributions and is mainly used to improve site’s performance as content is cached on multiple edge locations around the world.It provides DDOS protection and integrates with AWS firewall Shield

CloudFront can provide origins from:

S3 buckets for distributing and caching files at the edge
Other custom origins like ALB, EC2, S3 websites and any HTTP backend you want

Overall, clients send requests to any of the multiple edges around the world where the requests get forwarded to the origin along with any query params and headers. Then the origin responds with the available assets which then get cached in the edge location for future requests.

Summary

Both CloudFormation and CloudFront are services that I have not used much in the past but I understand their importance in our AWS stack. CloudFormation is definitely a service that requires a deeper dive with some coding examples (maybe a future blog post) to really understand its value.

Next week we're finally going to talk about the most interesting service in AWS for being a developer. Shall I ruin the surprise? Whatever. Next week will be about AWS Lambda!

AWS Learn In Public Week 8, Elastic Beanstalk

Harris Geo 👨🏻‍💻 — Sun, 18 Apr 2021 19:53:31 +0000

This has been an exciting week as London is starting to open up little by little. We agreed with the rest of my team at work that whoever wants can start to the office twice per week. Finally after a long winter of staying at home the whole day every day, we can start seeing some people again. The only downside of it is that it broke my routine of tweeting about AWS services.

In this blog post we're going to talk about Beanstalk which is a really interesting service as it can glue the majority of the stuff you need when creating an environment. I don't understand why not everyone uses it.

Definitely once I finish this project and get my certification this will be one of the first services that I'm going to dive deep into it. Without further ado let's check it out.

AWS Beanstalk

What is AWS Elastic Beanstalk?

A platform as a service for deploying applications to AWS
It is a layer for configuring how to use other services like EC2, Auto Scaling Groups, Load Balancers, RDS etc.
Using Elastic Beanstalk is free but you only pay for the underlying resources
Elastic Beanstalk is a managed service and can also be used for deployment strategies
The idea behind it is that the developer is responsible for the code and Beanstalk for the infrastructure

There are 3 architecture models

Single Instance Deployment which is great for dev environments
Load Balancer with Auto Scaling Groups which is the standard model for production web apps
Auto Scaling Groups only which is mainly for analytics and workers services

We can version our applications to environments and promote them to the next environment until we reach production. We can customise these stages to whatever we want. e.g. dev - staging - prod. Rollback feature is also available

Deployment options

All at once

We have the "all at once" option where you can deploy all instances in one go.

This option has downtime but it is the fastest way to deploy
It is great for dev environments that require quick iterations and also there are no additional costs to it

Rolling

We have the rolling option where slow update the current instances with new once until our application only contains the new code. Let's say our app has 4 instances. 2 of them are going to be updated (below capacity) with the new version and then the next 2.

Rolling with additional batches

We have the rolling with additional batches option. Here we use the same logic as before but instead of updating the current instances, we add a few extra. The deployment is longer, has small extra costs but is good for production

Immutable

We have the immutable option where we spin up a complete new set of instances (double the amount in total) and once the new version is out and running, we terminate the old ones. This option has 0 downtime, is great for prod but is quite costly.

Blue / Green deployments

Finally we have the blue / green deployment option

We create a new env with the new app version (green) and direct 10% of the traffic to it
The old env (blue) will handle 90% of the traffic
We setup weighted policies in Route53
Once we are happy, Beanstalk can swap urls

Beanstalk under the hood

Let's talk a bit more about how AWS Beanstalk works under the hood. It basically relies on AWS CloudFormation to provision any other AWS services (Infrastructure As Code). To do that we can define a .ebextensions folder inside which we provision any service we want

Running Docker with Beanstalk

Single Docker Container

Single Docker for simple setups where we run our app as a single Docker container.

We provide:

A Dockerfile which will be used to build and run our container
A Dockerrun.aws.json v1 file for existing images which can be in ECR or Dockerhub
Uses EC2 under the hood

Multi Docker Container

Multi Docker which runs multiple containers per EC2.

It will create and ECS cluster, EC2 instances for it, a Load Balancer in High Availability mode, task definitions and execution.
It requires a Dockerrun.aws.json v2 config file at the root of the project.

Multi Docker also uses the Dockerrun.aws.json v2 config file to generate the ECS task definition. We need to have our docker images prebuilt and stored in ECR

Summary

I really don't understand why not more people use Beanstalk. It really simplifies the whole deployment process and putting all the services together.

On the surface this looks really really simple but Beanstalk is just the tip of the iceberg. Next week we are going to talk about CloudFormation and a little bit of Cloudfront.

AWS Learn In Public Week 7, ECR, ECR and Fargate

Harris Geo 👨🏻‍💻 — Sun, 11 Apr 2021 21:19:25 +0000

How's everyone doing? This section I have to admit is the one that I found the most difficult to understand all of the concepts within ECS so I hope that my notes will help explaining it in a simple way. Let's get started.

ECS

What AWS ECS (Elastic Container Service)?

ECS is Amazon's platform for managing Docker containers.
ECS clusters are groups of EC2 instances that run the ECS agent for Docker containers which then register the instance to the cluster.
EC2 instances run an AMI designed for ECS

ECS Task Definitions, Services and Clusters

Task Definitions

Task Definitions are metadata in JSON format that give instructions to ECS on how to run a Docker container.

The contain crucial information about image names, port bindings for container and host, memory and cpu required, environment variables and networking information

ECS Services

We can configure the way tasks run and also how many should run and spread them amongst your EC2 instances and can also be linked to load balancers

Clusters

We need to create an ECS Cluster in which we add services and task definitions

Clusters can be of type EC2 or Fargate. For EC2 Clusters the corresponding EC2 instance are automatically created along with an autoscaling group

A task definition depends on an ECS service in order to run
Creating an ECS service also adds a Docker container image for it inside the EC2 instances
We can update the number of tasks in the ECS service and corresponding ASG group if we want to scale

ECR

Let's talk about AWS ECR (Elastic Container Registry) which is a private repo for storing our Docker images

It is used for creating custom images locally and then push them to ECR so that they are available to use in ECS

To create a repo, in our CLI we need to

Authenticate our Docker client to the ECR registry
Build our Docker images
Tag the image
Push the image into ECR

Fargate

Let's talk about AWS ECS with Fargate

Fargate is a Serverless way of launching ECS Clusters

We only create the task definitions and AWS will run our containers. To scale we only increase the task number

We don't have to worry about managing EC2 instances anymore

ECS Task placement

Let's talk about ECS tasks placement

Task placement strategy: Determine where to place newly launched EC2 type tasks. Task placement constraints: Based on CPU, memory and available port. The same logic constraints also apply when scaling in and need to terminate tasks

Now about Task placement strategies we have

Binpack: Based on the least available amount of memory or CPU which is great for cost savings
Random: without any logical order
Spread: Based on specified value like instanceId
Mix them together

Summary

Most of these concepts mentioned above may not make perfect sense if you don't see them being applied on the AWS console but still they are good theory to know. Maybe that can be a future version of this sets of tutorials :)

Next week we're talking about Elastic Beanstalk where things start getting more interesting as we can see combination of several services.

AWS Learn In Public Week 6, Advanced S3, Glacier And Athena

Harris Geo 👨🏻‍💻 — Mon, 05 Apr 2021 15:13:04 +0000

Week 6 of my AWS learning journey. This week is diving deeper into what S3 can provide us. We're going to do a quick overview of a couple new services Glacier and Athena. Let's go into the details.

S3 Replication

Let's talk about AWS S3 Replication

To do that we must enable versioning in both source and destination buckets
We have Cross Region Replication (CRR) which is ideal for compliance, lower latency access and also when you want to replicate across accounts

Then we have same region replication (SRR) which can be for log aggregation or live replication between production and test accounts

In either case, buckets can be in different accounts
Copy is asynchronous and we must give proper IAM permissions

After enabling S3 Replication, only new objects are replicated and not everything.

When deleting. Deletes without version ID add a delete marker which is not replicated.
Deleting with a version ID, it deletes in the source and is not replicated

We cannot do "chaining" of replication.

That means that if Bucket A has a replication into Bucket B which then has a replication into Bucket C and we add an object into bucket A, it won't make it all the way to Bucket C

S3 pre-signed URLs

Let's talk about AWS S3 pre-signed URLs

We can generate links that have the same permissions on the file as when we open it via the AWS console.
Such links can be generated using the CLI (downloads) or SDK (uploads)
By default they are valid for 3600 seconds but we can change the timeout with the --expires-in x seconds argument
We can use them for cases like
- Share link to content only with logged in users
- Allow temporary actions for users

Here's the CLI command for it

https://pbs.twimg.com/media/Exuf1ucXAAM8HgC?format=jpg&name=medium

S3 Storage classes and Glacier

We're going to do a quick overview of AWS S3 Storage classes and Glacier!

1. S3 standard which is a general purpose with high durability across multiple AZs.

99.99% availability throughout the year.
S3 standard is great for Data analytics, mobile & gaming applications and more

2. S3 Standard-Infrequent Access (IA)

For data that are less frequently accessed but require almost instant access.
It is also high durability across multiple AZs
It has 99.9% availability
IA is good for Disaster recovery, backups etc

3. S3 One Zone-Infrequent Access

The same principle but only in a single AZ but 20% cheaper
It has 99.5% availability
Supports SSL for data in transit and encryption at rest
Good for secondary backups and other data you can recreate

4. S3 Intelligent tiering

Similar to S3 standard but automatically moves objects between tiers based on access patterns
It has a small monthly monitoring and auto-tiering fee

5. Glacier

Low cost and meant for archiving or backups
Good for long term retention like 10s of years
Archives are stored in vaults
There is a cost to retrieve which gets more expensive for faster retrievals (1 minute to 12 hours)
The minimum storage duration is 90 days.

6. Glacier Deep Archive

For looong term storage and really cheaper than all the other options
However the fastest way to retrieve vaults is 12 hours
The minimum storage duration is 180 days (half a year)

To give you an idea of the costs, for S3 standard it is $0.023 per GB. For deep Glacier it is $0.00099 per GB

https://aws.amazon.com/s3/pricing/

S3 lifecycle rules

Let's talk about AWS S3 lifecycle rules

We can move between storage classes based on how often we access our objects.
For infrequently accessed object, move them to standard IA
For archives and object we don't need instantly we can move them to Glacier or Deep Archive
We can even automate that with transition actions which are definitions when objects are transitioned to another storage class
- Example: Move objects to Standard IA class 60 days after creation. Then archive them to Glacier after 6 months
We can also set expiration actions where we configure object to be deleted after a set amount of time
- Example: Access log files can be set to delete after 365 days. Such files can be old versions or incomplete multi-part uploads
We can create these rules based on prefixes like s3://somebucket/archives/*
We can also add rules based on certain objects tags like Department: Sales

AWS Athena

Today let's talk about AWS Athena

It is a Serverless service to perform analytics directly against S3 files and uses a SQL language to query these files
It supports CSV, JSON and more.
Athena is quite common for cases like BI, analytics, reporting, ELB and CloudTrail logs and more

Summary

I have to admit, most of these features are not really that common out there. In my last 2-3 jobs I have only seen a very basic way of using S3 comparing to what we discuss here. However, based on the examples these features must definitely be used by companies especially when there are so many different storing tiers.

Next week we are going to talk about the service that caused me to want to learn more about how AWS works. That is ECS where we are going to talk about managing Docker containers, ECR and Fargate.

AWS Learn In Public Week 5, S3

Harris Geo 👨🏻‍💻 — Sun, 28 Mar 2021 19:56:08 +0000

This week we're going to talk about S3. S3 is the service that the majority of developers are the most familiar with. Yet it is a service that can go really deep and has some crazy features that are not that known to the public. For that reason, I have split it in 2 parts.

In the first one we're going to talk about the basics, along with terminology and some concepts like CORS and more. Let's get started.

S3 Overview

Time for AWS S3 (Simple Storage Service)

A service to store objects (files) in buckets (directories)
Each bucket is created on a region level and its name needs to be globally unique

S3 Objects

Let's talk about AWS S3 Objects (files)

Everything after the bucket name is called a key
It is composed by a prefix and the object name
The UI will trick you that S3 has a concept of directories within buckets but it doesn't
Each object is max 5GB, otherwise multi-part upload

S3 Versioning

Let's talk about versioning in AWS S3

It can be enabled on a bucket level and same key will increment its version by 1,2,3
It is a good practice that can protect against accidental deletes and we can easily rollback
Files without prior version will have version set to null

S3 Object Encryption

SSE-S3. It encrypts S3 objects using keys handled and managed by AWS.

It is a server side encryption with the AES-256 algorithm and requires the header "x-amz-server-side-encryption": "AES256"

SSE-KMS which is managed by KMS (Key management Service).

Here we can manage access and also create an audit trail with access history.
It is also a server side encryption and requests the header "x-amz-server-side-encryption": "aws:kms"

SSE-C which is for managing your own keys. S3 doesn't store the key you provide.

It is HTTPS only and the encryption key is required in the header in every request

Finally we have the Client Side Encryption which requires the a library to manually be configured on the client side.

Such library can be Amazon S3 Encryption Client.
For that encryption type, the client needs to encrypt their data before upload and decrypt them after download

S3 Security

Today let's talk about AWS S3 Security.

We can have

👤 User based.

IAM policies for API calls allowed per user from the IAM console

🪣 Resource based.

Bucket policies that allow cross account access
Object ACL (Access control list) for finer grain
Bucket ACL less common

An IAM principal can access an S3 object if

The user IAM permissions allow it

or the resource policy allows it

and there is no explicit deny

We can enable MFA delete.

This needs the bucket to be version and will ask the user to add an MFA code in order to delete an object.

This can only be enabled from the AWS CLI

Finally we have pre-signed URLs.

These are URLs that give the same permissions as the user generating them and are valid for limited time.

These can also only be generated from the AWS CLI

An common use case is websites that offer premium video service for logged in users

S3 bucket policies

Let's talk a bit more about AWS S3 bucket policies

They are JSON based policies for

Resources like buckets and objects
Actions like set of API to allow or deny
Principal where the account or user to apply the policy to

The AWS recommended way for creating policies is via the policy generator

https://awspolicygen.s3.amazonaws.com/policygen.html

Examples of policies.

Grant public access to the bucket
Force upload encryption for objects
Grant access to another account (cross account)

CORS

Let's talk about the CORS concept and how is it applied in AWS S3

CORS stands for Cross-Origin Resource sharing and is a web browser based mechanism to allow requests to other to other origins while visiting the main origin

CORS has 3 components.

For https://example.com

Protocol which in this case is HTTPS
Host which is http://example.com
Port which is 443 due to HTTPS (80 for HTTP)

Websites on the same domain and subdomain are same origin

Like https://example.com/hello or https://example.com/about

Different origins are all the rest like https://client.example.com and https://api.example.com

For S3 CORS

If a client does a cross-origin request on our S3 buckets, we need to enable the correct CORS headers
We can add * to allow all origins or specify the one we want

S3 Consistency model

Let's talk about the AWS S3 consistency model.

That problem is due to data replication after we add new objects.

Sometimes if we query that object straight away instead of a 200, we may get a 404.

This is called "eventually consistent"

Eventual consistency can also happen when updating or deleting objects.

For updates we might get the old version of the object and for deletes we might still get a 200.

Here's how to enable "strong consistency" https://aws.amazon.com/s3/consistency/

MFA Delete

We can now go a bit deeper and talk about some more advanced AWS S3 concepts and more specifically MFA Delete.

This is a security setting which requires the user to put their MFA code in order to delete an object
It can only be enabled or disabled via the CLI of the AWS root account and requires bucket versioning to be enabled.
MFA code will be required only for permanently deleting an object version or suspending versioning on the bucket.

S3 Access Logs

Let's talk about AWS S3 access logs

For audit purposes, we can log all access to S3 buckets

Any request to S3 from any account authorised or denied will be logged into another S3 bucket

We can then use Amazon Athena (or other data analysis tools) to analyse these data

A huge warning here is to not enable the same bucket for monitoring and logging

That is because we can end up in an infinite loop and wake up to a gigantic bill due to the exponential growth of the bucket size

Summary

I really enjoyed learning more about S3. I now feel comfortable that I can do some more advanced configuration with S3 and feel confident using it. Where there any facts out of the ones I posted today you would like to add to? Next week is getting deeper into S3 and we will also talk about Glacier and Athena.

AWS Learn In Public Week 4, Route53 and VPC

Harris Geo 👨🏻‍💻 — Sun, 21 Mar 2021 19:34:05 +0000

The AWS learning challenge continues. This week was about Route53 which is literally the first time I touch this service and then some deeper dive into VPCs. Route53 gives a lot of flexibility and customisation but it is quite expensive for buying new domains and also you have to pay to transfer your domain which was a bummer. Nevertheless I gave it a try and it now makes a lot of sense.

Afterwards, in the VPC section there were some a bit advanced topic which I have to admit are difficult to make sense in a tweet. One of the biggest advantages I have seen so far is that it is a lot easier to follow devops related conversations and understand why certain decisions were made. Enough with me blabbing let's go to the content.

Route53

Let’s talk about AWS Route53

It is a managed DNS (Domain Name System) service which contains a collection of rules and records for reaching a server through its domain name

The most common records include

A which is a hostname to IPv4
AAAA which is hostname to IPv6
CNAME which is hostname to hostname
Alias which is hostname to an AWS resource like ELB

CNAME vs Alias

Let's do a comparison of when to use CNAME vs Alias in AWS Route53

AWS resources expose a hostname that looks like

http://abc-123.eu-west-2.elb.amazonaws.com. You want that to point to http://yoursite.example.com

The CNAME points a hostname to another hostname. Only for non root domains

e.g. http://yoursite.example.com -> http://another.site.com

An Alias points a hostname to an AWS resource, is free and provides native health check

e.g. http://another.site.com -> http://abc-123.eu-west-2.elb.amazonaws.com

Route 53 Overview

Let's do an overview of AWS Route53

It can use

🔓 Public domain names you own

🔒 Private domains within your VPC

Advanced features

⚖ Client load balancing (through DNS)

🩺 Health checks

📃 Multiple routing policies

Each site lives in a hosted zone which is $0.50 per month

Route53 Routing policies

Let's talk about AWS Route53 routing policies

Weighted routing policy

Control the % of the requests that go to specific endpoints

Helpful for AB testing

Helpful for splitting traffic between regions

Can be associated with health checks

Latency routing policy

Redirect to the server that has the least latency close to us

Latency of users is a priority

Latency is evaluated in terms of users designated AWS region

Some from Greece can be redirected to the UK is the latency is lower

Failover Routing Policy

Route53 does a health check to the primary instance

If it is unhealthy then Route53 will failover to the secondary instance (Disaster Recovery)

Geo Location routing policy

Routing based on location

Different from latency based policy

Traffic form the UK should go to specific IP

Should create a default policy in case there's no match

Multi Value routing policy

Use when routing traffic to multiple resources

Want to associate Route53 health checks with records

Up to 8 healthy records are returned for each Multi Value query

Multi Value is not a substitute of having an ELB

Route53 health checks

Finally let's talk about health checks in AWS Route53

We ping a server and expect it to respond

A server becomes unhealthy when X health checks fail (default is 3)

Then when Y health checks pass it is marked as healthy (default is 3)

The default interval is 30s between each health check but can be set to 10s but it's more expensive

There are around 15 health checkers that will check the endpoint health

That means that there's 1 request every 2 seconds for the default interval

There are HTTP, TCP and HTTPS healthchecks and don't use SSL verification

We can integrate them with CloudWatch

As we saw in the routing policies, health checks can be linked to Route53 DNS queries

VPC Summary

Let's do a summary of VPC components in AWS

☁️ VPC: A private network (Virtual Private Cloud) to deploy your resources

🆎 AZ: Availability Zones, different data centres in a AWS region

📡 Internet Gateway: At the VPC level to provide Internet Access

🚦 NAT Gateway / Instances: Giving internet access to private subnets

🚥 NACL: Stateless subnet rules for inbound and outbound traffic

🛡 Security Groups: Stateful rules that operate at the EC2 instance level or ENI (Elastic Network Interfaces)

🤝 VPC Peering: Connect 2 VPCs with non overlapping IP ranges

📃 VPC Flow logs: Logs for network traffic

🔐 Site to Site VPN: VPN over public internet between on-premises data centre and AWS

🔌 Direct Connect: Direct private connection to AWS

3 tier solution architecture

Let's talk about a typical 3 tier solution architecture in AWS

1. Public subnet

Route53 talking to our ELB
Our public ELB

2. Private subnet

Our EC2 Instances
ASG amongst AZs
ELB connects to EC2 instances using route tables

3. Data subnet

RDS & ElastiCache

Internet Gateways and NAT Gateways / instances

Let's talk about AWS IGWs (Internet Gateway) and NAT gateways / instances

IGWs help our VPC instances connect to the internet with Public Subnets
NAT Gateways and NAT instances allow our VPC instances in your Private Subnets to access the internet while remaining private

In simple words

If we want to give internet access to a VPC in a Public Subnet then we need a IGW
If the VPC is in a Private Subnet then we need a NAT gateway (AWS managed) or a NAT instance (self managed)

Security Groups and NACL

Let's talk about AWS Network security with NACLs (Network ACL) and SGs (Security Group)

🔥 Network ACL is a firewall which controls traffic to and from subnets

🚦 Supports Allow and Deny rules

📗 Attached at the subnet level and only allow rules for IP addresses

Security groups are firewalls which control traffic to and from an ENI (Elastic Network Interface) or EC2 instance

✅ Can only have ALLOW rules

📒 Can include IP addresses and other security groups

Security Groups VS NACL

Let's do a quick comparison of AWS SG (Security Groups) vs NACL (Network Access Control Lists)

For SGs

Operate at the instance level
Support allow rules only
Stateful: Traffic is automatically allowed regardless of any rules
Rules need to be evaluated before deciding whether to allow traffic
Can be applied when launching the instance or can be edited later on

For NACLs

Operate at the subnet level
Support allow and deny rules
Stateless: Traffic to be explicitly allowed by the rules
Process rules in order when deciding to allow traffic
Automatically applies to all instances in the subnets and doesnt rely on users creating SGs

VPC Flow logs

Let's talk about AWS VPC Flow logs

Capture information about IP traffic into your interfaces (VPC, Subnet Flow Logs, ENI Flow Logs)
Helps to monitor and troubleshoot connectivity issues like Subnets to the internet, Subnets to other Subnets and Internet to Subnets
Captures network information from AWS managed interfaces too. (ELB, ElastiCache, RDS etc)
VPC Flow logs data can go to S3 or CloudWatch logs

VPC Peering

Let's talk about AWS VPC

It is a connection for allowing 2 VPC being part of the same network
They are privately connection and they behave like they are on the same network.
They must not have overlapping IP address ranges (CIDR)
VPC peering connections are not transitive
- VPC1 is connected to VPC2
- VPC2 is connected to VPC3
- VPC1 is NOT connected to VPC3

VPC Endpoints

Let's talk about AWS VPC Endpoints

They allow you to connect to AWS services using a private network instead of a public one
That results in enhanced security and lower latency to access AWS services
VPC Endpoints gateways are popular for S3 and DynamoDB
For all the rest of the AWS services we have VPC Endpoint Interfaces
VPC Endpoints are only used within your VPC

Site to Site VPN and Direct Connect

To close the VPC section, let's talk about AWS Site to Site VPN and Direct Connect

Site to Site VPN is for connecting at on-premises VPN to AWS with an automatically encrypted public connection

Direct Connect (DX) is a physical connection between on premises and AWS. That connection is private, secure and fast. However, it takes at least a month to establish.

Note. Neither of them can access VPC endpoints

Summary

Route53 was a lot more interesting than originally anticipates. It had some great examples of ways we can split traffic which I thought would normally be handled by the load balancer. Then diving deeper into VPCs was quite interesting because they are great for sharpening my networking skills which are not my strongest card.

I would love to hear from you about the format and any suggestions on how to make this challenge more beneficial for the people out there that follow my updates. Next week is all about S3!

AWS Learn In Public Week 3, EBS, EFS, RDS and ElastiCache

Harris Geo 👨🏻‍💻 — Sun, 14 Mar 2021 21:34:48 +0000

The learning AWS journey continues. This week was all about storage. Because there were too many things to share, my timeline was quite busy with multiple tweets per day.

At the beginning I started with EBS (Elastic Block Store) and EFS (Elastic File System) which are about HDD type of storage for EC2. That was interesting because I learned about a few hardware terms that I was not aware of.

The week continued with RDS which I initially thought of skipping as "it was the AWS service i'm the most familiar with" but thankfully ended up doing it and learned a lot about how to scale Databases in AWS. Especially the part about Aurora DB was really exciting.

The week ended with ElastiCache which also revealed some great techniques for managing cache. Let's go straight to the Tweets.

EBS

What is an AWS EBS (Elastic Block Store)

A volume in which you store your EC2 data so that they don't get lost when the instance is terminated

An EBS is network drive which works like a USB stick on the cloud.

⏳ It has a bit of latency

🔌 It can quickly be attached to other instances

🔒 It is locked to an AZ. To move it across you need to snapshot it

💰 You are billed for the provisioned capacity, not the one you use

EBS Types

There are 4 types of AWS EBS (Elastic Block Store)

GP2: A general purpose SSD that balances performances and price
IO1: A high performance SSD for mission critical low latency / high throughput work (good for large DBs) Only these 2 can be boot volumes.
ST1: Low cost HDD for frequently accessed and throughput intense workloads
SCI: Lowest cost HDD for less frequently access workloads.

All 4 are characterised in size, throughput and IOPS (I/O Ops Per Second)

Instance Store

Finally for AWS EBS we have the instance store.

It is a physical HDD attached to the instance.

➕ I/O performance, good for cache / temp content, survives reboots

➖ Lost on stop / termination, can't resize and require manual backups

EFS

What is an AWS EFS (Elastic File System)

A managed NFS (Network File System) that can be mounted on multiple EC2 instances

It is multi AZ, highly available and scalable
It is pay per use and is good for web serving, data sharing and Wordpress.

Unlike EBS, it has no capacity planning and uses a pay-per-use pricing model which scales automatically.

📈 Can have thousands of concurrent NFS clients and can grow to Petabytes scale.

EBS vs EFS

Let's compare EBS and EFS

For EBS

1 instance at a time locked in 1 AZ.

To migrate we need to take snapshots and restore them to another AZ.

When running backups they use a lot of IO and shouldn't be run when app handles lot of traffic.

Root volumes are lost upon termination

For EFS

We can mount hundreds of of instances across multiple AZs

We can use them to share website files but is Linux only.

Is ~ 3 times more expensive than EBS. For cost savings can use EFS-IA (cost to retrieve files, lower to store)

RDS

What is an AWS RDS (Relation Database Service)

An AWS managed SQL only database which can be one of: Postgres, MySQL, MariaDB, Oracle, MSSQL and Aurora

As it is a managed service, we cannot SSH into our machine but only connect to the remote DB

Why RDS and not deploying a DB on EC2?

Automated provisioning, OS patching. and maintenance windows for upgrades

Continuous backups and can restore to specific timestamps

Option for multi AZ setup for Disaster recovery

Horizontal / vertical scaling

EBS storage (GP2 or IO1)

RDS Backups

Let's talk about backups in AWS RDS

They are automatically enabled

Daily during the maintenance window

Transaction logs every 5 minutes with ability to
restore to any point in time up to 5 minutes ago

7 days retention of backups (up to 35)

Now let's talk about snapshots.

The difference between a backup and a snapshot is that a snapshot is manually triggered by the user unlike the backups which are automatic.

Because they are manual, the retention period is as long as the user wants.

For IAM auth, we don't need a password, just authenticate with IAM authentication tokens.

That IAM authentication token is short lived and has lifetime of 15 minutes

RDS Read Replicas

What are AWS RDS Read Replicas?

They are additional servers which have copies of our DB and are used for read scalability.

RDS supports up to 5 read replicas that can be within AZ, across AZ and even cross region.

Replication is ASYNC so reads are eventually consistent.

Replicas can be promoted to their own DB.

Each replica has its own connection string which an app needs to have to connect to.

A use case of using read replicas.

Your prod DB is taking on normal load and you want to run a reporting app to run some analytics.

You can create a read replica to run the new workload there and the prod DB is not affected.

RDS Disaster Recovery

Now let's talk about RDS multi AZ DR (Disaster Recovery)

It is a SYNC replication which is when a change happens on main, also needs to happen on the secondary replicas.

There's 1 DNS name and supports automatic app failover to secondary replicas to increase availability.

The failover might happen in case of loss of AZ, loss of network, or instance / storage failure.

No manual intervention in apps and used for scaling.

Can set read replicas as multi AZ to ensure even higher availability.

RDS Encryption and Security

Let's talk about AWS RDS encryption and security.

There are 2 types of encryption.

🛏 At rest encryption (data not in movement)

🛩 In flight encryption

For rest encryption, we can encrypt master & read replicas and is defined at launch time. Master not encrypted = replicas not encrypted

For in-flight encryption, we use SSL certificates to encrypt data to RDS in flight. SSL options will trust certificates when connecting to DB

To encrypt and un-encrypted RDS DB

✅ We create a snapshot

✅ We copy is and enable encryption

✅ We restore the DB from the encrypted snapshot

✅ Then migrate to new DB and delete old one

More about RDS security

More about AWS RDS security 🔐

RDS clusters are deployed within private subnets and not public ones.

We can add Security Group rules which uses the same inbound / outbound logic as EC2.

We can also setup IAM policies about who can manage RDS.

To login to the Database we use the traditional username and password way.

For MySQL and Postgres in RDS we can also use IAM based authentication

Aurora DB

Today we're going to talk about AWS Aurora DB.

🥷 Not open sourced

✅ Supports MySQL and Postgres

💪 Cloud optimised with 5x performance over classic RDS

🏔 Automatically grows up to 64TB (starts with 10GB)

📈 Can have up to 15 replicas with replication process of 10ms

🌋 Instantaneous failover being high availability native

💸 20% more expensive than classic RDS

Aurora provides Serverless plans in certain regions which is great for automated DB instantiation and auto-scaling

It's great for infrequent or unpredictable workloads

Requires no capacity planing and you pay per second which makes it super cost effective

Aurora Global

Let's talk about AWS Aurora Global

Cross Region read replicas that are easy to setup and useful for disaster recovery

The Aurora Global database gives 1 primary region for reads and writes and can have up to 5 secondary read regions with 16 replicas (80 in total)

ElastiCache

Today let's talk about AWS ElastiCache

It is a managed Redis or Memcached service

Caching is a great way to help reduce load off our DB and make our app stateless

Elastic cache has write scaling using sharing and read scaling using Read Replicas

ElastiCache Redis

It is multi AZ with auto failover

You can enhance the reads with read replicas for high availability

Even if your cache restarts, you still have access to your data using AOF (Append Only File) persistence

Then you can backup and restore features

ElastiCache Memcached

Uses multiple nodes for partitioning data (sharding)

Unlike Redis, if the cache restarts, all of the data has been lost

Also there is no option to backup and restore your data

ElastiCache implementation methods

Lazy loading 🥱 (or cache-aside or lazy population)

When reading from the cache

If data found in the cache, return them (cache hit)

If they aren't there (cache miss), read from the DB, write them into the cache and then return them

➕
we only cache data that is used
if there's a failure, it's not fatal as we still have the DB

➖
In case of cache miss, we do 3 round journeys to the DB and cache
We might cache data that will not be used again and need to also implement an invalidation strategy

Write through ✍🏼

That's when we write to the DB and also to the cache

➕
Data in cache is always up to date

Reads are always fast

No one expects writes to be ultra fast

Makes sense from a UX point of view

➖
In case a page needs to read before a write takes place, there will be no data in the cache

In many cases we rely lazy loading to be there as well

We add too much into the cache and it is very likely that a lot of data will never be read

Cache evictions and TTL (Time to Live) ⌛

When we explicitly define that cache will be available for x seconds and that it will automatically be deleted

The items that were Least Recently Used (LRU) can be evicted

TTL are good for leaderboards, comments on social media and activity streams

They can last from seconds, hours or even days

If there are too many evictions due to memory, then we need to scale either vertically or horizontally

Summary

Uuu boy, that was a lot this week. I really enjoyed it though, especially the RDS and ElastiCache part. It is really good to see that AWS training not only educates you on their services but on architectural / scaling concepts as well. These concepts are pretty much applicable to any cloud service.

Next week we're looking at Route53, some more stuff about VPCs and S3.

AWS Learn In Public Week 2, Load Balancers and Auto Scaling

Harris Geo 👨🏻‍💻 — Sun, 07 Mar 2021 16:41:35 +0000

Week 2 of my AWS challenge and things are going great! Learning how ELBs (Elastic Load Balancer) and ASGs (Auto Scaling Group) work connects many dots on the wider topic of understanding the cloud infrastructure.

The deeper I dive into all these concepts, the more interesting things get and it makes a lot more sense on many aspects of full stack development. Here's a breakdown of what happened this week.

28/02

What is an AWS ELB (Elastic Load Balancer)?

A server that evenly distributes internet traffic to multiple EC2 instances.

Load balancing is a good way to ensure high availability to your system in case one of the instances goes down.

01/03

AWS provides 3 kinds of load balancers

✅ CLB (Classic Load Balancer)

✅ ALB (Application Load Balancer)

✅ NLB (Network Load Balancer)

CLB is old generation and supports HTTP, HTTPS and TCP

ALB is the most common amongst modern applications and supports HTTP, HTTPS and Websockets

NLB is mainly used for high performance job and supports TCP, TLS and UDP

02/03

What is Load Balancer Stickiness?

A cookie to instruct our load balancer to redirect each independent user to the same instance.

➕ works for CLB and ALB

➕ can control the expiration date

➖ can bring imbalance to the load balancer

03/03

What is SSL (Secure Socket Layer) and TLS (Transport Layer Security) in AWS?

Both of them are certificates which allow traffic between your clients and your load balancer to be encrypted in transit (also referred to as in flight encryption)

Good to knows.

TLS is a new version of SSL.

People quite often use TLS but refer to them as SSL.

04/03

What is Connection Draining in AWS Load Balancers?

The time to complete "in-flight requests" while the instance is de-registering or unhealthy.

Once the instance is de-registering, it stops receiving requests.

This process can take 1-3600 seconds.

05/03

What is an AWS ASG (Auto Scaling Group)

A setting for managing changes to traffic with a goal to:

📈 scale out (add EC2 instances) to match increased load

📉 scale in (remove EC2 instances) to match decreased load

What does ASG look in AWS? We have

✅ Minimum size which is the lowest amount of instances required for our system to be functional.

✅ Actual size / desired capacity: the setup normally we have

✅Maximum size - the amount of instances we can allocate to handle the extra load

06/03

Let's talk about AWS ASG policies. We have 3 kinds 💈

Target tracking scaling

It is quite simple and the easiest one to setup.

It is when you set rules of e.g. the CPU usage to stay at around 40%

Simple or step scaling

➕ When a CloudWatch alarm is triggered (e.g. CPU > 70%) then add 2 units

➖ When a CloudWatch alarm is triggered (e.g. CPU < 30%) then remove 1 unit

Scheduled Actions

This is based on know usage patterns like on a busy weekend or on BlackFriday

Finally let's talk about AWS ASG scaling cooldowns

A cooldown is a period to ensure your ASG doesn't add or remove instances before the previous scaling activity takes place.

The default cooldown period is 300 seconds. Can be reduced to 180.

Example: A policy that terminates instances based on criteria or metric, EC2 Auto Scaling needs less time to determine wether to terminate additional instances

If your application is scaling up or down multiple times each hour, then that's a good indicator that you need to modify your ASG cooldown timers.

Week 2 Summary

Learning AWS is quite interesting because it feels like even the small bits help a lot. I find it quite easy to wake up in the morning because early AM before work now means AWS time.

Understanding load balancing answers many questions on how big sites handle their traffic. Some things I have noticed is that getting this concept of how things work in the cloud changes a lot the way I started thinking about development. More of that in the next blog posts to come.

If you want to follow my journey, feel free to follow me on Twitter and please reach out :). Next week I will be looking at AWS EBS (Elastic Block Store), EFS (Elastic File System) and RDS (Relational Database System).

Setup Your AWS Free Tier Alerts and avoid any surprise charges

Harris Geo 👨🏻‍💻 — Thu, 04 Mar 2021 07:47:03 +0000

This is the second AWS blog as part of my 100DaysOfAWS challenge. Let's see how far I can reach.

If you're like me and you want to explore the world of AWS but are sceptical about not ending up with a gigantic bill then that blog post is for you.

Let's login with our root account and click on your username and then my billing dashboard. Warning, this only works with the root account. If you have another account that is given same permissions as the root one it will not work.

Under preferences, select billing preferences. In case you leave in a country other than the US you might want to be charged not in USD.

Billing preferences

In that page you will see a few options on how to receive alerts. For now let's check the receive free tier usage alerts as well as receive billing alerts. Under the latter option, there is a link the manage billing alerts which will lead us to CloudWatch to do a bit further setup let's click on that. This is where we can also change our billing currency. You can also search for the CloudWatch service if you cannot see this option.

On the main CloudWatch page, on the left we can see a section for alarms. Let's click on the billing option of it. Here we can create billing alarms to ensure that we're not going above our free tier limit. This tier gives us 10 free alarms and 1000 free email notifications each month. Now let's click on the create alarm button.

Here there are some steps we need to go through. In the first one for metric and conditions, we can see that there is an option for currency. Let's change that to GBP because we live in the UK. If we scroll down there is an option to define the threshold value. Let's put 5 GBP there. That means that if our bill goes above £5 per month, we will get notified. Modify that amount to something you are comfortable with. Let's click next.

Now let's create an SNS topic so that we are subscribed to that alert. In the SNS topic section, click on create new topic. Now let's enter a name for that topic. That name has to be unique so the more descriptive the better. I named mine Free_tier_exceeded. Then we can add the email we want this subscription to notify. Click on create topic.

There is a link to view that notification in the SNS (Simple Notification Service) console, let's click there. It opens the SNS console in a new tab. Under subscriptions we see the one we just added which has status pending confirmation. If we open our email we put earlier, there is a new email to confirm the subscription. Before we confirm anything let's first finish creating the alarm.

Back at CloudWatch tab, for now let's ignore the rest of the options and go straight to the next step. Now let's give our alarm a name and description. I put Free_tier_alert and Email me when my bill goes above £5. Let's click next.

Now we can see a summary of everything we just put along with a Graph for estimated charged. Finally let's click on create alarm. The alarms page in CloudWatch should look like that.

Now it's time to approve the subscription email we received earlier. Clicking on the refresh button of the billing alarms page, now shows the actions set to 1 action(s). The state however is still showing us insufficient data which is ok as we do not have any charges yet.

Now we can go and start playing around with AWS services and in case we forget to remove something that charges us over time, we will be notified. Happy hacking.

AWS Learn In Public Week 1, the EC2 basics

Harris Geo 👨🏻‍💻 — Sun, 28 Feb 2021 16:36:58 +0000

Hello there,

Last weekend due to a gaffe at work, I decided to invest and improve my AWS skills. It has been a desire of mine for a few months now to go for the AWS developer associate certification.

I really like the bite size blogging experience in Twitter and felt like that this method could motivate me to do it consistently and also others can learn along with me. I "signed up" to the 100 days of AWS challenge and also to learn in public hashtags.

I got some really positive feedback when I made this announcement on Twitter. My goal was to post at least one fact about what I was studying that day every single day.

My learning usually happens early in the morning before work and my tweet has been at around the time I take my lunch break. Let's see how did the first week go. I started with the AWS EC2 basics.

21/02

What is an AWS Virtual Private Cloud (VPC)?

An Amazon VPC enables you to launch AWS resources into a virtual network that you've defined.
Think of it as a traditional network within your data centre that uses AWS infrastructure.

22/02

What is an AWS Availability Zone (AZ)?

AZs are AWS data centres that can be found within AWS regions. Each region has multiple AZs.
As a best practice to ensure high availability for your system, a VPC can span multiple AZs.

23/02

What is an AWS subnet?

A range of IP addresses in your VPC.

✅ you can launch AWS resources into a subnet that you select

❌ you cannot launch any instances without subnets

A subnet is always mapped to a single AZ
As a best practice subnets should be spread amongst AZs for redundancy and failover purposes.

24/02

Types of AWS subnets

Public subnets for things that are connected to the internet. e.g. web servers
Private subnets for things that are not connected to the internet e.g. Databases

25/02

Time for Networking in AWS.
What is an AWS internet gateway?
A horizontally scaled, redundant and highly available VPC component that allows communication between instances in our VPC and the internet.

❗ Each VPC can only have 1 internet gateway

26/02

What is an AWS AMI (Amazon Machine Image)?
The software and operating system that will be used in our system when launching EC2 instances.

27/02

AWS Tip 💡

Every time you get timeout errors it's very likely to be related to your security group settings.
Start by investigating your inbound / outbound rules.

Week 1 Summary

The experience has totally been positive! Still trying to figure out what is the best way to share that information but come on, the purpose of such challenge apart from learning a new skill, is to see how much you can improve during time.

If you want to follow my journey, feel free to follow me on Twitter and please reach out :). Next week I will be looking at AWS ELB (Elastic Load Balancer).

Docker in plain English part 1. Building and running Docker containers

Harris Geo 👨🏻‍💻 — Sun, 24 Jan 2021 13:55:43 +0000

Docker is a great tool and it is really useful for automating our workflows. I have been using Docker for many years, yet sometimes I find myself forgetting the basic commands.

The scope of these series is to give some really simple examples of how to use Docker. In this article we will dockerise a static website. For those not familiar with the Docker vocabulary, dockerise is when we put some app into a Docker container. Before we start, let's download and install the docker CLI from the official website.

The static website

Here's a dead simple website that we are going to dockerise.

<html>
  <head>
    <title>Simple website</title>
  </head>
  <body>
    <h1>Hello there</h1>
    <p>
      I am a simple website and I live inside a Docker container.
    </p>
  </body>
</html>

Let's copy that code, open our terminal / code editor and paste it into an index.html file. Now let's create a Dockerfile.

The Dockerfile

We can now create a new file with the name Dockerfile and add the following code into it.

FROM php:7.0-apache
COPY . /var/www/html/

What we're doing here is we're telling the container to install apache. Then the next step is to copy everything from the current directory and put it into /var/www/html/ which is the directory apache will look at for HTML (and not only) files. Now we have everything that we want in order to build our Docker image.

Building the Docker image

We have configured what the Docker image is going to contain so let's build it.

docker build -t static-website .

The -t flag stands for tag and is a good practice that will simply make our lives so much easier when looking at long lists of Docker images.

The first time we run that command, it is going to take a few seconds or minutes depending on our internet connections. That is because Docker has to download the image we specified in the Dockerfile.

We can see all of the available images with the docker images command. Our command line should look like this.

➜  docker images
REPOSITORY           TAG          IMAGE ID       CREATED              SIZE
static-website       latest       3152d04a164f   About a minute ago   368MB
php                  7.0-apache   aa67a9c9814f   2 years ago          368MB

Now that we have a Docker image for our website, we can spin up some Docker containers.

Running our Docker container

To spin up our Docker container we simply run the following command.

docker run -d -p 4000:80 --name my-website static-website

Wow there's a lot of things happening here. Let me quickly explain what's going on.

-d is a flag that is telling Docker to detach that container from the process and run in the background. If we don't include that flag then killing the root process or in other words closing the terminal will also stop the container
-p is a flag that stands for port. This where we tell docker what port to make that container available on in the outside world and then map it to the port used internally. In this case apache uses port 80 and we want to make that available on the port 4000 of our localhost
--name is quite straightforward. It is the name we want to give to our Docker container
the final argument static-website refers to the Docker image we want to use to run the Docker container

You may notice that once we run that command a long string is returned. That string is the id of the container and we can use it to stop it. However due to the amount of output that might be generated while running that command, we can also view all of the containers that are currently running with docker ps. That should look like this.

➜  docker ps
CONTAINER ID   IMAGE             COMMAND                  CREATED          STATUS          PORTS                            NAMES
9254225687cd   static-website    "docker-php-entrypoi…"   11 minutes ago   Up 11 minutes   3000/tcp, 0.0.0.0:8001->80/tcp   my-website

Now if we open http://localhost:4000 we should be able to see our static website which runs inside a Docker container we just created! 🎉

Stopping containers

While experimenting with Docker we might have noticed that we have created multiple images and containers. Given the example from above, to stop the container we can do the following.

docker stop 9254225687cd
# or
docker stop my-website

This is where I find that naming containers comes in quite handy. When stopping a container, that does not mean that we have removed it. Running docker start my-website should run it again.

Tip: Remember that docker ps will only show us the containers that are currently running. To see everything, including the ones that are stopped we can run docker ps -a

Cleaning up Docker containers and images

All of the images and containers we have created can take a lot of space. Now let's talk about how to clean things up. Running docker ps -a, we can view and then copy the ids of the containers we want to remove.

docker rm ac6f4b61de14

Tip: If you have more than once containers you want to remove you don't have to remove them one by one. You can pass all of them in the docker remove command like docker rm id1 id2 ....

There are also more commands that can help with cleaning up. However, they can be subject to different opinions of whether you should or should not use them so let's not talk about them yet. For the Docker images the logic is quite similar.

docker image rm edec9bdcc8d2

Aaaand that's it 🙌

Congratulations we now know how to get started with our Docker journey. Next step I would recommend to do is to try and Dockerise our React or you Node.js apps. The getting-started page of the official docs have some great examples.

In the next part we are going to talk about spinning up multiple containers using docker-compose.

Please subscribe to my newsletter if you enjoyed this post and you would like to get notified when new ones come out.

Did you enjoy this content? Subscribe to my newsletter and get notified when I post new stuff.