DEV Community: Deepak Porwal

Bridging vs Bonding in Linux Networking – Explained with Real-Life Analogies

Deepak Porwal — Mon, 02 Feb 2026 07:16:24 +0000

Hey folks! So I've been studying for my LFCS exam lately, and I hit this networking section about bridges and bonds. At first, I was confused - aren't they basically the same thing? Nope. Turns out they're completely different, and once I understood it, it clicked.

Let me share what helped me finally get it.

The "Aha!" Moment

You know how sometimes you need a simple analogy to make tech concepts stick? Here's what worked for me:

Bridging is like connecting two different neighborhoods with a flyover. People from both sides can now meet each other easily.

Bonding is like widening a single road by adding more lanes. Same road, just more capacity.

See the difference? One connects separate places, the other makes one place handle more traffic.

When I Actually Use Bridges

Last month, I was setting up my homelab (yeah, I know, typical DevOps thing). I had this Ubuntu server with one NIC, and I wanted my VMs to be on the same network as my laptop.

What did I do? Created a bridge.

Basically, a bridge in Linux is like a virtual switch living inside your server. You plug in your physical NICs and maybe some virtual interfaces (like for VMs), and boom - everything talks to each other like they're all connected to the same switch.

Here's the deal:

Each NIC you add to a bridge becomes a "port"
Devices on different segments suddenly act like they're on the same LAN
No routing needed, it's all Layer 2 magic

Real scenario from my homelab

I had:

My server's enp0s3 connected to my home router
A couple of VMs that needed IPs from the same DHCP pool

Solution? Bridge enp0s3 with the VM tap interfaces. Now my router sees the VMs as if they're physical machines on my network. Pretty neat.

This is super common in:

KVM/libvirt setups
Docker networking (container bridges)
Transparent firewall configurations

Now Let's Talk Bonding

Bonding is a different beast. You're not connecting networks here - you're combining multiple NICs to act as ONE.

Think about it like this: you have two 1Gbps cables going to the same switch. Instead of using them separately, you bond them into bond0, and from your server's perspective, there's just one fat pipe.

Why would you do this?

Three solid reasons:

1. High Availability

If one cable gets unplugged (trust me, it happens), traffic automatically fails over to the other NIC. Your app keeps running. No downtime.

2. More Bandwidth

Depending on the bonding mode, you can actually push traffic through multiple NICs simultaneously. More lanes = more cars.

3. Peace of Mind

One flaky link? No problem, the bond shifts traffic to the healthier ones.

I've seen this save production systems multiple times. One time at work, someone accidentally knocked a cable loose in the datacenter. We didn't even notice until the monitoring alert came in - the bond had already failed over.

Bonding Modes - This Confused Me At First

Okay, so Linux supports 7 bonding modes (0 through 6), and honestly, you don't need to memorize all of them. But here are the ones I actually use:

Mode 1: Active-Backup (My Go-To)

This is the simplest and most common one I've used. One NIC is active, the others are just chilling on standby. If the active one dies, a standby instantly takes over.

When I use it:

Production servers where uptime matters
When I can't touch the switch config (which is... often)

The catch:

You're not getting any extra throughput. Only one link is active at a time.

Mode 4: LACP (The "Proper" Way)

This is 802.3ad - the official standard for link aggregation. Both your server and the switch negotiate and form a team.

When I use it:

When I have access to managed switches (datacenter setups)
Need both redundancy AND more bandwidth

The catch:

Your switch needs to support LACP, and you need to configure it on both ends.

Mode 0: Round Robin (Lab Only)

Sends packets round-robin across all NICs. Great for testing, but can cause packet reordering issues in production.

The Others

There's Mode 2 (XOR), Mode 3 (Broadcast - rarely used), Mode 5 (Transmit Load Balancing), and Mode 6 (Adaptive Load Balancing). Honestly, I've barely touched these in real work. Mode 1 and Mode 4 handle 90% of my use cases.

Quick Decision Tree (This Helped Me)

Whenever I'm confused about which to use, I ask myself:

Question 1: Do I have multiple NETWORKS that need to talk?

Yes → Use a bridge
No → Keep reading

Question 2: Do I have multiple cables to the SAME network?

Yes → Use bonding
No → You probably don't need either

Question 3 (if bonding): Can I configure the switch?

Yes → Use Mode 4 (LACP)
No → Use Mode 1 (Active-Backup)

Seriously, this little mental checklist has saved me so much time.

A Quick Comparison Table

Thing	Bridging	Bonding
What it does	Connects different networks	Combines NICs on same network
Best analogy	Flyover connecting neighborhoods	Adding lanes to a highway
What apps see	Multiple networks reachable	One interface (bond0)
Common use	VMs, containers	Production servers, storage
My usage	Homelab, dev environments	Work servers, HA setups

Real-World Examples From My Experience

Example 1: My Homelab VMs

I create a bridge br0, plug in my physical NIC and VM interfaces. My VMs get IPs from my home router's DHCP. Easy.

Example 2: Production App Server

Two NICs, same VLAN, can't modify the switch. I create bond0 in Mode 1. One cable goes bad? Server keeps running. Simple as that.

Example 3: Database Server With Storage

Two 10GbE NICs, heavy storage traffic, and I control the switch. Mode 4 (LACP) all the way. I get aggregated bandwidth and failover.

For LFCS Exam Folks

If you're studying for the LFCS like me, remember these key points:

Bridge = virtual switch connecting interfaces; makes separate networks behave as one
Bond = multiple NICs presented as one logical interface
Mode 1 = Active-backup, simple HA, no switch config needed
Mode 4 = LACP, needs switch support, gives you both HA and bandwidth

In the exam, they'll probably ask you to configure one or the other using nmcli or ip commands. Just remember the concept - the syntax you can look up.

Final Thoughts

Honestly, once the bridge vs bond concept clicked for me, a lot of other networking stuff started making sense too.

Bridge = connect networks

Bond = combine NICs

That's it. That's the tweet.

If you're setting up a homelab, start with a simple bridge for your VMs. If you're managing production servers, look into bonding for HA. And if you're studying for LFCS, just understand the concepts - the commands are easy once you know what you're trying to achieve.

Hope this helps someone out there who was as confused as I was!

Feel free to drop any questions below - I'm still learning too, so we can figure it out together.

Cheers! ✨

AWS CDK Python

Deepak Porwal — Wed, 04 May 2022 05:21:14 +0000

Basics:

Resources and Identifiers

Any piece of Infrastructure that is created via CDK is a Resource.
Every Resource has an identifier which will help you to refrence it later.
A Resource in the CDK maps to a Resource in CloudFormation
The Resource Identifierwill also be the Identifierwith CloudFormation.

Constructs

A Construct is a logical grouping of one or more Resources.
Constructs are the building blocks with the CDK.
Constructs can be programmatically customized.
Constructs enable customizable reuse within an Organization.

We can also create another resource out of the construct.
Eg. we need to have a Resource (security group) and an Identifier with (WebSG)

To deploy all of the above resources, called Stack.

Stack

A stack is the unit of deployment within CDK.
A stack in the CDK corresponds to a stack in Cloudformation.
Stacks share cloudformation Stack limitations.
Every stack has an Environment that specifies account and region.
Environments can be either explicity or implicitly define.

App

A CDK App is the root of the context tree for a CDK Project.
An App can contain one or more stack.
Each stack within an App can contain its own environment.

CDK Workflow

Init --> Project is created using the command line tool
Bootstrap --> Create needed AWS Resources for CDK Enviroment.
Synth --> Generate Cloudformation template from Code.
Deploy --> Template are launched by Cloudformation
Update --> CDK Project is updated with new Infrastructure.
Diff --> Update Against deployed stack are Identified
Again Deploy.

Why use the AWS CDK

Let's look at the power of the AWS CDK. Here is some code in an AWS CDK
project to create an Amazon ECS service with AWS Fargate launch type (this is
the code we use in the Creating an AWS Fargate service using the AWS CDK).

class MyEcsConstructStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        vpc = ec2.Vpc(self, "MyVpc", max_azs=3) # default is all AZs in region
        cluster = ecs.Cluster(self, "MyCluster", vpc=vpc)

        ecs_patterns.ApplicationLoadBalancedFargateService(
            self,"MyFargateService",
            cluster=cluster, # Required
            cpu=512, # Default is 256
            desired_count=6, # Default is 1
                                task_image_options=ecs_patterns.ApplicationLoadBalancedTaskImageOptions(
            image=ecs.ContainerImage.from_registry("amazon/amazon-ecssample")),
            memory_limit_mib=2048, # Default is 512
            public_load_balancer=True) # Default is False

This class produces an AWS CloudFormation template of more than 500
lines.

Other advantages of the AWS CDK include:

Use logic (if statements, for-loops, etc) when defining your infrastructure
Use object-oriented techniques to create a model of your system
Define high level abstractions, share them, and publish them to your team, company, or community
Organize your project into logical modules
Share and reuse your infrastructure as a library
Testing your infrastructure code using industry-standard protocols
Use your existing code review workflow
Code completion within your IDE

Developing with the AWS CDK

The AWS CDK Toolkit is a command line tool for interacting with CDK apps. It enables developers to synthesize artifacts such as AWS CloudFormation templates, deploy stacks to development AWS accounts, and diff against a deployed stack to understand the impact of a code change.

The AWS Construct Library includes a module for each AWS service with constructs that offer rich APIs that encapsulate the details of how to create resources for an Amazon or AWS service. The aim of the AWS Construct Library is to reduce the complexity and glue logic required when integrating various AWS services to achieve your goals on AWS.

Note:
There is no charge for using the AWS CDK, but you might incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage. Use the AWS Pricing Calculator to estimate charges for the use of various AWS resources.

Getting Started:

Create the App:

Each AWS CDK app should be in its own directory, with its own local module dependencies. Create a new directory for your app. Starting in your home directory, or another directory if you prefer, issue the following commands.

mkdir hello-cdk
cd hello-cdk

Important
Be sure to use the name hello-cdk for your project directory, exactly as shown here. The AWS CDK project template uses the directory name to name things in the generated code, so if you use a different name, some of the code in this tutorial won't work.

Now initialize the app using the cdk init command, specifying the desired template ("app") and programming language.

cdk init TEMPLATE --language LANGUAGE

i.e

cdk init app --language python

![[Pasted image 20220405151619.png]]

After the app has been created, also enter the following two commands to activate the app's Python virtual environment and install its dependencies.

source .venv/Scripts/activate
python -m pip install -r requirements.txt

Tip -->
If you don't specify a template, the default is "app," which is the one we wanted anyway, so technically you can leave it out and save four keystrokes. If you have Git installed, each project you create using cdk init is also initialized as a Git repository. We'll ignore that for now, but it's there when you need it.

Here in Python Building is not required.

List the stacks in the app
Just to verify everything is working correctly, list the stacks in your app.

cdk ls

If you don't see HelloCdkStack, make sure you named your app's directory hello-cdk.

Add an Amazon S3 bucket

At this point, your app doesn't do anything useful because the stack doesn't define any resources. Let's define an Amazon S3 bucket. Install the Amazon S3 package from the AWS Construct Library.

pip install aws-cdk.aws-s3

Replace the first import statement in hello_cdk_stack.py in the hello_cdk directory with the following code.

from aws_cdk import (aws_s3 as s3,core)
#Replace the comment with the following code.
bucket = s3.Bucket(self,"MyFirstBucket",versioned=True,)

Bucket is the first construct we've seen, so let's take a closer look. Like all constructs, the Bucket class takes three parameters.

scope:
Tells the bucket that the stack is its parent: it is defined within the scope of the stack. You can define constructs inside of constructs, creating a hierarchy (tree).
Id:
The logical ID of the Bucket within your AWS CDK app. This (plus a hash based on the bucket's location within the stack) uniquely identifies the bucket across deployments so the AWS CDK can update it if you change how it's defined in your app. Buckets can also have a name, which is separate from this ID (it's the bucketName property).
props:
A bundle of values that define properties of the bucket. Here we've defined only one roperty: versioned, which enables versioning for the files in the bucket.
All constructs take these same three arguments, so it's easy to stay oriented as you learn about new ones. And as you might expect, you can subclass any construct to extend it to suit your needs, or just to change its defaults.

Tip -->
If all a construct's props are optional, you can omit the third parameter entirely.

Synthesize an AWS CloudFormation template

Synthesize an AWS CloudFormation template for the app, as follows.

cdk synth

If your app contained more than one stack, you'd need to specify which stack(s) to synthesize. But since it only contains one, the Toolkit knows you must mean that one.

Tip -->
If you received an error like "--app" is required..., it's probably because you are running the command from a subdirectory. Navigate to the main app directory and try again.

The cdk synth command executes your app, which causes the resources defined in it to be translated to an AWS CloudFormation template. The output of cdk synth is a YAML-format AWS CloudFormation template, which looks something like this.

Resources:
  MyFirstBucketB8884501:
    Type: AWS::S3::Bucket
    Properties:
      VersioningConfiguration:
        Status: Enabled
    UpdateReplacePolicy: Retain
    DeletionPolicy: Retain
    Metadata:
      aws:cdk:path: HelloCdkStack/MyFirstBucket/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/yXIQQ5AMBBA0bPYt0NZ2XIB4QBSVTHKNNFpRMTdEav/8nNQJWSJPoI0o5MrDnB1rI0T7+pDAVcVjbMs6ol+3R9bG3zcjf1cexqR0dMtmpNnT2kBJSiVLAFR7pEYNwvt3wd3SI6vcAAAAA==
    Metadata:
      aws:cdk:path: HelloCdkStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Conditions:
  CDKMetadataAvailable:

Even if you aren't very familiar with AWS CloudFormation, you should be able to find the definition for an AWS::S3::Bucket and see how the versioning configuration was translated.

Note:
Every generated template contains a AWS::CDK::Metadata resource by default. The AWS CDK team uses this metadata to gain insight into how the AWS CDK is used, so we can continue to improve it. For details, including how to opt out of version reporting, see Version reporting. The cdk synth generates a perfectly valid AWS CloudFormation template. You could take it and deploy it using the AWS CloudFormation console. But the AWS CDK Toolkit also has that feature built-in.

Deploying the stack

To deploy the stack using AWS CloudFormation, issue:

cdk deploy

As with cdk synth, you don't need to specify the name of the stack since there's only one in the app. It is optional (though good practice) to synthesize before deploying. The AWS
CDK synthesizes your stack before each deployment. If your code changes have security implications, you'll see a summary of these, and be asked to confirm them before deployment proceeds. cdk deploy displays progress information as your stack is deployed. When it's done, the command prompt reappears. You can go to the AWS CloudFormation console and see that it now lists HelloCdkStack. You'll also find
MyFirstBucket in the Amazon S3 console.
You've deployed your first stack using the AWS CDK—congratulations! But
that's not all there is to the AWS CDK.

Modifying the app

The AWS CDK can update your deployed resources after you modify your app. Let's make a little change to our bucket. We want to be able to delete the bucket automatically when we delete the stack, so we'll change the RemovalPolicy.

Update:

# hello_cdk/hello_cdk_stack.py
bucket = s3.Bucket(self,
    "MyFirstBucket",
    versioned=True,
    removal_policy=core.RemovalPolicy.DESTROY)

Now we'll use the cdk diff command to see the differences between what's already been deployed, and the code we just changed.

cdk diff

The AWS CDK Toolkit queries your AWS account for the current AWS CloudFormation template for the hello-cdk stack, and compares it with the template it synthesized from your app. The Resources section of the output should look like the following.

[~] AWS::S3::Bucket MyFirstBucket MyFirstBucketB8884501
├─ [~] DeletionPolicy
│ ├─ [-] Retain
│ └─ [+] Delete
└─ [~] UpdateReplacePolicy
├─ [-] Retain
└─ [+] Delete

As you can see, the diff indicates that the DeletionPolicy property of the bucket is now set to Delete, enabling the bucket to be deleted when its stack is deleted. The UpdateReplacePolicy is also changed.
Don't be confused by the difference in name. The AWS CDK calls it RemovalPolicy because its meaning is slightly different from AWS CloudFormation's DeletionPolicy: the AWS CDK default is to retain the bucket when the stack is deleted, while AWS CloudFormation's default is to delete it. See Removal policies for further details.
You can also see that the bucket isn't going to be replaced, but will be updated instead.
Now let's deploy.

cdk deploy

Enter y to approve the changes and deploy the updated stack. The Toolkit updates the bucket configuration as you requested.

Destroying the app's resources

Now that you're done with the quick tour, destroy your app's resources to avoid incurring any costs from the bucket you created, as follows.

cdk destroy

Enter y to approve the changes and delete any stack resources.

Note
This wouldn't have worked if we hadn't changed the bucket's RemovalPolicy
just a minute ago!
If cdk destroy fails, it probably means you put something in your Amazon S3 bucket. AWS CloudFormation won't delete buckets with files in them. Delete the files and try again.

Working with the AWS CDK in Python

Python is a fully-supported client language for the AWS CDK and is considered stable. Working with the AWS CDK in Python uses familiar tools, including the standard Python implementation (CPython), virtual environments with virtualenv, and the Python package installer pip. The modules comprising the AWS Construct Library are distributed via pypi.org. The Python version of the AWS CDK even uses Python-style identifiers (for example, snake_case method names).
You can use any editor or IDE; many AWS CDK developers use Visual Studio Code (or its open-source equivalent VSCodium), which has good support for Python via an official extension. The IDLE editor included with Python will suffice to get started. The Python modules for the AWS CDK do have type hints, which are useful for a linting tool or an IDE that supports type validation.

Prerequisites

To work with the AWS CDK, you must have an AWS account and credentials and have installed Node.js and the AWS CDK Toolkit. See AWS CDK Prerequisites.
Python AWS CDK applications require Python 3.6 or later. If you don't already have it installed, download a compatible version for your platform at python.org. If you run Linux, your system may have come with a compatible version, or you may install it using your distro's package manager (yum, apt, etc.).
Mac users may be interested in Homebrew, a Linux-style package manager for Mac OS X.
The Python package installer, pip, and virtual environment manager, virtualenv, are also required. Windows installations of compatible Python versions include these tools. On Linux, pip and virtualenv may be provided as separate packages in your package manager. Alternatively, you may install them with the following commands:

python -m ensurepip --upgrade
python -m pip install --upgrade pip
python -m pip install --upgrade virtualenv

If you encounter a permission error, run the above commands with the --user flag so that the modules are installed in your user directory, or use sudo to obtain the permissions to install the modules system-wide.

Note
It is common for Linux distros to use the executable name python3 for Python 3.x, and have python refer to a Python 2.x installation. You can adjust the command used to run your application by editing cdk.json in the project's main directory.

Creating a project

You create a new AWS CDK project by invoking cdk init in an empty directory.

mkdir my-project
cd my-project
cdk init app --language python

cdk init uses the name of the project folder to name various elements of the project, including classes, subfolders, and files. After initializing the project, activate the project's virtual environment. This allows the project's dependencies to be installed locally in the project folder, instead of globally.

source .env/bin/activate

Note:
You may recognize this as the Mac/Linux command to activate a virtual environment. The Python templates include a batch file, source.bat, that allows the same command to be used on Windows. The traditional Windows command, .env\Scripts\activate.bat, works, too. Then install the app's standard dependencies:

python -m pip install -r requirements.txt

Important
Activate the project's virtual environment whenever you start working on it. Otherwise, you won't have access to the modules installed there, and modules you install will go in the Python global module directory (or will result in a permission error).

Managing AWS construct library modules

Use the Python package installer, pip, to install and update AWS Construct Library modules for use by your apps, as well as other packages you need. pip also installs the dependencies for those modules automatically. To run pip without needing it installed in a special directory, invoke it as:

python -m pip PIP-COMMAND

AWS Construct Library modules are named like aws-cdk.SERVICE-NAME. For
example, the command below installs the modules for Amazon S3 and AWS Lambda.

python -m pip install aws-cdk.aws-s3 aws-cdk.aws-lambda

Similar names are used for importing AWS Construct Library modules into your Python code (just replace the hyphens with underscores).

import aws_cdk.aws_s3 as s3
import aws_cdk.aws_lambda as lam

After installing a module, update your project's requirements.txt file, which lists your project's dependencies. It is best to do this manually rather than using pip freeze. pip freeze captures the current versions of all modules installed in your Python virtual environment, which can be useful when bundling up a project to be run elsewhere.
Usually, though, your requirements.txt should list only top-level dependencies (modules that your app depends on directly) and not the dependencies of those modules. This strategy makes updating your dependencies simpler. Here is what your requirements.txt file might look like if you have installed the Amazon S3 and AWS Lambda modules as shown earlier.

aws-cdk.aws-s3==X.YY.ZZ
aws-cdk.aws-lambda==X.YY.ZZ

You can edit requirements.txt to allow upgrades; simply replace the == preceding a version number with ~= to allow upgrades to a higher compatible version, or remove the version requirement entirely to specify the latest available version of the module.
With requirements.txt edited appropriately to allow upgrades, issue this command to upgrade your project's installed modules at any time:

pip install --upgrade -r requirements.txt

Note
All AWS Construct Library modules used in your project must be the same version.

AWS CDK idioms in Python

Props

All AWS Construct Library classes are instantiated using three arguments: the scope in which the construct is being defined (its parent in the construct tree), a name, and props, a bundle of key/value pairs that the construct uses to configure the resources it creates. Other classes and methods also use the "bundle of attributes" pattern for arguments.

In Python, props are expressed as keyword arguments. If an argument contains nested data structures, these are expressed using a class which takes its own keyword arguments at instantiation. The same pattern is applied to other method calls that take a single structured argument.

For example, in a Amazon S3 bucket's add_lifecycle_rule method, the transitions property is a list of Transition instances.

bucket.add_lifecycle_rule(
    transitions=[
        Transition(
            storage_class=StorageClass.GLACIER,
            transition_after=Duration.days(10))
        ]
)

When extending a class or overriding a method, you may want to accept additional arguments for your own purposes that are not understood by the parent class. In this case you should accept the arguments you don't care about using the kwargs idiom, and use keyword-only arguments to accept the arguments you're interested in. When calling the parent's constructor or the overridden method, pass only the arguments it is expecting (often just kwargs). Passing arguments that the parent class or method doesn't expect results in an error.
Future releases of the AWS CDK may coincidentally add a new property with a name you used for your own property. This won't cause any technical issues for users of your construct or method (since your property isn't passed "up the chain," the parent class or overridden method will simply use a default value) but it may cause confusion. You can avoid this potential problem by naming your properties so they clearly belong to your construct. If there are many new properties, bundle them into an appropriately-named class and pass it as a single keyword argument.***

AWS: Launch an EC2 Instance from the Web Console

Deepak Porwal — Fri, 18 Feb 2022 07:52:41 +0000

You are a Systems Adminstrator at Globomantics corporation. You have been asked to investigate the IaaS (Infrastructure as a Service) offerings in Amazon Web Services. To do this, you've been asked to launch an EC2 instance so that you and your team can test out the Virtual Machine capabilities on AWS.

Click Open AWS console to access the lab environment, then use the provided credentials to log in.

Note: Ensure you are in the US West (Oregon) region in the top-right corner of your screen.
Use the search bar at the top to navigate to the EC2 service.
Click the Launch instance button and then click Launch instance.
At the top of the list of AMIs on the Choose an Amazon Machine Image start page, click the Select button for the option labeled Amazon Linux 2 AMI (HVM), SSD Volume Type. Keep the radio button for 64-bit (x86) selected.

✅ Note

This Quick Start page allows you to choose from a variety of different Amazon approved Quick Start images. Options include: Amazon Linux, Red Hat, Suse, Ubuntu, and Windows. Amazon curates this selection of Quick Start options and vendors must agree to continually update the images that appear here, to ensure they are kept up-to-date with security patches.

✅ Note

If you wanted to launch an EC2 option from an AMI (Amazon Machine Image) that does not appear in the limited Quick Start list, you could select from the other tabs on the left hand side. These options include My AMI's (for images your organization has built themselves), AWS Marketplace (which includes AMI's from many different vendors, often using a pay per minute model), and Community AMI's, where you can find images for additional operating systems, such as CentOS and Debian.

On the Choose an Instance Type page, keep the t2.micro instance type selected, then click the Next: Configure Instance Details button at the bottom right. This instance type creates a virtual machine with 1 vCPU (virtual CPU) and 1 GiB of memory.

✅ Note

Notice that the t2.micro instance type has a green label for "Free tier eligible." AWS offers a free tier that allows you to launch and run a t2.micro instance for 1 year without incurring any charges. Be aware that, if this were your own AWS account, other AWS services may cause your account to incur charges – but it is possible to run some things in AWS for a year without paying anything, thanks to the generous Free tier.

On the Configure Instance Details page, review the available options, but keep the defaults selected. Scroll down to the Advanced Details section and expand it if it is collapsed.
In the User data section, keep As text selected, and then paste the following lines into the text box:

#!/bin/bash -xe
yum install -y ruby
cd /opt
curl -O https://aws-codedeploy-us-west-2.s3.amazonaws.com/latest/install
chmod +x ./install
./install auto

Then click the Next: Add Storage button at the bottom right.

✅ Note

The above script will download and install the AWS CodeDeploy agent immediately after the EC2 instance is launched. The usage of EC2 User data scripts is the most common way to initialize or bootstrap EC2 virtual machines.

On the Add Storage page, review the EBS (Elastic Block Store) Volume attributes, keep the default 8 GiB size, and then click the Next: Add Tags button at the bottom right.
On the Add Tags page, click the Add Tag button, enter a Key of env and set its Value to test.
Add another tag with the Key Name and the Value webserver1.

✅ Note

The Name tag (case sensitive) is a special tag that AWS uses to display the user-defined instance name on the Instances web console page. Notice that these tags will be applied to both the EC2 instance and the EBS volume.

Click the Next: Configure Security Group button.
On the Configure Security Group page, keep the Create a new security group radio button selected, keep the default Security group name of launch-wizard-1, and keep the auto generated Description.

ℹ️ Note

Notice that the launch wizard has populated a single Security Group Rule with Type: SSH, Protocol: TCP, Port Range: 22, and Source: 0.0.0.0/0 (this means any IP address). This rule opens up access for the entire internet to SSH into the EC2 instance on port 22.

Click the Review and Launch button.

✅ Note

For any AWS account and EC2 instance of real importance, you should never use a Security group named after the launch-wizard, and you should not allow SSH access from the entire internet. Instead, you should give your Security Group a name that is descriptive of its purpose or function, and you should use a VPN or a Bastion host for SSH access into your EC2 instances.

On the Review page, you can review all of the options that you have selected throughout the Launch wizard process. Click the Launch button at the bottom right.
You will now see a Select an existing key pair or create a new key pair modal. Click on the Choose an existing key pair dropdown and select Proceed without a key pair, then add a checkmark to the acknowledgement box that informs you that you will not be able to connect to the instance.
Click the Launch Instances button.

✅ Note

Typically when launching an EC2 instance you would create and select an existing key pair that you've already added to your AWS account. This key pair allows you to SSH into the EC2 instance using the default user.

After a brief Initiating Instance Launches page, you'll be taken to a Launch Status page, and you should see a green box at the top of the page that says Your instances are now launching. You should also see a unique instance identifier, which looks like i-XXXXXXXXXX.

Congratulations! You have successfully used the EC2 Launch Wizard to create an EC2 instance on a t2.micro instance type, using the Amazon Linux 2 operating system, with a User data script that installs the AWS Code Deploy agent.

Verify the Instance Status

Now that your EC2 Instance has been launched, you will verify its status.

You should still be on the Launch Status page from the last challenge. Click the View Instances button on the bottom right, which will take you to the Instances page.
You should see your instance named webserver1 in the Instances list.

Under the Instance state column, your instance should say Running.
Check the box to the left of webserver1. You should now see details for the instance.
Click on the Status checks tab. You should see green checkmarks and a passed message under both System status checks and Instance status checks. This indicates that your instance is viewed as 100% healthy and is ready for use.

✅ Note

If both of these statuses are not yet green, you may need to wait a few minutes for the status checks to complete their runs, which can take some time after the initial instance launch. The "System status checks" monitor the underlying AWS systems on which your instance runs. Only AWS can resolve "System Status check" issues, although you can Stop and then Start your instance to move it to new underlying hardware. The "Instance status checks" monitor the software and network configuration of your individual instance. If there is an issue with the Instance status check, and not an issue with the System status checks, then you can often resolve the issue on your own.

Click on the Monitoring tab and you should see some activity in the graphs for CPU and Network activity. This activity confirms that your EC2 instance is using its vCPU and that it is communicating on the network.

Note: If no activity is shown on the graphs, wait a few minutes and then click the refresh button at the top right, above the graph boxes, inside the Monitoring tab.

Congratulations! You have verified the status checks on your new EC2 instance and have verified that it is using its CPU and network.

Your team can now begin experimenting with this EC2 instance and others, in order to evaluate the IaaS capabilities of AWS. Nice work!

POD Design

Deepak Porwal — Fri, 28 Jan 2022 07:19:26 +0000

Labels

Labels are nothing but the tags to the kubernetes object for the identification. Doubtfulness of the servers would lead to mistakes on stopping or terminating.
AWS TAG = Labels

Labels are key/value pairs that are attached to objects, such as pods.

eg:
name: dporwal-server
env: prod

Set Labels to the kubernetes objects

# creating pods
kubectl run nginx --image=nginx
kubectl run nginx2 --image=nginx

#give label to pods
kubectl label pods nginx env=dev
kubectl label pods nginx2 env=prod

#see labels of pods
kubectl get pods --show-labels

DEMO:

Selectors

Basically to filter-out the k8s objects is the functionality of selectors.

Suppose you only want to see the prod server

kubectl get pods -l env=prod

#similarly to dev
kubectl get pods -l env=dev

Suppose you want to get all the pods that are not prod.

kubectl get pods env!=prod

Labels In YAML file

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
  labels:
    env: prod
    app: nginx
spec:
  containers:
  - name: nginx
    image: democontainer
    ports:
      - containerPort: 8080

ReplicaSets (AWS = Auto Scalling)

A ReplicaSet purpose is to maintain a stable set of replica Pods running at any given of time.

Desired State - Number of Pods you want
Current State - Number of Pods are currently running

It always try to maintain the Desired State with the current state.

We will create YAML file to launch our first ReplicaSet.

kubectl apply -f replicaset.yaml
kubectl get pods --show-labels
kubectl delete rs dporwal-replicaset

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: dporwal-replicaset
spec:
  replicas: 5
  selector:
    matchLabels:
      tier: frontend
  template:
    metadata:
      labels:
        tier: frontend
    spec:
      containers:
        - name: php-redis
          image: gcr.io/google_samples/gb-frontend:v3

Deployments

Suppose replicaset is having 3 desired tomorrow you need 5, then every time we need to have make changes on replicaset.yaml. Here Deployments come for this solution

Benefits is Rolling out changes, which make the replicaset and deploy the latest version and then redirect from older version replicaset to latest version replicaset.

Creating first Deployment set

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dporwal-deployment
spec:
  replicas: 5
  selector:
    matchLabels:
      tier: frontend
  template:
    metadata:
      labels:
        tier: frontend
    spec:
      containers:
        - name: php-redis
          image: gcr.io/google_samples/gb-frontend:v3

Now lets suppose we made the changes and bring-up the new version.
we changed image from gcr.io/google_samples/gb-frontend:v3 to nginx

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dporwal-deployment
spec:
  replicas: 5
  selector:
    matchLabels:
      tier: frontend
  template:
    metadata:
      labels:
        tier: frontend
    spec:
      containers:
        - name: php-redis
          image: nginx

Now, when we will run this yaml file, it will bring the previous version replicaset down and will create a new replicaset.
let me show you.

Here is a strategy of rolling-out the changes.

25% max unavailable, 25% max surge

kubectl describe deployment

Here Deployment ensures that only certain number of pods are down which the changes are rolling out. By default it ensures that least 25% of desired pods are up(25% max unavailable).
Deployments keep the history of the version that are made.

To check the revisions of the deployment.

kubectl rollout history deployment.v1.apps/dporwal-deployment

Rollback

Lets rollback it to revision 1.

kubectl rollout undo deployment.v1.apps/dporwal-deployment --to-revision=1

Deployment Configuration

There are 2 main configs for the deployments.

maxSurge - Max number of pods can be scheduled above the existing pods.
maxUnavailable - Max number of pods that can be unavailable during the update.

for eg:
maxUnavailable=0 and maxSurge=20% << Full Capacity is maintained
maxUnavailable=10% and maxSurge=0 << Update with no extra capacity, In-place update.

Default:
maxSurge: 25%
maxUnavailable: 25%

You can edit the deployement config, by following command.

$ kubectl edit deployment dporwal-deployment

#making the changes to the deployment set and apply it
$ kubectl set image deployment dporwal-deployment nginx=nginx:alpine

#To scale Pods for the deployments
$ kubectl scale deployment dporwal-deployment --replicas=10

References:
Official Documentation
Udemy Course
Getting Started with Kubernetes
Kubernetes PODs

Credit:
Zeal Vora

Kubernetes PODs

Deepak Porwal — Tue, 12 Oct 2021 12:04:12 +0000

Here we will see the practical aspects of Orchestration Containers.
We will see how same task is perform by the Docker set of Commands comparing to the Kubernetes commands.

Architecture

Run Our First Container

Docker Command

docker run --name mywebserver nginx

Kubectl command

kubectl run mywebserver --image=nginx

Exec into Container

Docker Command

docker exec -it mywebserver bash

K8s Command

kubectl exec -it mywebserver -- bash
OR
kuberctl exec -it mywebserver -- ls -l /

Removing Container

Docker commands

Before removing the particular container we have to stop the container and then we can remove it

docker container stop mywebserver
docker container rm mywebserver

kubectl command

We don't need to stop the pod before removing, k8s take care of it.

kubectl delete pod mywebserver

Benefits of Pods

Many applications might have more than one container which is tightly coupled in one-to-one relationship.

Here you need to know the relation between the containers, so to keep track on all the containers having dependencies. As if one goes down, complete application goes down.

Linking in Docker commands,

docker run -dt --name myweb01 function01
docker run -dt --name myapp01 function02

Whereas, in K8s Containers within a pod share an IP address and port space, and can find each other via localhost.
Not need to create individual container, just need to refer as a pod. No worries of one-to-one connection.
If pod1 is not working it can create another pod2 and be available whenever needed.

* Pod always runs on a Node

* A node is a worker machine in k8s

* Each node is managed by master

* A node have multiple pods

Launch multi containers Pod

We need to use yaml file to mention the different containers and Objects/Volumes.

apiVersion: v1
kind: Pod
metadata:
  name: sidecar-pod-1
spec:
  volumes:
  - name: log
    emptyDir: {}

  containers:
  - image: busybox
    name: main-container
    args:
     - /bin/sh
     - -c
     - >
      while true; do
        echo "$(date) INFO hello from main-container" >> /var/log/myapp.log ;
        sleep 1;
      done
    volumeMounts:
    - name: log
      mountPath: /var/log

  - name: sidecar-container
    image: busybox
    args:
     - /bin/sh
     - -c
     - tail -fn+1 /var/log/myapp.log
    volumeMounts:
    - name: log
      mountPath: /var/log

run the above yaml file, which I have named as pod.yaml

kubectl apply -f pod.yaml

Let us understand this YAML file:

For the sake of understanding I have named my containers as main-container and sidecar-container.
The main container will be our application which will continuously write something to /var/log/myapp.log
The /var/log/ path is mounted on the containers using separate volume. This path is mounted using volumeMounts in both the containers so that the path is shared across both the containers.
The sidecar container will read the log file content using tail -fn+1 /var/log/myapp.log

Benefits of Configuration File

Integrates well will change review processes
Provides the source of record on what is live within the Kubernetes cluster.
Easier to troubleshoot changes with version control

Understanding POD Configurations in YAML

In the above pod.yaml you would have seen many Important fields like apiVersion, kind, metadata, container etc.

key	Description
apiVersion	Version of API
kind	kind of object you want to create
metadata name	name of the object that name uniquely identifies it
spec	Describe state of the object

To check apiVersion we need to access API Primitives at path localhost:8080/api through

kubectl proxy --port 8080

For more fields references you can check
API Documentation
Github Documentation

you can also take reference from CLI using below commands, but these could be not updated details.

kubectl api-resources
#specific to pod details
kubectl explain pod

Exposing Ports for Pods

Here we have the yaml file that, to demonstrate how we expose port. It work similar way as in Docker.

Reference

Lets create a pod with container at port 8080 and we will see the pod details.

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx
    image: democontainer
    ports:
      - containerPort: 8080

and checking details

to check for more details about the port field in document. you can run following command.

kubectl explain pod.spec.containers.ports

Generating Pod Manifests using CLI

# Create a pod from nginx Image
kubectl run nginx --image=nginx

#Create a pod and expose to port 8080
kubectl run nginx --image=nginx --port=8080

# Output the manifest file
kubectl run nginx --image=nginx --port=80 --dry-run=client -o yaml

detele pods

#delete specific pod
kubectl delete pod command

#delete all pods
kubectl delete pod --all

References:
Official Documentation
Udemy Course
Getting Started with Kubernetes
Docker Commands

Credit:
Zeal Vora

Getting Started with Kubernetes

Deepak Porwal — Tue, 05 Oct 2021 16:34:25 +0000

Orchestration

Container orchestration is all about managing the life cycles of containers, especially in large, dynamic environments.

Container Orchestration can be used to perform a lot of tasks, some of them includes:

Provisioning and deployment of containers
Scaling up or removing containers to spread application load evenly
Movement of containers from one host to another if there is a shortage of resources
Load balancing of service discovery between containers
Health monitoring of containers and hosts

There are many container orchestration solutions which are available, some of the popular ones include:

Docker Swarm
Kubernetes
Apache Mesos
Elastic Container Service (AWS ECS)

There are also various container orchestration platforms available like EKS.

Introduction to Kubernetes

Kubernetes (K8s) is and open-Source container Orchestration engine developed by Google.

It is originally designed by Google, and is now maintained by the Cloud Native Computing Foundation.

Installation Options for K8s

Installation Aspects

Things to configure while working with Kubernetes.

Sr No	Things to Install	Description
1	kubectl	CLI for running user commands against cluster
2	Kubernetes Master	Kubernetes Cluster by itself
3	Worker Node Agents	Kubernetes Node Agent

Understanding Installation Methods

There are multiple ways to get started with fully functional k8s environment.

Use the Managed K8s Service
Use Minikube
Install & Configure K8s Manually (Hard Way)

Install & Configure K8s Manually(Hard Way)

In this approach you have to install all the components ok K8s Individually

Components to be Configured - Managed Service

You have to just download kubectl and that's it. As rest all the things are taken care by Managed Service. Download CLI and just need to connect with K8s master.

Environment Setup

Components to be Configured - Minikube

Take care of K8s master also the worker node, then we need to configure the kubectl.
It has disadvantage that it provision single node cluster.

Kubernetes API Primitives

Depending on the Operations there are various APIs availble.

eg: /apis , /metrics ,
/api/v1/pods or /api/v1/nodes or /api/v1/services

to explore APIs, run below command and goto: localhost:8080

kubectl proxy --port 8080

Kubernetes POD

A pod is a collection of containers and its storage inside a node of a Kubernetes cluster. It is possible to create a pod with multiple containers inside it. For example, keeping a database container and data container in the same pod.

There are two types of Pods −

Single container pod
Multi container pod

Single container pod

$ kubectl run <name of pod> --image=<name of the image from registry>

eg:

kubectl run tomcat --image = tomcat:8.0

Multi container pod

Multi container pods are created using yaml mail with the definition of the containers.

apiVersion: v1
kind: Pod
metadata:
   name: Tomcat
spec:
   containers:
   - name: Tomcat
    image: tomcat: 8.0
    ports:
containerPort: 7500
   imagePullPolicy: Always
   -name: Database
   Image: mongoDB
   Ports:
containerPort: 7501
   imagePullPolicy: Always

For more deep on Pods goto: Kubernetes Pods

Kubernetes Objects

Kubernetes Objects are basically a record of intent that you pass on to the Kubernetes Cluster.
Once you create the object, the kubernetes system will constantly work to ensure that object exists.

Commands and Arguments in K8s

In Docker main difference between ENTRYPOINT and CMD is that Override the main command in Dockerfile with the command passing as arguments.

Whereas, in Kubernetes we can override both the ENTRYPOINT and CMD with command and arguments field.

Dockerfile VS K8s Menifest Perspective

Docker Field Name	K8s field Name	Description
ENTRYPOINT	command	Command that will run by the container
CMD	args	Argument passed to the container

Let me show you how command and args work on kubernetes.

Reference: Define a Command and Arguments for a Container

References:
Official Documentation
Udemy Course

Credit:
Zeal Vora

Specific Kubernetes setups, coming soon!!!!!!!!!!!!!!!!

DigitalOcean Managed Kubernetes Service

Deepak Porwal — Sat, 02 Oct 2021 18:28:07 +0000

Setting Up Kubernetes using Digital Ocean K8s Managed Service

You can use my referral you will get $100 free usage for 3 months.
By clicking below.

Downloading the Config file and Setting up

And when we open the config file we will see these details.

Setting up CLI kubectl

Kubectl need to connect to the k8s master, we nees:

DNS/IP of the cluster
Authentication Credentials

which will found in kubeconfig.

I'm setting up in windows. So, will be using below reference.

https://kubernetes.io/docs/tasks/tools/install-kubectl-windows/

Download this Kubectl binary

Next Step, Move that Binary to C:\Binaries. If Binaries folder is not there, create one.

Create a Environment Variable to call this Binary.

Setting up config to connect with Kubernetes master

Move the kubeconfig file to C:\Users\ .kube and name it as config ( removing .yml extension)

To Verify, run kubectl , it will work from any path.

For, linux and MacOS installation refer below link.
https://kubernetes.io/docs/tasks/tools/

References:
Official Documentation
Udemy Course

Credit:
Zeal Vora

Docker Networking

Deepak Porwal — Thu, 30 Sep 2021 16:14:56 +0000

Docker takes care of the networking aspects so that containers can communicate with other containers and also with the Docker Host.

Docker networking subsystem is pluggable, using drivers.

There are several drivers available by default and provides core networking functionality.

bridge
host
overlay
macvlan
none

We can check what all the network drivers are supported by Docker Containers and also at what ip that driver is configured to connect to the other containers.

docker network ls

if you want to check particular network driver is running at what IP, we can inspect that network driver

docker inspect bridge

We can check what all drivers what containers are running and what are not by simply inspecting that particular network driver.

docker inspect network host

We can also create a docker container and can launch it to the available network drivers. Like here we have will launch a container on host driver.

docker container run -dt --name myhost --network host ubuntu

Bridge Network

A bridge network uses a software bridge that allows containers connected to the same bridge network to communicate while providing isolation from containers which are not connected to that bridge network.

C1,C2, C3 are the containers in the bridge network, can also communicate with eachother

If want to see the bridge in the docker container, you can simply inspect any docker container and there you will get the bridge details.
While creating a container if you don't mention the network, by default it is bridge network
We also can create User-defined Bridge Network which are superior to the default bridge.

Let me show you how two docker containers can connect with each other.

Step 1:Created 2 docker containers bridge01 and bridge02

docker container run -dt --name bridge01 ubuntu
docker container run -dt --name bridge02 ubuntu

Step 2: Enter into the bridge01 container and installed the net-tools(ifconfig) and iputils-ping(ping) using below commands.

docker container exec -it bridge01 bash
apt-get update && apt-get install net-tools
apt-get install iputils-ping

Step3: We will check the ip address for bridge02 . So, to ping to this container.

docker inspect bridge02

Step 4: Here we ping the bridge02 container from bridge01

docker container exec -it bridge01 bash
ping 172.17.0.7

User-Defined Bridge Network

Differences between user-defined bridges and the default bridge

User-defined bridges provide automatic DNS resolution between containers.
User-defined bridges provide better isolation
Containers can be attached and detached from user-defined networks on the fly.
Each user-defined network creates a configurable bridge.
Linked containers on the default bridge network share environment variables.

Reference: https://docs.docker.com/network/bridge/

Lets create a custom user-defined bridge.

docker network create --driver bridge mybridge

If we don't define the driver, it will by default take the bridge driver.

docker network create mybridge-demo

To get the subnet and Gateway details of our custom network.

docker network inspect mybridge

Now, we can see that new network containers will be created inside the ubuntu docker container.

Now, let me show you the best feature of the custom user-defined network. which is ( User-defined bridges provide automatic DNS resolution between containers. )

Step1 : Creating 2 docker containers(mybridge01 and mybridge01) on custom network (mybridge) that we have created.

docker container run -dt --name mybridge01 --network mybridge ubuntu
docker container run -dt --name mybridge02 --network mybridge ubuntu
apt-get update && apt-get install net-tools && apt-get install iputils-ping

Step 2: Ping the mybridge02 by its name, And here we will be able to connect. This is what we call DNS resolution.

Step3: If we try to do the same with the previous containers that we created with the default bridge network(bridge01 and bridge02). We will not able to ping with the name defined.

You can also define the custom options for your network, which will give us more feasibility to customize with our use-case.

Like in user-defined network we don't have defined any options yet.

But, we can define options like we have in default bridge network.

Host Network

This driver removes the network isolation between the docker host and the docker containers to use the host’s networking directly.
For instance, if you run a container that binds to port 80 and you use host networking, the container’s application will be available on port 80 on the host’s IP address.

So, now lets see how host network removes isolation between the docker host and the docker containers.

Step1: We will create a myhostdemo1 container having host network and will install necessary packages.

docker network ls
docker container run -dt --name myhostdemo1 --network host ubuntu
docker container exec  -it myhostdemo1 bash
apt-get update && apt-get install net-tools && apt-get install iputils-ping

Step2: Now we will install and start nginx on the container bridge01, that we had already created above with the bridge network.

docker container exec  -it bridge01 bash
netstat -ntlp
apt install nginx
/etc/init.d/nginx start

Step 3: If we exit the container and try to find that nginx host address. As, it is bridge network we will not able to see it.

Step4: If we do the same with myhostdemo01. Installing and starting the nginx server, We will see that host address is reflecting outside the container as-well.

So, host network container can connect to all the docker containers internal container network drivers as-well external containers network drivers.

you can check it out by ifconfig in the host network container.

None Network

If you want to completely disable the networking stack on a container, you can use the none network.
This mode will not configure any IP for the container and doesn’t have any access to the external network as well as for other containers.
Eg. Virus affect testing

Demo:
Created one none network container and trying to ping google.com but we are not able to ping.

Publishing Exposed Ports of Container

We were discussing an approach to publishing container port to host.

docker container run -dt --name webserver -p 80:80 nginx

This is also referred to as a publish list as it publishes the only a list of the port specified.
There is also a second approach to publish all the exposed ports of the container.

docker container run -dt --name webserver -P nginx

This is also referred to as a publish all.
In this approach, all exposed ports are published to random ports of the host.

Legacy Approach for Linking Containers

Before the Docker networks feature, you could use the Docker link feature to allow containers to
discover each other and securely transfer information about one container to another container.
The --link flag is a legacy feature of Docker. It may eventually be removed. Unless you absolutely need
to continue using it, we recommend that you use user-defined networks to facilitate communication
between two containers instead of using --link

Step1: Created a container container01

Step2: Create a new container linking to the above launched container.

docker container run -dt --link container01:container --name container02 busybox sh

Step3: on login to container02 and pinging container01, we are able to ping

Step4: To check the link, you can cat the /etc/hosts file

References:
Official Docker
Udemy Course

Credit:
Zeal Vora

Prev: Image Creation, Management, and Registry(Part 2)

Image Creation, Management, and Registry(Part 2)

Deepak Porwal — Fri, 24 Sep 2021 12:20:03 +0000

Tagging Docker Images

Docker tags convey useful information about a specific image version/variant.

They are aliases to the ID of your image which often look like this: 8f5487c8b942

Assigning tag while building image

docker build -t demo:v1 .

Assigning tag if no tag default

Lets build a image without tag.

docker build .

Now lets assign tag to the existing image without tag.

docker tag adc07a47930e demo:v2

tag for existing tag of the image

This will create another image with the same IMAGE ID.

docker tag demo:v2 demo2:v3

Docker Commit

Whenever you make changes inside the container, it can be useful to commit a container’s file changes or settings into a new image.

By default, the container being committed and its processes will be paused while the image is committed.

Syntax:

docker container commit CONTAINER-ID myimage01

Created a container containing context01.txt file in root directory.
Then committed the container to the images and then creating another container from the same image, where we can see the same file/changes made present.

We can also define the Commands while committing the images from the containers.

The --change option will apply Dockerfile instructions to the image that is created.

Supported Dockerfile instructions:

CMD | ENTRYPOINT | ENV | EXPOSE
LABEL | ONBUILD | USER | VOLUME | WORKDIR

command

docker container commit --change='CMD ["ash"]' modified-container

Docker Image Layers

A Docker image is built up from a series of layers.
Each layer represents an instruction in the image’s Dockerfile.

Here is the best resource I found for the presentation.

As, the container have the R/W layer a top layer which is connected with the base image.
OR we can say that one base image is connected to different containers by Writable layer.

Lets, see some scenarios. Will build a docker image using dockerfile and will check the layers.

Dockerfile:

FROM ubuntu
RUN dd if=/dev/zero of=/root/file1.txt bs=1M count=100
RUN dd if=/dev/zero of=/root/file2.txt bs=1M count=100

Command to check the layers of the image, where layerdemo01 is the image name.

docker image history layerdemo1

Below Screenshot clearly shows how 2 layers are of same volume, as they are not adding the volume of previous layer. Every layer has its separate size according to the command or the changes.

Now, lets try to remove these files and I found that 2 layers were created with 0B but the image still contains the same 282MB.

So, Basically here we have 5 layers.
Layer1 -> FROM Ubuntu
Layer2 -> RUN dd if=/dev/zero of=/root/file1.txt bs=1M count=100
Layer3 -> RUN dd if=/dev/zero of=/root/file2.txt bs=1M count=100
Layer4 -> RUN rm -f /root/file1.txt
Layer5 -> RUN rm -f /root/file2.txt

Here Only Layer4 and Layer5 is dealing with the removing of those files, but the file are still there in Layer3 and Layer2.
That is the reason image still have the same volume.

Dockerfile:

FROM ubuntu
RUN dd if=/dev/zero of=/root/file1.txt bs=1M count=100
RUN dd if=/dev/zero of=/root/file2.txt bs=1M count=100
RUN rm -f /root/file1.txt
RUN rm -f /root/file2.txt

For the requirement where we need to remove the files after creation, we can use && to run both the command at one-go.
This will conserve the volume of the image

FROM ubuntu
RUN dd if=/dev/zero of=/root/file1.txt bs=1M count=100 && rm -f /root/file1.txt
RUN dd if=/dev/zero of=/root/file2.txt bs=1M count=100 && rm -f /root/file2.txt

Managing Images using CLI

So basically the best practice of using the command for images is

docker image <command>

docker pull ubuntu == docker image pull ubuntu
docker images == docker image ls
docker image build
docker image history
docker image import
docker image inspect
docker image load
docker image prune
docker image push

Inspecting Docker Images

A Docker Image contains lots of information, some of these include:

Creation Date
Command
Environment Variables
Architecture
OS
Size

docker image inspect command allows us to see all the information associated with a docker image.

Suppose we need to get the particular field from the inspect data.
e.g Hostname

we can use

docker image inspect ubuntu | grep 'Hostname'
docker image inspect ubuntu --format='{{.Id}}'

There are certain things that have parent child details. Like, ContainerConfig have Hostname, Domainname etc.
But, this will only gives the values not the key.

docker image inspect ubuntu --format='{{.ContainerConfig}}'

If you want the key and value both.

docker image inspect ubuntu --format='{{json .ContainerConfig}}'

If you just want the hostname value you can use below caommand to filter out the information from the inspect data.

docker image inspect ubuntu --format='{{.ContainerConfig.Hostname}}'

Docker Image prune

Docker image prune command allows us to clean up unused images.
By default, the below command will only clean up dangling images.

Dangling Images = Image without Tags and Image not referenced by any container

To prune all the images that has no container refrenced, we can use below commands.

docker image prune -a

If you want to remove all the images only which don't have tag associated, you can use below commands.

docker image prune

Before Prune we had these many images.

After running prune command.

Those images got removed which were not referenced to the container.

Here is the below command the image without the tag ( ) tag which is a Dangling image, but it can't be prune because it has containers associated.

Flattening Docker Images

Modifying Image in a single Layer or specific Layer.

As we know ubuntu has many layers. So, to merge all layers to the single layer. There is one approach that to,
Import and Export to a container

Commands:

docker export myubuntu > myubuntudemo.tar
cat myubuntudemo.tar | docker import - myubuntu:latest

Building Docker Registry

A Registry a stateless, highly scalable server-side application that stores and lets you distribute Docker images.

Docker Hub is the simplest example that all of us must have used.

There are various types of registry available, which includes:

Docker Registry
Docker Trusted Registry
Private Repository (AWS ECR)
Docker Hub

To push the image to a central registry like DockerHub, there are three steps:

1. Authenticate your Docker client to the Docker Registry Refrence for setting up Docker Registry https://hub.docker.com/_/registry/

docker run -d -p 5000:5000 --restart always --name registry registry:2

2. Tag Docker Image with Registry Repository and optional image tag.

docker tag myubuntu:latest localhost:5000/myubuntu

3. Push Image using docker push command To push the docker image to the AWS ECR. Refrence: https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html

 docker push localhost:5000/myubuntu

Now, lets pull the image from the registry. For that first we need to untag the registy located image and then delete the image.

docker pull localhost:5000/myubuntu

Pushing Docker Image to Docker Hub

I have my account on Docker Hub and created had 1 repository.

So, first we will login to the docker hub by CLI and then create the tag and push it. Then before pulling the image I have removed the tag to the image and removed all the containers associated to that container and then pull the image from the Docker Hub Repository.

docker login
docker tag busybox deepakporwal95/mydemo:v1
docker push deepakporwal95/mydemo:v1
docker pull deepakporwal95/mydemo:v1

Pushing out custom image to docker hub.

Pulling the image from Docker Hub.

Searcing and Filtering Images from Docker Hub

Description	Command
Search for Busybox image	docker search busybox
Search for Busybox image with Max Result of 5	docker search busybox --limit 5
Filter only official images	docker search --filter is-official=true nginx

On searching nginx images.

We will limit the number of results.

On searching images from Docker Hub we get many results. We can filter those results by three filter supporters.

stars
is-automated
is-official This will bring to us specific required result only.

Moving Images Across Hosts

Suppose we want to send docker image to other hosts or instances from admin box or master server. In this case we save the image as a zip and then transfer that image to host and at last will load that image in the host.

Cache in Docker

While building a container or image it uses the cache of each layer which has been already been there.

Here is the Dockerfile and requirements.txt details that we will be using for this usecase.

Dockerfile

FROM python:3.7-slim-buster
COPY . .
RUN pip install --quiet -r requirements.txt
ENTRYPOINT ["python", "server.py"]

requirements.txt

certifi==2018.8.24
chardet==3.0.4
Click==7.0
cycler==0.10.0
decorator==4.3.0
defusedxml==0.5.0

Here are the commands that will be using to build the images.

docker build -t without-cache .
docker build -t with-cache .

without cache

with cache

References:
Official Docker
Udemy Course

Credit:
Zeal Vora

Prev: Image Creation, Management, and Registry(Part 1)

Next: Docker Networking

Image Creation, Management, and Registry(Part 1)

Deepak Porwal — Sun, 12 Sep 2021 12:35:35 +0000

Working with Docker Images

Every Docker container is based on an image.

Till now we have been using images that were created by others and available in Docker Hub.

Docker can build images automatically by reading the instructions from a Dockerfile

A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image.

Overview of Dockerfile

Basic format of Dockerfile:

FROM busybox
COPY cat.txt
ADD add.txt
ADD compress.tar.gz /tmp
CMD ["sh"]

A Dockerfile must start with a FROM instruction.
The FROM instruction specifies the Base Image from which you are building.

There are multiple INSTRUCTIONS that are available in Dockerfile, some of these include:

FROM
RUN
CMD
LABEL
EXPOSE
ENV
ADD
COPY
ENTRYPOINT
VOLUME
USER

Many more....

COPY vs ADD Instruction

COPY and ADD are both Dockerfile instructions that serve similar purposes.

They let you copy files from a specific location into a Docker image.

Difference between COPY and ADD

COPY takes in an src and destination. It only lets you copy in a local file or directory from your host

ADD lets you do that too, but it also supports 2 other sources.

First, you can use a URL instead of a local file/directory. Secondly, you can extract a tar file from the source directly into the destination.

Use WGET/CURL wherever possible

Using ADD to fetch packages from remote URLs is strongly discouraged; you should use curl or wget instead.

This will execute all the lines repeatedly, will execute everytime same file. So, its not a good practice

ADD http://example.com/big.tar.xz /usr/src/things/
RUN tar -xJf /usr/src/things/big.tar.xz -C /usr/src/things
RUN make -C /usr/src/things all

This will not create unnecessary files and will make a sequence, which will deal with only file file and then process.

RUN mkdir -p /usr/src/things \
    && curl -SL http://example.com/big.tar.xz \
    | tar -xJC /usr/src/things \
    && make -C /usr/src/things all

Lets have a demo to show you how ADD work.

Create a Compressed file
Create a Dockerfile and define the file to copy from to destination
Run that Dockerfile
Verify the Output on the Destination directory.

Step 1. Create Compressed file

root@dporwal-docker:~# touch compress.txt
root@dporwal-docker:~# tar -czvf compress.tar.gz compress.txt 
root@dporwal-docker:~# docker build -t demobusybox .

Step 2. Create a Dockerfile and define the file to copy from to destination

vi Dockerfile

dockerfile1

Step 3. Run that Dockerfile

$ docker build .

Step 4. Verify the Output on the Destination directory.

Start the container that is build by DOcerfile

EXPOSE Instruction

The EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime and it does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published.

In above dockerfile1 you can see the scenario where person will not be able to get the port it should intends to publish because of not using EXPOSE command

So, now we will use this EXPOSE command Dockerfile.

Here I'm creating a dockerfile, which will launch nginx server on port 9080.

FROM ubuntu:latest
LABEL version="0.0.1"
LABEL maintainer="porwal.deepak2000@gmail.com"

RUN apt-get update && apt-get upgrade -y

RUN apt-get install nginx -y 

EXPOSE 9080

CMD ["nginx", "-g", "daemon off;"]

This is my initial environment of my docker server.

Now we will run below commands to build the custom image exposedemo with tag 0.0.1

docker image build --tag exposedemo:0.0.1 .

And here we will create a container from the exposedemo image we build.

docker container run -d exposedemo:0.0.1

And in this image you can see in yellow box the container is running on port 9080

to Verify you can also use this command.

docker inspect exposedemo:0.0.1

HEALTHCHECK Instruction

HEALTHCHECK instruction Docker allows us to tell the platform on how to test that our application is healthy.

When Docker starts a container, it monitors the process that the container runs. If the process ends, the container exits.

That's just a basic check and does not necessarily tell the detail about the application.

We can specify certain options before the CMD operation, these includes:

HEALTHCHECK --interval=5s CMD ping -c 1 172.17.0.2

--interval=DURATION (default: 30s)
--timeout=DURATION (default: 30s)
--start-period=DURATION (default: 0s)
--retries=N (default: 3)

Lets create a container demohealthcheck which we will be using to check the connection, and will create another container monitor where we will implement HEALTCHECK command by checking connection with demohealthcheck container.

Commands:

docker container run -dt --name demohealthcheck busybox sh

We will get the IP Address of this container to connect.

docker inspect demohealthcheck

Now lets create dockerfile and implement HEALTHCHECK command

FROM demohealthcheck
HEALTHCHECK --interval=5s CMD ping -c 1 172.17.0.3

Building image and container from the above docker file and check the connection.

We can also check the HEALTHCHECK Logs by

docker inspect monitor

if ExitCode: 1 then, there is something wrong with container.

Exit Code	Description
0: Success	the container is healthy and ready to use
1: Failure	the container is not working correctly
2: Reserved	do not use the exit code

We can also use HealthCheck commands while building the container.
like below we are using --health-cmd

command:

docker run -dt --name tmp --health-cmd "curl -f http://localhost" busybox sh

and we will see that container is unhealthy.On doing docker inspect tmp we found that curl command is not found.

Now we will remove this container and will try to check the container after every 5 sec and it will retry 1 time on failure.

you can try below command and let me know in discussion if you are good to do so.

docker run -dt --name tmp --health-cmd "curl -f http://localhost" --health-interval=5s --health-retries=1 busybox sh

for more commands you can check:
https://docs.docker.com/engine/reference/run/#healthcheck

Dockerfile ENTRYPOINT

The best use for ENTRYPOINT is to set the image’s main command
ENTRYPOINT doesn’t allow you to override the command.
It is important to understand the distinction between CMD and ENTRYPOINT.

As, CMD command in Dockerfile can be overridden by runtime command. Whereas ENTRYPOINT command will append to the given command given at runtime.

Sample Code Snippet:

FROM ubuntu
ENTRYPOINT ["top", "-b"]
CMD ["-c"]

WORKDIR Instruction

The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile

FROM busybox
RUN mkdir /root/demo
WORKDIR /root/demo
RUN touch dporwal.txt
CMD ['/bin/sh']

The WORKDIR instruction can be used multiple times in a Dockerfile

Sample Snippet:

WORKDIR /a
WORKDIR b
WORKDIR c
RUN pwd

Output = /a/b/c

ENV Instruction

Passing as argument during run time.
The ENV instruction sets the environment variable to the value .

You can use -e, --env, and --env-file flags to set simple environment variables in the container you’re running or overwrite variables that are defined in the Dockerfile of the image you’re running.

Example Snippet:

docker run --env VAR1=value1 --env VAR2=value2 ubuntu env | grep VAR

Eg.
docker run -dt --name env01 --env USER=USERADMIN busybox sh

implementation of ENV in dockerfile

Example Snippet:

FROM busybox
ENV NGINX 1.2
RUN touch web-$NGINX.txt
CMD ["/bin/sh"]

References:
Official Docker
Udemy Course

Credit:
Zeal Vora

Prev: Getting Started with Docker

Next: Image Creation, Management, and Registry(Part 2)

Getting Started With Docker

Deepak Porwal — Mon, 06 Sep 2021 12:28:37 +0000

Introduction to Docker

Docker is a technology designed to make it easier to create, deploy, and run applications by using containers. Docker is an open platform, once we build a docker container, we can run it anywhere, say it windows, Linux, mac whether on a laptop, data center, or in the cloud. It follows the build once, run anywhere approach.

Installing Docker

I will not be showing you the steps to install docker here.
You can refer this.

https://dev.to/whattosay/docker-environment-setup-4hac

Docker works on a wide variety of operating systems, this includes:

Windows
Linux
MAC

The installation of Docker is pretty straight forward in each one of them.
https://docs.docker.com/get-docker/

Docker Containers vs Virtual Machines

Virtual Machine contains the entire Operating System.
The container uses the resource of the host operating system

Docker [Image vs Containers]

Docker Image is a file that contains all the necessary dependency and configurations which are required to run an application.

Docker Containers is basically a running instance of an image.

Docker Image Creation and Identification

When you create a Docker container, it is assigned a universally unique identifier (UUID).

These can help identify the docker container among others.
To help humans, Docker also allows us to supply container names.
By default, if we do not specify the name, docker supplies a randomly-generated name from two words, joined by an underscore
By adding --name=meaningful_name argument during the docker run command, we can specify our own name to the containers

Creating nginx Docker image and getting IPConfig

$ docker pull nginx
$ docker images
$ docker run 
$ docker run -dt -p 80:80 nginx
$ docker ps
$ apt install net-tools
$ ifconfig

Here is a Snapshot for your reference, Do click on image to see it properly.

To check the nginx server
Goto: http://165.22.212.229:80

Here 165.22.212.229 is droplet Public IP and 80 is the port on which image has been mapped(docker run -dt -p 80:80 nginx)
By default Docker containers can make connections to the outside world, but the outside world cannot connect to containers.
If we want containers to accept incoming connections from the world, you will have to bind it to a host port.

Port Binding

$ docker inspect mynginx

This command shows the config details of docker container
In below Screenshot first I deleted the container by

$ docker rm 6e80b1835da72da248ae2e2d2b35045c3cd736196159a030f76bd31fb9f16f33

then did the porting from 8000 --> 80

$ docker run -d -p 8000:80 --name mynginx nginx

Attach and Detached Mode

When we start a docker container, we need to decide if we want to run in a default foreground mode or the detached mode.
You may want to use this if you want a container to run but do not want to view and follow all its output.

Detached Mode -d

$ docker run -d --name detached -p 8081:80 nginx

Removing Docker Containers

Docker containers can be removed with the help of docker container rm command.

Description	Command
Remove single container	docker container rm CONTAINER
Stop all the containers	docker container stop $(docker container ls -aq)
Remove all the containers	docker container rm $(docker container ls -aq)

Troubleshooting Issue

While I was removing Image I found that images were running. SO, wasn't able to remove.
Therefore, you can't remove the image until you stop.

For more commands related to Docker
https://docs.docker.com/engine/reference/commandline/docker/

The cli commands were then refactored to have the form docker COMMAND SUBCOMMAND, wherein this case the COMMAND is container and the SUBCOMMAND is run

Older Approach: docker run
Newer Approach: docker container run

Both of these approaches will work perfectly.

Docker container exec

The docker container exec command runs a new command in a running container.

The command started using docker exec only runs while the container’s primary process (PID 1) is running, and it is not restarted if the container is restarted.

# here are the commands to run the commands inside container from outside 

$ docker container exec -it dporwalexec netstat -ntlp

#you can login to the container and run the commands

root@dporwal-docker:~# docker container exec -it dporwalexec bash

#here you are inside the container with the bash Interactive
root@c0efbecf71a2:/# /etc/init.d/nginx

Here is the snapshot of above commands.

Importance of the IT flag

-it
Every process that we create in the linux environment , has three open files descriptions, stdin, stdout, stderr

-i --> interactive, keeps stdin open even if not attached

-t --> tty, flag allocated a paseudo-TTY

Default Container Command

Whenever we run a container, a default command executes which typically runs as PID 1.

We can override the default container command by manually specifying the command.

$ docker container run -d nginx sleep 500

In the above command, the sleep 500 will run as a PID 1 process overriding any default command that would be present in the nginx container.

Restart Docker Policies

By default, Docker containers will not start when they exit or when docker daemon is restarted.

Docker provides restart policies to control whether your containers start automatically when they exit, or when Docker restarts.

We can specify the restart policy by using the --restart flag with docker run command

Flag	Description
no	do not automatically restart the container(the default)
on-failure	Restart the container if it exits due to any error, which manifest as a non-zero exit code
unless-stopped	Restart the container unless it is explicitly stopped or Docker itself is stopped or restarted
always	Always restart if its stopped

$ docker container run -d --restart unless-stopped nginx

Here I have created on nginx container with unless-stopped restart command.
After restarting Docker service the container should also get start, as it will run restart command.

Disk Usage montior docker containers

The docker system df command displays information regarding the amount of disk space used by the docker daemon.

Commands:

$ docker system df
$ docker system df -v

In the below Snapshot I'm creating 500MB file using command.
I'm not sure about the below mechanism, Do let me know in discussion section if you know anything about this, how it works.

dd if=/dev/zero of=bigfile.txt bs=1M count=500

Remove Docker Images

To remove one image at a time, we will use below command.

docker rmi nginx

To remove all the docker images at once, we will use first to list all the images and then run all the commands

docker images -a -q
docker rmi $(docker images -a -q)

References:
Official Docker
Udemy Course

Credit:
Zeal Vora

Prev: Setting Docker Environment

Next: Image Creation, Management, and Registry(Part 1)

Docker Environment Setup

Deepak Porwal — Sun, 05 Sep 2021 21:12:59 +0000

* Create DigitalOcean Account

You can use my referral you will get $100 free usage for 3 months.
By clicking below.

* Create a Droplet

* Use the following Configuration for Droplet.

You have to open image and zoom

In case of ssh key generation

Click on New SSH Key as shown in above image.

And Run these commands in Local System.

$ ssh-keygen
$ cat /c/Users/<Your User>/.ssh/docker_rsa

* Droplet after Creating

In you Project, you will be able to see one droplet.
Where you will get the public IP of that droplet and we will access that using that.

* Login and update Droplet

$ ssh -i docker_rsa root@165.22.212.229
$ sudo apt-get update

After updating kernal, installing docker to the droplet.
Below is specific to Ubuntu

$ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Then we need to install Docker.

$ echo \
"deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null 

$ sudo apt-get update && sudo apt-get install docker-ce docker-ce-cli containerd.io

Verify Docker

To Verify Docker is running.

$ systemctl status docker

Hurray!! you are inside your first Droplet....