DEV Community: mohammed afif ahmed

Everything you need to know about linux File System directories!

mohammed afif ahmed — Tue, 15 Jun 2021 03:05:53 +0000

History of Linux and Windows

Okay, before starting with the actual topic, let’s discuss what makes Linux different from windows. If you are a Windows user you might have seen different drives such as C, D, E, etc which are absent in Linux instead it has folders named /bin, /sbin, /usr, /etc.

For new Linux users, let us tell you about how Linux and windows evolved. Windows was installed on top of DOS(Disk operating system), which was a command-line tool where you can run programs, games, etc. It used letters to assign removable discs such as floppy drives, i.e. A and B. When the hard drive was introduced, the letter C was assigned for the internal disk and the next available letter for the next disk.

Microsoft evolved their kernel so that windows boots are less dependent on DOS and eventually without DOS at all. Linux follows the Unix tradition which is why it uses forward slash unlike backslash in windows. It is also case sensitive, also macOS has similar features as it is a common Unix-based ancestor.

So now let’s jump into the Linux filesystem.

A filesystem is used to control the flow of how data is stored or distributed or retrieved from a physical storage device such as HDD, SSD, etc. Its main purpose is to make users and the operating system store files in such a manner that all the different directories can utilize them efficiently.

The OS plays the role of intermediate that facilitates the data transfer for storage on a storage device. Linux uses a directory tree to manage directories and files. The tree information is also stored on a storage device and this part is called the root file system or root directory.

The root directory is the most important as all other directories are derived from it and it is responsible for booting, repairing, and restoring the Linux system.

The File hierarchy for Linux defines the Linux directory structure. We can refer to the sequential directories in those directories by using directory names connected by a forward slash (/) such as /var/log and /var/spool/mail. These are called paths.

Now let us explore these sub-directories of the root directory one by one.

/bin

This is a subdirectory of the root which stands for binaries and contains the executable programs, which are used for minimal functionality for booting or repairing. It usually contains the shell commands like cp(copy), rm(remove), ls, etc. It also contains programs that boot scripts may depend on. Bin folders can also be found in other sections of the file system tree.

/sbin

This stands for system binaries that a system administrator may use and a standard user would not have access without permission. This folder, along with the one above it,contains the files that must be accessible while running in single-user mode (this mode boots you in as a root user to enable you to perform device repairs,updates, and testing) rather than multi-user mode.

/boot

This directory contains all the executables or programs responsible for booting up a Linux machine whether it is ubuntu or kali or mint. As a result, the data used until even the Linux kernel starts to run some program is stored in the /boot directory.

/dev

This directory houses some exclusive or device-specific files. Everything in Linux is categorized as a file or a directory, as this command emphasizes.



$ ls -la /dev

We can see all the partitions on the system. /dev/cdrom represents our CD-ROM. Here the nested files represent a hardware device and any changes to them will be reflected in our hardware. For example, /dev/dsp represents the speakers and if you make changes to it, it will be reflected in the speakers. For instance, it will make a sound if you cat something on it.



$ cat /boot/vminux > /dev/dsp

/etc

This directory contains all your configurations. Here configuration refers to those which are system-wide and not just for a particular user such as /etc/apt contains the sources list which contains the repo’s the system connects to and its various settings.

/lib, /lib32, /lib64

These are the directories where the libraries are stored. Libraries are files that the application uses to perform various functions, and these are required by the binaries in the /bin directory.

/media and /mnt (mount)

These are the directories where we can find our other mounted drives such as USB,floppy disks, or external hard drives, etc. The /media folder wasn’t there before.
It was just /mnt, but nowadays most Linux distros automatically mount devices in /media directory. But why two directories for mounting? Well, When manually mounting files, we use the /mnt directory and leave the /media directory to the operating system.

/opt (this is the optional folder)

This folder usually contains manually installed software from any third-party vendors. This is the place where you can install the software created by you.

/proc

This directory contains all the pseudo files that have information about the system processes and resources. Every process has a directory (named after the process ID) that contains all the necessary information on that process and these are not saved on the hard drive. The files here are translated by the kernel to generate some other information. for example



$ cat /proc/cpuinfo

This command prints out the information about the CPU.

/root

This is the home directory of the root user. It does not contain typical directories and does not reside in the /home directory, unlike a user's /home directory. You can store the files here but you need to have root access. This directory's location also means that the root has constant access to its /home folder.

/run

This directory is relatively new, and various distributions use it in different ways. It's a tempfs file system, which means everything in it is deleted when the system is rebooted or shut down. It is used to store runtime information that processes use to work early in
the boot method.

/srv

This is called a service directory where data from the service is saved. For you, it will most likely be null, but if you run a web server or an FTP server, you will store files here that will be accessed by other users. Since it is at the root of the drive, it provides better
protection.

/sys

This is called a system directory, It's been around for quite some time. It's a method of communicating with the kernel. This directory is similar to the /run directory in that it is not written to the disc physically. It's generated every time the machine starts up, so you wouldn't save anything here, and nothing is mounted here.

/tmp

It is a temp (temporary) directory. This is where the program temporarily stores files that could be used during a session. For example, when you write a document in a program in vscode, it will save a temporary copy of what you're writing here on a regular basis. If the program crashes, you can search here to see if you have a recently saved copy that you can restore.

When you reboot your computer, this folder is normally empty. Any files or directories may still be present or may have been stuck because the machine was unable to remove them. This isn't a concern unless there are hundreds of files taking up disc space, in which case you'll need to log in as root and manually delete them.

/usr

In contrast to the /bin directory, which is used by the system and system administrator to perform maintenance, this is the user application space where programs that are used by the user will be installed. Any program installed here is considered non-essential for basic system service, and it is also known as Unix System Resource.

The installed programme can be found in a variety of locations, including /usr/bin, /usr/sbin, and /usr/local. The local directory is where most programs installed from source code end up. The /usr/share is where many larger programs mount themselves.

/var

This is the directory for variables. It contains files and directories that are expected to grow in size over time, such as dynamic data. /var/log contains system and application log files, which will grow in size as you use the system. Other items included here include mail databases and temporary storage for printer queues, also known as /spool.

/home

Each user has this directory. The /home directory is where you store your file and documents. Each user can access only their folder unless admin permissions.

Conclusion

And, phew...! finally we have reach the end. In this article, we have covered all the different Linux File system directories. We have also seen what makes Linux FS different from Windows FS and how Linux evolved. I certainly hope that you now have knowledge of what each directory in Linux FS means and its true functionality as well as its role in OS.

liked the post?

Introduction to code-splitting in reactjs.

mohammed afif ahmed — Tue, 25 May 2021 11:00:46 +0000

Introduction

Imagine you have an application with many different components and pages with routes for that pages. So when you run your application it takes a large amount of time time to load or display content. So what the problem and how can it be solved.

That's when code-splitting comes in, it makes sure only those components are fetched which are being displayed on the webpage at that particular moment. For example, if you have a Homepage component and an AboutUs component. Homepage component is displayed when you're on root route i.e., / and AboutUs at /about, When you're on the home route you don't need AboutUs javascript right? but it is fetched on the initial load, which makes the site load time-consuming, and eventually leads to losing viewers.

We will look at an example site and how to perform code-splitting with just a few lines of code.

Let's get started:

fork or clone (might give a star as well 😀) this repo from GitHub, it is a single page application built using react. You can apply code spitting anywhere in the components. For example, where you import any third-party library. But an easier place to identify is at the route level, where you write the routing rules.
In the repo, you clone navigate to/src/MainComponent.js to see all the routes.

We have a route called /pricing which renders the PricingPage.js component, we will split this particular component.

But, before applying code-splitting let's see what the browser fetches or tries to load.

Start the app To start the app run the below commands(assuming you have some react knowledge)

$ npm install
$ npm start

The app must be live at http://localhost:3000

In Chrome dev tools open the network tab and select js as a filter, you can see that on the initial page load the browser fetches bundel.js.
This is where react takes all the javascript written in the application and it into this file(it contains all the components).
The index page contains all the js.
As a result, the page load is slow. We are going to exclude some components from going into that bundle and instead fetch only when needed, here when someone navigates to /pricing.

The latest version of React uses a combination of two things to accomplish this: Suspense and React.lazy.

Replace the code in MainComponent.js with the below code:

import React, { Component, Suspense, lazy } from 'react';
import { BrowserRouter, Route } from 'react-router-dom';
import HomePage from './HomePage';
import ProductDetailsPage from './ProductDetailsPage';
import HowItWorks from './HowItWorks';

const PricingPage = lazy(() => import('./PricingPage'));

class MainComponent extends Component {
    render() {
        return (
            <Suspense fallback={<h1>Loading.....</h1>}>
                <BrowserRouter>
                    <Route exact path='/' component={HomePage} />
                    <Route
                        exact
                        path='/products/:productId'
                        component={ProductDetailsPage}
                    />
                    <PricingPage exact path='/pricing' />
                    <Route exact path='/working' component={HowItWorks} />
                </BrowserRouter>
            </Suspense>
        );
    }
}

export default MainComponent;

Now go back to the network tab and check the source, you can see one more 1.chunk.js file when you navigate to /pricing
which contains only the PricingPage component.

Also when you do npm run build it builds the components separately unlike bundling all together if not using code splitting. Below are the logs of build logs before and after applying code-spitting.

File sizes after gzip:

  76.03 KB  build/static/js/2.d23bfa23.chunk.js
  28.43 KB  build/static/js/main.b229bef3.chunk.js
  770 B     build/static/js/runtime-main.e43a4c19.js
  306 B     build/static/css/main.0fc1fc64.chunk.css

File sizes after gzip:

  76.03 KB (-2 B)    build/static/js/2.8bab3079.chunk.js
  28.07 KB (-368 B)  build/static/js/main.b6b9360c.chunk.js
  1.18 KB            build/static/js/3.58e0fecc.chunk.js
  1.16 KB (+418 B)   build/static/js/runtime-main.01e4ec24.js
  306 B              build/static/css/main.0fc1fc64.chunk.css

As you can see there is one extra file that is our js for PricingPage component, and also you can see the reduction in the size of other files because the pricing component is excluded from that.

And...that's a wrap, I hope you have learned how to go about splitting a react app, now you can apply the same approach to your application as well.
We looked at code-splitting with react-router a create-react-app template that uses webpack under the hood, but you can apply the same with parcel, babel, etc.

Liked the post?

Using multiple versions of nodejs.

mohammed afif ahmed — Mon, 24 May 2021 18:35:35 +0000

Introduction

Usually we work on different version for our nodejs project and its hard to manage them,but fortunately there is tool called NVM(node verion manager) which help to manage your node version and switch between them according to your projects.

Windows installation

Unfortunately nvm project is only for linux/macos, but...but there is another very similar project by Corey bulter,known as nvm-windows. Click this link download the nvm-setup.zip file and install it in a mundane fashion as on windows.
After installation you can use the same commands as on linux/macos.

Linux installation

In your terminal use curl to install

$ curl -o- https://raw.githubusercontent.com/nvm-sh/v0.34.0/install.sh |

after installation you need to add a bit of configuration in your .bashrc file or .zshrc etc. So, open the file and append the below code.

$ export NVM_DIR="${XDG_CONFIG_HOME/:-$HOME/.}nvm"
[ -s :$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

This sets the path to directory of installation.
Reload your blog terminal, for the changes to take effect and we are good to go.

usage

So let's jump into the terminal and look at some of the commands.

To Install latest version of node

$ nvm install node

Install specific version

$ nvm install {node_verion}

#example
$ nvm install 10.0.0

command to list out all the versions installed

$ nvm ls

Switching between different node versions

# to use latest version
$ nvm use node  

# for a specific verion
$ nvm use 10.0.0

Deleting node versions

$ nvm uninstall {node_version}

Conclusion

As this was an introductory post, we looked at some of the most used commands, this is a very useful tool if you are working on multiple projects which requires different version of node. You can have a look at the official nvm https://github.com/nvm-sh/nvm gitrepo to understand thoroughly.

Liked the content?

Everything you need to know about Docker Swarm

mohammed afif ahmed — Sun, 23 May 2021 05:16:10 +0000

Introduction

Docker is a free and open framework for building, delivering, and running apps. Docker allows you to decouple your code from your hardware, allowing you to easily deliver apps. You will handle the infrastructure the same way you manage your apps with Docker. One of the characteristics you have with docker is that you can run multiple docker environments in the same host environment.
However, docker may be able to maintain only a certain number of containers, because it runs on a single node. But what if someone wants to work with or develop thousands of containers.

This is where Docker swarm comes into the picture. A docker swarm is the virtualization of the large number of nodes(running Docker engine) running in a cluster. These nodes can communicate with each other, helping developers to maintain multiple nodes in a single environment.

Docker swarm

A swarm mode consists of Multiple Docker hosts which serve as managers (to handle membership and delegation) and workers which run swarm services. A Docker host may be a manager, a worker, or both at the same time. You determine the optimum state of service when you develop it (number of replicas, network and storage resources available to it, ports the service exposes to the outside world, and more). Docker strives to keep the optimal state. Docker schedules a worker node's activities on other nodes if that node becomes inaccessible. A job, as opposed to a standalone container, is a running container that is part of a swarm service and operated by a swarm manager.

When you're in swarm mode, you can change the configuration of a service, including the networks and volumes it's connected to, without having to restart it manually. Docker will upgrade the configuration, avoid any service tasks that have out-of-date configurations, and start new ones that fit the desired configuration.
So, the difference between swarm mode and standalone docker containers is that, when in swarm mode only managers can manage containers or swarm, unlike standalone containers which can be started on any daemon. But, the daemon can participate in swarm mode as a manager or worker, or both.

Docker swarm architecture

Previously, we have used terms such as manager, worker, nodes, etc, Now let us try to understand what it means and how the docker swarm works.

Node

A node is a Docker engine instance that is part of the swarm. This can also be thought of as a Docker server. One or more nodes may operate on a single physical device or cloud server. But in development, these swarm cluster nodes can be spread over several machines on the cloud.
There are two types of nodes, a manager node, and a worker node.

Manager
In the above image, we can see a swarm manager who is responsible to manage what a docker worker does. It maintains track of all of its workers' whereabouts. Docker Manager knows what job the worker is working on, what task it has been allocated, how assignments are distributed to all jobs, and whether the worker is up and running.
Docker Manager's API is used to build a new service and orchestrate it. Assigns tasks to workers using the worker’s IP addresses.
Worker
The Docker Manager has complete power over a Docker Worker. The Docker Worker accepts and executes the tasks/instructions that the Docker Manager has delegated to it. A Docker Worker is a client agent that informs the manager about the state of the node it’s been running on through a REST API over HTTP protocol.

services

The tasks to be executed on the manager or worker nodes are described by a service. It is the swarm system's core mechanism and the main point of user engagement with the swarm.
When you create a service, you also create containers and also specify tasks, which must be executed inside them.
The swarm manager distributes a certain number of replica tasks among the nodes in the replicated resources model depending on the scale you set in the desired state.

Load Balancing To expose the resources you want to make available externally to the swarm, the swarm manager uses ingress load balancing. The swarm manager can automatically add a PublishedPort to the service automatically or manually configure one. If you don't mention a port, the swarm manager assigns a port between 30000 and 32767 to the operation.

External modules, such as cloud load balancers, can access the service through the PublishedPort of any node in the cluster. regardless of whether that node is actually performing the service's task. Ingress links are routed to a running task instance by all nodes in the swarm.

Swarm features

After looking at what the docker swarm is and its related terminology, let us see what are the different features that swarm mode offers on the docker engine

Use the Docker Engine CLI to create a swarm of Docker Engines where you can deploy application services. You don’t need additional orchestration software to create or manage a swarm.
The Docker Engine manages some specialization at runtime, rather than managing distinction between node roles at deployment time. The Docker Engine can be used to deploy both manager and worker nodes. This means that a swarm can be created on just a disc image.
Docker Engine takes a declarative approach of defining the optimal state of your application stack's different resources. For example, a web front-end server with message queueing services and a backend database might be described as an application.
You should declare the number of tasks to be performed with each service. When the swarm manager scales up or down(this means that we are scaling up or down the number of services or containers), it automatically adapts to preserve the desired environment, by adding or deleting tasks.
The swarm manager node checks the cluster status continuously and reconciles any inconsistencies between the current state and the target state. For instance, you set up a service of running 10 container replicas on 5 workers, and two of the replicas on one of the worker crashes. Then, the manager creates two more replicas and assigns them to the worker who is up and running.
A network overlay for your services may be specified. When the application is initialized or modified the swarm manager immediately assigns addresses to the containers in the overlay network. The ports may be shown to an external load balancer for the utilities. Internally, you should decide how service containers can be distributed between nodes.

Now, let us get into some practical, we are going to create a cluster, add two worker nodes and then deploy services to that swarm.
Pre-requisites:

Three Linux machines, that can communicate over a network
Docker installed on all three of them.

This tutorial includes three Linux hosts with Docker-enabled and network-compatible communications. They can be physical, virtual, or Amazon EC2 instances or otherwise hosted.
One will be a manager and the other two will be workers.
We are going to use three Linux machines hosted on AWS, that is EC2 instances.

While creating EC2 instances add the following rules in the security group as follows:
The following ports must be available. On some systems, these ports are open by default.

TCP port 2377 for cluster management communications
TCP and UDP port 7946 for communication among nodes
UDP port 4789 for overlay network traffic

While creating the manager machine add these rules.
Then while creating the worker nodes use the same security group created for the manager machine.

Next, ssh into all the machines and install docker-engine.
Use the following commands to install docker-engine on all three machines.

Update and add packages of the apt installation index so that apt can use an HTTPS repository.

$ sudo apt-get update

$ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

Add Docker’s official GPG key:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Use the following command to set up the stable repository

$ echo \ "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install docker engine

$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io

Repeat this for all the three machines.

You are able to create a swarm after you have completed all the setup steps. Ensure that the host machines are running the Docker Engine daemon. Open a terminal and ssh on the computer where the manager node must be executed.

Run the following command to create a new swarm:

$ docker swarm init --advertise-addr <MANAGER-IP>

We have a manager machine with IP 172.31.80.181, so the command is:

$ docker swarm init --advertise-addr 172.31.80.181

The --advertise-addr flag configures the manager node as 172.31.80.181 to publish its address. The other swarm nodes must have access to the IP address of the manager.
The output consists of commands to attach new nodes to the swarm. Nodes will enter as managers or workers according to the flag value.

To see the current state you can use docker info:

$ sudo docker info

In this above image, we can see that there are no containers running on the docker server and the swarm flag is active. This also prints out the clusterId and number of managers and nodes etc.

To view information about nodes use the command

$ docker node ls

The * next to the ID represents that this is the node we are currently connected to.
The swarm mode Docker Engine names the machine hostname node automatically.

Adding Worker Nodes

It's time to add worker nodes to the above-created swarm cluster.
ssh into the machine where you want to run your worker.
Now, we must run the output of docker swarm init as a command in this worker terminal:

$ docker swarm join --token SWMTKN-1-05ikz9ituzi3uhn1dq1r68bywhfzczg260b9zkhigj9bubomwb-a003dujcz7zu93rlb48wd0o87 172.31.80.181:2377

You can execute the following command on a manager node to retrieve a worker's join command if your command is not available:

$ docker swarm join-token worker

Do the same with the other worker as well. SSH into another machine and run the join command.

To view worker nodes, open a terminal and ssh in the machine that runs the manager node and execute the Docker node ls command:

$ docker node ls

The manager nodes in the swarm are determined by the column MANAGER. Worker1 and worker2 are identified as working nodes by the empty state in this column.

Deploy service to the swarm

Now we have a cluster with a manager and two workers. We can now deploy services to the swarm cluster.

Open the terminal into the manager node after SSH, and run the following command:

$ docker service create --replicas 1 --name helloworld alpine ping docker.com

Let's break down the above command:

docker service create: to create a service
--replicas: this flag indicates the desired state of 1 running instance.
--name: used to name the service
alpine ping docker.com: this indicates that the services is going to run alpine Linux and the primary command to run inside the instance or service is ping docker.com

To see the list of running services run the below command:

$ docker service ls

This image lists the name of the service we just create and the number of replicas, along with the base image, which is alpine.

Conclusion

In this article, We started with a description of the docker, then we discussed the need for multiple docker hosts. We then described what is docker swarm, its uses, and its working through docker swarm architecture, we also covered, different docker swarm terminologies like manager node and worker node. After we thoroughly understood the docker swarm we stated to implement it or run services in a swarm cluster. We started with creating 3 Linux hosts in AWS as EC2 instances along with the security group configuration(by adding TCP and UDP rules).
We looked at how to create or initialize a swarm cluster through a manager node, then we added a couple of worker nodes to the same cluster. TO conclude we added a service running Linux alpine to run a ping command.
Also this is my first article on dev.to, and really enjoyed writing it.

liked the post?