DEV Community: Ayush Dutta

Web-Apps: Networks-based deep dive

Ayush Dutta — Fri, 18 Oct 2024 06:25:49 +0000

What's HTTP?

HTTP (Hypertext Transfer Protocol) is designed to interconnect nodes containing hypertext, which are documents that can embed text and multimedia. The global interconnection of these nodes forms the internet.

Building an Alternative to HTTP

HTTP has been the backbone of web communication for decades. However, evolving web technologies demand innovation. Let's explore the key components of HTTP and consider possible alternatives.

Understanding HTTP and Hypertext

Hypertext is the foundation of the World Wide Web. Browsers interpret it using HTML (Hypertext Markup Language). When a browser requests a webpage, it sends an HTTP request to a web server, which responds with an HTML document and other resources.

Client-Server Architecture

HTTP uses a client-server model:

The client (typically a browser) sends requests for resources.
The server processes requests and sends back responses.

Rethinking Client-Server Architecture

Alternatives to HTTP could focus on:

Persistent Connections: Keeping connections open for faster communication.
Streaming Data: Supporting continuous, real-time data streaming.
Bidirectional Communication: Allowing real-time, two-way messaging.

Security in Web Protocols

HTTPS (Hypertext Transfer Protocol Secure) encrypts data transfers between client and server. It uses TLS (Transport Layer Security) to ensure:

Data Integrity: Preventing data alteration.
Confidentiality: Encrypting data for intended recipients only.
Authentication: Verifying identities to prevent impersonation attacks.

You're absolutely right, and I apologize for that oversight. Let me add a section on APIs and the request-response model:

APIs and the Request-Response Model

What Are APIs?

Application Programming Interfaces (APIs) are crucial in modern web applications. They define rules for how different software systems communicate, enabling seamless integration between various applications and services.

Key Aspects of APIs:

Standardization: APIs standardize communication between different applications.
Functionality: They allow retrieval of data, execution of operations, and more.
Versatility: APIs power web applications, mobile apps, IoT devices, and more.

The Request-Response Model

Most APIs, including those based on HTTP, use the client-server request-response model:

Request: The client sends a request to the server for a specific resource or action.
Processing: The server processes the request.
Response: The server sends back a response, which may include data, status information, or both.

Example of API Request-Response:

Request:
GET /api/users/123 HTTP/1.1
Host: api.example.com

Response:
HTTP/1.1 200 OK
Content-Type: application/json

{
  "id": 123,
  "name": "John Doe",
  "email": "john@example.com"
}

HTTP Methods Guide

GET

GET /api/users/123 HTTP/1.1
Host: api.example.com

Used to retrieve resources
Should not modify server state
Data is sent in query parameters
Idempotent and cacheable

POST

POST /api/users HTTP/1.1
Host: api.example.com
Content-Type: application/json

{
  "name": "John Doe",
  "email": "john@example.com"
}

Used to create new resources
Can modify server state
Data is sent in the request body
Not idempotent, but can be made cacheable

PUT

PUT /api/users/123 HTTP/1.1
Host: api.example.com
Content-Type: application/json

{
  "name": "John Smith",
  "email": "john.smith@example.com"
}

Used to update existing resources
Replaces the entire resource
Idempotent but not cacheable

PATCH

PATCH /api/users/123 HTTP/1.1
Host: api.example.com
Content-Type: application/json

{
  "email": "john.smith@example.com"
}

Used for partial updates to resources
Only specified fields are updated
Not guaranteed to be idempotent

DELETE

DELETE /api/users/123 HTTP/1.1
Host: api.example.com

Used to delete resources
Idempotent but not cacheable

These methods form the core of RESTful API design, each serving a specific purpose in resource manipulation. The choice of method depends on the operation you want to perform on the resource.

Common API Architectures:

RESTful APIs: Use HTTP methods (GET, POST, PUT, DELETE) to perform operations on resources.
GraphQL: Allows clients to request specific data structures, reducing over-fetching.
WebSocket APIs: Enable real-time, bidirectional communication.

Understanding DNS and Domain Names

The Domain Name System (DNS) translates human-readable domain names into IP addresses. This process involves:

Browser cache check
Query to DNS resolver
Resolver queries DNS servers
Resolution returns the response

Domain Name Hierarchy

Domain names have a hierarchical structure:

Top-Level Domain (TLD): e.g., .com, .org
Second-Level Domain (2LD): e.g., example in example.com
Third-Level Domain (3LD) or Subdomain: e.g., www in www.example.com

DNS Resolution Process

User enters URL
Local cache lookup
Query sent to DNS resolver
Root DNS server queried
TLD DNS server queried
Authoritative DNS server queried
IP address returned to browser

Ports in System Processes

Ports are logical constructs that identify specific processes or services in network communication. They allow computers to direct network traffic to the correct applications.

Types of Ports

Well-Known Ports (0-1023): Used by system processes and main network services.
Registered Ports (1024-49151): Used by applications not part of core services.
Dynamic or Private Ports (49152-65535): Used for temporary connections.

Security Concerns for Ports

Close unused ports
Use firewalls to control port access
Conduct regular port scans

Cloud Machines

Cloud machines are virtual computing instances provided by cloud service providers like AWS, Google Cloud, or Azure. They offer scalability, on-demand availability, and remote access.

Benefits of Cloud Machines

Scalability
Pay-as-you-go pricing
OS flexibility
Managed infrastructure

Containers

Containers are lightweight, portable units that package an application and its dependencies. They share the host OS kernel, making them more resource-efficient than VMs.

Advantages of Containers

Lightweight
Portable
Fast deployment
Consistent environments

Containers vs. Virtual Machines (VMs)

Characteristic	Containers	Virtual Machines
Architecture	Share host OS kernel	Each has its own OS
Startup Time	Fast (seconds)	Slow (minutes)
Resource Usage	Low	High
Isolation	Process-level	Full OS-level
Portability	Highly portable	Less portable

Deploying a Web App to AWS EC2 and Mapping a Domain

Step 1: Create an AWS EC2 Instance

Log in to AWS Management Console
Launch a new EC2 instance:
- Choose an AMI and instance type
- Configure security group (allow HTTP, SSH, and your app's port)
- Launch and download SSH key

Step 2: Deploy the Web Application

SSH into your EC2 instance
Install required software
Upload your web app
Run your application

Step 3: Set Up a Domain Name and Map it to EC2 IP

Buy a domain name
Create an A record pointing to your EC2 instance's public IP
Set TTL to a low value (e.g., 300 seconds)

Step 4: Access the Web App via the Domain

Wait for DNS propagation, then access your app using the domain name.

Step 5: Share the Website Link

Once everything is set up, share your website link (e.g., https://www.example.com).

🤖 Virtual Machines and Containers

Ayush Dutta — Sun, 13 Oct 2024 19:50:24 +0000

One of the most essential and important concerning isolated environments and modern cloud architecture in data centers. Virtual Machines and Containers provide a replication development/deployment environment for software. Despite both existing as an abstraction layer above the host system, they differ conceptually when concerning factors like their architecture, speed and resource footprint.

Virtual Machines

Virtual Machines are specialized environments that emulate a computer operating system. They can be further specialized to deploy particular software. Let's think of running an OS like Linux, but over the host system running Windows. Here, Linux will be treated as a "Guest OS" and will be running atop Windows as an Application.

Virtual machines are governed by a type of software called a "Hypervisor", these can control their deployment.

Hypervisors also enable us to distribute hardware resources among virtual machines according to preference. If we have a 128GB RAM server and a lot of clients that want to access it, we would split the hardware resource among several virtual machines, so that each client gets their own isolated space and OS to run with a particular quantity of machine resources (like CPU and memory) allotted to them.

What can be improved

An Operating system in general consists of two stacked layers over the hardware.

Going from top to bottom, we first encounter the OS Applications layer, which consists of all the software run on the machine.

Secondly, we get the OS kernel, the layer that deals with communication with hardware resources, and acts as a bridge between the hardware and the OS Applications layer.

A virtual machine emulates both the OS kernel and the applications layer stacked in the mentioned order. This results in excessive use of resources, however there is a solution to this. We can use the host's kernel instead of having multiple kernels run altogether.

Enter: Containers

With the added advantage of using the host's kernel, Containers save on resources and boost up speed. Docker is used specifically because it's one of the most popular tools and also the one that revolutionized this category, also its registry, Docker-Hub is one of the largest repositories for pulling dependencies for your project.

We have a container engine that manages these containers or isolated environments, Just like a hypervisor.

Spinning up a new container

Docker can be installed through the official set of instructions on their site.

It uses the Docker engine which the docker daemon responsible for all processes concerning it.

There are two options, one is to either pull an existing container through Docker's registry called Docker Hub. Secondly is to build custom images to suit our project's needs.

Using existing images

To clone a docker container off Docker Hub,

docker pull <image_name>:<version>

We can then start this container by getting the image ID through the docker images command.

docker start <image_ID>

Building a custom image

In order to build a custom image, a blueprint file called a Dockerfile needs to be added to the project directory. This will contain all the configuration code, regarding what dependency versions and base image (which is the OS image that will be emulated) will be included in the container.

A typical Dockerfile would look somewhat similar to the one below.

FROM <primary_service>:<base_image>
COPY <source code directory> /app/
WORKDIR <the working directory>
RUN <any terminal commands>

Now that the blueprint is made, time to build and deploy a docker container with the following configuration.

docker build <location_to_save_container>

This builds the docker container following the blueprint, now this can be uploaded to Docker Hub or any repository to ship it to other environments and servers.

Week 2 Extra: Docker Vs. Podman

Podman is a popular alternative to Docker and possibly is aiming to succeed it. It has now been adopted to be used in container orchestration tools like Kubernetes.

The Advantage

Podman optionally requires root/admin privileges to build/run containers. Which makes it safer for the deployment system.
Unlike Docker which uses a daemon (background service) that constantly listens for new requests using the client-server architecture. Podman's philosophy has a daemon-less architecture and hence saves up on system resources.
Mostly all docker commands work with Podman with the replacement of the word "docker" with "podman" (at least as far as the CLI is concerned).
Podman isn't a monolithic piece of software unlike Docker. It depends on software like systemd. Hence it is more lightweight.

In summary, Podman is overall a safer alternative than Docker.

Some Extra Reads:

❄️ NixOS: OS as Code

Ayush Dutta — Sun, 13 Oct 2024 19:49:46 +0000

There are many Linux-based distributions out there, some are flashy, some are minimal, some are bloated (i guess we all hate those ones, lol). Then we have NixOS, a very fundamentally different Linux distro from all its other relatives. It can be summarized in three words: declarative, reproducible, and immutable.

Declarative

Let's start with the first attribute, NixOS's declarative configuration makes it easy to write configurations for the various components in our operating system in an easy-to-understand language called "Nix". Nix is very similar to JSON syntax wise.
For declaring NixOS config, we write a file called configuration.nix which is stored in /etc/nixos/

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix

    ];

  # Bootloader.
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  networking.hostName = "endernix"; # Define your hostname...
  # Enable networking
  networking.networkmanager.enable = true;

  # Enable docker
  virtualisation.docker.enable = true;

  # Enable bluetooth
  hardware.bluetooth.enable = true;
  hardware.bluetooth.powerOnBoot = true;
...
}

The comments above each line or chunk of code describe their functionality. For example, the Bluetooth chunk describes that when re-building the system again, it needs to enable and start the bluetooth service on boot. We'll get into rebuilds later on, but how do we install packages? Easy. we declare it in the same file.

{ ...
users.users.ifkash = {
    isNormalUser = true;
    description = "Kashif";
    extraGroups = [ "networkmanager" "wheel" "docker" ];
    packages = with pkgs; [
      # Browsers
      brave
      firefox
      google-chrome
      # Fetch tools
      neofetch
      sysfetch
      nitch
    ];
};

...
}

Now these blocks can be placed anywhere in the file, because Nix is a purely functional programming language, which in layman's terms mean that positions of definitions does not matter sequentially.

Reproducibility

Once our system's file has been configured, it's time to build/modify our system. Running sudo nixos-rebuild switch will read the entire configuration file and build a new Nix system accordingly. Now comes the question, would it take nearly double the size of the previous version if i simply include small packages in the new rebuild? The answer is No.
/nix/store

Nix store is a location where Nix system stores all the names of the packages in the form of files with their dependency list hashed and attached to the file name. This way, whenever the system is rebuilt, if it is available in nix store, it is simply ignored. If not, a new file is created in nix store and added with appropriate rules.

We can also go back to our previous build with `nixos-rebuild --rollback switch
Rollbacks - Never break your system again

Hence, incase our new system breaks for some reason, we can always switch back to the previous version and get our work done. This feature gives it the deadly advantage against distros like Arch Linux.

Immutability

Since the system is defined declaratively, it's not possible to modify the system's existing state (like updating/upgrading packages, etc) This is not a bug, but a feature. It enables further safety against breaking our systems. This also means we cannot install additional things into the system in the current rebuild. It also makes the system more secure against attacks that intend to change ownership permissions or corrupt the boot files of the system.

The Nix package manager

The Nix package manager fetches from the Nix package repository, one of the largest repos among all operating systems, it even beats Arch's user repository well known for its enormous package support. Hence, Nix boasts better software support than most operating systems on the planet.

You can search your favorite packages here: https://search.nixos.org/packages

Why use NixOS as a daily driver?

Apart from excellent software support, and all its other features. NixOS doesn't feel different from other Linux distros at all on the surface level. There's that feel-at-home and works-out-of-the-box feeling. Now that NixOS's installation ISO comes with the graphical calamares installer, it makes installing it even easier.

Acknowledgements

🛠️ Setting up Arch

Ayush Dutta — Sun, 13 Oct 2024 19:48:19 +0000

Downloading the ISO

An ISO is an image of the operating system. It acts as the installer for the operating system.

The ISO can be downloaded from one of ArchLinux's global mirrors, and flashed to create a bootable USB.

Setting up Ventoy: Flashing multiple ISOs to one USB

Just to be safe, we will use Ventoy to flash at least two different ISOs to our USB stick to make sure that if installing Arch is not possible for some case, we can always install some other linux distribution. However this is a very rare case because people don't usually sole-install Arch on their devices.

Link to installing: Ventoy

Let's assume that we are already on a Linux distribution (If you are on Windows, please follow this guide ). After we install the Ventoy binary, we will run it with this command:

$ sh Ventoy2Disk.sh -i /dev/sdX

where /dev/sdX is the device file name for our USB, we can know what that is through the fdisk -l command.

After Ventoy is installed, we can simply drag and drop our ArchLinux ISO into our flash drive's directory.

Diving into Arch: First boot

We will be greeted with a command line after we select the first option from the boot loader.

Connecting wi-fi

In most cases of a home computer, we may not have an ethernet cable laying around, so, we will connect our computer to wi-fi with iwctl utility.

Simply typing iwctl will open the iwd shell prompt. From here we must type the following command to view all our network devices:

$ [iwd]: device list

After we grab our device. We will make it scan for active wi-fi networks and display them:

$ [iwd]: station <device_name> scan
$ [iwd]: station <device_name> get-networks

Then select and connect to the network:

$ [iwd]: station <device_name> connect <wifi_name>

We will be asked to pitch in the password for that SSID. After we connect, we can test our connection with the ping command. And press CTRL-C to stop pinging. This is optional.

Partitioning drives

We will use the cfdisk utility to edit our drive partitions.

$ cfdisk /dev/sdY

here /dev/sdY is the device file name for the drive whose partitions we want to edit. We will assume that we are clean installing Arch on the entire drive.

In order to delete existing partitions, we will simple hover over the partition section with arrow keys and press delete.

After we select and enter write, our formatting is done. Let's create our new partitions now.

We will click on new then give the first partition a size of 100M meaning 100 megabytes, this will house the boot loader for our distribution.

After pressing enter, we will hover back to free space and repeat the process. This time we will give it a size of 4G. This will be house the swap memory for our distribution. It is that part of the drive which can be used as extra memory incase our RAM fills up.

Lastly, we will select free-space and not edit the default size suggested. Which will be all of the remaining part. This will house our files.

After everything's done. We will click write then primary then enter. Then quit.

Building and mounting file systems

We can know our partition names and size from the lsblk command.

$ mkfs.fat -F 32 <boot_partition>
$ mkswap <swap_partition>
$ mkfs.btrfs <storage_partition>

Here, we are using FAT32, SWAP, and btrfs file systems for our partitions. They are, almost the most optimal choices for file systems for these partitions.

After that, we will mount or make the file systems accessible from Arch.

$ mkdir -p /mnt/boot/efi
$ mount <boot_partition> /mnt/boot/efi
$ swapon <swap_partition>
$ mount <storage_partition> /mnt

Here, we are creating the boot and then nesting that with the efi directory to store the boot partition.

We are mounting the rest of the partitions to their required position.

Installing essential packages

Now, we are ready to install the core packages of our system.

$ pacstrap -K /mnt base linux linux-firmware sof-firmware base-devel nano networkmanager grub

We are installing the Linux kernel, sound-card firmware (for newer) systems, build-tools, network manager, the GRUB boot-loader and a text editor for editing system config files.

Generating the fstab file

$ genfstab -U /mnt > /mnt/etc/fstab

fstab is our Linux system's filesystem table, aka fstab , which is a configuration table designed to ease the burden of mounting and unmounting file systems to a machine.

Time to enter our newly installed arch system:

$ arch-chroot /mnt

System configurations

We can set the time zone with the following command

$ ln -sf /usr/share/zoneinfo/Region/City /etc/localtime
$ hwclock --systohc

where Region/City is your region (Just click TAB to set it as you go). The second command syncs the system clock with the set timezone.

Generating 'locale's or general configurations that are to be used by all or most programs can be done with this:

$ locale-gen
$ echo 'LANG=en_US.UTF-8' > /etc/locale.conf
$ echo 'KEYMAP=us' > /etc/vconsole.conf

Creating the hostname file for our system

$ echo '<hostname>' > /etc/hostname

Root and User configuration

$ passwd
$ useradd -m -G wheel ‘USERNAME’
$ passwd ‘USERNAME’

This will prompt us for the root password. Then create a user and add it to the group wheel. Lastly we will set the password for this user.

It's time to add root privileges to this user.

$ EDITOR=nano visudo

Over here, we will head over to the first statement and look for the line:

#%wheel%   ALL=(ALL:ALL) ALL

we will uncomment this line to enable all users underneath this group to have root privileges. Exit nano with CTRL+S and CTRL+X.

Update the packages:

$ pacman -Syu

Installing GRUB boot-loader

$ grub-install <drive_name>

we can replace <drive_name> with the device file name for our drive (Not partition).

Finishing steps

we will unmount the /mnt directory and reboot into our new system!

$ umount -R /mnt
$ reboot

Please remove the USB after the screen goes blank after the reboot command. We now boot into a fresh install of ArchLinux!

It is advisable to install a desktop environment like GNOME if you are using it as a daily driver. Just login now. Welcome!

Some extra reads:

Zed's Linux port needs more punch

Ayush Dutta — Thu, 10 Oct 2024 15:18:47 +0000

I've been waiting for the Zed editor to release their Linux port for a long time now, ever since it was released. But as soon as it did, I wasn't impressed, but neither was i disappointed. Here are my thoughts...

Rather than being written in Javascript like VSCode, Zed offers better speeds, along with other benefits provided by the Rust language. Being a recent Rust fanatic myself, I appreciate it very much.

Whenever I open Zed project on my computer, it lags by a few seconds. Sure, but once it's open, it's as smooth as butter. However that is not the case with Neovim since it uses the terminal emulator as a base.

Sure, there are plenty of extensions out there, but the extensions marketplace doesn't really have as many extensions as something like Jetbrains or Neovim, given that they took ages to release the Linux port, I expected more. If this is the case, they should eliminate the marketplace feature fully and adopt a similar config system like neovim's for installing plugins.

Zed supports many languages out of the box, including syntax highlighting and auto complete. This is however, a trade-off because the user needs to tinker a lot to remove support for languages that are not needed. This is also NOT the case with VSCode, where you install the extension (LSP, Formatters,Linters, etc) of the language only when you need it.

Currently Zed editor supports native packages for only a few Linux distros like NixOS. Even for Arch, i believe it's only through the Arch wiki. Otherwise you may need to depend on curling an install script.

I personally tend to stay away from these AI coding assistants because they collect usage data, need API keys, or just ruin your ability to code overtime with the habit of reliance.

I feel that I might stay on the Neovim side longer as my config offers more than the Zed editor provides to my workflow. I know it is all about personal choices. I acknowledge that these were all purely my opinions on the topic (feel free to roast me if you're a Zed fanboy). There's just nothing on Zed currently that might attract me to make the switch, even with Ai code completion (which can be easily implemented in any mainstream editor nowadays). Zed needs to pack more of a punch.

Hacking the LinkedIN algorithm for developers

Ayush Dutta — Sat, 05 Oct 2024 20:09:33 +0000

First thing's first, this is not a post about penetration testing any systems. We're gonna talk about how to boost your Linkedin from zero to hero in a matter of months. We're also gonna talk about being real and raw in this post, not the shenanigans of how to get down to "influencing", we're here to do actual business. Let's kick the tires.

Diving into the essentials first, we'll need a profile that's cool and yet not to casual, It's okay to wear something casual (but not too fancy). Try to make the background as subtle or peaceful as possible or just replace it. Also try to make sure you have a banner following the same color-scheme. It can be a single color shade, have your logo or be a photo of you with your community.

Tagline: The most important part of one's profile. Here's the priority list you should sequence your tagline on:

What's your skill? (frontend,backend,fullstack,ML, etc)
What're you doing in your career life? (interns, jobs, etc)
Anything you're interested in (in terms of tech)
Anything you like doing (can be non-tech)

Now, we've got a lot of things done. Time to construct your bio, make sure it is short,crisp and upto 50 words. Distill your resume down to those 50 words. Yes, you are free to prompt ChatGPT or some other LLM.

Experiences (optional if you don't have any): Roles and companies don't matter on the profile unless you're some founder or revolutionary guy or you got a nice company. If you're not in this group, then make sure to bullet doing your job description clearly in the description section under the role.

Accordingly add your education, and add your volunteering experiences with any communities. Be sure to add pictures in all of these including experience sections.

Since we are just beginning, we can skip the featured section for now. And as for the certifications section, keep certificates that actually matter, like AWS ones, not those from coursera or udemy.

Time to connect with people, search people in your interested domain through #keyword on the search bar, keep connecting with people through search, networks section and DMing existing connections. Your Linkedin feed's content will determine the content you consume and that will be determined by the people you follow and connect with.