DEV Community: Syed Aun Abbas

Partial Pre-Rendering in Next.js

Syed Aun Abbas — Sun, 16 Mar 2025 19:44:23 +0000

Introduction

Next.js 14 introduces an exciting experimental feature called Partial Pre-Rendering (PPR). This new approach to rendering aims to improve performance and user experience by blending the best aspects of static rendering and dynamic rendering.

In this post, we'll explore what prerendering is, how it works, and why it’s crucial for modern web applications.

What is Prerendering?

Before diving into Partial Pre-Rendering, let's first understand prerendering and why it matters.

Dynamic Rendering: The Traditional Approach

Typically, when a server receives a request for a webpage, it generates the page dynamically and sends it to the client. If another user makes the same request, the process repeats. This is known as dynamic rendering.

While this approach ensures that the page is always up to date, it has some drawbacks:

Performance Issues: Generating a page dynamically for every request increases server load and response time.

Scalability Challenges: High-traffic applications may struggle to handle large volumes of dynamic requests.

However, dynamic rendering is necessary for certain types of content:

Personalized pages (e.g., an e-commerce site showing different recommendations for each user).

Frequently updated data (e.g., real-time stock prices or news feeds).

Static Rendering: The Optimized Alternative

On the other end of the spectrum, many web pages do not need dynamic rendering because their content remains the same for all users.

For such pages, we can optimize performance by prerendering them at build time. Instead of generating the page every time a user requests it, the content is rendered once and stored as a static file.

These prerendered pages are then pushed to a Content Delivery Network (CDN). When a user requests the page, it is served instantly from the closest edge location, reducing load times and improving scalability.

Example of Static Rendering

A company’s blog page is a great candidate for prerendering. Since the content does not change frequently, it makes sense to generate it once and serve it as a static file instead of re-rendering it dynamically on each request.

The Problem with Single Paradigm Frameworks

A Look Back in Time

A few years ago, web frameworks typically specialized in a single rendering strategy:

Server-Side Rendering (SSR)

Static Site Generation (SSG)

Client-Side Rendering (CSR)

Backend API Endpoints

This forced developers to make an all-or-nothing decision about their entire application’s rendering approach, even if different routes required different strategies.

As a result, it was common for teams to split their applications across multiple frameworks or even different programming languages. A company might host its blog, dashboard, and API on separate subdomains, each built using different technologies just to support different rendering needs.

The Need for Flexibility

As user expectations evolved, developers began encountering friction with these single-paradigm frameworks:

Dynamic-heavy apps wanted to statically render some pages ahead of time.

Static sites wanted to add personalization and interactivity.

Server-rendered applications wanted more client-side flexibility.

Client-rendered applications wanted better SEO and initial load performance.

How Next.js Solves This Problem

Next.js gained popularity because it broke free from the single-paradigm approach. It provided teams with:

A unified toolset for handling different rendering strategies.

A consistent language and routing system across all pages.

The ability to mix and match SSR, SSG, CSR, and API routes seamlessly.

This made it easier for developers to build modern web applications without managing complex handoffs between different frameworks and backend services.

The Problem with Static and Dynamic Rendering

Fast forward to today, and once again, we are starting to reach the limits of single rendering paradigms, but this time at a more granular page level.

Even the most static pages, like documentation or blog posts, sometimes require dynamic elements. For example:

Personalized code snippets tailored to the user.

A global navbar displaying the signed-in user's information.

At the same time, even highly dynamic pages share large portions of static content across users.

The Limitation of Prerendering

While prerendering is great for performance, it has an inherent limitation:

We cannot prerender a page that depends on runtime information before receiving the request that contains that data.

This forces developers into yet another all-or-nothing decision:

Prerender the page for better performance but lose the ability to personalize it.

Dynamically render the entire page on every request, making it slower and more resource-intensive.

This is exactly where Partial Pre-Rendering comes in.

Partial Pre-Rendering (PPR) allows us to blend prerendering and dynamic rendering within the same page.

Static parts of the page can be prerendered ahead of time.

Dynamic elements are loaded at runtime as needed.

This approach enables pages to be as static or as dynamic as they need to be, providing the best balance between performance and interactivity.

Current Next.js Behavior

As a reminder, Next.js currently prerenders a page at build time unless it uses dynamic APIs such as:

Incoming request headers

Uncached data requests

When Next.js detects the use of these APIs, it assumes the developer intends to use dynamic rendering and opts the entire page into runtime rendering.

The Problem with This Approach

Right now, if a page uses just one dynamic function, it forces all its parent components (up to the root) into dynamic rendering—even if those components don’t actually use any runtime data.

This means that adding a single dynamic element to an otherwise static page turns the entire page into a dynamically rendered one, eliminating the performance benefits of prerendering.

A Better Way: Isolating Dynamic Elements

What if we could prevent this side effect from spreading upwards?

What if dynamic elements could exist on a prerendered page without opting the entire page into runtime rendering?

What if we could mix and match prerendered and dynamic content more efficiently?

This is exactly what Partial Pre-Rendering (PPR) aims to achieve.

React Boundaries to the Rescue

To solve the challenge of combining static and dynamic rendering, we can use React Boundaries to account for different rendering modes.

Error Boundaries

A page can be in a functioning or non-functioning state. With React Error Boundaries, we can wrap specific parts of a page and define fallback UI in case of errors. If an error occurs inside the boundary, the rest of the page remains unaffected, and the fallback UI is displayed instead.

This prevents a single failure from breaking the entire page and improves resilience in complex applications.

Suspense Boundaries

Similarly, we can wrap components that include asynchronous operations in a React Suspense Boundary. This allows us to design fallback UI for temporary loading states.

When rendering on the server, React immediately streams the fallback UI to the client.

Once async operations complete, the actual content is seamlessly loaded without blocking the rest of the page.

This prevents slow async operations from delaying the initial page load, improving performance and user experience.

Extending Suspense for Partial Pre-Rendering

With Partial Pre-Rendering, Next.js extends Suspense Boundaries even further:

Developers can wrap components that use runtime dynamic APIs in a Suspense Boundary.

This allows the static parts of a page to be prerendered at build time.

Meanwhile, dynamic elements can load later without affecting the prerendered content.

By isolating dynamic elements inside Suspense, we prevent dynamic APIs from opting the entire page into runtime rendering, preserving the benefits of static optimization.

How Does Partial Pre-Rendering Work?

Deployment Process

When your application is deployed:

The partially prerendered result is pushed to an Edge Network for global distribution.

At runtime, when a user visits the page, edge compute serves the static prerendered result instantly.

Simultaneously, a request is sent to a runtime server to render dynamic parts.

Client-Side Rendering Process

On the client:

The browser starts rendering the prerendered HTML while downloading essential static assets like images, fonts, stylesheets, and JavaScript.

Client-side components become interactive as JavaScript loads.

The server uses runtime data (e.g., request headers, cookies) to fetch and render missing dynamic content.

The browser fills in the dynamic holes by replacing prerendered placeholders with fresh data as they stream in.

This approach ensures fast initial loading while still supporting personalized and real-time updates.

How Is Partial Pre-Rendering Different?

The Challenge with Dynamic Rendering

When a page is dynamically rendered, there is a delay between the client’s request and the server’s response. This delay can increase due to:

Long distances between the user and the rendering server.
Slow, uncached data requests.
Lack of streaming support.
Cold starts in serverless environments.

During this waiting period, users and web crawlers see nothing, and the browser remains idle.

The Advantage of Static Rendering

With static rendering, this delay is shorter since there’s no runtime computation, and the response is served directly from an Edge Network.

How Partial Pre-Rendering Bridges the Gap

Partial Pre-Rendering maintains the benefits of static rendering by:

Serving a fast initial response from the Edge.

Allowing the browser to begin rendering immediately.

Letting the server process dynamic content in parallel and stream updates.

Just as prerendering optimizes pages that don’t change between requests, Partial Pre-Rendering optimizes the parts of pages that remain unchanged, improving both performance and interactivity.

Summary

That covers the concept of Partial Pre-Rendering. While the details can get complex, the good news is:

No need to deeply understand how it works to benefit from it.
No new APIs to learn—it integrates seamlessly into existing workflows.
No upfront infrastructure concerns—Next.js automatically handles optimization.
No forced all-or-nothing rendering decisions—static and dynamic content can coexist.

Instead, developers can write their code as if the entire page is dynamic:

Use runtime APIs and uncached data requests as needed.
Add Suspense Boundaries to progressively stream and break up rendering.
Let React and Next.js automatically optimize each part, combining the best of static and dynamic rendering.

Credits: https://www.youtube.com/watch?v=MTcPrTIBkpA

Understanding the Kubernetes Architecture

Syed Aun Abbas — Sat, 17 Aug 2024 17:20:38 +0000

Kubernetes, often abbreviated as K8s, has rapidly become the go-to solution for container orchestration. Its architecture is both powerful and flexible, designed to manage containerized applications at scale. In this blog post, we'll delve into the core components of Kubernetes architecture, explaining how they work together to deliver the robustness that makes Kubernetes a leading choice for managing containerized workloads.

What is Kubernetes?

Before diving into the architecture, it's important to understand what Kubernetes is. Kubernetes is an open-source platform designed to automate the deployment, scaling, and management of containerized applications. Initially developed by Google, Kubernetes is now maintained by the Cloud Native Computing Foundation (CNCF).

The Building Blocks of Kubernetes Architecture

Kubernetes architecture is divided into two main components: Control Plane and Worker Nodes. Each of these plays a critical role in the overall functioning of a Kubernetes cluster.

1. The Control Plane

The control plane is the brain of the Kubernetes cluster, managing and maintaining the desired state of the applications running in the cluster. It consists of several key components:

a. API Server (kube-apiserver)

The API Server is the front end of the Kubernetes control plane. It exposes the Kubernetes API, which is used by all components to communicate with one another. The API Server processes REST operations, validates them, and updates the corresponding objects in the cluster.

b. Etcd

Etcd is a distributed key-value store used to store all cluster data. It’s the source of truth for the cluster state, including configuration data, secrets, and status information. Because etcd is so critical to the operation of Kubernetes, it’s typically run in a highly available configuration.

c. Controller Manager (kube-controller-manager)

The Controller Manager runs various controllers that handle routine tasks within the cluster. These controllers include:

Node Controller: Manages node lifecycle, detecting and responding to node failures.
Replication Controller: Ensures that the desired number of pod replicas are running at all times.
Endpoint Controller: Populates the Endpoints object, which is used to associate services with pods.
Service Account & Token Controllers: Manage service accounts and access tokens for pods.

d. Scheduler (kube-scheduler)

The Scheduler is responsible for assigning pods to nodes. It watches for newly created pods that have no node assigned and selects a node for them based on various factors like resource availability, taints, and affinities.

2. Worker Nodes

Worker nodes are the machines that run the actual applications or workloads in the form of containers. Each worker node contains the following components:

a. Kubelet

Kubelet is the primary node agent that communicates with the Kubernetes API Server. It ensures that containers are running in a pod and in the desired state as per the pod specifications.

b. Kube-proxy

Kube-proxy maintains network rules on nodes. It enables communication to pods from inside and outside the cluster by forwarding requests to the correct containers based on IP addresses and ports.

c. Container Runtime

The container runtime is the software responsible for running containers. Kubernetes supports various container runtimes, such as Docker, containerd, and CRI-O. The container runtime interfaces with Kubelet to manage the lifecycle of containers.

3. Pods: The Smallest Deployable Unit

In Kubernetes, the smallest deployable unit is the Pod. A Pod encapsulates one or more containers that share the same network namespace and storage. Pods are designed to be ephemeral; they can be destroyed and recreated at any time. Therefore, Kubernetes abstracts the concept of persistent storage through Persistent Volumes and Persistent Volume Claims.

4. Services, Ingress, and Network

Kubernetes provides networking solutions to ensure that pods can communicate with each other and with external services.

Services: A Kubernetes Service is an abstraction that defines a logical set of pods and a policy by which to access them. Services can be exposed within the cluster, or externally via NodePort, LoadBalancer, or Ingress.
Ingress: Ingress manages external access to services within a cluster, typically HTTP or HTTPS. It provides load balancing, SSL termination, and name-based virtual hosting.
Networking Model: Kubernetes assumes a flat network structure where every pod can communicate with every other pod without NAT. This model is realized using various CNI (Container Network Interface) plugins.

5. Storage in Kubernetes

Storage in Kubernetes is abstracted to allow applications to consume storage resources without needing to know the details of the underlying storage provider. Kubernetes supports several storage options:

Persistent Volumes (PV): Storage resources available in the cluster.
Persistent Volume Claims (PVC): Requests for storage by a user.
Storage Classes: Allow administrators to define different classes of storage.

6. Extensions and Add-Ons

Kubernetes is highly extensible. Some of the common extensions include:

Custom Resource Definitions (CRDs): Allow users to define their own custom resources.
Operators: Automate the management of complex applications.
Helm: A package manager for Kubernetes, which simplifies the deployment and management of applications.

Conclusion

Kubernetes architecture is a powerful and flexible system designed to manage containerized applications at scale. Its components work in harmony to maintain the desired state of the applications, ensuring high availability, scalability, and efficiency. Understanding the core elements of Kubernetes architecture is crucial for anyone looking to deploy, manage, or scale applications in a cloud-native environment.

Whether you're a beginner trying to get your head around Kubernetes or an experienced developer looking to deepen your knowledge, understanding the architecture is the first step towards mastering Kubernetes. With this foundation, you can confidently explore the advanced features and capabilities of this robust container orchestration platform.

Dockerizing a Node.js Application with MongoDB

Syed Aun Abbas — Fri, 28 Jun 2024 21:22:07 +0000

In this blog post, I will guide you through Dockerizing a Node.js application that uses MongoDB. We will cover setting up the Node.js app, creating a MongoDB database, and using Docker to containerize both services. By the end of this tutorial, you'll have a working application running in Docker containers.

Prerequisites

Basic knowledge of Node.js and Express
Basic understanding of Docker and Docker Compose
Node.js and Docker installed on your machine

Project Structure

Step 1: Setting Up the Node.js Application

First, let's set up our Node.js application. We'll create a simple product management API using Express and MongoDB. We'll use the src folder for this in our project.

`ProductController.js`

This file contains the controller logic for handling requests related to products.

import Product from '../models/Product.js';

async function index(req, res, next) {
    try {
        const products = await Product.find();
        return res.status(200).json({
            status: 200,
            data: products
        });
    } catch (e) {
        console.log(e);
        return res.status(500).json({
            message: e
        });
    }
}

async function save(req, res, next) {
    try {
        const product = new Product();
        product.title = req.body.title;
        product.description = req.body.description;
        product.price = req.body.price;
        await product.save();

        return res.status(201).json({
            status: 201
        });
    } catch (e) {
        console.log(e);
        return res.status(500).json({
            message: e
        });
    }
}

export default {
    index,
    save
};

`Product.js`

This file defines the Mongoose schema for our product model.

import mongoose from 'mongoose';

const ProductSchema = new mongoose.Schema({
    title: {
        type: String,
        required: true
    },
    description: {
        type: String,
        required: true
    },
    price: {
        type: String,
        required: true
    }
});

export default mongoose.model('Product', ProductSchema);

`products.js`

This file defines the routes for our product API.

import express from 'express';
import productsController from '../controllers/ProductController.js';

const router = express.Router();

router.get('/', productsController.index);
router.post('/', productsController.save);

export default router;

`app.js`

This file sets up our Express application and connects to MongoDB.

import express from 'express';
import productRoutes from './routes/products.js';
import mongoose from 'mongoose';
import config from '../config.js';
import bodyParser from 'body-parser';

const connectToDB = async () => {
    try {
        await mongoose.connect(config.db_uri, {});
    } catch (e) {
        console.log(e);
        process.exit(1);
    }
};

const app = express();

// middleware
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

app.use('/products', productRoutes);

await connectToDB();

export default app;

`index.js`

This file starts the Express application.

import app from './app.js';
import config from '../config.js';

app.listen(3000, () => {
    console.log(config.app_name + ' Started on Port 3000');
});

`config.js`

import dotenv from 'dotenv';
import mongoose from "mongoose";

dotenv.config();

const config = {
    app_name: process.env['APP_NAME'],
    port: process.env['PORT'] ?? 3000,
    db_uri: process.env['DB_URI'] ?? 'mongodb://localhost:27017/docker',
    db_options: {
        useNewUrlParser: true,
        useUnifiedTopology: true
    }
}
export default config;

`env`

APP_NAME=LearnDocker
PORT=3000
DB_URI=mongodb://127.0.0.1:27017/dockerlearn #This is a local Mongo database

Step 2: Dockerizing the Application

Now, let's create a Dockerfile to containerize our Node.js application.

Dockerfile

# Use the official Node.js image based on Alpine Linux, which is a lightweight distribution
FROM node:alpine

# Set the working directory inside the container
WORKDIR /usr/src/app

# Copy package.json and package-lock.json to the working directory
COPY ./package.json ./
COPY ./package-lock.json ./

# Install the Node.js dependencies
RUN npm install

# Copy the application source code to the working directory
COPY ./src ./src

# Copy the environment configuration file
COPY ./.env ./

# Copy the configuration file
COPY ./config.js ./

# Define the command to run the application
CMD ["npm", "start"]

Docker Compose

We'll use Docker Compose to set up our Node.js application and MongoDB as separate services.
docker-compose.yml

version: "3"

services:
  # Define the MongoDB service
  mongo_db:
    container_name: database_container  # Set a custom name for the MongoDB container
    image: mongo:latest  # Use the latest version of the MongoDB image
    restart: always  # Always restart the container if it stops or fails
    volumes:
      - mongo_db:/data/db  # Map the mongo_db volume to /data/db inside the container to persist data

  # Define the Node.js application service
  app:
    build: .  # Build the image from the Dockerfile in the current directory
    ports:
      - 4000:3000  # Map port 4000 on the host to port 3000 in the container
    environment:
      APP_NAME: LearnDocker  # Set the application name environment variable
      PORT: 3000  # Set the application port environment variable
      DB_URI: mongodb://mongo_db:27017/dockerlearn  # Set the MongoDB URI to connect to the MongoDB container
    depends_on:
      - mongo_db  # Ensure the mongo_db service is started before this service

volumes:
  mongo_db: {}  # Define a named volume for MongoDB data

Step 3: Running the Application

To run the application, follow these steps:

Build the Docker images:

   docker-compose build

Start the services:

   docker-compose up

You can test the application on Postman at the endpoint localhost:4000/products

You should now see the product management API in action.

Conclusion

Congratulations! You've successfully Dockerized a Node.js application with MongoDB. This setup provides a clean and efficient way to manage your application and its dependencies. Feel free to expand this project further and explore more features of Docker and Node.js.

Dockerfile Instructions

Syed Aun Abbas — Wed, 15 May 2024 20:29:27 +0000

Introduction:
In the fast-paced world of containerization, Docker has become the go-to solution for building, deploying, and managing applications. At the heart of Docker's efficiency lies the Dockerfile, a simple yet powerful script that automates the creation of Docker images. In this guide, we'll explore essential Dockerfile commands, unraveling their significance through practical examples.

1.FROM: Laying the Foundation
The FROM command defines the base image used to start the build process. It sets the starting point for your Docker image, ensuring compatibility and reproducibility across different environments.

Example:

FROM ubuntu:20.04

This sets the base image as Ubuntu 20.04, providing a stable platform for subsequent commands.

2.RUN: Executing Commands
The RUN command executes commands within the Docker image during the build process. It enables you to install dependencies, configure the environment, and perform other tasks necessary for preparing the image.

Example:

RUN apt-get update && apt-get install -y python3

This installs Python 3 within the Docker image, ensuring that the necessary dependencies are available.

3.CMD: Defining Default Command
The CMD command specifies the default command or executable to be executed when the container starts. It sets the primary functionality of the container, allowing you to define its behavior by default.

Example:

CMD ["python3", "app.py"]

This specifies that the default command to run when the container starts is to execute the Python script app.py.

4.ENTRYPOINT: Setting Default Application
The ENTRYPOINT command sets a default application to be used every time a container is created with the image. It provides a fixed entry point for the container, ensuring consistent behavior across deployments.

Example:

ENTRYPOINT ["python3", "app.py"]

This sets the default executable to run when the container starts, ensuring that our Python script app.py is executed.

5.ENV: Configuring Environment Variables
The ENV command sets environment variables within the Docker image. It allows you to customize the container environment, making it adaptable to different deployment scenarios.

Example:

ENV PORT=8080

This sets the environment variable PORT to 8080, allowing dynamic configuration of the port on which the application listens.

6.EXPOSE: Enabling Networking
The EXPOSE command associates a specific port to enable networking between the container and the outside world. It serves as documentation for users of the image regarding the network connectivity requirements of the containerized application.

Example:

EXPOSE 8080

This declares that the container listens on port 8080, allowing external services to communicate with the application running inside the container.

7.ADD: Copying Files
The ADD command copies files from a source on the host into the container's filesystem at the specified destination. It facilitates the inclusion of application code, configuration files, and other resources necessary for the container environment.

Example:

ADD . /app

This copies the contents of the current directory into the /app directory within the Docker image.

8.MAINTAINER: Defining Image Creator
The MAINTAINER command specifies the full name and email address of the image creator. While not mandatory, it provides essential metadata about the image.

Example:

MAINTAINER John Doe <john@example.com>

This defines John Doe as the image creator with the email address john@example.com.

9.USER: Setting User ID
The USER command sets the UID (or username) which is used to run the container. It allows you to specify the user context in which the containerized application operates.

Example:

USER appuser

This sets the user context to appuser within the container.

10.VOLUME: Enabling Access to Host Directory
The VOLUME command enables access from the container to a directory on the host machine. It facilitates data persistence and sharing between the container and the host.

Example:

VOLUME /data

This creates a volume mount at /data within the container, allowing access to a directory on the host machine.

11.WORKDIR: Setting Working Directory
The WORKDIR command sets the working directory within the container where subsequent commands are executed. It provides a convenient way to organize and manage the filesystem within the container.

Example:

WORKDIR /app

This sets the working directory to /app within the container.

12.LABEL: Adding Metadata
The LABEL command allows you to add metadata to your Docker image. It provides information about the image, such as version, description, or any other relevant details.

Example:

LABEL version="1.0" description="Sample Docker image for demonstration purposes"

This adds metadata to the Docker image, specifying the version and description.

Conclusion:
Dockerfile commands are the building blocks of efficient Docker image creation, enabling you to customize and streamline the containerization process. By mastering these commands and their practical applications, you empower yourself to create robust and reliable container environments tailored to your application's requirements. Whether you're deploying microservices, monolithic applications, or anything in between, understanding Dockerfile commands is essential for maximizing the potential of Docker and containerization technologies.

Exploring Apache Kafka

Syed Aun Abbas — Wed, 13 Mar 2024 23:15:59 +0000

Introduction

Embarking on the exploration of microservices inevitably leads us into the realm of various concepts, patterns, and tools. Among these, Apache Kafka stands out as a distributed streaming platform, often mistakenly pigeonholed as a mere messaging system. However, its intricacies and capabilities extend far beyond traditional messaging paradigms, making it a crucial component in modern data processing architectures.

Unveiling Apache Kafka

Apache Kafka is not just another messaging system; it is a distributed streaming platform designed to handle real-time data streams seamlessly across a cluster of machines. At its core, Kafka facilitates the processing of infinite data streams, distinguishing itself with its distributed architecture, scalability, and fault tolerance.

Demystifying Messaging

Before diving into Kafka's architecture, it's essential to grasp the fundamentals of messaging. Messaging involves producers generating messages, queues acting as buffers for message delivery, and consumers subscribing to queues to receive messages. However, unlike traditional messaging systems, Kafka introduces the concept of streams, enabling real-time data processing and distributed computing.

Deciphering Kafka's Architecture

Central to Kafka's architecture are topics, which serve as the conduits for data streams. Topics consist of partitions, with each partition distributed across brokers within a cluster. The replication factor ensures data durability by replicating partitions across multiple brokers. Additionally, Kafka employs a partition leader to manage data distribution and failover, ensuring seamless operation even in the face of broker failures.

Producers: The Catalyst of Data Streams

Producers play a vital role in Kafka's ecosystem by generating and sending messages to topics. Unlike traditional messaging systems, Kafka employs partitioning to distribute messages across partitions efficiently. Producers can specify message keys, allowing for deterministic message routing and enabling ordered processing within partitions.

Consumers and Consumer Groups

Consumers subscribe to topics to consume messages generated by producers. By leveraging consumer groups, Kafka enables scalable and fault-tolerant message consumption. Consumer groups facilitate load balancing, ensuring that each message is processed efficiently across multiple consumers within the group.

Unlocking Kafka's Potential

Apache Kafka's distributed architecture, coupled with its real-time processing capabilities, makes it indispensable in various use cases, including event-driven architectures, real-time analytics, and data integration pipelines. By understanding Kafka's core principles and features, organizations can leverage its full potential to build robust, scalable, and resilient data processing systems.

Conclusion

In conclusion, Apache Kafka transcends the boundaries of traditional messaging systems, offering a comprehensive solution for real-time data processing and distributed computing. As organizations navigate the complexities of modern data architectures, Kafka emerges as a cornerstone, empowering them to harness the power of real-time data streams for innovation and growth. Through continuous exploration and understanding, we can unlock Kafka's full potential and drive forward the evolution of data processing technologies.

The Git Guide: Understanding Core Concepts

Syed Aun Abbas — Mon, 19 Feb 2024 03:46:44 +0000

Introduction

In the world of software development and version control, Git stands out as one of the most powerful and widely used tools. Understanding Git and its core concepts is essential for any developer looking to efficiently manage their projects and collaborate with others. In this comprehensive guide, we'll explore the fundamentals of Git, including its purpose, workflow, and essential commands, while also delving into practical examples and best practices.

What is Git?

At its core, Git is a distributed version control system designed to track changes to files over time. Whether you're managing software code, documentation, or any other type of text-based files, Git enables you to record snapshots of these files, facilitating collaboration and ensuring project integrity.

Core Concepts

Working Tree
The working tree represents the visible state of the project on the filesystem. It's where developers interact with files, making changes such as additions, deletions, and edits. Every modification made in the working tree reflects the current state of the project.
Staging Area
Also known as the index, the staging area acts as an intermediary between the working tree and the git history. It allows developers to curate changes before committing them to the repository. By adding specific files or modifications to the staging area, developers gain fine-grained control over which changes are included in the next commit.
History
The git history encompasses the entire record of commits and project evolution. It's stored in a hidden directory named .git, which contains an object database and metadata. This history, represented graphically as a commit graph, preserves the chronological sequence of snapshots of the project at different points in time. Sharing the .git directory grants access to the complete project history, enabling collaboration and version control across different environments.

Workflow
In the Git workflow, developers make changes in the working tree, staging selected modifications in the staging area, and ultimately committing these changes to the git history. This workflow provides flexibility and control, allowing developers to manage project versions effectively.

Core Commands

Git Init: Initializing Repository
git init is used to initialize a new Git repository in a directory. When you run git init in a directory, Git creates a new subdirectory named .git inside that directory. This .git directory contains all the necessary files and subdirectories that Git needs to manage the repository.

Git Status: Assessing the State of Your Repository
When you're working in a Git repository, it's essential to know the status of your files—what's been modified, what's staged for commit, and what's untracked. This is where the git status command comes in handy. Let's walk through a scenario to illustrate its usage.

Imagine you've just created a new file named S1 in your project directory. At this point, Git considers S1 as an "untracked" file since it's new and hasn't been added to the repository yet. Running git status will give you an overview of the current state of your working tree and staging area. You'll see that S1 is listed as an untracked file, prompting you to take action.

Git Add: Staging Changes for Commit
To start tracking changes to S1, you need to add it to the staging area using the git add command. This action signals to Git that you want to include S1 in the next commit. Executing git add S1 moves the file from the untracked state to the staging area, preparing it for commitment to the repository.

Upon running git status again after staging S1, you'll notice a change in the output. Git now informs you that there are "changes to be committed," specifically mentioning the addition of S1 to the staging area. Additionally, git status no longer lists S1 as an untracked file since it's now being tracked by Git.

Git commit- Committing to the git history
After staging your changes using git add, the next step is to create a commit—a snapshot of the current state of your project. This is accomplished with the git commit command. Let's proceed with committing our newly added file, S1, to the repository.

Executing git commit -m "add file S1" initiates the commit process and adds the file to the git history. Here's what happens behind the scenes:

Creating a Commit: Git takes all the changes currently staged in the staging area and packages them into a commit. In our case, since we've only added S1 to the staging area, the commit will include this single file.

Adding a Commit Message: The -m option allows us to provide a concise message that describes the changes being made in this commit. It's essential to craft meaningful commit messages that convey the purpose of the changes, aiding in understanding the history of the project.

Git Reset -- files
The git reset -- files command is primarily used to unstage specific files from the staging area in Git. When you stage changes using git add, you're preparing those changes to be included in the next commit. However, if you accidentally add files or changes that you don't want to commit, you can use git reset -- files to undo the staging of those specific files.

Git Revert: Undoing changes
git revert is a command used in Git to undo changes made to a repository by creating a new commit that represents the inverse of the specified commit or commits. It's a safer alternative to commands like git reset, which can alter history in a way that's potentially destructive, especially if the changes have been shared with others.

Git Checkout -- files
The git checkout -- files command is used to discard changes made to specific files in the working directory and replace them with the version of the file from the staging area (index). It effectively reverts the specified files to the state they were in at the time of the last commit or the state they were in when they were last staged.

Git Log-Display Commit History
The git log command is used to display the commit history of a repository. When you run git log in your terminal or command prompt within a Git repository, Git retrieves and presents a chronological list of commits, starting from the most recent to the oldest.

Git diff- Display differences between 2 states of repository
The git diff command is used to display the differences between two states of the repository. These states could be between the working tree and the staging area (index), or between the staging area and the most recent commit.

Git rm-Removing files
To remove a file from Git, you typically use the git rm command followed by the filename. Here's how you can remove a file:
git rm filename
If you just want to remove the file from the Git repository but keep it in your local filesystem, you can use the --cached option:
git rm --cached filename
After running either of these commands, make sure to commit your changes to finalize the removal

.gitignore
The .gitignore file is a text file used by Git to specify intentionally untracked files that Git should ignore. These are typically files that are generated as a part of your build process or are specific to your development environment and don't need to be tracked by Git. By adding file patterns to the .gitignore file, you can tell Git not to consider those files when determining which files to track or stage for commits.

Branching

Git branching is a fundamental aspect of version control, empowering developers to work on different features, experiments, or bug fixes concurrently without interfering with the main codebase. In this guide, we'll delve into the core commands for branching in Git, providing practical examples to solidify your understanding.

Creating a New Branch:
To create a new branch in Git, you can use the git branch command followed by the desired branch name. For instance:
git branch feature-xyz
This command creates a new branch named feature-xyz based on the current state of your repository.

Switching to a Branch:
Once you've created a branch, you'll often need to switch to it to start working. Use the git checkout command followed by the branch name:
git checkout feature-xyz
This command switches your working directory to the feature-xyz branch, allowing you to make changes specific to that feature.

Creating and Switching to a Branch (Shortcut):
Git offers a convenient shortcut to create a new branch and immediately switch to it:
git checkout -b feature-xyz
This single command creates a new branch named feature-xyz and switches your working directory to it, streamlining your workflow.

Listing All Branches:
To view all branches in your repository, use the git branch command without any additional parameters:
git branch
This command lists all local branches, highlighting the current branch with an asterisk (*).

Merging Branches:
Merging combines the changes from one branch into another. To merge a branch into your current branch, use the git merge command:
git checkout master
git merge feature-xyz
In this example, we switch to the master branch and merge changes from the feature-xyz branch into it.

Deleting a Branch:
After completing work on a branch, you can delete it using the -d flag with the git branch command:
git branch -d feature-xyz
This command deletes the feature-xyz branch. Use with caution, as it's irreversible.

Force Deleting a Branch:
In some cases, Git may prevent branch deletion due to unmerged changes. To force delete a branch, use the -D flag:
git branch -D feature-xyz
Exercise caution when force deleting branches, as it can result in data loss.

Renaming a Branch:
To rename a branch, use the git branch -m command followed by the old and new branch names:
git branch -m feature-xyz new-feature-name
This command renames the feature-xyz branch to new-feature-name.

Conclusion

In this comprehensive guide, we've covered the essential concepts and commands of Git, from its core principles to practical usage scenarios. By understanding these fundamentals, developers can effectively manage their projects, collaborate with others, and navigate the complexities of version control with confidence. Whether you're just starting with Git or looking to deepen your expertise, mastering these concepts and commands will undoubtedly enhance your productivity and contribute to your success as a software developer. Happy coding!

Why Webhooks Matter

Syed Aun Abbas — Sat, 27 Jan 2024 01:05:54 +0000

Introduction

In the ever-evolving landscape of technology, where innovation is the heartbeat of progress, certain concepts play a pivotal role in shaping the architecture of systems. Today, we embark on a journey to explore one such concept that has woven itself into the fabric of Mach-based architectures – the intriguing world of webhooks. While not originally part of the Mac acronym, webhooks have become a cornerstone of modern-day technology.

Why Webhooks Matter

To truly grasp the significance of webhooks, let's delve into a common technological predicament – the incessant need for updates or status checks. Picture a car filled with enthusiastic children eagerly asking, "Are we there yet?" This persistent quest for information, whether from APIs or systems, is what we refer to as polling. It involves repeatedly seeking status updates, akin to incessantly refreshing a webpage to glean the latest information.
In response to the challenges posed by polling, webhooks emerge as a sophisticated alternative. At its core, a webhook allows developers to tap into a system's events and configure these events to trigger calls to other systems when specific occurrences take place. Rather than incessantly polling for updates, webhooks empower a system to push relevant data to another when a consequential event unfolds.

A Practical Example

Consider a scenario within the realm of e-commerce, where the journey of a product from warehouse to customer unfolds. The carrier system, responsible for tracking this journey, can utilize webhooks to seamlessly inform the e-commerce system of crucial events. Be it the arrival of a package at the distribution center, its journey through the delivery process, or the moment it's collected by the customer – each event triggers the webhook, sending a prompt message to the e-commerce platform's API. This real-time exchange enables instantaneous updates, order status modifications, customer communication, and even post-delivery review requests.

Why Webhooks are Essential

The paradigm shift from polling to webhooks mirrors the transition from pulling data from another system (polling) to pushing data precisely when necessary (webhooks). This transformative approach not only enhances operational efficiency but also mitigates unnecessary data duplication, ensuring that updates occur precisely when needed.

In the construction of intricate platforms or systems comprising various subsystems, preserving the independence of each system's domain is paramount. Webhooks play a pivotal role by facilitating nuanced adjustments across different systems when events unfold. This not only streamlines processes but also guarantees data integrity, significantly reducing the need for extensive business logic in the front end

Conclusion

In summary, webhooks emerge as the unsung heroes of modern technology, seamlessly connecting disparate systems and ensuring real-time updates without the need for constant polling. Understanding and implementing webhooks within Mach-based architectures not only boosts efficiency but lays the foundation for scalable, modular, and composable systems.

Type-Safe API Communication with tRPC

Syed Aun Abbas — Wed, 24 Jan 2024 23:58:06 +0000

In the ever-evolving landscape of web development, writing code that is not only efficient but also easy to maintain is a constant endeavor. One of the key aspects that developers grapple with is ensuring type safety, especially when it comes to the communication between a client and a server. In this blog post, we'll explore how tRPC (Typed RPC) emerges as a powerful tool to address this challenge, providing a seamless blend of familiarity and robust type safety.

The Challenge of Traditional API Communication

Traditional REST-based API communication, while widely adopted, often falls short in maintaining strong type safety. Developers face uncertainties when it comes to changes in route names, data structures, or response formats. The lack of immediate feedback can lead to runtime errors, making the debugging process more cumbersome.

Enter tRPC: Bridging the Gap Between REST and GraphQL

tRPC steps in as a bridge between traditional REST practices and the advanced type safety of GraphQL. It offers a fresh perspective on API communication, combining the best of both worlds.

Setting the Stage with a REST Example

Let's start by examining a scenario where a developer is working on a project using standard REST practices. An Express server is established, handling various routes.

import express from "express"
const router = express.Router()

router.get<{ names: string }>("/greetings", (req, res) => {
  res.send(`Hello ${req.query.names}`)
})

router.get<{ name: string }>("/error", (req, res) => {
  res.status(500).send("This is an error message")
})

export default router

In this scenario, a developer making changes to route names or data structures may face challenges in ensuring that the client-side code remains consistent with the server-side implementation. Type safety is not guaranteed, and errors might only surface at runtime. For example, if you change the endpoint to "/greetings" instead of "/greeting" and then attempt to request the "/greeting" endpoint from the client-side, you won't immediately see an error until you run the code. This is because the TypeScript types for the endpoint names are defined in the route handlers, and there is no compile-time checking to catch these mistakes.

When you run the code and make a request to "/greeting" (assuming the endpoint is defined as "/greetings"), you will likely encounter a runtime error, possibly resulting in a 404 Not Found response or some other unexpected behavior.

This is in contrast to using tRPC or a similar framework where the TypeScript types are generated based on your API definition. In such frameworks, if you attempt to make a request to a non-existent or misspelled endpoint, the TypeScript compiler will catch this error during the compilation process, providing early feedback and helping you avoid runtime issues.

Empowering Development with tRPC

Now, let's transition to a world where tRPC becomes a central player in API communication. The same scenario is reimagined, but this time with tRPC seamlessly integrated.

Server-Side Integration
On the server side, tRPC brings about a paradigm shift in route definition and input validation using Zod.

How to add tRPC to existing Express project

// tRPC Middleware for Express
import { createExpressMiddleware} from "@trpc/server/adapters/express"
import express from 'express'
import cors from "cors"
import {appRouter} from "./routers"

const app=express()
app.use(cors({origin:"http://localhost:5173"}))

app.use('/trpc', createExpressMiddleware({
  router:appRouter,
  createContext:({req,res})=>{ return {} },
}));
app.listen(3000)

Key features include:
tRPC Middleware: tRPC provides middleware for Express, making it easy to integrate with existing setups.

Type-Defined Routes: Routes are now explicitly defined, complete with input validation using Zod, a powerful TypeScript-first schema declaration library.

Setting up routes

// Importing necessary dependencies
import { t } from "../trpc";  // Assuming 't' is an object with utility functions for defining TRPC routers
import { z } from "zod";  // Assuming 'z' is an object with utility functions for defining schemas
import { usersRouter } from "./users";  // Importing another router from a different module

export const appRouter = t.router({
    greeting: t.procedure
     .input(z.object({ name: z.string() }))
     .query(requestObj => {
      console.log(requestObj);
      return `Hello ${requestObj.input.name}`;
    }),

    errors: t.procedure.query(() => {
      throw new Error("This is an error message");
    }),
    users: usersRouter,
});

export type AppRouter = typeof appRouter;

Inside the router definition, there are three properties:

greeting: A query that expects an input object with a name property of type string. The query logs the input object and returns a greeting message that includes the provided name.

errors: A query that intentionally throws an error with the message "This is an error message" when executed.

users: This property is assigned the value of another router (usersRouter) imported from a different module.

TypeScript's static type checking ensures that the code adheres to the specified types during development. This helps catch type-related errors at compile time, providing early feedback to developers.

Client-Side Integration
On the client side, the tRPC client takes center stage, providing developers with autocompletion, immediate feedback, and most importantly, type safety.

Now, let's say someone on the development team decides to change the name of the greeting query from "greeting" to "welcome":
Now, if the development team attempts to use the greeting query in the codebase after this change, TypeScript's static type checking will catch the error at compile time.

// tRPC Client Setup
import { createTRPCProxyClient, httpBatchLink } from "@trpc/client"
import type { AppRouter } from "../../server/routers"

const client = createTRPCProxyClient<AppRouter>({
  links: [
    httpBatchLink({
      url: "http://localhost:3000/trpc",
    }),
  ],
})

async function main() {
  const result = await client.greeting.query({ name: "Kyle" })
  console.log(result)
}

The code is now more intuitive and developer-friendly. The tRPC client allows developers to call API functions as if they were invoking regular functions. The TypeScript type system ensures that any inconsistencies, such as changes in route names or data structures, are immediately flagged.

Conclusion

tRPC emerges as a powerful ally in the quest for type-safe API communication. By seamlessly integrating with familiar tools and frameworks, tRPC brings a new level of confidence and efficiency to the development process.

In a world where the complexities of GraphQL meet the simplicity of REST, tRPC stands out as a beacon, offering developers a robust solution without compromising on ease of use. Embrace the power of tRPC and elevate your web development experience with enhanced type safety and streamlined API communication.

Typescript- What's the advantage?

Syed Aun Abbas — Mon, 22 Jan 2024 21:10:18 +0000

Introduction

In today's rapidly evolving world of web development, mastering TypeScript has become a valuable skill for JavaScript developers. In this blog, we'll explore the fundamental concepts of TypeScript. TypeScript brings numerous advantages, making you a more confident and efficient developer. Let's delve into the key concepts that make TypeScript a powerful tool for enhancing your coding experience.

Typescript vs Javascript

1. Static Typing
TypeScript introduces static typing, allowing developers to declare and enforce variable types. This feature catches type-related errors during development, providing early feedback and enhancing code reliability. This is especially beneficial in large codebases where maintaining consistency is crucial.

2. Code Readability and Documentation
By using explicit types and interfaces, TypeScript improves code readability and serves as self-documentation. Developers can easily understand the expected types of variables and function parameters, making the codebase more maintainable and facilitating collaboration.

3. Enhanced IDE Support
TypeScript enhances the capabilities of Integrated Development Environments (IDEs), particularly Visual Studio Code. With static typing information, IDEs can provide intelligent code completion, real-time error checking, and better navigation, resulting in a more efficient development workflow.

4. Early Detection of Bugs
The type-checking mechanism in TypeScript enables the early detection of potential bugs during the development phase. This proactive approach reduces the likelihood of runtime errors, contributing to more robust and stable applications.

5. Code Refactoring
TypeScript simplifies the process of code refactoring. With accurate type information, developers can confidently make changes to their codebase, knowing that the TypeScript compiler will highlight any inconsistencies or issues, allowing for safer and more straightforward refactoring.

6. Object-Oriented Programming Features
TypeScript supports object-oriented programming (OOP) features such as classes, interfaces, and inheritance. This makes it a more versatile choice for developers accustomed to OOP principles, enabling the creation of scalable and maintainable code.

7. Compatibility with JavaScript Ecosystem
TypeScript seamlessly integrates with existing JavaScript code and libraries. Developers can gradually adopt TypeScript into their projects, as any valid JavaScript code is also valid TypeScript. This flexibility makes it easier for teams to transition to TypeScript without a steep learning curve.

Implicit and Explicit Types

One of TypeScript's core features is its type system, designed to catch errors and enhance code reliability. TypeScript can infer types implicitly based on assigned values, reducing the chance of unexpected behavior.

// Implicit Type
let message = "Hello TypeScript"; // TypeScript infers 'message' as type string

// Explicit Type
let firstName: string = "John";
let age: number = 30;

Explicitly defining types adds clarity to your code and acts as a safety net against potential errors. TypeScript's ability to catch type-related bugs during development is a significant advantage.

Special Types: Tuple, Enum, and Interface

Tuple
Tuples in TypeScript enable the representation of an array with a fixed number of elements, each with a known but potentially different type.

// Tuple Example
let coordinates: [string, number] = ["Latitude", 40.7128];

Enum
Enums provide a convenient way to represent a set of named constants. They enhance code readability and reduce the likelihood of errors.

// Enum Example
enum Continents {
  Africa,
  Europe,
  Asia,
  NorthAmerica,
  SouthAmerica,
  Australia,
  Antarctica,
}
let chosenContinent: Continents = Continents.Africa;

Interface
Interfaces define the structure of objects, specifying the types of their properties. They promote code consistency and help catch errors early in the development process.

// Interface Example
interface User {
  name: string;
  id: number;
}

let newUser: User = { name: "John", id: 1 };

Conclusion

As we conclude our TypeScript exploration, we've touched upon the essential concepts that make TypeScript a powerful tool for JavaScript developers. The seamless integration, robust type system, and additional features contribute to a more efficient and reliable coding experience. Whether you're a seasoned developer or just starting, embracing TypeScript can elevate your skills and enhance your projects.

Zod: TypeScript-first schema validation library

Syed Aun Abbas — Sun, 21 Jan 2024 21:23:03 +0000

Introduction

Zod is a TypeScript-first schema declaration and validation library. In this blog post, we'll delve into the fascinating world of Zod, exploring its features, integration with TypeScript, and how it can revolutionize the way you approach data validation in your projects.

Why Zod?

Zod isn't just another validation library; it's a game-changer. Imagine a world where defining and validating your code is a breeze, and you only need to do it once. Zod brings this vision to life by being TypeScript-first, seamlessly integrating with TypeScript types. No more redundant type declarations – with Zod, your schema is your type, reducing redundancy and increasing efficiency.

Zod offers a number of advantages

Zero Dependencies: Zod prides itself on its lightweight nature. With zero external dependencies, it ensures a streamlined development process without unnecessary baggage. Weighing in at just 8kb when minified and zipped, Zod is a powerful tool that won't weigh down your projects.

Cross-Platform Compatibility: Whether you're working in Node.js or crafting cutting-edge applications for modern browsers, Zod has you covered. Its versatility extends across different environments, ensuring consistent and reliable validation wherever your code runs.

Immutable Architecture: Zod adopts an immutable approach, a design choice that pays off in terms of clarity and reliability. Methods such as .optional() return new instances, preserving the integrity of your data while allowing for seamless chaining.

Functional Approach: Zod challenges the conventional 'validate' mindset by promoting a functional approach – parse, don't validate. This shift in perspective simplifies your workflow and enhances the overall robustness of your validation process.

Plain JavaScript Compatibility: You don't need to be a TypeScript enthusiast to benefit from Zod's capabilities. While it integrates seamlessly with TypeScript, Zod is equally adept at working with plain JavaScript.

Basic Usage

Creating a simple string schema

import { z } from "zod";

// creating a schema for strings
const mySchema = z.string();

// parsing
mySchema.parse("tuna"); // => "tuna"
mySchema.parse(12); // => throws ZodError

// "safe" parsing (doesn't throw error if validation fails)
mySchema.safeParse("tuna"); // => { success: true; data: "tuna" }
mySchema.safeParse(12); // => { success: false; error: ZodError }

import { z } from "zod";
This line imports Zod library into your project

Creating a Schema for Strings:
const mySchema = z.string();
Here, you define a schema using Zod for validating strings. z.string() creates a schema that expects the data to be a string.

Parsing with parse method:
mySchema.parse("tuna"); // => "tuna"
mySchema.parse(12); // => throws ZodError
The parse method is used to validate and parse data according to the defined schema. The first line successfully parses the string "tuna" as it conforms to the schema. The second line attempts to parse the number 12, but it throws a ZodError since it doesn't match the expected string type.

"Safe" Parsing with safeParse method:
mySchema.safeParse("tuna"); // => { success: true; data: "tuna" } mySchema.safeParse(12); // => { success: false; error: ZodError }
The safeParse method is similar to parse, but it doesn't throw an error if the validation fails. Instead, it returns an object with a success property indicating whether the parsing was successful, along with the parsed data or an error object.

Creating an object schema

import { z } from "zod";

const User = z.object({
  username: z.string(),
});

User.parse({ username: "Ludwig" });

// extract the inferred type
type User = z.infer<typeof User>;
// { username: string }

const User = z.object({ username: z.string(), });
Here, you define a schema named User for an object using z.object(). The object schema specifies that it should have a property named username of type string.

Parsing an Object:
User.parse({ username: "Ludwig" });The parse method is used to validate and parse an object according to the defined schema. In this case, it successfully parses an object with the property username set to the string "Ludwig".

Extracting the Inferred Type:
type User = z.infer<typeof User>;
After defining the schema, you use the z.infer utility to extract the TypeScript type inferred from the Zod schema. This results in a TypeScript type User that represents the expected structure of the object. In this case, the User type is inferred as { username: string }.
Using z.infer in Zod simplifies TypeScript integration by automatically deriving TypeScript types from your Zod schema. This single-source approach ensures consistency, reduces redundancy, and adapts types dynamically to changes in your validation rules.

Primitives

import { z } from "zod";

// primitive values
z.string();
z.number();
z.bigint();
z.boolean();
z.date();
z.symbol();

// empty types
z.undefined();
z.null();
z.void(); // accepts undefined

// catch-all types
// allows any value
z.any();
z.unknown();

// never type
// allows no values
z.never();

Coercion for primitives

Zod has introduced a convenient way to coerce primitive values with the coerce function. This allows for seamless conversion of various primitive types during the parsing step. Here's a breakdown and summary:

Coercion Example:
const schema = z.coerce.string(); schema.parse("tuna"); // => "tuna" schema.parse(12); // => "12" schema.parse(true); // => "true"
With z.coerce.string(), the input values are coerced into strings during the parsing process. The String() function, a JavaScript built-in for string coercion, is applied. The returned schema is a ZodString instance, enabling the use of all string methods.

Chained Coercion and Validation:
z.coerce.string().email().min(5);
Chaining is supported, allowing you to perform additional validations after coercion. In this example, the schema coerces to a string, validates the value as an email, and then checks for a minimum length of 5 characters.

Supported Coercions:
z.coerce.string(); // String(input) z.coerce.number(); // Number(input) z.coerce.boolean(); // Boolean(input) z.coerce.bigint(); // BigInt(input) z.coerce.date(); // new Date(input)

Strings

Zod includes a handful of string-specific validations.

// validations z.string().max(5); z.string().min(5); z.string().length(5); z.string().email(); z.string().url(); z.string().emoji(); z.string().uuid(); z.string().cuid(); z.string().cuid2(); z.string().ulid(); z.string().regex(regex); z.string().includes(string); z.string().startsWith(string); z.string().endsWith(string); z.string().datetime(); // ISO 8601; default is without UTC offset, see below for options z.string().ip(); // defaults to IPv4 and IPv6, see below for options

// transformations
z.string().trim(); // trim whitespace z.string().toLowerCase(); // toLowerCase z.string().toUpperCase(); // toUpperCase

You can customize some common error messages when creating a string schema.

const name = z.string({ required_error: "Name is required", invalid_type_error: "Name must be a string", });
When using validation methods, you can pass in an additional argument to provide a custom error message.

z.string().min(5, { message: "Must be 5 or more characters long" }); z.string().max(5, { message: "Must be 5 or fewer characters long" }); z.string().length(5, { message: "Must be exactly 5 characters long" }); z.string().email({ message: "Invalid email address" }); z.string().url({ message: "Invalid url" }); z.string().emoji({ message: "Contains non-emoji characters" }); z.string().uuid({ message: "Invalid UUID" }); z.string().includes("tuna", { message: "Must include tuna" }); z.string().startsWith("https://", { message: "Must provide secure URL" }); z.string().endsWith(".com", { message: "Only .com domains allowed" }); z.string().datetime({ message: "Invalid datetime string! Must be UTC." }); z.string().ip({ message: "Invalid IP address" });

Conclusion

In a nutshell, Zod empowers developers to focus on building robust applications by providing a clear and efficient solution to data validation challenges. Its elegant design, TypeScript integration, and continuous evolution make it a valuable tool for simplifying validation logic and improving overall code quality. Consider integrating Zod into your projects, and experience a paradigm shift in how you approach schema validation in your applications.

Creating a Github workflow using Github Actions

Syed Aun Abbas — Sat, 20 Jan 2024 21:44:04 +0000

Automatic Deployment with GitHub Actions is a powerful and streamlined approach that enables developers to automate the deployment process of their applications directly from their GitHub repositories. GitHub Actions is a continuous integration and continuous deployment (CI/CD) platform provided by GitHub, allowing you to define workflows using YAML files.

In the context of automatic deployment, GitHub Actions can be configured to trigger specific tasks, such as building, testing, and deploying your application, whenever changes are made to the repository. This automation not only saves time but also ensures consistency and reliability in the deployment process.

Developers can leverage various predefined actions and customize workflows to meet the specific requirements of their projects. With GitHub Actions, you can seamlessly integrate deployment processes into your development workflow, facilitating faster and more efficient software delivery. Whether deploying to cloud services, hosting platforms, or custom servers, GitHub Actions provides a versatile and scalable solution for automating the deployment lifecycle.

In this tutorial, I'll guide you on how to setup a Github workflow using Github Actions. NOTE: This is for React applications on Windows.

Setting up the workflow

Initialize a Repository
Start by creating a repository on GitHub and pushing your project to this newly created repository.

Setting up the workflow file
Search for the Node.js workflow in the Actions tab in your Github repo. The specific workflow is shown below

Once you click configure. you will be redirected to a YAML file that you can customize. Some changes are made to the YAML file as shown below:

name: Node.js CI

on: #this tells Github when to trigger the job
  push: #will trigger whenever we push to the main branch
    branches: [ "main" ]
#pull request was removed from here because we will not be dealing with it


jobs: 
#this job will be run every single time this workflow file is triggered
  build:
    runs-on: self-hosted #we can specify where it runs

    strategy:
      matrix: 
        node-version: [18.x] #You can specify which versions you wanna test against

    steps: #Script for all the steps we want to execute
    - uses: actions/checkout@v3
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'npm'
    - run: npm i
    - run: npm run build --if-present
    - run: npm test

I've commented the file above for more clarity on what each statement is doing. After this, commit the file to the main branch.
Now we will set a runner on our app that is listening to jobs

Setting the runner
On your Github repository, navigate to 'Settings' > 'Actions' > 'Runners'. There you will see 'New self-hosted runner' on the top right- click this and configure the settings according to Windows. Now open Windows Powershell as an administrator, then on your powershell, run mkdir actions-runner; cd actions-runner then cd actions-runner then type and enter the command Set-ExecutionPolicy -ExecutionPolicy RemoteSigned. Then under 'Download" and "Configure" in your Settings>Actions tab, run the scripts on your Windows Powershell in order exactly as shown.
You can skip the first command i.e mkdir actions-runner; cd actions-runner as we're already in that folder.

Press enter for any default settings where the option is shown on your Powershell. After that, your Powershell should look something like this. It is now listening for any jobs triggered.

Now go back to Github Actions and rerun all the jobs. It should run successfully and pass all the tests. This will be the output

Your powershell should also indicate that the job has succeeded.

This means you have successfully setup the Github Actions Runner and created a workflow.

Introduction to Redis

Syed Aun Abbas — Fri, 19 Jan 2024 20:10:34 +0000

Overview

Redis is an in-memory data structure store that goes beyond the conventional role of a database, extending its capabilities to serve as a cache and a message broker. It is designed to handle data with unparalleled speed and efficiency, having the capability to perform 110,000 SETs and 81000 GETs per second.

As a NoSQL key-value database, Redis simplifies data access through a single or set of keys to retrieve or delete associated values. In contrast to traditional relational NoSQL databases, Redis uses key-value pairs, where keys can be simple strings, serving as unique identifiers tied to specific data locations.

Redis provides a rich array of data structures such as strings, hashes, lists, sets, sorted sets, bitmaps, hyperloglogs, or geospatial indexes. This versatility empowers developers to mold their data storage according to specific application requirements, allowing for seamless integration with a wide range of use cases.

Redis Use Cases

Session Data
Redis serves as a session store for sharing session data among stateless servers in a web application. When a user logs in, session data, including a unique session ID, is stored in Redis. The session ID is returned to the client as a cookie. During subsequent requests, the client includes the session ID, allowing stateless web servers to retrieve the session data from Redis. It's crucial to note that Redis is an in-memory database, and session data stored in Redis will be lost if the Redis server restarts.

Replication
In addition to persistence options like snapshots and AOF, Redis uses replication for session data backup. While options like snapshots and AOF may be slow to load on restart, replication involves duplicating data to a backup instance. In the event of a main instance crash, the backup is swiftly promoted to handle traffic, ensuring quick recovery in production scenarios.

Distributed Log
Redis can be utilized as a distributed lock for coordinating access to shared resources among multiple nodes in an application. Clients attempt to acquire the lock by setting a key with a unique value and a timeout. If successful, the lock is acquired; otherwise, the client retries until the lock is released. While a basic implementation may lack full fault tolerance, various Redis client libraries offer high-quality distributed lock implementations out of the box for production use.

Ray Limiter
Redis can function as a rate limiter by utilizing its increment command on counters with set expiration times. In a basic rate-limiting algorithm, each incoming request uses the request IP or user ID as a key, and the request count is incremented using Redis's INCR command. Requests exceeding the rate limit are rejected. Keys expire after a specified time window, resetting counts for the next window. Redis supports more advanced rate-limiting algorithms, such as the leaky bucket algorithm, offering flexibility in implementing sophisticated rate limiters.

Gaming Leaderboard
For gaming leaderboards, especially in games of moderate scale, Redis is a preferred choice. The implementation relies on Sorted Sets, a fundamental Redis data structure. Sorted Sets consist of unique elements, each with an associated score, and they are sorted by score. This structure enables efficient retrieval of elements by score in logarithmic time, making Redis a delightful solution for various types of gaming leaderboards.

Getting hands-on with Redis

You can connect to Redis in the following ways:

With the redis-cli command line tool
Use RedisInsight as a graphical user interface
Via a client library for your programming language

Let's go over some of the basic commands in Redis to familiarize ourselves with how Redis works

Set and get a key-value pair
SET foo 100
This sets a key value-pair in the database with foo as the key and 100 as the value
To get this value, we can simply use
GET foo
This will return the value 100

Check if a key exists
We can use the EXISTS command to check if a key exists in the database or not. It will return 1 if the key exists and 0 otherwise EXISTS foo will return 1 whereas EXISTS foo1 will return 0

Deleting a key value pair
DEL command deletes the key from the database. DEL foo will return 1 indicating foo has been deleted. EXISTS foo will now return 0

Setting multiple key value pairs
The MSET command in Redis allows you to set multiple key-value pairs in a single command.
MSET key1 "value1" key2 "value2"
In this example, the MSET command sets the values for two keys (key1, key2) in a single go. After executing this command, key1 will have the value "value1" and key2 will have the value "value2"

Appending to a string
The APPEND command allows you to append a value to an existing string. For instance, say you have a key value pair as bar: "Hello" so APPEND bar " World" will result in the value of 'bar' becoming "Hello World".

List operations
Redis supports lists. You can use the LPUSH and RPUSH commands to push elements to the left and right of a list, respectively.
LPUSH myList "value1"
LPUSH myList "value2"
LPUSH myList "value3"
After executing these commands, the myList will contain the values in the following order:
"value3" "value2" "value1"
To retrieve all elements from the list:
LRANGE myList 0 -1
The LRANGE command with the range 0 -1 will return all elements of the list.

Set operations
Redis supports sets. Sets can be manipulated using commands like SADD to add elements and SMEMBERS to retrieve all elements of a set.
SADD mySet "value1"
SADD mySet "value2"
SADD mySet "value3"
After executing these commands, the mySet will contain the elements: "value1" "value2" "value3"
To retrieve all elements from the set:
SMEMBERS mySet
The SMEMBERS command will return all elements of the set.

Sorted Sets
Redis also supports Sorted Sets, a data structure that extends the functionality of sets by associating each element with a score. Commands like ZADD are used to add elements to a Sorted Set along with their scores. For example:
ZADD mySortedSet 1 "value1"
ZADD mySortedSet 2 "value2"
ZADD mySortedSet 3 "value3"
After executing these commands, the mySortedSet will contain elements with associated scores: "value1" (score 1), "value2" (score 2), and "value3" (score 3).
To retrieve elements from a Sorted Set based on their scores, you can use commands like ZRANGE or ZREVRANGE:
ZRANGE mySortedSet 0 -1
This command will return all elements of the Sorted Set in ascending order of their scores. You can adjust the range parameters for more specific retrievals.
Sorted Sets in Redis provide a way to maintain an ordered collection of unique elements, allowing for various operations based on element scores.

These examples cover a range of basic Redis commands, showcasing its versatility in handling different data structures and operations.

Redis, with its speed, versatility, and simplicity, invites you to explore the vast possibilities it unfolds. Whether you're building a real-time application, optimizing data access, or architecting a scalable solution, Redis stands ready to elevate your data management experience.