DEV Community: Neha Sharma

How to Improve JavaScript Code's Performance

Neha Sharma — Fri, 19 Dec 2025 07:56:11 +0000

High-performance JavaScript is about predictability. Write code that is boring for humans but delightful for the JIT compiler.

Performance is not magic. It's mechanical sympathy.

JavaScript performance optimization is not about micro-tweaks or premature cleverness. It's about understanding how the JavaScript engine, memory model, and runtime environment behave under pressure. This article focuses on high-impact optimizations grounded in engine internals, execution patterns, and production-grade scenarios.

1 . Understand the JavaScript Execution Model

Modern JavaScript engines (V8, SpiderMonkey, JavaScriptCore) rely on Just-In-Time (JIT) compilation, Inline caching, Hidden classes / shapes ,Speculative optimizations

Avoid De-Optimization Triggers
Engines optimize functions based on observed types. Changing types mid-execution forces de-optimization.


// Avoid this
function sum(a, b) {
  return a + b;
}

sum(1, 2);       // optimized for numbers
sum("1", "2");   // de-optimizes

// Do this
function sum(a, b) {
  a = Number(a);
  b = Number(b);
  return a + b;
}

2 . Shape Stability and Object Access

JavaScript objects use hidden classes. Mutating object structure dynamically prevents property access optimization. When iterating over large datasets, stable shapes can yield 20–50% performance improvements.

// Avoid this
const user = {};
user.name = "Alice";
user.age = 30;
user.isAdmin = true;

// Do this
const user = {
  name: "Alice",
  age: 30,
  isAdmin: true
};

3 . Minimize Garbage Collection Pressure

Garbage Collection (GC) pauses execution. High allocation rates cause frequent GC cycles. Avoid Unnecessary Allocations in Hot Paths

// Excessive Allocation
function process(items) {
  return items.map(item => ({
    id: item.id,
    value: item.value * 2
  }));
}

// Object Reuse
function process(items) {
  const result = new Array(items.length);
  for (let i = 0; i < items.length; i++) {
    const item = items[i];
    result[i] = item.value * 2;
  }
  return result;
}

4 . Prefer Iterative Loops Over Abstractions in Hot Code

Functional methods (map, reduce, filter) are expressive-but slower in critical paths.

// Avoid this
const total = prices
  .filter(p => p > 10)
  .map(p => p * 1.2)
  .reduce((a, b) => a + b, 0);

// Optimized Loop
let total = 0;
for (let i = 0; i < prices.length; i++) {
  const p = prices[i];
  if (p > 10) {
    total += p * 1.2;
  }
}

5 . Async Performance: Avoid Accidental Serialization

async/await can introduce hidden performance bottlenecks when misused. This change alone can reduce execution time from O(n × latency) to O(max latency).

// Serialized Execution
async function fetchAll(urls) {
  const results = [];
  for (const url of urls) {
    results.push(await fetch(url));
  }
  return results;
}

// Parallel Execution
async function fetchAll(urls) {
  return Promise.all(urls.map(fetch));
}

6 . Memoization and Cache Invalidation

Avoid recomputation for deterministic, expensive functions. Memoization must be paired with:

Cache size limits
Clear invalidation rules

Otherwise, memory leaks will offset performance gains

const cache = new Map();

function expensiveCalc(n) {
  if (cache.has(n)) return cache.get(n);

  let result = 0;
  for (let i = 0; i < n * 1e6; i++) {
    result += Math.sqrt(i);
  }

  cache.set(n, result);
  return result;
}

7 . DOM and Browser Performance: Batch Everything

DOM access is orders of magnitude slower than pure JS.

// Avoid this (Layout Thrashing)
elements.forEach(el => {
  el.style.width = el.offsetWidth + 10 + "px";
});

// Do this ; This prevents forced synchronous reflows.
const widths = elements.map(el => el.offsetWidth);

elements.forEach((el, i) => {
  el.style.width = widths[i] + 10 + "px";
});

8 . Measure, Don't Guess

There are tools, and API available to measure performance. These will give engineer context what to fix. Optimization without measurement is cargo cult engineering .Use:

performance.now()
Chrome DevTools Performance tab
Node.js --prof
Flamegraphs

const start = performance.now();
// critical code
const end = performance.now();
console.log(`Execution: ${end - start}ms`);

Your code either cooperates with these systems or actively fights them. Stable types, stable shapes, controlled memory allocation, and intentional async behavior allow engines to optimize aggressively.

If you liked this then you will like my other writings too. If you found something to be fixed/added then let me know in comments. Don’t be a stranger and say hi — Linkedin , X and NehaSharma.dev.

I Passed the AWS AI Practitioner Exam!

Neha Sharma — Mon, 08 Dec 2025 19:06:54 +0000

Yay! I passed the AWS AI Practitioner exam.

Every year I set a few personal goals and try my best to hit them. For 2025, one of my goals was to get 2–3 certifications. That didn’t quite happen (hello, procrastination 👀), so before 2025 ends I was determined to at least get one done.

With all the AI buzz around me, the AWS GenAI Practitioner felt like the perfect place to start.

Preparing for the Exam

I started with the official website of AWS certification to understand exam’s outline, areas of topics with percentage to focus on.

Once I knew what to expect, I used the following study materials:

Freecodecamp YouTube Video by Andrew Brown watched 2 times
Udemy mock exam from Stéphane Maarek
ChatGPT (my beloved teacher)
Hands-on (optional)
Internal office videos

How I Studied With a Full-Time Job (Without Losing Sleep)

I already had some background in AI, LLMs, and transformer architecture, which definitely worked in my favour.

The most important thing I did was — I booked the exam date. (In Amazon we have the leadership principle of “working backwards”); this approach has always worked great for me. Once the date was locked in, I had a clear deadline, a stronger reason to focus, and a deadline to finish my content.

My Daily Study Routine

I dedicated 30 mins to 1 hour daily to watch the YouTube Video. Spent 1–2 weeks to digest the concepts (I was not able to digest the content 100% ). So, I moved to taking mock exams.

First Mock Test (Reality Check)

I took my first Udemy mock test and scored 60% which meant I failed. Instead of getting discouraged, I reviewd all the wrong questions, I used chatGPT, or youtube to get my concepts clear. I started passing the mock exam (good sign). This was my routine.

A few weeks ago before exam date, I got access to internal videos on GenAI. These videos helped me in revising the all concepts and I made lot of notes.

The day before the exam, I took the 4th Udemy mock test (which I had intentionally saved for the end), just to see where I stood and I passed that too. For extra confidence, I even did one more random mock after that.

Timelines

Sep : started but was not consistent
Oct and Nov :I try to be consistent with my study. I went on 2 weeks travel and then fall sick for a week. So I basically lost around 3 weeks in between
Exam Scheduled for 7th Dec
Study 30 mins to 1 hr daily

Exam Day

Initially, I booked my exam for 7th December, but due to travel I brought it forward by a week and took it from home 29th Nov.

There were 65 questions and mix of mult-choice, and match the right options. For me time management is not any issue. So, I was not concerned about the time.

tip: please login 30 mins before. They put you in the queue to scan your docs, and rooms.

Exam was straight forward. Questions were aligned to Udemy mock exams. No surprise there at all.

Now what?

Celebration and relaxation for now. Will start preparing for a new certificate soon.

Have questions? or just want to say? You can connect with me at X or Linkedln

Object-Oriented JavaScript: A Practical Guide with Examples

Neha Sharma — Sat, 20 Sep 2025 20:41:00 +0000

If you’ve been programming for a decade or you’re just getting started, you’ve probably heard of OOP (Object-Oriented Programming). It’s a powerful paradigm that support many modern programming languages, from Java to C#.

But when it comes to JavaScript, things are interesting. JavaScript wasn’t originally built with classes in mind—it was designed as a lightweight scripting language for browsers. Over time, as web applications became more complex, JavaScript evolved. Today, with ES6 and beyond, it supports OOP-like patterns while still being fundamentally prototype-based under the hood.

In this blog, we’ll explore:

1 . What OOP is

2 . The core concepts of OOP

3 . How OOP is implemented in JavaScript

4 . A real-world example you probably already use

What is OOP?

Object-Oriented Programming is a paradigm that organizes code around objects. An object represents a real-world entity (like a user, product, or order) and bundles both data (properties) and behavior (methods) together.

In programming, we treat every thing :

As a JavaScript developer, you can think of an object as a collection of related data and behavior.

An object is made up of:

1 . Properties (data): key–value pairs (like variables inside the object).

2 . Methods (behavior): functions stored as properties, which define what the object can do.

In simple terms: A property describes what it has and a method describes what it does.

Benefits of OOP:

1 . Reusability: Write once, use across projects.

2 . Encapsulation: Hide implementation details, expose only what matters.

3 . Maintainability: Easier to update and extend.

4 . Abstraction: Focus on “what” an object does, not “how” it does it.

5 . Polymorphism: Same method name, different behavior depending on context.

Concpets in OOP

1 . Object

2 . Class & constructors

3 . Inheritance

4 . Polymorphism

5 . Abstraction

6 . Encapsulation

JavaScript and OOPs concept

In Modern JavaScript, we can implement OOP concepts. As an engineer, you should have the goal of using OOP for various benefits. Eg: for reusability, encapsulation, inheritance, and so on. Remember these are the programming goals you should as an progarmmer you should have goals.

Do we need to implement all these concepts? No..but as needed.

1. Object

Objects are the foundation of OOP. In JavaScript, before modern JavaScript we used to refer JavaScript as prototypal scripting language. As everything used to be done through prototype. In JavaScript, we make an object like this. Now, from JS pov, why this is important. As it helps in have a strucuture your code , data, properties and methods.

// creating an object in JavaScript

const car = {
  brand: "Tesla", // data
  model: "Model 3", // data 
  drive() { // method 
    console.log(`${this.brand} ${this.model} is driving...`);
  }
};

car.drive(); // Tesla Model 3 is driving...

The “Car” example explains OOP well, but it doesn’t feel very connected to everyday JavaScript work. So let’s model something practical—a carousel. In JavaScript, we can represent a carousel as an object.

// Creating a Carousel using object

const Carousel = {
  images: ["image1.jpg", "image2.jpg", "image3.jpg"], 
  style: "default",                                   

  createCarousel() {                                  
    console.log("Carousel created with images:", this.images);
  },

  nextImage(index) {
    console.log("Showing:", this.images[index % this.images.length]);
  }
};

// Using the object
Carousel.createCarousel();

2 . Classes & Constructors

Before ES6, JavaScript didn’t have the class or constructor keywords.

Developers who came from languages like Java or PHP often mimicked these concepts using functions and prototypes.

With ES6, JavaScript introduced the class syntax.

It doesn’t add new functionality under the hood — classes are syntactic sugar over prototypes — but it makes object-oriented code more readable and familiar.

The constructor is a special method that runs automatically when you create a new instance of a class.

Class and constructors form the foundation for the rest of the OOP concepts we’ll cover — from inheritance to abstraction.

class Car {
  constructor(brand, model) {
    this.brand = brand;
    this.model = model;
  }

  drive() {
    console.log(`${this.brand} ${this.model} is driving...`);
  }
}

const myCar = new Car("Tesla", "Model Y");
myCar.drive(); // Tesla Model Y is driving...

Here, Car is a class, and the constructor sets up its properties. When we call new Car("Tesla", "Model Y"), the constructor runs automatically.

class Carousel {
  constructor(items = []) {
    this.items = items;
    this.currentIndex = 0;
  }

  showCurrent() {
    console.log(`Showing: ${this.items[this.currentIndex]}`);
  }

  next() {
    this.currentIndex = (this.currentIndex + 1) % this.items.length;
    this.showCurrent();
  }

  prev() {
    this.currentIndex =
      (this.currentIndex - 1 + this.items.length) % this.items.length;
    this.showCurrent();
  }

  goTo(index) {
    if (index >= 0 && index < this.items.length) {
      this.currentIndex = index;
      this.showCurrent();
    } else {
      console.log("Invalid index");
    }
  }
}

const carousel = new Carousel(["img1.jpg", "img2.jpg", "img3.jpg"]);
carousel.showCurrent(); 
carousel.next();        
carousel.goTo(2);

Here, Carousel organizes state (items, currentIndex) and exposes methods to control the slideshow. The constructor ensures every new carousel starts in a valid state.

Classes and constructors in ES6 make object-oriented JavaScript cleaner and more intuitive, while still being powered by the prototype system under the hood.

3 . Inheritance (Prototypes)

Inheritance allows a child class to reuse and extend functionality from a parent class. It supports the DRY (Don’t Repeat Yourself) principle by letting you share code instead of duplicating it.

In JavaScript, inheritance was originally implemented using prototypes. Since ES6, we have the class syntax, which makes inheritance much more intuitive though under the hood, it still uses prototypes.

class Vehicle {
  constructor(type) {
    this.type = type;
  }
  start() {
    console.log(`${this.type} is starting...`);
  }
}

class Car extends Vehicle {
  constructor(brand) {
    super("Car"); // call parent constructor
    this.brand = brand;
  }

  start() { // method overriding
    console.log(`${this.brand} ${this.type} is revving up!`);
  }
}

const myCar = new Car("Tesla");
myCar.start(); // Tesla Car is revving up!

Here, Car inherits from Vehicle and reuses its constructor logic. It also overrides the start() method to provide custom behavior.

Suppose we have a base Carousel class. We can extend it to create specialized carousels with extra features like styles or auto-scroll.

class Carousel {
  constructor(items = []) {
    this.items = items;
    this.currentIndex = 0;
  }

  showCurrent() {
    console.log(`Showing: ${this.items[this.currentIndex]}`);
  }

  next() {
    this.currentIndex = (this.currentIndex + 1) % this.items.length;
    this.showCurrent();
  }
}

class ImageCarousel extends Carousel {
  constructor(images = [], style = "default") {
    super(images); // call parent constructor
    this.style = style;
  }

  showCurrent() { // override method
    console.log(
      `Displaying image [${this.style} style]: ${this.items[this.currentIndex]}`
    );
  }
}

const imgCarousel = new ImageCarousel(
  ["img1.jpg", "img2.jpg", "img3.jpg"],
  "dark"
);

imgCarousel.showCurrent(); 
imgCarousel.next();

Here, ImageCarousel extends Carousel, reusing its logic for navigation, but overrides showCurrent() to add styling context.

Inheritance lets you reuse and extend base functionality. Overriding allows subclasses to adapt behavior to their needs.
Even with ES6 classes, JavaScript’s inheritance model is still prototype based under the hood.

4 . Polymorphism

Polymorphism allows different classes to define their own versions of the same method. In practice, a child class can override a method from its parent class to provide specialized behavior.

Polymorphism and inheritance often go hand in hand:

Inheritance lets you reuse a base structure (Vehicle → Car, Bike).
Polymorphism lets each subclass override behavior to suit its needs.

If you’re ever confused: Inheritance = reusing code and Polymorphism = redefining behavior.

class Vehicle {
  drive() {
    console.log("This vehicle is moving...");
  }
}

class Car extends Vehicle {
  drive() {
    console.log("Car is driving on the road...");
  }
}

class Bike extends Vehicle {
  drive() {
    console.log("Bike is zooming through traffic...");
  }
}

const myCar = new Car();
const myBike = new Bike();

myCar.drive(); // Car is driving on the road...
myBike.drive(); // Bike is zooming through traffic...

In our caraousel example

class Carousel {
  show() {
    console.log("Showing carousel item...");
  }
}

class ImageCarousel extends Carousel {
  show() {
    console.log("Displaying an image slide...");
  }
}

class VideoCarousel extends Carousel {
  show() {
    console.log("Playing a video slide...");
  }
}

const imgCarousel = new ImageCarousel();
const vidCarousel = new VideoCarousel();

imgCarousel.show(); // Displaying an image slide...
vidCarousel.show(); // Playing a video slide...

Here, both ImageCarousel and VideoCarousel extend the same Carousel base class, but each redefines the show() method — this is polymorphism in action.

5. Abstraction

Abstraction is about hiding implementation details and exposing only what’s necessary. It lets you define a contract (what methods must exist) without forcing subclasses to care about how they’re implemented.

JavaScript doesn’t have native abstract classes like Java or C#, but we can simulate abstraction by:

Throwing errors in base methods that must be overridden.
Using documentation or patterns to enforce a contract.

class Vehicle {
  start() {
    throw new Error("Method 'start()' must be implemented");
  }
}

class Car extends Vehicle {
  start() {
    console.log("Car engine started...");
  }
}

class Bike extends Vehicle {
  start() {
    console.log("Bike engine started...");
  }
}

const myCar = new Car();
const myBike = new Bike();

myCar.start(); // Car engine started...
myBike.start(); // Bike engine started...

Here, Vehicle enforces that every subclass must define start(), but the actual implementation is left to Car and Bike.

class BaseCarousel {
  render() {
    throw new Error("Method 'render()' must be implemented");
  }
}

class ImageCarousel extends BaseCarousel {
  render() {
    console.log("Rendering image carousel...");
  }
}

class VideoCarousel extends BaseCarousel {
  render() {
    console.log("Rendering video carousel...");
  }
}

const imgCarousel = new ImageCarousel();
const vidCarousel = new VideoCarousel();

imgCarousel.render(); // Rendering image carousel...
vidCarousel.render(); // Rendering video carousel...

Here, BaseCarousel defines the abstract method render(). Every type of carousel must implement it, but how it renders (images, videos, etc.) is up to the subclass.

6. Encapsulation

Encapsulation is about hiding internal details and exposing only what’s necessary through a public API. It prevents direct access to an object’s state, which makes code more secure, modular, and maintainable.

In modern JavaScript (ES2022+), we can achieve encapsulation using private fields (denoted by #), getters, and setters.

class Car {
  #fuel = 0; // private field

  refuel(liters) {
    this.#fuel += liters;
    console.log(`Added ${liters}L of fuel.`);
  }

  drive() {
    if (this.#fuel > 0) {
      this.#fuel--;
      console.log("Car is driving...");
    } else {
      console.log("No fuel left!");
    }
  }

  getFuelLevel() {
    return this.#fuel;
  }
}

const myCar = new Car();
myCar.refuel(2);
myCar.drive(); // Car is driving...
console.log(myCar.getFuelLevel()); // 1
// myCar.#fuel (Error: private field)

The car’s fuel state is hidden. You can only change it through refuel() and check it with getFuelLevel().

class Carousel {
  #currentIndex = 0; // private field

  constructor(items = []) {
    this.items = items;
  }

  next() {
    this.#currentIndex = (this.#currentIndex + 1) % this.items.length;
    this.show();
  }

  prev() {
    this.#currentIndex =
      (this.#currentIndex - 1 + this.items.length) % this.items.length;
    this.show();
  }

  show() {
    console.log(`Showing: ${this.items[this.#currentIndex]}`);
  }

  getCurrentIndex() {
    return this.#currentIndex;
  }
}

const carousel = new Carousel(["img1.jpg", "img2.jpg", "img3.jpg"]);
carousel.next(); // Showing: img2.jpg
console.log(carousel.getCurrentIndex()); // 1
// carousel.#currentIndex ❌ (Error: private field)

The carousel’s internal state (#currentIndex) is private. Consumers can only change slides via next() / prev() and can’t tamper with the internals directly.

Encapsulation protects internal state and provides a clear, safe API for interaction. This reduces bugs and makes codebases easier to evolve without breaking consumers.

Bringing It All Together

Let’s combine what we’ve learned so far: classes, constructors, inheritance, and polymorphism into one example.

class User {
  constructor(name) {
    this.name = name;
  }

  greet() {
    console.log(`Hello, I’m ${this.name}`);
  }
}

class Admin extends User {
  greet() { // method overriding (polymorphism)
    console.log(`👋 Admin ${this.name} here. I manage the system.`);
  }
}

const normalUser = new User("Alice");
const adminUser = new Admin("Bob");

normalUser.greet(); // Hello, I’m Alice
adminUser.greet();  // 👋 Admin Bob here. I manage the system.

Real world example

If you’ve worked with React, you’ve already used OOP principles. Before hooks become popular, we were using OOPs to make class based components. If you are someone who is working in building libraries or polyfills then you will be using OOPs a lot.

import React from "react";

// Base Component
class Button extends React.Component {
  render() {
    return <button>{this.props.label}</button>;
  }
}

// Specialized Component (Inheritance + Polymorphism)
class IconButton extends Button {
  render() {
    return (
      <button>
        <span role="img" aria-label="icon">
          🔍
        </span>{" "}
        {this.props.label}
      </button>
    );
  }
}

export default function App() {
  return (
    <div>
      <Button label="Click Me" />
      <IconButton label="Search" />
    </div>
  );
}

Summary

OOPs in JavaScript are important to write clean and maintainable code. OOPs also promotes Solid prinicples of programming.

OOP concept	What/Why
Object	A collection of related data (properties) and behavior (methods). Organizes code into real-world entities. Promotes Single Responsiblity Model (SRE) and Keep it simple silly (KISS)
Class & Constructor	A blueprint for creating objects. The constructor sets initial state when creating an instance.Provides structure, consistency, and easier instantiation of objects.
Inheritance	A child class reuses and extends functionality from a parent class. Promotes DRY principle, code reuse, and logical hierarchies.
Polymorphism	Child classes override or redefine parent methods with their own behavior. Flexibility: same method name, different outcomes depending on context.Promotes Open/Closed Principle
Abstraction	Hides implementation details and exposes only required functionality. Simulated in JS via base classes that throw errors. Defines clear contracts while allowing flexible implementations. Promotes KISS, and Interface Segregation Principle
Encapsulation	Restricting direct access to internal state; exposing it through public methods.Improves maintainability, security, and prevents unintended interference. Promotes SRP, Low Coupling / High Cohesion

I hope this help you to write your clean, and better code in your next project. If you like it then do share to your social media - X, Linkedin.

AI vs ML vs MLOps: A Developer’s Roadmap to Getting Started

Neha Sharma — Sat, 30 Aug 2025 14:56:47 +0000

If you’re asking yourself, “Where do I even start in AI engineering?” — you’re not alone. The landscape is crowded with buzzwords (AI, ML, DL, GenAI, MLOps), and it’s easy to feel lost. A few months ago, I had the same question: Where should I begin — AI, ML, or infrastructure?

So, I went back to step zero to get clarity. In today’s blog, we’ll unpack the differences between AI and ML, and how to decide where to start.

The Big Picture: AI, ML, and DL

You’ve probably seen the classic diagram that looks like nested circles. To understand from where to start in AI domain understanding this diagram will help you a lot. Let’s put some meaning behind it:

1. AI (Artificial Intelligence)

From a 10,000-foot view, AI is about building systems that mimic human-like intelligence. This includes everything from rule based expert systems companies used to today’s generative AI models.

2. ML (Machine Learning)

A subset of AI. Instead of writing explicit rules, ML systems learn from data to make predictions or classifications. eg, predicting house prices based on location, size, and historical sales.

3. DL (Deep Learning)

A subset of ML that uses artificial neural networks with multiple layers. Deep learning shines when you have massive datasets and GPU/TPU hardware. eg, computer vision (image recognition), natural language processing (chatbots, LLMs), and speech recognition.

The common thread across all three? Data.

No matter which path you choose — AI applications, ML models, or deep neural networks — everything starts with clean, structured, and sufficient data. In short: data is the new oil.

The Developer’s Perspective

As engineers, the question becomes: which domain should you pick — AI, ML, or DL? To decide, let’s break it down layer by layer.

1. AI → Application Layer

Leveraging pre-trained or foundation models (OpenAI GPT, Claude, Gemini, Llama 3, etc.) to build intelligent applications. Most GenAI startups fall into this category.

Tech stack examples: LangChain, LlamaIndex, Pinecone, FAISS, Hugging Face APIs.

2. ML/DL → Model Building Layer

Training your own models from scratch or fine-tuning existing ones. This requires understanding algorithms, data preprocessing, feature engineering, and evaluation.

Tech stack examples: Scikit-learn, TensorFlow, PyTorch, Hugging Face Transformers, Weights & Biases.

3. MLOps → Infra → Scaling Layer

Making sure models don’t just work on your laptop but actually run in production at scale. Think data pipelines, CI/CD, model monitoring, GPU clusters, and distributed systems.

Tech stack examples: Kubernetes, Airflow, MLflow, Kubeflow, Amazon SageMaker, Ray, Docker.

Choosing Your Path

Here’s the practical decision-making guide:

Want to build cool apps quickly?

Start with AI (application layer). You’ll work with APIs, prompt engineering, and frameworks like LangChain to turn pre-trained models into usable apps.

Fascinated by algorithms, optimisation, and math?

Go deeper into ML/DL. You’ll learn to design neural networks, tune hyperparameters, and explore architectures like CNNs, RNNs, and Transformers.

Love DevOps, cloud, and distributed systems?

Explore MLOps / infra. This path is about ensuring reliability, scalability, and automation of AI/ML workflows.

You don’t need to learn ML/DL before touching AI applications. In fact, many developers jump straight into building AI-powered apps first, then circle back to ML theory once they want more control.

How day-to-day work looks?

Let’s break down the day-to-day work across each role.

1. AI Engineering (Application Layer)

You have a product idea (eg, builiding a chatbot that answers customer queries by combining OpenAI GPT with your company’s documents (RAG pipeline).) and now as an AI engineer you will:

1 . Choosing and integrating pre-trained models (GPT, Stable Diffusion, Whisper).

2 . Building workflows around LLMs (RAG, agents, tool use).

3 . Handling prompt engineering, context windows, embeddings, and vector databases.

4 . Scaling apps to handle real-world usage.

5 . Considering hardware and latency constraints (e.g., running on GPU vs CPU, on-device inference).

2. ML/DL Engineering (Model Building Layer)

You are building, training, and testing models (e.g, Train a deep learning model to classify writing into shakespear vs not-shakespear using PyTorch and fine-tuning ResNet.)

1 . Collecting, cleaning, and annotating datasets.

2 . Splitting data into train/validation/test sets.

3 . Feature engineering (normalisation, categorical encoding).

4 . Training models (RandomForest, Logistic Regression, or Transformers).

5 . Hyperparameter tuning (Grid Search, Bayesian optimisation).

6 . Evaluating with metrics (accuracy, F1, ROC-AUC, BLEU).

7 . Optimising for inference speed and resource usage.

3. MLOps / Infra Engineering (Scaling Layer)

You are an engineer who is working with DL or AI engineers. eg Build an ML training pipeline with Airflow that trains models nightly, logs metrics to MLflow, and deploys the best-performing version to Kubernetes.

1 . Automating data ingestion with ETL pipelines.

2 . Containerising models with Docker.

3 . Orchestrating workflows with Airflow or Prefect.

4 . Deploying models to Kubernetes or SageMaker endpoints.

5 . Monitoring for data drift and model degradation.

6 . Handling distributed training (Horovod, Ray).

7 . Ensuring reproducibility (MLflow, DVC, GitHub Actions).

The Hardware Angle

Another important dimension is compute infrastructure:

1 . AI Applications: Often run on cloud APIs. You don’t worry much about GPUs, unless you want to self-host.

2 . ML/DL Models: Training deep networks can require powerful GPUs/TPUs (NVIDIA A100, H100, TPU v5).

3 . MLOps: Involves managing GPU clusters, storage, and networking (think: Kubernetes, Proxmox, or cloud GPU farms).

Final Thoughts

Think of it this way:

If you’ve got an product idea today → start with AI applications. Build a working prototype fast. This is where software engineers will shine.

If you’re curious about how models work under the hood → dive into ML/DL. This path requires programming skills and a solid foundation in math.

If you enjoy infrastructure and scaling challenges → focus on MLOps / infra. This requires expertise in cloud systems, networking, and automation.

The ecosystem is huge and evolving fast. You don’t need to master everything at once. Pick the layer that excites you most, get hands-on, and then branch out as your curiosity grows.

At the end of the day, all three paths are connected — many engineers eventually end up wearing multiple hats. Start where you are most comfortable, and build from there.

Finally Completed Designing Data Intensive Application Book

Neha Sharma — Thu, 29 May 2025 09:44:38 +0000

I Finally Finished Reading DDIA – Here’s What I Learned

I’ve finally completed Designing Data-Intensive Applications (DDIA). This book is one of the most popular and recommended book for engineers, and I now understand why it’s considered essential reading for anyone building scalable, resilient systems.

It took me three months of reading to complete this book, balancing a full-time job, personal commitments, and the occasional bout of procrastination.

Do watch my YT video on it

(https://youtu.be/4IkIyrQQoVU))

why I recommend this book to engineers, architects, and tech leads alike:

1 . Data Systems Demystified

The book begins by grounding readers in the fundamentals of data systems—storage engines, indexes, logs, and databases.

It dives deep into the trade-offs between different system designs (e.g., OLTP vs. OLAP, document vs. relational databases), helping you make better architecture decisions.

2 . Distributed Systems — With Real-World Trade-Offs

This is the heart of the book. From replication and partitioning to consensus algorithms and failure handling, Kleppmann explains the why behind every design choice.

You’ll walk away with a strong mental model for designing resilient, fault-tolerant, and eventually consistent systems.

3 . Batch and Stream Processing

The final section covers data processing pipelines—how data flows from ingestion to analytics, whether in real-time or batches.

It helped me better understand tools like Kafka, Flink, Hadoop, and how to design systems that evolve over time without losing integrity or availability.

My Takeaways

No One-Size-Fits-All: DDIA reinforces that system design is always about trade-offs. Consistency, availability, and partition tolerance—pick your battles.

You Need to Think in Systems: Modern architectures (microservices, event-driven design, etc.) require a solid grasp of distributed thinking, not just language-specific skills.

DB underlying: Understanding how databases evolved—from B-trees to LSM-trees to CRDTs—gives you better foresight into current and future trends.

Would I Recommend It?

Absolutely. If you’re serious about designing scalable systems—or leading teams that do—this book is non-negotiable. It will challenge your assumptions, solidify your fundamentals, and make you think like a systems designer.

Yes, it’s dense. Yes, it gets ~~boring~~ academic at times. But that’s where the value is. It doesn’t teach you a single tool or framework; it teaches you how to think.

Happy Learning!!

Sharding Demystified

Neha Sharma — Mon, 26 May 2025 19:42:50 +0000

Following my system design blogs, this is another blog where we will learn about "sharding vs partition".

A lot of folks assume that sharding and partition are same but they are not. When working with distributed systems, with data intensive applications, and data these 2 concepts will come handy.

To understand sharding and partition, we will look into a problem and then see how and where these will help.

This is a two-part blog series. In this post, we’ll focus on sharding, and in the second post, we’ll learn about partitioning.

Problem

Imagine you’ve built a Meetup-style web application. Users can register, browse events, and RSVP to the ones they’re interested in. Initially, everything runs smoothly, the platform uses a single relational database like PostgreSQL or MySQL, and the traffic is manageable.

But soon, your platform takes off. The user base grows rapidly. Now, thousands of users are:

1 . Creating and updating profiles

2 . Browsing through hundreds of upcoming events

3 . Sending RSVPs in real time

As a result, you start noticing signs of stress in your backend:

1 . Fetching meetups takes longer than expected

2 . RSVP actions are delayed or time out

3 . Pages load slowly, especially during peak hours

On deeper inspection, you identify the core issues:

1 . Heavy read/write patterns on the same database tables (users, meetups, RSVPs)

2 . Query latency growing with data size; as rows pile up, indexes become bloated and less efficient

3 . Write contention on hot tables like RSVPs

4 . Disk IOPS and CPU usage on your database instance hitting their limits

It’s clear that your single-database setup isn’t scaling well. You need to rethink how your data is stored and accessed — in a way that distributes the load, reduces contention, and keeps performance fast and consistent as you grow.

Solution

To fix the above issues, partitioning and sharding come in. In this blog we will explore sharding.

Sharding

Sharding is a process of splitting a large database into smaller, independent pieces called shards, which are distributed across multiple servers. Each shard contains a subset of the data. A shard key determines how the data is distributed.

This enables horizontal scaling, allowing the system to handle more traffic and larger datasets by adding more servers.

Advantages of Sharding:

1 . Scaling : Sharding is "horizontal scaling". It helps in scaling the system well as the shard are distributed across multiple servers.

2 . Fault tolrance : If one shard fail then this failure won't affect the other shards. The system will continue working as expected and this helps ensure fault tolrance. A trait every architecture should have.

3 . Performance : As the data is spread across multiple shard and servers the performance of query is improved and it also reduce the risk of "hot spots".

4 . Supports High Concurrency : Each shard can process queries independently.

5 . Tailored Data Placement : Shard based on geography, tenant, or access patterns

Types of shard:

	Type	What?
1	Range based	Data is divided based on a continous range of values of the shard key. Eg: users of meetups from IDs 1–100 go to shard A, and so on.
2	Geo Based	Data is sharded based on a business rule or real-world division. Eg: users in the USA will go to shard A, users from Asia to shard B and so on.
3	Hash based	A hash function is applied to the shard key, and the result determines the shard: hash(user_id) % number_of_shards
4	Directory based	A central lookup table maps each key to a specific shard. Eg: The system maintains a table like user_id → shard_id.
5	Featured based	Data is divided based on the feature.

Magic Pill?

Sharding is not the magic pill that will solve all the problems. It comes with a few dis-advantages. basically "trade-offs". In our meetup problem, Sharding by region helps scale: US → Shard A, EU → Shard B.But users attending events across regions may face latency or cross-shard lookups.

1 . Operational Complexity : More moving parts: monitoring, backups, and deployments become more complicated.

2 . Data Complexity : Handling data in sharding is complex as compare to non-sharding solution. You need to get the data and store them to correct shard, as well as manage retrieve the data too.

3 . Re-sharding : If you have to re-shard then it will be another complexity

4 . cross shard queries : Queries between shards are not easy. Joins, aggregations, and transactions across shard are difficult and slower.

5 . Inconsistent Load Distribution : Poor shard key choice can lead to "hot shards" with unequal load.

Data consistency and sharding

With sharding, we are distrubtuing, storing, and retrieving data from multiple shards distributed across multiple servers. This leads us to question - "How the data consistency will be? - Strong, weak, or eventually".

In sharding, we will have the eventual consistency

How to shard?

Now that we know what sharding is, trade-offs, types time to learn about how to do sharding. For our meetup problem, let's start with:

1 . Understand the Data and Workload : Which table is large, and gets high read/write. Eg: in Meetup table is it users, rsvps, meetups.

2 . Choose a Shard Key : Time to choose a shard key, this determines how data is distributed. It is important to identify the shard key which evenly distributes load, High cardinality (lots of unique values), meet most common query patterns. Eg: users -> user_id , meetups -> location or organizer_id

3 . Pick a Sharding Strategy : what should the sharding strategy be? In our case, we can go with the Geo-based sharding. Eg: when user logs in app identifies user’s region, Routes requests to the correct shard and all operations (fetch meetups, RSVP, etc.) to that shard. Eg: User from USA will go to shard A (having USA data).

4 . Update Application Logic : This is where the code will go. Writing Route requests, handle cross-shard queries, etc.

Once your data is sharded, another important task to take care who decides where the data goes: client (backend application) or server?

	Type	What?
1	client side	The client (usually backend application) contains the logic to determine which shard to read/write from.
2	server side	The server (or middleware/proxy) handles which shard stores or retrieves data, abstracting this away from the client.Eg: AWS aurora with global DB, MongoDB (mongos router), Vitess etc.

Setup: Code Time

PS: The below code examples are only for understanding the sharding concept. These are not "prod ready" code.

Shard config

A configuration file that contains the database connection string against the region. Horizontal scaling is possible (add more regions as shards grow)

// shard-config.js
export const shardMap = {
  US: {
    name: 'Shard A',
    connectionString: 'postgres://user:pass@shard-us.example.com/meetupdb',
  },
  EU: {
    name: 'Shard B',
    connectionString: 'postgres://user:pass@shard-eu.example.com/meetupdb',
  },
  ASIA: {
    name: 'Shard C',
    connectionString: 'postgres://user:pass@shard-asia.example.com/meetupdb',
  },
};

Shard Router

Router will be responsible for routing the users to the correct shard. Queries are routed only to relevant shards, reducing latency.

// db-router.js
import { shardMap } from './shard-config.js';
import { Client } from 'pg';

export function getShardClient(region) {
  const shard = shardMap[region];
  if (!shard) throw new Error(`No shard found for region: ${region}`);

  const client = new Client({ connectionString: shard.connectionString });
  return client;
}

RSVP

// rsvp.js
import { getShardClient } from './db-router.js';

export async function rsvpToMeetup(region, userId, meetupId) {
  const client = getShardClient(region);
  await client.connect();

  await client.query(`
    INSERT INTO rsvps (user_id, meetup_id) VALUES ($1, $2)
  `, [userId, meetupId]);

  await client.end();
  console.log(`User ${userId} RSVPed to meetup ${meetupId} in ${region}`);
}

Tools for Sharding

1 . Vitess (Sharding layer for MySQL)

2 . MongoDB (Built-in sharding)

3 . Citus (Scalable Postgres with sharding)

4 . ProxySQL (MySQL routing/sharding proxy)

How sharding solves our problem?

After implementing sharding in our meetup project we can see the following improvements:

1 . Performance improvement

2 . Low latency

3 . Scaling

4 . Fault tolrance (no SPoF)

Summary

1 . Sharding = Data Splitting across multiple databases to handle scale, performance, and fault tolerance.

2 . It's not the same as partitioning—sharding involves physical distribution.

3 . Sharding improves scalability but adds complexity in querying, consistency, and rebalancing.

4 . Choosing the right shard key is critical—aim for high cardinality and balanced access patterns.

5 . Common strategies include geography-based, tenant-based, or feature-based sharding.

6 . Be aware of challenges like cross-shard queries, re-sharding, and operational overhead.

7 . Use sharding only when needed—it's powerful, but not a silver bullet.

Congrats!! you have reached to the end of the blog. If you liked this blog, found a mistake, or just want to say hi then do reach me at my socials.

Happy Learning!!

Understanding eventually consistency

Neha Sharma — Sun, 18 May 2025 19:21:00 +0000

This is from my series of System design for Developers and based on the book I am reading "Database Internals".

Before we jump into understanding eventually consistency. We need to have a clear understanding of 2 questions:

1 . What is consistency?

2 . Why do we need it?

For the above questions, let's try to understand by a simple daily life analogy - Coffee shop.

Your Coffee Shop Becomes a Chain

Imagine you own a local coffee shop that rewards customer through an app. Every time someone buys a coffee, they earn credits. This means 1 shop, 1 app, 1 database - everything is perfect.

and then your coffee becomes so popular that you expand into a nationwide chain. Now your system needs to scale.

Suddenly, you’re dealing with:

Multiple locations
More users logging in, especially during peak hours (mornings & weekends)

This means we have to scale our App to manage the lodas. It means - More databases, load balancers, and monitoring tools and we ended up with the distributed system.

Now here comes the problem.

A customer buys a coffee in Location A, but doesn’t immediately see their reward points reflected in the app. Is this a bug? Why? is it happening?

Because in a distributed system, writes (like earning credits) may take time to propagate across replicas.

This delay is known as the consistency delay window — the time between the data being written and it becoming visible across all nodes.

So, this is not a bug but a trade-offs (we will see this ahead).

What is Consistency?

In databases, consistency means showing the most recent and correct data to the reader. When consistency is guaranteed, users always get the latest view of the data.

Why Do We Need It?

From the example you must have understood that

To avoid:

1 . Stale reads (seeing outdated data)

2 . Confusing user experiences (e.g., missing points or duplicate messages)

3 . Conflicting writes

Consistency ensures that your app behaves as expected, especially when it spans multiple systems.

Now that we understand what consistency is and why it matters, let’s explore eventual consistency.

Distributed databases often follow one of the following consistency models:

1 . Strong Consistency – Always shows the latest data.

2 . Eventual Consistency – Guarantees that, eventually, all replicas will converge to the latest data.

3 . Loose (or Weak) Consistency – Does not guarantee a consistent view at all times.

Every model comes with its pros and cons. Eg: Strong consistency will cost performance. So, as an architect one needs to evaluate trade-offs and decide accordingly which consistency model to have.

This blog will focus on "eventually" consistency.

What is Eventual Consistency?

In systems with eventual consistency, the data might be out of sync temporarily (remember delay/inconsistency window??), but it will converge over time.

Examples: Banking apps (a pending transaction might not show instantly), Social media likes/comments, Messaging apps (seen/read receipts can lag), Shopping carts in e-commerce platforms

in our coffee app: A user earns credits at Store A. The app may not reflect it instantly across all devices or stores but eventually, once all replicas sync up, the correct balance is shown.

This model works well for high-traffic systems where availability and scalability are more important than instant consistency.

How Do Systems Handle It?

Is eventual consistency a problem? Not necessarily. Like any design decision, “it depends.” As I mentioned earlier, every consistency model has trade-offs. One need to evalaute the trade-offs accordingly.

For eventually Consistency, goal should be - Minimize inconsistency windows.

But how can we handle such "inconsitency"?

1 . Ensure "read your own write" – where the user sees their most recent action. Eg: When the user bought a coffee, in the app we can show coffee is purchase and points are credit (which will eventually in future will synced)

2 . Repair data automatically through background sync (active/passive repair): This is a way to repair the replicas - Active repair : running job at scheduled time. If there are any mistamtches in replicas then sync them. Passive repair: App notices missing points at a store and silently fixes it

3 . Allow tunable consistency – some databases let developers choose the level of consistency per operation (like Cassandra, DynamoDB).

Summary

When we scale databases across nodes and regions, maintaining consistency becomes harder. Eventually consistent systems trade off immediacy for scalability and availability.

As a developer, it’s crucial to:

Understand the trade-offs
Know your system’s consistency model
Design features with the delay window in mind
Use monitoring and repair mechanisms wisely

In short, not all apps need strong consistency—but all developers need to understand consistency.

Database Replication Strategies

Neha Sharma — Wed, 23 Apr 2025 16:23:52 +0000

If you are a beginner then read my blog : What is Database.

This blog is the part of my series on System Design Roadmap for Beginners

In large-scale, high-traffic applications, scaling the database layer is crucial for performance, fault tolerance, and availability.

One of the core techniques used to achieve this is database replication—the process of copying and maintaining database objects (like records and schemas) in multiple database servers.

At its core, databases handle two types of operations:

Write operations: Modifying or inserting new data.
Read operations: Fetching or querying existing data.

To manage these efficiently under load, replication allows writes to go to one or more write nodes (leaders) and reads to be served from read-only replicas (followers).

Why Do We Need Database Replication?

As applications scale, relying on a single database node can lead to performance bottlenecks, single points of failure, and latency issues for global users. Replication addresses these challenges by distributing data across multiple nodes.

Key Reasons for Replication:

1 . High Availability & Fault Tolerance
If one node fails, others can take over seamlessly, ensuring the application remains operational.

2 . Read Scalability
Read-heavy applications benefit from offloading traffic to replicas, reducing latency and improving throughput.

3 . Geo-Distributed Access
Place replicas closer to end users to minimize latency and enhance performance for global audiences.

4 . Backup & Recovery
Replicated data can act as live backups, enabling faster disaster recovery and business continuity.

5 . Analytics and Reporting
Heavy analytical queries can be run on replicas, preventing slowdowns on the primary write node.

Pros and Cons of Replication:

It is not like that replication has no cons. There are reasons for using replication but there are a few trade offs too:

Pros:

1 . Improved Read Performance

2 . Fault Tolerance

3 . Horizontal Scalability for Reads

4 . Redundancy and Backup

5 . Lower Latency for Global Users

Cons

1 . Replication Lag: Followers might not have the most recent data.

2 . Conflict Resolution Complexity: Especially in multi-leader and leaderless setups.

3 . Operational Overhead: More infrastructure and management required.

4 . Eventual Consistency Risks: Temporary inconsistencies are possible.

There are three common replication strategies:

1. Single-Leader Replication (Primary-Replica or Master-Slave)

In this model, only one node (leader) accepts write operations. All other nodes (followers/replicas) asynchronously or semi-synchronously replicate the data from the leader and serve read requests.

Pros	Cons
Strong consistency on writes (since all writes go to one node).	Single point of failure: If the leader fails, failover must happen.
Simpler conflict resolution (as only one node handles writes).	Write scalability limits: All writes go to one node, creating a bottleneck.
Well-supported by traditional RDBMS like PostgreSQL, MySQL.	Lag in replicas: Followers may lag behind the leader, leading to stale reads.

🛠 Who uses it:

PostgreSQL, MySQL, MongoDB (Replica Sets), Oracle RAC in primary-replica setups.
Web applications, CMS platforms, analytics dashboards.

2. Multi-Leader Replication (Multi-Master)

In this strategy, multiple nodes act as leaders, and can accept write operations. Changes made on one node are propagated to others, often asynchronously.

Pros	Cons
High availability and write scalability: Clients can write to the nearest or most available leader.	Conflict resolution is hard: If two nodes write conflicting data simultaneously, you need conflict resolution strategies (last-write-wins, custom merge, etc.).
Good for geo-distributed systems: Reduces latency by writing locally.	Eventual consistency: Inconsistencies may appear temporarily.
High availability and resilience critical	Complex replication logic: Increased operational complexity.

🛠 Who uses it:

CouchDB, Active-Active Redis, MySQL with Galera Cluster, Cassandra (when not using quorum).
Collaborative apps, CRMs, mobile-backend-as-a-service (MBaaS).

3. Leaderless Replication

In this model, there’s no central leader. All nodes are equal and can accept both reads and writes. Coordination happens via quorum-based protocols, where a write is successful only if it’s acknowledged by a certain number of nodes.

For example, using:

W = write quorum
R = read quorum
N = total number of replicas

The system ensures consistency if W + R > N

Let’s say: N = 3 (Data is replicated on 3 nodes), W = 2 (A write must succeed on 2 out of 3 nodes) , R = 2 (A read must query at least 2 out of 3 nodes).

On appling the formula: W + R = 2 + 2 = 4
Since 4 > 3 (N) → The quorum condition is satisfied.

What This Means?

When a write is acknowledged by 2 nodes, at least 2 nodes have the latest data. When a read queries 2 nodes, it’s guaranteed to intersect with at least one of the nodes that saw the latest write.

That way, the system can resolve inconsistencies and return the most up-to-date value.

If W + R < N (N = 3, W = R = 1) ; then Consistency is not guaranteed. A read might only hit a stale node that hasn't received the latest write.

Pros	Cons
High fault tolerance: Even if some nodes are down, operations can still proceed.	Eventual consistency: Data may be inconsistent across nodes temporarily.
Highly available and partition-tolerant (favors AP in CAP theorem).	Conflict resolution required.
Flexible consistency trade-offs.	Write amplification: Every write must go to multiple nodes.

🛠 Who uses it:

Amazon DynamoDB, Apache Cassandra, Riak.
High-availability, low-latency systems like shopping carts, IoT ingestion, logging platforms.

Choosing the Right Strategy

Use Case	Best Strategy
Read-heavy workloads with strong consistency	Single-Leader
Global applications with local writes needed	Multi-Leader
High availability and resilience critical	Leaderless

Happy Learning!!

APIs Are Like Grocery Lists: A Fun Guide to API Design

Neha Sharma — Sat, 12 Apr 2025 08:10:25 +0000

This blog is part of my System Design series, where I break down key concepts in a simple and practical way.

Today, we’ll explore APIs (Application Programming Interfaces)—a fundamental part of modern software development.

At first glance, API design may seem straightforward, but as you dive deeper, you realize it's more complex than it appears. Before we get hands-on, let's first understand the basics of an API.

💻 What is an API?

An API (Application Programming Interface) allows different applications to communicate with each other by defining a set of rules for interaction.

Let's try to understand API with a real world analogy: A Grocery List.

Imagine you ask your mom, *"What should I buy from the grocery store?"

She gives you a grocery list—a structured set of instructions specifying what you need to get.

Think of this list as an API.

You (the client) request information.
Your mom (the server) provides a structured response.
The grocery list (the API) acts as an intermediary that ensures clear communication.

Now in digital world, when you browse X (formerly Twitter), an API fetches tweets for your feed. When you check a food delivery app, an API retrieves restaurant listings.

🔥 Why Do We Need APIs?

APIs simplify communication between the client (browser, mobile app) and the server. As applications become more data-driven, APIs help manage data by:

Fetching, sending, and updating data efficiently.
Enforcing structured interactions between different services.
Supporting architectural patterns like microservices, where different components interact through APIs.

In development, APIs are often referred to as services or contracts, emphasizing their role in connecting the frontend and backend.

📚 Deep Dive

Now, we know what is API and why we need it. Let's try to understand it more in detail to create, and use them.

To understand API, we need to understand the following:

Status code - What happened with the request?
Communication protocol - How the client and server talk to each other?
API Data Formats - How data is sent or received?

Let's explore the above one by one. As the all the above are crucial for API designing.

🚀 Status code (What happened with the request?)

Now, when you have asked your mom to share the grocery list how you will get to know that request is fulfilled or not?

If your mom has handover the list then it is successful.

If you are waiting then how long shall you wait?

If she rejected then you don't need to do anything and so on.

These responses are helpful for you to understand what is happening with your request and you can act accordingly.

Similarly, When an API responds to a request, it includes a status code indicating success or failure.

Understanding these codes helps developers debug issues effectively.

Common Status Codes:

CODE	meaning	Explanation
successful
	200	A new resource was successfully created
	201	The client sent an invalid request
	204	Authentication is required but missing or incorrect
Redirection
	300	The requested resource was not found
	301	The client sent an invalid request
Error
	403	The client does not have permission to access the resource
	404	The requested resource was not found
Server
	500 Internal Server Error	Something went wrong on the server
	502 - Bad Gateway	The server received an invalid response from an upstream server

These codes help developers manage API behavior and improve user experience.

📱 API Communication Protocols - (How the client and server talk to each other?)

Now, when you have requested the grocery list your mother then how you have communicated?

Is it you decided to ask the grocery list in one go? or one by one? or you will call her from grocery store and at that time you will start taking the order.

This is basically how the "communication" will happen. Similarly, we need to define in API this is known as "communication protocols"

Different APIs use different communication protocols to exchange data. Here are some common ones:

communication protocol	explanation
REST (Representational State Transfer)	HTTP API
GraphQL	Request only what is needed
gRPC (Google Remote Procedure Call)	binary-based API protocol
WebSockets	Real time

REST (Representational State Transfer) – HTTP API

The most widely used protocol that follows standard HTTP methods (GET, POST, PUT, DELETE). It uses URLs to access resources and typically exchanges data in JSON or XML.

Your mom gives you a grocery list with everything written down at once. You don’t ask for items one by one; you take the list and go shopping.

2 . GraphQL

Unlike REST, where you get a fixed response, GraphQL allows clients to request only the data they need. This minimizes over-fetching and under-fetching of data.

Instead of receiving a full grocery list, you ask your mom item by item (e.g., “What fruits do I need?”). You get exactly what you ask for, nothing more, nothing less.

3 . gRPC (Google Remote Procedure Call)

A high-performance, binary-based API protocol designed for microservices and inter-service communication. Uses Protocol Buffers (protobufs) instead of JSON.

Instead of speaking in full sentences, you and your mom use secret codes to communicate faster (like shorthand). It’s more efficient but requires both sides to understand the codes.

4 . WebSockets

Enables real-time, two-way communication (e.g., chat apps, live notifications). Unlike REST, which requires repeated requests, WebSockets keep a persistent connection open.

You call your mom while shopping, and she updates the list in real-time based on what’s available.

🗒️ API Data Formats - (How data is sent or received?)

Now, in which format you want the grocery list to be shared.

When making API requests, data can be exchanged in different formats:

Format	Usage	Example
JSON (JavaScript Object Notation)	The most commonly used format, lightweight and easy to parse	A neatly written, structured list
XML (eXtensible Markup Language)	Common in enterprise applications and SOAP APIs	A detailed, old-fashioned document
HTML	Used when APIs return web page content	A full-page magazine with images
Binary (e.g., images, videos, PDFs)	For handling multimedia files	A voice recording of your mom telling you what to buy

❓ How Does an API Look Like?

I am sure now you would be keen to see how API looks like in code.

In the below example, we have a simple example where the format is JSON and key-value format.

{
  "items": ["Apples", "Milk", "Bread"],
  "vegetables": ["spinach", "mint", "peas"]
}

This would be the response when you request (GET) the below URL.

GET https://api.example.com/groceries

The response contains a list of groceries in JSON format, following the communication protocol (HTTP), and returning a status code (200 OK).

Similarly, for POST request we have:

POST https://api.example.com/groceries
Content-Type: application/json

{
  "items": ["Bananas", "Oats"]
}

Response (201 Created):

{
  "message": "Items added successfully",
  "items": ["Bananas", "Oats"]
}

Conclusion

APIs are crucial, enabling seamless communication between different systems. Just like a grocery list ensures you get the right items, an API ensures efficient data exchange between applications.

If you like this blog, please give a like, share in your network, and connect with me at X and Linkedin.

Deploying my ReactJS and HTML5 app to AWS S3

Neha Sharma — Sun, 30 Mar 2025 16:09:38 +0000

This is a submission for the Pulumi Deploy and Document Challenge: Fast Static Website Deployment

What I Built

I wanted to host my ReactJS and HTML5 project to AWS S3. So, I used Pulumi to upload the files from my VSCode CLI.

Live Demo Link

HTML 5 web app

ReactJS

Project Repo

https://github.com/Neha/pulumi-demo

My Journey

Starting with Pulumi was easy because of the documentation. However, for ReactJS and HTML5 web app. I faced a few common issues:

Access to S3
ES modules issue
Access to upload to S3 (file vs folder)

Using Pulumi

2 files (yaml, and index.js) step up is easy to follow.
I used Pulumi's web integrated AI support. It was not able to provide me guidance that was helpful
Loved the preview URL generation, and access to the preview is amazing.

System Design Roadmap for Beginners

Neha Sharma — Fri, 14 Mar 2025 08:03:45 +0000

System design sounds overwhelming. If you are an engineer who has decided to learn system design, this blog will help you start with a structured plan of essential topics.

An important thing to remember is that system design has nothing to do with the "programming language." System design requires knowledge of the fundamentals of software design, independent of the language you use.

Important Concepts

Every system (software) should be designed while keeping in mind the following:

Performance – Ensuring fast response times and efficiency.
Scalability – Designing systems to handle growing workloads.
Reliability – Making sure the system functions correctly under different conditions.
Observability – Monitoring system behavior through metrics, logs, and traces.
Maintainability – Ensuring the system is easy to update and debug.
Logging – Capturing important events and errors for troubleshooting.
Telemetry – Collecting and analyzing real-time system data.

These are the goals of every application and software you will design, whether it is front-end or back-end.

Concepts for a Deep Dive

Once you have a basic understanding, explore these topics in depth:

API Design – Structuring APIs for scalability and efficiency.
Load Balancers – Distributing traffic effectively.
Servers – Understanding types of servers and their roles.
Caching Strategies – Optimizing response times with caching layers.
Databases – SQL vs. NoSQL, indexing, replication, and partitioning.

How to Start?

Now, you might be thinking, "Oh! This is a lot!" Yes, it is—but taking small, manageable steps is the key. The best way to start is by training your system design thinking, especially if you're from a non-backend background.

System Design Through a Real-World Analogy

Imagine you are a chef at a new restaurant. As a chef, your responsibilities include:

Ensuring the kitchen operates without disruptions.

System-design concepts: System availability & reliability (CAP Theorem, Fault Tolerance)

Just like a kitchen should always function despite small failures (e.g., a missing ingredient or a broken appliance), a system should be designed to be available and resilient even when some parts fail. Concepts like fault tolerance and CAP theorem help achieve this balance.

2 . Maintaining harmony among the staff.

System design concept: Distributed systems and consistency models.

A well-managed kitchen has different chefs handling various tasks, such as baking, grilling, and plating, yet everything must come together harmoniously. In distributed systems, multiple servers manage data and requests while maintaining consistency to ensure the system functions as one.

3 . Keeping ingredients fresh and well-stocked.

System design concept: Database caching and indexing

A restaurant ensures essential ingredients are always available and quickly accessible, similar to how databases use caching and indexing to provide fast data retrieval and improve performance.

4 . Ensuring customer orders are prepared and served efficiently.

System design concept: Load balancing and API request handling

When a restaurant receives multiple orders, chefs must manage them efficiently without overloading any single person. Similarly, load balancers distribute user requests across multiple servers to ensure smooth system performance.

5 . Managing customer expectations and handling peak-hour demand.

System design concept: Auto-scaling and performance optimization.

A restaurant must be ready to handle increased customers during peak hours by adding extra staff or prepping ingredients in advance. Similarly, auto-scaling enables a system to handle high traffic by dynamically adjusting resources.

6 . Designing a menu that balances variety and efficiency.

System design concept: Architecting a scalable and maintainable system**

A restaurant menu should be carefully designed to offer variety without overwhelming the kitchen. If the menu is too large, it slows down operations. Similarly, system architecture should be structured efficiently to avoid unnecessary complexity while ensuring scalability and maintainability.

7 . Setting up an ordering system for seamless customer interactions.

System design concept: API design and microservices

Just as customers need a simple way to place orders (dine-in, takeaway, online), applications use well-designed APIs to handle requests efficiently. A well-structured API ensures smooth communication between different system components, just like an ordering system ensures a seamless experience for customers.

Just like a chef ensures smooth restaurant operations, a system designer ensures a scalable, efficient, and resilient system. By understanding these concepts step by step, you will build a strong foundation in system design.

Summary

System design isn't something you master overnight. Start small, focus on understanding real-world applications, and gradually explore complex architectures.

Keep learning, building, and experimenting!

Happy coding! 🚀

React Native New Architecture

Neha Sharma — Sat, 09 Nov 2024 16:47:33 +0000

React Native has announced the release of version 0.76. With this update, the new architecture will now be enabled by default. Prior to version 0.76, the only way to enable the new architecture was by opting in.

This blog will focus on what's new in the new architecture.

The new React Native architecture has been making headlines for over a year, and for all the right reasons. This new architecture, also known as 'Bridgeless'. Why is it called Bridgeless? We’ll explore this in the blog."

Before we explore the new architecture, let's quickly recap the previous one.

📱 Recap of old architecture

a. There are two lands - JavaScript, and Native.

b. There are 3 threads:

JavaScript Thread (JavaScript): Responsible for the JavaScript bundle code.
Main, UI Native Thread : Responsible for the native modules.
Shadow, or background Thread (Yoga): Responsible for the layout.

c. The only way JavaScript and Native code can communicate to each other is through the Bridge.

d. Any native component, such as Button or Alert, is serialized into JSON in the JavaScript layer and sent through the bridge to the native thread. In the native thread, this JSON is then converted into a native (iOS or Android) component.

e. In the native thread, when an event occurs on a native component, it sends the event as JSON to the JavaScript thread through the bridge. This communication is asynchronous, enabling the bridge to facilitate interaction between JavaScript and native components.

🚩 Problems with old Architecture

Communication between the JavaScript and native layers relies on the bridge, which is the main limitation of the old architecture.

Performance Issues
Jerky or empty frames
Duplication of the nodes

✨ Goal of new architecture

The new architecture focuses on resolving the challenges of the previous one. Its goals are to:

Fast startup 🥳
Concurrent rendering 🎟️
Responsive apps 🧑‍💻
Support on multiple platforms 🎮
Less crashes 🧨
Better memory management 💃
Synchronous execution 🎢

🆕 New Architecture

The new architecture is a rewrite in C++, which has unlocked two major improvements:

A. Direct communication between JavaScript and native layers without a bridge. This is why the new architecture is commonly known as 'bridgeless.'

B. Support for multiple platforms (as long as the platforms are using React Native)

🧩 New Architecture's components:

1. JavaScript Interface (JSI)

JSI is JavaScript Interface, this is the layer written in C++. Any JS engine can be use with this and this enables the cross platform support - not just on IOS, android but also on smart TVs, smart watches, etc.

JSI enables the JavaScript to hold a reference to the native module. These enables the JavaScript to communicate directly with native modules as well as this enables the synchronous communication between JavaScript and Native thread

PS: Your react native code gets bundle through Metro and goes to the JSI.

2. New Native Modules

New Native Modules is the new and improved native modules. This is written in C++ and it enables the synchronous access from JS/TS apis to Native. This means that there will be direct communication between Native, and JavaScript thread without the need of bridge. C++ also enables to write your own native modules for cross platform sharing.

New native modules allows to handle events, read layout, schedule updates both async, and sync.

As we learned earlier, JSI keeps the reference of objects in the TurboModules, this will allow JavaScript code to load each module only when it is required (dynamic loading of modules). This improves the startup time of app as compared to the old architecture.

3. Codegen

Codegen is a tool to create the strongly typed contracts. These contracts are helpful for developers by saving their time and make the communication easier between cross programming languages.

In React Native, JavaScript and Typescript are not strongly typed language but C++ is strongly typed. To make the communication between JavaScript, and C++ , codegen generates interfaces (types). This happened at the build time for fast execution at runtime.

Because of the Codegen JSI (JavaScript Interface) directly communicate with Turbo modules without any bridge.

4. New Renderer

New Renderer is known as Fabric. This is also written in C++.
Remember we have 3 threads in React Native? - JavaScript, Main/UI Native, and Shadow/background thread. In old architecture, the problem was our main thread used to get blocked and this leads to the:

performance issues
empty frames (missing 60FPS)
also, old architecture has to maintain the 2 copy of node and DOM. This leads to the memory issue
There was no way to interrupt the low priority tasks to give priority to urgent updates.

In the Fabric these issues were taken care. With new renderer now we can use transition to interrupt the low priority tasks for urgent. This will make the app responsive & Main/UI native thread won't be block. The events will be async execute. With new renderer system, there will an immutable tree of the view hierarchy.

Immutable means that it won't be changeable. Benefits of immutable:

This allow for thread-safe processing of updates.
This also means there will be multiple in-progress trees, each representing a different version of the user interface.
As there are multiple trees at different version of UI, *updates can be rendered in the background without blocking the UI *(such as during transitions) or on the main thread (in response to user input)
The new renderer can also read layout information synchronously and across different threads. This enables background computation for low-priority updates and synchronous reads when needed.

🎯 End to End Architecture

🎉 Summary

New architecture is available by default from 0.76
New architecture has introduced: JavaScript Interface (JSI), new native modules (Turbo Modules) , codegen, new renderer system (fabric)
JSI is JavaScript Interface it is based on C++ and make the JavaScript and Native modules direct communication possible without any need for Bridge
New Native modules are improved current native modules. Written in C++ and enables lot of benefits: synchronous communication to and fom JavaScript and native without bridge, lazy loading.
Codegen generates type interfaces for JavaScript, and C++ to communicate with each other.
New renderer (Fabric) is new renderer system written in C++. This enables the better performance by enabling multiple threading, and interrupting low priority tasks.
New event loop to make the react native more closer to DOM
Good read GitHub issue

❤️ Say Hi!!

Twitter

YouTube

Happy Learning!!