DEV Community: Ashis Chakraborty

Ever tried comparing a basketball to a watermelon...??!!

Ashis Chakraborty — Fri, 25 Jul 2025 16:05:32 +0000

Ashis Chakraborty

Jul 25 '25

Why Feature Scaling Matters in ML?!

#featurescaling #machinelearning #ai #datascience

Comments

2 min read

Why Feature Scaling Matters in ML?!

Ashis Chakraborty — Fri, 25 Jul 2025 16:02:08 +0000

Ever tried comparing a basketball to a watermelon…?! That's what your machine learning model feels without feature scaling!
If you feed your AI model age in years and net worth in millions.
One feature overpowers the other. The result? Terrible predictions.
No balance. No fairness.

Fig. ML model without feature scaling

Imagine a dataset with two features:
Age: 20 to 70
Income: 15,000 to 2,500,000

Without scaling, income will dominate distance-based decisions just because of its larger numbers. Now, incase of a insurance premium calculation, income may become deciding factor instead of age.

Fig. Wrong prediction because of net worth's high value

Feature Scaling:

According to wiki:
Feature scaling is a method used to normalize the range of independent variables or features of data.
Feature scaling is a data preprocessing technique that transforms the values of numeric features into a similar scale or range. It is done typically so that no one attribute dominates others due to its high numeric value.
Fig. Scaling using Age divided by 100 and net worth by 100,000 in the previous figure

It makes sure all the features are in the same range numerically so that they can be compared without much external factors. A feature is a characteristic or attribute of your data.

Why is Feature Scaling Important ?

Many ML models - especially those based on distance calculations or gradient descent - are sensitive to the scale of data. Without scaling:
Attributes with larger values can unfairly influence the model.
Models like K-Nearest Neighbors (KNN) or Support Vector Machines (SVM) can give wrong predictions.

Feature scaling Methods:

We can rescale all features to the same range - like 0 to 1 or -1 to 1 -
so every input gets equal importance.
Two common methods:

i) Standardization -

Centers data around zero. Data can be negative or positive. Formula for standardization is:
st. value = (original value - mean of the feature) / standard deviation of the feature

Fig. Apply Standaradization on data

ii) Normalization -

Compress values between 0 and 1. It is often called Min-Max Scaling. In this feature scaling technique it resizes all values in a feature to a fixed range, typically between 0 and 1.
norm. value = (original value - min value of the feature) / (max value of the feature - min value of the feature)

Fig. Apply Normalization on data

You may check this video link for more detail.

Find Largest Value in Each Tree Row

Ashis Chakraborty — Wed, 25 Dec 2024 16:50:47 +0000

Leetcode daily challenge: 25 Dec,2024.
Problem link

BFS is perfect when we are dealing specifically with rows/levels of a binary tree. With BFS, we handle one row of the tree at a time.

Here, we need to find the maximum value in each row. We can simply perform a BFS and for each row, keep track of the maximum value we have seen so far.

void bfs( TreeNode* node, vector<int>& result){

        queue<TreeNode *>Q;
        Q.push(node);

        int qSize = Q.size();
        int count =0;

        while(!Q.empty()){
            count = 0;
            priority_queue<int>heap;
            qSize = Q.size();

            while(count<qSize){// for each level/row find max value
                count++;
                TreeNode* temp = Q.front();
                Q.pop();
                heap.push(temp->val);

                if(temp->left != NULL)
                Q.push(temp->left);

                if(temp->right != NULL)
                Q.push(temp->right);

            }//while ends

            if(heap.size())
            result.push_back(heap.top());
        }
    }
    vector<int> largestValues(TreeNode* root) {
        if(root == NULL)
        return {};

        vector<int>result;
        bfs(root, result);///call bfs

        return result;
    }

Video link with detailed explanation will be found here.

Steps to solve Recursive problems

Ashis Chakraborty — Mon, 13 May 2024 18:37:45 +0000

Recursion is the technique of a function calling itself repeatedly till the base condition is satisfied.
It reduces the length of our code. Recursion is helpful for solving Graph Traversal and Dynamic programming..types of problems.

5 steps to solve a problem using recursion:

Problem Statement: Given an input n. Calculate the sum of all the integers from 1 up to n. Solve it using a recursive function.

Step 1. Find the base case The base case is the condition, which tells the function when to stop.
When there is only 1 element, sum is 1. In our example problem. base case is when n = 1, answer = 1.

Step 2. Write some example cases with consecutive inputs (preferably more than 5 example)
n = 5; answer = 1 + 2 + 3 + 4 + 5 = 15

Step 3. Find a pattern between two consecutive answers
n = 4; sum(4) = 1 + 2 + 3 + 4 = 10
n = 5; sum(5) = 1 + 2 + 3 + 4 + 5 = 15 = sum(4) + 5

Step 4. Generalize a pattern in function terms
n = 5; sum(5) = 1 + 2 + 3 + 4 + 5 = 15 = sum(n-1) + n
n = 4; sum(4) = 1 + 2 + 3 + 4 = 10 = sum(n-1) + n

Step 5. Write code using recursion, with base case

int  sum ( int n ) {
           if( n == 1)
             return 1;
           return n+ sum(n-1);                    
 }

For more details you may check this video:

Solve Recursion in Five Steps | @designUrThought - YouTube

Welcome to @designUrThought for an in-depth exploration of recursion, a fundamental concept in computer science and programming! Whether you're a coding nov...

youtube.com

Designing a Rate Limiter

Ashis Chakraborty — Mon, 09 Aug 2021 19:35:30 +0000

How do you design a rate limiter service?

Rate limiting is a simple concept to grab, but it’s vital for large-scale systems. It is used for security purposes as well as performance improvement. For example, a service can serve a limited number of requests per second. However, if a service is receiving a huge number of requests at a time, it might be too much for the server to handle. In that case, even the server might be shut down. To handle this type of problem, we would need some kind of limiting mechanism that would allow only a certain number of requests to the service.

Why do Need Rate Limiting?

Rate limiting is needed to protect a system from being brought down by hackers. To know the importance of rate-limiting, we have to know about the DOS attack.
The denial of service attack, which is mostly known as a DOS attack, is when hackers try to flood a system with many requests within a short period of time to shut the system down. Because a server has a limit of how many requests it can serve within a certain period of time. So, the system can’t handle the floods of requests properly; it can not handle them properly.
Rate limiting can prevent that from happening as it protects the system from being flooded with intentional unnecessary requests. After a threshold value, the system returns an error.

Sliding Window Algorithm:

If we keep track of each request per user in a time frame, we may store the timestamp of each request in a Sorted Set in our ‘value’ field of hash-table. But, it would take a lot of memory. So, we can use a sliding window with the counters. Now, consider this if we keep track of request counts for each user using multiple fixed time windows.
For example, if we have an hourly window, when we receive a new request to calculate the limit, we can count requests each minute and calculate the sum of all counters in the past hour. This would reduce the memory we need for the set.
Suppose we rate-limit at 500 requests per hour with an additional limit of 10 requests per minute. This means that the client has exceeded the rate limit when the sum of the counters in the past hour exceeds the limit request(500).
For a smaller time frame, the client can’t send more than ten requests per minute. This would be a hybrid and reasonable consideration as we may think that none of the real users would send such frequent requests. Even if they do, they will get success with retries since their limits get reset every minute.
Continue reading on Towards Data Science »

System Design Analysis of Instagram

Ashis Chakraborty — Mon, 22 Mar 2021 17:40:16 +0000

How do you design a photo-sharing service like Instagram?

Performance, availability, consistency, scalability, reliability, etc., are important quality requirements in system design. We need to analyze these requirements for the system.

As a system designer, we might want to have a design that will be highly available, very high performant, top-notch consistency in the system, highly secured system, etc. But it’s not possible to achieve all these targets in one system. We need to have requirements that will work as restrictions on the design of a system. So, let’s define our NFRs:
Our system should be highly available. In the case of any web service, it’s a mandatory requirement. Home page generation latency should be at most 200 msec. If the home page generation takes too long, users will be dissatisfied, which is not acceptable.
As we choose for the system’s high availability, we should keep in mind that may hamper consistency across the system. The system should also be highly reliable, which means any uploaded photo or video by users should never be lost.
In this system, photos search and views would be more than uploading. As the system would have more read-heavy operations, we will focus on building a system that can retrieve photos quickly. While viewing photos, latency needs to be as low as possible.

Data flow:

If you are not sure where to start in a system design, always start with the data storing system. It will help to keep your focus aligning with the requirements of the system.
We need to support two scenarios at a high-level, one is to upload photos, and another is to view/search photos. Our system would need some object storage servers to store photos and some database servers to store metadata information.
Defining the database schema is the first phase of understanding the data flow between different components of the system. We need to store user profile data like the follower list, uploaded photos by users. There should be a table that stores all data regarding photos.

Load balancer:

As we use multiple copies of a server, we need to distribute the user traffic to those servers efficiently. The load balancer will distribute the user requests to various servers uniformly. We can use IP-based routing for the Newsfeed service, as the same user requests go to the same server, and caching can be used to get a faster response.
If there are no such requirements, round-robin should be a simple and good solution for the server selection strategy of load balancers.

API GateWay:

We have a lot of services for our system. Some will generate newsfeed, some help storing photos, some viewing the photos, etc. We need to have a single entry point for all clients. This entry point is API Gateway.
It will handle all the requests by sending them to multiple services. And for some requests, it will just route to the specific server. The API gateway may also implement security, such as verifying the client’s permission to perform the request.

Pagination:

The newsfeed of users can be a large response. So, we may design the API to return a single page of the feed. Let’s say we are sending at most 50 posts every time a feed request is made.
The user may send another request for the next page of feeds for the next time. And within that period, if there is not enough feed, Newsfeed may generate the next page. This technique is called pagination.

Continue reading on Towards Data Science for detailed design analysis»

Designing Notification System with Message Queues

Ashis Chakraborty — Mon, 15 Feb 2021 15:29:34 +0000

What is Message Queue?

From the name itself, you can understand it is a queue. But it’s a particular type of queue. It routes messages from the sender to the receiver.

It is a queue; it follows the FIFO (First In First Out) policy. It means the message that is sent first will be delivered first. In some cases, we can add priority to a message, which will make it a priority queue.

So, where do we use a message queue? Say we are designing an email server. The sender sends an email, but the receiver is offline. The mail server needs to temporarily store the mail until the recipient comes online and gets the message.

Message queues can also be used for implementing notification systems. Message queues also allow us to run background processes, tasks.

Properties Of A Message Queue

Message queues are asynchronous in style. Asynchronous behavior allows the components(sender and receiver) to communicate with each other in the background.

An important property of message queues is that they give temporary storage for storing messages until they are processed & received by the receiver. This is the property that makes the message queue an important component while designing a system.
We can add priority to the messages in case of the requirement of the system.

Other important message queuing properties include message acknowledgments when a message is received successfully, retrying for failed messages in case of failure scenarios.

We now need to look into a popular messaging model widely used in the industry, the publish-subscribe message routing model.

Publish-Subscribe Pattern

A Publish-Subscribe model is where many consumers receive the same message sent from a single or many sources.

Current days, we often subscribe to various topics on websites for being notified of the new updates on any particular interest. Be it from Facebook groups, Livescore for games, etc. We will discuss pub-sub in another article. For now, we should know, message queues have components that push the messages to the queues based on the message type and the rules which are set.
The exchange component will push the message to the queue, and the consumers will receive the message. The relationship between exchange and the message queue is known as Binding.

Facebook newsfeed notification system with the message queue
In the latter part of this article, we will discuss how notification systems and real-time feeds are implemented using message queues.

For more detailed implementation, check this link

The main article is published in Towards Data Science »

System Design Basics: Getting started with Caching

Ashis Chakraborty — Sat, 16 Jan 2021 18:12:44 +0000

Caching is an essential topic of system design. It is one of the essential techniques of system design. In almost all the systems, we might need to use caching. It is a technique used for the performance improvement of a system.

What is Cache?

The cache is a piece of hardware or software that stores data that can be retrieved faster than other data sources. Caches are generally used to keep track of frequent responses to user requests.

We need to increase the number of cache hits and decrease the cache miss rate for performance improvement.

Cache Invalidation:

In the case of data modification in DB, if the cache contains the previous data, then that is called stale data. So, we need to have a technique for invalidation of cache data. Else, the application will show inconsistent behavior. As cache has limited memory, we need to update data stored in it. This process is known as Cache Invalidation. Some techniques for cache invalidation are Write-through Cache, Write-back Cache, Cache-Aside, Read-Through Cache, etc.

Cache Eviction policy:

A cache is a backup of primary data storage; It has a limited amount of space. We may need to remove them from the cache. These are called eviction policies. Some well-known policies are First In First Out, Last In First Out, Least Recently Used(LRU), Least Frequently Used (LFU), etc. According to the requirements, we have to select an eviction policy for the system.

Continue reading on Towards Data Science »
If you don't have a medium account check this link

System Design Basics: Getting Started With the Client-Server Architecture

Ashis Chakraborty — Tue, 22 Dec 2020 13:02:58 +0000

Client-Server architecture

Client/server architecture is an important concept for system design newbies. You may think it like the foundation of how the modern internet works. Nowadays, digital devices like computers, laptops, mobile devices are everywhere. Client-server architecture is the foundation of knowing how these computers talk to one another.

What do client and server mean in the client-server model? For simplification, we may say that a client is a machine that requests some data or service. And the server returns some service or data to the client. Servers listen to the incoming network requests. So, the client-server model is a modern system designed in such a way that clients request data or service from the server, and the servers provide data or service to the client.

Now we may check how this architecture works with an example. For example, what happens when we type www.medium.com in our browser and press the enter button. The client device, our own laptop or computer, does not really know what the medium.com URL means. The browser needs to communicate with the server machine where the medium URL is located.

The users who type a specific URL in this system are clients. The server provides information through the internet. So, these computers can be in different parts of the world.

A specific set of rules are needed for the interaction of two systems. The most popular are the HTTP and HTTPS.

The client computers request the required information from the server. That could be in any structure of formatted data. Mainly implemented and popular formats are in XML and JSON. Server and client request and respond mostly using these formats.

Now, we are going a bit into detail into what happens after the user types a URL in a browser. How does the browser know where the server is located? The browser performs a DNS query.

Continue reading on Towards Data Science »
If you don't have a medium account check this link

System Design Basics: Load balancer 101

Ashis Chakraborty — Fri, 06 Nov 2020 00:17:16 +0000

Getting started with Load balancer

The load balancer is a server that usually sits between client devices and a set of servers and distributes client requests across servers. Load balancers can be placed in various places of a system. The loads on the servers need to be distributed in a balanced way; that’s why they are called a load balancer.

Where do we use a Load balancer?

Typically, we can put a load balancer between the client and the server to handle incoming network requests. Another very normal use of an LB is for distributing the network traffic across backend servers. A load balancer is used to reduce individual server load. And it prevents a single point of failure for any server. So, it improves the overall availability and responsiveness of the system. We may add LBs at various places in the system. Especially where we have multiple resources like servers or database or cache.

Load Balancing Techniques:

There are various types of load balancing methods. Every type uses different algorithms for different purposes. These techniques are actually different types of strategy for server-selection. Here is a list of load balancing techniques:

Random selection:

In this method, the servers are selected randomly. There are no other factors calculated in the selection of the server. There might be a problem with some of the servers sitting idle, and some are overloaded with requests in this technique.

Round Robin:

This is one of the most common load balancing methods. It’s a method where the LB redirects incoming traffic between a set of servers in a certain order. Let's say, there is a list of five servers; the first request goes to server 1, the second one goes to server 2, and so on. When LB reaches the end of the list, it starts over at the beginning, from server number 1 again. It almost evenly balances the traffic between the servers. But in this method, server specifications are not considered. The servers need to be of equal specification for this method to be useful. Otherwise, a low processing powered server may have the same load as a high processing capacity server.

Continue reading on codeburst »
If you don't have medium account check this link

System Design 101

Ashis Chakraborty — Wed, 07 Oct 2020 13:56:59 +0000

Step by step guide on designing a system
System design is one of the most important and feared aspects of software engineering. One of the main reasons was that everybody seems to have a different approach; there is no clear step by step guidelines.

The struggle of software engineers with system design can be divided into two parts:

1) The system design process is unstructured by nature; there is no proper answer to say binary right or wrong.

2) Software developers lack in experience developing complex and large-scale systems.

In this article, we’ll go through steps to approach solving a design problem. This guideline may help you to design a system. This one is based on my experience of learning architecture courses.

★Step 1: Requirements clarifications
★Step 2: Estimation of important parts
★Step 3: Data Flow
★Step 4: High-level Component design
★ Step 5: Detailed design
★ Step 6: Identify bottlenecks and resolve them
For detailed information,
Continue reading on Towards Data Science »

Those who don't have medium account may use this link

Five Daily Activities to Overcome Negative Thoughts

Ashis Chakraborty — Wed, 30 Sep 2020 06:50:18 +0000

Simple activities to remove negative thoughts in your daily life:

★Write down your problems.

Writing is like a mental exercise. It can bring psychological relief to the mind.

★Forget what others are thinking about you.

Most of our negative thoughts are about what people may think if you do or do not do something.
Care about what other people think and you will always be their prisoner. — Lao Tzu
The interesting fact is others don’t really care as much as you think. Unless your activity directly affects another person or their life, they are unlikely to spend much time thinking about you at all.

★Take deep breaths

Take a 1–2 minutes pause and take some deep breaths. Feel that you’re breathing with stomach and through your nose. Focus on the air going in and out, don’t rush them, be calm; just breath in and out

★Listen to music

Let the body heal from the outside, when there’s a storm brewing on the inside.

★Focus on the solution, not the problem

If you focus on the solution, you will get more ways to succeed. But if you focus on the problem, you will find more problems.

Continue reading on ILLUMINATION »