DEV Community: Ranuj Mahajan

Neural networks don’t understand words, sentences, or punctuation. They only understand numbers.

Ranuj Mahajan — Sun, 13 Sep 2026 09:00:58 +0000

Core Definition: Tokenization is the process of breaking raw text into smaller discrete units—called tokens—which can be words, subwords, characters, or symbols that a computer model can process.

Numerical Mapping: Once split, each unique token is assigned a numerical identifier (Token ID) using a predefined vocabulary, allowing neural networks to perform mathematical operations on text.

Modern Standard (Subword Tokenization): Most large language models use subword algorithms (such as Byte-Pair Encoding or WordPiece) to balance vocabulary size, efficiently handle compound words, and avoid "out-of-vocabulary" errors for rare or unseen terms.

Neural networks don’t understand words, sentences, or punctuation. They only understand numbers and vector spaces.

Ranuj Mahajan — Sun, 13 Sep 2026 08:56:19 +0000

Visual breakdown: How raw text moves through the LLM ingestion pipeline (Tokenize Embed Position Attend)

Ranuj Mahajan — Sun, 13 Sep 2026 08:45:05 +0000

Liquid syntax error: Tag '{% %}' was not properly terminated with regexp: /\%\}/

How Embeddings are generated in AI

Ranuj Mahajan — Sat, 12 Sep 2026 13:00:51 +0000

Why Most Engineers Struggle with System Design (And How Visuals Fix It)

Ranuj Mahajan — Fri, 11 Sep 2026 10:57:02 +0000

Most engineers prepare for system design interviews by reading 600-page textbooks or memorizing generic bullet-point summaries.

Then, they get hit with an open-ended interview question like:

"Design a globally distributed URL shortener handling 50k writes per second."

Suddenly, the disconnected facts freeze up. You know what a load balancer is. You know what Redis does. But visualizing how data moves end-to-end under load is an entirely different skill.

The Flaw with Text-Heavy System Design
Text explains definitions, but distributed architecture is all about flow and trade-offs:

How request routing navigates an API Gateway and Load Balancer before touching application instances.

Where decoupled workers write back output after an asynchronous queue is triggered.

How cache invalidation propagates when a write hits the primary database.

When you only memorize terms, you miss the mental map of where bottlenecks emerge.

System Design You Can See
To design resilient architectures, you need to build intuition visually:

End-to-End Tracing: Follow a packet from client to CDN, load balancers, caches, and sharded storage tiers.

Visualizing Trade-offs: See the direct cost of consistency vs. availability, or horizontal scaling vs. vertical read replicas.

Reusable Blueprints: Move beyond ad-hoc diagrams to structured visual mental models that apply to real-world architectures (like Netflix, Uber, or WhatsApp).

Level Up Your Architecture Intuition
If you want to stop getting lost in abstract jargon and start designing systems with confidence, check out EngineeringDepth.

We teach system design visually—giving you clear, practical diagrams, lessons, and deep-dive architectural breakdowns designed for modern engineers and tech interviews.

👉 Explore the visual course at engineeringdepth.com

What’s the hardest concept in distributed systems for you to visualize? Let’s discuss in the comments below!

Basics of the RAG

Ranuj Mahajan — Mon, 03 Aug 2026 11:20:10 +0000

Artificial Intelligence becomes much easier to understand when you see the flow visually.

Ranuj Mahajan — Wed, 29 Jul 2026 14:00:54 +0000

How does global Load balancer works

Ranuj Mahajan — Tue, 07 Oct 2025 14:58:42 +0000

How does Load Balancer works?

Load balancer uses an algorithm to forward the request to the different based on some filters.

CPU based algorithm
round robin technique
Load balancer also keeps the heart beat to check if services connected to it are working fine.
global load balancers distributes the request based on IP and geographical locations.

![ ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t9r3klrv9627ue7yqdyn.png

Mardown to the PDF converter.

Ranuj Mahajan — Mon, 15 Sep 2025 15:41:27 +0000

https://pdffrommarkdown.com is the place where you can convert your markdown files to pdf .
you can give your custom names to the pdf and you can encrypt them with password as well.

Improve your understanding of SOLID principles in OOPS.

Ranuj Mahajan — Mon, 26 Aug 2024 07:38:02 +0000

What Does Solid principles mean ?

S-> Single responsibility principle : it states that a class or an object in the OOPS programming realm should carry only one responsibility. if there are actions that are taken by student
pay for the fees
enrolment to new course
subjects studying

Assume we can create a class with all the responsibility encapsulated in student class, but that would make our code complex and difficult to maintain.

we have to separate the concerns and make sure that each class is only responsible for one task . this way each class will perform task.

O-> Open and closed principle :
This principle states that we should not change existing functionality or program in the class. we can extend our functionality of the class, but we shouldn't be able to modify it.

this looks really easy while we write it in theory, but we have to think of creating interface and make sure that we even if new classes are added, they implement from the interface and are able to execute the existing core logic.

Why do we need a NoSQL Databases

Ranuj Mahajan — Tue, 23 Jul 2024 13:16:57 +0000

In this post, I would explain why do we need NoSQL databases from a developer perspective.

Developer Friendly : NoSql databases are developer friendly, we can store JSON that is generated from the UI to the database, there is no transformation required at the application server end. this data can be read through a key from the database.
Data schema: we can store Json documents of different format in the collection. we dont have to comply to the any data schema. data of different schema's can be stored in the collection. this ensures that a very less transformation or no transformation is required for the data retrieval and data storage.
NoSql horizontal scaling : it can store huge amount of data. Nosql data can be scaled out easily. as data grows, we can add more and more servers to the Nosql databases.

Why do we need NoSql Database

Ranuj Mahajan — Thu, 18 Jul 2024 10:48:53 +0000

Data is new oil, we need database management technique to store the data of the application. data can be categorised into two types

Relational data : this data has relations and is required for the transaction. it is really time consuming or not easy to horizontal scale SQL data though not impossible( so we need NOSQL data to scale our data management system easily)

*Below are some important reason why we need NoSql Databases
*

Nosql databases are easily scalable, do you why are they scalable, because data is not dependent on each other. so i give you an example if you store 1000 movies in your hard drive and create a index somewhere to get the movies from the harddrive, you can store another 1200 movies in other harddrives and scale it. so as movie storage grow, you can buy a storage and create an index, this is the case with nosql databases, as data grow, you can buy a NoSql database and start experimenting with our data. this is very important. we need quick data and we need less overhead if data grows exponentially.
data can grow exponentially while application is running, we can do sharding on the data. we will use shards and each shards will store some kind of common data.
NoSQL system support security, you can use kerberos, password based and encryption algorithms to the NoSql database.
latency, if we have data that is scattered across the server, then we may need to create an index . we can create an index on nosql servers as well, index is nothing complex but is a lookup, how we will search for an data . nosql databases if properly managed can reduce latency and respond super fast.
fault tolerance and high availability: these database are well suitable for the cloud and distributed environments, they are highly available. if in case any node or any system is down, nosql databases recover well on time.