DEV Community: Ranuj Mahajan

How AI Learns: Forward Pass and Loss Explained with 2 + 1

Ranuj Mahajan — Sat, 03 Oct 2026 09:14:52 +0000

Our inputs are 2 and 1, and the correct answer is 3. But the network initially predicts 0.7.

Why? A neural network doesn’t automatically know the rules of addition. Its prediction depends on its current weights and biases.

The forward pass

A forward pass means sending inputs through the network to produce a prediction.

Each neuron multiplies its inputs by weights, adds a bias, and applies an activation function.

For example, a hidden neuron might calculate:

(2 × 0.5) + (1 × -1) + 0.5 = 0.5

If we use ReLU as the activation function, positive values stay unchanged and negative values become zero.

In our example, two hidden neurons each produce 0.5. The output neuron combines them:

Prediction = (0.5 × 0.8) + (0.5 × 0.4) + 0.1
= 0.7

The forward pass gives us a prediction. It does not update the weights.

Calculating the loss

Now we compare the prediction with the correct answer:

Correct answer = 3
Prediction = 0.7
Error = 0.7 − 3 = -2.3

Using squared-error loss:

Loss = (Prediction − Correct answer)²
= (0.7 − 3)²
= 5.29

Loss is a number that measures how wrong the prediction is. For this loss function, a smaller value means a closer prediction.

Learning from the mistake

Calculating loss alone doesn’t teach the network.

Next, backpropagation calculates gradients: how changes to each weight and bias would affect the loss. An optimizer uses those gradients to update the parameters.

Training repeats this process across many examples:

Make a prediction
Calculate the loss
Calculate gradients
Update weights and biases
Repeat

One example isn’t enough to show that the network has learned addition. We also need to test it on input pairs it hasn’t trained on.

The key distinction

Forward pass: What does the network predict?

Loss calculation: How wrong is that prediction?

Training: How should its weights and biases change?

That’s the foundation of neural network training.

For more such posts, visit EngineeringDepth.com.

Follow Planet AI & Tech for simple explanations of AI and technology.

Neural networks don’t understand words, sentences, or punctuation. They only understand numbers.

Ranuj Mahajan — Sun, 13 Sep 2026 09:00:58 +0000

Core Definition: Tokenization is the process of breaking raw text into smaller discrete units—called tokens—which can be words, subwords, characters, or symbols that a computer model can process.

Numerical Mapping: Once split, each unique token is assigned a numerical identifier (Token ID) using a predefined vocabulary, allowing neural networks to perform mathematical operations on text.

Modern Standard (Subword Tokenization): Most large language models use subword algorithms (such as Byte-Pair Encoding or WordPiece) to balance vocabulary size, efficiently handle compound words, and avoid "out-of-vocabulary" errors for rare or unseen terms.

Neural networks don’t understand words, sentences, or punctuation. They only understand numbers and vector spaces.

Ranuj Mahajan — Sun, 13 Sep 2026 08:56:19 +0000

Visual breakdown: How raw text moves through the LLM ingestion pipeline (Tokenize Embed Position Attend)

Ranuj Mahajan — Sun, 13 Sep 2026 08:45:05 +0000

Liquid syntax error: Tag '{% %}' was not properly terminated with regexp: /\%\}/

How Embeddings are generated in AI

Ranuj Mahajan — Sat, 12 Sep 2026 13:00:51 +0000

Why Most Engineers Struggle with System Design (And How Visuals Fix It)

Ranuj Mahajan — Fri, 11 Sep 2026 10:57:02 +0000

Most engineers prepare for system design interviews by reading 600-page textbooks or memorizing generic bullet-point summaries.

Then, they get hit with an open-ended interview question like:

"Design a globally distributed URL shortener handling 50k writes per second."

Suddenly, the disconnected facts freeze up. You know what a load balancer is. You know what Redis does. But visualizing how data moves end-to-end under load is an entirely different skill.

The Flaw with Text-Heavy System Design
Text explains definitions, but distributed architecture is all about flow and trade-offs:

How request routing navigates an API Gateway and Load Balancer before touching application instances.

Where decoupled workers write back output after an asynchronous queue is triggered.

How cache invalidation propagates when a write hits the primary database.

When you only memorize terms, you miss the mental map of where bottlenecks emerge.

System Design You Can See
To design resilient architectures, you need to build intuition visually:

End-to-End Tracing: Follow a packet from client to CDN, load balancers, caches, and sharded storage tiers.

Visualizing Trade-offs: See the direct cost of consistency vs. availability, or horizontal scaling vs. vertical read replicas.

Reusable Blueprints: Move beyond ad-hoc diagrams to structured visual mental models that apply to real-world architectures (like Netflix, Uber, or WhatsApp).

Level Up Your Architecture Intuition
If you want to stop getting lost in abstract jargon and start designing systems with confidence, check out EngineeringDepth.

We teach system design visually—giving you clear, practical diagrams, lessons, and deep-dive architectural breakdowns designed for modern engineers and tech interviews.

👉 Explore the visual course at engineeringdepth.com

What’s the hardest concept in distributed systems for you to visualize? Let’s discuss in the comments below!

Basics of the RAG

Ranuj Mahajan — Mon, 03 Aug 2026 11:20:10 +0000

Artificial Intelligence becomes much easier to understand when you see the flow visually.

Ranuj Mahajan — Wed, 29 Jul 2026 14:00:54 +0000

How does global Load balancer works

Ranuj Mahajan — Tue, 07 Oct 2025 14:58:42 +0000

How does Load Balancer works?

Load balancer uses an algorithm to forward the request to the different based on some filters.

CPU based algorithm
round robin technique
Load balancer also keeps the heart beat to check if services connected to it are working fine.
global load balancers distributes the request based on IP and geographical locations.

![ ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t9r3klrv9627ue7yqdyn.png

Mardown to the PDF converter.

Ranuj Mahajan — Mon, 15 Sep 2025 15:41:27 +0000

https://pdffrommarkdown.com is the place where you can convert your markdown files to pdf .
you can give your custom names to the pdf and you can encrypt them with password as well.

Improve your understanding of SOLID principles in OOPS.

Ranuj Mahajan — Mon, 26 Aug 2024 07:38:02 +0000

What Does Solid principles mean ?

S-> Single responsibility principle : it states that a class or an object in the OOPS programming realm should carry only one responsibility. if there are actions that are taken by student
pay for the fees
enrolment to new course
subjects studying

Assume we can create a class with all the responsibility encapsulated in student class, but that would make our code complex and difficult to maintain.

we have to separate the concerns and make sure that each class is only responsible for one task . this way each class will perform task.

O-> Open and closed principle :
This principle states that we should not change existing functionality or program in the class. we can extend our functionality of the class, but we shouldn't be able to modify it.

this looks really easy while we write it in theory, but we have to think of creating interface and make sure that we even if new classes are added, they implement from the interface and are able to execute the existing core logic.

Why do we need a NoSQL Databases

Ranuj Mahajan — Tue, 23 Jul 2024 13:16:57 +0000

In this post, I would explain why do we need NoSQL databases from a developer perspective.

Developer Friendly : NoSql databases are developer friendly, we can store JSON that is generated from the UI to the database, there is no transformation required at the application server end. this data can be read through a key from the database.
Data schema: we can store Json documents of different format in the collection. we dont have to comply to the any data schema. data of different schema's can be stored in the collection. this ensures that a very less transformation or no transformation is required for the data retrieval and data storage.
NoSql horizontal scaling : it can store huge amount of data. Nosql data can be scaled out easily. as data grows, we can add more and more servers to the Nosql databases.