In the previous article, we defined an example neural network and started creating variables for the weights and biases using PyTorch.
In this article, we will continue further by exploring how to perform forward passes through the neural network.
For reference: This is the neural network which we are going to recreate.
So, lets begin!
Creating the First Calculation
We will start by creating input_to_top_relu.
def forward(self, input):
input_to_top_relu = input * self.w00 + self.b00
Here, we take the input, multiply it by the weight w00, and add the bias b00.
This gives us the input for the first ReLU activation function.
Applying the ReLU Activation Function
Next, we pass input_to_top_relu through the ReLU activation function using F.relu().
def forward(self, input):
input_to_top_relu = input * self.w00 + self.b00
top_relu_output = F.relu(input_to_top_relu)
The F comes from the following import:
import torch.nn.functional as F
This module gives us access to activation functions such as ReLU.
Scaling the ReLU Output
Now we use top_relu_output to calculate scaled_top_relu_output.
Here, we simply multiply the output by another weight.
def forward(self, input):
input_to_top_relu = input * self.w00 + self.b00
top_relu_output = F.relu(input_to_top_relu)
scaled_top_relu_output = top_relu_output * self.w01
Repeating the Same Process for the Bottom Path
We perform the same set of operations for the bottom part of the network.
def forward(self, input):
input_to_top_relu = input * self.w00 + self.b00
top_relu_output = F.relu(input_to_top_relu)
scaled_top_relu_output = top_relu_output * self.w01
input_to_bottom_relu = input * self.w10 + self.b10
bottom_relu_output = F.relu(input_to_bottom_relu)
scaled_bottom_relu_output = bottom_relu_output * self.w11
Combining Everything Together
Finally, we add the outputs from the top and bottom paths along with the final bias.
Then, we pass the result through one final ReLU activation function.
def forward(self, input):
input_to_top_relu = input * self.w00 + self.b00
top_relu_output = F.relu(input_to_top_relu)
scaled_top_relu_output = top_relu_output * self.w01
input_to_bottom_relu = input * self.w10 + self.b10
bottom_relu_output = F.relu(input_to_bottom_relu)
scaled_bottom_relu_output = bottom_relu_output * self.w11
input_to_final_relu = (
scaled_top_relu_output
+ scaled_bottom_relu_output
+ self.final_bias
)
output = F.relu(input_to_final_relu)
return output
What We Have Built So Far
At this point, our neural network has two methods:
__init__()
This method is used to initialize the weights and biases.
forward()
This method performs a forward pass through the neural network.
It takes an input value and calculates the output using:
- weights
- biases
- activation functions
Now that we have built the forward pass, the next step is to test whether everything works correctly by plugging in some input values.
We will explore that in the next article.
AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.
git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.
Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.
Give it a ⭐ star on Github
Top comments (1)
The way you've broken down the forward pass with explicit intermediate variables (input_to_top_relu, scaled_top_relu_output) is pedagogically brilliant—it makes the gradient flow paths crystal clear when learners eventually move to backprop. Most tutorials skip over why you'd want to expose these intermediate values, but seeing the explicit computation graph here really highlights why autograd can trace through each operation. This step-by-step approach builds intuition that generic frameworks sometimes obscure.