DEV Community: Rijul Rajesh

Pytorch for Neural Networks Part 10: Completing Training and Verifying the Results

Rijul Rajesh — Wed, 10 Jun 2026 19:17:41 +0000

In the previous article, we completed the implementation for optimizing final_bias using gradient descent.

In this final part, we will run the code and observe what happens before and after optimization.

Before Optimization

When we first run the code, the value of final_bias is:

0.0

This is the starting value before any optimization takes place.

If we graph the model at this stage, we get the following result:

As we can see, the model does not fit the training data very well.

The predictions are far from the values we want, which means the loss is relatively large.

Watching Gradient Descent Update the Bias

As the training loop runs, we can observe final_bias changing at each step of gradient descent.

With every epoch:

The model calculates the loss.
Backpropagation computes the derivatives.
The optimizer updates final_bias.
The process repeats.

Over time, the model gradually moves toward a better value for final_bias.

After Optimization

After 34 steps, the total loss becomes very small.

At this point, the optimization process stops.

The final optimized value becomes:

-16.0019

Verifying the Result

Finally, we can verify that the optimized model now fits the training data correctly by graphing the updated outputs.

Using the graphing code from earlier, we get:

Now the model fits the training data much better.

This shows that gradient descent successfully optimized final_bias, allowing the neural network to learn the correct relationship from the training data.

So that is it for this small introduction to building and training neural networks with PyTorch.

We will explore more areas related to coding neural networks in the coming articles.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 9: Taking Steps Toward Better Predictions

Rijul Rajesh — Tue, 09 Jun 2026 19:14:51 +0000

In the previous article, we went through the optimization loop and passed all three training inputs through the model.

In this article, we will explore the additional steps we need to take when the total loss is not yet small enough.

Taking a Step Toward a Better Bias

If total_loss is still too large, we need to take a small step toward a better value for final_bias.

We do this using:

optimizer.step()

In the previous article, we saw that:

loss.backward()

calculates derivatives and stores them inside the model parameters.

The optimizer can then use these stored derivatives to determine the correct direction to update the parameter.

Clearing Old Derivatives

After updating the model, we need to clear the stored derivatives.

We do this using:

optimizer.zero_grad()

Here is the updated training loop:

for epoch in range(100):
    total_loss = 0

    for iteration in range(len(inputs)):
        input_i = inputs[iteration]
        label_i = labels[iteration]

        output_i = model(input_i)

        loss = (output_i - label_i) ** 2

        loss.backward()

        total_loss += float(loss)

    if total_loss < 0.0001:
        print("Num steps: " + str(epoch))
        break

    optimizer.step()
    optimizer.zero_grad()

    print(
        "Step: "
        + str(epoch)
        + " Final Bias: "
        + str(model.final_bias.data)
        + "\n"
    )

Why Do We Need `zero_grad()`?

We clear the derivatives because of how PyTorch works.

Earlier, we saw that:

loss.backward()

accumulates derivatives.

This means that if we do not clear them, then the next time we enter the loop, the new derivatives will be added to the old derivatives from the previous step.

That would lead to incorrect updates.

So after every optimization step, we reset the stored derivatives using:

optimizer.zero_grad()

Tracking the Final Bias

At the end of each loop, we print:

the current epoch number
the current value of final_bias

This allows us to track how final_bias changes during training.

The process continues until:

the total loss becomes very small, or
we finish all 100 epochs.

Printing the Final Optimized Bias

Once training is complete, we can print the final optimized value for final_bias.

print(
    "Final bias, after optimization: "
    + str(model.final_bias.data)
)

In the next article, we will see how this training process actually runs and how the value of final_bias changes over time, to get our final result from the script.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 8: Training with Multiple Inputs

Rijul Rajesh — Mon, 08 Jun 2026 19:24:32 +0000

In the previous article, we began the optimization loop using the first input value.

In this article, we will continue the same process for the remaining inputs.

Processing the Second Training Point

We start by selecting the input dose and the label for the second point in the training dataset.

Next, we run this second input through the model to get a predicted output.

Calculating the Loss

Now, we calculate the loss, which in this case is the squared residual between the predicted value and the observed value.

Calculating the Derivative

After calculating the loss, we call:

loss.backward()

This calculates the derivative of the loss function with respect to final_bias.

However, there is something important to understand here.

loss.backward() does not replace the derivative from the previous training point.

Instead, it adds to it.

This means:

The derivative from the first point is remembered.
The derivative from the second point is calculated.
Both derivatives are added together.

In other words, loss.backward() accumulates derivatives each time we go through the nested loop.

After this, we add the new loss value to total_loss, just like before.

Processing the Third and Final Input

Now, let us process the third and final training point.

Again, we calculate the squared residual for the last point.

When we call:

loss.backward()

PyTorch adds the derivative for this final point to the derivatives from the previous two points.

Then, we add this squared residual to total_loss.

At this stage, we are done processing all the training points for one epoch.

Checking Whether Training Should Stop

After the loop finishes, we check whether total_loss has become very small.

for epoch in range(100):
    total_loss = 0

    for iteration in range(len(inputs)):
        input_i = inputs[iteration]
        label_i = labels[iteration]

        output_i = model(input_i)

        loss = (output_i - label_i) ** 2

        loss.backward()

        total_loss += float(loss)

    if total_loss < 0.0001:
        print("Num steps: " + str(epoch))
        break

If total_loss becomes very small, it indicates that the model fits the training data well.

At that point, we can stop training.

The break statement exits the optimization loop and ends the training process.

However, if the loss is still too large, there are a few more steps we need to perform.

We will explore those in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 7: Training with Loss and Derivatives

Rijul Rajesh — Sun, 07 Jun 2026 20:21:14 +0000

In the previous article, we explored concepts such as total loss and epochs.

Now, we will continue with the training process.

for epoch in range(100):
    total_loss = 0

    for iteration in range(len(inputs)):
        input_i = inputs[iteration]
        label_i = labels[iteration]

        output_i = model(input_i)

        loss = (output_i - label_i) ** 2

        loss.backward()

        total_loss += float(loss)

Running Through the Training Data

We start with a nested for loop.

This loop runs each data point from the training dataset through the model and calculates the total loss.

The inner loop starts with the first training point and determines:

the input value (or dose)
the known output value (or effectiveness)

We store these values in:

input_i
label_i

Getting the Predicted Output

Next, we run the input dose through the neural network:

output_i = model(input_i)

This gives us the predicted output from the model.

Calculating the Loss

Now, we calculate the difference between the predicted output and the known label using a loss function.

In this example, we use the squared residual, which is simply the square of the difference between the predicted value and the known value.

loss = (output_i - label_i) ** 2

For example, if:

Predicted output = 0
Known label = 0

then:

(0 - 0)^2 = 0

So the loss would be 0, which means the prediction is perfect.

Calculating Derivatives with Backpropagation

After calculating the loss, we use:

loss.backward()

This calculates the derivative of the loss function with respect to the parameters we want to optimize.

In our case, this helps PyTorch determine how the final bias should change to reduce the loss.

Tracking the Total Loss

Finally, we add the loss value to total_loss:

total_loss += float(loss)

This allows us to keep track of how well the model fits the entire training dataset during each epoch.

We have only gone through the first training point so far.

We still need to repeat this process for the remaining training points.

We will continue that in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Identity Federation Explained: Understanding SSO, OAuth, OIDC, and SAML

Rijul Rajesh — Sat, 06 Jun 2026 20:14:00 +0000

Have you ever clicked "Sign in with Google" instead of creating a new account for a website?

If so, you've already used Identity Federation.

Identity Federation allows applications to trust a central identity source, known as an Identity Provider (IdP), to authenticate users. Instead of managing separate usernames and passwords for every application, users can sign in once and access multiple services.

Identity Federation in Action

Imagine your company uses:

Slack
Jira
Salesforce

Without identity federation, you would need separate credentials for each application.

With identity federation, you sign in using your company account, and the other applications trust that authentication. This creates a smoother and more secure user experience.

This capability is known as Single Sign-On (SSO).

Key Components

Identity Provider (IdP)
The system that verifies a user's identity. Examples include Microsoft Entra ID, Okta, Google Identity, and Keycloak.

Service Provider (SP)
The application a user wants to access, such as Slack, Jira, or Salesforce.

When an application claims to support federation, it means it can integrate with external Identity Providers and trust them for user authentication.

Common Federation Protocols

OAuth 2.0

OAuth 2.0 is an authorization framework that allows applications to access resources on a user's behalf without requiring their password.

For example, a photo-printing app can request access to your Google Photos account. After you approve the request, Google grants the application limited access without exposing your password.

OpenID Connect (OIDC)

OpenID Connect (OIDC) is built on top of OAuth 2.0 and adds user authentication.

When you click "Sign in with Google", OIDC helps the application verify who you are and obtain basic profile information, such as your name or email address.

SAML

Security Assertion Markup Language (SAML) is an XML-based standard used to exchange authentication information between an Identity Provider and a Service Provider.

SAML is commonly used in enterprise environments to enable Single Sign-On across multiple business applications.

Identity Federation enables organizations to centralize authentication and provide Single Sign-On across multiple applications.

The most common technologies involved are:

OAuth 2.0 for authorization
OIDC for authentication and identity information
SAML for enterprise Single Sign-On

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 6: Understanding Epochs and Loss

Rijul Rajesh — Fri, 05 Jun 2026 12:10:59 +0000

In the previous article, we prepared everything needed to optimize our neural network and find the ideal value for the final bias.

In this article, we will begin implementing the optimization process.

Creating the Optimizer

First, we create an optimizer object.

We will use Stochastic Gradient Descent (SGD) to optimize final_bias.

optimizer = SGD(model.parameters(), lr=0.1)

To optimize final_bias, we pass:

model.parameters()

to SGD.

PyTorch will automatically optimize every parameter where:

requires_grad=True

In our case, only final_bias has requires_grad=True, so that is the only parameter that will be updated during training.

Here, lr stands for learning rate, which is set to 0.1.

The learning rate controls how large each update step is during optimization.

Understanding Epochs

Before continuing, there is one important term we need to understand: epoch.

An epoch is one complete pass through the entire training dataset.

In this example, our training data contains 3 data points.

Every time all 3 training points are passed through the model once, we call it one epoch.

Running the Optimization Loop

We can now start the optimization process using a for loop that counts the number of epochs.

for epoch in range(100):
    ...

This loop will run the training process 100 times.

In other words, the model will see the full training dataset 100 times.

Tracking the Loss

Next, we initialize a variable called total_loss.

This stores the loss, which is a measure of how well the model fits the training data.

To better understand total_loss, let us look at an example.

In the figure above, the unoptimized model fits the training data poorly.

The residuals (the difference between what the model predicts and what we know is true) are large.

Because the residuals are large, the loss will also be relatively large.

Now imagine the model improves and fits the training data more closely.

The residuals become smaller.

In this case, the loss becomes smaller because the model predictions are closer to the correct values.

So, during each epoch, we use total_loss to keep track of how well the model fits the training data.

We will continue building the optimization process in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 5: Preparing the Model for Training

Rijul Rajesh — Thu, 04 Jun 2026 12:05:01 +0000

In the previous article, we created input values for our neural network and tested it out.

In this article, we will start preparing for backpropagation so that we can handle cases where the final bias is unknown.

Creating a Trainable Version of the Neural Network

To begin, we will create a copy of our neural network.

Let us call it MyBasicNN_train.

We name it this way because this version of the neural network will be trained.

class MyBasicNN_train(nn.Module):
    def __init__(self):
        super().__init__()

        self.w00 = nn.Parameter(torch.tensor(1.7), requires_grad=False)
        self.b00 = nn.Parameter(torch.tensor(-0.85), requires_grad=False)
        self.w01 = nn.Parameter(torch.tensor(-40.8), requires_grad=False)

        self.w10 = nn.Parameter(torch.tensor(12.6), requires_grad=False)
        self.b10 = nn.Parameter(torch.tensor(0.0), requires_grad=False)
        self.w11 = nn.Parameter(torch.tensor(2.7), requires_grad=False)

        self.final_bias = nn.Parameter(
            torch.tensor(0.0),
            requires_grad=True
        )

Making the Final Bias Trainable

Here, we initialize final_bias with a value of 0.0.

We also set:

requires_grad=True

for final_bias.

This tells PyTorch that this parameter should be optimized during training.

Unlike the other weights and biases, which remain fixed, the final bias will be updated using backpropagation.

Visualizing the Untrained Model

Now, let us visualize what this untrained model looks like.

This time, we will use MyBasicNN_train instead of MyBasicNN.

model = MyBasicNN_train()

output_values = model(input_doses)

sns.set(style="whitegrid")

sns.lineplot(
    x=input_doses,
    y=output_values.detach(),
    color="green",
    linewidth=2.5
)

plt.ylabel("Effectiveness")
plt.xlabel("Dose")

Notice that we use:

output_values.detach()

This separates the tensor from PyTorch’s gradient tracking system so that we can safely plot the values.

The result looks like this:

Why Training Is Needed

In the original graph, we had:

Effectiveness = 1 when Dose = 0.5

which is the correct value.

However, in this new graph, we get:

Effectiveness = 17 when Dose = 0.5

which is clearly far too high.

This tells us that the final bias is not correct and needs to be optimized.

To fix this, we will train the neural network.

Training Data

To train the model, we need two things:

1. Input values

We will use three input doses:

0.0, 0.5, 1.0

2. Labels

These are the correct output values we want the model to learn:

0.0, 1.0, 0.0

With these inputs and labels, we are ready to start optimizing the final bias.

In the next article, we will explore how to do this using backpropagation.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 4: Testing the Neural Network

Rijul Rajesh — Tue, 02 Jun 2026 19:30:55 +0000

In the previous article, we defined the forward pass for our neural network.

Now, we will provide inputs to the network so that we can test whether it works correctly.

Creating Input Values

To create a sequence of input values, we can define them like this:

input_doses = torch.linspace(start=0, end=1, steps=11)

Here, we use the PyTorch function linspace() to create a tensor containing 11 evenly spaced values between 0 and 1, including both endpoints.

The resulting tensor is stored in a variable called input_doses.

We can print input_doses to see what it looks like:

tensor([0.0000, 0.1000, 0.2000, 0.3000, 0.4000, 0.5000, 0.6000, 0.7000, 0.8000,
        0.9000, 1.0000])

Creating the Neural Network

Now, we need to create an instance of our neural network.

model = MyBasicNN()

Passing Inputs Through the Neural Network

Next, we pass the input values through the model.

output_values = model(input_doses)

By default, PyTorch automatically calls the forward() method that we defined earlier.

The outputs from the neural network are stored in the variable output_values.

Plotting the Results

Now, let us visualize the outputs using a graph.

For this, we will use Seaborn and Matplotlib.

import seaborn as sns
import matplotlib.pyplot as plt

sns.set(style="whitegrid")

sns.lineplot(
    x=input_doses,
    y=output_values,
    color="green",
    linewidth=2.5
)

Next, we set labels for the graph:

plt.ylabel("Effectiveness")
plt.xlabel("Dose")

This gives us the following result:

You can try out the code yourself by checking out this colab notebook

In the next article, we will explore a scenario where the final bias is unknown.

We will solve this problem by implementing backpropagation in code.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 3: Forward Passes

Rijul Rajesh — Mon, 01 Jun 2026 19:18:57 +0000

In the previous article, we defined an example neural network and started creating variables for the weights and biases using PyTorch.

In this article, we will continue further by exploring how to perform forward passes through the neural network.

For reference: This is the neural network which we are going to recreate.

So, lets begin!

Creating the First Calculation

We will start by creating input_to_top_relu.

def forward(self, input):
    input_to_top_relu = input * self.w00 + self.b00

Here, we take the input, multiply it by the weight w00, and add the bias b00.

This gives us the input for the first ReLU activation function.

Applying the ReLU Activation Function

Next, we pass input_to_top_relu through the ReLU activation function using F.relu().

def forward(self, input):
    input_to_top_relu = input * self.w00 + self.b00
    top_relu_output = F.relu(input_to_top_relu)

The F comes from the following import:

import torch.nn.functional as F

This module gives us access to activation functions such as ReLU.

Scaling the ReLU Output

Now we use top_relu_output to calculate scaled_top_relu_output.

Here, we simply multiply the output by another weight.

def forward(self, input):
    input_to_top_relu = input * self.w00 + self.b00
    top_relu_output = F.relu(input_to_top_relu)
    scaled_top_relu_output = top_relu_output * self.w01

Repeating the Same Process for the Bottom Path

We perform the same set of operations for the bottom part of the network.

def forward(self, input):
    input_to_top_relu = input * self.w00 + self.b00
    top_relu_output = F.relu(input_to_top_relu)
    scaled_top_relu_output = top_relu_output * self.w01

    input_to_bottom_relu = input * self.w10 + self.b10
    bottom_relu_output = F.relu(input_to_bottom_relu)
    scaled_bottom_relu_output = bottom_relu_output * self.w11

Combining Everything Together

Finally, we add the outputs from the top and bottom paths along with the final bias.

Then, we pass the result through one final ReLU activation function.

def forward(self, input):
    input_to_top_relu = input * self.w00 + self.b00
    top_relu_output = F.relu(input_to_top_relu)
    scaled_top_relu_output = top_relu_output * self.w01

    input_to_bottom_relu = input * self.w10 + self.b10
    bottom_relu_output = F.relu(input_to_bottom_relu)
    scaled_bottom_relu_output = bottom_relu_output * self.w11

    input_to_final_relu = (
        scaled_top_relu_output
        + scaled_bottom_relu_output
        + self.final_bias
    )

    output = F.relu(input_to_final_relu)
    return output

What We Have Built So Far

At this point, our neural network has two methods:

`init()`

This method is used to initialize the weights and biases.

`forward()`

This method performs a forward pass through the neural network.

It takes an input value and calculates the output using:

weights
biases
activation functions

Now that we have built the forward pass, the next step is to test whether everything works correctly by plugging in some input values.

We will explore that in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 2: Initializing Weights and Biases

Rijul Rajesh — Sun, 31 May 2026 20:11:34 +0000

In the previous article, we got started with expressing a neural network in the form of Python code.

In this article, we will continue building on that.

This is the neural network that we will recreate using code.

You can see the weights and biases shown in the diagram above. Let us now add them to our code.

We start with the basic neural network class:

class MyBasicNN(nn.Module):
    def __init__(self):
        super().__init__()

Our first weight has the value 1.70.

We can represent it like this:

class MyBasicNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.w00 = nn.Parameter(torch.tensor(1.7), requires_grad=False)

Here, we initialize a new variable called w00 and make it a neural network parameter.

When we define a weight as a parameter, PyTorch treats it as part of the neural network and gives us the option to optimize it during training.

Since this value is stored as a tensor, the neural network can take advantage of features such as:

automatic differentiation
accelerated mathematical operations

If you are unfamiliar with tensors, check out my earlier article on tensors.

Since we do not need to optimize this weight, we set:

requires_grad=False

requires_grad is short for requires gradient.

By setting it to False, we tell PyTorch that this parameter should not be updated during optimization.

In a similar way, we can define the rest of the weights and biases.

class MyBasicNN(nn.Module):
    def __init__(self):
        super().__init__()

        self.w00 = nn.Parameter(torch.tensor(1.7), requires_grad=False)
        self.b00 = nn.Parameter(torch.tensor(-0.85), requires_grad=False)
        self.w01 = nn.Parameter(torch.tensor(-40.8), requires_grad=False)

        self.w10 = nn.Parameter(torch.tensor(12.6), requires_grad=False)
        self.b10 = nn.Parameter(torch.tensor(0.0), requires_grad=False)
        self.w11 = nn.Parameter(torch.tensor(2.7), requires_grad=False)

        self.final_bias = nn.Parameter(torch.tensor(-16.), requires_grad=False)

Now that we have initialized the weights and biases, the next step is to create a forward pass through the neural network.

The forward pass defines how the input moves through the network using these weights and biases.

To handle this logic, we need to define another method called forward().

We will cover that in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Pytorch for Neural Networks Part 1: Writing Your First Neural Network in Pytorch

Rijul Rajesh — Fri, 29 May 2026 21:16:24 +0000

In my previous series of articles, we mainly explored the theory behind various neural network concepts.

In this new series, we will focus on putting that knowledge into practice using code. This will be a fun way to turn what we have learned into something more practical.

We will start with the basics and build things step by step.

For this article, we will be using the following modules.

Importing PyTorch

import torch

torch is used to create tensors, which store all the numerical data in neural networks, such as:

raw input data
weights
biases

import torch.nn as nn

This module helps us define and build neural network components.

It also allows us to make weights and biases part of the neural network.

import torch.nn.functional as F

This module gives us access to various activation functions and other useful operations.

from torch.optim import SGD

SGD, which stands for Stochastic Gradient Descent, is an optimization algorithm used to fit the neural network to data.

Creating a Neural Network

Now let us begin building our neural network.

When creating a neural network in PyTorch, we usually start by creating a class.

class MyBasicNN(nn.Module):

Here, we create a class named MyBasicNN.

This class inherits from a PyTorch class called nn.Module.

By inheriting from nn.Module, our class gains all the functionality needed to behave like a neural network in PyTorch.

Initializing the Neural Network

Next, we define the initialization method.

class MyBasicNN(nn.Module):
    def __init__(self):
        super().__init__()

Here, we define the constructor (__init__) for our neural network.

The line:

super().__init__()

calls the initialization method of the parent class nn.Module.

This ensures that all the necessary PyTorch functionality is properly set up for our neural network.

What Comes Next?

The next step is to initialize the weights and biases for our neural network.

Before doing that, we first need an example problem so we know what kind of neural network we want to build.

We will explore that in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Tensors Explained Part 2: Why Tensors Are Useful

Rijul Rajesh — Fri, 29 May 2026 03:52:37 +0000

In the previous article, we started with a brief introduction to tensors.

In this article, we will explore why tensors are useful.

Why Tensors Matter

Unlike normal scalars, arrays, matrices, and multi-dimensional matrices, tensors are designed to take advantage of hardware acceleration.

Tensors do not just store data in different shapes.

They are also designed to perform mathematical operations on that data efficiently and quickly.

Tensors and Hardware Acceleration

Tensors can take advantage of GPUs (Graphics Processing Units), which many of us use in our day-to-day devices.

GPUs are very good at performing many mathematical calculations in parallel, making them useful for training neural networks.

There is also specialized hardware called TPUs (Tensor Processing Units).

TPUs are specifically designed to work with tensors and help neural networks run even faster.

Automatic Differentiation

Another important use case of tensors is in backpropagation.

In neural networks, we estimate the optimal weights and biases using backpropagation.

This process requires calculating many derivatives and applying the chain rule.

Instead of manually calculating all these derivatives, tensor frameworks can handle this automatically using something called automatic differentiation.

This means that even as neural networks become more complex, tensors help manage the difficult mathematical calculations behind the scenes.

So that is it for tensors.

In the next article, we will explore another topic

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

DEV Community: Rijul Rajesh

Pytorch for Neural Networks Part 10: Completing Training and Verifying the Results

Before Optimization

Watching Gradient Descent Update the Bias

After Optimization

Verifying the Result

Pytorch for Neural Networks Part 9: Taking Steps Toward Better Predictions

Taking a Step Toward a Better Bias

Clearing Old Derivatives

Why Do We Need zero_grad()?

Tracking the Final Bias

Printing the Final Optimized Bias

Pytorch for Neural Networks Part 8: Training with Multiple Inputs

Processing the Second Training Point

Calculating the Loss

Calculating the Derivative

Processing the Third and Final Input

Checking Whether Training Should Stop

Pytorch for Neural Networks Part 7: Training with Loss and Derivatives

Running Through the Training Data

Getting the Predicted Output

Calculating the Loss

Calculating Derivatives with Backpropagation

Tracking the Total Loss

Identity Federation Explained: Understanding SSO, OAuth, OIDC, and SAML

Identity Federation in Action

Key Components

Common Federation Protocols

OAuth 2.0

OpenID Connect (OIDC)

SAML

Pytorch for Neural Networks Part 6: Understanding Epochs and Loss

Creating the Optimizer

Understanding Epochs

Running the Optimization Loop

Tracking the Loss

Pytorch for Neural Networks Part 5: Preparing the Model for Training

Creating a Trainable Version of the Neural Network

Making the Final Bias Trainable

Visualizing the Untrained Model

Why Training Is Needed

Training Data

1. Input values

2. Labels

Pytorch for Neural Networks Part 4: Testing the Neural Network

Creating Input Values

Creating the Neural Network

Passing Inputs Through the Neural Network

Plotting the Results

Pytorch for Neural Networks Part 3: Forward Passes

Creating the First Calculation

Applying the ReLU Activation Function

Scaling the ReLU Output

Repeating the Same Process for the Bottom Path

Combining Everything Together

What We Have Built So Far

__init__()

forward()

Pytorch for Neural Networks Part 2: Initializing Weights and Biases

Pytorch for Neural Networks Part 1: Writing Your First Neural Network in Pytorch

Importing PyTorch

Creating a Neural Network

Initializing the Neural Network

What Comes Next?

Tensors Explained Part 2: Why Tensors Are Useful

Why Tensors Matter

Tensors and Hardware Acceleration

Automatic Differentiation

Why Do We Need `zero_grad()`?

`init()`

`forward()`