DEV Community

Cover image for Pytorch for Neural Networks Part 5: Preparing the Model for Training
Rijul Rajesh
Rijul Rajesh

Posted on

Pytorch for Neural Networks Part 5: Preparing the Model for Training

#ai #machinelearning

In the previous article, we created input values for our neural network and tested it out.

In this article, we will start preparing for backpropagation so that we can handle cases where the final bias is unknown.

Creating a Trainable Version of the Neural Network

To begin, we will create a copy of our neural network.

Let us call it MyBasicNN_train.

We name it this way because this version of the neural network will be trained.

class MyBasicNN_train(nn.Module):
    def __init__(self):
        super().__init__()

        self.w00 = nn.Parameter(torch.tensor(1.7), requires_grad=False)
        self.b00 = nn.Parameter(torch.tensor(-0.85), requires_grad=False)
        self.w01 = nn.Parameter(torch.tensor(-40.8), requires_grad=False)

        self.w10 = nn.Parameter(torch.tensor(12.6), requires_grad=False)
        self.b10 = nn.Parameter(torch.tensor(0.0), requires_grad=False)
        self.w11 = nn.Parameter(torch.tensor(2.7), requires_grad=False)

        self.final_bias = nn.Parameter(
            torch.tensor(0.0),
            requires_grad=True
        )
Enter fullscreen mode Exit fullscreen mode

Making the Final Bias Trainable

Here, we initialize final_bias with a value of 0.0.

We also set:

requires_grad=True
Enter fullscreen mode Exit fullscreen mode

for final_bias.

This tells PyTorch that this parameter should be optimized during training.

Unlike the other weights and biases, which remain fixed, the final bias will be updated using backpropagation.

Visualizing the Untrained Model

Now, let us visualize what this untrained model looks like.

This time, we will use MyBasicNN_train instead of MyBasicNN.

model = MyBasicNN_train()

output_values = model(input_doses)

sns.set(style="whitegrid")

sns.lineplot(
    x=input_doses,
    y=output_values.detach(),
    color="green",
    linewidth=2.5
)

plt.ylabel("Effectiveness")
plt.xlabel("Dose")
Enter fullscreen mode Exit fullscreen mode

Notice that we use:

output_values.detach()
Enter fullscreen mode Exit fullscreen mode

This separates the tensor from PyTorch’s gradient tracking system so that we can safely plot the values.

The result looks like this:

Why Training Is Needed

In the original graph, we had:

Effectiveness = 1 when Dose = 0.5

which is the correct value.

However, in this new graph, we get:

Effectiveness = 17 when Dose = 0.5

which is clearly far too high.

This tells us that the final bias is not correct and needs to be optimized.

To fix this, we will train the neural network.

Training Data

To train the model, we need two things:

1. Input values

We will use three input doses:

0.0, 0.5, 1.0
Enter fullscreen mode Exit fullscreen mode

2. Labels

These are the correct output values we want the model to learn:

0.0, 1.0, 0.0
Enter fullscreen mode Exit fullscreen mode

With these inputs and labels, we are ready to start optimizing the final bias.

In the next article, we will explore how to do this using backpropagation.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Top comments (0)