DEV Community

Cover image for Pytorch for Neural Networks Part 8: Training with Multiple Inputs
Rijul Rajesh
Rijul Rajesh

Posted on

Pytorch for Neural Networks Part 8: Training with Multiple Inputs

#ai #machinelearning

In the previous article, we began the optimization loop using the first input value.

In this article, we will continue the same process for the remaining inputs.

Processing the Second Training Point

We start by selecting the input dose and the label for the second point in the training dataset.

Next, we run this second input through the model to get a predicted output.

Calculating the Loss

Now, we calculate the loss, which in this case is the squared residual between the predicted value and the observed value.

Calculating the Derivative

After calculating the loss, we call:

loss.backward()
Enter fullscreen mode Exit fullscreen mode

This calculates the derivative of the loss function with respect to final_bias.

However, there is something important to understand here.

loss.backward() does not replace the derivative from the previous training point.

Instead, it adds to it.

This means:

  • The derivative from the first point is remembered.
  • The derivative from the second point is calculated.
  • Both derivatives are added together.

In other words, loss.backward() accumulates derivatives each time we go through the nested loop.

After this, we add the new loss value to total_loss, just like before.

Processing the Third and Final Input

Now, let us process the third and final training point.

Again, we calculate the squared residual for the last point.

When we call:

loss.backward()
Enter fullscreen mode Exit fullscreen mode

PyTorch adds the derivative for this final point to the derivatives from the previous two points.

Then, we add this squared residual to total_loss.

At this stage, we are done processing all the training points for one epoch.

Checking Whether Training Should Stop

After the loop finishes, we check whether total_loss has become very small.

for epoch in range(100):
    total_loss = 0

    for iteration in range(len(inputs)):
        input_i = inputs[iteration]
        label_i = labels[iteration]

        output_i = model(input_i)

        loss = (output_i - label_i) ** 2

        loss.backward()

        total_loss += float(loss)

    if total_loss < 0.0001:
        print("Num steps: " + str(epoch))
        break
Enter fullscreen mode Exit fullscreen mode

If total_loss becomes very small, it indicates that the model fits the training data well.

At that point, we can stop training.

The break statement exits the optimization loop and ends the training process.

However, if the loss is still too large, there are a few more steps we need to perform.

We will explore those in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Top comments (0)