DEV Community

Cover image for Pytorch for Neural Networks Part 7: Training with Loss and Derivatives
Rijul Rajesh
Rijul Rajesh

Posted on

Pytorch for Neural Networks Part 7: Training with Loss and Derivatives

#ai #machinelearning

In the previous article, we explored concepts such as total loss and epochs.

Now, we will continue with the training process.

for epoch in range(100):
    total_loss = 0

    for iteration in range(len(inputs)):
        input_i = inputs[iteration]
        label_i = labels[iteration]

        output_i = model(input_i)

        loss = (output_i - label_i) ** 2

        loss.backward()

        total_loss += float(loss)
Enter fullscreen mode Exit fullscreen mode

Running Through the Training Data

We start with a nested for loop.

This loop runs each data point from the training dataset through the model and calculates the total loss.

The inner loop starts with the first training point and determines:

  • the input value (or dose)
  • the known output value (or effectiveness)

We store these values in:

input_i
label_i
Enter fullscreen mode Exit fullscreen mode

Getting the Predicted Output

Next, we run the input dose through the neural network:

output_i = model(input_i)
Enter fullscreen mode Exit fullscreen mode

This gives us the predicted output from the model.

Calculating the Loss

Now, we calculate the difference between the predicted output and the known label using a loss function.

In this example, we use the squared residual, which is simply the square of the difference between the predicted value and the known value.

loss = (output_i - label_i) ** 2
Enter fullscreen mode Exit fullscreen mode

For example, if:

  • Predicted output = 0
  • Known label = 0

then:

(0 - 0)^2 = 0
Enter fullscreen mode Exit fullscreen mode

So the loss would be 0, which means the prediction is perfect.

Calculating Derivatives with Backpropagation

After calculating the loss, we use:

loss.backward()
Enter fullscreen mode Exit fullscreen mode

This calculates the derivative of the loss function with respect to the parameters we want to optimize.

In our case, this helps PyTorch determine how the final bias should change to reduce the loss.

Tracking the Total Loss

Finally, we add the loss value to total_loss:

total_loss += float(loss)
Enter fullscreen mode Exit fullscreen mode

This allows us to keep track of how well the model fits the entire training dataset during each epoch.

We have only gone through the first training point so far.

We still need to repeat this process for the remaining training points.

We will continue that in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Top comments (0)