Understanding Backpropagation with Python Examples — Part 2

In the previous article – Part 1 of this series, we took the weights and biases, plotted our curve, and started computing the sum of squared residuals.

Let us now compare the predicted curve with the actual curve.

Try executing the following snippet. Refer to the Colab notebook.

def plot_predicted_vs_observed(b3):
    # Combined predicted curve
    y_combined = y_upper_final + y_lower_final + b3

    plt.figure()

    # Predicted curve
    plt.plot(
        x,
        y_combined,
        color="tab:green",
        label="Predicted (squiggle)"
    )

    # Observed points
    dosage_points = np.array([0.0, 0.5, 1.0])
    observed_values = np.array([0.0, 1.0, 0.0])

    plt.scatter(
        dosage_points,
        observed_values,
        color="black",
        zorder=5,
        label="Observed (0,1,0)"
    )

    plt.title(f"Predicted vs Observed (b3 = {b3})")
    plt.xlabel("Input (Dosage)")
    plt.ylabel("Output")
    plt.legend()
    plt.show()

b3 = 0
plot_predicted_vs_observed(b3)

This will compare the observed values 0, 1, 0 with the predicted values.

We will get the following result:

Now let us compute the sum of squared residuals, which is simply the squared errors summed together.

def compute_and_plot_ssr(b3_values):
    # Ensure iterable
    b3_values = np.atleast_1d(b3_values)

    # Dosage points and observed values
    dosage_points = np.array([0.0, 0.5, 1.0])
    observed_values = np.array([0.0, 1.0, 0.0])

    # Upper path (fixed)
    z_u = w1 * dosage_points + b1
    y_u = w3 * softplus(z_u)

    # Lower path (fixed)
    z_l = w2 * dosage_points + b2
    y_l = w4 * softplus(z_l)

    ssr_values = []

    for b3 in b3_values:
        # Combined prediction
        y_pred = y_u + y_l + b3

        # SSR
        residuals = observed_values - y_pred
        ssr = np.sum(residuals ** 2)
        ssr_values.append(ssr)

        print(f"b3 = {b3:.2f}")
        print("  Predicted:", y_pred)
        print(f"  SSR: {ssr:.2f}")

    # Plot all points
    plt.figure(dpi=200)
    plt.scatter(b3_values, ssr_values, s=80)
    plt.title("SSR vs Bias (b3)")
    plt.xlabel("Bias (b3)")
    plt.ylabel("Sum of Squared Residuals")
    plt.show()

    return np.array(ssr_values)

b3 = 0
compute_and_plot_ssr(b3)

We will observe the following:

Our goal is to make the sum of squared residuals as small as possible.

Let us plot multiple values for b3 and see how it behaves.

compute_and_plot_ssr([0, 1, 2, 4])

I tried plugging in b3 values of 0, 1, 2, and 4. Let’s see what happens.

We can see that it dips at a certain point and then goes back up.

We can create a curve from this and then find the ideal bias. We will explore this in the next part of the article.

Looking for an easier way to install tools, libraries, or entire repositories?
Try Installerpedia: a community-driven, structured installation platform that lets you install almost anything with minimal hassle and clear, reliable guidance.

Just run:

ipm install repo-name

… and you’re done! 🚀

🔗 Explore Installerpedia here