Understanding Multiple Input and Output Neural Network

Hello, I'm Ganesh. I'm building git-lrc, an AI code reviewer that runs on every commit. It is free, unlimited, and source-available on Github. Star git-lrc on GitHub to help more developers discover the project. Do give it a try and share your feedback for improving the product.

In the previous article, we discussed how ReLU activation function works. Now let's see how multiple input and multiple output neural networks work.

How Multiple Input and Output Neural Network works

Until now, we worked with a neural network that had a single input and a single output. In real-world problems, we usually have multiple inputs and multiple outputs.

In this article, we will build a neural network with:

2 inputs (input1 and input2)
1 hidden layer with 2 neurons
3 outputs (output1, output2, output3)
ReLU as the activation function

Network Structure

input1 ──┐
         ├──► hidden_neuron1 ──┬──► output1
input2 ──┤                     ├──► output2
         └──► hidden_neuron2 ──┴──► output3

Each input is connected to each hidden neuron, and each hidden neuron is connected to each output neuron.

Forward Pass Equations

All weights and biases are assigned based on normal distribution.
Hidden Layer Calculation
For hidden neuron 1:

x1 = (input1 * w1) + (input2 * w2) + b1
y1 = ReLU(x1) = max(0, x1)

For hidden neuron 2:

x2 = (input1 * w3) + (input2 * w4) + b2
y2 = ReLU(x2) = max(0, x2)

Output Layer Calculation

For output1:

output1 = (y1 * w5) + (y2 * w6) + b3

For output2:

output2 = (y1 * w7) + (y2 * w8) + b4

For output3:

output3 = (y1 * w9) + (y2 * w10) + b5

Example with Numbers

Let's work through a concrete example.

Given inputs:

input1 = 2
input2 = 3

Assume the following initial weights and biases:

Hidden layer weights:
  w1 = 0.5,  w2 = -0.3,  b1 = 0.1
  w3 = -0.4, w4 = 0.8,   b2 = 0.2

Output layer weights:
  w5 = 0.6,  w6 = 0.7,  b3 = 0.1
  w7 = -0.5, w8 = 0.4,  b4 = 0.2
  w9 = 0.3,  w10 = -0.6, b5 = 0.0

Calculate hidden neuron values

Hidden neuron 1:

x1 = (2 * 0.5) + (3 * -0.3) + 0.1
   = 1.0 - 0.9 + 0.1
   = 0.2

y1 = ReLU(0.2) = max(0, 0.2) = 0.2

Hidden neuron 2:

x2 = (2 * -0.4) + (3 * 0.8) + 0.2
   = -0.8 + 2.4 + 0.2
   = 1.8

y2 = ReLU(1.8) = max(0, 1.8) = 1.8

Calculate outputs

Output 1:

output1 = (0.2 * 0.6) + (1.8 * 0.7) + 0.1
        = 0.12 + 1.26 + 0.1
        = 1.48

Output 2:

output2 = (0.2 * -0.5) + (1.8 * 0.4) + 0.2
        = -0.10 + 0.72 + 0.2
        = 0.82

Output 3:

output3 = (0.2 * 0.3) + (1.8 * -0.6) + 0.0
        = 0.06 - 1.08 + 0.0
        = -1.02

Final Results

output1 = 1.48
output2 = 0.82
output3 = -1.02

Why ReLU works here

Notice that x1 = 0.2 and x2 = 1.8 — both are positive, so ReLU passes them through unchanged.

If any x value were negative (say x1 = -0.5), then:

y1 = ReLU(-0.5) = max(0, -0.5) = 0

That neuron would contribute nothing to the outputs, effectively "turning off" and making the network sparse and efficient.

Matrix Representation

You can think of all the weights as a matrix of connections:

Hidden Layer (2x2 weight matrix + 2 biases):

  w1   w2   b1        w3   w4   b2
[ 0.5  -0.3  0.1 ]  [ -0.4  0.8  0.2 ]


Output Layer (3x2 weight matrix + 3 biases):

  w5   w6   b3
[ 0.6  0.7  0.1 ]   → output1

  w7   w8   b4
[-0.5  0.4  0.2 ]   → output2

  w9  w10   b5
[ 0.3 -0.6  0.0 ]   → output3

Each output neuron learns a different combination of the hidden neuron outputs, allowing the network to produce multiple independent predictions.

Conclusion

We now understand how a neural network with 2 inputs and 3 outputs works step by step using the ReLU activation function:

Each hidden neuron receives all inputs, computes a weighted sum plus bias, and applies ReLU.
Each output neuron receives all hidden neuron outputs and computes its own weighted sum plus bias.
The network can produce multiple distinct outputs simultaneously from the same inputs.