Neural Networks

what it is -

• it is a system of connected nodes that learns patterns by adjusting the weights.
• it is a type of model used in machine learning.
• it is made by layers of nodes, where each layer can have multiple nodes.

Other things.

• node is a small unit.
• neural network is trained through books , websites and not just text data things like images, audio data too(everything becomes numbers/vectors so that the model can learn).
• but mainly used for images.
• node performs calculation and prediction done using these nodes.
• node are responsible for simple calculation in the network.
• node size/layers are fixed.
  • meaning number of layers, neurons per layer and how they are connected( the architecture of the network).
• weights and bias values changes through out training.
• the problem is processed parallelly within layers.
  • neurons in same layer computing simultaneously is called parallel.
  • the same input is sent to all neurons and they process it differently( basically with different weights).
  • as the layer increases the input is refined and transformed and the finally a output is created

steps in processing the questions.

• when we give a input.
  • the input is converted into vectors.
  • the first layer is used and the vectors are processed
  • the output = new set of numbers or vectors.
  • next, this output(above output) is taken processes again (more pattern)
  • going through each layers refines the output.
  • finally a output is generated.

Key Words in Neural Networks

• neurons == nodes small unit that performs math.
• Weights are nothing but tells how much influence one neuron's has on the next neuron.
• Connections are the links between the nodes.
  • the connection is also responsible for the importance of that specific node.
  • lets say A node, B node and C node, node A and B are connected to C, then after calculation of that node, the connection between those decided the importance, if connection between a and c is more weighted then it is given more importance in the output.
• Propagation function is the weighted sum + activation that happens inside the neuron as the data moves forward.
• learning Rule tells how the model updates the weights after prediction using loss and gradients.
• Learning rate tells the model how much the model updates the weights after the error.
• Hidden Layers middle layers that process and learn patterns.
• gradient is a number which tells how loss changes, if +ve add weights and vice versa.

Basic Flow in Neural Network.

1. Input (a)

2. For each connection:
   input × weight

3. At neuron:
   sum all inputs + bias → z

4. Apply activation:
   output = activation(z)

5. Pass output to next layer

6. Final output → compare with actual

7. Loss calculated

8. Backpropagation (error goes back)

9. Gradient Descent (update weights)

10. Repeat

Activation Function

• it is the math that is in the node.
• in each neuron, the output transforms, this happens in each layer and helping the network getting a meaningful output.
• activation function introduced non-linearity transformation.
• this avoids the problem of straight line (avoids linear limitation).
• each layer changes the output differently.
• this makes the model learn and understand complex pattern like images, speech and languages.

Loss Function

• measures the error.
• a function that tells you weather the output is wrong.
• this is at the output layer of the neural network.
• this is used only in the training phase and can be used for evaluation.
• input given -> prediction -> loss function(compares it with actual) -> loss calculated -> output.
• after each loss is calculated the weights are updated using backpropagation + gradient descent.

Backpropagation

• computes gradients(how error changes when weights are updated).
• process of sending error back to update the weights.
• the error flows backward through the layers.
• it is done using gradient descent. the gradient descent uses them to update weights.

Gradient Descent

• method to update the weights to reduce loss.
• basically it moves the weights to where the error reduces.
• the gradient tells which connection influenced the error and the weights are updated accordingly(all the connection's weights are updated accordingly).
• it computes gradient mathematically, which connection affect the error and change accordingly.

Types of Activation Function

ReLU → hidden layers
Sigmoid → binary output
Tanh → alternative hidden layer

ReLU - Rectified Linear Unit

• this is a activation function that keeps positive values and removes negative values.
  • this helps in faster computation(makes the functions simple by removing negatives)
  • not just by removing also because it is computationally simple.
  • negative values are ignored.
  • basically acts like filters.

If z < 0 → output = 0  
If z ≥ 0 → output = z

Sigmoid

• function that converts any values into 0 to 1 range.
• easy interpreting proper yes or no classification.
• converting of raw output to confidence score

f(x) = 1 / (1 + e^(-x))

Tanh - Hyperbolic tangent

• converts the output between -1 and 1.
• exponential-based function.
• centered at 0 meaning well balanced.
• better than sigmoid (negative values are considered).

tanh(x) = (e^x - e^(-x)) / (e^x + e^(-x))

Types of Neural Network

Feedforward Neural Network (FNN)

• the input flows in one direction there is no memory saved.
• this is the simplest form of neural network.
• Used in
  • basic classification
  • Regression

Convolutional Neural Network(CNN)

• best for images, can be used for others too.
• designed to process grid like data
• uses the grid of pixel values (numbers) to understand the image. key components of CNN
• input layer - gets the raw image data and passes it to the network.