DEV Community: Antonio Triguero

Types of Software Licenses that you can use in your projects

Antonio Triguero — Wed, 30 Jun 2021 06:20:52 +0000

Deep Learning for Dummies #2 - Learning Process

Antonio Triguero — Sun, 27 Jun 2021 15:07:58 +0000

Internally, each neuron inside a neural network detect patterns or characteristics inside your data for feeding back to other neurons. I pretend to show you how neural networks can learn to solve a problem, and what is our tasks as developers in this learning process. First, we review typical mathematical concepts used for training neural networks and next, I will explain the algorithms that permit the learning task.
For understanding the content of this post, you should read my last article, where I explain in detail the machine learning and deep learning problem of how the base neural network's architecture is.
At this point, we know that a learning process consists in finding the weights of the net that better estimate the values of a function, but what is the real problem that we are solving? For answer this questions, we need to define some mathematical concepts of mathematical functions analysis.

Derivate of a function

At this point you could be thinking “WTF is this? I arrive to this post for learning deep learning and this guy start with a math class” and my answer is “Yes, but relax because you only need to understand the concepts because in a real world problem probably you will use neural networks and no research about these”.
Returning to the problem, the derivative of a function represent the tangent line that pass for this point, and it can be interpreted as slope. If you drop a ball at this point, the ball will go to the direction of the derivate, and this concept is the gradient.

Gradient

The gradient is a derivative of a function, but with direction and step magnitude. It is like the velocity of the ball, it always goes to the bottom of the line but with the velocity that you can set up. Mathematically this is more complicated, but the concept is sufficient.
And now we know how to calculate the derivate of function and the direction of the derivative, but if you look at each derivate of function, you will notice that each gradient point to a minimal point of a function. A minimal point is a point where the function return a low value. If this value is the lower, it will be the minimal value and if not, it will be a partial minimal value. And you think this more, you can implement an algorithm that can find any minimal point of a function from any point in a function moving in the direction of the gradient and this algorithm is the gradient descent.

Gradient Descent

Gradient descent is a mathematical algorithm that permit to find the minimal of a function moving in the direction of the gradient. The steps of this algorithm are:

Calculate the gradient at the actual point.
Look at the direction of the gradient.
Move actual point following this direction with a step size.
Execute step 1, 2 and 3 until a number of iterations. The process is illustrated in this image: This algorithm resolve an optimization problem in which you want to find a minimal value inside a function, but how can apply this method to network's learning process. So first, we need to define a function to minimize (cost function) and the network will learn the weights that minimize this function. This function link the outputs of the network with the value that we expect. Next, we calculate the gradient but for each weight of the network (this is back propagation because you need the next neurons gradients for calculating it) and finally, we advance on the direction of the gradient with a step size. This process is repeated over a number of epochs, and it's the process of training.

Deep Learning for Dummies #1 - Introduction

Antonio Triguero — Mon, 21 Jun 2021 13:59:06 +0000

Deep Learning is a state-of-art today for machine learning problems. With artificial neural networks, we automate tasks that we thought that only humans could do, but the future is now and technology is closer to humans with these algorithms. And the best feature is that engineers haven't care about code the logic for the computer, now, they can learn how to do tasks based on the data that we throw into the world. In this post, I explain what is deep learning and how it works.

The problem

The machine learning problems can be generally two types:

Classification: In this problem, the machine has to learn how to differentiate between some classes. It can be able to classification the input data and assign a class to each. For example, you introduce an image with a cat inside and it should learn to identify this image as a cat.
Regression: In this problem, the machine tries to predict a value from input data. This value can be any number and it should try to estimate this value. For example, if you introduce the last 100 days of SP500 close price, it should be able to predict the next day's value. There are other problems that you can find in the literature but these two are the typical problems.

The Maths and Neurons

Let's think about the regression problem. In this problem, you can think that the machine is learning to return an output value based on the input value. It is similar to a mathematical function where you introduce a series of numbers and it results in another number. For example, the next function:

It can be a function that given a value return the next number in the natural numbers ladder. But in the regression problem, you don't have the mathematical expression that links your input data (x) to your output data (f(x)) and your goal is to find this expression. But you think, how can you find this expression if you don't how it is? The answer is Taylor polynomials.

The Taylor polynomials have an expression like this:

And you can interpret it as a polynomial that estimates the real function value based on his inputs multiplied by weights. This expression can be larger and when larger, more precise in his estimations.
In deep learning, we are trying something like this but with a more complex approach because this expression is similar to the work process of a neuron in deep learning. The minimal computation step in deep learning is the neuron and it works following the next diagram:

This neuron is the most simple in deep learning and it takes some inputs and returns one output. The output is calculated based on a linear combination of his inputs. It is trying to approximate a function based on an addition of the inputs multiplied by weights and these weights are the values that a neuron will learn to approximate better this function.
With this neuron, you can try to approximate a function but the power of these neurons is magic where they are combined.

Neuronal Networks

A neuron is a good approach for estimate function but it works like a linear regression problem, but if the function that you should estimate is more complex than a line, what should you do? You can try to combine these neurons linking outputs as inputs of other neurons that build neural networks.
Let's think about a neural network of two neurons like this:

With this approach, you are encapsulating the value of the first neuron inside the function that the second neuron is trying to estimate. Mathematically it is something like this expression:

With this expression, we can estimate functions more complex and if you want to estimate more complex functions, you can append more neurons. But, if neural networks are famous, it is for their layers. The layers are neurons that are positioned at the same level inside the last chain of neurons. These neurons will be connected to the next layer of neurons building deep neural networks. These neurons can estimate functions with good performance and we can call them universal function approximators. An example is shown in the following image:

This architecture is the base of all neural network architectures and they can estimate functions better than the art-of-art machine learning algorithms with good data and good training.
In the next post, I will so you how these neural networks learn and how is the process of training.