DEV Community: SoftarexTechnologies

HOW DEEP IS YOUR NETWORK? FIGURING OUT HOW DEEP NEURAL NETWORKS WORK

SoftarexTechnologies — Fri, 26 Jun 2020 13:46:57 +0000

Every day we are facing AI and neural network in some ways: from common phone use through face detection, speech or image recognition to more sophisticated — self-driving cars, gene-disease predictions, etc. We think it is time to finally sort out what AI consists of, what neural network is and how it works.

In the beginning was the Artificial Neural Network

In general terms, an Artificial Neural Network (ANN) is a technology for pattern recognition and the passage of input through various layers of simulated neural connections. It was inspired by the human brain and the way it works.

In its simplest form, an ANN can have only three layers of neurons: the input layer (where the data enters the system), the hidden layer (where the information is processed) and the output layer (where the system decides what to do based on the data).

An ANN that is made up of more than three layers – i.e. an input layer, an output layer, and multiple hidden layers – is called a ‘deep neural network’, and this is what underpins deep learning. A deep learning system is self-teaching, learning as it goes by filtering information through multiple hidden layers, in a similar way to humans.

Deep Neural Networks (DNN) is a neural network with a certain level of complexity — more than two layers. DNN combines simple mathematical operations in layers, which allows expressing more complex dependencies through them. And as the depths increase, complexity and abstraction level increase too, i. e. the data is processed in more complex ways.

How Neural Network Works

A neural network is a directed graph of nodes connected by synaptic and activation connections, which is characterized by the following properties:

Each neuron is represented by a set of linear synaptic connections, and, possibly, by a nonlinear activation connection.
Neuron synaptic connections are used to weigh the corresponding input signals.
The weighted sum of the input signals determines the induced local field of each particular neuron.
Activation links modify the induced local field of the neuron, creating an output signal.

*The neural network is trained on examples, making the input-output correspondence table for a specific task defined by data.

From a mathematical point of view, learning neural networks is a multiparameter problem of nonlinear optimization.

The signal propagates from the input layer to the output layer of the neural network and collides with parameterized transformations. Deep learning algorithms are contrasted to shallow learning algorithms by the number of these transformations. It is believed that deep learning is characterized by several non-linear layers (> 2).

Thus, deep learning of neural networks is machine learning algorithms for modeling high-level abstractions using numerous non-linear transformations.

Deep training solves the central problem of teaching performances. It introduces representations that are expressed in terms of simpler representations obtained at lower levels. Thus, deep learning allows a computer to build complex concepts from simpler ones. A typical example is a deep direct distribution network, or a multilayer perceptron (MLP).

It wasn't smooth at first

For a long time in neural networks application there were problems with the DNN training. First of all, it was determined by the problem of vanishing gradients and exploding gradients. These problems were solved after the following innovations:

Increasing the size of data sets to several TB.
Increasing the size of the models.
The use of massively parallel computing with GPGPU for training and increasing hardware performance.
The use of piecewise linear functions as a nonlinear connection, for example, rectified linear units (ReLU).

Gradient descent algorithms with adaptive learning speed (AdaDelta, AdaGrad, RMSPror, Adam).
Development of regularization methods in neural networks: ** Convolutional networks, where a priori knowledge of the input data is determined by regularization (spatial image relations) ** Dropout

** Normalization of mini-batch

The method of constructing the architecture Network-In-Network.
Dimensionality reduction via 1x1 convolutions in Convolution networks.
Development of residual learning method (deep residual learning)

All this made it possible in the end to build neural networks with sizes greater than 150 layers — now the size is unlimited and can reach thousands of layers.

Updates are coming

Based on the general definition of DNN, they can be used to solve any issues where artificial neural networks were previously used. However, DNN shows significantly better results and opens up more opportunities.

Using DNN methods in Computer Vision it is possible to create an application that will analyze different sports games and give detailed statistics about a player's performance.

All the analytics will be made via DNN video recognition algorithms. Such a system is capable to process and transform data into required stat: players/ball speed, expected goals, successful entries, failed passes, and etc. Recognizing both individual player movements with a ball and a whole team performance is not a problem thanks to DNN. It can even recognize players by numbers on their jerseys.

Having a good coach under a team’s belt is really good, but when they are empowered with such an analytical system — isn’t it a double-threat, huh?

Or it is possible to develop a system for manufacturing needs that will define fragments in a video stream with required objects to gather the necessary information. For example, such a system can:

Monitor and control different moving parts of assembly lines in real-time;
Collect information about particular coordinates of selected devices parts from a camera video stream;
Сheck the state of any mechanical system, people movements, and gestures in real-time;
Measure the coordinates of the necessary fragments or parts of the required machines or use DNN to process information from the video streams.

These are just brief examples of DNN capabilities. If we dig deeper, there probably wouldn't be any boundaries for them.

We really mean to learn

As DNN “grows stronger” it soon will be almost at any sphere of our life, we suppose. And since modern technologies developed in the way to understand humans and their needs, it's a good point to learn and understand how these technologies work. As Immortal Technique once said — "you never know" (Hi, Skynet).

Keen to learn about some more DNN use examples? We have them! Check out the system which remotely measures the mass of homogeneous products in real-time. Or the real example of a DNN use in CV — the system for valuing sports players' performance and helping them to improve their trainings.

We are always eager to share our best practices and wide open to learn something new, so if you have any questions or ideas – fill free to write to us. Let’s develop the world together!

Computer Vision Analysis of the Dynamic Object State

SoftarexTechnologies — Mon, 08 Jun 2020 15:25:03 +0000

If you were following close to our blog then you likely to know quite a lot already about modern Computer vision, AI and its capabilities. If not, then it’s probably the time you should start – the world of IT develops too fast and it’s better to hold your finger on its pulse in case not to fall out of life.

At the previous articles, we’ve discussed how Computer Vision can help to improve the way we know the sport and our approach to it. Here we’ll go into the details about soccer especially.

Landmark Localization - Computer Vision Analysis

The object’s state in a video can be described by the key points. These points are called landmarks and give great insight into the analyzed object structure. For example, face landmarks can be very useful for a wide range of applications like face recognition, face animation, emotion recognition, blink detection, photography, and more.

It also applies to body landmarks — they too have a wide range of applications: recognition of person gestures and posture, contactless game controllers interaction, environment interactions in AR-applications, and sports analytics – sportsmen movements.

Object Landmark Localization, or Object Alignment, is the process of a set of key points extrapolation from a given object image. For face alignment, we are only interested in the landmarks that describe the shape of face attributes like eyes, eyebrows, nose, mouth, and chin. Or it can be the shape, for example, of a body: head, shoulders, elbows, hands, hips, knees, and feet.

The main methods of Computer Vision Analysis

There are a lot of methods that are able to detect these points but let’s take a closer look at the main three of them:

The first one achieves superior accuracy and robustness by analyzing a 3D face model extracted from a 2D image;
The second relies on the power of CNN's (Convolutional Neural Networks) or RNN’s (Recurrent Neural Networks);
The third one - ERT – utilizes simple, but fast features to estimate the location of the points.

Dlib, a famous machine learning library written in C++, offers the third one. It implements a wide range of algorithms that can be used either on the desktop or mobile platforms. The Face Landmark Detection algorithm offered by Dlib is an implementation of the Ensemble of Regression Trees – ERT.

ERT is a cascade of high capacity regression functions learned via gradient boosting. This technique uses pixel intensities differences, a simple and fast feature, to directly estimate the landmark positions. These estimated positions are subsequently refined with an iterative process, which is done by a cascade of regressors and learning through gradient boosting.

The regressors produce a new value from the previous one, trying to reduce the alignment error of the estimated points at each iteration. The algorithm is really fast – it takes from 1 to 3 ms on a desktop platform to align a set of 68 landmarks.

Basically, a shape predictor can be generated from a set of images, annotations, and training options. A single annotation consists of an object region and labeled points, that we want to localize. An object region can be easily obtained by any detection algorithm, but points have to be labeled only manually.

Haar Cascade – HOG in combination with SVM detector, or single CNN detector, can automatically detect such objects as people or their faces. This stage produces a bounding box around the object.

We can solve the problem of high-precision analysis of soccer players' movements during training and practice, using this approach in sports analytics.

To do this, we need to set a range of key points for a human body and points for a ball, as at the principle above. A set of marking key points is formed based on training videos. A base of 300 tagged images is enough for a high precision point recognition.

The next step is teaching the ERT model and analyzing the accuracy of execution. If the analysis is not correct enough, it is necessary to expand the data set and adjust the parameters of the training model.

A fully trained model processes new videos in two steps:

The first step – the object detection model forms a bounding box around the analyzed objects – the player and the ball, on the new frame.

Second – the ERT model calculates the coordinates of key points inside this bounding box.

By connecting these key points we get the skeleton of the player and the position of the ball at the moment of impact.

The movement trajectory of each point of a player’s body and a ball is formed for the entire video. The model also calculates the parameters of their speed throughout the entire video and acceleration at specific time horizons.

Thanks to the received information, coaches can make adjustments to training or exercises to improve players' performance.

Looking ahead

As you can see there are almost endless opportunities of using Computer Vision analysis in soccer because tracking and analyzing winger’s, forward’s or full-back’s stats are just a beginning. It also can be used to track a goalkeeper’s performance, measure the speed of stricken balls and more. We have several projects based on Computer Vision.

This case is just one of the many — don’t hesitate to learn more about our experience. We are always eager to share our best practices and wide open to learn something new, so if you have any questions or ideas — fill free to write to us. Let’s develop the world together!