DEV Community: AI Pool

Confidence Interval Understanding

AI Pool — Wed, 09 Jun 2021 17:12:20 +0000

Introduction

Suppose you are looking for a particular value of a characteristic of a population, firstly data is obtained from a sample of that population(could be the mean), and the parameter is guessed from the sampled data. Due to not using the whole population, one may ask if the sampled data is a reliable estimation of the whole population. This is where confidence interval comes in.

Confidence Interval

As earlier mentioned, the range of values that you expect your estimate to lie within, within a certain level of confidence is the confidence interval. For instance, if you say you have a 90% confidence level, it means you are very confident that 90% of the time, your estimate falls between the lower and upper bounds which are specified by that confidence interval.

There's a value called the alpha value, which indicates a threshold, for instance, an alpha of 0.1, which means that there's a less than 10% chance that the test could have occurred under the null hypothesis. Coming back to our confidence level, the desired confidence level is normally 1 minus this alpha value used in the statistical testing.

Therefore, if an alpha value of p < 0.1 is chosen for statistical significance, then the confidence level would ultimately be 1-0.1= 0.9 or 90%.

Confidence intervals are used for various statistical estimates such as

Proportions
Mean of a population
Differences that exist between the mean and proportions of a population
the variation estimates among groups

As the same suggests, these are all estimates of a number and do not give valuable information around that number, but the confidence interval is a good way to know more information about the variations of the number.

You can find more in the following article Confidence Interval Understanding

Other Resources

Normalization in Deep learning

AI Pool — Sat, 05 Jun 2021 08:21:02 +0000

Introduction

Deep learning is an exciting field in Artificial intelligence, it is at the forefront of the most innovative and exciting fields such as computer vision, reinforcement learning, and natural language processing. Deep learning has a complex architecture, which comes with some problems. These deep neural networks have tons of layers, which are difficult to train because they are responsive to the learning algorithm's initial random weights and configuration.

The input layer may have certain features which dominate the process, due to having high numerical values. This can create a bias in the network because only those features contribute to the outcome of the training. For example, imagine feature one having values between 1 and 5, and feature two having values between 100 and 10000. During training, due to the difference in scale of both features, feature two would dominate the network and only that feature would have a contribution to the outcome of the model.

Batch Normalization

Batch normalization is the most common form of normalization in deep learning. It scales the inputs to a layer to a common value for every mini-batch during the training of deep neural networks. This stabilizes the learning process and significantly reduces the number of epochs required to train deep networks, enabling the network to train faster.

The way batch normalization works are by calculating the mean and variance of every feature in the mini-batch, then the mean is subtracted and each feature is divided by the standard deviation of the mini-batch.

You can find more in the following article Normalization in Deep learning

Other Resources

Understanding of Probability Distribution and Normal Distribution

AI Pool — Mon, 31 May 2021 17:44:56 +0000

Introduction

Statistics is a key component in data science, which deals with gathering, analyzing, and drawing conclusions from data. An aspect of statistics is the probability distribution, which gives an idea of the likelihood of an event occurring, for example, there is an 80% chance of rain tonight.

Regarding probability, the common notation used is p(X), which means the probability that a random variable X is equal to a particular value, therefore p(X=0.8), in the example given, indicates that there's an 80% chance of X occurring. The sum of all probabilities should be equal to 1, therefore if there's a 0.8 chance of rain, then there's a 0.2 chance of no rain. Probabilities are also between 0 and 1. There are two types of probability distributions,

Discrete probability distribution
Continuous probability distribution

The following sections talk a bit more about both distributions

Visualization of different types of distribution

There are various types of a discrete probability distribution, some of which are

Poisson, for counting situations, such as the counts of televisions sold at a video store per week
Binomial for the binary situations, such as if it would rain or not
Uniform distribution for multiple situations that have the same probability such as a die roll

The following plot is a visualization of Normal Distribution

The normal distribution has certain characteristics which make it a bit easier to spot, some of which are:

The mean, median, and mode are equal
There is no skew(whether left or right), meaning 50% of the values are on the left of the mean, and the other 50% on the right
The mean and standard deviation are the key terms that characterize this

You can find more about Gaussian Distribution in the Following Article

Other Resources

AlexNet

AI Pool — Sat, 29 May 2021 16:01:14 +0000

ImageNet Classification with Deep Convolutional Neural Networks

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.

Paper

Implementations

In 2012, AlexNet significantly outperformed all the prior competitors and won the challenge by reducing the top-5 error from 26% to 15.3%. The second place top-5 error rate, which was not a CNN variation, was around 26.2%.

import keras
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D
from keras.layers.normalization import BatchNormalization
import numpy as np
np.random.seed(1000)
#Instantiate an empty model
model = Sequential()

# 1st Convolutional Layer
model.add(Conv2D(filters=96, input_shape=(224,224,3), kernel_size=(11,11), strides=(4,4), padding=’valid’))
model.add(Activation(‘relu’))
# Max Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding=’valid’))

# 2nd Convolutional Layer
model.add(Conv2D(filters=256, kernel_size=(11,11), strides=(1,1), padding=’valid’))
model.add(Activation(‘relu’))
# Max Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding=’valid’))

# 3rd Convolutional Layer
model.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding=’valid’))
model.add(Activation(‘relu’))

# 4th Convolutional Layer
model.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding=’valid’))
model.add(Activation(‘relu’))

# 5th Convolutional Layer
model.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), padding=’valid’))
model.add(Activation(‘relu’))
# Max Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding=’valid’))

# Passing it to a Fully Connected layer
model.add(Flatten())
# 1st Fully Connected Layer
model.add(Dense(4096, input_shape=(224*224*3,)))
model.add(Activation(‘relu’))
# Add Dropout to prevent overfitting
model.add(Dropout(0.4))

# 2nd Fully Connected Layer
model.add(Dense(4096))
model.add(Activation(‘relu’))
# Add Dropout
model.add(Dropout(0.4))

# 3rd Fully Connected Layer
model.add(Dense(1000))
model.add(Activation(‘relu’))
# Add Dropout
model.add(Dropout(0.4))

# Output Layer
model.add(Dense(17))
model.add(Activation(‘softmax’))

model.summary()

# Compile the model
model.compile(loss=keras.losses.categorical_crossentropy, optimizer=’adam’, metrics=[“accuracy”])

Author of the implementation: @engmrk

AlexNet

Other Resources

Recommendation Systems by Matrix Factorization and Collaborative Filtering

AI Pool — Sat, 29 May 2021 15:39:06 +0000

Introduction

We're all using any platform which has some items and these items are made for us, for users. In general, we use some of them and get suggestions for other items as well, but what suggestions are they, how do they even know what to suggest, which type of items each user likes and what is the probability, that they are going to like something platform suggests?

Recommendation systems are created just for this. Let's take Netflix as an example. It has a lot of movies and users watch them on their platform. When each user watches any movie, Netflix suggests other movies relevant to the user or the item, which the user has watched.

Nowadays almost all platforms like Facebook, Youtube, Twitter, Pinterest, Amazon, etc. use recommendation systems, to suggest relevant content to their users. It gives good growth in terms of revenue or other business metrics, which is important to the company.

Now let's formulate a recommendation system and show how it's going to suggest content relevant to users.

Recommendation Systems, Problem Formulation

Let's assume we have multiple users and movies. Each user has watched any movie and liked it. It means different movies can be liked by the same person and different users can like the same movie. It kind of like a matrix for users and items. This matrix will contain users and movies as rows and columns and each (i,j) position of the matrix will correspond to the rating of the movie or user has liked it or not. Something like this.

It is a sparse matrix. If the user has not watched the movie, then the corresponding place of this matrix will be undefined. Our goal is to find a model, which will fill all missing places in the matrix we showed above.

Find more about collaborative filtering and matrix factorization in the Following Article

Other Resources

Bias-Variance Tradeoff in Machine Learning

AI Pool — Sat, 29 May 2021 14:54:23 +0000

Hypothesis Space

Before speaking about bias and variance, let's understand what hypothesis set is and how we are going to define it. First of all, when you train a model, you are seeking a hypothesis function over the entire space. It does not matter you train a linear regression, logistic regression or a deep network, you always have to understand what a hypothesis set is and how you're going to find a function you are looking for.

If we create a model to approximate the given target function, it means we define a hypothesis set. Our trained model is a point from it, which can be far or close to the target function.

Now let's define our hypothesis set. Let's assume we have chosen a model, which defines hypothesis set H.

Bias-Variance Tradeoff

There is an optimal point, where bias and variance are in a good position and their values are reasonable. To find that optimal point, we need to draw the curves for every value, which depends on the complexity of the model. By saying the complexity of the model, we mean the complexity of the hypothesis set, the size of it.

Our goal is to minimize the total loss, which consists of bias, variance, and small noise. These curves show that increasing the complexity of the model, will decrease the bias, but the variance will increase and as a result, the total loss will be high.
We can't take a too simple model, which can't even approximate the target function and can't take too big one either, because it has high variance.

This is a known problem in the machine learning sphere, specifically in deep learning. Every specialist knows about Underfitting or High Bias and Overfitting or High Variance. These are the main problems everybody faces and there are a lot of approaches to fix them. People tried to solve this in the following ways.

Regularization
Model Selection / Early Stopping
Cross-Validation
Normalization Functions
Augmentation Techniques
Others

You can find more in the Following Article

Other Resources

Diving into Object Detection Basics

AI Pool — Sat, 29 May 2021 13:57:47 +0000

Intro

The prospects of Artificial Intelligence (AI) are not just limited to predicting if a person will get a loan or not by giving his credit history, annual income, annual expenses, criminal records, etc. Computer Vision is a trending topic for AI enthusiasts of any experience level.

Let me give you a brief idea about Computer Vision. It is nothing but when a machine identifies an 'object' in an image/video after learning from the data that was fed to it. For example, when we see an object first time, we are mostly aware of it like what it is called. But after getting to know its name, the next time we see it we know exactly what it is. Exactly like our brain, Computer Vision, to be specific Object Detection works.

Introduction of Object Detection

Object detection is locating and identifying an object in an image or in a video. Locating an object is nothing but giving the exact position where the object resides in the frame. (Here frame can be a single image or a sequence of frames that is a video). To locate an object, we can either use a bounding box or any other geometrical shape like a circle. The easiest and standard approach is by using the bounding box, where we first obtain the center coordinates (x, y) and the width (w) and height (h) of the box.

To identify an object, the network must be trained on data, for example, images of the person. This step is called the classification of objects and it is very essential for the bounding box to be formed correctly. To ensure the correct training of the network, ensure the data is correct.

Anchor Boxes

How does the network predict or identify the box?

The network first makes a random guess of the coordinates and assigns them a value w for width and h for height. It assigns (0, 0) for the center of the box (x, y). Of course, this is not the actual prediction. So after every step of training, which is termed as an iteration, the network performs regression to get the correct estimates.

Datasets to start with

There are many datasets to start training your first object detection model. These datasets are open source meaning anyone is free to use them. These datasets have a large collection of classes of objects to choose from. So have fun while exploring these datasets

COCO Dataset
ImageNet
Open Image Dataset V6
Labelme
CelebFaces
50 other datasets

You can find more in the Following Article

Other Resources

Introduction of Fast Fourier Transformation (FFT)

AI Pool — Sat, 29 May 2021 13:45:45 +0000

Introduction

Suppose you have an audio file, or even a video file, or let's say you have an image file. But the only problem is it has lots of noise. You can't distinguish between the required audio, frames, or the object in the three files respectively. So how are you supposed to distinguish between the noise and the actual signal?

The odds are quite high that the file may contain a term called noise. Though the terms are of the communication domain, they are also involved in the AI domain.

Noise

Noise is a random signal which consists of equal intensities or powers at every frequency. In computing, it is statistically defined as a sequence of random variables. So basically, in very simple terms, it is a random thing that may be a part of your signal.

Fourier Transformation (FT)

Fourier Transformation which is the main highlight of this article is a very useful tool for analyzing signals, especially noisy signals. It transforms or converts complex mathematical equations into simpler trigonometric functions in terms of sin or cos. Sin or Cos are used because the signal is easier to analyze in their format. In other terms, Fourier transformation is used to convert time signals into frequency signals and power signals. You are using the applications of Fourier Transformation unknowingly every day.

Implementation of FFT

Note: The codes are written and tested by the author. The outputs are the screenshots of Jupyter Notebook cells.

For this implementation, we will be using scipy library as a Fourier transformation calculator.

1. Let's import the libraries for python fft

import numpy as np
import matplotlib.pyplot as plt
from scipy.fftpack import fft, fftfreq

Note: If you have the latest package of scipy, use scipy.fft instead of scipy.fftpack

2. Defining a random signal

n = 500 # Number of data points
dx = 5.0 # Sampling period (in meters)
x = dx*np.arange(0,n) # x coordinates
w1 = 100.0 # wavelength (meters)
w2 = 50.0 # wavelength (meters)

fx = np.sin(2*np.pi*x/w1)
plt.title("Signal as a function of time")
plt.plot(x, fx)

3. Getting the Discrete Fourier Transform (DFT) using FFT

Fk = fft(fx)/n # Fourier coefficients (divided by n)

# To plot in the frequency domain
Fk =  fftshift(Fk) # Shift zero frequency to center
nu = fftfreq(n,dx) # Natural frequencies
nu = fftshift(nu) # Shift zero frequency to center

plt.title('DFT of the signal')
plt.plot(nu, np.abs(Fk))  # Get the absolute values of DFT

As you can see in the above image, the DFT of our signal is a simple graph in which we have a single pair of frequencies as I had mentioned earlier that DFTs are symmetrical. As the signal is a simple continuous signal and contains only one frequency component, it was expected to get a single pair of frequencies.

4. Adding a random noise

new_fx = np.sin(2*np.pi*x/w1) + 2*np.cos(2*np.pi*x/w2) 
# new = old + random signal/noise
plt.title('Modified Signal')
plt.plot(x, new_fx)

5. Getting the DFT for the modified signal

new_Fk = fft(new_fx)/n  # Fourier coefficients (divided by n)
new_Fk = fftshift(new_Fk)  # Shift zero frequency to center
plt.title('DFT of Modified Signal')
plt.plot(nu, np.abs(new_Fk))  # Get the absolute values of DFT

You can find more details with plotted images in this article