DEV Community: Shiv Prasad

McCulloch Pitts Neuron

Shiv Prasad — Mon, 10 Aug 2020 16:47:31 +0000

An artificial neuron consists of multiple inputs x_1 , x_2 and x_3. Each of these inputs have their weights w_1 , w_2 and w_3 respectively. The neuron takes multiple inputs and weighted aggregates and applies a function to give an output.

The early model of an artificial neuron was proposed by Warren McCulloch and Walter Pitts.

In a simplified computational model of a neuron the inputs (x_i) are in boolean and the aggregates of which represent the function g(x). The function g(x) is passed into function f which gives the output 1 or 0 based on the threshold/parameter b.

Let us assume that a model predicts whether a person will like a smartphone or not. The wrong predictions made is measured by the loss function.

In a primitive algorithm like brute force search the threshold is assumed and predictions are made for different thresholds and the model is selected based on the lowest value of loss function.

In the graph we can see that threshold was increased from 1 to 10 and the lowest loss was at 5 where loss = 0 i.e correct predictions were made.

We can find the accuracy of the neuron with an assumed threshold of 5 by applying the model on the test data and finding the fraction of number of correct predictions to the total number of predictions in the test data.

The geometric interpretation of MP neuron is that it separates the outcome into two different regions. The region above the line gives a positive result and region below the line gives negative result. Sometimes the negative points lie above the line which makes the line inseparable which is a limitation. So zero loss is not possible.

The data is in boolean.
Only classification tasks can be performed
The model is linear and separates the outcome into two different regions.
The loss is never zero.
The parameter b can range between 0 to n and find loss near to or equal to 0.
The accuracy is the fraction of number of correct prediction to total predictions.

Vectors and Matrices

Shiv Prasad — Wed, 05 Aug 2020 13:53:34 +0000

Vectors are a collection of co-ordinates that a point has in a given space. They are defined by their magnitude and direction.

For n-dimensions there are n-coordinates in the given space.

Unit vector has a magnitude of one. To find the unit vector of any vector divide each element of the vector by the vectors length.

The projection of vector x on vector y can be found by dividing the dot product of x and y by the magnitude of the vector y and multiplied by vector y . The projection of vector x is in the direction of y vector.

This is an example on projecting vector x on vector y.

Above is an example to calculate the angle between two vectors.

Two vectors are said to be orthogonal when the angle between them is 90°. That is the dot product between the vectors should be zero.

Matrices :

Matrices are a collection of row vectors or column vectors. Data we deal are mostly is in the form of matrices. We manipulate these matrices to find relations and make predictions.

Here is a useful playlist :

https://www.youtube.com/playlist?list=PL2jykFOD1AWazz20_QRfESiJ2rthDF9-Z

Expert systems - 6 Jars

Shiv Prasad — Sun, 02 Aug 2020 16:59:25 +0000

In an expert system a human establishes the rules and hand codes the rules and the machine executes the rules. In machine learning the machine learns from the data curated by a human and uses functions and parameters to establish a relationship between input and output.

Let us split the machine learning process into 6 jars and see how these interact with each other to form the complete process of machine learning.

The basic idea of machine learning is to find a function that depends on the features of the given data. It is used to find the output from a number of given inputs. A function can be a series of if-else statements, weights applied to different parameters / features or from a family of functions such as linear , polynomial or non-linear.

The machine learning algorithm tries all the functions and different parameters in the function and ultimately selects the one which best explains the relationship between input and output.

The domain of machine learning has been more successful because of abundant data that is available in different forms such as text, image, video, pictures, voice and tabular data.

The models and learning algorithms are already available open source for everyone to use.

The hardware limitations for computing has been broken with the help of cloud computing and complex tasks can be performed using a simple machine with a web browser.

Lets split machine learning process into six different jars and look into each one of them in detail.

Jar 1 — Data

For machine learning we need useful data and lets look at what form that these data should be in to get meaningful insights from them. So the data can be in the form of text, audio, image and structured data in tabulated form.

For example this is a medical image with input part (x) and output part (y) which tells the presence of an anomaly. The question that the hospital can raise is " Can I tell whether a person is normal or not with these images ? ". For that question to be answered we need the output (y) given by the doctors to each and every image of the scan and x is the data of each and every pixel in the scan. If the scan is a 30x30 image then there are 900 RGB values of the image stored in a array. This data can be read by a machine. Here all the data is encoded as numbers and contains 900 real numbers. This is an example of high dimensional data.

The data should be in x and y.
The data should be in machine readable format.

In this example the text data in the document is represented as real numbers in x and output is represented in y.

Jar 2 — Tasks

In this example the input x ( red box ) is the text data from description of product and output y (green box)is the structured tabulation extracted from the unstructured input x.

The tasks can be supervised that is, they have labelled data. Here from the labelled images we can tell classifier to learn the relationship between x and y and find out whether there's any text in the image (y = 1) or not ( y = 0 ).

In this task it is expected to find the box in which the text is present in the image. Here we are trying to find the co-ordinates (left_x, left_y, width, height) of the box from the values (x) of the image given.

In unsupervised tasks we don’t know the y labels. We don't know what each cluster means, like they can be clustered based on colours of the sign or based on the shapes of the sign board. The machine will try to find the pattern and it is upto us to interpret those patterns.

In this type of unsupervised task the machine generates x from x. That is from the images given as input it produces an output image that is similar to input images. This is known as generations and is achieved with the help of GAN's (Generative Adversarial Networks).

Jar 3 — Models

The image is classified as a dog because there is a function which takes the image and classifies it as a dog. The problem that we face at the beginning is that we don’t know what the function is. This is where machine learning comes in to find the function and establish a relationship with x and y which is the closest approximation of the true relation.

The ŷ is the close approximation of the function. The green line in the graph is our model developed by close machine learning.

Trying to fit a function through all the points makes it more complex. So we use the concepts of bias-variance tradeoff, overfitting and regularization to control the complexity of the model.

Jar 4 — Loss Function

Let’s assume that three people build three models for a given function. To determine which model suits the best we can use loss function to solve the problem.

L1,L2,L3 are the loss functions of the three models. We can see that L1 has less value and hence it is a close approximation of the function. A model can be said to have perfect estimate when the loss function equals to 0.

Jar 5 — Learning

The learning algorithm is used to find the parameters a,b,c such that the loss is minimized. There are many optimization solvers which can be used to find a,b,c and this forms the crux of machine learning.

Jar 6 — Evaluation

The data is split into test and train before training the model on the test data. The predictions are made on the test data and accuracy is used to evaluate the model built. All the data collection process , defining the tasks in terms of x and y , defined a learning algorithm and loss function is done on training data. The goal is to estimate the parameters a,b,c so that the loss on the training data is minimized.

Intoduction to SQL

Shiv Prasad — Sat, 01 Aug 2020 16:51:47 +0000

A database is a set of data stored in a computer. This data is usually structured into tables. Tables can grow large and have a multitude of columns and records.

SQL (pronounced “S-Q-L” or “sequel”) allows you to write queries which define the subset of data you are seeking. Unlike Excel and Sheets, your computer and SQL will handle how to get the data; you can focus on what data you would like. You can save these queries, refine them, share them, and run them on different databases.

What is a Database?

A database is a set of data stored in a computer. This data is usually structured in a way that makes the data easily accessible.

Tables: Rows and Columns

Tables can have hundreds, thousands, sometimes even millions of rows of data. These rows are often called records.

Tables can also have many columns of data. Columns are labeled with a descriptive name (say, age for example) and have a specific data type.

For example, a column called age may have a type of INTEGER (denoting the type of data it is meant to hold).

In the table above, there are three columns (name, age, and country).

The name and country columns store string data types, whereas age stores integer data types. The set of columns and data types make up the schema of this table.

The table also has four rows, or records, in it (one each for Natalia, Ned, Zenas, and Laura).

What is a Relational Database Management System (RDBMS)?

A relational database management system (RDBMS) is a program that allows you to create, update, and administer a relational database. Most relational database management systems use the SQL language to access the database.

What is SQL?

SQL (Structured Query Language) is a programming language used to communicate with data stored in a relational database management system. SQL syntax is similar to the English language, which makes it relatively easy to write, read, and interpret.

Many RDBMSs use SQL (and variations of SQL) to access the data in tables. For example, SQLite is a relational database management system. SQLite contains a minimal set of SQL commands (which are the same across all RDBMSs). Other RDBMSs may use other variants.

(SQL is often pronounced in one of two ways. You can pronounce it by speaking each letter individually like “S-Q-L”, or pronounce it using the word “sequel”.)

Additional info :

Popular Relational Database Management Systems

SQL syntax may differ slightly depending on which RDBMS you are using. Here is a brief description of popular RDBMSs:

MySQL

MySQL is the most popular open source SQL database. It is typically used for web application development, and often accessed using PHP.

The main advantages of MySQL are that it is easy to use, inexpensive, reliable (has been around since 1995), and has a large community of developers who can help answer questions.

Some of the disadvantages are that it has been known to suffer from poor performance when scaling, open source development has lagged since Oracle has taken control of MySQL, and it does not include some advanced features that developers may be used to.

PostgreSQL

PostgreSQL is an open source SQL database that is not controlled by any corporation. It is typically used for web application development.

PostgreSQL shares many of the same advantages of MySQL. It is easy to use, inexpensive, reliable and has a large community of developers. It also provides some additional features such as foreign key support without requiring complex configuration.

The main disadvantage of PostgreSQL is that it is slower in performance than other databases such as MySQL. It is also less popular than MySQL which makes it harder to come by hosts or service providers that offer managed PostgreSQL instances.

Oracle DB

Oracle Corporation owns Oracle Database, and the code is not open sourced.

Oracle DB is for large applications, particularly in the banking industry. Most of the world’s top banks run Oracle applications because Oracle offers a powerful combination of technology and comprehensive, pre-integrated business applications, including essential functionality built specifically for banks.

The main disadvantage of using Oracle is that it is not free to use like its open source competitors and can be quite expensive.

SQL Server

Microsoft owns SQL Server. Like Oracle DB, the code is close sourced.

Large enterprise applications mostly use SQL Server.

Microsoft offers a free entry-level version called Express but can become very expensive as you scale your application.

SQLite

SQLite is a popular open source SQL database. It can store an entire database in a single file. One of the most significant advantages this provides is that all of the data can be stored locally without having to connect your database to a server.

SQLite is a popular choice for databases in cellphones, PDAs, MP3 players, set-top boxes, and other electronic gadgets. The SQL courses on Codecademy use SQLite.