DEV Community: Anita Okoh

Linear regression

Anita Okoh — Thu, 23 Apr 2020 17:04:30 +0000

Disclaimer: My study note on Linear Regression

Linear Regression is an algorithm used to find the best fit line( trend) for a set of data points.
It revolves around the straight-line equation

y = m X + b

where

y is the dependent variable( the variable we went to predict),
X is the dependent variable ( or the predictor or feature variable) and
the two parameters or weights which are the slope and intercepts respectively.

The whole game or idea in a linear regression model is to adjust the weights to find the best-suited line that fits the data enough to predict the y variable accurately or near-accurately. In turn, the best-fit line is the equation which has the least error/lowest error

There are two major ways of getting the best line by adjusting the weights

The absolute and square trick
The error functions (MAE OR MSE) along with gradient descent

Types of Gradients

There are 3 major types

Batch Gradient descent: This involves summing all the errors and updating the weights
Stochastic Gradient Descent: This involves using each point to update the weights
Mini-Batch Gradient Descent: This involves splitting the data into small equal batches and use each batch to update the weights.

(N:B, the mini-batch method is most times used because the earlier two types can be computationally expensive.)

It would be great to understand the mechanics behind each Gradient descent visually

Dimensions in Linear Regression Models

Two-dimension: This includes one feature variable and a dependent variable. In this case, the prediction is in the form of a line
Three-dimension: This includes two feature variables and a dependent variable. In this case, the prediction is in the form of a plane
N-dimension: This includes n-number of features along with a dependent variable. In this case, the prediction is in the form of a hyperplane.

It is good to note that when we have one feature(X) variable, it is known as a Simple Linear Regression because it builds a simple model. More than one feature variable makes it a Multiple Linear regression and as more features are introduced, the more the model becomes complex.

There are two major red flags to note about Linear Regression Model.

It works best when the data is linear or has a linear relationship. If the x and y variable has no linear relationship, you may need to
- Make adjustments (or transform the data)
- Add more features
- Use another type of non-linear model
Linear Regression is sensitive to outliers.

Polynomial Regression is a method to transform the data that have a non-linear relationship. It is more of a preprocessing/data transformation method. It also adds more complexity to the linear model. It would be great to understand more on its intuition.

Regularization is a way to penalize complex models and help smoothen the line to make the model more generalized.
There are two types of Regularization: L1 and L2 which are penalties that can be found in the lasso regression and Ridge Regression respectively( These are variations of the vanilla Linear Regression). In each of the Variation models, the parameter lambda is used to regulate these penalties.

Below is an image that shows the difference between L1 and L2

Feature Scaling

This is a way of transforming your data into a common range of values. There are two types

Standardizing - This is a process of subtracting each data point with the mean and dividing by the standard deviation.
Normalizing - This is a process where the data is scaled between 0 and 1.

When to feature scale

When your algorithm uses a distance-based metric to predict like SVMs, KNN, K-means, etc.
When you incorporate regularization.

Some questions to ponder are:

Can feature scaling improve the linearity of the variables?
How does scaling affect distance-based models or regularization?

Some Software engineering Concepts

Anita Okoh — Thu, 09 Apr 2020 11:57:13 +0000

Lately, I have been taking some software engineering classes to combine the knowledge with my data science skills

These are some of the concepts I have picked up so far

Refactoring: This means restructuring your code to improve the internal structure without changing external functionality. In other words, rearranging your code to improve efficiency and readability without change the purpose or the original functions of the code. This includes cleaning your code by using descriptive but concise variable names and decluttering the lines of codes in a function as well as using appropriate white spaces. It also includes proper documentation, be it an inline-level, function/module level or project level
Modularization: This means converting your codes into modules based on their functionalities. Someone once told me that "similar functions should be grouped together". This helps for each module to be independent in function, improve readability and reusability for future references
Test-Driven Development: This is the process of writing tests for tasks before developing or writing the code to implement those tasks. This is such an abstract technique in data science and it would be nice to practice this.

One thing to keep in mind is: Classes, inheritance, object, attributes, and methods are common to all object-oriented programming language

If one were to think about python concepts ( like packages, modules, functions, etc) in a hierarchical method, I would say it would look like this below (from small to big)

Classes , Magic Methods and Inheritances

Anita Okoh — Wed, 08 Apr 2020 15:26:39 +0000

CLASSES

A Class is a blueprint of an object. Classes are a great way for object-oriented programs to group similar data and functions in a manner that is reusable and to avoid redundancy.

Magic Methods

Python Magic Methods are methods specifically to override default python behaviors or rules, most of the time, operator overrides. They can be identified with two prefix and suffix underscores in the method name. Examples of Magic codes are

the __init__() method in a Class which overrides how python instantiate an object
the __add__() method which overrides the behaviour of the plus sign. These methods could be the reason we can add two strings with ‘+’ operator with no error and without any explicit typecasting(i.e Converting one data type to another data type)
The __rep__() method which overrides what gets printed as a variable value
The __len__() method which probably has to do with the length of a data structure

And etc.

Magic Methods seem to be super cool but super technical and seem to be only applicable in a Class

Inheritance

Inheritance is the technique where a class inherits the properties of another class(which most times is considered as the parent class). And these properties being both attributes and methods. And the main rationale for this is so that the child class can have more varieties without affecting the design of the parent class.

Combinatorics

Anita Okoh — Sat, 04 Apr 2020 18:53:57 +0000

Combinatorics is the combination of outcomes from a finite set.
There are two major types of combinatorics.

Permutation
Combination Factors that differentiate each combinatorics are
Order
Repetition
Entire set/Subset

Permutation and combination both have to do with the number of different ways to arrange or pick certain elements of a set

The major differences between the two are

Order: For Permutation, how the set/subset is arranged or the order is very critical while for combination, the order is irrelevant. For example, trying to decode the passcode of a locker has a lot to do with how the elements/letters would be arranged. Therefore, this is a permutation problem
Double Counting: For Permutation, no matter how identical two subsets are in terms of elements( not order), there are still considered distinct sets. Unlike the combination, where each combination of sets should not be identical in elements. An example is a collection of the following subsets: AB, BA, AC, CA, BC. Although there are 5 permutations, there are just 3 combinations because for combinations AB & BA are not distinct selections. This is why there would be more permutation selections than combination selections in a given set. Therefore, permutation does not take into account double counting

Permutation can be split further into two

The vanilla Permutation
Variation

Vanilla Permutation in the sense that all elements in the set would be used in the selection ways problem.
Variation, as the name implies, infers that we pick and arrange some elements in the given set i.e a subset(s) from a total set(n)

For example, how many ways can the sets: A, B, C, D, E be arranged? can be a vanilla permutation problem while how many ways can the sets: A, B, C, D, E be arranged in twos? is a variation problem

I would like to say that combinations can be confused with variation permutation. One major difference is if the order is relevant. ( A second difference is obviously the formulas)

It is also good to know that both permutations and combinations have different formulas based on if the elements in question should be repeated or not. (combination problems with repetitions are very hard to identify)

Let's talk about the formulas. Take note of the difference in the order and repetition constraints from the image below

I also created a decision tree to help as well

The above information is not exhaustive therefore I would update as I learn more about the topic.

So many Types of Probability distributions

Anita Okoh — Sat, 04 Apr 2020 10:54:52 +0000

Today in my Nanodegree course, I got to be aware of some of the different types of probability distributions
The common ones are

the continuous probability distribution(CPD) and
the discrete probability distribution (DPD)

CPD in terms of Normal Distribution( Gaussian distribution)
DPD in terms of Binomial Distribution

I guess for CPD, the random variable ( variable in question) can be any range of number while fo the DPD, the random variable are discrete( integers) only

Some questions that popped up to my minds are

What are the other PDs?
How are the Bernoulli and multinominal different?
And when do you use each in naive Bayes

I also have to know how to build the gaussian distribution and any other distribution from scratch.