DEV Community: Zarin Saima Roza

Feature Scaling in ML

Zarin Saima Roza — Sun, 18 Feb 2024 15:49:23 +0000

Imagine you're making a dessert and following a recipe that lists ingredients in different units—some in grams, some in tablespoons. Now, you want to ensure each ingredient plays a fair role in the final dish. If you don't scale them properly, one ingredient might overpower the others because it's measured in larger units. Let's say you unintentionally put a lot of cocoa powder in your dessert than the actual amount required so your dessert now tastes bitter. We don't want it right?

In the world of data and machine learning, our "ingredients" are features or variables in a dataset. Feature scaling is like ensuring each of these "ingredients" gets a fair portion in the analysis so that we don't end up with a dessert that tastes too bitter, or we can say end up making a wrong model. We want to avoid letting one feature overshadow others just because it's naturally on a different scale.

Common methods for feature scaling include:

Min-Max Scaling (Normalization)
standardization (Z-score normalization)

(these methods are applied column-wise, which means we will normalize all the values of one column, we won't be doing it row-wise)

Let's understand with an example:

Think, we have a dataset that contains age and salary.

The differences in values of salaries are way bigger than the differences in values of ages, we can't put them in the same position. On one side the difference is 10K another side we have 1. Now if I want to group them I would surely group red and purple together as their difference is 8k; 2k lesser than blue and purple. But it may give us wrong ans. So we have to bring them down to a position where we can compare them. If Normalizing is used we'll get-

Now, you can easily say that the purple person is closer to the blue person. And we would group blue and purple.

The Great Showdown: Python list Vs. NumPy Array. Who wins?

Zarin Saima Roza — Fri, 24 Nov 2023 22:24:41 +0000

In Python, a list is a built-in data type used to store a collection of items. It is a mutable, ordered sequence that can contain elements of different data types, including numbers, strings, and even other lists. On the other hand, NumPy is a Python library that helps with dealing with n-dimensional arrays. It's used in hard mathematical operations. We can efficiently use it as a fundamental tool for scientific computing and data analysis in Python.

Now, in 3 different scales, we will judge which one is better to use.
The ultimate battle round begins 🥁!

Round1:storage efficiency

lis=range(1000) #a list of 1000 element

#print(lis) #range(0, 1000)
#print(sys.getsizeof(5)) #any num would give 28
#print(len(lis))#1000
print(sys.getsizeof(1)*len(lis))# 5 denotes element num

arr=np.arange(1000)#same as range func, gonna create an array
print(arr.size*arr.itemsize)
# print(arr.itemsize)

# 28000 bytes -taken by lis
# 8000 bytes -taken by arr(numpy)

#Byte size of elements varies with Operating system

In the above code, it's evident that numpy takes less space to store. We created a list of 1000 elements. Where each element takes 28 bytes in Python conventional list,numpy stores each element in 8 bytes. So, clearly, numpy wins!

Round 2:Time taken to execute

**
When the dataset is very small it doesn't really matter. But let's say you are working with a very large amount of data then who'll be ahead in the race?

In Python everything is an object. The objects of Python list are not stored in a contiguous manner. Rather it stores the location of the object. Then pointers point towards the object located in its given memory location.So it takes while to execute.

On Numpy's case it has contiguous memory location. So takes less time to execute.

Let's see it in the code-


sz=1000000

lis1=range(sz)
lis2=range(sz)

arr1=np.arange(sz)
arr2=np.arange(sz)

#python list

start=time.time()
print(time.time()-start)
res= [(i+j)for i,j in zip(lis1,lis2)]
#it will take 1st ele from lis1 and lis2 and add them togrther in ele1 of res
print("Time taken by python list",(time.time()-start)*1000) #sec


#numpy array

start=time.time()
result=arr1+arr2
print("Time taken by nparray",(time.time()-start)*1000)

we see,
Time taken by python list 113.46936225891113 mili seconds
Time taken by nparray 4.576921463012695 mili seconds

python list takes a lot more time than numpy to work. So, clearly numpy again takes a lead.

Round3: Conveniency

**
We are already familiar with the usage of numpy. But as numpy and List do a lot of similar jobs. In which basis numpy is convenient.

It's very easy to add, subtract,multiply, and divide two matrices in numpy. In python, we don't get to do these.
NumPy offers a rich set of mathematical functions for linear algebra, Fourier analysis, random number generation, and more. This makes it a powerful tool for scientific computing.


#numpy why convenient
arr1=np.array([1,2,3])
arr2=np.array([6,90,5])
print(arr1+arr2,arr1-arr2,arr1*arr2,arr1/arr2)

Undoubtedly, numpy aces this round too!

To conclude, when it comes to fast computation, memory efficiency and convenience numpy without a say beats Python convenient List!💪

To know more functions of numpy, Check out this ipynb file-
[(https://colab.research.google.com/drive/1-WoYuFd_iZiTZ8gcRuzrbvjx6D1Mq4Rh?usp=sharing)
]

Optimization of log loss

Zarin Saima Roza — Tue, 21 Nov 2023 22:08:37 +0000

Let's think of a scene where to win you need 7 H and 3 T in 10 times of tosses. And, three are 3 different coins has different possibilities of having H and T. Which coin has the probability of winning?

Surely, number 1 wins!

Let's see how we can solve this,

This can be used using the chain rule, But then it will be lengthy. So, using the log function on g(p)(the chances of winning) is the wisest decision. Then finding the derivative of the log loss will give the correct answer.

So what is the log loss?
According to a data scientist, log-loss is indicative of how close the prediction probability is to the corresponding actual/true value (0 or 1 in the case of binary classification). The more the predicted probability diverges from the actual value, the higher the log-loss value.

The relationship of log loss with ml is shown below-

Here, we minimize the log loss and get that the optimal p is 0.7.
:D

Optimization of The Square Loss

Zarin Saima Roza — Tue, 21 Nov 2023 21:22:12 +0000

The square loss function is the average of squared differences between the actual and the predicted values. the squared error is one of the most important functions in machine learning. It is used to train linear regression problems and certain neural networks.

In this problem, there are 3 power lines and our task is to place the house at a point where it is the most cost-effective.

Firstly, let's take the first power line. Where the house be situated to get the least cost? Obviously, where the distance is zero, so that we get zero cost.

Now, let's bring another power line to the discussion.

Let, a= the distance of pl1 from origin
b= the distance of pl2 from origin
x= the distance of the house from origin

So, we can see the cost of connecting both of the powerlines is quite the same when the house is placed in the middle.

We don't want any of these above situations cause the cost area is very big.

We see that the tangent is zero in the lowest point. (x-a)^2 +(x-b)^2 is the cost function here, the derivate of this function is-

So, if the house is placed between two poles the cost will be minimized.

Now, let's see what happens if a 3rd powerline is in the scene,
Again, if we repeat the same procedure,

If we sum the heights of the three parabolas, we get the black parabola.

finding the derivative of the cost function-

So, the generalized solution is-

Finally, the solution would be to put the house in the average of the coordinates of all the power lines. And this is the square loss function.

**The pictures are taken from different source.