DEV Community: Shubham Saxena

How to Judge your Recommendation System Model ?

Shubham Saxena — Tue, 09 Feb 2021 16:17:52 +0000

There are many metrics to judge a classical machine learning model like Accuracy, Precision, Recall, F1 etc, but they cant be directly applied to a recommendation system model.
So in this post I'm writing about the metrics we can use to judge a recommendation system model.

Recommended Vs Relevant Items

Recommended items: Items which our model predicted for a given user. We can get these items from the model predictions for a given user.
Relevant items: Items in which the user is actually interested in. These can be obtained from several methods like:
1. Items on which the user has clicked.
2. Items which the user has searched for.

The goal of a Recommendation model is simply to maximize the intersection between recommended and relevant items for all users.

Here are some metrics we can use for checking the performance of our model:

Precision @K:

How many relevant items are present in top k recommendations?

For eg. P@5 would be calculated by taking top 5 recommendations from the system for each user and checking how many are relevant ones.
If 3 items are relevant then P@5 = 3/5 = 0.6.

Average Precision @K:

The mean of P@i for i = 1..K

For eg, AP@3 = SUM(P@1 + P@2 + P@3)/3. This will be calculated for each user.

Mean Average Precision @K:

Mean of AP@K for all users.

For eg, MAP@3 = SUM(AP@1 + AP@2 + AP@3)/3

These metrics are straightforward to implement, also can be obtained from here.
Happy Learning !

Calculating Run Time of a function using Python Decorators

Shubham Saxena — Fri, 01 Jan 2021 11:32:49 +0000

Many times we need to check the running time of a function, so that we can see which functions are taking a lot of time and improve them.
In this post I'm writing about a simple but elegant way to do this using Python Decorators.

But, What is a Decorator?

Decorators are a design pattern in python that lets us modify the functionality of an object without changing its structure. Here I'll use this to modify a function to also print the time taken by it.

Lets see how I'm using this to track function run time:

1. Target Function:

Suppose we have a basic function which reads a csv and returns the pd.DataFrame. Now depending on the size of the csv, it can take some time to run.

2. Decorator Function:

Here we can define the function calc_runtime which will act as a decorator for calculating runtime. This function will take a function f as input and will return an updated version of the same function.

3. Using the Decorator over target function:

As the decorator is ready, we can simply call this over any function to print its runtime. Here we are calling over our target function like this :

Basically the load_csv gets passed on as an argument to the calc_runtime, there we handle its arguments by using args
and kwargs easily. Finally we return the value to the calling space.

By using this kind of decorators, its straightforward to change the return value of a function, or print some additional layers over a function.

How to? : *args and **kwargs in Python

Shubham Saxena — Sat, 26 Dec 2020 16:03:52 +0000

Python offers several ways to pass on arguments to a function body.
Suppose we are making a function that returns the area of a rectangle. We can write the basic code like this:

This given function will calculate area of a given rectangle given its height and width.

Now, if we want to make this function generic to find hyper volume of a cuboid given its n dimensions, we have two ways to move forward:

Change the function to take n dimensions as arguments: calculate_volume(x1, x2, x3 ... xn), this seems really messy and wont even work on n-1 dimensions.
Change the function to take a list of n dimensions as a single argument:calculate_volume([x1, x2, x3, ... xn]), this seems much better, although we will have to pass on a list of dimensions which isn't really that nice.

Understanding *args:

The important part here is the *, and not the args. We can use any variable name according to the function use case.
Now let's use this to solve our current problem:

As the dimensions argument can take any number of arguments, we can now pass several dimensions in it to return the volume.

We can further improve this logically by using both fixed and extended arguments :

Now, as a valid shape will be having at least one dimension, so we can pass it as first_dimension and all other dimensions in extra_dimensions.

Understanding **kwargs

This is used for variable length keyword arguments. Use case is a function that where we dont know the arguments or we need to pass on all the arguments to some other function.
It is basically a dictionary that can hold all the keyword arguments together and we can simply iterate over its key-value pair to access them.
Example:
Here I'm making an automatic html generator which generates the tag given all its attributes. We dont need to worry about handling all the keyword arguments, we simply pass them in **tags.

Output: '< img src = "mountain.jpg" width = "500" height = "600" >'

Combining everything:

One real life use for these are in arguments forwarding.
Imagine we are working on a big project with several functions, and now we want to monitor the time taken for each function in the code.
Basic way would be to go to function definition and encapsulate it with in start_time and end_time, then logging out the difference as time taken. This will require heavy code change and is not advisable at all.

Another way would be to make a generic trace function that will take any other function as argument and returns the time taken by it.

Since the trace function is generic in nature, it doesn't care about the arguments of f. We can simply pass them as args and kwargs without changing the function definition.

For example, here we have a function that calculates the exponent value given 2 numbers (which of course has been modified for this example)

Although we can do this in a much elegant way, that I'll continue in the next post.