DEV Community: Azad Kshitij

Organize Your Godot Exports with export_category in Godot 4

Azad Kshitij — Thu, 28 Sep 2023 13:00:00 +0000

Have you ever wondered how to neatly categorize your exported variables in Godot? The answer lies in the export_category keyword, a fantastic new feature introduced in Godot 4. It allows you to group your exported variables, maintaining organization and tidiness in your scripts.

YouTube

How to Use export_category:

It's remarkably simple. Just place @export_category("YourCategoryName") before the @export keyword. For instance, if you're working on a player script and want to create a category called "Movement," it's as easy as:

@export_category("Movement")
@export var speed = 10
@export var jump_height = 5

With this, you'll have a "Movement" category neatly displayed in your inspector, containing all related exported variables. To add more categories, simply place @export_category above your desired exported variables.

@export_category("Player Info")
@export var name = "Bit Of Code"

Why Use export_category:

Keep your projects organized, even with numerous exported variables.
Enhance script readability and maintainability.
Simplify tweaking and fine-tuning during development.

Conclusion:

I hope you found this tutorial helpful and discovered a new way to organize your Godot projects. If you did, please give it a like and if you prefer a video version watch the short to better understand export_category

Lookup in 1 min

Azad Kshitij — Tue, 11 Jul 2023 16:57:38 +0000

This is a handy function when you need to look up something quickly and easily.

LOOKUP has two forms: vector and array. The vector form is simpler and more common, so I'll focus on that one. The vector form of LOOKUP has three arguments:

lookup_value: This is the value that you want to find in a row or column. It can be a number, text, a logical value, or a reference to a cell.
lookup_vector: This is the range that contains the values that you want to match with lookup_value. It can be a row or a column, but it must be sorted in ascending order.
result_vector: This is the range that contains the values that you want to return from the same position as lookup_value. It can be a row or a column, but it must be the same size as lookup_vector.

For example, let's say you have a table of products and prices like this:

Product	Price
Apple	$1
Banana	$2
Cherry	$3
Date	$4
Eggplant	$5

If you want to find the price of a product, you can use LOOKUP like this:

=LOOKUP("Cherry",A2:A6,B2:B6)

This formula will return $3, because "Cherry" is in cell A4, and $3 is in cell B4.

You can also use LOOKUP to find the product name for a given price, like this:

=LOOKUP(4,A2:B6,A2:A6)

This formula will return "Date", because 4 is in cell B5, and "Date" is in cell A5.

That's it for this blog post. I hope you learned something new about LOOKUP in Excel. If you have any questions or comments, feel free to leave them below. Thanks for reading!

Data Science: %%timeit in python

Azad Kshitij — Thu, 26 Jan 2023 06:30:00 +0000

Youtube Short

In Python, the %%timeit magic command is a handy tool for measuring the execution time of small bits of code. It can be used in interactive environments like Jupyter notebooks.

To use %%timeit, you simply place it at the beginning of a cell in a Jupyter notebook, followed by the code you want to measure the execution time of. %%timeit will then run the code multiple times and return the fastest run time among all the runs.

The output shows the time it took to run the code in microseconds, as well as the mean and standard deviation of the run times across multiple runs.

The number of runs and loops can be adjusted by passing arguments to %%timeit.

In this example, %%timeit will run the code 10 times and do 100 loops per run.

%%timeit is a useful tool for identifying performance bottlenecks in your code and optimizing the most heavily used sections. It can be especially helpful for comparing the performance of different algorithms or implementations.

Data Science: What is a box plot?

Azad Kshitij — Thu, 05 Jan 2023 09:30:00 +0000

Youtube Short

A box plot, also known as a box and whisker plot, is a graphical representation of a dataset that shows the distribution of values in the data. It is a useful tool for visualizing the spread and skewness of a dataset, as well as identifying outliers.

The box plot can be used to compare the distribution of multiple datasets by creating a box plot for each dataset and placing them side by side. It is also possible to overlay box plots on top of each other to compare the distributions more closely.

Box plot is a graphical representation of a dataset that shows the distribution of values in the data.
- The top line is maximum value.
- Bottom line is minimum value.
- The Centre line is Median.
- Top of the box is 75th percentile value.
- Bottom of the box is 25th percentile value.
- You see those circles outside yes those are called 'outliers'.
Lets see how to create one with python.
- Start by importing necessary packages.
- We will use seaborn to create the plot.

import seaborn as sns
import matplotlib.pyplot as plt

Lets use some inbuilt dataset that comes with seaborn. called taxis. and set the style of the graph as white grid.

sns.set(style="whitegrid")

df = sns.load_dataset("taxis")

Now define values for the x-axis and y-axis. and define a list of cities you want to create box plot for.

x = "pickup_borough"
y = "total"
cities = ["Queens"]

Create the plot with sns.boxplot() function, and provide df as data. set x as x y as y and order boxplot in order of cities list. Now use plt.show() function to show the graph.

ax = sns.boxplot(data=df, x=x, y=y, order=cities)

plt.show()

Final

import seaborn as sns
import matplotlib.pyplot as plt

sns.set(style="whitegrid")

df = sns.load_dataset("taxis")

x = "pickup_borough"
y = "total"
cities = ["Queens"]
ax = sns.boxplot(data=df, x=x, y=y, order=cities)

plt.show()

Result

I hope this tutorial has helped you understand the basics of box plots. If you have any questions comment them down below I will be more than happy to answer them.

Data Science: 10 Ways to improve your pandas code

Azad Kshitij — Wed, 21 Dec 2022 11:30:00 +0000

In this post we will talk about writing better pandas code just by some simple set of rules and using inbuild function provided by pandas. Your code might be working before but these changes will make your code more readable and more manageable and some time run faster. Don't let the reader know that you are new.

Writing into csv with unnecessary index.

When you save a DataFrame to a csv format, by default it will be saves with a index column. This will unnecessarily increase the size of our csv file. We can change this behaviour by doing index = False in the to_csv() function.

df.to_csv("output_index.csv", index=False)

Using column names which include spaces.

It is some time okay if you are sending the file to some one non technical but for actual work it sometime creates many problem. use _ instead of spaces, after doing this it will be easy to access them by . notation in pandas.

df.columns 
# Output: Index(['Series_reference', 'Period', 'Data_value', 'Suppressed', 'STATUS', 'UNITS', 'MAGNTUDE', 'Subject', 'Group', 'Series_title_1'], dtype='object')
df.Series_reference

Filter dataset like a PRO with QUERY method

Use .query() function provide by pandas instead of using manual conditions.
ex:

# This
df.loc[ (df.Period >= 1971) & (df.Period <=  1973) ]
# Can be simplified as this...
df.query(' 1971 <= Period <= 1973 ')

query strings with(@ symbol) to easily reach variables

This might be bit controversial as some people like have references of their variables and to make it more readable, but just to inform you about this feature that you can use @ to reference variable in a query string. like this...

min_year = 1971
max_year = 1973
# This:
df.query(f' {min_year} <= Period <= {max_year} ')
# Can be simplified as this...
df.query(' @min_year <= Period <= @max_year ')

"inplace" method could be removed in future versions, better explicitly overwrite modifications

inplace method could be removed in future versions so better explicitly modify the DataFrame by reassigning.

# This 
df.fillna(inplace=True)
# Should be changed to this... 
df = df.fillna()

Better Vectorization instead of iteration

As I mentioned about the performance of you function the loops are no top of that discussion. You can iterate over each row of you DataFrame but that is not a good idea if you just want to create a new column by applying conditions on you DataFrame.

%%time
for i, row in df.iterrows():
    if row['Period'] < 2000:
        df.loc[i, 'is_recent'] = False
    else:
        df.loc[i, 'is_recent'] = True

df.is_recent
"""
CPU times: total: 34.3 s 
Wall time: 37.9 s
"""

But by using Vectorization we can significantly reduce the time

%%time
df['is_not_recent'] = df['Period'] >= 2000
"""
CPU times: total: 15.6 ms 
Wall time: 993 µs
"""

Vectorization method are preferable than Apply

The apply method does take less time then iteration it is still not as good as the vectorization.

%%time
df['recent'] = df.apply(lambda row : row['Period'] < 2000, axis=1)
df.recent
"""
CPU times: total: 2.59 s 
Wall time: 2.92 s
"""

df.copy() method

If you want to use a single column as a new DataFrame then you shoud use .copy() otherwise if you change the newly created dataframe it will also change the same in original dataframe.

new_df = df.copy()

Chaining formulas is better than creating many intermediate dataframes

Chaining operations will be a lot more useful than you think just look at this example...

# This 
df1 = df.query('Period > 2000')
df2 = df1.groupby(["STATUS"])[['Period']].min()
df_out = df2.sort_values('Period')

# Can be simplified as this...
df_out = (df.query('Period > 2000')
    .groupby(["STATUS"])[['Period']].min()
    .sort_values('Period'))
df_out

Now, you decide by your self which one you want to use.

Properly set column dtypes

When it comes to memory pandas can be a 'memory hog' but there are ways to keep the memory usage low. You can try changing dtypes of a column. Sometime you have a column that stores 'Y' for True and 'N' for 'False' you can change the type of that column which will speedup the operations on that column as well as reduce the size of DataFrame. Try applying these conversions...

string -> category or bool or number
float63 -> Smaller float values like float32 or float16
int64 -> smaller int values like int32 or int16

These are the 10 ways in which you can improve your pandas code, in future article we will talk about more ways to improve your datascience projects. If you have anything to say comment down below I'm new to blog writing so any type of feedback is appreciated Thanks.

References

https://www.youtube.com/watch?v=_gaAoJBMJ_Q

Machine Learning: Basic Terms

Azad Kshitij — Fri, 16 Dec 2022 06:30:00 +0000

In these series of articles we will talk about ML algorithms and implement them in python and the main focus of these articles is to understand math and logic behind these algorithms. We will start by understanding basic terms of ML.

1. Classification Algorithms

As the name suggest these are classification algorithms that categorizes the data into different category. It is a Supervised Learning technique.

2. Supervised Learning

Supervised learning is associated with learning with supervision or training. In supervised learning, the algorithms are trained with data which is labeled or classified.

3. Unsupervised learning

Unsupervised learning is associated with learning without supervision or training. In unsupervised learning, the algorithms are trained with data which is neither labeled nor classified. In unsupervised learning, the agent needs to learn from patterns without corresponding output values.

4. Cross-Entropy Loss

Is a metric used to measure how well a classification model in machine learning performs. The loss (or error) is measured as a number between 0 and 1, with 0 being a perfect model.

5. Condition positive (P)

The number of real positive cases in the data

6. Condition negative (N)

The number of real negative cases in the data

7. True positive (TP)

A test result that correctly indicates the presence of a condition or characteristic

8. True negative (TN)

A test result that correctly indicates the absence of a condition or characteristic

9. False positive (FP)

A test result which wrongly indicates that a particular condition or attribute is present

10. False negative (FN)

A test result which wrongly indicates that a particular condition or attribute is absent

11. Confusion Matrix

It provides us the result of our model in matrix form (as shown below). We use this to measure accuracy of the mode.

	Actual positive	Actual Negative
Predicted Positive	True Positive	False Negative
Predicted Negative	False Negative	True Negative

12. Area Under the Curve (AUC)

Is the area encapsulated between the curve and x-axis.

13. Receiver Operating Characteristic (ROC)

The receiver operating characteristic (ROC) curve, which is defined as a plot of test sensitivity as the y coordinate versus its 1-specificity or false positive rate (FPR) as the x coordinate

\frac{hits}{hits + misses}

This image from Wikipedia shows exact definition of ROC.

If you want me to include any other terms in this article comment them and I will do my best to include them.

References

https://www.javatpoint.com/classification-algorithm-in-machine-learning
Park SH, Goo JM, Jo CH. Receiver operating characteristic (ROC) curve: practical review for radiologists. Korean J Radiol. 2004 Jan-Mar;5(1):11-8. doi: 10.3348/kjr.2004.5.1.11. PMID: 15064554; PMCID: PMC2698108.
https://en.wikipedia.org/wiki/Receiver_operating_characteristic
https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc

Data Science: Saving Pandas Dataframe the Efficient Way

Azad Kshitij — Sun, 04 Dec 2022 11:30:00 +0000

So you are done processing your data, now what? You might want to send the processed data to someone else or save the result for later. In such case you you will use pandas method to_format(file_name). let say you want to save to .csv you will write .to_csv("csv.csv"). The most common way is to save it as .csv form because is the most versatile and anyone can open and understand the content of file.

But But But saving it to .csv is not the best way to save a DataFrame, as there are many flaws of saving it to a .csv file. Let say our original dataframe had some columns with object as a datatype and durign the processing we converted it into a category datatype now if we save the dataframe to a .csv, it will loos the changed datatype and we have to start all over again. Now what is the solution?

Pandas support saving to many other file formats such as parquet, feather, pkl, hd5 etc. All these formats saves the information about the datatype. Lets do some comparison between all these formats.

I used this dataset to perform the benchmarking.
I read the CSV file and saved them into different files.
Used seaborn to plot the graphs.

Write Speed

From the above graph we can see that saving to csv takes the most time and all other formats takes significantly less time to save. Feather is the clear winner followed by h5 and pkl then parquet.

Read Time

From the above graph we can see that reading a csv takes the most time and all other formats takes significantly less time to read. PKL is the clear winner followed by feather and h5 then parquet.

File Size

Now this is something interesting as you can see parquet is the clear winner and the runner up feather takes almost 3 times more than that.

From the above results you can decide by yourself which format you want to use. You might stick to csv if you are working with non technical people, but if you are working in tech and collaborating you might find parquet the perfect fit for sharing online because of its low file size.

If you have anything to say comment down below I'm new to blog writing so any type of feedback is appreciated Thanks.

Data Science: Linear Algebra with Python

Azad Kshitij — Fri, 11 Nov 2022 11:30:00 +0000

Linear Algebra

Linear algebra is the branch of mathematics that deals with vector spaces. It contains concept of vector, matrix etc. Linear algebra is widely used by data scientists (frequently implicitly, and not infrequently by people who don’t understand it). It wouldn’t be a bad idea to read a textbook.

Vector

Vectors, in general, are objects that can be added together (to form new vectors) and that can be multiplied by scalars (numbers) to create new vectors Vectors are concretely (for us) points in some finite-dimensional space. Although you may not think of your data as vectors, they are a good way to represent numerical data. For instance, if you have a large number of people's heights, weights, and ages, You can think of your data as three-dimensional vectors (height, weight, age). If If you're teaching a class with four exams, you can treat student grades as four-point scale. vectors with dimensions (exam1, exam2, exam3, exam4). The most basic method is to represent vectors as lists of numbers. A list of three numbers corresponds to a vector in three-dimensional space, and vice versa:

weight = [
          10, # kg
          20, # kg
          30  # Kg
      ]

length = [
          15, # meter
          25  # Meter
    ]

One issue with this approach is that we will want to perform arithmetic on vectors. Because Python lists aren't vectors (and thus don't support vector operations), arithmetic), we'll have to create our own arithmetic tools. So let's start there. To begin, we'll frequently need to combine two vectors. Vectors are added component by component. This means that if two vectors v and w have the same length, their sum is the vector itself. whose first element is $v [0] + w [0]$ , second element is $v [1] + w [1]$ , and so on on. (If they're not the same length, we can't combine them.)

Adding $[a, b]$ and $[c, d]$ will result in $[a + c, b + d]$ .

We can easily implement this by zip-ing the vectors together and using a list comprehension to add the corresponding elements:

def vector_add(v, w):  
    """adds corresponding elements"""  
    return [v_i + w_i  
        for v_i, w_i in zip(v, w)]

Similarly, to subtract two vectors we just subtract corresponding elements:

def vector_subtract(v, w):  
    """subtracts corresponding elements"""  
    return [v_i - w_i  
        for v_i, w_i in zip(v, w)]

We’ll also need to be able to multiply a vector by a scalar, which we do simply by multiplying each element of the vector by that number:

def scalar_multiply(c, v):  
    """c is a number, v is a vector"""  
    return [c * v_i for v_i in v]

We also need dot product of two vectors. The dot product of two vectors is the sum of their component wise product:

V⋅W=Vx⋅Wx+Vy⋅WyV{\cdot}W = Vx {\cdot} Wx + Vy{\cdot}Wy

def dot(v, w):
    return sum(v_i * w_i for v_i, w_i in zip(v, w))

Another value which can be useful is the Magnitude (or length) of the vector which is square root of sum of square of values.

\sqrt{(Vx)^2+(Vy)^2}

Distance between two vectors can be calculated by

\sqrt{(vx-Wx)^2 + (Vy-Wy)^2}

import math

def sum_of_squares(v):  
    """v_1 * v_1 + ... + v_n * v_n"""  
    return dot(v, v)

def magnitude(v):  
    return math.sqrt(sum_of_squares(v))

def squared_distance(v, w):  
    """(v_1 - w_1) ** 2 + ... + (v_n - w_n) ** 2"""  
    return sum_of_squares(vector_subtract(v, w))

def distance(v, w):  
    return math.sqrt(squared_distance(v, w))

Matrices

A matrix is a two-dimensional number collection. Matrixes will be represented as lists of lists, with each inner list the same size and representing a row of the matrix. If A is a matrix, then $A [i] [j]$ is the element in the ith row and the jth column. Per mathematical convention, we will typically use capital letters to represent matrices.

Same as vectors we can represent matrix in terms of 2D list.

A = [
     [1,2,3],
     [4,5,6]
]

The matrix A contains len(A) rows and len(A[0]) columns, which we take to be its shape given this list-of-lists representation:

def shape(A):
    n_rows = len(A)
    n_columns = len(A[0])
    return n_rows, n_columns

We shall refer to a matrix as a $\times k$ matrix if it contains n rows and k columns. Each row of a n by k matrix can be thought of as a vector of length k, and each column as a vector of length n:

def get_row(A, i):  
    return A[i] # A[i] is already the ith row  

def get_column(A, j):  
    return [A_i[j] # jth element of row A_i  
            for A_i in A] # for each row A_i

If a diagonal of a matrix contains only 1s then its called Diagonal Matrix.

We can use matrix to represent relations between different entities ex:

friendships = [(0, 1), (0, 2), (1, 2), (1, 3), (2, 3), (3, 4),  (4, 5), (5, 6), (5, 7), (6, 8), (7, 8), (8, 9)]
############# Which Can Also be Represnted as ###############

friendships = [ [0, 1, 1, 0, 0, 0, 0, 0, 0, 0], # user 0  
                [1, 0, 1, 1, 0, 0, 0, 0, 0, 0], # user 1  
                [1, 1, 0, 1, 0, 0, 0, 0, 0, 0], # user 2  
                [0, 1, 1, 0, 1, 0, 0, 0, 0, 0], # user 3  
                [0, 0, 0, 1, 0, 1, 0, 0, 0, 0], # user 4  
                [0, 0, 0, 0, 1, 0, 1, 1, 0, 0], # user 5  
                [0, 0, 0, 0, 0, 1, 0, 0, 1, 0], # user 6  
                [0, 0, 0, 0, 0, 1, 0, 0, 1, 0], # user 7  
                [0, 0, 0, 0, 0, 0, 1, 1, 0, 1], # user 8  
                [0, 0, 0, 0, 0, 0, 0, 0, 1, 0]] # user 9

If there are very few connections, this is a much more inefficient representation, since you end up having to store a lot of zeroes. However, with the matrix representation it is much quicker to check whether two nodes are connected, you just have to do a matrix lookup instead of (potentially) inspecting every edge:

friendships[0][2] == 1 # True, 0 and 2 are friends  
friendships[0][8] == 1 # False, 0 and 8 are not friends

friends_of_five = [i # only need  
                    for i, is_friend in enumerate(friendships[5]) # to look at  
                    if is_friend] # one row

Resource

References

This article is highly influenced by the book Data Science from Scratch by Joel Grus, this is a fantastic book to read for people getting started with data science. If you don't like reading book then save the series as there is more to come.

Data Science: Basic Mathematics

Azad Kshitij — Tue, 08 Nov 2022 11:30:00 +0000

You might be thinking how much math we need to know to get started with Data Science? There is no specific answer to that but I have the way for you to get started.

Why?

Unlike software engineering data-science is not mostly about programming, it's more about data and understanding relation between datapoints. In order to do that we need eye for that and most of us don't have that so we need math to make sense of the data. A significant portion of your ability to translate your data science skills into real-world scenarios depends on your success and understanding of mathematics. Mathematical knowledge is necessary for data science careers because machine learning algorithms, data analysis, and insight discovery all depend on it. Although there are other requirements for your degree and employment in data science, math is frequently one of the most crucial.

Data Science 💗 Maths

Let's talk about the most common types of math that you will use in your data science career.

Linear Algebra

Resource

Statistics & Probability

Statistics refers to the mathematics and techniques with which we understand data. This is essential in machine learning when working with classifications such as logistic regression, discrimination analysis and hypothesis testing and distributions.

Resource

Calculus

Calculus is used in machine learning to create loss/cost/objective functions, which are used to train algorithms to achieve their goals. It contains study of derivatives, curvature, divergence, and quadratic approximations.

Resource

In future article we will talk about each topic in detail and how to use them and when to use them so stay tuned and save the series. If you have anything to say comment down below I'm new to blog writing so any type of feedback is appreciated Thanks.

References

Data Science: Better graph with matplotlib and python

Azad Kshitij — Sat, 29 Oct 2022 12:30:00 +0000

Data visualization is used to shows the data in a more straightforward representation and more comfortable to be understood. It can be formed in histograms, scatter plots, line plots, pie chart, etc. Many people are still using Matplotlib as their back-end module to visualize their plots. In this story, I will give you some tricks, 5 powerful tricks in using Matplotlib to create an excellent plot.

Why use plots and charts?

Plot is the representation of data graphically in any format. It is the most efficient way of communicating facts with non-technical professionals and helps them draw inferences from the data. Plot is a powerful way to graphically represent your data. It makes it easier for Data Analysts, and Data Scientists to analyze data and derive meaningful insights. There are a lot of tools available that can help you visualize your data such as Tableau, Looker, Sisense, etc. As Data Science is all about Numerical Data that is processed with scientific methods and algorithms, plotting is a way to converts the dataset into visual content. Datasets can be represented graphically with plots or graphs.

Important steps for better plots in data science

Shown below are the important aspects of Data Science Visualization in the field of Data Science:

Data Cleaning
Data Exploration
Identifying Trends
Presenting Results

1) Data Cleaning

Visualization can help detect Null values of data items in large datasets by representing them distinctively. Data could consist of redundancy and noise which needs to be eliminated before analysis. Visualizing these datasets gives you a complete overview without assumptions about the correctness of the data.

2) Data Exploration

The visual representation of data helps both technical and non-technical professionals/personnel have an overview of what the data is about. They can then tinker and also draw conclusions based on what they see.

3) Identifying Trends

Data Scientists and Data Analysts, at times, work with real-time data to derive meaningful trends. As real-time data is always fluctuating, it becomes difficult to analyze it. This is where the data can be visualized using charts and graphs for better understanding. This helps in making informed decisions not just in Data Science but in Business Intelligence in general.

4) Presenting Results

The result of analysis at any point of processing can always be visualized. The visualization can be done by anyone with knowledge of Data Science Visualization Tools, not just a Data Scientist. So far the data is from a supported data source, a Data Science Visualization Tool can represent it in its supported formats such as Graphs, Curves, or Charts

Type of Data Plots & their Significance

Following are the 7 most commonly used Data Plotting Types in the field of Data Science Visualization:

Bar Plots
Line Plots
Scatter Plots
Area Plots
Histogram
Pie Chart
Bubble plot

1) Bar Plot

A Bar Plot is very easy to understand and therefore is the most widely used plotting model. Simplicity and Clarity are the 2 major advantages of using a Bar Plot. It can be used when you are comparing variables in the same category or tracking the progression of 1 or 2 variables over time. For example, to compare the marks of a student in multiple subjects, a Bar Plot is the best choice.

2) Line Plot

A Line Plot is widely used for the comparison of stockpiles, or for analyzing views on a video or post over time. The major advantage of using Line Plot is that it is very intuitive and you can easily understand the result, even if you have no experience in this field. It is commonly used to track and compare several variables over time, analyze trends, and predict future values.

3) Scatter Plots

A Scatter Plot uses dots to illustrate values of Numerical Variables. It is used to analyze individual points, observe and visualize relationships between variables, or get a general overview of variables.

4) Area Plots

An Area Plot displays Quantitative Data graphically. It is very much like Line Plot but with the key difference of highlighting the distance between different variables. This makes it visually clearer and easy to understand. It is generally used to analyze progress in Time Series, analyze Market Trends and Variations, etc.

5) Histogram

A Histogram graphically represents the frequency of Numerical Data using bars. Unlike Bar Plot, it only represents Quantitative Data. The bars in the Histogram touch each other i.e. there is no space between the bars. It is generally used when you are dealing with large datasets and want to detect any unusual activities or gaps in the data.

6) Pie Chart

A Pie Chart represents the data in a circular graph. The slices in a Pie Chart represent the relative size of the data. Pie Chart is generally used to represent Categorical Data. For example, comparison in Areas of Growth within a business such as Profit, Market Expenses, etc.

7) Bubble Plots

A bubble plot is a scatter plot where the circle size is mapped to the value of a third numeric variable and You can also map a 4th value to the color of bubble.

Enough talking let get to the spicy stuff 😁😁. We will be using python for plotting graphs, all of the above graphs are created with python and matplotlib.

Get Started

Matplotlib

Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python. Matplotlib makes easy things easy and hard things possible.

Create publication quality plots.
Make interactive figures that can zoom, pan, update.
Customize visual style and layout.
Export to many file formats.
Embed in JupyterLab and Graphical User Interfaces.
Use a rich array of third-party packages built on Matplotlib.

To install matplotlib run pip install matplotlib or if you are using conda environment (as you should be) use conda install matplotlib. That's it we are all set...

[!note]-
For the tutorial I will be using some dataset and some libraries which are not part of this tutorial. This is a intermediate level tutorial if you want to start with basic follow this tutorial.

Styling Graph There are lot of ways to style our graphs we can use built in styles, create our own styles or can combine both ways and get benefit of both worlds. Theme:

colors = cycler(color=plt.get_cmap("tab20").colors)  # ["b", "r", "g"]

mpl.style.use("seaborn-pastel")
# mpl.style.use("seaborn-whitegrid")
mpl.rcParams["figure.figsize"] = (20, 5)
mpl.rcParams["axes.facecolor"] = "white"
mpl.rcParams["axes.grid"] = True
mpl.rcParams["grid.color"] = "lightgray"
mpl.rcParams["axes.prop_cycle"] = colors
mpl.rcParams["axes.linewidth"] = 1
mpl.rcParams["xtick.color"] = "black"
mpl.rcParams["ytick.color"] = "black"
mpl.rcParams["font.size"] = 12
mpl.rcParams["figure.titlesize"] = 25
mpl.rcParams["figure.dpi"] = 100
mpl.rcParams["savefig.dpi"] = 100
mpl.rcParams["legend.fontsize"] = 20
mpl.rcParams["legend.frameon"] = False
mpl.rcParams["legend.facecolor"] = "white"
mpl.rcParams["legend.edgecolor"] = "white"
mpl.rcParams["legend.fancybox"] = False
mpl.rcParams["legend.numpoints"] = 1
mpl.rcParams["legend.scatterpoints"] = 1
mpl.rcParams["legend.markerscale"] = 1
mpl.rcParams["legend.labelspacing"] = 0.5
mpl.rcParams["legend.handlelength"] = 2
mpl.rcParams["legend.handletextpad"] = 0.8
mpl.rcParams["legend.borderpad"] = 0.5
mpl.rcParams["legend.borderaxespad"] = 0.5
mpl.rcParams["legend.columnspacing"] = 2
mpl.rcParams["legend.framealpha"] = None
mpl.rcParams["legend.edgecolor"] = "inherit"

You can use seaborn-pastel or seaborn-whitegrid as a base theme and add your own parameters to customize. As I mostly have starched image that's why I have figure size to (20,5), you can also change the font size and fonts etc. That's it now you will have better plots to present to your boss or non-technical personnel. In future article we will talk about each type of chart and how to use them and when to use them so stay tuned and save the series.

References

https://www.youtube.com/watch?v=1CQZibJdN7Q&t=11s

Data Science: Best Vscode settings

Azad Kshitij — Thu, 27 Oct 2022 11:30:00 +0000

VS code has become the tool for any programming project and data science is one of them. Most data scientists like to use jupyter notebook or jupyter lab but what they don't know is VS Code support jupyter notebook and you can do much more than just use jupyter notebook. one of the best feature of vscode for data science is Interactive python window will talk about that later, lets talk about some other features.

1. IntelliSense

IntelliSense is a general term for various code editing features including: code completion, parameter info, quick info, and member lists. IntelliSense features are sometimes called by other names such as "code completion", "content assist", and "code hinting."

When we install python extension for vscode it comes with IntelliSense and it works as intended. You can improve IntelliSense with type annotation in your project.

As you can see in above image there is no suggestions for the name parameter but as soon as we define type of parameter vscode will show all the available methods related to the datatype (below image).

2. Indent (Space vs Tabs)

At the end of the day, tabs versus spaces is truly a matter of preference, however the tab is still the character specifically designed for indentation, and using one tab character per indentation level instead of 2 or 4 spaces will use less disk space / memory / compiler resources and the like.

It depends on individual, I prefer tab and set it as 2 space (2 column) width as it occupies less space and I only have to key only one time and every stroke matters. To change indent size in vscode follow below steps...

3. Text Warp

Most of us don't prefer text to run off the screen. To toggle wrapping on or off on a per-file basis, simply press ALT + Z . If you almost never want to turn text-wrapping off, you can make it the default. Inside settings.json, simply paste in the following code, and your text will wrap by default:

{ "editor.wordWrap": "on" }

4. Evaluate Math Expression

You know vscode have inbuild calculator? You can evaluate basic math operation. Just select the expression, press ctrl+shift+p and search for "Evaluate and you will find the option".

5. Format on Save

Install black, a python formatter that will help us follow a similar formatting for all our data science projects. command:

pip install black

Open your VSCode settings, by going 'Code -> Preferences -> Settings'. Search for "python formatting provider" and select "black" from the dropdown menu:

In the settings, search for "format on save" and enable the "Editor: Format on Save" option:

6. Snippets

You can install extensions for snippets but you can also define your own snippets. With this features you can type few character to type repeated code like imports statements, plotting functions etc.

7. Interactive Python Window

VSCode support interactive python window, where you can run python code just like jupyter notebook but even better as your main code is stored in a single python file. This is a very important and productive features for data scientist. For this to work you need to change the setting with...

"jupyter.sendSelectionToInteractiveWindow": true // <---- Very Important

8. Remote access to VMs

Odds are that you’ll run code in a server, a virtual machine or some other hardware that you might want to connect to via SSH. Fortunately, you can connect to any of them and still carry on using your local VS Code installation, by using the Remote SSH extension.

9. CSV

As a data scientists you have to deal with a lot of file types that stores the data, CSV is the most famous file type to store huge amount of data. As CSV is a txt file so you can open it directly in vscode but to make it more usable and readable we can use extensions such as...

Rainbow CSV which will provide highlighting for your csv file..

Edit csv which has a excel like UI to edit and update your csv file.

As the article is getting longer we will talk about some more settings in future articles so make sure to follow the series to not miss any updates. If you have anything to say comment down below I'm new to blog writing so any type of feedback is appreciated Thanks.

Data Science: Setup Visual Studio Code with Python

Azad Kshitij — Wed, 19 Oct 2022 03:47:47 +0000

Visual Studio Code

Visual Studio Code(VS Code) is one of the most famous code editor build by Microsoft which free and build on open source that runs everywhere. VS Code support almost every language and have support of many different extensions, with many themes. But usually most data scientists prefer to use Jupyter Notebook or Jupyter Lab for their data science projects as, visual studio support jupyter notebook files as well but there are many other features to choose VS Code over Jupyter Notebook.

Reasons to choose VS Code

Better auto completion
- We all know how bad the auto completion in in Jupyter Lab, we need to press ctrl+space to get suggestions. VS Code provides smart completions based on variable types, function definitions, and imported modules.
Extensions
- I know there are many extensions available in Jupyter Lab but are not enough and not that powerful. With VS Code's powerful extension system community have created many really impressive extensions and there are verified extensions provided by Microsoft.
Debugging
- Debugging is not that common thing for Data scientists but with the debugging features we can analyze intermediate variables to find problems in our code.
Live share
- you can collaboratively edit documents in real-time in VS Code, allowing you to do some pair programming, being apart.
Support for all Common Files
- As a data scientist we often need to open .csv, .json, .yaml etc. files and you know what VS Code support these out of box and you can install extensions to better handle there file types.

Enough talking about stuff lets start doing...

Get Started

Install VS Code

Go to the official download page for VS Code and follow usual installation steps. make sure to tick...

Add "Open with Code" action to Windows Explorer file context menu
Add 'Open with Code" action to Windows Explorer directory context menu this will allow us to open any folder directly in vscode.

After you will open Vscode for the first time it will look like this. Choose your proffered theme. Lets install some extensions...!

Install Extensions

Click on the extension button on the left side bar.
Search for python in the search bar.
Click on the first extension you find and install it. (make sure it is provided by Microsoft).

This will install all the necessary extensions for python

Now you are all set to start working on your project, but there are some more setting that you can change to make it even more suitable as per your need. You can change layout (I prefer the side bar on the right as it doesn't change the view when I open and close it). You can change color theme if you want to my personal favorite is Atom One Dark Pro.

Binaryify / OneDark-Pro

Atom's iconic One Dark theme for Visual Studio Code

One Dark Pro

GitHub Repo

Atom's iconic One Dark theme, and one of the most installed themes for VS Code!

MySQL Proxy

A plugin for IDEA that records code CRUD operations, helping you identify potential issues in SQL and providing optimization suggestions.

一个记录代码 CRUD 操作的 IDEA 插件，帮助您识别潜在问题的SQL并提供优化建议。

SCREENSHOT

Italic

Default theme

Flat theme

Darker theme

Retro theme

Gnome theme

Terminal

Donation

If you like this extension, you could donate via PayPal It will encourage me to make this extension better and better!

Thanks List:

Andreas Georgiadis

Setting

Built in themes

Notice

Setting only support default theme(One Dark Pro).

Markdown preview style

You can toggle whether to use markdown style in vscode setting (default use)

Suggest Editor Settings

 "editor.fontSize": 20,
 "editor.lineHeight": 30,
 "editor.fontFamily": "JetBrains Mono",

JetBrains Mono Download: https://www.jetbrains.com/lp/mono

Tweaks & theming

If you want to play around with new colors, use the setting workbench.colorCustomizations to customize the currently selected…

View on GitHub

Settings

Now lets change some settings for productivity.

Change font size as per your liking here are my font settings.

 "editor.fontSize": 22,
 "terminal.integrated.fontSize": 20,
 "editor.fontFamily": "Fira Code",
 "editor.fontWeight": "450",
 "editor.fontLigatures": true,

Necessary fonts...

tonsky / FiraCode

Free monospaced font with programming ligatures

Fira Code: free monospaced font with programming ligatures

Read in Español | 简体中文 | 日本語

Problem

Programmers use a lot of symbols, often encoded with several characters. For the human brain, sequences like ->, <= or := are single logical tokens, even if they take two or three characters on the screen. Your eye spends a non-zero amount of energy to scan, parse and join multiple characters into a single logical one. Ideally, all programming languages should be designed with full-fledged Unicode symbols for operators, but that’s not the case yet.

Solution

Fira Code is a free monospaced font containing ligatures for common programming multi-character combinations. This is just a font rendering feature: underlying code remains ASCII-compatible. This helps to read and understand code faster. For some frequent sequences like .. or //, ligatures allow us to correct spacing.

Download & Install

Then:

View on GitHub

Jupyter Notebook

"jupyter.askForKernelRestart": false,
"jupyter.interactiveWindowMode": "perFile",
"jupyter.themeMatplotlibPlots": true,
"jupyter.textOutputLimit": 0,
"jupyter.sendSelectionToInteractiveWindow": true // <---- Very Important
"notebook.outline.showCodeCells": true,
"notebook.lineNumbers": "on",
"notebook.outputFontFamily": "CaskaydiaCove NF",
"notebook.cellToolbarVisibility": "hover",
"notebook.undoRedoPerCell": false,

Now you are all set to change the future. In this article I talked about setting VS Code for data science in the future I will share how to properly use vscode to speed up your process and be more productive so make sure to follow the series and like to post if you have anything to say comment down below I'm new to blog writing so any type of feedback is appreciated Thanks.