DEV Community: Aziza Afrin

Powerful Pandas! Part 2

Aziza Afrin — Sun, 04 Feb 2024 08:48:50 +0000

This is the second blog on Pandas Library. In this write up we would show what else can be done with Pandas library apart from content mentioned in Powerful Pandas! Part 1.

Dealing with Missing Values

While analyzing data we often face a problem of missing values in our data. Let's see how can we deal with that in python with pandas library.
Let's import pandas and numpy library into our workspace.

import numpy as np
import pandas as pd

Now create a data-frame with some missing value.

d={'A':[1,2,np.nan],'B':[5,np.nan,np.nan],'C':[1,2,3]}

df=pd.DataFrame(d)
df

1. Dropping Missing Values:

To drop all the rows with missing values:

df.dropna()

To drop all the column with missing values:

df.dropna(axis=1)

We can also set limit like how many missing entries are allowed.

#it keeps row 1, as it has at lest 2 not NaN entry
df.dropna(thresh=2)

2. Replacing Missing Values:

we can replacing missing entries with anything like string, average values etc.

#replacing missing values
df['A'].fillna(value=df['A'].mean())

Joining Rows

Using group by method to light a group rows of data together and call aggregate functions.
Groupby allows you to group together rows based off of a column and perform an aggregate function on them

data={'Company':['Gp','Gp','Teletalk','Teletal','Robi','Robi'],
                'Person':['Alam','Mim','Tina','Rimi','Rafi','Pranto'],
                'Sales':[200,170,225,439,410,324]}

df=pd.DataFrame(data)

df_new=df.groupby('Company')

df_new.mean()

Check out this link for practice and comment down your thoughts.(https://www.kaggle.com/code/azizaafrin/powerful-pandas-part-2)
Happy Learning!❤️

Aziza Afrin

All about MatPlotlib

Aziza Afrin — Sat, 24 Jun 2023 16:57:31 +0000

MatPlotlib

Today, we are gonna talk about a python library called MatPlotlib. It allows to create different types of visualizations. It also can produce animated and interactive plots with simple syntax. If you are interested to learn more about MatPlotlib, here is the link of the matplotlib official website: matplotlib

Installation

Install and import matplotlib with following code:

!pip install matplotlib

import matplotlib.pyplot as plt

Intro

Let's start creating plot. We can go for two different approaches:

Functional Method
Object Oriented Method

1. Functional Method

Let's define two array x and y.

import numpy as np
x=np.linspace(0,5,11)
y=x**2

And now plot x and y.

#functional method for plot
plt.plot(x,y,'r')   
plt.xlabel('X')
plt.ylabel('Y')
plt.title('Title')

Here,
'r' will gives a red coloured plotted line.
plt.xlabel('X') will label the x axis.
plt.ylabel('Y') will labl the y axis.
plt.title('Title') wiil gives a title for the plot.

Multiplot

We can create several plot at a time using subplot. For example:

plt.subplot(1,2,1)
plt.plot(x,y,'r')

plt.subplot(1,2,2)
plt.plot(y,x,'b')

Run this code on the linked kaggle notebook to see these subplot.

2. Object-Oriented Method

We have a lot more control on the figure comparing with functional approach. We start by creating an empty figure object and then adding feature simultaneously.

fig=plt.figure()   #it will create an empty canvus figure

axes=fig.add_axes([0.1,0.1,0.8,0.8])

axes.plot(x,y)
axes.set_xlabel('X')
axes.set_ylabel('Y')
axes.set_title('Title of the plot')

We can also plot a figure on another plot like this:

fig=plt.figure()    #it will create an empty canvus figure
axes1=fig.add_axes([0.1,0.1,0.8,0.8])
axes2=fig.add_axes([0.2,0.5,0.4,0.3])

axes1.plot(x,y)
axes1.set_title('Larger plot')

axes2.plot(y,x)
axes2.set_title('Smaller plot')

Sub Plots

We can also create sub plots like functional approach.

fig,axes=plt.subplots(nrows=1, ncols=2) #arranging plots in rows and column

axes[0].plot(y,x,'r')
axes[0].set_title("The red line")

axes[1].plot(x,y,'g')
axes[1].set_title("The green line")

Figure Size and DPI

We can also control the figure size and dpi of the plot.

figsize is a tuple of the width and height of the figure in inches
dpi is the dots-per-inch (pixel per inch).

fig = plt.figure(figsize=(8,4), dpi=100)

Saving Figure

Simply save the produced figure with:

fig.savefig("filename.png", dpi=200)

Legends, labels and titles

The legends, labels and titles of the figure can also be controlled.

#Legend

fig = plt.figure()

ax = fig.add_axes([0,0,1,1])

ax.plot(x, x**2, label="x**2")
ax.plot(x, x**3, label="x**3")
ax.legend(loc=3)

# Lots of options for legend locations

ax.legend(loc=1) # upper right corner
ax.legend(loc=2) # upper left corner
ax.legend(loc=3) # lower left corner
ax.legend(loc=4) # lower right corner
ax.legend(loc=10) # center
ax.legend(loc=0) #let matplotlib to choose location

# Commonly used
ax.legend(loc=0) # let matplotlib decide the optimal location
fig

Plot Appearance

The plot appearance can also be controlled by matplotlib.

#using RGB hex code for diffrent colour
fig=plt.figure()
ax=fig.add_axes([0,0,1,1])
ax.plot(x,y,color='#FF8C10')

The line width and transparency of the plotted line can be controlled in matplotlib library.

#setting alpha for transparency of the plotted line
fig=plt.figure()
ax=fig.add_axes([0,0,1,1])
ax.plot(x,y,color='#FF8C10',linewidth=8, alpha=0.6)

We can transform the plotted line from a simple line to a -. line or many more.

#using ls or line style for different typs of plotted line
fig=plt.figure()
ax=fig.add_axes([0,0,1,1])
ax.plot(x,y,color='violet',lw=5,alpha=0.7,linestyle="-.")

Marker

Different markers can also be added to the plotted line.

#using ls or line style for different typs of plotted line
fig=plt.figure()
ax=fig.add_axes([0,0,1,1])
ax.plot(x,y,color='violet',lw=5,alpha=0.7,linestyle="-.")

An example of different makers:

Code of this plot is available in the linked Kaggle notebook.

Axis Range Customization

We can customize the axis range for our convenience

fig, axes = plt.subplots(1, 3, figsize=(12, 4))

axes[0].plot(x, x**2, x, x**3)
axes[0].set_title("default axes ranges")

axes[1].plot(x, x**2, x, x**3)
axes[1].axis('tight')
axes[1].set_title("tight axes")

axes[2].plot(x, x**2, x, x**3)
axes[2].set_ylim([0, 60])
axes[2].set_xlim([2, 5])
axes[2].set_title("custom axes range");

Different Plots

Plots like histogram, boxplot, scatter plot can also be produced with matplotlib.

data = [np.random.normal(0, std, 100) for std in range(1, 4)]

# rectangular box plot
plt.boxplot(data,vert=True,patch_artist=True);

You can practice more example at your own. The notebook link is given below. Follow this link to practice on your own.
Notebook Link: [https://www.kaggle.com/code/azizaafrin/matplotlib]
Github Link: Github

Happy Learning!❤️

Aziza Afrin

Powerful Pandas! Part-1

Aziza Afrin — Sun, 25 Dec 2022 14:36:45 +0000

Today we gonna cover Pandas library.
Pandas is a python library which usually use for data manipulation and data analysis. Mostly used in Data Science and Machine Learning. In this notebook we gonna show how powerful pandas library is!

Let's get started!

Let's call the numpy and pandas library into our workspace. Here, we are using kaggle notebook where these libraries are already installed.

import numpy as np
import pandas as pd

If these libraries aren't installed in you IDE, you have to install them before calling them.

1. Pandas Series

Let's create a series with pandas.

a1=['a','b','c']
my_data=[50,70,30]
ar=np.array(my_data)
d={'a':50,'b':70,'c':30}

pd.Series(data=my_data, index=a1)

Same thing could be done with:

pd.Series(my_data,a1)

and also with:

pd.Series(d)

Indexing in series

series1=pd.Series([1,2,3,4],['A','B','C','D'])
series1

series1['C']

2. Pandas DataFrames

Call the required library for creating data frame in python with pandas.

import numpy as np
import pandas as pd

from numpy.random import randn

Setting a fixed seed point as we want to draw the same set of random numbers each time we run the code. Otherwise our result would be vary every time we run the code.

np.random.seed(1011)

df=pd.DataFrame(randn(5,4),['A','B','C','D','E'],['W','X','Y','Z'])
df

Here is our data frame.

If we want to grab the column 'W', output gives a series

df['W']

another way to grab a column like sql

df.W

If we want to grab multiple column, output gives a dataframe

df[['W','Z']]

Add a column
Let's add a column to the data frame

df['H']=df['W']+df['Z']

Delete a column
To delete a column we will use drop function

df.drop('H',axis=1)

But if you run again the dataframe new column is still there, so we have to add another argument.

df.drop('H',axis=1,inplace=True)

this permanently deletes the column.

Selecting rows, labelbased index:

df.loc[['A','B'],['W','Y']]

Conditional selection
Select rows where W column value is greater than zero along with Y and X column.

df[df['W']>0][['Y','X']]

Multiple selection: Can you explain what result will give the following code?

df[(df['W']>0) & (df['Y']>1)]

df[(df['W']>0) | (df['Y']>1)]

Multi-level index or index higher key
Now we will create a data frame with index more than one level.

outside=['G1','G1','G1','G2','G2','G2']
inside=[1,2,3,1,2,3]
hi_index=list(zip(outside,inside))
hi_index=pd.MultiIndex.from_tuples(hi_index)

df=pd.DataFrame(randn(6,2),hi_index,['A','B'])

df

To grab everything under G1

df.loc['G1']

Try to explain which value we want to grab with following code:

df.loc['G2'].loc[2]['B']

3. Read CSV file

CSV files contains plain text and is a well know format that can be read by everyone including Pandas.

df = pd.read_csv('/kaggle/input/pandas/data_set.csv')

print(df.to_string())

4. Correlations

The relationship between each column in your data set can be calculated by cor() method. The relationship between the columns of our data

df.corr()

Correlation value varies from -1 to 1. Negative value indicate negative relationship that is if values of variable increases, other will decreases. Positive value mean a positive relationship, values of variable increases, other will increase too. 1 indicates perfect relationship.

You can practice more example at your own. The notebook link is given below. Go to the link and practice.
Notebook Link: [https://www.kaggle.com/code/azizaafrin/powerful-pandas-part-1]

Happy Learning!❤️

Aziza Afrin

New to NumPy?

Aziza Afrin — Tue, 29 Nov 2022 15:22:22 +0000

In python, the library which is used to create and manipulates matrices is NumPy. Basically, NumPy works with arrays. For instances array is an ordered series or arrangement of numbers/objects. NumPy stands for "Numerical Python".

Let's see how can we use Numpy to work with matrices.

First import NumPy module with running the following code.

import numpy as np

1. NumPy Arrays

Let us create a list.

my_list=[1,2,3]

Now call np.array to craete a NumPy array with the list.

arr=np.array(my_list)

And type arr to see the array.

arr

We can create this array by directly putting the values like:

arr=np.array([1,2,3])

The array we have created was an one dimensional array. We can create two dimensional array. Let's create a three by three array.

mt=[[1,2,3],[4,5,6],[7,8,9]]
np.array(mt)

Let's create a sequence starting at zero, step size or difference is 2, to 11

np.arange(0,11,2)

Here, we are creating a five by five dimensional array containing only zeros:

np.zeros((5,5))

We can also create matrics having only value 1, identity matrics etc (example are given in the notebook).

We can create array with values from a specific distribution such as standard normal distribution, uniform distribution, poisson distribution, binomial distribution, chi square distribution etc. For example we can create a two dimensional array with sample from uniform distribution(0 to 1).

np.random.rand(5,5)

We can find the maximum or minimum values is the array.

#array returning random integers
ranarr=np.random.randint(0,50,10)
ranarr.max()

While returning the maximum value of the array we can also find the location for the maximum value.

ranarr.argmax()

2.NumPy Indexing and Selection

Let us create a two dimensional array first.

arr=np.array([[5,10,15],[20,25,30],[35,40,45]])
arr

and we get
array([[ 5, 10, 15], [20, 25, 30], [35, 40, 45]])

If we want to get the value 10 we type

arr_2d[0][1]

In python, indexing starts from 0, so if we want to grab except the third row we type, because third row indexed as 2.

arr_2d[:2]

3. NumPy Operations

Let's do some arithmatic operations with numpy

ar=np.arange(1,11)

Addition:

ar+ar

Substraction: Substract array from a array and substract a scaler from array

ar-ar
ar-4

Multiplication:

ar*3

4. Universal Array Function

Universal Functions used to implement vectorization in NumPy which is more faster than iterating over elements. For example:
To find a array containing square root of each values in array we type

np.sqrt(ar)

Taking exponential of each values of the array:

np.exp(ar)

To find a product of arrays:

a= np.array([1, 2, 3, 4])
b = np.array([5, 6, 7, 8])

np.prod([a, b])

You can practice more example at your own. The notebook link is given below. Go to the link and practice.
Notebook Link: [https://www.kaggle.com/code/azizaafrin/python-numpy]

Happy Learning!<3