DEV Community: Tifey_

How to use Pandas for Csv data in Python

Tifey_ — Wed, 19 Jan 2022 12:05:47 +0000

In the last article, you learnt how to use the python csv module in order to work with csv files, you learnt the conversion of csv files into dictionaries and lists.

In this article, we are going to treat a software module called Pandas. It mainly works with structured data I.e csv data, tabular data, etc. It's no doubt that most Python programmers make use of this library when they come across csv files and this is due to it flexibility and usability.

How does Pandas work

.
The library arranges csv files into a tabular form making the first rows of each columns the head of the table. These heads could be used as an attribute to access the items in its columns.

What you'll learn

Installing Pandas
Data frame and series.
Using Pandas to convert a csv into list.
Pandas to convert csv into dict
Calculating mean, median mode and other statistical functions.

Installing Pandas

Visit the pypi website and search for the pandas library. Click no its latest version and install. It will be configured in you python editor automatically.

Using Pandas to read csv files
Pandas is a library made specially for structural and tabulated datas so it does work on a csv file.
To do this:

import the pandas in your main.py
call the read_csv() function. The read_csv() takes in the file path of the csv file in a strung type. Data = pandas.read_csv("my_cs_file.csv")

DATAFRAME AND SERIES

.
DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dict of Series objects. It is generally the most commonly used pandas object.

How to create a dataframe.
A dataframe accepts list nested in a dictionary. You will need to create the nested dictionary before you could create a dataframe

Import pandas as pd
Dict = {"Names" : ["Scott", "Abey", "Boluwatife ", "Thomas ", "Mike",],
"Scores" : [67, 82, 98, 46, 89]
}
df =pd.DataFrame(Dict)

You can print the df to see how it looks, exactly like a table.

N:B a list is nested in the dictionary. The keys in the dictionary will be the heads of the tables and each value which is a list will be the items in heads

Series:
Series is a one-dimensional labeled array capable of holding any data type (integers, strings, floating point numbers, Python objects, etc.). The axis labels are collectively referred to as the index. The basic method to create a Series is to call:

s = pd.Series(data, index=index)

How to create a Series

Dict = {"Names": "Fola",
"Score":12} 

df_series = pd.Series(Dict)

Get columns of dataframe
Columns can be accessed by columns heads. Taking the dict as an example. Of you want to access the name of students in the table.

Names = df.Names Or Names = df["Names"]

Get rows in a Dataframe
The best way to access a row in pandas dataframe is to check through the column of a table and locate the name of the item at the row we need.
row1= df[df.Names == "Scott"]

This will print every item in the same row with the name Scott

USING PANDAS TO CONVERT CSV INTO LIST.

Unlike the python pre built csv module, the pandas library already has a method for creating a list from a csv file.

-----to_list()---
This method takes a file path as a string input. The csv path we need to convert.
Example:
Import pandas as pd new_list = pd.to_list("file.csv")

With these lines of codes, the output will be the list of items in your csv file. Unlike the csv module where you run the for loop for number of times to get what pandas has in just two(2) lines.

USING PANDAS TO CONVERT CSV INTO DICTIONARY.

-------to_dict()--------
The method also takes the dataframe as an input and convert its item into dictionary using the head words as the key and other rows as its value.

USING PANDAS FOR STATISTICS

The pandas library has also made available method for executing statistical tasks.

Mean : .mean() The method takes a list of numbers and return the mean value of it list
Mode : .mode()
It takes a list of numbers and returns the number with the highest frequency
Median: .median()
It takes a list of numbers, arranges them in an ascending order and returns the number at the intersection (middle)
Product of numbers: .prod()
Accepts a an array of numbers and returns its product.
Cumulative numbers. .cumin()
It takes an array of numbers and returns the cumulative value.

Other available methods are:
.max(), .min(), .quantile, .cummax(), .var(), etc.

To know more about the pandas library visit the pandas library documentation

To have more insight about pandas library;
Visit the pandas documentation library
https://pandas.pydata.org/docs/user_guide/index.html

----------------If this was helpful, kindly react and comment----------------

How to use Python Csv Module to analyse Csv Files

Tifey_ — Sat, 15 Jan 2022 23:39:23 +0000

A CSV file (Comma Separated Values file) is a type of plain text file that uses specific structuring to arrange tabular data. Because it's a plain text file, it can contain only actual text data—in other words, printable ASCII or Unicode characters .

When time comes for you to use a Csv file or a spreadsheet in your program. This article will be of great help as you will be able to open and work with csv files, know what modules are used for csv files, modify and work with csv as if they were they were yours

Let's get Started

!!!
Python as a high level programming has a pre built module for csv files which is called "csv".
Below are the steps for using Python pre built module to analyse csv files

Importing the Csv Module
Since the module is already prebuilt you will only import else you will need to install

import csv
Opening the csv file using the with statement.
The with statement is preferred by most Python developers because it automates the close() function after using its open().
The with statement

with open ("notes.csv") as csv_file:
data=csv.reader(csv_file)

open() : is a python function used to open files in a main.py
csv_file : is a variable which stores the file that is been open
data : is a variable assigned to the information extracted from the csv_file after its been accessed by the csv module.
.reader() : is a csv function for reading the content of a opened csv file.

N:B the reader() is read only. If you print data variable at this stage, it will generate numbers and code which represents the location of the csv file on the computer ROM.

Converting a csv file to List and Dictionary**

After the file is been read by the csv module. You could start making your modifications. You can convert it to either a list or dictionary. We will walk you through the two conversions.

1). Converting csv file to a list.

a. Create an empty stored in a variable.
List = [].
b. Loop through the data( variable which you use to store the content of the file when the reader() was used).
c. Append each item in the data to your new list.

Your codes should be in the below format

When the above code is run. It will outpu the list of items in the csv file beginning from the first row and convert each row to a list type object. It will output the below

With this, you can work with your csv files like you do with your list objects.

2). Converting a csv file to Dictionary.
A dictionary works with its keys and values.
To convert a csv file into a dictionary, we will need a key and value.

In this article, we will use an item in the csv_file as the key and the other as its value since it is of two columns.
a). Create an empty dictionary and assign it to a variable.
b). Loop through the data
c). Add the key and value to the empty dictionary.
Your codes should be in the below format:

The key used in the above is the item at index 0 on each row of the Csv file and its value is that at index 1.
The following would be its output

Also with the above out you can also treat your Csv file like that of a dictionary. For example you could modify its value, its key, change the values and keys, and other dictionary features.

Now let's advance a little.

How would it be if we could make all letters in the csv file become a value to a key called Letters and do likewise to the words too? It sound nice right?.

Steps
A). Create a new empty list for the letters.
B). Use a for loop to append all the letters to the new list excluding the "letter" string.
It should be like the below

import csv

> list = []
with open("Notes.csv") as csv_file:
    data = csv.reader(csv_file)
    for each in data:
        list.append(each)
    Letters = []
    for Char in list:
    If char[0] != "letters":
        Letters.append(char[0]

The if statement in the above is used to exclude the letter string from the list
Char = a loop variable
Char[0] = the item at index 0 of Char variable which is the letter in csv file.

C). Create another empty list with a variable
words.
D). Use a for loop to append all the word to the words list excluding the "words" string.
Your codes should be like below

import csv

    list = []
    with open("Notes.csv") as 
    data_file:
    data = csv.reader(data_file)
    for each in data:
        list.append(each)

    Letters = []
    for char in list:
        if char[0] != "letters":
            Letters.append(char[0]

    Words = []
    for word in list:
        if word[0] != "words":
             Words.append(word[0])

E). Create an empty dictionary called dict.
F). Add a key called Letters to the empty dict and makes it value to be the Letters list object
G). Add another key called Words to the dict and makes it value to be words list object.

Your code should be like the following

Following the above steps, you have completed the task. If you print the dict object you will get what task we want.

It's result will be:

The letters are now the value of Letters and the words is now the value of Words.

It is very obvious that it took many lines of code to carry out a mini task using the python csv module.

Another module which python programmers use when the come across csv files is a module called Pandas.

Pandas is a software library written for the Python programming language for data manipulation and analysis. In particular, it offers data structures and operations for manipulating numerical tables and time series

In our next article, we will explain more about pandas

Kindly like, comment and follow !!!!!

How to Use Python Csv Module

Tifey_ — Sat, 15 Jan 2022 22:19:14 +0000

How to Use Csv Module with Csv files in Python

Csv file which means Comma separated values files. It is a type of file with extension ".csv" which is used as a spreadsheet i.e it is embedded with data in a grid of rows and columns.

Let's get started!!!

Python as a high level programming has a pre built module for csv files which is called "csv".
Importing the Csv Module
Since the module is already prebuilt you will only import else you will need to install
import csv

Opening the csv file using the with statement.
The with statement is preferred by most Python developers because it automates the close() function after using its open().
The with statement

with open ("notes.csv") as csv_file:
data=csv.reader(csv_file)
open() : is a python function used to open files in a main.py
csv_file : is a variable which stores the file that is been open
data : is a variable assigned to the information extracted from the csv_file after its been accessed by the csv module.
.reader() : is a csv function for reading the content of a opened csv file.

N:B the reader() is read only. If you print data variable at this stage, it will generate numbers and code which represents the location of the csv file on the computer ROM.

Converting a csv file to List and Dictionary

After the file is been read by the csv module. You could start making your modifications. You can convert it to either a list or dictionary. We will walk you through the two conversions.

Converting csv file to a list.
Create an empty stored in a variable.
List = []
Loop through the data( variable which you use to store the content of the file when the reader() was used)
Append each item in the data to your new list.

Your codes should be in the below format

When the above code is run. It will outpu the list of items in the csv file beginning from the first row and convert each row to a list type object. It will output the below

With this, you can work with your csv files like you do with your list objects.

Converting a csv file to Dictionary. A dictionary works with its keys and values. To convert a csv file into a dictionary, we will need a key and value.

In this article, we will use an item in the file as the key and the other as its value since it is of two columns.

Create an empty dictionary and assign it to a variable.
Loop through the data
Add the key and value to the empty dictionary. Your codes should be in the below format:

The key used in the above is the item at index 0 on each row of the Csv file and its value is that at index 1.
The following would be its output

Also with the above out you can also treat your Csv file like that of a dictionary. For example you could modify its value, its key, change the values and keys, and other dictionary features.

<|||||||||||-----------------|||||||||||||>

Now let's advance a little.
How will it be if we could make all letters in the csv file become a value to a key called Letters and do likewise to the words too? It sound nice right?.

Steps

Create a new empty list for the letters
Use a for loop to append all the letters to the new list excluding the "letter" string It should be like the below

**The if statement in the above is used to exclude the letter string from the list
Char = a loop variable
Char[0] = the item at index 0 of Char variable

Create another empty list with a variable words
Use a for loop to append all the word to the words list excluding the "words" string.
Your codes should be like below
Create an empty dictionary called dict.
Add a key called Letters to the empty dict and makes it value to be the Letters list object
Add another key called Words to the dict and makes it value to be words list object.
You code should be like the following

Following the above steps, you have completed the task. If you print the dict object you will get what task we want.

It's result will be:

The letters have now become the value of Letters and the words is now the value of Words.

It is very obvious that it took many lines of code to carry out a mini task using the python csv module.

Another module which python programmers use when the come across csv files is a module called Pandas

Pandas is a module used to work with csv files. It allows programmers to work with the csv more efficiently.
Using Pandas mores task can be accomplished, you can get the statistical information of a csv file like the mean, median, mode, standard deviation, list conversion, etc using the built functions.
Pandas is occupies a wide range when working with csv files.

In our next article, we will explain more about pandas

Kindly comment !!!!!