DEV Community: Adarsh Rawat

How Call by Value and Call by Reference Work in Python?

Adarsh Rawat — Mon, 15 Nov 2021 15:12:49 +0000

In this Article we're going to understand how call by value and call by reference work in python

Let's first understand what is call by value and call by reference

Call By Value :- In Call by value the value(copy) of the parameters passed as formal argument ina function instead of the variable itself, so in this case if any modification applied to the function parameters it will not be reflected on the actual variable.**

let's take example in C programming langauge to better understand call by value and call by reference
Example :-

#include <stdio.h>

void call_by_value(int a , int b){
    a = 44;
    b = 55;
}

int main() {
    int a  = 1;
    int b = 2;
    printf("value of a and b before function call :- %d and %d",a,b);
    call_by_value(a, b);
    printf("\nvalue of a and b after function call :- %d and %d",a,b);
    return 0;
}
Output :-
value of a and b before function call :- 1 and 2
value of a and b after function call :- 1 and 2

In above example the value of varible a and b not modified because a copy of a,b passed instead of their addresses

Call By Reference :- In Call By Reference, the address(actual varible) of the variable is passed into the function call as the actual parameter.

So, if any modification applied to the function parameters it will be reflected on the actual variable.**

Example :-

#include <stdio.h>

void call_by_reference(int *a , int *b){
    *a = 44;
    *b = 55;
}

int main() {
    int a  = 1;
    int b = 2;
    printf("value of a and b before function call :- %d and %d",a,b);
    call_by_reference(&a, &b);
    printf("\nvalue of a and b after function call :- %d and %d",a,b);
    return 0;
}
Output :-
value of a and b before function call :- 1 and 2
value of a and b after function call :- 44 and 55

In above example the address of varible a and b using & followed by variable name , where in function defination
* followed parameter known as pointer variable which used
to storing addresses, because of this values in varible a and b is modified.

Now we have seen what is call by value and call by reference with example

But, Python programming language uses the mechanism of the call-by-object and also call by object reference.

Each object whenever it is instantiated ,it is assigned unique id and its type if defined at runtime and id(object) returns a unique identity (integer value) of that object.

to move further into the topic first we have to understand mutable object and immutable object in python.

Mutable Objects
Datatypes that can be modified or if there state can be changed those objects are known as Mutable datatype
like (list, dictionaries, set) .
Immutable Objects
Datatypes that cannot be modified or if there state cannot be changed those objects are known as Immutable datatype like **(int, float, bool, string, unicode, tuple).

So when Mutable objects are passed to the function call then it is treated as **call by value.**

And When Immutable objects are passed to the function call and there value is modified then it is treated as **call by reference.**

For the sake of simplicity we're going to use the word call by value and call by reference instead of (call by object) and (call by object reference) respectivly.

let's take examples for call by reference and call by value

Example :- Call by reference (with Immutable objects (int, float, bool, string) and no object state modified) in python

num1  = 43
num2 = 12.44
num3 = False
name = 'hello'
print("Id in main function scope")
print(f"id of num1  = {id(num1)}")
print(f"id of num2  = {id(num2)}")
print(f"id of num2  = {id(num3)}")
print(f"id of name = {id(name)}")


def call_by_reference(num1,num2,num3,name):
  print("\nId within function")
  print(f"id of num1  = {id(num1)}")
  print(f"id of num2  = {id(num2)}")
  print(f"id of num2  = {id(num3)}")
  print(f"id of name = {id(name)}")


call_by_reference(num1,num2,num3,name)

Output :-
Id in main function scope
id of num1  = 94157957099328
id of num2  = 140438713493200
id of num2  = 94157956423968
id of name = 140438848940272

Id within function
id of num1  = 94157957099328
id of num2  = 140438713493200
id of num2  = 94157956423968
id of name = 140438848940272

So we can see that the id's the objects (num1, num2, num3) has not change becuase have not applied modification to those objects.

Example :- Call by reference (with Mutable objects (list ,dict) and object state modified) in python

names = ['ram', 'shyam', 'bharat']
d =  {'bob':21,'max':22}
print("Id in main function scope")
print(f"id of names list  = {id(names)}")
print(f"id of d dict  = {id(d)}")

def call_by_reference(name, d):
  print("Id within function")

  d['sam'] = 12
  names.append('john')

  print("\nModified Objects id :-")
  print(f"id of names list  = {id(names)}")
  print(f"id of d dict  = {id(d)}")

call_by_reference(names, d)
print("Modfied list and dict")
print(f"names =   {names}")
print(f"dict = {d}")

Output :-

Id in main function scope
id of names list  = 140439297460096
id of d dict  = 140438712179600
Id within function

Modified Objects id :-
id of names list  = 140439297460096
id of d dict  = 140438712179600
Modfied list and dict
names =   ['ram', 'shyam', 'bharat', 'john']
dict = {'bob': 21, 'max': 22, 'sam': 12}

We can see even after modifying the objects state the id's of the object remains unchanged because the objects datatype are mutable objects here by ,even if the object state is modified
it is treated as call by reference(call by object reference).

==============================================

Example :- Call by Value (with Immutable objects (int, float, bool, string) and object state modified) in python

num1  = 43
num2 = 12.44
num3 = False
name = 'hello'
print("Id in main function scope")
print(f"id of num1  = {id(num1)}")
print(f"id of num2  = {id(num2)}")
print(f"id of num2  = {id(num3)}")
print(f"id of name = {id(name)}")


def call_by_value(num1,num2,num3,name):
  print("\nId within function")
  num1 = 1
  num2 = 2
  num3 = 3
  name = 'sam'
  print("Modified Objects id")

  print(f"id of num1  = {id(num1)}")
  print(f"id of num2  = {id(num2)}")
  print(f"id of num2  = {id(num3)}")
  print(f"id of name = {id(name)}")


call_by_value(num1,num2,num3,name)
print("\Unmodified int, float, bool, string objects value in global scope")
print(f'num1 = {num1}')
print(f'num2 = {num2}')
print(f'num3 = {num3}')

Output :-
Id in main function scope
id of num1  = 94157957099328
id of num2  = 140438713494320
id of num2  = 94157956423968
id of name = 140438848940272

Id within function
Modified Objects id
id of num1  = 94157957097984
id of num2  = 94157957098016
id of num2  = 94157957098048
id of name = 140438713157936

Unmodified int, float, bool, string objects value in global scope
num1 = 43
num2 = 12.44
num3 = False

In the above example we can see that after modifying the Immuatable objects the object id's were diffrent , this is because we cannot modify the value of Immutables and because of this new 3 objects named (num1, num2, num3, name) where created in the function local scope after the function call we have also print the value to verify the objects in the global scope remain unchanged.

Example :- Call by Value (with Immutable objects (tuple) and object state modified) in python

names = ('ram', 'shyam', 'bharat')
print("Id in main function scope")
print(f"id of names tuple  = {id(names)}")

def call_by_value(name, d):
  print("Id within function")
  names = ('ram','shyam','bharat','john')
  print(f"modified object value \n{names}")
  print("\nModified Objects id :-")
  print(f"id of names tuple  = {id(names)}")

call_by_value(names, d)
print("\nUnchanged tuple object in global scope")
print(f"names = {names}")

Output :-
Id in main function scope
id of names tuple  = 140438712974144
Id within function
modified object value 
('ram', 'shyam', 'bharat', 'john')

Modified Objects id :-
id of names tuple  = 140438712255888

Unchanged tuple object in global scope
names = ('ram', 'shyam', 'bharat')

In the above example we can have assigned a new tuple with same elements + one more element because (tuple datatype does not support any add method) andchecked the object id which is diffrent because a new object gets created and also the global object remain unchanged because of it's Immutable nature.

With this example we have come to an end
hope you all have understood the topics
covered in this article.

:-)

File handling in Python

Adarsh Rawat — Sat, 13 Nov 2021 12:58:53 +0000

In this Article we going to learn following topics :

Read data from a file.
Write data to a file .
Append Data to a file.
Delete a file.

We're going to cover each of the above point with meaningful examples.

In python open() function is used to perform various operations on files, operation such as
(read,write,append,delete).

open()

Syntax :- open(filename, mode)

The open() function takes two argument filename and mode, and return a TextIOWrapper class object.

There are four different modes available :-

r mode is the default when no mode passed, used to read file, it throws FileNotFoundError if the file passed in first argument does not exist.
w mode is write data into a file, if data exist in the file it removes that data then, start writing the data into the file , if the filename passed does not exist,it creates a new file.
a mode write data to existing file, creates a file if it does not exist.
delete file with specified filename using os module ,if the file does not exist it throws FileExistsError error.

Other then that , you are specify the mode in which file will we handled.

t in Text mode.
b in Binary mode.
For Example :-

file1 = open("file.txt",'wb') # means write in binary mode
file2 = open("file.txt",'wt') # means write in text mode

Default mode is Text mode

After doing all the operation on file use can call file.close() method to close a file.

Throughout the examples we're going to us the below file

filename :- students.txt

Name : Ram Kumar
Age  : 13
City : Ajmir

Name : Alice Pen
Age  : 12
City : Boston

Name : Marcus Lee
Age  : 15
City : Tokyo

Read data from file

file = open("students.txt",'r')

for line in file:
    print(line,end="")

file.close()

Output :-
Name : Ram Kumar
Age  : 13
City : Ajmir

Name : Alice Pen
Age  : 12
City : Boston

Name : Marcus Lee
Age  : 15
City : Tokyo

In the above code , used for to the read one line at a time and print it, we're added end="" in print statement because , in file , a line ends with \n implicitly.

Write data to file

file = open("students_2.txt",'w')
student_info = { 
    "Name" : "Mike Javen",
    'Age' : 15,
    'City' : 'Minisoda'
}

for field,data in student_info.items():
    file.write(f"{field} : {data}  \n")

file.close()

students_2.txt file content :-

Name : Mike Javen  
Age : 15  
City : Minisoda

Here, we have used write() function , new file students_2.txt gets created, because it does not exist in the first place ,and we have added a student info to that file, we not added this student data to students.txt file because in w mode the data in the file first gets deleted and then writing starts.

Append data to file

file = open("students.txt",'a')
students_info = [ { 
    "Name" : "John Lenon",
    'Age' : 14,
    'City' : 'Navi Mumbai'    
 }, 
 { 
    "Name" : "Sam Dune",
    'Age' : 11,
    'City' : 'Boston'    
 }
]

for student_info in students_info:
    file.write("\n")
    for field,data in student_info.items():
        file.write(f"{field} : {data}  \n")

file.close()

Students.txt file.content :-

Name : Ram Kumar
Age  : 13
City : Ajmir

Name : Alice Pen
Age  : 12
City : Boston

Name : Marcus Lee
Age  : 15
City : Tokyo

Name : John Lenon  
Age : 14  
City : Navi Mumbai  

Name : Sam Dune  
Age : 11  
City : Boston

In the above code we added 2 new students details in students.txt file using 'a' append mode,

Delete a file

import os

filename = 'students_2.txt'
os.remove(filename)

print(f"successfully deleted file {filename}")

Output :-
successfully deleted file students_2.txt

The above example is fairly simple one in which we import python os module to delete the file in the current directory i.e., students_2.txt if the file does not exist in the diretory it throws error FileNotFoundError error.

With this we have seen all 4 opearation on a file, I'd like to mention another way in which we can perform same opeartions we talked about, where we does not have to explicitly close() the file working with the file.

Alternative way

with open("students.txt",'r') as file:
    for line in file:
        print(line, end="")

Output :-
Name : Ram Kumar
Age  : 13
City : Ajmir

Name : Alice Pen
Age  : 12
City : Boston

Name : Marcus Lee
Age  : 15
City : Tokyo

Name : John Lenon
Age : 14
City : Navi Mumbai

Name : Sam Dune
Age : 11
City : Boston

Here, we have used with keyword and as alias as soon as the with block ends, the file gets closed implicitly, we can use the same syntax with other modes

Writing file in Binary Mode

file = open("new_students.txt",'wb')
students_info = b'''
    "Name" : "Sam Dune",
    'Age' : 11,
    'City' : 'Boston'
     }
     '''
file.write(students_info)
file.close()

Output :-

    "Name" : "Sam Dune",
    'Age' : 11,
    'City' : 'Boston'

Here we have write bytes to a files

Before Finishing this Article I'd like to cover some useful examples of file processing.

Example :-

Reading the entire file content and storing it list for further processing

# file content :-
#my name is john levi and i
#live in boston and study in
#xyz univeristy doing majors
#in computer science.

person_info = [line for line in open('info.txt')]

for line in person_info:
    print(line,end="")

Output :-
my name is john levi and i
live in boston and study in
xyz univeristy doing majors
in computer science.

Here we fetched the content of entire file using list compreshension in just single line remember this file could be of 1000 of line for some operation we have to read file again and again, in this way we can just used the file content stored in the list.

Example :-

Reading the file content and storing it content in some collection like dictionaries datatype

# File Content :-
# Ram Kumar , 12 , Indore
# Shayam Kumar, 13 , Mumbai
# Ajay Kumar, 11 , Delhi
# Harris javen, 15 , New Jersey


students_info = []
for line in open("info.txt"):
    name, age, city = line.split(',')
    student = {
        "Name" : name.strip(),
        "Age" : age.strip(),
        "City" : city.strip()
    }
    students_info.append(student)

print("Content tetched from file and stored in dictionary\n")
for student in students_info:
    print(student)

Output :-

Content tetched from file and stored in dictionary

{'Name': 'Ram Kumar', 'Age': '12', 'City': 'Indore'}
{'Name': 'Shayam Kumar', 'Age': '13', 'City': 'Mumbai'}
{'Name': 'Ajay Kumar', 'Age': '11', 'City': 'Delhi'}
{'Name': 'Harris javen', 'Age': '15', 'City': 'New Jersey'}

with this example we have come to an end hope you all have understood the topics covered in this article.

:-)

Learn Generators in Python with (Examples)

Adarsh Rawat — Fri, 12 Nov 2021 17:47:28 +0000

In this Article we will going to look following topics

Understanding Generators in Python,with Examples
Also how to construct our own Generator function
Advantages of using Generator function instead of regular function in some cases

Generators :-

Generator are the special function which returns an traversal object which used to iterate over a collection of values, it is also used to create iterators.
It is can also be expressed in a way which is simiar to list compreshension
Just like iterator next() method we can also iterate over generator using generator.next() to iterate over the generator.

Creating a Geneartor :-

It is simple to create a Generator function if you have worked with regular function.
In Generator function instead of using return statement we use yield statement.
A function with yield keyword in it is called a Generator function
generator throws StopIteration Exception when next() method called on the last element in the genearator,
We have to explicitly handle StopIteration Exception when
using next() method with generator

let's use take an example to understand the above points :-

Example :-

# Normal function
def func():
    letters = ['a', 'b', 'c', 'd', 'e']
    for c in letters:
        return c

# Generator function
def generator_func():
    letters = ['a', 'b', 'c', 'd', 'e']
    for c in letters:
        yield c

print("Normal function :-")
print(func())

print("\nGenerator function :-")
print(generator_func())

print("\nIterating over generator using next() method")
gen = generator_func()
print(next(gen))
print(next(gen))
print(next(gen))
print(next(gen))
print(next(gen))
print(next(gen))

Output :-

Normal function :-
a

Generator function :-
<generator object generator_func at 0x00000180318F9A10>

Iterating over generator using next() method
a
b
c
d
e
Traceback (most recent call last):
  File "C:\Users\adars\Desktop\test.py", line 26, in <module>
    print(next(gen))
StopIteration

Now , you can se a normal function return the value which we have return but , Generator function return generator object (traversal object)

we can iterate over the generator object, let's see how?

Example :-

for i in generator_func():
    print(i)

Output :-
a
b
c
d
e

we iterated over the generator object in the above code,

Now you're probably thinking what is the difference between :-

function with return
function with yield

**function with return statement**

when the return statement called , the function terminates it execution and return the value, removing the function from the call stack.

**function with yield statement**

yield controls the flow of Genearator
yield pauses the function state and yielded value to the caller
yield resume the function execution on yield statement successfully
there can be multiple yield statement per Generator function

Generator with multiple yield statements

Example :-

def generator_func2():
    numbers = [1, 2, '3', '4', 5, 6, '7', 8, 9, '10']
    for num in numbers:
        if type(num) == int:
            yield f"{num} is an integer form"
        elif type(num) == str:
            yield f"{num} is an string form"



for i in generator_func2():
    print(i)


Output :-
1 is an integer form
2 is an integer form
3 is an string form
4 is an string form
5 is an integer form
6 is an integer form
7 is an string form
8 is an integer form
9 is an integer form
10 is an string form

In the above example we created a Generator function which uses
multiple yield statments

Creating Generator in one line

Just like list comprehension , we can create Generators in single line
Instead of [ ] brackets ,we use ( ) to create a generator function

Example :-

numbers_generator = (i for i in range(10))
print(type(numbers_generator))

for num in numbers_generator:
    print(num)

Output :-
0
1
2
3
4
5
6
7
8
9

In the above example we have created a Generator function which belong to generator class, in a way similar to list comprehension

let's take some more example of generators function

If your fimilar with range() function in python , we will going to implement our own range() function with the help of generators.

Example :-

def range(start = 0,stop = 0,stepby = 1):
    yield start
    while(start < stop):
        yield start+stepby
        start += stepby

print("range() function implementation :-")
print(type(range(0,10)))


for i in range(0,10):
    print(i)

print("\n---------\n")

for i in range(0,10,2):
    print(i)

Output :-
range() function implementation :-
<class 'generator'>
0
1
2
3
4
5
6
7
8
9
10

---------

0
2
4
6
8
10

We have created a simple implementation of range() function
using generators

*We will going to write two implementation of *fibonacci sequence using generators

Example

def fibo_seq_generator(n):

    # Initialize first two Fibonacci Numbers 
    a, b = 0, 1

    # One by one yield next Fibonacci Number
    while a < n:
        yield a
        a, b = b, a + b

print("Fibonacci sequence using generator :- ")
for i in fibo_seq_generator(20):
    print(i)

Output :- 
Fibonacci sequence using generator :-
0
1
1
2
3
5
8
13

Now that we have covered many examples of generators and
learned how to create generators, now we're going to see
the advantages of generator :-

Advantages of Genearators

It is Easy to Implement a generators relatively compared to
an iterator , which requires you to implement iter_() and
next() function.
Genearators are memory efficient when working with large
sequence of numbers because yield statement give value to
the caller function whenever yield line gets executed ,
So it doesn't have to store the entire sequence in the memory,
whereas in case of normal function ,it stores the sequence
somewhere in memory(in call stack) before returning the result.
With Generators we can generate infinite sequence of numbers without needing to worry about computer memory usage.
Generators used in Data Pipeline which provides the facility to process large datasets or stream of data without using extra computer memory.

with this we have successfully completed
Generators in Python with (Examples) Article
hope you all have understand the topics
covered in this Article.

:-)

Object Serialization and Deserialzation in Python

Adarsh Rawat — Fri, 12 Nov 2021 10:48:13 +0000

In this Article we're going to see :-

serializing object *(a process called serialization)
de-serializing object *(a process called de-serialization)

let's start with the definations :-

Serialization :-

Serialization is the process of converting data structures or object state into a format that can be stored or saved in memory for latter use, that format in which the object state saved will follow some rules which will help in de-serializing the object later.

De-serialization :-

De-serialization is just opposite of serialization in De-serialization ,we get the object state back to recreate the original object from the serialized format.

Now that we have understand defination of serialization and de-serialization.

let's see some place where they are very helpful :-

While sending data over the internet the data is transfered mostly in Json form (JSON is a format that encodes objects in a string and deserialization it (convert string -> object))
Hugh Machine Learning Model that are trained on hugh amount of data ,need to stored in some form for later use ,we cannot re-train them again and again , that is where serialization help
to store ML model state.
Big Data system uses serialization and deseralization to store large amount of data.

we have seen only few applications, serialization and deserialization are pretty much applied whererever there is need to store data in some form to use it for later use.

**Now we going to look how to do Serialzation and Deserializatiom in Python**

Pickle :-

Python pickle module is used for serializing and de-serializing a Python object structure. Any object in Python can be pickled so that it can be saved on disk.
Pickling is a way to convert a python object (list, dict, etc.) into a character stream
It saves the find the .pkl format

methods available :-

dump() − The dump() method serializes to an open file (file-like object).
dumps() − Serializes to a string
load() − Deserializes from an open-like object.
loads() − Deserializes from a string.

Serializing the object

import pickle
class Student:
  def __init__(self,firstname,lastname,age,standard):
    self.firstname = firstname
    self.lastname = lastname
    self.age = age
    self.standard = standard

  def showinfo(self):
    print(f"Firstname :- {self.firstname}")
    print(f"Lastname :- {self.lastname}")
    print(f"Age :- {self.age}")
    print(f"Standarad :- {self.standard}")


student1 = Student("Adarsh","Raven",21,"12th")
student2 = Student("Ankit",'Raven',24,'11th')


# Student Info
print("Student1 :- ")
student1.showinfo()


print("\nStudent2 :- ")
student2.showinfo()

# Serializing object
picked_student1 = pickle.dumps(student1)
picked_student2 = pickle.dumps(student2)

# Object stored in Byte steam
print("serialized student",picked_student1)


Output :-

Student1 :- 
Firstname :- Adarsh
Lastname :- Raven
Age :- 21
Standarad :- 12th

Student2 :- 
Firstname :- Ankit
Lastname :- Raven
Age :- 24
Standarad :- 11th
serialized student b'\x80\x03c__main__\nStudent\nq\x00)\x81q\x01}q\x02(X\t\x00\x00\x00firstnameq\x03X\x06\x00\x00\x00Adarshq\x04X\x08\x00\x00\x00lastnameq\x05X\x05\x00\x00\x00Ravenq\x06X\x03\x00\x00\x00ageq\x07K\x15X\x08\x00\x00\x00standardq\x08X\x04\x00\x00\x0012thq\tub.'

Deserializing the same object

orignal_student1 = pickle.loads(picked_student1)
orignal_student2 = pickle.loads(picked_student2)

print("\n\nAfter Getting object back from the saved state :- \n")
print("Student1 :- ")
orignal_student1.showinfo()

print("\nStudent2 :- ")
orignal_student2.showinfo()

Output :-

After Getting object back from the saved state :- 

Student1 :- 
Firstname :- Adarsh
Lastname :- Raven
Age :- 21
Standarad :- 12th

Student2 :- 
Firstname :- Ankit
Lastname :- Raven
Age :- 24
Standarad :- 11th

there are also other ways available to do the same task
we have covered in the above example, it will mention resources to those if you're intrested :-

joblib
json

:-)

Most Used Built-in functions in Python in

Adarsh Rawat — Thu, 11 Nov 2021 17:27:15 +0000

In this article we will going to look some very useful built-in functions that will always come handly in your python journey,
these functions will save you writing extra lines of code,

We will going to understand the defination and look some useful use-cases for each built-in function.

We will going to go through the following built-in functions :-

map()
filter()
reduce()
zip()
enumerate()
all()
any()

1. map()

map() is a built-in Python function that takes in two or more arguments: a function and one or more iterables, in the form:
Syntax :- *map(callable, iterable)
map() returns an iterator - that is, map() returns a special object that yields one result at a time as needed.
if you're not fimiliar with iterator :-
refer to this post :- iterator,iterable in python
for n size of input , it will return n size output

let's see some example of map()

Task :- convert list of string integer to to int

Example:-

lst_of_str_ints = ['1','2','3','4','5','6','7','8','9','10']
print("list of str ints :- ",lst_of_str_ints)

lst_of_ints = list(map(int, lst_of_str_ints))
print("list of ints :- ",lst_of_ints)

Output :-
list of str ints :-  ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
list of ints :-  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

we passed int object as first argument in the map function, second argument is the list_of_str_int, in this example
the iterator iterate each element at a time and map it to the corresponding callable argument int() passed as first argument,
then after iterating over all the elements, the map() function returns a map object will can then be casted to iterable like
(list,tuple,set).

we can perform the same task using lambda function:-

Example:-

lst_of_str_ints = ['1','2','3','4','5','6','7','8','9','10']
print("list of str ints :- ",lst_of_str_ints)

lst_of_ints = list(map(lambda x:int(x), lst_of_str_ints))
print("list of ints :- ",lst_of_ints)

Output :-
list of str ints :-  ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
list of ints :-  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Example :-

Task :- we will convert temprature from celsius to fahrenheit

temps_in_celsius = [0, 22.5, 40, 100]
print("temprature in celsius :-",temps_in_celsius)

temps_in_fahrenheit = list(map(lambda celsius : (9/5)*celsius + 32, temps_in_celsius))
print("temprature in fahrenheit :-",temps_in_fahrenheit)

Output :-
temprature in celsius :- [0, 22.5, 40, 100]
temprature in fahrenheit :- [32.0, 72.5, 104.0, 212.0]

In the above example we converted list of temperature in celsius to "temprature in fahrenheit"

Example :-

Task :- capitalize each name in a list to uppercase

names = ['Ram','Shyam','Abhiansh','Raven','Marcus']
print("Original list :-",names)

upper_cased_names = list(map(lambda x : x.upper(), names))
print("Upper case name :-",upper_cased_names)

Output :-
Original list :- ['Ram', 'Shyam', 'Abhiansh', 'Raven', 'Marcus']
Upper case name :- ['RAM', 'SHYAM', 'ABHIANSH', 'RAVEN', 'MARCUS']

2. filter()

filter(function, iterable)( offers a efficient way to filter out all the elements of an iterable, based on if a condition evaluates to treu
**Syntax :- *filter(callable, iterable)*
It takes first argument as function which evaluates to some boolean condition (True or False), second argument as iterable
The element will only be included if the function return **
for n size of input , it will return n-m size output
where m is the no. of element for which the function evaluates to False

let's see some example of filter()
Task :- fitler ints from a list of object of various type i.e. (float,string,lst,...)

Example :-

lst = [12, 43.42, 'hello', [1,3,4], 54,92, (1,2), 12.22, 9]
print("Orignal list of object without filteration :- ",lst)

lst_of_ints = list(filter(lambda x:type(x)== int, lst))
print("Updated list of ints  :- ",lst_of_ints)

Output :-
Orignal list of object without filteration :-  [12, 43.42, 'hello', [1, 3, 4], 54, 92, (1, 2), 12.22, 9]
Updated list of ints  :-  [12, 54, 92, 9]

In the above example we have filtered ints from a list of objects of different type, we used the same approch as in the previous example, but not here we're using a function which evaluates to two condition ,i.e., if the type of object is int then it returning True, and that element get included in the result list , Otherwise if the type in not int then it returning False , means item will not be included in the result list

let's see some other example :-

Task :- filter even integer from a list of integers

Example :-

list_of_ints = [1,2,3,4,5,6,7,8,9,10]

print("Orignal list of ints :- ",list_of_ints)

list_of_even_ints = list(filter(lambda x: x % 2 ==0, list_of_ints))
print("Updated list of only even ints  :- ",list_of_even_ints)

Output :- 
Orignal list of ints :-  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Updated list of only even ints  :-  [2, 4, 6, 8, 10]

In the above example we filtered list of even integer from a list of integers.

Task :- we have a list of random emails, so of which are valid and some are invalid , we have to filter out only valid emails

Example :-

import re
list_of_emails = ['nullchar@aol.com',
                  'salesgeek@yahoo.com',
                  'luebkehotmail.com',
                  'fmergeslive.com',
                  'jonathan@outlook.com',
                  'lydia@gmail.com',
                  'grdschl@comcast.net',
                  'catalogsbcglobal.net',
                  'glenz@outlook.com',
                  'cumarana@outlook.com',
                  'rsteinericloud.com',
                  'mosses@mac.com']

email_regex = r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'

print("Orignal list of emails :- ",list_of_emails,"\n")

valid_emails = list(filter(lambda email:re.match(email_regex, email), list_of_emails))
print("Updated list of valid emails  :- ",valid_emails)

Output :-
Orignal list of emails :-  ['nullchar@aol.com', 'salesgeek@yahoo.com', 'luebkehotmail.com', 'fmergeslive.com', 'jonathan@outlook.com', 'lydia@gmail.com', 'grdschl@comcast.net', 'catalogsbcglobal.net', 'glenz@outlook.com', 'cumarana@outlook.com', 'rsteinericloud.com', 'mosses@mac.com']

Updated list of valid emails  :-  ['nullchar@aol.com', 'salesgeek@yahoo.com', 'jonathan@outlook.com', 'lydia@gmail.com', 'grdschl@comcast.net', 'glenz@outlook.com', 'cumarana@outlook.com', 'mosses@mac.com']

In the above example we make use of re regex module to match if the email is valid or not using the regular expression in email_regex variable, and after iterating over each element at matching it to the email_regex we get a list of valid emails

3. reduce()

The function reduce(function, sequence) continually applies the function to the sequence. It then returns a single value. Syntax :- *reduce(callable, iterable)

i.e., If seq = [ s1, s2, s3, ... , sn ], calling reduce(function, sequence) works like this:

At first the first two elements of seq will be applied to function, i.e. func(s1,s2)
The list on which reduce() works looks now like this: [ function(s1, s2), s3, ... , sn ]
In the next step the function will be applied on the previous result and the third element of the list, i.e. function(function(s1, s2),s3)
The list looks like this now: [ function(function(s1, s2),s3), ... , sn ]
1. It continues like this until just one element is left and return this element as the result of reduce()

let's see some example of reduce()
Task :- reduce a list of int to a single integer using + operator

Example :-

from functools import reduce

lst =[47,11,42,13]
print("Original list :- ",lst)
print("reduced value :- ",reduce(lambda x,y: x+y,lst))

Output :-
Original list :-  [47, 11, 42, 13]
reduced value :-  113

let me explain what is going on in the above example

45    11     42    13
 \    /      /     /
  \  /      /     /
   58      /     /
    \     /     /
     \   /     /
      100     /
       \     /
        \   /
         \ /
         113

numbers at index 0 and 1 first added
then their sum added with value at index 3
then their sum added with value at index 4
then after completing the sequence 
we the reduced value  :- 113

let's see one more example of reduce
Task :- reduce a list of int to a single integer using * operator

from functools import reduce

lst =[47,11,42,13]
print("Original list :- ",lst)
print("reduced value :- ",reduce(lambda x,y: x*y,lst))

Output :-
Original list :-  [47, 11, 42, 13]
reduced value :-  282282

The visual that i have demostrated in the previous example ,in this example the same process going , but this time using x operator.

4. zip()

The function zip() makes an iterator that combines elements from each of the iterables
Syntax :- zip(iter1, iter2, iter3, iter4....)
zip() function return zip object which iterate over a list of tuples where tuples size is equal to the no. of list passed in the zip() function.
The iterator stops when the shortest input iterable is exhausted
i.e it will return list of tuples of size equal to the length of (minimum size *iterable among all the iterables passed in the zip() function)*

let's see some example of zip()
Task :- we will combine user firstname,lastname,hometown using
zip() function

Example :-

first_names = ['ram','shyam','ajay','bipin','manoj','alex']
print("list of first names :- ", first_names,"\n")

last_names = ['gupta','tiwari','yadav','rawat','desai','khan','raven']
print("list of first names :- ", last_names,"\n")

home_towns = ['ayodhya','vrindavan','bihar','jhnasi','boston','delhi','lanka']
print("list of hometowns :-" , home_towns,"\n")


persons_info = list(zip(first_names,last_names,home_towns))
print("data after using zip() function :-",persons_info)

Output :-
list of first names :-  ['ram', 'shyam', 'ajay', 'bipin', 'manoj', 'alex'] 

list of first names :-  ['gupta', 'tiwari', 'yadav', 'rawat', 'desai', 'khan', 'raven'] 

list of hometowns :- ['ayodhya', 'vrindavan', 'bihar', 'jhnasi', 'boston', 'delhi', 'lanka'] 

data after using zip() function :- [('ram', 'gupta', 'ayodhya'), ('shyam', 'tiwari', 'vrindavan'), ('ajay', 'yadav', 'bihar'), ('bipin', 'rawat', 'jhnasi'), ('manoj', 'desai', 'boston'), ('alex', 'khan', 'delhi')]

In the above example we have combined inforation from three iterable (firstnames,lastnames,hometowns) into list of tuples with information combined from all three iterables.

also notice that,
it has returned a list of tuples of size equal to the length of (minimum size *iterable among all the iterables passed in the zip() function)
i.e
* first_names list has length of 6
* last_names list has length of 6
* home_town list has length of 7

5. enumerate()

The function enumerate() return an iterates to a list of tuples , where first element of each tuple is it's corresponding index (by default is 0) and second element is the value at store at the index
Syntax :- enumerate(iterable, start)
the second argument start denotes from where the index start
i.e., if start = 4 then [(value1,4), (value2,5), (value3,6)...]

let's us take an example with start argument and without start arguement

Task :- without start argument

Example :-

names = ['ram','shyam','ajay','bipin','manoj','alex']

names_with_indexes = list(enumerate(names))
for name in names_with_indexes:
  print(name)

Output :-
(0, 'ram')
(1, 'shyam')
(2, 'ajay')
(3, 'bipin')
(4, 'manoj')
(5, 'alex')

Task :- with start argument

Example :-

names = ['ram','shyam','ajay','bipin','manoj','alex']

names_with_indexes = list(enumerate(names, 4))
for name in names_with_indexes:
  print(name)

Output :-
(4, 'ram')
(5, 'shyam')
(6, 'ajay')
(7, 'bipin')
(8, 'manoj')
(9, 'alex')

you can see the difference in the above two examples

6. all()

all() are built-in functions that allow us to conveniently check for boolean matching in an iterable.
Syntax :- all(iterable)
all() will return True if all elements in an iterable are True.

let's understand with examples

Example :-

lst = [True,True,False,True]
print(all(lst))

Output :-
False

Since element at index 2 is false , all() returned False

Example

lst = [True,True,True,True]
print(all(lst))

Output :-
True

Now the result is True

Example

lst = [True,[],[1,2,3],True]

print(all(lst))

Output :-
False

if any one coufused why the result is False, there is a concept of Truthy and Falsy in python.
here is a good example your can check Truthy and Falsy in python

7. any()

any() are built-in functions that allow us to conveniently check for boolean matching in an iterable.
Syntax :- any(iterable)
any() will return True if any elements in an iterable are True.

let's understand with examples

Example :-

lst = [True,False,False,False]
print(any(lst))

Output :-
True

Since element at index 0 is True, any() returned True

Example

lst = [False,False,False,False]
print(any(lst))

Output :-
False

Now the result is False

Example

lst = [(),[],[1,2,3],False]

print(any(lst))

Output :-
True

Now the result is true because at index 2, there is a list of size greater than 0 , which is a Truthy value.

with this example we have come to an end,
we have covered various useful built-in functions
which are very useful in certain scenarios

Hope you all have learned some useful stuff today. :-)

Getting Started with Collections Module in Python

Adarsh Rawat — Thu, 11 Nov 2021 10:52:03 +0000

In This Article we will go through various example covering the data structures containers offered by python collections module.

The collections module provide us with various different container datatypes, which are used to stored objects, and
access them, iterate over the objects, these all container datatypes , all have there usecases, which we will look later in the Article.

Before going forward, i'd like to mention one more point, all the container datatypes are built on top of the existing datatypes available in python like (Tuple,List,Set,Dict), these are introduced to improve the functionalities of the built-in container datatype in python.
As we progess further in the Article things will get clear to you.

We will going to the look at containers available in collections module :-

Counters
OrderedDict
DefaultDict
ChainMap
NamedTuple
DeQue

to use containers available in collections module, you will first have to import them,
there are mutiple ways to import :-

Example :-

#Import all container at once
from collections import *

#Import single container
from collections import Counter

# Importing and accessing using alias
import collections as clns
counter = clns.Counter()

1. Counter :-

A Counter is a sub-class of the dictionary.
It is used to keep the count of the elements in an iterable
It is an unordered dictionary where the key represents the element in the iterable and value represents the count of that element in the iterable

Example :-

from collections import Counter

names = ['George', 'Paul', 'George', 'Ringo', 'John', 'John', 'George', 'John', 'Ringo', 'Paul', 'Ringo', 'Ringo', 'George', 'Paul', 'John', 'John', 'John', 'John', 'John', 'Paul']

names_counter = Counter(names)
print(names_counter)

Output :- 
Counter({'John': 8, 'George': 4, 'Paul': 4, 'Ringo': 4})

We can see that it return Counter object , which we can cast to dict for further use.

key as the name
value as the no. of occourances of that key

Counter container has most_common method which returns List the n most common elements and their counts from the most common to the least. If n is None, then list all element counts.

calling most_common() on names_counter will return in :-

names_counter.most_common()

Output :-
[('John', 8), ('George', 4), ('Paul', 4), ('Ringo', 4)]

where the most frequent element with its count is at the from and least frequent element with its count at the end of the list
to know more about other methods available in Counter use can call help(counter_object) which will give you documentation for Counter

2. OrderedDict:-

A OrderedDict is a sub-class of the dictionary.
It remember the order in which the keys are inserted.
A regular dict does not preserve the order in which the keys are inserted but an OrderedDict maintains the order in which keys are inserted , , , , this is the major difference between a regular dict and an OrderedDict

Example :-

# Regular dict
reg_dict = dict()
reg_dict['apples']  = 3
reg_dict['pineapple'] = 5
reg_dict['mango'] = 1
reg_dict['orange'] = 7
reg_dict['grapes'] = 2
print(f"Regular Dict {reg_dict} \n\n")

from collections import OrderedDict

# Ordered Dict
ord_dict = OrderedDict()
ord_dict['apples']  = 3
ord_dict['pineapple'] = 5
ord_dict['mango'] = 1
ord_dict['orange'] = 7
ord_dict['grapes'] = 2
print(f"Ordered Dict {ord_dict}")

Regular Dict {'apples': 3, 'pineapple': 5, 'mango': 1, 'orange': 7, 'grapes': 2} 


Ordered Dict OrderedDict([('apples', 3), ('pineapple', 5), ('mango', 1), ('orange', 7), ('grapes', 2)])

Deleting elment from OrderedDict :-

del ord_dict['mango']
print(ord_dict)

Output :-
OrderedDict([('apples', 3), ('pineapple', 5), ('orange', 7), ('grapes', 2)])

Inserting element in OrderedDict :-

ord_dict['mango'] = 11
print(ord_dict)

Output :-
OrderedDict([('apples', 3), ('pineapple', 5), ('orange', 7), ('grapes', 2), ('mango', 11)])

As we can see first we deleted the key mango from OrderedDict then re-inserted it at the end of the OrderedDict.
for more method use can call help(OrderedDict_object)

3. DefaultDict :-

A DefaultDict is a sub-class of the dictionary.
The functionality of both dictionaries and defualtdict are almost same except for the fact that defualtdict never raises a KeyError.
It provides a default value for the key that does not exists.
We can pass a default values while initialising the DefaultDict as *DefaultDict(default_value=None)
It can take any type of default value

Example :-


from collections import defaultdict

d_dict = defaultdict(lambda : "Not Available") # passing default value

d_dict['John'] = 11
d_dict['Paul'] = 12
d_dict['George'] = 9
d_dict['Rango'] = 14

print(d_dict['Rango'])
print(d_dict['John'])
print(d_dict['max'])

Output :-
14
11
Not Available

4. ChainMap :-

A ChainMap is a container which stores many dictionaries into single ChainMap object.
We can perform operations on ChainMap object using following methods :-
keys() :- This function is used to display all the keys of all the dictionaries in ChainMap.

values() :- This function is used to display values of all the dictionaries in ChainMap.
maps() :- This function is used to display keys with corresponding values of all the dictionaries in ChainMap.

Example :-

from collections import ChainMap  


d1 = {'Ram': 12, 'Shyam': 12} 
d2 = {'Mango': 3, 'Orange': 4} 
d3 = {'Car': 4, 'Bus': 8} 

# Defining the chainmap  
c = ChainMap(d1, d2, d3)  

print(c)

Output :- 
ChainMap({'Ram': 12, 'Shyam': 12}, {'Mango': 3, 'Orange': 4}, {'Car': 4, 'Bus': 8})


# printing chainMap using maps
print ("All the ChainMap contents are : ")
print (c.maps)

# printing keys using keys()
print ("All keys of ChainMap are : ")
print (list(c.keys()))

# printing keys using keys()
print ("All values of ChainMap are : ")
print (list(c.values()))

Output :-

All the ChainMap contents are : 
[{'Ram': 12, 'Shyam': 12}, {'Mango': 3, 'Orange': 4}, {'Car': 4, 'Bus': 8}]
All keys of ChainMap are : 
['Car', 'Bus', 'Mango', 'Orange', 'Ram', 'Shyam']
All values of ChainMap are : 
[4, 8, 3, 4, 12, 12]

5. NamedTuple:-

It contain keys that are hashed to a particular value.
It supports both access from key-value and iteration.
following methods are availble in namedtuple to access the value :-

access by index: we can pass index within square brackets to access the value ,unlike dictionaries which are not accessible by index.
access by keyname: access value by using . operator followed by name of the attribute.
access using getattr(): This is another way to access the value by giving namedtuple and key value as its argument.

Example :-

from collections import namedtuple

Book = namedtuple('Book', ['title', 'author', 'price'])

b = Book('Xyz', 'xyz', 50)

print("accessing using index")
print(b[0])
print(b[1])
print(b[2])

print("\n------------------\n")

print("accessing using name")
print(b.title)
print(b.author)
print(b.price)

print("\n------------------\n")

print("using getattr() ")
print(getattr(b, 'title'))
print(getattr(b, 'author'))
print(getattr(b, 'price'))

Output :-
accessing using index
Xyz
xyz
50

------------------

accessing using name
Xyz
xyz
50

------------------

using getattr() 
Xyz
xyz
50

6. DeQue:-

Deque (Doubly Ended Queue) in Python is implementation available in collections module
Deque helps in fast insertion and deletion operation which list datatypes lacks
Deque offers insertion and deletion in O(1) time complexity whereas list offer insetion and deletion in O(n) time complexity.
It offers operations :-

append() :- This function is used to insert the value in its argument to the right end of deque.
appendleft() :- This function is used to insert the value in its argument to the left end of deque.
pop() :- This function is used to delete an argument from the right end of deque.
popleft() :- This function is used to delete an argument from the left end of deque. these are the frequenty used methods, you can find other method by calling help(deque_object).

Example :-

from collections import deque

# initializing deque
dq = deque(['John','Max','Harry','Marcus','Robin','Jackub'])

# using append() to insert element at right end
dq.append("Jimmy")

print ("The deque after appending at right is : ")
print (dq,"\n")

# using appendleft() to insert element at left end
dq.appendleft('Alex')
print ("The deque after appending at left is : ")
print (dq,"\n")

# using pop() to delete element from right end
dq.pop()
print ("The deque after deleting from right is : ")
print (dq,"\n")

# using popleft() to delete element from left end
dq.popleft()
print ("The deque after deleting from left is : ")
print (dq,"\n")

Output :-

The deque after appending at right is : 
deque(['John', 'Max', 'Harry', 'Marcus', 'Robin', 'Jackub', 'Jimmy']) 

The deque after appending at left is : 
deque(['Alex', 'John', 'Max', 'Harry', 'Marcus', 'Robin', 'Jackub', 'Jimmy']) 

The deque after deleting from right is : 
deque(['Alex', 'John', 'Max', 'Harry', 'Marcus', 'Robin', 'Jackub']) 

The deque after deleting from left is : 
deque(['John', 'Max', 'Harry', 'Marcus', 'Robin', 'Jackub'])

We have completed 6 containers available in the collections module , there are other container datatype available in collections module you can check by clicking the link below :-
other collections module containers

Hope you guys have learned all the topics covered in this Article :-)

Getting Started with Decorators in Python

Adarsh Rawat — Wed, 10 Nov 2021 16:55:18 +0000

To get started with Decorators , First we have to understand few things :-

everything we see in python is an object.
function in python is also an object.
we can declare function within a function.
these inner function have access to local variables in th e enclosing function.
a function can return any value, including a function

to understand the above point let us take an example

Example :-

>>> def hello(name):
...     return f"Hello {name}"
...
>>> a = hello
>>> # as we can see variable a referencing to a function object
>>> a("Ram")
'Hello Ram'
>>>

with the above example we can,

functions are objects
they can be assigned/reference by other name ,such as variable a

let's take another example to clear other remaining
points

>>> def foo(name):
...     def bar():
...             print(f"Hey I'm the inner function bar enclosed inside foo function , {name}")
...     return bar
...
>>> f = foo("Ram")
>>> f.__name__
'bar'
>>> type(f)
<class 'function'>
>>> f()
Hey I'm the inner function bar enclosed inside foo function , Ram
>>>

In the above example we have declared a function foo which contains another function bar, and when we called the outer function foo we get back reference to bar function object,
we also verified the same by,

checking f.name attribute which return name of the function
checked type(f) to see that it is of type function
Inner function bar has access to all varibles of enclosing function like variable name

The concepts that we have covered in the above example
will help in the understanding of Decorators

Now we see some examples of special type of decorator

Function Wrapper

Wrappers around the functions are also knows as decorators which are a very powerful and useful tool in Python since it allows programmers to modify the behavior of function or class.

Let's understand with an example

Example :-

lines = '-' * 60 + '\n'

def with_lines(func):
    def wrapper():
        return f'{lines}{func()}{lines}'
    return wrapper

def a():
    return f'I am in a!\n'
new_a = with_lines(a)


def b():
    return f'I am in b!\n'
new_b = with_lines(b)

print(a())
print(b())
print(new_a())
print(new_b())

Output :-
I am in a!

I am in b!

------------------------------------------------------------
I am in a!
------------------------------------------------------------

------------------------------------------------------------
I am in b!
------------------------------------------------------------

In the above example we created a with_lines function which takes func (callable object ) as and argument and with_lines function has an inner function wrapper
which has access to the outer function variable,
So we call the function passed by outer function inside the wrapper function by adding global varible lines at the start and end and return wrapper function object.

With this now we have a with_lines function wrapper which can we used with any function to change behiviour of the function,

As we used it with new_a and new_b functions that have some formating added to them using with_lines wrapper function.

Wrapper are one of the most powerful tools of python which can we used to change or added some extra behiviour to the existing function.

Now we have gathered enough information to understand Decorators.

What is a Decorator?

In simple terms :-

A function
that takes function as an argument
and return a function as its output
Because of the @ syntax in Python, decorating a function replaces that function with the result of calling the decorator on the function.

What can I use a Decorator For?

Anything that is repeated across multiple functions
Grab / modify / filter input arguments
Grab / modify / filter outputs
Change the state
Stop a function from being called under certain conditions
Choose a different function to be called instead

Now we will understand decorators with couple of examples

Example :-

lines = '-' * 60 + '\n'

def with_lines(func):
    def wrapper(*args, **kwargs):
        return f'{lines}{func(*args, **kwargs)}{lines}'
    return wrapper

@with_lines
def a():
    return f'I am in a!\n'
# a = with_lines(a)   # this line (after) is the same as @with_lines (before)


@with_lines
def b():
    return f'I am in b!\n'
# b = with_lines(b)  # this line (after) is the same as line 14

@with_lines
def add(x, y):
    return f'{x} + {y} = {x+y}\n'

print(a())
print(b())
print(add(3,5))  # add is actually wrapper! so when I call add(3,5), it's saying wrapper(3,5)

OUTPUT :- 

------------------------------------------------------------
I am in a!
------------------------------------------------------------

------------------------------------------------------------
I am in b!
------------------------------------------------------------

------------------------------------------------------------
3 + 5 = 8
------------------------------------------------------------

Using the same with_lines wrapper function used in the above example we added this decorator with the existing method a() ,b() and add() method using using @ Symbol which replaces that function with the result of calling the decorator on the function.

Let's take another example to see how we can filter the input using decorator :-
We have given a list of int from which we have to all numbers which are only even, let's see how we can do it using decorator

Example :-



# Wrapper that filter even from list of ints
def only_evens(func):
    def wrapper(*args):
        even_numbers = [num for num in args if num % 2 == 0]
        print("\n\nWrapper added")
        print(f"even numbers {even_numbers}")
        result = f"sum of even numbers :- {func(*even_numbers)}"
        return result

    return wrapper



def add_nums(*numbers):
    sum = 0
    for num in numbers:
        sum += num
    return sum


print("Original add_nums function output :-",add_nums(1,2,3,4,5,6,7,8,9,10))

@only_evens
def add_nums(*numbers):
    sum = 0
    for num in numbers:
        sum += num
    return sum


print("Original add_nums function output :-",add_nums(1,2,3,4,5,6,7,8,9,10))

Output :-
Original add_nums function output :- 55


Wrapper added
even numbers [2, 4, 6, 8, 10]
Original add_nums function output :- sum of even numbers :- 30

In the above example
first we called the orignal method add_nums() which add all the numbers.
then we declared the add_nums() function again but this time
added @only_evens wrapper to it , the wrapper only_evens filter the list of values first then the filters values passed to the original method add_nums() which return the sum of only even values.

with this example we have come to an end,
hope you all have learn some basis of decorators
this topic , at first is not easy to grasp,
but as you explore more about decorators
you will know how useful it is.
See you next time :-)

Types of iterators in Python, Which One to Use?

Adarsh Rawat — Wed, 10 Nov 2021 10:34:53 +0000

In this Article we will write

code for various iterators in python
comparing the time taken by the iterator to iterate
when to use which iterator

if you're new to this topic, i would prefer you go with this article linked below first:-*

---> What is iterator,iterable in Python

Types of Iterators in Python

Forward Iterator (for iterator)
Reverse Iterator
Index Based Iterator
Enumerate Iterator
Parallel Iterator
File Iterator

Forward Iterator

It is a read-only iterator, means you can only travese over it items in iterable, cannot modify values
It is the fastest way to iterate/traverse over the iterable (Collection on values)
As it's name suggest it can move only forward, going at the previous element is not possible.

Example :-

>>> names = ['ram','shyam','ajay','bipin','manoj','alex']
>>> for name in names:
...     name = name.upper()
...     print(name)
...
RAM
SHYAM
AJAY
BIPIN
MANOJ
ALEX
>>> names
['ram', 'shyam', 'ajay', 'bipin', 'manoj', 'alex']

As we can see in the above example for loop uses forward iterator,to iterable over the name in the names list,
we also tried to modify each value, by explicitly assigining
it to new uppercase value but, it does not modifiy the name in the names list, as we can see on the last line.

Reverse Iterator

It is a read-only iterator, means you can only travese over it items in iterable, cannot modify values
Iterate over element in the reverse order ,starting from last element to first element in the iterable.
it has same efficiency as forward itertor
reversed() function is used to return list_reverseiterator object

Example :-

>>> names
['ram', 'shyam', 'ajay', 'bipin', 'manoj', 'alex']
>>> reverse_iter = reversed(names)
>>> reverse_iter
<list_reverseiterator object at 0x000001D28CA21D20>
>>> for name in reverse_iter:
...     print(name)
...
alex
manoj
bipin
ajay
shyam
ram
>>>

The reversed object return list_reverseiterator object which then traveres from last element to first element.

Index Based Iterator

It uses a special function -
- range(start,stop,stepby),
- range(start,stop),
- range(stop)
Suitable when we have to perfrom read/write operation on the iterable object.
Suitable for INPLACE list operations
It is slower compared to for iterator

Example :-

>>> names = ['ram','shyam','ajay','bipin','manoj','alex']
>>> for i in range(len(names)):
...     names[i] = names[i].upper()
...     print(names[i])
...
RAM
SHYAM
AJAY
BIPIN
MANOJ
ALEX
>>> names
['RAM', 'SHYAM', 'AJAY', 'BIPIN', 'MANOJ', 'ALEX']

As you can see we used range() function to traverse/iterate over the list using index, and also able to modify the value INPLACE.

Enumerate Iterator

It uses a special function -
- enumerate(iterable), return enumerate object
Suitable for Mutable iterable objects
Can be used for INPLACE Operations
Slow as compared to forward Iterator

Example :-

>>> names = ['ram','shyam','ajay','bipin','manoj','alex']
>>> names_iter = enumerate(names)
>>> names_iter
<enumerate object at 0x000001D28CA413C0>
>>> for index,value in names_iter:
...     names[index] = value.upper()
...
>>> names
['RAM', 'SHYAM', 'AJAY', 'BIPIN', 'MANOJ', 'ALEX']

enumerate() function return enumerate object which iterate over a list of tuples where

first element of each tuple is it's corresponding index
second element is the value at store at the index
with the help of index we can modify the value in the names list

Parallel Iterator

It uses a special function -
- zip(iter1, iter2, iter3, iter4....)
Suitable when we have to iterate over mutile iterable object at the same time.
Return zip object

Example :-

>>> first_names = ['ram','shyam','ajay','bipin','manoj','alex']
>>> last_names = ['gupta','tiwari','yadav','rawat','desai','khan','raven']
>>> home_town = ['ayodhya','vrindavan','bihar','jhnasi','boston','delhi','lanka']
>>>
>>> detail_iter = zip(first_names, last_names,home_town)
>>> detail_iter
<zip object at 0x000001D28CA41B00>
>>> for detail in detail_iter:
...     print(detail)
...
('ram', 'gupta', 'ayodhya')
('shyam', 'tiwari', 'vrindavan')
('ajay', 'yadav', 'bihar')
('bipin', 'rawat', 'jhnasi')
('manoj', 'desai', 'boston')
('alex', 'khan', 'delhi')

zip() function return zip object which iterate over a list of tuples where tuples size is equal to the no. of list passed in the zip() function

also, it will return list of tuples of size equal to the length of (minimum size iterable among all the iterables passed in the zip() function) i.e
- first_names list has length of 6
- last_names list has length of 6
- home_town list has length of 7

So, it return list of tuples with list size 6

File Iterator

It uses a special function -
- open(filename)
It is used to iterate over the contents of the file

Example :-

File Content :- 
Details :-
('ram', 'gupta', 'ayodhya')
('shyam', 'tiwari', 'vrindavan')
('ajay', 'yadav', 'bihar')
('bipin', 'rawat', 'jhnasi')
('manoj', 'desai', 'boston')
('alex', 'khan', 'delhi')

Code :-

>>> for line in open("hello.txt"):
...     print(line)
...
Output :-

Details :-

('ram', 'gupta', 'ayodhya')

('shyam', 'tiwari', 'vrindavan')

('ajay', 'yadav', 'bihar')

('bipin', 'rawat', 'jhnasi')

('manoj', 'desai', 'boston')

('alex', 'khan', 'delhi')

It reads only line at a time

Now, that we have covered all the 6 iterators, we now see when to use which iterator

When to Use Which Iterator?

when we need to :-

traverse on a TUPLE (readonly) - forward iterator
traverse on a LIST (readonly) - forward iterator
traverse modify list inplace - index based iterator
traverse mutiple iterable (readonly) - parallel iterator

Comparing Performance of Iterators

Now, we will going to compare performance of

(forward iterator, reverse iterator, index based iterator, enumerate iterator)
by iterating a list of size 10000
comparing the time taken by various iterator

Example :-

Code :- 
import time
lst = [i for i in range(10000000)]
print("Traversing over list of size 10000000 \n-----------------------------------------------------\n")

start_time = time.time()
for i in lst:
     print("",end="")
end_time = time.time()
print(f"Time take by forward iterator {round(end_time - start_time ,2)} Seconds")

##################################################################

start_time = time.time()
for i in reversed(lst):
     print("",end="")
end_time = time.time()
print(f"Time take by reverse iterator {round(end_time - start_time ,2)} Seconds")


##################################################################

start_time = time.time()
index_based_iter = range(len(lst))
for i in index_based_iter:
     print("",end="")
end_time = time.time()
print(f"Time take by index based iterator {round(end_time - start_time ,2)} Seconds")


##################################################################

start_time = time.time()
index_based_iter = range(len(lst))
for i in index_based_iter:
     print("",end="")
end_time = time.time()
print(f"Time take by enumerate iterator {round(end_time - start_time ,2)} Seconds")


Output :-
Traverseing over list of size 10000000
-----------------------------------------------------

Time take by forward iterator 6.99 Seconds
Time take by reverse iterator 7.45 Seconds
Time take by index based iterator 8.32 Seconds
Time take by enumerate iterator 7.15 Seconds

Now, we have some to end, hope you all are now fimilar with various types of iterators in python and when to use which iterator.
See you next time , covering some other topic :-)

What is Iterable,iterator in Python?

Adarsh Rawat — Wed, 10 Nov 2021 07:37:59 +0000

First , we have to understand what is itertors, and why it is used?

Iterator :-

An iterator is an object that contains a countable number of values, in simple words an iterators stores numbers of values,
through which we can iterated upon (traversing over the values).

In Python an iterator is an object which implements the iterator protocol, which consist of the methods iter() and next().

Iteratble :-

objects like (strings, list, tuple, sets, dict) are all iterable (container of values).

Iterator :-

each of above iterable has an iter() method which can be used to get iterator of that particular object i.e, for list we get list_iterator object , for tuple we get tuple_iterator and so on...
let us understand with an example

>>> student_name = ['ram','shyam','john','harry','ravi']
>>> student_iter = iter(student_name)
>>> student_iter

<list_iterator object at 0x000001E7C37E1F30> # list_iterator object
>>>
>>> print(next(student_iter))
ram
>>> print(next(student_iter))
shyam
>>> print(next(student_iter))
john
>>> print(next(student_iter))
harry
>>> print(next(student_iter))
ravi

Using the next(iterable) we can move over elements in the iterator, Remeber on important in iterator object does not have any exception handler , so if we try to call next() ,when we are on the last element, it will throw an StopIteration Exception
the last element in the list is 'ravi' and we are currently on ravi, then let's see what happen what happen if we call next().

>>> print(next(student_iter))
ravi
>>> print(next(student_iter))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

There are mutiple ways to handle this StopIteration Exception
let's see each one of them.

1. We can use try and except block

>>> next(student_iter)
'ravi'
>>> try:
...     print(next(student_iter))
... except StopIteration:
...     print("StopIteration Exception Handled")
...
StopIteration Exception Handled
>>>

Here, we have handled the exception use try and except block,
it is still a solution to the block , but at the same time , every time we have to use extra lines of code whenever we use next() on the last element.

2. Using for loop

>>> for student in student_iter:
...     print(student)
...
ram
shyam
john
harry
ravi
>>>

for loop internally handles the StopIteration Excetion whenever it next() called on the last element.

By Passing Second Argument in next(iterable,sentinel) you can think sentinel as the default value return whenever *next * called on the last element.

>>> student_name = ['ram','shyam','john','harry','ravi']
>>> student_iter = iter(student_name)
>>> next(student_iter,None)
'ram'
>>> next(student_iter,None)
'shyam'
>>> next(student_iter,None)
'john'
>>> next(student_iter,None)
'harry'
>>> next(student_iter,None)
'ravi'
>>> next(student_iter,None)
>>>

You can see that ,we passed second argument as None , so when we called next on the last element, it does not throw the StopIteration Excetion instead it return the sentinel (default value). In this way we don't don't have to write extra lines of code , only have to pass the second argument.

That's it for now,
i think it would to enough for now,
I will cover some advance topics about iterator in python in the
next article ,*(types of iterators in python, and which is the most optimal one to use).

Getting started with Python List Comprehension

Adarsh Rawat — Tue, 09 Nov 2021 18:11:50 +0000

To know the importance of Comprehension in Python, first we have to understand what is Comprehension?

List Comprehension

List Comprehension is defined as an elegant way to define, create a list in Python and consists of brackets that contains an expression followed by for clause. It is efficient in both computationally and in terms of coding space and time.

Syntax :- The list comprehension starts with '[' and ']'.
[ expression for item in list if conditional ]

Let's learn it with the help examples

Example 1:
Suppose you have generate a list of square over a range of values
first ,you will try to do it with using for loop

Using For Loop

>>> lst = []
>>> for i in range(1,11):
...     lst.append(i**2)
...
>>> lst
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

do this way will definately do the job but, here we can make use of list comprehension to make this a one liner, let's see

Using List comprehension

>>> lst = [i**2 for i in range(1,11)]
>>> lst
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

in the above , the left most expression will be evaluated and store in the list for every value in the for loop

Conditionals in List Comprehension

We can also add conditional statements to the list comprehension.
to store values in the list based wheather the condition is True or False, We can create a list using range(), operators, etc.
let's understand this with an example.

Syntax: newlist = [expression for item in iterable if condition == True]

Example 2:
Suppose there are students in a class and you have to filter the student names based on the performance in the exam, you have a dict of students with key as there name and value as there marks
So, if a student get marks greater than or equal to 40 then he/she will be included in the list else he/she will not be included.

First we will try to do it with for loop, then
using list comprehension

Using For loop

>>> students = {'ram':44,'shyam':22,'harry':75,'john':54,'robert':25,'seta':65,'max':23,'harry':75,'robin':24,'philip':56}
>>> selected_students = []
>>> for student,marks in students.items():
...     if marks >= 40:
...             selected_students.append(student)
...
>>> selected_students
['ram', 'harry', 'john', 'seta', 'philip']
>>>

Using List comprehension

>>> students = {'ram':44,'shyam':22,'harry':75,'john':54,'robert':25,'seta':65,'max':23,'harry':75,'robin':24,'philip':56}

>>> selected_students = [student for student,marks in students.items() if marks >= 40]

>>> selected_students
['ram', 'harry', 'john', 'seta', 'philip']

With the help of list comprehension we have achived the same task with the same Ridability

Nested List Comprehensions

Nested List Comprehensions are nothing but a list comprehension within another list comprehension which is quite similar to nested for loops. let's understand it with an example.

Example 3:-

Suppose you have to generate a matrix where each row contain value from 1 to 4, we will try to do it using for loop then using nestd list comprehensions

Using For loop

>>> for i in range(3):
...
...     # Append an empty sublist inside the list
...     matrix.append([])
...
...     for j in range(5):
...         matrix[i].append(j)
...
>>> matrix
[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]

Using Nested List Comprehensions

>>> matrix = [[i for i in range(5)] for _ in range(3)]
>>> matrix
[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]

Dictionary Comprehension

Just like we use list comprehension ,in the similar way we can also create dictionary in one or two line instead of writing extra line . we use { , } instead of [ , ] braces,
all the other syntax is similar.
let's understand with the example

Example 4:-

you have given a lits of fruits and list list of calories corresponding to each of the fruits , now you have to generate a dict of fruits with
key as name of the fruit
value as it calories it has

Using For loop

>>>
>>> fruits = ["apple","pineapple","banana","orange","papaya","strawberry","grapes","watermalon"]
>>> calories = [100,141,65,43,120,80,110,200]
>>> fruit_dict = {}
>>> for i in range(len(fruits)):
...     fruit_dict[fruits[i]] = calories[i]
...
>>> fruit_dict
{'apple': 100, 'pineapple': 141, 'banana': 65, 'orange': 43, 'papaya': 120, 'strawberry': 80, 'grapes': 110, 'watermalon': 200}

Using Dict Comprehension

>>> fruits = ["apple","pineapple","banana","orange","papaya","strawberry","grapes","watermalon"]
>>> calories = [100,141,65,43,120,80,110,200]
>>> fruits_dict = { fruits[i]:calories[i] for i in range(len(fruits))}
>>> fruits_dict
{'apple': 100, 'pineapple': 141, 'banana': 65, 'orange': 43, 'papaya': 120, 'strawberry': 80, 'grapes': 110, 'watermalon': 200}

Use can also Use Condtional Statement in Dict Comprehension as we used in List Comprehension
let's take small example

Example 5 :-

We will added a student name and his marks based on wheather he/she has got marks greater than or equal to 40 , Refer to example 4.

Using For loop

>>> students = {'ram':44,'shyam':22,'harry':75,'john':54,'robert':25,'seta':65,'max':23,'harry':75,'robin':24,'philip':56}
>>> selected_students_dict = {}
>>> for student,marks in students.items():
...     if marks >= 40:
...             selected_students_dict[student] = marks
...
>>> selected_students_dict
{'ram': 44, 'harry': 75, 'john': 54, 'seta': 65, 'philip': 56}

Using Dict Comprehension

>>> students = {'ram':44,'shyam':22,'harry':75,'john':54,'robert':25,'seta':65,'max':23,'harry':75,'robin':24,'philip':56}
>>> selected_students_dict = {student:marks for student,marks in students.items() if marks >= 40}
>>> selected_students_dict
{'ram': 44, 'harry': 75, 'john': 54, 'seta': 65, 'philip': 56}

Now we have come to an end , the above examples will be enough for you to started with list comprehension , after all it one of the most powerful features of python that every pythonista must now,
i'm added some key takeways below. All the best for your future python explorations :-)

Key Takeawys

-> List comprehension is an elegant way to define and create lists based on existing lists.

-> List comprehension is generally more compact and faster than normal functions and loops for creating list.

-> we should avoid writing very long list comprehensions in one line to ensure that code is user-friendly.

-> every list comprehension can be rewritten in for loop, but every for loop can’t be rewritten in the form of list comprehension.