DEV Community: Sahil

Mastering Python Comprehensions: Crafting Efficient and Readable Code

Sahil — Tue, 04 Jun 2024 22:31:17 +0000

There are a lot of things that Python can do, and one of the things Python is good at is making code more readable and easy to follow. Python comprehension is one of those patterns that will make your code more efficient and readable, unless you go overboard.

So what is comprehension? In technical terms, comprehension is a concise way in Python to create sequences (like lists, sets, and dictionaries) using a single line of code.

So simply put, comprehension will help you create list, set, dict, and generators efficiently and in a more readable format.

Let's start then!

Types of Comprehension

1. List Comprehension

As the name suggests, it is a compact way to create a list. Let's say you want to create a list of all numbers from 1 to 10. How would you do it?

You might say you can just use range(1, 11) to get 1 to 10. But now if you want to get only odd numbers, how would you get it?

To get odd numbers:

odd_nums = [x for x in range(1, 11) if x % 2 != 0]
print(odd_nums)  # Output: [1, 3, 5, 7, 9]

Now with odd numbers, if you want the square of those numbers as well, then you could just do this:

squared_odd_nums = [x**2 for x in range(1, 11) if x % 2 != 0]
print(squared_odd_nums)  # Output: [1, 9, 25, 49, 81]

Now, if you do it the traditional way, this is how your code will look:

squared_odd_nums = []

for num in range(1, 11):
    if num % 2 != 0:
        squared_odd_nums.append(num**2)
print(squared_odd_nums)  # Output: [1, 9, 25, 49, 81]

2. Set Comprehension

No need for more explanation here; set comprehension is the same as list comprehension but instead of composing with [] (square brackets), we will use {} (curly brackets).

squared_odd_nums = {num**2 for num in range(1, 11) if num % 2 != 0}
print(squared_odd_nums)  # Output: {1, 9, 25, 49, 81}

3. Dictionary Comprehension

One more benefit of comprehension is you can easily create a dictionary with one line. Let's say you want a dictionary of all odd numbers and their squared values. How would you do it?

It's easy!

squares = {num: num**2 for num in range(1, 11) if num % 2 != 0}
print(squares)  # Output: {1: 1, 3: 9, 5: 25, 7: 49, 9: 81}

4. Generator Comprehension

Last but not least! Let's see how we can create a generator using comprehension.

Note: Generators look like tuples but they are not.

squared_odd_nums = (num**2 for num in range(1, 11) if num % 2 != 0)
print(type(squared_odd_nums))  # Output: <class 'generator'>
print(list(squared_odd_nums))  # Output: [1, 9, 25, 49, 81]

Now if you really want a tuple, you can do something like this:

squared_odd_nums = tuple(num**2 for num in range(1, 11) if num % 2 != 0)
print(type(squared_odd_nums))  # Output: <class 'tuple'>
print(squared_odd_nums)  # Output: (1, 9, 25, 49, 81)

You might ask how do we use nested loop and conditions with Python comprehension like we do with normal for loop. So here is your answer.

If/Else with Python Comprehension

I showed you earlier how to use if statement with comprehension but what if you want to use if and else both.

Let's see it with similar odd number example that we used earlier. Let' say we want odd number and squared of even number in between 1 to 10. How would we do it?

numbers = [num**2 if num % 2 == 0 else num for num in range(1, 11)]
print(numbers)  # Output: [1, 4, 3, 16, 5, 36, 7, 64, 9, 100]

Here, we have checked that if condition num % 2 == 0 (even number) becomes true then square it otherwise just pass the number (odd number).

Nested loop with Python Comprehension

So now you want to know how the hell we will do the nested loop. So. let's say we want to flatten the 3x3 matrix.

Here is how you would do it with list comprehension.

matrix_3d = [
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9]
]
flattened_list = [col for row in matrix_3d for col in row]
print(flattened_list)
# Output: [1, 2, 3, 4, 5, 6, 7, 8, 9]

Here, it will take each row and then iterate through each element of that row.

Now, if you want to use condition with nested loop comprehension here is how you would do it.

matrix_3d = [
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9]
]
flattened_list = [col for row in matrix_3d for col in row if col % 2 != 0]
print(flattened_list)
# Output: [1, 3, 5, 7, 9]

You can use if-else same way we use for single loop comprehension. So, that was a guide for comprehension in Python.

Conclusion

From the above, we can conclude that comprehensions are a powerful feature in Python that can help you write cleaner, more efficient code. If you think, "How can this be faster than a normal loop?" check out my blog titled How to read and write sequential data gracefully in Python for a better understanding of that.

Embrace comprehensions to write more Pythonic and maintainable code.

How to read and write Sequential data gracefully in python

Sahil — Sat, 15 Apr 2023 18:58:55 +0000

In this blog, I will teach you different ways to use list in Python. You might be thinking I know what a list is, I have used it numerous times. But there is a difference between using a list and using a list efficiently. So, I will show you what I have learned so far as a Software Engineer.

We can create a list basically two ways:

using list() constructor
using [] list comprehension (or using square brackets you might say)

There is no difference in output when we use these methods. Let's see!

Add/create a list using list comprehension `[]`

def list_comprehension() -> list:
    return [num for num in range(10)]

data = list_comprehension()
print(data)

"""OUTPUT:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
"""

Add/create a list using `for` loop

def list_for() -> list:
    data_list = []
    for num in range(10):
        data_list.append(num)
    return data_list

data = list_for()
print(data)

"""OUTPUT:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
"""

As I said, there is no difference. We will see the difference when we try to add a huge amount of data. Let's see!

Time took to add data for list comprehension

import time

def time_process(func):
    """Measure time"""
    def inner():
        start_time = time.time()
        func()
        end_time = time.time()
        elapsed_time = end_time - start_time
        print('Execution time:', time.strftime("%H:%M:%S", time.gmtime(elapsed_time)))
    return inner

@time_process
def list_comprehension() -> list:
    """Return list comprehension"""
    return [num*2 for num in range(1000000000)]

list_comprehension()

"""OUTPUT:
Execution time: 0:00:39
"""

Time took to add data for the `for` loop

@time_process
def list_for() -> list:
    data = []
    for num in range(1000000000):
        data.append(num*2)
    return data

list_for()

"""OUTPUT:
Execution time: 0:00:52
"""

You might say this is not that big of a difference. But it is! In real-time it makes a difference. And when you don't know how much data you will receive from DB or API, comprehension is the best option to create a list.

You might say, "what if we have condition?"

If you need to use only an if, then comprehension got your back!

Comprehension with for loop

@time_process
def list_comprehension() -> list:
    return [num for num in range(1000000000) if num%2 == 0]

list_comprehension()

"""OUTPUT:
Execution time: 0:00:35
"""

Append with for loop

@time_process
def list_for() -> list:
    data = []
    for num in range(1000000000):
        if num%2 == 0:
            data.append(num)
    return data

list_for()

"""OUTPUT:
Execution time: 0:00:42
"""

This is about writing, what about reading, you might say! To read data efficiently, instead of returning a list we can use generators.

Why? You might ask?

@time_process
def read_from_generator():
    """Return list comprehension"""
    generator_data = (num for num in range(1000))
    return generator_data

"""OUTPUT:
AVG time took to read: 0.03886222839 ms
"""

@time_process
def read_from_list():
    """Return list  generated by for without comprehension"""
    list_data = list(num for num in range(1000))
    return list_data
"""OUTPUT:
AVG time took to read: 0.04118919373 ms
"""

To read 1000000000 data from generators, it took 23 sec, whereas it took 53 secs for List. But for the time being, let's ignore the time they took to read the data.

Let me give you an inside secret if you don't need to re-read the data always use generators. Why? You might ask!

The reason to use generators is, Generators tend to be very memory efficient. Generators return data only when it is needed or asked for. Let's see with an example.

# read 3 values from generator
from_generator = read_from_generator()

for i in from_generator:
    print(i, end=' ')
print('\\\\nCompleted first read!')

for i in from_generator:
    print(i, end=' ')
print('\\\\nCompleted last read!')

"""OUTPUT:
0 1 2
Completed first read!

Completed last read!
"""

# read 3 values from generator
from_list = read_from_list()

for i in from_list:
    print(i, end=' ')
print('\\\\nCompleted first read!')

for i in from_list:
    print(i, end=' ')
print('\\\\nCompleted last read!')

"""OUTPUT:
0 1 2
Completed first reaad!
0 1 2
Completed last reaad!
"""

In the above code block, if I get data from read_from_generator method and iterate through that and I get data from read_from_list and iterate through that, I will not see any difference. But now if I try to iterate through both of them again, I will not see any iterations in the case of generators.

If you want the data again, you need to call read_from_generator method again. You might say why the hell do I want that. The reason is MEMORY EFFICIENCY!

Generators don't keep all the data in the memory all the time, unlike lists. Once you read the data it is gone. So, if you know you know you will iterate through data only once, always choose Generators.

I ran every method we discuss in this blog 25 times and the below table is the result of that.

Conclusion

Always use List comprehension and Generators whenever possible.

Upload CSV Data To MongoDB

Sahil — Mon, 09 Jan 2023 13:46:37 +0000

As developers, we face many cases when we import data from one source and want to upload this data into another source. This was the same situation I was facing. I exported data from Notion and wanted to import it into the MongoDB Atlas.

To insert data into MongoDB, we will use pymongo python package. To install pymongo use the following command:

 pip install pymongo==3.12.3

Note: Please note that there is some issue with a newer version of pymongo. So, I strongly suggest using version 3.12.3.

I will also use the pandas library. I find it easier to work with pandas than with python's default csv module. You can install pandas the same way we installed pymongo.

 pip install pandas

How to read CSV data in Python

To read CSV files, as I mentioned earlier, we will use the pandas library. To read a CSV file we need only 2 lines of code.

import pandas as pd

df = pd.read_csv("CsvFile.csv") # return a dataframe
print(df.head())  # df.head() return first 5 records

Now, after reading the CSV file we need to convert it into dict or json format. Why? We need to convert it into dict because MongoDB is JSON based database. And we can easily convert dict to json format and insert data into MongoDB.

How to convert pandas `dataframe` into `dict`

Pandas provide a method called df.to_dict() which converts dataframe to an array of dictionaries. We have multiple rows, so we need an array of dictionaries.

data_dict = df.to_dict("records")

Here, the "record" keyword is important. That's the argument which converts it into a proper array of dict. You can check out the official documentation here.

How to insert data to MongoDB

As I mentioned earlier, we will use pymongo it to connect with MongoDB. You can do that using the following code:

from pymongo import MongoClient

with MongoClient(URL) as client:
    db = client.prod  # prod is a database name
    tools = db.tools  # tools is a collection name
    ...

Here, we have used context manager (With statement), so we don't need to manually close the connection after operations are done. We will need the URL of our database, regardless of whether it is local or hosted somewhere.

Now, we got out a collection, to upload data into this collection we have 2 options. Either we insert row by row or all row data at once. First I will show you how you can insert all data at once.

How to insert multiple rows at once in MongoDB

To insert multiple data at once, we can use insert_many() method.

with MongoClient(URL,connect=False) as client:
    db = client.prod
    tools = db.tools
    result = tools.insert_many(data_list)

If you do not want any other operation during insertion and have already prepared an array of dict as we did earlier. This is the way to go.

How to insert data in MongoDB

To insert data row by row, we can use insert_one() the method. For example, we want to add it to every row.

with MongoClient(URL,connect=False) as client:
    db = client.Tools
    tools = db.tools
    for (index, data) in enumerate(data_list):
        data['id'] = index  # id added to existing data
        tools.insert_one(dub)

Conclusion

There are so many things we can do with Python and MongoDB. This was one of the examples. If you want to know or learn any other MongoDB operations apart from insertion let me know on Twitter or LinkedIn

A Date With the Datetime Module in Python | Part - 2

Sahil — Mon, 09 Jan 2023 13:44:13 +0000

We have explored how to get current date, month, year and day of the week in the previous blog. In this blog post we will explore more of a datetime and its corresponding methods with some tutorial.

Get the Current Week Number using Python

There are two ways to get current number of week using datetime.

isocalendar()
strftime()

# isocalendar()
from datetime import datetime

today_iso = datetime.today().isocalendar()
print("ISO Calendar:", today_iso)
print("Current Week:", today_iso.week)

"""
OUTPUT:

ISO Calendar: datetime.IsoCalendarDate(year=2022, week=43, weekday=2)
Current Week: 43

"""

As we can see, isocalender() method can provides year, week, weekday. We can extract all of them in similar way we got week.

# strftime()
from datetime import datetime

today_iso = datetime.today().strftime("%V")
print("Current Week:", today_iso)

"""
OUTPUT:

Current Week: 43
"""

There are so many possibilities with strftime() method. Let's see what we cna do with strftime().

strftime() in Python

The strftime() method helps us to represent date in appropriate string format. We can use the any of the following format codes as standalone or in combination.
Syntax: strftime(format)

Note: The strftime() method can be used with and datetime, date and time objects.

Directive	Meaning	Example
%a	Weekday in short-form.	Sun, Mon
%A	Weekday in full-form.	Sunday, Monday
%w	Weekday as number. (Starts from 0 as Sunday)	0, 1, …, 6
%d	Day of the month as a zero-padded decimal number.	01, 02, …, 31
%b	Month in short-form	Jan, Feb
%B	Month in full-form	January, February
%m	Month as a zero-padded decimal number.	01, 02, …, 12
%y	Year without century as a zero-padded decimal number. 22 for 2022.	00, 01, …, 99
%Y	Year with century as a decimal number.	0001, 0002, …, 2013, 2014, …, 9998, 9999
%H	Hour (24-hour clock) as a zero-padded decimal number.	00, 01, …, 23
%I	Hour (12-hour clock) as a zero-padded decimal number.	01, 02, …, 12
%p	Locale’s equivalent of either AM or PM.	AM, PM (en_US); am, pm (de_DE)
%M	Minute as a zero-padded decimal number.	00, 01, …, 59
%S	Second as a zero-padded decimal number.	00, 01, …, 59
%f	Microsecond as a decimal number, zero-padded to 6 digits.	000000, 000001, …, 999999
%z	UTC offset in the form ±HHMMSS[.ffffff].	(empty), +0000, -0400, +1030, +063415, -030712.345216
%Z	Time zone name (empty string if the object is naive).	(empty), UTC, GMT
%j	Day of the year as a zero-padded decimal number.	001, 002, …, 366
%U	Week number of the year (Sunday as the first day of the week) as a zero-padded decimal number. All days in a new year preceding the first Sunday are considered to be in week 0.	00, 01, …, 53
%W	Week number of the year (Monday as the first day of the week) as a zero-padded decimal number. All days in a new year preceding the first Monday are considered to be in week 0.	00, 01, …, 53
%c	Locale’s appropriate date and time representation.	Tue Aug 16 21:30:00 1988 (en_US); Di 16 Aug 21:30:00 1988 (de_DE)
%x	Locale’s appropriate date representation.	08/16/88 (None); 08/16/1988 (en_US); 16.08.1988 (de_DE)
%X	Locale’s appropriate time representation.	21:30:00 (en_US); 21:30:00 (de_DE)

How to Get Name of Month in Python

To get name of the month, we can use strftime method with datetime module.

# Month Name using Python
from datetime import datetime

today = datetime.today()
month_full = today.strftime("%B")
month_short = today.strftime("%b")

print("Today:", today)
print("Month in Full:", month_full)
print("Month in Short:", month_short)

"""
OUTPUT:

Today: 2022-10-30 16:56:49.644084
Month in Full: October
Month in Short: Oct
"""

How to Get Name of Weekday in Python

To get name of the week-day, we can use strftime method with datetime module.

# Week-Day Name using Python
from datetime import datetime

today = datetime.today()
day_full = today.strftime("%A")
day_short = today.strftime("%a")

print("Today:", today)
print("Day in Full:", day_full)
print("Day in Short:", day_short)

"""
OUTPUT:

Today: 2022-10-30 17:01:19.545097
Day in Full: Sunday
Day in Short: Sun

"""

Combine Multiple Format Codes in strftime() Method

So how to show date in a specific format. We can use any special character to separate format codes. To use % we have to write %%.

# Custom Date
from datetime import datetime

today = datetime.today()
custom_date = today.strftime("%d %B, %Y")

print("Today:", today)
print("Custom Date:", custom_date)

"""
OUTPUT:

Today: 2022-10-30 17:14:15.280584
Custom Date: 30 October, 2022
"""

Want to learn more?

Signup for my Newsletter and get the best articles into your inbox.

Till the next time 👋

A Date With the Datetime Module in Python | Part - 1

Sahil — Sat, 12 Nov 2022 10:39:57 +0000

We can extract current date using two classes.

Using date class

from datetime import date

today = date.today()

Using datetime class

from datetime import datetime

today = datetime.today()

from datetime import datetime

now = datetime.now()

The main difference between date.today() and datetime.today() is that date class just return the date object. Whereas, datetime class returns not only date but time as well.

Extract Current Day, Month and Year using Python

today = datetime.today()

print(f"Today: {today}")
print(f"Year: {today.year}")
print(f"Month: {today.month}")
print(f"Day: {today.day}")

OUTPUT:

Today: 2022-10-22 17:13:37.918735
Year: 2022
Month: 10
Day: 22

Extract Day of the Week using Python

today = datetime.today()

print(f"Today: {today}")
print(f"Weekday: {today.weekday()}")

"""
OUTPUT:

Today: 2022-10-22 17:32:48.396792
Weekday: 5
"""

This weekday() method will work with both the date and datetime class.

Note: Here, weekday() method will return index between 0 to 6.

Extract Current Time using Python

today = datetime.now()

print(f"Today: {today}")
print(f"Hour: {today.hour}")
print(f"Minute: {today.minute}")
print(f"Second: {today.second}")

OUTPUT:

Today: 2022-10-22 17:41:01.076569
Hour: 17
Minute: 41
Second: 1

Want to learn more?
Signup for my Newsletter and get the best articles into your inbox.

Till the next time 👋

Quick Guide to Object-Oriented Programming With Python

Sahil — Tue, 01 Nov 2022 05:07:29 +0000

What is OOP (Object-Oriented Programming)?

If you try to look on the internet or in any textbooks, there is no specific definition of OOP. In general, object-oriented programming is considered a combination of 4 pillar concepts. Those concepts are:

Abstraction (To hide something)
Encapsulation (To encapsulate or join many things)
Inheritance (To inherit something)
Polymorphism (To use for many purposes)

I will explain these concepts in detail as the blog progresses. But the main thing to know in OOP is Classes & Objects. So, what are these Classes and Objects?

Class & Object in Python

In simple terms, A class is just a skeleton or structure of an actual thing (object). For example, a sketch/outline of a building is a class, whereas, an actual building is an Object.

Syntax to create a class:

class ClassName:
        <statement-1>
    .
    .
    .
    <statement-N>

Now, if we apply this Class-Object concept to all humans, then we can have Person as a class with many properties and functionality.

class Person:
    age = 30
    height = 180
    weight = 65

    def say_hello(self):
        print('Hello Person')

Here, we have created a Person class with age, height and weight properties and the say_hello() method or functionality. We have created this Person class, now what?

To use this person class we need to create an object from this Person class. To create an object we use the following syntax:

object_name = ClassName()

To create an object of Person class:

person = Person()
print(person)

""" OUTPUT:

<__main__.Person object at 0x000001C47ED317B0>

"""

When we try to print person, we can see that it is an object of the Person class located at 0x000001C47ED317B0 in memory. To use the property and methods of the class, we can use the following syntax:

class Person:
    age = 30
    height = 180
    weight = 65

    def say_hello(self):
        print('Hello Person')

person = Person()
print("AGE:", person.age)
print("HEIGHT:", person.height)
print("WEIGHt:", person.weight)

person.say_hello()

""" OUTPUT:

AGE: 30
HEIGHT: 180
WEIGHt: 65
Hello Person

"""

This Person class looks very static and with fixed functionality then why I was calling it a schema/outline you might ask?

I kept the class Person simple on purpose to show you the basic use of Class and Objects. So if I were to show you the proper yet simple use of class and object then:

class Person:

    def __init__(self, name, age, height, weight):
        self.name  = name
        self.age = age
        self.height = height
        self.weight = weight

    def show_data(self):
        print('Details: ')
        print('AGE:', self.age)
        print('HEIGHT:', self.height, "cm")
        print('WEIGHt:', self.weight, "kg")

person1 = Person('John Doe', 33, 180, 75)
person1.show_data()

print("-"*15)

person2 = Person('Jane Doe', 31, 160, 65)
person2.show_data()

""" OUTPUT:
Details:
NAME: John Doe
AGE: 33
HEIGHT: 180 cm
WEIGHt: 75 kg
---------------
Details:
NAME: Jane Doe
AGE: 31
HEIGHT: 160 cm
WEIGHt: 65 kg
"""

As you can see in the above code example, I was able to pass different values for name, age, height and weight and created 2 objects from the Person class. I can pass the same or different values as per my need and create an N number of objects because they will be stored at different locations anyway.

person1 = Person('John Doe', 33, 180, 75)
print(person1)
# OUTPUT: <__main__.Person object at 0x000001E6B1C93FA0>

person2 = Person('Jane Doe', 31, 160, 65)
print(person2)
# OUTPUT: <__main__.Person object at 0x000001E6B1C93EB0>

person3 = Person('Jane Doe', 31, 160, 65)
print(person3)
# OUTPUT: <__main__.Person object at 0x000001E6B1C93E50>

In the real-life scenario, when we have to create multiple users (signup functionality), we do not write code for every user. We use class and create objects based on the received details as we did with person1 and person2.

Now, you might have noticed the self keyword and __init__() method in the above code. So, what are these?

What is the init() method in Python?

If you know other languages such as Java, C# or JavaScript, you might be familiar with the concept of a constructor. The init() method is a constructor for a class in Python. For non-technical people, consider the constructor a way to tell the compiler/interpreter to initiate an operation or allocate memory to the object that we are planning to create. But it is not necessary to use the constructor in Python. If we do not provide the __init__() method, the interpreter will create on its own.

In general, we use the init() method to initialize some properties and for inheritance purposes. As we saw earlier, this is how we initialize the variables:

def __init__(self, name, age, height, weight):
        self.name  = name
        self.age = age
        self.height = height
        self.weight = weight

Note: We will discuss how we use the __init__() method for inheritance later.

Now, you might ask, but Sahil, what the heck self is?

What is the ‘self’ keyword in Python?

The self keyword represents the instance of a class. (Just for simple understanding, consider instance as an object generated by the interpreter. Objects and instances are not the same but here consider them the same, just for simple understanding.)

Using this self keyword, we can access all attributes/properties and methods across the class. For instance, in the example below, we have used the self keyword to access name, age, height and weight attributes.

class Person:

    def __init__(self, name, age, height, weight):
        self.name  = name
        self.age = age
        self.height = height
        self.weight = weight

    def show_data(self):
        print('Details: ')
        print('NAME:', self.name)
        print('AGE:', self.age)
        print('HEIGHT:', self.height, "cm")
        print('WEIGHt:', self.weight, "kg")

We can also access methods using the self keyword.

class Person:

    def __init__(self, name, age, height, weight):
        self.name  = name
        self.age = age
        self.height = height
        self.weight = weight

    def show_data(self):
        print('Details: ')
        print('NAME:', self.name)
        print('AGE:', self.age)
        print('HEIGHT:', self.height, "cm")
        print('WEIGHt:', self.weight, "kg")

    def self_demo(self):
        self.show_data()

person1 = Person('John Doe', 33, 180, 75)
person1.self_demo()

""" OUTPUT:
Details:
NAME: John Doe
AGE: 33
HEIGHT: 180 cm
WEIGHt: 75 kg
"""

It is not necessary to use the self keyword. You can use any name, but the condition is, that it should be the first argument of a method. (For __init()__ as well.)

class Person:

    def __init__(random_replacement, name):
        random_replacement.name = name
        pass

    def show_data(random_replacement_1):
        print('Just a Random Thing!')
        print('Name:', random_replacement_1.name)

person1 = Person('John Doe')
person1.show_data()

""" OUTPUT:
Just a Random Thing!
Name: John Doe
"""

As you might have guessed, we don't need to use the same name across the class. The one and only rule is,it should be the first argument! But generally, we prefer the self keyword as it has become almost a standard. If we want to pass any arguments, we must pass them after the self keyword. As we did in the __init__() method.

You can also add any attribute at any time using the self keyword.

class Person:

    def __init__(self, name):
        self.name = name

    def add_age(self, age):
        self.age = age

    def show_data(self):
        print('Name:', self.name)
        print('Age:', self.age)

person1 = Person('John Doe')
person1.add_age(31)
person1.show_data()

""" OUTPUT:
Name: John Doe
Age: 31
"""

Here, we have added age to the class attributes. I do not suggest you do that but just for demo purposes that it is possible. Consider the self keyword as the dictionary and all the attributes and methods as the key of that dictionary.

In the above code example, if we call the show_data() method first. It will throw an error, as we do not have any attribute named age at that time.

This process of combining attributes and methods under one umbrella is called encapsulation.

So, how do we extend the usage and functionality of our class? We can do that using Inheritance and Polymorphism.

Inheritance in Python

As the name suggests, inheritance is the capability to inherit something. In this case, we will be inheriting attributes and methods. For example, we created a class named Person which is too generic. By inheriting the generic class, we can create another class that will be more specific to our needs.

We want to create an Employee class that inherits everything or is inherited from the Person class. So, how do we do it?

Inheritance syntax in Python:

# Base Class/Parent Class
class BaseClass:
        <statement-1>
    .
    <statement-N>

# Derived Class/Child Class
class DerivedClass(BaseClass):
        <statement-1>
    .
    <statement-N>

To create the Employee class from the Person class we can write the following code:

class Person:
    def __init__(self, name, age, height, weight):
        self.name  = name
        self.age = age
        self.height = height
        self.weight = weight

    def show_basic_data(self):
        print('Details: ')
        print('NAME:', self.name)
        print('AGE:', self.age)
        print('HEIGHT:', self.height, "cm")
        print('WEIGHt:', self.weight, "kg")

class Employee(Person):
    def __init__(self, employeeId, department, salary):
        super().__init__(name, age, height, weight)
        self.employeeId = employeeId
        self.department = department
        self.salary = salary

    def show_data(self):
        self.show_basic_data()
        print('Employee Id:', self.employeeId)
        print('Department:', self.department)
        print('salary: $', self.salary)

employee = Employee('John Doe', 33, 180, 75, 1 ,'IT', 100000)
employee.show_data()

""" OUTPUT:
Details:
NAME: John Doe
AGE: 33
HEIGHT: 180 cm
WEIGHt: 75 kg
Employee Id: 1
Department: IT
salary: $ 100000
"""

You might have noticed super() in Employee class. What is super() you might ask? The super keyword refers to the Parent class. We use the super() method to access attributes and methods of the parent class. And that is what we did in the __init__() method of the Employee class. We passed some data to the parent class.

Once we have called super() in the __init__() method of the child class, we can use all the attributes and methods of the parent class in our child class. That is what we did in the show_data() method of the Employee class. We could use super().show_basic_data() as well.

Conclusion

Finally! We are at the end of this section 😁.

Let me know if you need any help or want to discuss something. Reach out to me on Twitter or LinkedIn. Make sure to leave any thoughts, questions, or concerns in the comments below. I would love to see them.

Want to learn more?
Signup for my Newsletter and get the best articles into your inbox.

Till the next time 👋

Delete Files & Directories Using Python

Sahil — Mon, 31 Oct 2022 03:54:19 +0000

So far, we have learned many basic concepts of Python. We learned about:

Data types & Variables
String Manipulation Methods
List & Dictionary Manipulation Methods
Conditional Statements
Loops
File Handling

But now what?
We can not be good at something just by reading blogs or watching YouTube tutorials. We have to practice and keep practicing until we become good at it. So, in this blog, I will try to show most of the concepts we have learned so far.

“For the things we have to learn before we can do them, we learn by doing them.”

― Aristotle

So, what we are gonna do?

Planning

In this blog, I will show you

How can we list all the files and directories of a specific location using the os module
Separate files and directories, and add them to their respective lists using the isfile() method
Go through these lists of files and directories using for loop
Delete files using the os module
Delete directories using the shutil module

Apart from these, we will see a bit about error handling in Python as well. Even if we test our code, in a real-world environment, unexpected errors can or will arise eventually. To tackle these scenarios we use error handling. Error handling is a general programming concept it is not a Python-only concept.

Coding

Before we get into the code, I would like to mention that if you don't understand any line or have any doubt, you can always reach out to me on Twitter or LinkedIn. I will explain the code in little chunks and I will also add the whole code file at the end of this blog. We will proceed with:

1. Import necessary modules

import os
import shutil

2. Get the list of all files and directories of a specific location

# location of temp files in windows
location = 'C:\\Windows\\Temp'
file_dir = os.listdir(location)

Note: Mac people, you can use any temporary folder or directory you wish program won't stop working. I am a Windows user so don't know where temp files are stored on a Mac devices.

3. Separation of Files and Directories

The listdir() method returns files and directories both. So, we need to separate files and directories from file_dir.

only_files = []
only_dir = []
for i in file_dir:
       # generate full path
       full_path = os.path.join(location, I)

       # checks if the given path is of file or not
       if os.path.isfile(full_path):
           only_files.append(full_path)

       # checks if the given path is of directory or not
       if os.path.isdir(full_path):
           only_dir.append(os.path.join(full_path)

4. Delete files

# loop through the list that contains only files
for i in only_files:
    try:
        # os.chmod(i, 0o777)
        os.remove(i) # removes the files
    except Exception as e:
        print(e)

Here, we have kept the whole delete logic (one line 😂) into the try and except block. Why do we do this? It is possible that the file has been moved or we do not have access to the file. This situation can raise the error/exception and our code will stop working. There can be many ways to handle exceptions. But, I have selected the simple and easy path by just printing the error message. Don't worry we will learn about exception handling in the next blog.

5. Delete directories

# loop through the list that contains only directories
for i in only_dir:
    try:
        # os.chmod(i, 0o777)
        shutil.rmtree(i, ignore_errors = False) # removes the the directory and all the sub-files and sub-directory that it might have
    except Exception as e:
        print(e)

The shutil.rmtree() method will take input and will try to delete the directory and all its sub-files and directories. Here, we want to raise exceptions when the method fails to remove some files/directories. So, we passed the ignore_errors parameter with false as a value.

Note: In Windows, sometimes you will face permission issues. To avoid those, open your editor or console wherever you are trying to run this code as an administrator. For mac, uncomment os.chmod(i, 0o777) part. It should work but not sure because I have not tried it in the mac system.

This demo is just to understand the concepts better that we have learned so far. We can optimize code by deleting files/directories at the same time we are splitting them into two lists. But again this is to make you understand how to use python.

6. Convert whole logic into a function

Re-usability is very important in the programming world. It is not necessary for this program but just to show you we will convert the whole logic into a function. And we will call this function only.

def delete_temp_file(locations):
    for location in locations:
        try:
            # get list of all files and directories
            file_dir = os.listdir(location)

            only_files = []
            only_dir = []

            for i in file_dir:
                # checks if the given path is of file or not
                # print(location+'\\'+i)
                # print(os.path.join(location, i))
                full_path = os.path.join(location, i)
                if os.path.isfile(full_path):
                    only_files.append(full_path)

                # checks if the given path is of directory or not
                if os.path.isdir(full_path):
                    only_dir.append(full_path)
        except Exception as e:
            print(e)

        # loop through the list that contains only files
        for i in only_files:
            try:
                # os.chmod(i, 0o777)
                os.remove(i) # removes the files
            except Exception as e:
                print(getattr(e, 'message', repr(e)))

        # loop through the list that contains only directories
        for i in only_dir:
            try:
                # os.chmod(i, 0o777)
                # removes the the directory and all the sub-files and sub-directory it has
                shutil.rmtree(i, ignore_errors = False)
            except Exception as e:
                print(e)

Now, execution time! 😎

Final Touch & Execution

This is repetitive, but I have added the whole code so that you can see it in one flow. Before you run it edit the location list as per your desire. Then go to the location where you saved the codebase and run python file_name.py . You can find whole codebase on my Gist

import shutil
import os

def delete_temp_file(locations):
    for location in locations:
        try:
            # get list of all files and directories
            file_dir = os.listdir(location)

            only_files = []
            only_dir = []

            for i in file_dir:
                # generate full path
                full_path = os.path.join(location, i)

                # checks if the given path is of file or not
                if os.path.isfile(full_path):
                    only_files.append(full_path)

                # checks if the given path is of directory or not
                if os.path.isdir(full_path):
                    only_dir.append(full_path)
        except Exception as e:
            print(e)

        # loop through the list that contains only files
        for i in only_files:
            try:
                # os.chmod(i, 0o777)
                os.remove(i) # removes the files
            except Exception as e:
                print(getattr(e, 'message', repr(e)))

        # loop through the list that contains only directories
        for i in only_dir:
            try:
                # os.chmod(i, 0o777)
                shutil.rmtree(i, ignore_errors = False) # removes the the directory and all the sub-files and sub-directory it has
            except Exception as e:
                print(e)


locations = ['C:\\Users\\Sahil\\AppData\\Local\\Temp', 'C:\\Windows\\Temp','C:\\Windows\\Prefetch']

# call function
delete_temp_file(locations)

That was it. Thank you for reading. 😊

I know it is a lot, but, I hope, you were able to absorb some knowledge. Let me know if you need any help or want to discuss something. Reach out to me on Twitter or LinkedIn. Make sure to share any thoughts, questions, or concerns. I would love to see them.

You can subscribe to my Newsletter for more amazing resources and articles.

Till the next time 👋

Organize Any Files and Folders Using Python

Sahil — Mon, 24 Oct 2022 11:20:21 +0000

Organize Download Folder

3 main things will help us decide how we want to organize our files.

How do we want to manage files?
What kind of files do we want to organize?
Which directory do we want to organize?

How do we want to organize files?

We can organize files based on various factors such as name, creation date, modified date and size. We can also manage files based on the type/extension of a file. That is where the next question comes into the picture.

What kind of files do we want to organize?

There are many types of files we might find in our download folder. There can be music, video, images, zip, executable, document and many more. After answering the first two questions, we must decide which directory we want to organize.

This question is important if you are organizing the subdirectories as well. For example, there can be a coding project that can contain many kinds of files. To organize this directory based on file type is not a good idea because it can mess up our whole project.

So for this tutorial, we will mostly focus on the root directory.

Plan For Creating Auto File Organizer

How to get the list of all files in Python

To organize any directory, first, we need a list of files. To get the list of all files in the given directory we can use the listdir() method from the os module.

Syntax: os.listdir(path)

import os

total_list = os.listdir("Downloads")
print(total_list)

""" OUTPUT:

['elements.svg', 'Export.zip', 'favicon_io', 'Frame 5.png',
'Screenshot.png', 'Temp', 'Video-1.webm']

"""

Here, we get a list that contains files and directories both.

If we want to use this, then we have to separate files and directories by ourselves using the os.path. To identify if the given path is a file or directory we can use the following code snippet:

from os import listdir, path

total_list = listdir(".")

file_list = []
dir_list = []

for file in total_list:
    if path.isfile(file):
        file_list.append(file)
    else:
        dir_list.append(file)

print(file_list)
print("-"*15)
print(dir_list)

""" OUTPUT:
['elements.svg', 'Export.zip', 'Frame 5.png', 'Screenshot.png', 'Video-1.webm']
---------------
['favicon_io', 'Temp']
"""

When I run this code, I was in the Download directory. So, if I want a list from the current directory, I can use os.listdir("./") or os.listdir(".") or os.listdir().

You might say this is a lot of work just to get a list of filenames. So, let me present to you an easy way. I did not know about this method as well. I found it when I was researching for this tutorial. We can use the os.walk() method to retrieve all the files. You can check out the blog post here from where I got to learn about this.

Syntax: os.walk(path)

from os import walk

files = []
for (dirpath, dirnames, filenames) in walk("./"):
    files.extend(filenames)
    break

print(filenames)

""" OUTPUT:
['elements.svg', 'Export.zip', 'Frame 5.png',
'Screenshot.png', 'temp.py', 'Video-1.webm']
"""

First of all, let me explain why I use the break keyword here. The walk() method not only lists out directories and files of the given location. But, it tries to scan all the subdirectories as well. Let me show you.

from os import walk

files = []
for (index, data) in enumerate(walk("./")):
    if index == 2:
        break
    print(index)
    print(data)
    print("-"*15)

""" OUTPUT:

0
('./', ['favicon_io', 'Temp'], ['elements.svg', 'Export.zip', 'Frame 5.png', 'Screenshot.png', 'temp.py', 'Video-1.webm'])
---------------
1
('./favicon_io', ['2021', '2022'], ['android-chrome-192x192.png', 'android-chrome-512x512.png', 'apple-touch-icon.png', 'favicon-16x16.png', 'favicon-32x32.png', 'favicon.ico', 'site.webmanifest'])
---------------

"""

As you can see, the first iteration yields the current path and two lists*.* The first list contains all the directories and the second list contains all the file names. Now, during the second iteration, the walk() method starts scanning the subdirectories which are “favicon_io” and “Temp” in our case. We are currently focusing only on the “Downloads” directory. So, I use the break keyword so that after one iteration it stops scanning subdirectories.

Once we got the list of all files, we need to separate them based on file type. To do that we will use the extension of each file.

How to Get File Extension Using Python

To get the file extension, we can use the path module from the os. The path module has a splitext() method that will return a tuple containing the file name and its extension.

Syntax: os.path.splitext(path)

from os import path

file_tuple = path.splitext('Screenshot.jpg')
print(file_tuple)
print(file_tuple[1])

""" OUTPUT:
('Screenshot', '.jpg')
.jpg
"""

Now, we have all files and we know how to get the extension. So, the next part is to move files according to their types.

How to Move Files Using Python

We can move files with the os module but this time we will use shutil which I think is way better than os for this task. We will use the move() method from the shutil module.

Syntax: shutil.move(src, dst, copy_function=copy2)

Note: Third argument is optional.

import shutil

shutil.move("Screenshot.png", "Temp/")
shutil.move("elements.svg", "Temp")
shutil.move("Frame 5.png", "Temp/Frame 5.png")

We can move files three ways using shutil.move()

Last and final step. We need to create a new folder if it does not exist to move all files with the same type.

Create a Folder/Directory Using Python

We can create a folder using the os.mkdir() method.

Syntax: os.mkdir(path, mode=0o777, *, dir_fd=None)

Note: The mode and dir_fd arguments are optional.

from os import mkdir, path

if not path.exists("Temp"):
    mkdir("Temp")

The path.exists() method checks if the given location exists or not. We need to validate the existence of the directory because if a directory already exists, mkdir() will throw an error. And we want to create a directory only if it does not exists.

Now, as we have gone throw all the steps. It’s time to put them together.

Organize Files Based on File Type Using Python

To recap, we need to

Get a list of files
Extract extension for each file
Create a folder if it does not exist
Move files to the respective directory

from os import mkdir, path, walk
from pathlib import Path as pathlib
from shutil import move

IMAGE_EXT = [".jpg", ".png", ".jpeg", ".gif", ".webp", ".eps"]
DOWNLOAD_LOCATION = str(pathlib.home() / "Downloads")

file_list = []
for (dirpath, dirnames, filenames) in walk(DOWNLOAD_LOCATION):
    file_list.extend(filenames)
    break

def check_directory(dir_location):
    """Check if directory exists or not. If not create one."""
    if not path.exists(dir_location):
        mkdir(dir_location)

def file_mover(extention_list, new_location):
    """move file based on extension list to new location."""
    for file in file_list:
        if (path.splitext(file)[1]).lower() in extention_list:
            move(file, new_location)

def organize_images():
    """Organize images based IMAGE_EXT list"""
    image_location = path.join(DOWNLOAD_LOCATION, "Organized Images")
    check_directory(image_location)
    file_mover(IMAGE_EXT, image_location)

organize_images()

Here, most of the code is from earlier or very common. The one difference that you might see is that I have used the *pathlib module. To make a generic code that can work with Linux, Windows and macOS, I used the* pathlib *module. It can give me the direct path to the default home directory for every operating system. I have tested it on Ubuntu, Windows and macOS.*

I have tried to keep it simple as much as I could. In this example, I have shown how to organize image files. But you can do it for all kinds of files. It will be hard to read and understand here that is why I have pushed code that you can use for any type of file on my GitHub or Gist.

Conclusion

It is not difficult to operate and use files and directories in Python. If you know what you want to do and what you are looking for you can find almost everything that you need by googling. That is how I came across the walk() method.

If you want to learn more about Python, check out my ongoing Python series.

That was it. Thank you for reading.

References

You can subscribe to my Newsletter for more amazing resources and articles.

Till the next time 👋

File Handling in Python

Sahil — Tue, 04 Oct 2022 11:45:03 +0000

Just to refresh our memory, so far, we have seen all basics of python including variables, data types, functions, conditional statements and loops. You can check out them all on my Python 101 series page.

In every software development, we have to do some kind of operation on the file. We can do n number of operations using python. We can:

List of all Files
Move & Rename File
Delete File
Read File
Write File
Append File

I am sure we can do a lot more that I might have forgotten to mention. But, these are the basic operation that we can do using Python. So, let’s get started!!!!

Just to save time and to create a better understanding, I have merged some topics. Don’t worry, you will know what I am talking about. 😁

List & Rename files

In this section, I will show you what we can do with the os module of python. If you are a geek like me and want to rename files according to the same conventions then you can do that with python.

So, to do this, what do we need? 🤔

First, we need a list of all files. After that, we need to pick each file and rename it.

So, how do we get a list of all files?

As far as I know, there is no direct method in the os module that returns only file names. So, once we get a list of all files and directories, we need to take out the directory or we need to keep just files. To do that we can use the os.path.isfile() method of the os module. This method will return true if the given argument is a file. Same way, os.path.isdir() will return true if the given argument is a directory.

import os

# returns name of all files & directory exist in current location
files_dir = os.listdir('./blogs')
print(files_dir)

only_files = []
for i in files_dir:
    if os.path.isfile('./blogs/'+i):
        only_files.append(i)

only_dir = []
for i in files_dir:
    if os.path.isdir('./blogs/'+i):
        only_dir.append(i)

print('-'*15)
print(only_files) # prints all files
print('-'*15)
print(only_dir) # prints all directories

"""
OUTPUT:
['1.txt', '2.txt', '3.txt', '4.txt', '5.txt', '6.txt', '7.txt', '8.txt', 'Test Directory 1', 'Test Directory 2']
---------------
['1.txt', '2.txt', '3.txt', '4.txt', '5.txt', '6.txt', '7.txt', '8.txt']
---------------
['Test Directory 1', 'Test Directory 2']
"""

So, we got a list of all files.

Now, how do we rename all these files? 🤔

To rename the files, we can use the rename() method and apply it to each file. Let’s assume, we want to rename files with odd and even prefixes based on the file index.

# only_files is a list of all files in the blogs directory

for index, file_name in enumerate(only_files):
    if index % 2 == 0:
        os.rename('./blogs/'+file_name, './blogs/'+'Even-'+file_name)
    else:
        os.rename('./blogs/'+file_name, './blogs/'+'Odd-'+file_name)

Here, enumerate() method returns the counter (starting from 0) and value from iterator (only_files). So, we are checking if the index/counter is even then the code will add the prefix ‘Even-’ to the file name otherwise it will add ‘Odd-’. Find out more on enumerate() on Programiz.

You might have guessed the syntax of the os.rename() method but just in case:
Syntax: os.rename(source, destination) or os.rename(current_name, new_name)

One thing to remember here is that we need to give a whole or proper relative path to use this method. In our case, I have given ‘./blog’ and the ‘/’ to go inside the blogs directory. So the path of the file will become ‘./blogs/1.txt’.

Move & Delete Files

To move a file, we can use either the os or shutil module. In this blog, I will show you how to move a file using the rename() method of the os module. The syntax of rename() is the same, but the only catch is, as the second argument we have to use the destination path with a file name. It might be confusing, so let me show you.

import os

os.rename('./blogs/1.txt', './blogs/Test Directory 1/1.txt')

os.rename('./blogs/2.txt', './blogs/Test Directory 2/1.txt')

So, in the first rename method, we are taking 1.txt from the blogs directory and moving it to Test Directory 1 which is a subdirectory of blogs. In the second scenario, we are taking 2.txt and moving it to the Test Directory 1 directory with the name 1.txt. Yes, we just moved and renamed the file at the same time. If you know a little bit about the Linux commands then you might have observed that os.rename() is similar to the mv Linux command.

Now, how do we delete these files? 🤔

No worries! OS module got your back there as well. By using the remove() method we can remove the file. For example, if we want to remove the 3.txt file from the blogs directory. To do this, we can write the following code:

import os

os.remove('./blogs/3.txt')

Read, Write & Append Operations on File

To do any kind of operation we have to follow 3 basic steps:

Open File
Do Operation
Close File

We can follow these steps using two patterns. It is very hard to explain these patterns in words so, I will explain you using code.

# Open the file
file = open('./blogs/1.txt', 'r', encoding="UTF-8")

# Do operation
data = file.read()
print(data)

# Close the file
file.close()

"""
OUTPUT:

Python 101 Series
"""

So, in this first pattern, we need to open a file, do the operation and close the file manually. Here, open() is provided by Python.
Syntax: open(file, access_mode, encoding)
Here, the file represents the location or a location-like object of a file. The access_mode represents in which mode we want to open the file. We can do more than one task at a time which I will list out later. The encoding represents the encoding format in which we want to operate the file but it is optional. In most cases, we use UTF-8.

with open('./blogs/1.txt', 'r', encoding="UTF-8") as file:
    data = file.read()
    print(data)

In this second pattern, we are using the context manager pattern. If I have to say in short, I would say we use this context manager pattern to use our resources mindfully. The issue with the first approach is that sometimes we might forget to close the file. In this case, it won't affect much but when we handle thousands and millions of files it will affect our system. Context manager pattern will free up the resources as soon as we are done with the more efficient task. If you want to know more about it, you can check out this geeksforgeeks blog.

So, on the 1st line, we open a file using the with open() syntax and assign it to the file variable. It is the same as file = open(). But the main benefit of using this syntax is, python will clear this resource automatically after line 3. As we are not in the context of with open() all resources will be freed related to the file variable after line number 3. In industries, while handling files, most of the time you will see context manager patterns.

As we read the file, we can write and append data to the file in a similar manner. So, to write data into the file we can use the following code snippet:

with open('./blogs/temp.txt', 'w', encoding="UTF-8") as file:
    data = "Python 101 Series"
    file.write(data)

In the case of the write operation, if a file does not exist, it will create a new file and write data into it. And if a file is already with the same name, it will override it with the new data. This will only create just a new file, this open() method will not create a directory for use. So, if I write open("./blogs/temp/temp.txt", "w"), it will throw an error as there is no directory named temp exist.

Now, to solve the overriding issue of writing, we can use the append method. The append method will append the new data at the end of the file we opened.

with open('./blogs/temp.txt', 'a') as file:
    data = "\n New Python 101 Series"
    file.write(data)

So, from the above code, we can say that python will check if the file named temp.txt exists then it will open and append \n New Python 101 Series at the end of it. Otherwise, it will create a new file and write the data. You can see the output in the image below.

File Access Modes

In the table below, I have shown basic access modes. But some modes let you operate on binary data and also in binary format. You can check out the full list on this blog post.

Mode	Explanation
r	read operation
w	write operation
a	append operation
r+	To read and write data into the file. The previous data in the file will not be deleted.
w+	To write and read data. It will override existing data.
a+	To append and read data from the file. It won’t override existing data.

Conclusion

Finally! I hope you learned something after 8 minutes of reading😁.

In this blog, we understood how to handle files in python with some basic programs. I know it is a lot to take in at a time. But, you don't need to remember everything that I mentioned here.

That was it. Thank you for reading.

You can subscribe to my Newsletter for more amazing resources and articles.

Till the next time 👋

Modular Modules and Packages in Python

Sahil — Thu, 22 Sep 2022 09:02:41 +0000

In real-life development, we do not write every single line of code. If there are any resources or libraries available we generally use those as long as the source is trustable. It is just like the old saying, "If something is available use it, don't need to reinvent the wheel."

So, in this tutorial, I will show you

How to install any third-party package
How to use any package (inbuilt and/or third-party)

Once we understand the basic concepts of packages in python. We will work on a small project in which you will take input from a user and save it in different file formats.

What is a Module in Python?

Before we understand what is a package, we need to understand what are modules in python. Because all packages are a combination of different modules. So, what is a module?

In real-life development, code files can be of hundreds or even thousands of lines. To manage and handle all code in one file is very difficult and not at all the suggested way to write the code. That is why we divide our code into different parts. These separate parts are called modules.

What is a Module in Python?

Now, after creating all the modules, we zip up all the related modules and get one package. So, the package is nothing but a combination of the related module(s). Most of the time, when the codebase became too large we create a package. But, you may see most of the time, people treat packages and modules the same way in python.

How to install a Package / Module in Python?

To install any third-party packages we need pip. Pip is a package manager for Python. If you are coming from a JavaScript background you might know about npm or yarn. So, before we install any packages in python we need to confirm if pip is installed on our system or not. Generally, the python installer installs pip alongside python. To verify pip installation, you can use the following command in your terminal

pip --version

If this command throws an error or won't show any pip version you can re-install pip using the following command.

python -m pip install --upgrade pip

After the installation process finishes check the pip version again. If you are still facing an issue mention issue in the comment section. I will try my best to help.

Syntax to install the package using pip:

pip install package_name

This is just a basic syntax there are plenty of operations that we can do with pip. Once we have verified pip installation, we need the name of the package. As we become more experienced developers, we tend to know which package to use and if we do not know we can always ask on online forums such as StackOverflow.

In this blog, I will show you the package that will help us to save data in excel (Xls) format. I did a quick google search and found we can write data into excel using two packages

xlsxwriter
pandas

There can be any other packages but I came across these two quite often. We can just write into excel format using xlsxwriter. To read excel files we need another package named xlrd. To install xlsxwriter use the following command.

pip install xlsxwriter

How to use Package / Module in Python?

Now, if we do not face any errors while installing the package, let's verify the installation of the package. To verify you can open a terminal on your system and follow the steps shown in the image:

If xlsxwriter is not installed properly on your system, you will see something like this:

This is about third-party packages. But, how do we use inbuilt modules and packages, you ask? We can use inbuilt modules/packages the same way we use third-party packages. One difference is we do not have to install any modules/packages provided by python.

In python, we have csv and json named inbuilt packages. To use these packages we need to import them the same way we imported the xlsxwriter package.

Save all files of package locally

Sometimes we need to upload our codebase to specific cloud services, and it is possible that we won't be able to install the module the same way we installed it on our local system. Amazon lambda is an example of such a cloud service. In this kind of scenario, we need to zip the module together with our code file. So, to install the module locally at the specific location we can use the following command:

pip install package_name -t PATH_TO_ROOT_FOLDER

Let's assume we want to install xlsxwriter in the current directory. To install/download xlsxwriter in the current directory we can use the following command in our terminal.

pip install xlsxwriter -t .

Here, full stop (.) represents the current directory. If you want, you can provide the full path to the current or specific directory as well. Once the package is downloaded we will see the following files in the current directory.

Conclusion

Finally! We are at the end of this section 😁.

In this blog, we understood how to install and use packages. But, there is more to package concepts in python. You can create and publish your own package. We will see this in some other blog.

I know, it is a lot to take in at a time. But, you don't need to remember everything that I have mentioned here. I just showed you so that you can recall what is possible and what is not. There are some other methods as well that I have not mentioned here.

That was it. Thank you for reading.

Let me know if you need any help or want to discuss something. Reach out to me on Twitter or LinkedIn. Make sure to share any thoughts, questions, or concerns. I would love to see them.

Till the next time 👋

Create and Use Functions in Python

Sahil — Sat, 17 Sep 2022 13:33:58 +0000

What is a function? What does a function do? As the name suggests function does a specific task. This task can be used repeatedly or once as per the use case. The main purpose of a function is to separate the logic into different blocks. We can create our functions or use any of the available in-built functions in python.

Syntax:

def function_name(parameters):
    """docstring"""
    statement(s)
    return expression

We can see in the syntax that to create a function we need to use the def keyword following the function name. Parameter(s)/ Argument(s) is the only way through which we can pass data to the function. We can add a docstring to explain the use of function and its parameter(s). A function can return a value if desired. It is not compulsory to return something.

Once we define the function, to use it we need to call that function. Let's assume I have created a function named odd_or_even. To call this function, I can write like odd_or_even(). And if this function is expecting some argument(s), I can write it like odd_or_even(2).

Sum of Two Numbers

If we want to just print the sum of two numbers or just to see the sum of two numbers, we do not need to return anything. And if we try to assign it to a variable we get None as the value.

def sum_num(num1, num2):
    total = num1 + num2
    print(total)

sum_num(5,10)

# OUTPUT: 15

def sum_num(num1, num2):
    total = num1 + num2
    print(total)

temp = sum_num(5,10)
print(temp)

""" OUTPUT:
15
None
"""

Now, if we want to return the sum then we can do using the following code.

```python:1.py showLineNumbers
def sum_num(num1, num2):
total = num1 + num2
return total

temp = sum_num(5,10)
print(temp)

OUTPUT: 15




## Keyword arguments

In general, we need to pass arguments in a specific order to a function. But, using keyword arguments we can pass arguments in any order. Let's understand it by an example of division. In the example below, if we want `3/2` then we will write `divide(3,2)`.



```python
def divide(num1, num2):
    return num1/num2

ans = divide(3,2)
print(ans)

# OUTPUT: 1.5

But now, by mistake, we wrote divide(2,3). This error can make the whole logic wrong.

def divide(num1, num2):
    return num1/num2

ans = divide(2,3)
print(ans)

# OUTPUT: 0.6666666666666666

To save yourself from this kind of mistake we can use keyword arguments. This is just a normal argument passing, but instead of passing only value, we will pass the variable name as well.

def divide(num1, num2):
  return num1/num2

ans = divide(num1=3, num2=2)
print(ans)

# OUTPUT: 1.5

So, now we can pass data in any order and this way logic will be intact.

def divide(num1, num2):
  return num1/num2

ans = divide(num2=2, num1=3)
print(ans)

# OUTPUT: 1.5

This is one of the best practices for function in python.

Variable Length Arguments

In some cases, we don't know the number of arguments we will receive in function. Or we do not want to restrict the function with a specific number of arguments. In this kind of situation, we can use variable-length arguments. There are two types of variable-length arguments:

*args (Non-Keyword Arguments)
**kwargs (Keyword Arguments)

1. *args (Non-Keyword Arguments)

In non-keyword arguments, we just pass n number of data to the function. *args is not the fixed name of non-keyword arguments. We can use any name. It is just a standard to use *args.

Let's assume, we are asking users to enter the fruits they have. Noe, every user might have a different number of fruits. To save or to operate on all fruits entered by users, we can use non-keyword arguments.

def fruit_list_operation(*fruits):
  for fruit in fruits:
    print(fruit, end=",")

fruit_list_operation("apple", "banana")

print()

fruit_list_operation("apple", "banana", "grape", "mango")

""" OUTPUT:
apple,banana,
apple,banana,grape,mango,
"""

Here, we have used for loop as *args will create a sequence data. As you see, when we pass 2 fruits loop runs for 2 times and 4 times in the second case.

The print(fruit, end=",") function is used to notify the interpreter that does not end the statement with a new line. Instead, use comma(,) to end the statement. And when we used the print() function, it just wrote a new line and moved on.

2. **kwargs (Keyword Arguments)

The variable-length keyword argument is almost the same as the variable-length non-keyword arguments. The only difference we use this variable-length keyword argument function when we do not know how many keyword arguments we will be getting.

def my_project(**kwargs):
  for key, value in kwargs.items():
        print("%s == %s" % (key, value))

my_project(frontend = "Angular", backend = "Python", database="MongoDB")

""" OUTPUT:
frontend == Angular
backend == Python
database == mongodb
"""

Default Parameter Value

When we want to use some default value if the argument is not passed to a function, we can use the default parameter. We can use any number and any datatype as the default parameter. There is just one condition we have to use the default value argument as the last parameter of the function.

def division(num1, num2=2):
  print(num1/num2)

division(16,4)
division(16)

""" OUTPUT:
4.0
8.0
"""

Let's see what happens if we keep the first argument as the default argument.

def division(num1=2, num2):
  print(num1*num2)

division(16,4)
division(16)

""" OUTPUT:
File "functions.py", line 1
    def division(num1=2, num2):
                         ^^^^
SyntaxError: non-default argument follows default argument
"""

Docstring

As we have seen in the syntax, the first commented statement in any function is considered as the docstring. Docstring is nothing but some commented statements explaining the use of the function, its arguments and the return expression. To print or to get the docstring we can use the following syntax: *function_name*.__doc__. Let's see an example to understand the concept of the docstring.

def is_odd(num1):
  """Function to check if the number is odd or not.

      Parameters:
        num1 (float): number to be checked

      Returns:
        is_odd (boolean): true if number is odd

  """
  is_odd = num1 % 2 != 0
  print(is_odd)
  return is_odd

is_odd(16)
is_odd(1)
print("-"*15)
print(is_odd.__doc__)

""" OUTPUT:
False
True
---------------
Function to check if the number is odd or not.

      Parameters:
        num1 (float): number to be checked

      Returns:
        is_odd (boolean): true if number is odd
"""

When we use a third-party library or even some python in-built library we can use docstring to understand the use of a specific function.

print(range.__doc__)

""" OUTPUT:
Return an object that produces a sequence of integers from start (inclusive)
to stop (exclusive) by step.  range(i, j) produces i, i+1, i+2, ..., j-1.
start defaults to 0, and stop is omitted!  range(4) produces 0, 1, 2, 3.
These are exactly the valid indices for a list of 4 elements.
When a step is given, it specifies the increment (or decrement).
"""

Conclusion

That was it. Thank you for reading.

Let me know if you need any help or want to discuss something. Reach out to me on Twitter or LinkedIn. Make sure to share any thoughts, questions, or concerns. I would love to see them.

Till the next time 👋

If you like this blog you can subscribe to my Newsletter for more amazing content.

Conditions and Loops in Python

Sahil — Tue, 13 Sep 2022 13:02:24 +0000

What if you want to do some specific tasks when the string is "Python" and some other tasks when the string is "Java". To achieve this conditioning we can use conditional statements in python or any other language. When we want to do some task for n number of times or we want to loop through some specific list, we can use different types of loops in this kind of situation.

Conditional Statements in Python

In python, we have two types of conditional statements.

If-else
if-elif-else (if - else if - else)

Both conditional statements seem almost similar, and actually, they are. There is just a minor difference between these two statements. We will understand this difference using code examples. If you know any other language(s), you might be wondering if python has a switch case or not. NO, python does not have a switch case.

if-else

Syntax:

if(conditional expresson):
    statement
else:
    statement

If I have explained the if-else statement in plain English then it would be the "either this or that" statement.

Let's assume if the string is java, I want to do the sum of two numbers else I want to multiply two numbers.

num1 = 5
num2 = 2

lang = 'java'

if(lang=="java"): # check if value of lang variable is 'java'
    c = num1 + num2
    print(c)
else:
    c = num1 * num2
    print(c)

# OUTPUT: 7

num1 = 5
num2 = 2

lang = 'java'

if(lang!="java"): # check if value of lang variable is NOT 'java'
    c = num1 + num2
    print(c)
else:
    c = num1 * num2
    print(c)

# OUTPUT: 10

It is not compulsory to use else part. You can use only if part as well. For example, add the string "Programming" to the existing string if it is "Python" or does nothing.

lang = 'Python'

if(lang=="Python"):
    lang = lang +" Programming"

print(lang)
# Output: Python Programming


lang = 'Java'

if(lang=="Python"):
    lang = lang +" Programming"

print(lang)
# Output: Java

Sometimes, you can go into the if statement using only variable as well. For example,

lang = 'Python'
if(0):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python

lang = 'Python'
if(1):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python Programming

lang = 'Python'
if(True):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python Programming

lang = 'Python'
if(False):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python

lang = 'Python'
if(""):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python

lang = 'Python'
if("random string"):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python Programming

lang = 'Python'
if(None):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python

lang = 'Python'
if([1]):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python Programming

lang = 'Python'
if([]):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python

lang = 'Python'
if({'a':1}):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python Programming

lang = 'Python'
if({}):
    lang = lang +" Programming"
print(lang)
# OUTPUT: Python

If we want to combine multiple conditions we can use and and or within the if statement.

lang = 'PYTHON'

if(lang.isupper() and len(lang) > 2):
    print("Worked!")
# OUTPUT: Worked!

if(lang.islower() or len(lang) == 6):
    print("Worked again!")
# OUPUT: Worked again!

if-elif-else

Syntax:

if(conditional expresson):
    statement
elif(conditional expresson):
    statement
else:
    statement

We use this conditional statement when we want to test more than one scenario. Let's understand this by an example.

marks = 75

if(marks >= 34 and marks <= 50):
    print("Grade C!")
elif(marks > 50 and marks <= 75):
    print("Grade B!")
elif(marks > 75 and marks <= 100):
    print("Grade A!")
elif(marks >= 0 and marks < 34):
    print("FAIL!")
else:
    print("Something is wrong!")

# OUTPUT: Grade A!

Be aware when you use multiple if and if-elif. A single error in your logic and all your code-base can throw an error. To understand this, see the below example.

marks = 75

if(marks > 75 and marks <= 100):
    print("Grade A!")
elif(marks > 50):
    print("Grade B!")
elif(marks >=34):
    print("Grade C!")
else:
    print("Something is wrong!")

print("-----------------------------------")

marks = 75

if(marks > 75 and marks <= 100):
    print("Grade A!")
if(marks > 50):
    print("Grade B!")
if(marks >=34):
    print("Grade C!")

""" OUTPUT:
Grade B!
-----------------------------------
Grade B!
Grade C!
"""

Loops in Python

In python, we have two kinds of loops.

for Loop
while Loop
Loop Control Statements

for Loop in Python

Syntax:

for iterator in sequence:
    statement(s)

For loop is used when we want to traverse through the sequential datatype. For example traversing through lists, tuples, strings etc.

fruits = ['apple', 'banana', 'mango', 'grapes', 'peach']

for fruit in fruits:
    print(fruit)

""" OUTPUT:
apple
banana
mango
grapes
peach
"""

for i in range(len(fruits)):
    print(fruits[I])

""" OUTPUT:
apple
banana
mango
grapes
peach
"""

In the first example above, in the fruit variable, we are directly getting each element of the fruits list. Whereas, in the second example, we are accessing each element by index reference. The range(n) function returns a sequence of numbers.
Syntax: range(start, stop, step).

for i in range(2, 13, 3):
    print(i)

""" OUTPUT:
2
5
8
11
"""

We can use for loop with dictionaries as well.

dict1 = {"lang": "python", "year": "2021", "month": "november"}

for item in dict1:
    print(item, ":" , dict1[item])

""" OUTPUT:
lang : python
year : 2021
month : november
"""

while Loop in Python

Syntax:

while expression:
    statement(s)

When you need to do some work repeatedly until some condition satisfies, in that situation you can use a while loop.

Let's assume we need to print "Hello World" until the count variable became 5.

count = 0
while (count < 5):
    print("Hello World")
    count = count + 1

Loop Control Statements

There are times when you need to break the loop or skip some iterations in the loop or do nothing. This kind of thing can be done using Loop Control Statements. In python, we have three looping control statements.

Break statement
Continue statement
Pass statement

We use a break statement when we want to break the loop when certain conditions match. For example,

fruits = ['apple', 'banana', 'mango', 'grapes', 'peach']

for fruit in fruits:
    if(fruit == 'mango'):
        break
    print(fruit)

""" OUTPUT:
apple
banana
"""

While the break statement breaks the loop, the continue statement makes the loop skip the current iteration and forces it to go to the next iteration.

fruits = ['apple', 'banana', 'mango', 'grapes', 'peach']

for fruit in fruits:
    if(fruit == 'mango'):
        continue
    print(fruit)

""" OUTPUT:
apple
banana
grapes
peach
"""

Pass statement is a meaningless piece of code that suggests the interpreter to do nothing. It is similar to the comment but comments are ignored by interpreters while the pass statement executed by the interpreter and as per the name does nothing. Pass statement is used with function and class as well, which I will explain in the next section.

fruits = ['apple', 'banana', 'mango', 'grapes', 'peach']

for fruit in fruits:
    if(fruit == 'mango'):
        pass
    print(fruit)

""" OUTPUT:
apple
banana
mango
grapes
peach
"""

for...else & while...else

During the development journey, there are times when you might need to use for..else and while...else. These are just normal for and while loop with special else cases. Let's see an example to understand it.

fruits = ['apple', 'banana', 'mango', 'grapes', 'peach']

for fruit in fruits:
    if(fruit == 'mango'):
        break
    print(fruit)
else:
    print("Loop Did Not Break!")

""" OUTPUT:
apple
banana
"""

fruits = ['apple', 'banana', 'mango', 'grapes', 'peach']

for fruit in fruits:
    if(fruit == 'abc'):
        break
    print(fruit)
else:
    print("Loop Did Not Break!")

""" OUTPUT:
apple
banana
mango
grapes
peach
Loop Did Not Break!
"""

So, you might have guessed. When the loop does not break OR the loop completes all the iterations without interruption, else block is executed. The same happens with the while loop as well.

i = 1
while i < 6:
  if(i == 3):
    break
  print(i)
  i += 1
else:
  print("All iterations of While are executed successfully.")

""" OUTPUT:
1
2
"""

Conclusion

Finally! We are at the end of this section 😁.

If you want to find out more about control statements and loops in python, check out Programiz.

That was it. Thank you for reading.

Make sure to share any thoughts, questions, or concerns. Let me know if you need any help or want to discuss something. Reach out to me on Twitter or LinkedIn. Make sure to share any thoughts, questions, or concerns. I would love to see them.

Till the next time 👋

DEV Community: Sahil

Mastering Python Comprehensions: Crafting Efficient and Readable Code

Types of Comprehension

1. List Comprehension

2. Set Comprehension

3. Dictionary Comprehension

4. Generator Comprehension

If/Else with Python Comprehension

Nested loop with Python Comprehension

Conclusion

How to read and write Sequential data gracefully in python

Add/create a list using list comprehension []

Add/create a list using for loop

Time took to add data for list comprehension

Time took to add data for the for loop

Conclusion

Upload CSV Data To MongoDB

How to read CSV data in Python

How to convert pandas dataframe into dict

How to insert data to MongoDB

How to insert multiple rows at once in MongoDB

How to insert data in MongoDB

Conclusion

A Date With the Datetime Module in Python | Part - 2

Get the Current Week Number using Python

strftime() in Python

How to Get Name of Month in Python

How to Get Name of Weekday in Python

Combine Multiple Format Codes in strftime() Method

A Date With the Datetime Module in Python | Part - 1

Extract Current Day, Month and Year using Python

Extract Day of the Week using Python

Extract Current Time using Python

Quick Guide to Object-Oriented Programming With Python

What is OOP (Object-Oriented Programming)?

Class & Object in Python

What is the init() method in Python?

What is the ‘self’ keyword in Python?

Inheritance in Python

Conclusion

Delete Files & Directories Using Python

Planning

Coding

1. Import necessary modules

2. Get the list of all files and directories of a specific location

3. Separation of Files and Directories

4. Delete files

5. Delete directories

6. Convert whole logic into a function

Final Touch & Execution

Organize Any Files and Folders Using Python

Organize Download Folder

Plan For Creating Auto File Organizer

How to get the list of all files in Python

How to Get File Extension Using Python

How to Move Files Using Python

Create a Folder/Directory Using Python

Organize Files Based on File Type Using Python

Conclusion

References

File Handling in Python

List & Rename files

Move & Delete Files

Read, Write & Append Operations on File

File Access Modes

Conclusion

Modular Modules and Packages in Python

What is a Module in Python?

What is a Module in Python?

How to install a Package / Module in Python?

How to use Package / Module in Python?

Save all files of package locally

Conclusion

Create and Use Functions in Python

Sum of Two Numbers

OUTPUT: 15

Variable Length Arguments

1. *args (Non-Keyword Arguments)

2. **kwargs (Keyword Arguments)

Default Parameter Value

Add/create a list using list comprehension `[]`

Add/create a list using `for` loop

Time took to add data for the `for` loop

How to convert pandas `dataframe` into `dict`