Bad loop/good loop in Python

Python is a powerful language with a fast learning curve. However, there are some traps.

Bad loop

Loops can be tricky, especially in Python. I've tested some existing scripts, and, most of the time, it's working fine, but sometimes it's quite slow.

The culprit is often a lousy loop. There are multiple ways to write loops in Python, e.g., with for.

There are mutable and immutable objects, and it's quite essential to understand the difference. Unlike lists, strings are immutable, so if you make some hazardous operations involving strings inside your loop, and you have a lot of items, the execution time might be terrible.

The following is bad:

concat = ''
for d in data:
    concat += d

Assuming that the list data is extracted from a .csv file, it could be 1 million rows, especially if you are manipulating some public Government data. With this code, you are recreating your variable for each row. It could take a lot of time to execute!

Good loop

A better approach could be:

concat = []
for d in data:
    concat.append(d)
result = '' . join(concat)

It takes only a few seconds to process billions of rows. With a manual concatenation (first example), it takes minutes!

Imagine that you have to tweak your code several times to make it work, it becomes a nightmare to test it with the first example. It's even worse with nested loops.

But wait, do you always need a loop?

Most of the time, you write loops to extract information or to concatenate things. Unfortunately, the best tricks are often counterintuitive.

Besides, fewer lines make the code more readable.

There are fantastic tools such as itertools with a lot of functions you can use, which prevents you from reinventing the wheel.

Make functions

Functions are great. It allows you to restructure your code into reusable blocks (Single responsibility principle):

def looper(d):
    # you can make other treatments here
    return d

concat = [ looper(d) for d in data ]
result = '' . join(concat)

Wrap up

Love Python, make functions and don't overuse loops.

Comments

[Prayson Wilfred Daniel]

In most case, I would avoid the loop all together. We can refactor the code:

# 🫣
concat = []
for d in data:
    concat.append(d)
result = '' . join(concat)

# 😊
result = ''.join(d for d in data)

And this:

def looper(d):
    # you can make other treatments here
    return d

result = '' . join(looper(d) for d in data)

[Banji]

Tnx bro for sharing this great post with us
Keep it up :)