Studying again the basic concepts of Python, a question that had always arisen came back to me, what is the difference in performance of each of the Loop options that we have within the language? (For Loop, List comprehension and High order functions), for this I had to understand how each one worked inside and find the best way to measure them, this is the reason why I began to investigate and test.
However, first I would like to explain what was the experiment that I carried out and how I came to think that this would be the best option.
Experiment
The experiment consisted of performing three functions in order to monitor the performance and not the process carried out by each of the iterables. At the beginning, a person from the community told me that the psutil module could help me by measuring the performance; however, this tool allows me to measure a complete process and it was very difficult and time-consuming to measure each of the functions separately, additionally we had a small detail and that is that these tests could not be done on large servers with a capacity of RAM and processors multicore, my machine was just a Macbook with 8 Gb of ram and a dual-core I5 core, a joke if we start to see the types of machines we have today.
With this in mind I decided to go for a basic mathematical iterative process that would start consuming enough resources that not too many processes would stall without affecting upcoming loops.
The solution was to iterate x number of times a number that was multiplied by the previous sequence and store everything in a list, something similar to fibonacci, but with much higher values provided that the list began to be much larger.
Hoping that the garbage collector would remove the variables in each iteration, I used the same names in all the calls and only changed the process, the code was as follows.
Code
def high_order_func(rang):
start_time = time.time()
n = 2
my_list = map(lambda i: i * n, range(rang))
my_list = list(my_list)
end_time = time.time()
how_much_time = end_time - start_time
return how_much_time
def comprehension_loop(rang):
start_time = time.time()
n = 2
my_list = [n * i for i in range(rang)]
end_time = time.time()
how_much_time = end_time - start_time
return how_much_time
def loop(rang):
start_time = time.time()
my_list = []
n = 2
for i in range(rang):
n = i * n
my_list.append(n)
end_time = time.time()
how_much_time = end_time - start_time
return how_much_time
if __name__ == '__main__':
import time
rang = 1000
high_order_func = high_order_func(rang)
comprehension_loop = comprehension_loop(rang)
loop = loop(rang)
print("loop ", str(loop), " sec")
print("list comprehension ", str(comprehension_loop), " sec")
print("Higher order funciton ", str(high_order_func), " sec")
Results
Each outcome is separated by its respective value of the rang variable and before you read on I would like you to try to guess which one is going to win and try to explain why you think this will happen.
Initially, it was tested with a small rang in order to see in small processes what the real difference was.
rang = 1000
Test 1
loop 0.009955167770385742 sec
list comprehension 0.02051258087158203 sec
Higher order funciton 8.821487426757812e-06 sec
Test 2
loop 8.0108642578125e-05 sec
list comprehension 5.507469177246094e-05 sec
Higher order funciton 0.0001220703125 sec
Test 3
loop 8.082389831542969e-05 sec
list comprehension 5.507469177246094e-05 sec
Higher order funciton 0.00012302398681640625 sec
At this point we can see that there is no clear difference between the three functions, let us remember that these values do not even take milliseconds yet, since at the end as you can see they have the scientific notation e-05 or e-06 which tries to say that they are They add that amount of zeros to the value at the beginning, so we can say that there is no winner since the values have a clear fluctuation of times but do not exceed e-05.
rang = 100000
Test 1
loop 0.008324146270751953 sec
list comprehension 0.007503986358642578 sec
Higher order funciton 0.012623071670532227 sec
Test 2
loop 0.009238958358764648 sec
list comprehension 0.007218837738037109 sec
Higher order funciton 0.015091180801391602 sec
Test 3
loop 0.00857686996459961 sec
list comprehension 0.007061004638671875 sec
Higher order funciton 0.013355016708374023 sec
This Output gives us an overview of how the tests are going to evolve from now on, since it shows us that the higher-order functions begin to slowly fall behind, however the other two remain quite similar in execution times.
rang = 100000000
Finally, the definitive test, since if I continued to increase the rang the machine began to throttle, which affected the tests beyond giving me a more realistic result than the one I was looking for.
Test 1
lloop 9.187693119049072 sec
list comprehension 9.024915933609009 sec
Higher order funciton 14.004309892654419 sec
Test 2
loop 9.79594111442566 sec
list comprehension 9.057484865188599 sec
Higher order funciton 15.923699855804443 sec
Test 3
loop 8.725109577178955 sec
list comprehension 8.681398868560791 sec
Higher order funciton 14.65591287612915 sec
In this last test we have a fairly clear difference and two winners that are close to being quite tied, since the higher order functions lagged far behind compared to the other two, but what happened here? What could have happened?
Conclusión
These results surprised me more than I expected, as at one point I thought I was doing double processing with these functions, however reading a bit of the [real python] article(https://realpython.com/python-map-function/) I understood that the map iterates over the first element creating an object with those values, but when passing it to the list it has to do a bigger process by operating all those bytes again, therefore, it makes them more expensive.
What do you think? Tell me in the comments your opinion about it, if this test seemed reliable or you think I had to take it from another point of view or something in my test is wrong.
I also invite you to read the following articles that allows you to understand some more things about loops.
List-comprehension: https://realpython.com/list-comprehension-python/
For Loops: https://realpython.com/python-for-loop/
Top comments (0)