Concurrency in python, what kind of implementation you really need.

We've all heard the word "concurrency" before, but what it actually means? Concurrency is basically the concept of executing two or more tasks at the same time. This can help us to improve i.e. response time and computation speed of our programs. To answer the question; "which implementation of concurrency you should go with" first you'll need to ask yourself what kind of situation you're dealing with.

1. CPU bound
The term CPU-bound describes a scenario where the execution of a task or program is highly dependent on the CPU, i.e computation power of the processor. Some examples of CPU bound tasks are computing SHA-1 checksums, doing binary searches on very large arrays, manipulating some images in which we'll need to recalculate the pixels from an original image, etc.

2. I/O bound
The term I/O bound refers to a condition in which the time taken to complete the task is primarily determined by the period spent waiting for input/output operations. Examples of I/O bound tasks are making HTTP API requests, making database queries etc, in which we need to wait for response of external entities.

So now that you can figure out what kind of situation you're dealing with, let's proceed with making our program faster with the power of concurrency!

Implementing concurrency for I/O bound tasks in python.
Python provides two interfaces to implement I/O bound concurrency namely threading and asyncio, in which it utilizes a mechanism called "Context-Switiching". Now you might ask what is context switching?. Context switching involves storing the context or state of a process so that it can be reloaded when required and execution can be resumed from the same point as earlier. This allows a single CPU to be shared by multiple processes. Yes, even though we only use single CPU we can still execute our processes concurrently since CPU stays idle for most of the time due to very nature of I/O bound tasks. Let's consider a case where your program makes API request to some external server and while that server is processing your request your CPU is just waiting in idle mode doing nothing! We can use context switching here to save current state and do something else while that external server is processing our request and resume when response is received from external server.

Here's an example of above said scenario with the help of threading.

import time
import threading


def io_bound_func():
    print("api request sent, waiting for response. . .")
    time.sleep(1)
    print("response recived!")


all_threads = []
for i in range(4):
    thread = threading.Thread(target=io_bound_func, name=f"Thread :: {i}")
    all_threads.append(thread)
    thread.start()

for thread in all_threads:
    thread.join()

On running this we get this output.

api request sent, waiting for response. . .
api request sent, waiting for response. . .
api request sent, waiting for response. . .
api request sent, waiting for response. . .
response recived!
response recived!
response recived!
response recived!

Now let's try to understand this. Here we tried to simulate time taken by external API server to process the request with the help of time.sleep(1) in normal single threaded programs the output should've been like this

api request sent, waiting for response. . .
response recived!
api request sent, waiting for response. . .
response recived!

i.e. our program should've waited for good one second before making another request, but in above example we can see that program did context switching and started working on another thread while current thread is waiting for response, savings CPU from going idle and improving execution speed at the same time!

Here's the same example but with the help of asyncio.

import asyncio


async def io_bound_func():
    print("api request sent, waiting for response. . .")
    await asyncio.sleep(1)
    print("response recived!")


async def main():
    await asyncio.gather(
        io_bound_func(), io_bound_func(), io_bound_func(), io_bound_func()
    )


asyncio.run(main())

Now I'm not going to explain all of the advantages and disadvantages of asyncio and threading over one another here, that might be the topic of another article, but basic difference here is while in threading context switching happens automatically, asyncio gives programmer more control to manually handle context switching with the help of async and await keywords.

Implementing concurrency for CPU bound tasks in python.
Python provides multiprocessing module to process CPU bound tasks concurrently. Here it uses the concept of parallelism. i.e. program actually run parallel to one another on multiple CPU cores at the same time, unlike context-switiching where single CPU handles all of the processes. Let's consider a case where we are trying to calculate a sum of all elements of a huge list. If our machine has 4 cores, we can "cut" the list into four smaller lists and calculate the sum of each of those lists separately on separate core and then just add up those numbers. We'll get a ~4x speedup by doing that! Here's a code example of the above scenario.

import multiprocessing
from multiprocessing import Process

manager = multiprocessing.Manager()
return_dict = manager.dict()


def chunks(lst, n):
    """Yield successive n-sized chunks from lst."""
    for i in range(0, len(lst), n):
        yield lst[i : i + n]


def cal_sum(idx, arr):
    return_dict[f"part-{idx}"] = sum(arr)


# divide array containing 100 elements into
# four parts containing 25 elements  each.
array = list(chunks(range(1, 101), 25))

processes = []

for idx, arr in enumerate(array):
    proc = Process(target=cal_sum, args=(idx, arr))
    processes.append(proc)
    proc.start()

for proc in processes:
    proc.join()


print(return_dict.values())
print(sum(return_dict.values()))

Here we have a large list containing 100 elements, well not really "large" but let's suppose it is. We have made a function called chunks() which splits it into four successive list containing 25 elements each, now with the help of multiprocessing.Process we're calculating sum of each part on different CPU cores in parallel and at the end we add all four parts into one to print resultant sum of all the elements of an array. Here's the output:

[1575, 950, 2200, 325]
5050

Where 1st line prints sum of four different parts calculated on multiple CPU cores and in the end it prints sum of all four parts.

Conclusion
So to summarise all of that;

if io_bound_task:
   print("Threading or Asyncio")
elif cpu_bound_task:
   print("Multiprocessing")

Simple as that! That's it for today, hope you enjoyed reading the post. Thank you!