Recently I had a problem, where I had to run multiple tasks in python. The issue was each task consisted of smaller operations, one of which was waiting for external service.
Let's say we have three tasks:
task_A: task_B: task_C:
+------------+ +------------+ +------------+
| | | | | |
| A_1 | | B_1 | | C_1 |
| | | | | |
+------------+ +------------+ +------------+
| | | | | |
| A_wait | | B_wait | | C_wait |
| | | | | |
+------------+ +------------+ +------------+
| | | | | |
| A_2 | | B_2 | | C_2 |
| | | | | |
+------------+ +------------+ +------------+
Tasks A_1, B_1, C_1, A_2, B_2, C_2 are what we actually want to execute, but we also need to wait for some period of time between the tasks - that's represented as tasks A_wait, B_wait, C_wait.
If we were to execute these in order we would execute
+-------+ +-----------------------+ +-------+
| | | | | |
| A_1 | | A_wait | | A_2 |
| | | | | |
+-------+ +-----------------------+ +-------+
+-------+ +-----------------------+ +-------+
| | | | | |
| B_1 | | B_wait | | B_2 |
| | | | | |
+-------+ +-----------------------+ +-------+
+-------+ +-----------------------+ +-------+
| | | | | |
| C_1 | | C_wait | | C_2 |
| | | | | |
+-------+ +-----------------------+ +-------+
Here's the gist with source:
So that means if, e.g. each wait lasts for one minute, the total time of waiting will be three minutes. And if we add another task, this time will then be four minutes.
What we could do instead is we could start executing task_B while we're waiting for A_wait to complete, and when we hit B_wait, we can start task_C. If we do that we could reduce the time of waiting to 1 minute in total, even after adding another task.
This is exactly what we can do using python's asyncio module. It allows us to write asynchronous code.
The order of execution that I wanted to achieve was:
+-------+ +-----------------------+ +-------+
| | | | | |
| A_1 | | A_wait | | A_2 |
| | | | | |
+-------+ +-----------------------+ +-------+
+-------+ +-----------------------+ +-------+
| | | | | |
| B_1 | | B_wait | | B_2 |
| | | | | |
+-------+ +-----------------------+ +-------+
+-------+ +-----------------------+ +-------+
| | | | | |
| C_1 | | C_wait | | C_2 |
| | | | | |
+-------+ +-----------------------+ +-------+
here's my first attempt:
Basically, we need to define our tasks as async and perform await on some other async call. In our case it will be wait call lasting for one minute. This will cause current async task to be suspended and next one to be launched.
The issue with this code is that the order in which tasks are executed is not defined. So, the first task to execute could be task_C, we just can't know.
Fortunately there's a way to specify the order in which we want our tasks to be executed. All we need to do is to create Task objects in the order in which we want them to be scheduled for execution.
With that I have asynchronous execution of tasks with predictable task scheduling.
Top comments (0)