maryu0

Posted on Jun 5

Python AsyncIO Explained: Coroutines, Tasks, Queues, Locks & Semaphores with Examples

#resources #api #python #tutorial

Asynchronous programming is one of those topics that feels confusing until it suddenly clicks.

When I first started learning Python's asyncio, I understood the syntax but struggled to understand why things behaved the way they did. Why does await sometimes run sequentially? Why do we need tasks? When should we use locks, queues, or semaphores?

To build a stronger intuition, I created a small repository of focused examples that explore the core concepts of asyncio step by step.

Repository: https://github.com/maryu0/python-asyncio

Why AsyncIO?

Traditional Python code executes one operation at a time.

For CPU-heavy work, this is often fine. However, many modern applications spend most of their time waiting for external resources:

API calls
Database queries
File operations
Network requests
Message queues

While the program is waiting, the CPU is mostly idle.

asyncio allows Python to switch to other work during these waiting periods, improving efficiency for I/O-bound applications.

Learning Path

The repository is organized from beginner-friendly concepts to more practical concurrency patterns.

1. Coroutines

File: coroutine.py

The starting point of asyncio.

You'll learn:

How to create async functions
What await does
How the event loop executes coroutines

Example:

async def greet():
    print("Hello")
    await asyncio.sleep(1)
    print("World")

Understanding coroutines is the foundation for everything else.

2. Why Tasks Matter

File: Need_for_TASKS.py

One of the biggest beginner misconceptions is assuming that multiple await statements automatically run concurrently.

They don't.

Consider:

await task1()
await task2()
await task3()

This executes sequentially.

This example demonstrates why asyncio.create_task() exists and how tasks enable concurrent execution.

3. Running Concurrent Work

File: tasks.py

Once tasks are introduced, we can run multiple coroutines at the same time.

Example:

t1 = asyncio.create_task(worker())
t2 = asyncio.create_task(worker())

await t1
await t2

This significantly reduces waiting time for I/O-heavy operations.

4. gather() and TaskGroup

File: gather.py

When managing multiple concurrent operations, Python provides powerful abstractions.

asyncio.gather()

Run multiple coroutines together and collect their results.

results = await asyncio.gather(
    task1(),
    task2(),
    task3()
)

TaskGroup

Introduced in newer Python versions, TaskGroup provides safer task management and structured concurrency.

This file compares both approaches and explains when each is useful.

5. Protecting Shared Resources

File: Lock.py

Concurrency introduces a new challenge: race conditions.

When multiple coroutines access shared data simultaneously, unexpected behavior can occur.

asyncio.Lock ensures only one coroutine modifies a shared resource at a time.

lock = asyncio.Lock()

async with lock:
    shared_counter += 1

This pattern is essential whenever multiple tasks update shared state.

6. Practical AsyncIO Patterns

File: Practice.py

This file combines multiple concepts into realistic examples:

Concurrent execution with gather
Timeout handling using wait_for
Fallback strategies
Async generators
Streaming-style output

These patterns are commonly used in production systems interacting with APIs and external services.

7. Producer-Consumer Queues

File: queue.py

Real-world systems often produce work faster than it can be processed.

asyncio.Queue acts as a buffer between producers and consumers.

Common use cases include:

Job processing systems
Event pipelines
Background workers
Message handling

This example demonstrates how queues help smooth bursts of incoming work.

8. Limiting Concurrency with Semaphores

File: semaphore.py

Sometimes running everything concurrently is actually a bad idea.

Imagine sending 1,000 API requests simultaneously.

You might:

Hit rate limits
Overload a service
Consume excessive resources

asyncio.Semaphore limits how many tasks run at once.

semaphore = asyncio.Semaphore(3)

async with semaphore:
    await make_request()

This pattern is extremely useful when working with external APIs.

Key Takeaways

After working through these examples, a few ideas became much clearer:

Coroutines define asynchronous work.
Tasks enable concurrent execution.
gather() and TaskGroup help coordinate multiple tasks.
Locks prevent race conditions.
Queues provide buffering between producers and consumers.
Semaphores prevent excessive concurrency.
Async generators enable streaming-style workflows.

Most importantly, asyncio isn't about making code magically faster.

It's about making better use of waiting time.

Repository Structure

python-asyncio/
│
├── coroutine.py
├── Need_for_TASKS.py
├── tasks.py
├── gather.py
├── Lock.py
├── Practice.py
├── queue.py
├── semaphore.py
└── Concepts.md

Recommended Learning Order

coroutine.py
Need_for_TASKS.py
tasks.py
gather.py
Lock.py
Practice.py
queue.py
semaphore.py
Concepts.md

Following this order helps build intuition gradually, from basic coroutines to advanced concurrency control patterns.

Who Is This For?

This repository is intended for:

Python beginners learning asyncio
Students exploring concurrent programming
Developers preparing for backend engineering
Anyone who wants hands-on asyncio practice before using frameworks or production systems

The examples are intentionally small and educational, focusing on clarity rather than production architecture.

Explore the Repository

GitHub: https://github.com/maryu0/python-asyncio

If you're learning asyncio, I'd love to know:

Which asyncio concept was the hardest for you to understand when you first started?

Tags: #python #asyncio #beginners #programming

DEV Community