DEV Community: Christophe Bornet

Introducing BlockBuster: is my asyncio event loop blocked?

Christophe Bornet — Wed, 08 Jan 2025 16:47:56 +0000

Asynchronous I/O was introduced in Python 3.5 as an alternative to threads to handle concurrency.
The promises of asynchronous I/O, and of the asyncio implementation in Python, are that by not spawning memory-expensive OS threads, systems use less resources and are more scalable. Also, in asyncio, schedule points are explicit through the await syntax whereas in thread-based concurrency, the GIL may be released at hard to guess points of the code. So asyncio based concurrent systems are easier to reason about and to debug. Eventually, it's possible to cancel asyncio tasks which is not easily doable with threads.

But to really benefit from these promises, it is very important to not do blocking calls in async coroutines. A blocking call can be a network call, a file system call, a sleep call and so on.
These blocking calls are harmful because under the hood, asyncio uses a single-threaded event loop where coroutines are run concurrently. So if a blocking call is made in a coroutine, it blocks the entire event loop and all the coroutines. This impacts the overall performance of the application.

Here is an example where a blocking call prevents concurrent execution of the code:

import asyncio
import datetime
import time

async def example(name):
    print(f"{datetime.datetime.now()}: {name} start")
    time.sleep(1) # time.sleep is a blocking function
    print(f"{datetime.datetime.now()}: {name} stop")

async def main():
    await asyncio.gather(example("1"), example("2"))

asyncio.run(main())

When run, this outputs something like:

2025-01-07 18:50:15.327677: 1 start
2025-01-07 18:50:16.328330: 1 stop
2025-01-07 18:50:16.328404: 2 start
2025-01-07 18:50:17.333159: 2 stop

We see that the 2 coroutines were not run concurrently.

To overcome this you need to use a non-blocking equivalent or defer the execution to a thread-pool:

import asyncio
import datetime
import time

async def example(name):
    print(f"{datetime.datetime.now()}: {name} start")
    await asyncio.sleep(1) # replaced the blocking time.sleep call by the non-blocking asyncio.sleep coroutine
    print(f"{datetime.datetime.now()}: {name} stop")

async def main():
    await asyncio.gather(example("1"), example("2"))

asyncio.run(main())

When run, this outputs something like:

2025-01-07 18:53:53.579738: 1 start
2025-01-07 18:53:53.579797: 2 start
2025-01-07 18:53:54.580463: 1 stop
2025-01-07 18:53:54.580572: 2 stop

Here the 2 coroutines were run concurrently.

Now the problem is that it is not always easy to identify whether a method is blocking or not. Especially if the code base is big or uses third-party libraries. Sometimes blocking calls are made in deeply buried parts of the code.

For instance, is this code blocking ?

import blockbuster
from importlib.metadata import version

async def get_version():
    return version("blockbuster")

Did Python load package metadata in memory at startup? Is it done when loading the blockbuster module? Or when we call version()? Is the result cached and subsequent calls will be non-blocking?
The correct answer is that it is done when calling version() and it involves reading the METADATA file of the installed package. And the result is not cached. So version() is a blocking call and should always be deferred to a thread. This fact is hard to know without diving into the code of importlib.

One way of detecting blocking calls is to activate asyncio's debug mode to log blocking calls that take too long. But that's not the most efficient way as a lot of shorter than the trigger timeout blockings are still harmful for the performance and blocking times in test/development may be different than in production. For instance a database call may take longer in production if it has to fetch a lot of data.

This is where BlockBuster saves the day!
When activated, BlockBuster will monkey-patch several blocking Python framework methods to make them raise an error if they are called from an asyncio event loop.
The default patched methods include methods for os, io, time, socket, sqlite modules. For a full list of methods detected by BlockBuster, see the project README.
Then you can activate BlockBuster during your unit tests or in development mode to catch any blocking calls and fix them.
If you know the awesome BlockHound library in the JVM, it's the same principle but for Python. BlockHound was a great inspiration for BlockBuster, kudos to the creators.

Let's see how to use BlockBuster on the above snippet of blocking code.

First, we need to install the blockbuster package

pip install blockbuster

Then we can use a pytest fixture and the blockbuster_ctx() method to activate BlockBuster at the beginning of each test and deactivate it during teardown.

import asyncio
import datetime
import pytest
import time
from blockbuster import blockbuster_ctx

async def example(name):
    print(f"{datetime.datetime.now()}: {name} start")
    time.sleep(1)
    print(f"{datetime.datetime.now()}: {name} stop")

async def main():
    await asyncio.gather(example("1"), example("2"))

@pytest.fixture(autouse=True)
def blockbuster():
    with blockbuster_ctx() as bb:
        yield bb

async def test_main():
    await main()

If you run this with pytest you get

FAILED test_example.py::test_main - blockbuster.blockbuster.BlockingError: Blocking call to sleep (<module 'time' (built-in)>)

Note: typically, in a real project the blockbuster() fixture would be set in a conftest.py file.

Conclusion

I believe BlockBuster is quite useful in asyncio projects. It has already helped me detect a lot of blocking call issues on projects I work on.
But it's not a silver bullet. In particular, some third-party libraries don't use Python framework methods to interact with the network or the file system and instead wrap a C library. For these, it is possible to add rules to trigger on blocking calls of these libraries in your test setup. Also BlockBuster is open-source: contributions are very welcome to add rules for your favourite library in the core project.
And if you see issues and things that could be improved, I will be pleased to get your feedback in the project issue tracker.

Some links:

OHM, the mediatype for REST/HATEOAS powered by OpenAPI

Christophe Bornet — Tue, 10 Nov 2020 15:42:20 +0000

Today I'd like to present the draft of the OHM format, a new media type for Hypermedia/REST level 3 applications that relies on OpenAPI to describe the Hypermedia controls.

Description of the OHM format

TL;DR : the format is called OHM for Openapi-HyperMedia. Its mediatype is application/ohm+json and the definition is :

{
  "content": <JSON representation of the resource>,
  "controls": <An OpenAPI Specification (OAS) in JSON format containing the possible operations from this state>
}

... and that's it. Have you already seen a format definition as small as this one ?

For instance a OHM message for a Person resource which has a collection of Address could look like this:

{
  "content": {
    "firstName": "Jane",
    "lastName": "Doe"
  },
  "controls": {
    "openapi": "3.0.1",
    "info": {
      "title": "rest",
      "version": "0.0.1"
    },
    "paths": {
      "/api/addresses/1": {
        "get": {
          "summary": "John Doe's address"
        }
      }
    }
  }
}

The content field contains the information describing a Person called Jane Doe in the usual application/json media type. And the controls field is an OAS with a link to Jane Doe's address. An OHM client can then be used to navigate from the Person to its address without prior knowledge of the path, method, arguments, etc... that must be used.

It's also possible to use OHM to describe write operations:

GET /api/orders/106

{
  "content": {
    "id": 106,
    "product": "Licensed Concrete Keyboard",
    "cost": 571.0,
    "customer": {
      "id": 1,
      "name": "Ricky Swift"
    }
  },
  "controls": {
    "openapi": "3.0.1",
    "paths": {
      "/api/orders/106": {
        "put": {
          "summary": "Update order 106",
          "requestBody": {
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Order"
                }
              }
            }
          }
        },
        "delete": {
          "summary": "Delete order 106"
        }
      },
      "/api/customers/1": {
        "get": {
          "summary": "Get this order's customer (id=1)"
        }
      }
    },
    "components": {
      "schemas": {
        "Order": {
          "type": "object",
          "properties": {
            "cost": {
              "type": "number",
              "example": 571.0
            },
            "customer": {
              "type": "object",
              "properties": {
                "id": {
                  "type": "integer",
                  "example": 1
                }
              }
            }
          }
        }
      }
    }
  }
}

This is a classical CRUD resource where a PUT operation can be used to update the resource or a DELETE one to remove it. We can note that OpenAPI helps to fully describe the expected request body of the PUT operation and its media type.

The rest of this post will explain how to use this format to build better APIs truly embracing REST concepts with HATEOAS.

OHM is a media type for REST

First, what is a true REST application (also called REST level 3)? If you already know, you can skip this whole chapter. But I talk with a lot of people who say they do REST APIs but don't realize that what they do is actually good old RPC (Remote Procedure Call) over HTTP.

Quoting Roy Fielding, the inventor of REST:

"What needs to be done to make the REST architectural style clear on the notion that hypertext is a constraint? In other words, if the engine of application state (and hence the API) is not being driven by hypertext, then it cannot be RESTful and cannot be a REST API."

A key constraint of REST is thus to use Hypermedia As The Engine Of Application State (HATEOAS). It was at the core of the REST dissertation and even in the name "REpresentational State Transfer" and that's what makes this architectural style so unique. HATEOAS means that messages received from the server contain all the necessary information to drive the application. That's exactly how navigation works with HTML documents : you go from one state to the other using links and forms. So REST is used everyday when you go on the Web and navigate HTML documents with your browser. But never when you call a Web service through an HTTP+JSON API...

OHM provides HATEOAS support through the controls field. If we compare with HTML, it's the equivalent of anchor links (GET) and forms (POST). But as it relies on OpenAPI, it can use all the HTTP methods. The content field is the equivalent of all the divs and CSS that contain the representation of a resource. In a sense, OHM is a format oriented towards machine (easy to parse) whereas HTML is oriented towards humans (with graphical information).

When to use a REST approach with the OHM format ?

At the time of Roy Fielding's dissertation, the state of Web applications was fully residing in the server.
But with the advent of JavaScript in the browser and the mobile revolution, the state machine of the application has now moved inside the various clients and there's no need anymore to transfer the state transitions from the server. So we just expose Web services from the server and use them with RPC HTTP calls that we describe with OpenAPI.

With a REST format such as OHM, you can put the state management of the application back into the server. This gives the possibility to share parts or the whole application logic between the different types of client (browser, mobile, ...). And you can evolve the application logic without having to update the client which is a major gain when working with native mobile apps.

The advantage compared to HTML is that the view is still built by the client and so it can be used with non-HTML clients such as native mobile ones.

There's also the case where we don't expose an application but a Web service. In that case the application stays in a client that we don't control and REST has maybe small advantage here. It provides a browsable API for the consumers so they can better understand the natural chaining of API calls but you could see that as a too small benefit compared to the additional work needed to add Hypermedia links.
Except if you use the same endpoints for both your applications and your services. In that case OHM will provide HATEOAS for the application and your clients can still get the service content in the content identical as if they where using the application/json mediatype.

Another advantage of using REST and OHM for Web services is that you can provide an endpoint that lists all the possible operations that are available for clients. A bit like if you provided a single HTML page with all the links and forms that exist on your Web site. With a shared semantic between the client and the API (for instance using the OpenAPI field operationId), you can create clients that build the URLs and requests dynamically and will adapt even if you change the URLs or the parameter disposition on the server. Of course, if non-REST clients use the API, it will still be needed to version the API and ensure backward compatibility for these clients.

It's not the primary intent of OHM to provide full automation and interoperability between systems. Although this can certainly be built by adding vocabularies on top with things like JSON-LD. I'd be very interested to get opinions and ideas on this.

The power of the OHM browser

One big advantage of a REST format such as OHM is that you can build generic clients that understand any application that support the media type. For instance, for OHM you can use the OHM Browser which is a modified swagger-ui that navigates the links when you execute instead of displaying the HTTP call response (the sample app it points to by default is hosted on Heroku free tier, so it can take a little time to wake up. Please be patient 😅)

You can test it with a sample OHM application built with JHipster that represents customers that have orders that you can create, read, update, delete with navigation links and that also supports pagination. The source code for this application is located here.

How does OHM compare to other REST formats?

A lot of HATEOAS formats only support navigation links (GET) and not actions that modify the remote system (POST, PUT, ...). As it relies on OpenAPI, OHM supports all the common HTTP methods. It also fully describes the operation links, their call parameters and can embed documentation on how to use them in the messages.

OHM supports all the H Factors described by Mike Amundsen.

You don't need to learn a new format if you already know OpenAPI.

There's a lot of Open-Source tools in the OpenAPI landscape that can be reused to support OHM. For instance the OHM-browser is based on swagger-ui, the sample app uses the OAS generated by Springfox, ...

It's relatively easy to transition an HTTP+JSON API to OHM by putting the former JSON response in the content field and adding the links progressively.

Conclusion

I truly believe that REST can bring a lot to web applications. Hopefully, OHM makes it easy to adopt if you're familiar with OpenAPI. Don't hesitate to give it a try and show what you build with it !
The next steps for me will be to grow the eco-system starting by an helper Java library (which just needs to be extracted from the sample application code). I also think it should be possible to integrate with Spring-Hateoas so I'll have a look into that. Don't hesitate to reach me if you have a language of heart on which you would want to have a helper library. Oh, and I won't blame you if you continue to call RPC HTTP APIs as REST, everybody does it anyway 😉 !

You can follow me on Twitter.

OHM links:

The OHM specification: https://github.com/cbornet/ohm
The OHM browser: https://raw.githack.com/cbornet/swagger-ui/ohm/dist/
A sample OHM application: https://rest-openapi-demo.herokuapp.com/api (https://github.com/cbornet/sample-rest-app)