DEV Community

Aivars Kalvāns
Aivars Kalvāns

Posted on • Originally published at aivarsk.com on

Running your pytests faster

Open your conftest.py and type the following lines:

import gc
# Like gc.disable() but overrides your dependencies that do
# gc.disable() and gc.enable()
gc.set_threshold(0)

Enter fullscreen mode Exit fullscreen mode

Depending on the project size and number of tests there is a speedup from a few percent in smaller projects up to 10% in a project with 30,000 tests. Your experience may vary.

I went down the Python GC rabbit hole and there will be a long story about it so here is a short one.

Most of the memory in Python is released when the reference count reaches zero. For a few cases when you might get cyclic references, there is a stop-the-world garbage collector. However, the trigger condition for starting the GC is when allocated but not released instances of non-atomic types reach a small threshold. That leads to a lot of GC cycles when you load something into memory for legitimate reasons: your Django QuerySet caching, file records, and collecting unit tests to run.

For my test case, 2nd generation gets more than 400,000 objects while pytest is running the tests. It takes some time to get there from the 0th generation and it just increases the n in the complexity of the collection algorithm.

If Instagram can run services without GC in production so can our tests.

AWS Security LIVE!

Join us for AWS Security LIVE!

Discover the future of cloud security. Tune in live for trends, tips, and solutions from AWS and AWS Partners.

Learn More

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay