Tuning your Python applications with Cython

Introduction

Python is a high-level interpreted and multi-purpose language created in 1991. The standard and most common version of its implementation was written in the C language and is known as CPython.

Despite being widely used for various purposes, Python, being an interpreted language, has a slower execution speed compared to its compiled counterparts like Rust, Golang, C, and others. However, it is possible to combine the ease of development in Python with the execution speed of its parent language C by using the Cython module.

Cython is an extended version of the Python language that utilizes its static compiler (which compiles only once, generating executable files) to convert Cython code into C. Through this process, the code originally written in Python can be compiled into C, drastically increasing its execution speed.

Hint: There are several implementations of Python, including versions in Java (Jython), C# (IronPython), and Python itself (PyPy).

Why use Cython?

• Widely used and mature
• Execution speed comparable to C
• You can directly call functions written in C or C++
• Enables you to create your own libraries such as Numpy or Pandas in an efficient and safe way.

Here we can see the difference between the syntax of a Python code and a Cython code.

Python

import math

def calc(start: int, end: int) -> None:
    p: int = start
    factor: int = 1000*1000

    while p < end:
        p +=1
        math.sqrt((p - factor) * (p - fator))

Cython

import cython
from libc.math cimport sqrt

def calc(start: cython.int, end: cython.int):
    p: cython.int = start
    factor: cython.in = 1000*1000

    while p < end:
        pos +=1
        sqrt((p - factor) * (p - factor))

We can see that the main difference in the code is due to the data typing and the declarations of functions in C. This process enables the compilation of our file.

Implementation

Creating the project

mkdir python_to_cython
cd python_to_cython
python3 -m venv venv
source venv/bin/activate

Installing the module

pip install cython

With this, we can begin our implementation. We will create a function to calculate the factorial of a number. As we know, a standard factorial calculation function has a Big O notation of 'n', so we will be able to see a clear difference in execution time.

Hint: You can use Cython to create compilations of various different files beyond simple functions. This includes API calls in views of Django, Flask, or FastAPI, asynchronous functions, ORM calls like Django-ORM or SQLAlchemy, and more.

example.py

def factorial(n) -> int:
    result = 1
    for i in range(1, n+1):
        result *= i
    return result

After creating our function, we will create a new file with the *.pyx extension that contains our code to be converted to C. In this file, we can use the same code from our example.py file.

example.pyx

def factorial(n) -> int:
    result = 1
    for i in range(1, n+1):
        result *= i
    return result

Next, we will create our setup.py file. In this file, instructions will be defined for importing Python modules, including information on how they should be built and installed. When used in conjunction with Cython, the script in setup.py takes information about the Cython files, which are converted into C files and subsequently transformed into language extensions.

setup.py

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize(['example.pyx'])
)

After executing this file with the command python3 setup.py, some files will be generated, and our structure will look like this:

root
├── build
│ └── temp
│ └── example.so
├── venv
├── example.c
├── example.cython
├── example.py
├── example.pyx
└── setup.py

With this, we will have our executable file compiled in C, and we can proceed to perform a performance test.

We will create a new file called test.py and import both our original function and our compiled function. However, before doing this, we will rename our example.py file to avoid import issues, by using the command:

mv example.py example_python.py

Now we will import both modules into our test file and set up a simple counter to measure the execution time of the functions using the time module.

test.py

import time
import example
import example_python

N = 100000 #200000, 300000, 400000, 500000
start_time = time.time()
perms = example.factorial(N)
end_time = time.time()
print("Execution time:", end_time - start_time, "seconds")


start_time = time.time()
example_python.factorial(N)
end_time = time.time()
print("Execution time:", end_time - start_time, "seconds")

To complete our test, we just need to execute it to see the difference in execution speed between the two functions. For a better visualization, we will run the test five times with different values of N. Below are the results:

Through a simple metric of variance, we can see that the variance of the code executed in Cython (2988.98 seconds²) is notably lower than the variance of the execution in Python (7636.24 seconds²), which directly reflects on the stability of the response times of the calls.

As we can see, we can increase the execution speed of our application by using Cython. This allows us to compile sections of our code originally written in Python, which are bottlenecks in the flow, into Cython, thus optimizing our application in a simple and efficient manner.

Hint: Next time you create your functions, use Cython instead of using them in Python.

You can access the repository of this project by clicking on this link

Click here plzzz