DEV Community: Ciro Spaciari

Adding better DX to the fastest Python WebFramework

Ciro Spaciari — Mon, 09 Jan 2023 10:30:11 +0000

After more than a year of active development, I finally publish my package to PyPI!

pip install socketify

After some people asks to also create a WSGI and ASGI server to improve projects using other frameworks, I worked hard to get it right.

But in the end, we outperform uvicorn and meinheld with Flask, Django, Emmett, FastAPI, and Falcon in TechEmPower plaintext with our WSGI and ASGI servers!

And we got the Top 1 with socketify itself!

https://www.techempower.com/benchmarks/#section=test&runid=7ce481b2-49ec-4a4d-952d-bb1334d4a4ad&test=plaintext&l=hra0hr-35r

Also added a CLI tool with very familiar options for uvicorn and gunicorn users

CLI: https://docs.socketify.dev/cli.html

Also added lifespan (start and shutdown) events to socketify itself, and added an improved DX for routing and support to Extensions/Plugins.

# main.py
from socketify import App

def run(app: App):
    @app.on_start
    async def on_start():
        # we can start connection pools here
        pass

    @app.on_shutdown
    async def on_shutdown():
        # and here we can clean connection pools
        pass

    router = app.router()
    api = app.router(prefix="/api")
    private = app.router("/api", auth_middleware)

    @router.get("/")
    def home(res, req):
        res.send("Hello, World!")

    # will serve in /api/hello
    @api.get("/hello") 
    def hello(res, req):
       res.send("Hello API!")

    # will serve in /api/users and use auth_middleware
    @private.get("/users")
    def get_users(res, req, auth):
       res.send("Hello private API!")

 python3 -m socketify main:run --port 8080 --workers 2

About Extensions: https://docs.socketify.dev/extensions.html

My focus until v0.1.0 now will be to add features to help developers and add more optimizations.

Are there any features that you want to see in the future? I will love to get community feedback!

Project: https://github.com/cirospaciari/socketify.py

Issues and Planned Features: https://github.com/cirospaciari/socketify.py/issues

Releases: https://github.com/cirospaciari/socketify.py/releases

The Status of Python Web Performance at the end of 2022

Ciro Spaciari — Mon, 14 Nov 2022 13:14:57 +0000

Let's talk about the current state of python web framework throughput performance, using the very popular TechEmPower as reference, looking only to PlainText to check only the framework throughput not the DB library or JSON library performance, we can see that for a few years japronto dominates the top.

Round 21:

Check the results here: https://www.techempower.com/benchmarks/#section=data-r21&test=plaintext&l=zijzen-6bj

The state of japronto

The source code from the project resides in the github, with more than 8.6k stars and 596 forks is a very popular github, but no new releases are made since 2018, looks pure much not maintained anymore, no PR's are accepted no Issues are closed, still without windows or macOS Silicon, or PyPy3 support. Japronto it self uses uvloop with more than 9k stars and 521 forks and different from japronto is seems to be well maintained.

So how the only Python framework hiting more than 1 million, actually more than 2.6 million of requests per second, more than 4.7 times than the second place falcon with only 564k requests per second, is an obsolete project with no secure updates since 2018 and still keeped by TechEmPower in the charts? I don't know but japronto github says that is an early preview and recomend the use of sanic with hits 277k requests per second.

Sanic is very very popular with 16.6k stars, 1.5k forks, opencollective sponsors and a very active github.
Falcon is more popular than japronto with 8.9k stars, 898 forks, opencollective sponsors and a very active github too.
Despite Japronto been keeped as first place by TechEmPower, Falcon is a way better solution in general with performance similar to fastify an very fast node.js framework that hits 575k requests per second in this benchmark.

So... Is still hope for hitting 1 million requests or more with Python?

After year of japronto dominance as the firsy place of TechEmPower PlaintText for Python,
a new library called vibora appears... and it's another project without any updates since 2019. Ok let's filter vibora and japronto and go for the next one that is active, a new library called socketify.py and also another new library called robyn arrives on the millions req/s in the live results.

Check the results here: https://www.techempower.com/benchmarks/#section=test&runid=43469a25-670e-4bda-a3c3-652839435784&test=plaintext&l=hra0hr-35n&w=18y6o-0-0-0&f=0-0-0-0-0-w-0-0-0-0-0-8vn08w-0-0

With socketify.py hiting more than 2 million of requests with Python or PyPy3, and robyn with more than 1 million the hope of an stable and well maintained library that can hit in the millions is born again.

But how it compare in others benchmarks?

I pick uvicorn, falcon and robyn for comparison using oha using 40 concurrent requests for 5s, running 1 time for warmup and getting an average of 3 runs, just sending back an "Hello, World!" message.



oha -c 40 -z 5s http://localhost:8000/

HTTP requests per second (Linux x64)

It also show that PyPy3 will not magically boost your performance, you need to integrate in a manner that PyPy3 can optimize and delivery CPU performance, with a more complex example maybe it can help more. But why socketify is so much faster using PyPy3? The answer is CFFI, socketify did not use Cython for integration and cannot delivery the full performance on Python3, this will be solved with HPy.

Also we can see that the Cython + meinheld (that is builded with C) are WAY faster than other options of falcon, but you will not get an optimized JIT for all python code that you write in a real life project.

HTTP pipelining is buggy and not really supported by browsers, the technique was superseded by multiplexing via HTTP/2, which is supported by most modern browsers, TechEmPower don't test HTTP/2 or HTTP/3.

In HTTP/3, multiplexing is accomplished via QUIC which replaces TCP. This further reduces loading time, as there is no head-of-line blocking even if some packets are lost.

Supporting HTTP/2 and HTTP/3 is way more important than winning TechEmPower HTTP/1.1 benchmarks, this only means you need to take any benchmark with a grain of salt, and run bencharks in an more close to reality scenario by our self, Caching strategy, Bandwidth, Latency, CPU time, Memory, all are relevant and not really tested here.

WebSocket Messages per second (Linux x64)

Socketify.py got almost 900k messages/s with PyPy3 and 860k with Python3 the same performance as Bun,
with Falcon 35k messages/s and with Falcon with PyPy3 56k messages/s, node.js manages 192k.

Bun is REALLY fast! But why the performance of Bun is almost identicall to socketify.py? The answer is that it also uses uWebSockets as base, actually the same code of the PR i made to the C API is been used in Bun with some tweaks, and it endup with almost the same performance that socketify.py, showing that both have some DNA in common.

I will do a more deep look and benchmarks in future, but for now it's it.

This library still very young with less than 100 stars on github, still a very young library that gives commercial support and are looking for sponsors, It's built on the shoulders of giants (very well tested, maintained and supported giants), bringing HTTP, HTTP/2 and WebSockets support, with experimental HTTP/3 in the roadmap, is already a much more complete than japronto.

Runtime versions: PyPy3 7.3.9, Python 3.10.7, node v16.17.0, bun v0.2.2

Framework versions: gunicorn 20.1.0 + uvicorn 0.19.0, socketify alpha, gunicorn 20.1.0 + falcon 3.1.0, robyn 0.18.3

HTTP tested with oha -c 40 -z 5s http://localhost:8000/ (1 run for warmup and 3 runs average for testing)

WebSocket tested with Bun.sh bench chat-client check the source code in this link

Machine OS: Debian GNU/Linux bookworm/sid x86_64 Kernel: 6.0.0-2-amd64 CPU: Intel i7-7700HQ (8) @ 3.800GHz Memory: 32066MiB

Follow @cirospaciari on twitter for more updates about socketify.py and if you have some time check out the github https://github.com/cirospaciari/socketify.py this project can be even better with some community love ;)

This is how i got more performance in WebSockets than Node or Bun using Python

Ciro Spaciari — Fri, 11 Nov 2022 20:44:57 +0000

Bun is changing out the scenario for Node.js, with much more performance in HTTP and WebSockets, but how FAST it is compared with Python?

In the Bun website https://bun.sh/, the source code of the WebSocket benchmark is available, so... let's try to run Falcon + uvicorn on it!

Benchmark

I just simplified a little the benchmark, to remove a string concatenation, to measure more of the raw throughput with less impact other things, and runned using Falcon + uvicorn and socketify.py, when benchmarking i was talking with Vytautas Liuolia in the Falcon Gitter for tunning HTTP performance in other raw throughtput testing as well, and Falcon is not so much focused on WS performance, this also depends a lot of ASGI server too. For this test i choose uvicorn with gunicorn as ASGI server.

Messages per second with Linux x64:

Runtime versions: PyPy3 7.3.9, Python 3.10.7, node v16.17.0, bun v0.2.2

Framework versions: gunicorn 20.1.0 + uvicorn 0.19.0, socketify alpha, gunicorn 20.1.0 + falcon 3.1.0, robyn 0.18.3

WebSocket tested with Bun.sh bench chat-client check the source code in this link

Machine OS: Debian GNU/Linux bookworm/sid x86_64 Kernel: 6.0.0-2-amd64 CPU: Intel i7-7700HQ (8) @ 3.800GHz Memory: 32066MiB

Socketify.py got almost 900k messages/s with PyPy3 and 860k with Python3 the same performance as Bun,
with Falcon 35k messages/s and with Falcon with PyPy3 56k messages/s, node.js manages 192k.

Socketify.py - Maybe the fastest web framework for Python and PyPy

Ciro Spaciari — Tue, 08 Nov 2022 21:19:58 +0000

This year i started a journey to bring a really fast framework for Http, Http2, Http3 and WebSockets to Python and PyPy.

The problems with japronto and uvloop

Japronto don't get any github updates since 2020 and don't get any src update since 2018, japronto don't support SSL, WebSockets, PyPy3, Windows or macOS Silicon.
I really needed good support + performance, not only performance.
Almost any solution out there have a really big performance hit when using PyPy3 or are not really fast enougth

The Solution

We discover a really fast, small, and well maintained C++ Library called uNetworking/uWebSockets, but no C API available, so we create and adapt the full C API from uNetworking/uWebSockets and will integrate libuv powered fetch and file IO, this same C API is used by Bun

⚡ Benchmarking

HTTP requests per second (Linux x64)

WebSocket messages per second (Linux x64)

It's a really an over simplificated microbenchmark, but shows a big difference between TechEmPower benchmarks using wrk and + 16 pipelining with an lua script, and of course will not hit 1 million req/s on my machine, running tfb tools in my machine socketify.py + PyPy3 hits 770k req/s vs 582k from japronto and in WebSockets almost 900k messages/s with PyPy3 and 860k with Python3 the same performance as Bun,
with Falcon 35k messages/s and with Falcon with PyPy3 56k messages/s, node.js manages 192k, and still can get at least 30% or more performance once fully integrated with libuv + asyncio.

See more benchmark info here: https://github.com/cirospaciari/socketify.py#zap-benchmarks

💡 Features

WebSocket with pub/sub support
Fast and realiable Http/Https
Support for Windows, Linux and macOS Silicon & x64
Support for PyPy3 and CPython
Dynamic URL Routing with Wildcard & Parameter support
Sync and Async Function Support
Really Simple API
Fast and Encrypted TLS 1.3 quicker than most alternative servers can do even unencrypted, cleartext messaging
Per-SNI HttpRouter Support
Proxy Protocol v2
Shared or Dedicated Compression Support
Max Backpressure, Max Timeout, Max Payload and Idle Timeout Support
Automatic Ping / Pong Support
Per Socket Data

🔎 Upcoming Features

Fetch like API powered by libuv
Async file IO powered by libuv
Full asyncio integration with libuv
Full Http3 support
HPy integration to better support CPython, PyPy and GraalPython
Hot Reloading

We created and adapt the full C API from uNetworking/uWebSockets and will integrate libuv powered fetch and file IO, this same C API is used by Bun

📦 Installation

For macOS x64 & Silicon, Linux x64, Windows



pip install git+https://github.com/cirospaciari/socketify.py.git
#or specify PyPy3
pypy3 -m pip install git+https://github.com/cirospaciari/socketify.py.git
#or in editable mode
pypy3 -m pip install -e git+https://github.com/cirospaciari/socketify.py.git@main#egg=socketify

🤔 Usage

Hello world app



from socketify import App

app = App()
app.get("/", lambda res, req: res.end("Hello World socketify from Python!"))
app.listen(3000, lambda config: print("Listening on port http://localhost:%d now\n" % config.port))
app.run()

SSL version sample


 python
from socketify import App, AppOptions

app = App(AppOptions(key_file_name="./misc/key.pem", cert_file_name="./misc/cert.pem", passphrase="1234"))
app.get("/", lambda res, req: res.end("Hello World socketify from Python!"))
app.listen(3000, lambda config: print("Listening on port http://localhost:%d now\n" % config.port))
app.run()

WebSockets



from socketify import App, AppOptions, OpCode, CompressOptions

def ws_open(ws):
    print('A WebSocket got connected!')
    ws.send("Hello World!", OpCode.TEXT)

def ws_message(ws, message, opcode):
    #Ok is false if backpressure was built up, wait for drain
    ok = ws.send(message, opcode)

app = App()    
app.ws("/*", {
    'compression': CompressOptions.SHARED_COMPRESSOR,
    'max_payload_length': 16 * 1024 * 1024,
    'idle_timeout': 12,
    'open': ws_open,
    'message': ws_message,
    'drain': lambda ws: print('WebSocket backpressure: %i' % ws.get_buffered_amount()),
    'close': lambda ws, code, message: print('WebSocket closed')
})
app.any("/", lambda res,req: res.end("Nothing to see here!'"))
app.listen(3000, lambda config: print("Listening on port http://localhost:%d now\n" % (config.port)))
app.run()

And click here more than 20 examples

💼 Commercially supported

Socketify is an active project and will be maintained over time with security updates and new features and will be commercial support available

Check our github page: https://github.com/cirospaciari/socketify.py

Faster and simple Http and WebSocket Server for Ruby

Ciro Spaciari — Tue, 18 Jan 2022 18:04:06 +0000

uWebSockets for Ruby - uWebSockets.rb

I started to implement uWebSockets C API wrapper and a Ruby version a few days ago.
I got some good performance results (about 5x agoo in using techempower benchmarks for plaintext), for now i just implemented http/https apis but i will implement websockets this week and write some documentation (only linux support, mac should works fine as well, but i will add windows and mac tests and support too)

I will be glad for some feedback and help for testing in the near future https://github.com/cirospaciari/uWebSockets.rb

Overly simple hello world app

require "uws"

UWS::App.new()
.get("/", lambda {|response, request| response.end("Hello World uWS from Ruby!")})
.listen(8082, lambda {|config| puts "Listening on port #{config.port}" })
.run()

Gemfile

gem 'uws', git: 'https://github.com/cirospaciari/uWebSockets.rb.git', branch: 'main', submodules: true

Run

bundle exec ruby ./hello_world.rb

SSL version sample

require "uws"

UWS::SSLApp.new({
    key_file_name: "./misc/key.pem",
    cert_file_name: "./misc/cert.pem", 
    passphrase: "1234"
})
.get("/", lambda {|response, request| response.end("Hello World uWS from Ruby!")})
.listen(8082, lambda {|config| puts "Listening on port #{config.port}" })
.run()