Pickling Python in the Cloud via WebAssembly

I have just finished publishing an article, "Leveraging Python Standard Library via WebAssembly" and no sooner am I struck with the immediate urge of Pickling Python in the Cloud via WebAssembly (Wasm).

Writing Wasm-powered serverless applications in Python is irresistible. I love Python, and I love Wasm, so I guess that makes sense.

With the latest version of Spin I can use the spin new command to scaffold out a new HTTP Python application in seconds:

Creating an Application

$ spin new -t http-py
Enter a name for your new application: pickling-python
Description: Pickling Python
HTTP path: /...

From there if I change into the application's folder and open the application manifest file (spin.toml) for editing:

$ cd pickling-python
$ vi spin.toml 

I see the following (a pretty straight forward .toml file):

spin_manifest_version = 2

[application]
authors = ["tpmccallum <[e-mail]>"]
description = "Pickling Python"
name = "pickling-python"
version = "0.1.0"

[[trigger.http]]
route = "/..."
component = "pickling-python"

[component.pickling-python]
source = "app.wasm"
[component.pickling-python.build]
command = "spin py2wasm app -o app.wasm"
watch = ["app.py", "Pipfile"]

Adding Redis Storage

I want all this processing and pickling in the cloud (e.g., once built and deployed, no processing or storage will be performed locally). For this to become a reality, I go to my Redis Cloud account and copy the credentials of an existing database, e.g. username, password, URL and port.

I then add the values in the environment and allowed_outbound_hosts configurations at the component.pickling-python "component" level (as shown below):

[component.pickling-python]
environment = { REDIS_ADDRESS = "redis://username:[e-mail]:16978" }
allowed_outbound_hosts = ["redis://my-redis-cloud.redislabs.com:16978"]

There is granting network permissions to components and Python component configuration documentation available if you need it. But you should be ok just mirroring what I am doing here (using your credentials, of course).

The Python

The app.py file in the application directory holds all the Python source code needed. You can open your app.py and replace it with the following code (if you are following along):

import os
import json
import pickle
from spin_http import Response
from spin_redis import redis_set, redis_get

# Define a simple example class
class ExampleClass:
    def __init__(self, attribute1, attribute2):
        self.attribute1 = attribute1
        self.attribute2 = attribute2

def handle_request(request):
    # Obtain the Redis address
    redis_address = os.environ['REDIS_ADDRESS']
    if request.method == 'POST':
        # Read the body of the request
        json_str = request.body.decode('utf-8')
        json_object = json.loads(json_str)
        json_object["attribute1"]
        json_object["attribute2"]
        # Create an instance of our ExampleClass using the values from the incoming request
        example_instance = ExampleClass(json_object["attribute1"], json_object["attribute2"])
        # Serialize the instance of our ExampleClass and set the  into Redis as a value
        redis_set(redis_address, "example_instance", pickle.dumps(example_instance))
        # Fetch the previously stored value from Redis and deserialize the value so we can work with an instance of our class
        fetched_instance = pickle.loads(redis_get(redis_address, "example_instance"))
        answer = {"Attribute 1": fetched_instance.attribute1, "Attribute 2": fetched_instance.attribute2}
    return Response(200,
                    {"content-type": "text/plain"},
                    bytes(json.dumps(answer),"utf-8"))

I will pause a moment and explain what this Python code is doing...

The code above:

• receives a JSON object as part of an incoming request,
• instantiates our ExampleClass using the JSON object's data,
• uses pickle.dumps to serialize the instance of the ExampleClass,
• stores the instance as bytes in Redis (with the attribute1 and attribute2 values intact),
• retrieves the bytes back from Redis and deserializes them into the form of our original ExampleClass instance (with the attribute1 and attribute2 values intact),
• creates a Response object,
• returns a JSON string that describes the correct state of the ExampleClass.

The build and deploy part is super easy, we just run the following 2 commands:

$ spin build
$ spin deploy

Once deployed, a secure HTTP request from a client can be made. Here is an example of a client's request using the curl command:

$ curl -X POST "https://dev-to-example-zwirrxzu.fermyon.app/" -H "Content-Type: application/json" -d '{"attribute1": 123, "attribute2": 456}'

And voilà, we get the correct response (as follows):

{
    "Attribute 1": 123,
    "Attribute 2": 456
}

How did this go for you? You can contact us on Discord if you get stuck or have any questions.

What's next

In my experience so far, I can use a vast amount of the Python Standard Library to build Wasm-powered serverless applications. The caveat I currently understand is that Python’s implementation of TCP and UDP sockets, as well as Python libraries that use threads, processes, and signal handling behind the scenes, will not compile to Wasm. It is worth noting that a similar caveat exists with libraries that I find on The Python Package Index (PyPI) site. While these caveats might limit what can be compiled to Wasm, there are still a ton of extremely powerful libraries to leverage.

My takeaway is that it is now so easy and quick to develop very powerful serverless web applications using a combination of Python, Spin and Fermyon Cloud.

What combinations of Python can you think of to launch applications for the web? Perhaps you can develop in tandem with a front-end web developer and create dynamic functionality using a combination of HTML/CSS/Javascript on the client side.

I hope this inspires you to create something great!

References:

• Leveraging Python Standard Library via WebAssembly
• YouTube - Leveraging Python Standard Library via WebAssembly
• Fermyon Developer Home
• Python logo image attribution Dnu72, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons