3 Ways to Implement Network Requests in Pyodide

Question

I mentioned an Online Python Tool before, and the core technology is to use a tool called Pyodide library, which can make Python run on web pages, but I found in the learning process that not all Python built-in libraries or extension libraries can run, such as requests is not supported.

According to the discussion under this issue, requests relies on Lib/http.client.py which relies on Lib/sockets.py which relies on socketmodule.c which needs <sys/socket.h>. Emscripten provides <sys/socket.h> support but if we used this, then the http requests would only work if we used them with a custom server that accepts web socket connections (or proxied all requests through such a server that could forward the requests as true http requests). So that wouldn't be ideal.

So if your local Python program uses the requests method, when switching to our online Python tool to run, you need to update the usage of network requests. Here we briefly introduce the three methods of network request.

Solution

1. http.open_url

Fetches a given URL synchronously.

import pyodide
print(pyodide.http.open_url('https://mocki.io/v1/d4867d8b-b5d5-4a48-a4ab-79131b5809b8').read())

Online Demo: Python Playground - http.open_url

2. http.pyfetch

Fetch the url and return the response.

import pyodide

async def get_data():
    response = await pyodide.http.pyfetch('https://mocki.io/v1/d4867d8b-b5d5-4a48-a4ab-79131b5809b8')
    data = await response.json()
    print(data)

get_data()

Online Demo: Python Playground - http.pyfetch

This method is recommended for obtaining image data

# request image
response = await pyodide.http.pyfetch('https://gcore.jsdelivr.net/gh/openHacking/static-files@main/img/16576149784751657614977527.png')
# Return the response body as a bytes object
image_data = await response.bytes()

Detailed reference case: Python Image to ASCII Online

3. fetch in js module

Pyodide wraps the js API, and uses the native js fetch API to implement network requests

import json
from js import fetch, JSON

async def get_data():
    response = await fetch('https://mocki.io/v1/d4867d8b-b5d5-4a48-a4ab-79131b5809b8',{'method':'GET'})
    data = await response.json()
    print(JSON.stringify(data))

await get_data()

Online Demo: Python Playground - js fetch

Conclusion

The above are the three network request methods commonly used in Pyodide that I have summarized. There may be many shortcomings. If you have a better method, please share it.

Reference

• 3 Ways to Implement Network Requests in Pyodide
• pyodide python api
• pyodide.http.open_url
• pyodide.http.pyfetch
• pyodide.runPythonAsync
• pyodide - requests module is not available #398