In this blog post you will learn how to stream OpenAI API responses directly in the browser, without the need for a backend as a middle-man between OpenAI and the browser application and without leaking your OpenAI API key to the world.
Stream the response through Signway
Signway is an open source and managed solution for generating pre-signed URLs, like the ones AWS S3 uses for sharing objects, but in this case for proxying correctly signed requests to destination APIs, and stream the response back. Its core is open source, so you can also host it yourself https://github.com/gabotechs/signway.
For following this tutorial you will need
- Python +3.9 or greater installed in you system
- An OpenAI Api key
Create a Signway Application
- Go to Signway's web page and log into the console.
- Once you log in you will be prompted to create an Application.
- After giving a nice name to the Application, you will see a pop-up sowing the newly created Application ID and Secret. Copy them and store them in a safe place, we will need them later.
- Once the Application is created, go to "Custom Headers" and add the "Authorization" header that will be sent to OpenAI.
Code a function that generates pre-signed URLs for Signway in Python 🐍
We will be using Python for this, but other languages could be used for generating pre-signed URLs.
First, create a virtual environment and install the Signway SDK:
python3 -m venv venv
source venv/bin/activate
pip install signway-sdk
Then create a script that generates pre-signed URLs. We will call this script sign.py:
# sign.py
from signway_sdk import sign_url
import os
print(sign_url(
id=os.environ['SW_ID'],
secret=os.environ['SW_SECRET'],
host="https://api.signway.io",
proxy_url="https://api.openai.com/v1/completions",
expiry=900,
method="POST"
))
Export the application secrets saved from the earlier step:
export SW_ID=""
export SW_SECRET=""
Remember to populate this variables with the Application ID and Application Secret that the Signway page gave you.
Executing this script gives us a URL that looks like this:
$ python3 sign.py
https://api.signway.io/?X-Sw-Algorithm=SW1-HMAC-SHA256&X-Sw-Credential=app-id%2F20230619&X-Sw-Date=20230619T195222Z&X-Sw-Expires=300&X-Sw-Proxy=https%3A%2F%2Fapi.openai.com%2Fv1%2Fcompletions&X-Sw-SignedHeaders=&X-Sw-Body=false&X-Sw-Signature=4aa60955f477eeb2bd556c41ecce3508d02b5b83bec9c0c188737998f73c5176
That is a signed URL, it can be safely transferred to clients as it does not contain any sensible information. In this case, we have set an expiration time of 15 mins, after that time the URL will be useless.
Create the frontend code that will stream the response
For this we will use a very simple example without using any framework:
Remeber to execute
python3 sign.pyand copy-paste the generated URL in theconst URL = ""variable
<script>
// Execute 'python3 sign.py' and paste the result URL
const URL = ""
fetch(URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: "text-davinci-003",
stream: true,
prompt: "Say this is a test"
}),
}).then(async res => {
const p = document.querySelector('#response')
const reader = res.body.getReader()
const decoder = new TextDecoder()
while (true) {
const { done, value } = await reader.read()
p.innerText += decoder.decode(value)
if (done) break
}
})
</script>
<html>
<p id="response"/>
</html>
If you open this file in your browser you will see how the response from Open AI is streamed.
What's next?
- You can do more things with Signway, like providing extra security by signing more things in the request, like the body or some headers. There is a cool tutorial about this here
- The natural thing to do is to generate pre-signed URLs from a trusted source, like your backend, you can use the excellent FastAPI framework for that.
- Make sure to pass by the Github repo and give it a star if you like it.
- Feel free to play around the free tier of Signway Managed, but join the waitlist if you are willing to take it to the next level.
Top comments (0)