Introduction
Recently, I had an idea to add my own AI Assistant to my website - www.akshaymakes.com. I wanted it to answer questions about my projects and myself and talk a lot about machine learning and artificial intelligence in general. My website is built on SvelteKit and I could have chosen to use OpenAIβs API directly from the front end. But, I wanted the assistant to be scalable. That is, in the future, I wanted it to be able to browse through my blogs, projects, and other material on my website and answer questions much better. So for this purpose and to keep the website simple, I created a LangChain application using FastAPI which would integrate with my website with REST API. I wanted the experience to be similar to ChatGPT i.e. it should be able to remember the context of the conversation and continue the conversation naturally. I deployed the application on Deta Space which was quite simple to do. Let me take you step by step through the process. We will keep the application simple right now. In future blogs, I will explain how you can add your website as context using Vector Databases like Weaviate or Pinecone to make the chat assistant more knowledgeable about you.
So in this tutorial I will show you how to create an API for getting OpenAI outputs using LangChain, FastAPI and Deta Space. Letβs begin.
Setting Up
Start with a new Python project in a new directory. In our example, let us call the project directory as
LangChainAPI.
Create a directory in
LangChainAPI
calledapp
and new file.env
Inside the app folder, create an empty
__init__.py
file and a newmain.py
andconversation.py
file.-
In this new directory, initiate a virtual environment with the following terminal command.
python -m venv venv
-
This is how the project structure will look like
βββ appβ β βββ __init__.py β βββ main.py β βββ conversation.py βββ .venv| βββ .gitignore βββ .env
-
Activate the environment.
For Windows
venv\Scripts\activate.bat
For MacOS/Linux
source venv/bin/activate
-
Install the dependencies.
pip install langchain fastapi "uvicorn[standard]" openai python-dotenv
-
Install Deta Space CLI
For Windows
iwr <https://deta.space/assets/space-cli.ps1> -useb | iex
For MacOS/Linux
curl -fsSL get.deta.dev/space-cli.sh | sh
-
Initiate Git Repository and commit
git init git add . git commit -m "First Commit"
-
Create an account on https://deta.space/signup and get your access token from the settings.
-
Login to Deta Space in the CLI. It will ask for the access token. Paste it.
space login
That is all for the setup. Now let us create the API.
Creating the API
In the app folder open conversation.py
. This is where we will write the logic for LangChain.
from langchain import OpenAI, ConversationChain, LLMChain, PromptTemplate
load_dotenv()
def conversation(human_input):
template = """Assistant is a large language model trained by OpenAI.
Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.
Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.
Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.
{history}
Human: {human_input}
Assistant:"""
prompt = PromptTemplate(
input_variables=["history", "human_input"],
template=template
)
chatgpt_chain = LLMChain(
llm=OpenAI(temperature=0),
prompt=prompt,
verbose=True,
)
output = chatgpt_chain.predict(human_input=human_input)
return output
In the main.py
file.
from fastapi import FastAPI
from langcorn import create_service
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from app.conversation import conversation
class Input(BaseModel):
human_input: str
class Output(BaseModel):
output: str
app=FastAPI()
@app.post("/conversation")
async def input(input: Input):
output = Output(output=conversation(input.human_input))
return output
origins = [
"<http://localhost>",
"<http://localhost:5173>",
"...Your Domains..."
]
app.add_middleware(
CORSMiddleware,
allow_origins=origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
In the origins
you can add other domains that will be making requests to your api.
Run the API Server Locally
In the terminal use this command to start the terminal locally. It will start on localhost:8000
.
uvicorn app.main:app --reload
To test your API, go to localhost:8000/docs
in your browser. This should open Swagger Docs.
You can input your prompt and check if you are getting the response. Once this works, feel free to play around with LangChain. You can add memory, context etc. At this moment, Deta Space does not support local Vector Databases. So you will have to use remote Vector Databases if you need to save your context files and embeddings.
Deploy to Deta Space
Once you are happy with the API, commit the changes to git
git add .
git commit -m "API Works"
Initiate Deta Space:
space new
This will create a new SpaceFile
in the project. Open this file and make and overwrite this to it.
# Spacefile Docs: <https://go.deta.dev/docs/spacefile/v0>
v: 0
micros:
- name: LangChainAPI
src: ./
engine: python3.9
primary: true
run: uvicorn app.main:app
presets:
env:
- name: OPENAI_API_KEY
description: Secret message only available to this Micro
default: "OpenAPI Key"
api_keys: true
Save the file and run this command in the terminal.
space push
This will create an instance of your API on the Deta Space Dashboard. In this case, it is named βgpt_serverβ. In your case, it will be βLangChainAPIβ.
Go to the instance settings and add your OpenAI API Key from the βConfigurationsβ tab. Then, go to the domains tab and get the base URL of your API. You can test it out in the browser first using the Swagger Docs and then use it in your application as REST API.
Want to connect?
πMy Website
π¦My Twitter
π¨My LinkedIn
Top comments (4)
the deta cli doesn't install - I've never heard of them - does this work with vercel?
Hey there. I hope @wh Chan has answered your problem with the deta cli. Regarding vercel.... Vercel has a much lower response time before the serverless function times out. Its 20 seconds on Vercel. So for longer chat responses, you will timeout. You need to limit your response tokens. I havent faced this issue with deta yet.
if you are using MacOS, the cmd is wrong, it should be
curl -fsSL get.deta.dev/space-cli.sh | sh
deta.space/docs/en/build/fundament...
Thanks a lot for correcting the mistake. I will update it.