Deploy a Langchain agent as an API endpoint in 10 minutes

Github repository at the end of the article

So you coded a cool Langchain agent in Python. Congrats! In this tutorial, we will walk you through the process of making it an OpenAPI endpoint, which can be deployed and called as an API, allowing you to seamlessly integrate it into your product or workflows. All that in only 10 minutes. Let’s get started!

---

Installation and setup

You need to have Python 3.8 or higher installed.

(Optional) Create a new virtual environment

python3 -m venv venv
source venv/bin/activate

Install the required packages

• langchain for the orchestration of the agent
• openai for the OpenAI API
• phospho for the deployment of the agent
• python-dotenv for the environment variables (like api keys)

pip install --upgrade langchain openai phospho python-dotenv
pip freeze > requirements.txt

Store your OpenAI API key

In this example, we are going to use OpenAI as our LLM provider for our Langchain agent. But you can use any LLM provider you wish.

OPENAI_API_KEY=your-api-key

Create an .env file and store your OpenAI API key in it. For extra security, you can create a new OpenAI key for this project.

Build the agent logic

Create a new langchain agent

Create a main.py python file at the route of the project. You must name it main.py since phospho will look for this file to initialize the agent.

Let’s load the environment variables from the .env file :

# main.py

# Load environment variables

from dotenv import load_dotenv
load_dotenv()

Now, let’s create a basic chatbot using the OpenAI API. As you can imagine, the process would be roughly the same for more complex agents.

# main.py

# Load environment variables

from dotenv import load_dotenv
load_dotenv()

# use langchain to easily interact with the OpenAI API

from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.chat_models import ChatOpenAI

chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)
messages = [
    SystemMessage(content="You are a funny and helpful assistant."),
    HumanMessage(content="Explain to me quantum physics."),
]
response=chat(messages)

print(response.content,end='\n')

Try it locally :

python main.py

Now, let’s package this logic into a function that we can use in our agent :

# main.py

# Load environment variables

from dotenv import load_dotenv
load_dotenv()

# import schema for chat messages and ChatOpenAI in order to query chatmodels GPT-3.5-turbo or GPT-4

from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.chat_models import ChatOpenAI

chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)

def my_agent(input):
    messages = [
        SystemMessage(content="You are a funny and helpful assistant."),
        HumanMessage(content=input),
    ]

    response=chat(messages)

    return response.content

Deploy the agent

Having the agent run locally is cool. But now, let’s deploy it so everyone can use it.

To do so, we are going to package our langchain agent into a phospho agent. phospho is an agnostic Plateform as a Service to deploy and manage autonomous agents. They also provide an SDK to help developers go from prototype to production with their agents.

Here, we only want to create a basic conversational agent. So we only need to create a chat route.

# main.py

from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.chat_models import ChatOpenAI
from phospho import Agent, Message

# Load environment variables

from dotenv import load_dotenv
load_dotenv()

# Define our langchain agent
chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)

def my_agent(input):
    messages = [
        SystemMessage(content="You are a funny and helpful assistant."),
        HumanMessage(content=input),
    ]

    response=chat(messages)

    return response.content

# Package it into a phospho agent

agent = Agent()

@agent.chat()
def my_chat(message):
    # Use our agent logic to generate a response
    response = my_agent(message.content)
    # Return the response in a Message object
    return Message(response)

That’s it!

With this code, we have created a new agent with a single route /chat that will call our my_agent function and send the response back to the user. Note that you need to send an object of type Message as a response (view the documentation for more info).

Create a new Github repository​

At the root of your project, create a new Github repository and push your code to the main branch. Do not forget to add a .gitignore file to ignore the .env file (there is your API key inside) and the venv folder :

# Ignore virtualenv
venv/

# Ignore .env
.env

# Ignore pycache
__pycache__/

Create a new phospho project

Go to your phospho account dashboard and create a new project.

For the project name :

• respect naming conventions, use just alphanumeric characters (no spaces, no special characters, no emojis, etc…)
• if the name is already taken, choose a new one. Come on, be creative!

Select the github repository you want to link to your phospho project.

Setup the environment variables

Remember, we have stored our OpenAI API key in a .env file. But this file is not available on the server since we didn't push it to Github. Our code won't work if the API key is not available as an environment variable to the langchain agent.

On the phospho webapp, go to your project page and click on the Manage Env Variables tab. Then, click on the Environment variables tab and add a new variable called OPENAI_API_KEY with your OpenAI API key as a value.

At the next restart of your agent, it will be able to use the OpenAI API (to force a restart, you can trigger a new deployment).

Deploy the agent

Once project is created, just push some code to the main branch or go to the project page and trigger the deployment manually using the button.

After a one or two minutes, your agent should be deployed and ready to use. You have the endpoint URL on your project page.

Test your agent

Your agent is now deployed as an OpenAPI endpoint. You can find the URL of the API endpoint of your agent on the project page. You can test it directly using the API or by going in your browser to the URL and adding /docs at the end.

Unless you go viral, you should be able to use the free plan (no credit card required to start).

Github repo with the source code

You can find all the source code of this tutorial in this github repo

Congratulations! You have successfully deployed a simple conversational agent using Langchain, OpenAI and phospho. Feel free to explore and enhance the capabilities of your agent further.

Want to give anonymous feedback? Fill in the google form here.