Deploy your own ChatGPT in 5 minutes

Disclaimer: The files for this tutorial are available at: our Github Tutorial Repo

If you ever wondered about how you could deploy your own version of ChatGPT with your/your company's content, this is your time to shine: I have a wonderful solution for you.

For those who don't know me or my project, I'm Vinnie, a software engineer and co-founder of the Talkd.AI project. We are a simple and easy to deploy app that allows any developer with some context of Docker and Python to deploy their own RAG (Retrieval-augmented Generation) the name of the type of applications that use a (vector) database and your custom content to return data from and to your LLM.

What you will need to have in order to use our Dialog server?

• A computer/server with a docker instance up and running
• A CSV file with some contents from your company (we will specify the format later on)
• A TOML file for setting up your model and prompts
• An OpenAI API Key (here is a tutorial to generate one)

Getting started

In order to start your own deployment of our Dialog server, you will need to use the following docker-compose file:

services:
  db:
    image: pgvector/pgvector:0.6.2-pg15
    restart: always
    ports:
      - '5432:5432'
    environment:
      POSTGRES_USER: talkdai
      POSTGRES_PASSWORD: talkdai
      POSTGRES_DB: talkdai
    volumes:
       - ./etc/db-extensions.sql:/docker-entrypoint-initdb.d/db-extensions.sql
    healthcheck:
      test: ["CMD", "pg_isready", "-d", "talkdai", "-U", "talkdai"]
      interval: 10s
      timeout: 5s
      retries: 5
  dialog:
    image: ghcr.io/talkdai/dialog:latest
    volumes:
      - ./data/:/app/data/
    ports:
      - '8000:8000'
    depends_on:
      db:
        condition: service_healthy
    environment:
      - PORT=8000
      - DATABASE_URL=postgresql://talkdai:talkdai@db:5432/talkdai
      - OPENAI_API_KEY=sk-your-openai-api-key
      - STATIC_FILE_LOCATION=/app/static
      - DIALOG_DATA_PATH=../data/your.csv
      - PROJECT_CONFIG=../data/your.toml

PGVector - Our first service

The first service of this docker-compose file is a PostgreSQL instance equipped with pgvector, our vector database implementation of choice since it's already inside a major open-source database vendor.

On this service, we've configured a volume mapping to a file, this file's content is the following:

CREATE EXTENSION IF NOT EXISTS vector;

This file only enables the vector plugin if it doesn't exist already in your database. If you are using another implementation of a PGVector Docker image that already runs this command, you can ignore this volume mounting.

Second Service - Dialog Server

As you can see, the second service is our Dialog Server, it's our implementation of the RAG approach built on top of FastAPI, LangChain and PGVector Python extension.

We also have a volume mapping inside this service:

    volumes:
      - ./data/:/app/data/

This volume mapping allow us to provide our prompt information and your custom content in the CSV format.

The CSV file for loading your own ChatGPT must have the following columns:

• category - The category of that content
• subcategory - The subcategory of that content
• question - The question/title of that content
• content - The content in itself, this content will generate the embeddings for our database.

On our example, we will add some questions regarding sports:

category,subcategory,question,content
faq,football,"Whats your favorite soccer team","My favorite soccer team is Palmeiras, from Brazil."
faq,football,"Whats your favorite soccer player","My favorite soccer player is Neymar, from Brazil."

Inside our prompt file (the .toml one I've mentioned before), we will configure our prompt and how the bot will speak, which model and temperature it will use and a fallback message in order it can generate answers on topics that it didn't find a content for a certain user message.

[model]
model = "gpt-3.5-turbo-1106"
temperature = 0.1

[prompt]
header = """You are Robert, an AI bot that answers any questions that a user may have."""
question_signalizer = "Answer the following question:"

suggested = """Just answer the following question, don't change context and don't reply using more than 50 words."""

fallback = """
Your a nice bot, say something nice to the user and try to help him with his question, but also say to the user that you don't know totally about the content he asked.
"""

Finishing the setup

With all of these files saved inside the ./data/ directory, we are able to do the last part of the setup: setting our environment variables in order to deploy our bot!

The environment variables that must be filled are:

• PORT - the Port where the web server is allowed to be started on (we recommend using 8000, but you can set your own)
• DATABASE_URL - the database URL (like postgresql://talkdai:talkdai@db:5432/talkdai)
• OPENAI_API_KEY - The generated OpenAI API Key
• DIALOG_DATA_PATH - The relative path from /app/ for the data CSV file you mapped inside your docker compose volume. (Our standard is to use the folder /app/data to host this file, so the value of this variable must be: ../data/your.csv
• PROJECT_CONFIG - The Prompt file location. The explanation is the same as above, but for the toml file. In this tutorial, we will set this as ../data/your.toml

Final step: Deploying and running

Now that we finally have our setup done, we can start our Dialog Server instance by running:

docker-compose up # -d # if you want to run it on background

You will need to wait a while until it builds everything, get our pgvector instance up and running and create the embeddings for your custom content.

When it finishes, your docker-compose log will look like this:

To access the API Swagger, you will go in your browser and use the following URL (or the equivalent for your setup): http://localhost:8000/docs.

If everything is fine, the following page must have loaded and you can now start asking questions to your own ChatGPT instance.

Using the /ask endpoint, you will be able to send any message you want the ChatGPT to answer, such as Whats your NBA team of heart?:

And after hitting send, the answer comes:

Conclusion

If you follow through this tutorial, you will be able to deploy your own ChatGPT with custom content for your company or your website.

Leave a like in this post if you enjoyed and share it with your friends.

Also, if you want more information on dialog and talkd.ai, access our GitHub page: https://github.com/talkdai/

Comments

[Beeblebrox]

What about the hardware requirements? How responsive would this be on relatively modern developer PC?