If you're building any kind of AI app that requires storage of vectors for embeddings, and you need an open source storage solution, it's hard to go wrong with PostgreSQL + the pgvector extension.
You could go with a hosted solution, such as Supabase's vector storage. You can also host it yourself.
It's easy and I show you how to do it for the Debian OS below, which would allow you to add this to a Debian hosted website. This is also the exact setup I'm using for clarifypdf.com, a "Chat with PDF" AI app.
Prerequisities
You'll need root access to your Debian environment.
Steps
SSH into your server, with your username and server's IP address
$ ssh user@ip.add.re.ss
Install pgvector
sudo apt install postgresql-15-pgvector
Login to PostgreSQL
$ psql -U db-username db-name
Enable pgvector
# CREATE EXTENSION pgvector;
Check that it's enabled
# \dx
You should see the extensions list, with pgvector in it.
That's it! You're ready to create embeddings with PostgreSQL!
Top comments (6)
Would love to see it in action. Any tutorials on how to use it in a ML context you are willing to share?
Sure! I'm using this exact setup in production for clarifypdf.com. So the DB contains embeddings for all documents and using it to store document sumarries.
Is there a specific aspect of the setup you're interested in?
Just a production ready setup would be interesting. I setup this myself using AWS RDS (with pgvector) and pipedream. It works but I am not sure whether this would be production ready or not. So having a point of reference would be really helpful :)
Sorry, missed this reply.
We don't have any fancy setup TBH. It's a PostgreSQL db with pgvector extension. The main "meat" of it is a Documents table and Embeddings table (one to many relation from Documents to Embeddings).
Documents store references to uploaded PDFs. Embeddings store content from the PDF. Each row in Embeddings represent one page of the PDF.
Embeddings has a 'content' column that stores the content from the PDF and an 'embeddings' column that stores the embeddings generated from the content.
When querying, we do a 'cosine similarity' search on the embeddings, which returns the rows from Embeddings table, whose 'content' values we then use to create the prompt for GPT (up to the token limit).
That's about it!
Interesting. We have the exact same setup :D I stumbled across Quivr in the meantime. This is an open source project that does basically this but utilizes Supabase and its vector databases. I use it ever since.
I run Postgresql 16; I have to use
CREATE EXTENSION vector;
Replace pgvector with vector to enable pgvector