PrivateGPT - Running "ChatGPT" offline on local documents

PrivateGPT is a python script to interrogate local files using GPT4ALL, an open source large language model. It is pretty straight forward to set up:

1. Clone the repo
2. Download the LLM - about 10GB - and place it in a new folder called models.
3. Place the documents you want to interrogate into the source_documents folder - by default, there's a text of the last US state of the union in there.
4. Run the ingest.py script - this can a long time. On this MacBook M1 it was 2 minutes though
5. Run the privateGPT.py script and you get a prompt in your Terminal:

I asked it what the US president has to say about the situation in the Ukraine and it gave me a synopsis and where in the document the information is from.

Neat!

Comments

[Alexander]

can I use this to do my own chatgpt on a webpage?

[Christian Heilmann]

You could but there are easier implementations like this one: github.com/mlc-ai/web-llm

[Alexander]

I know is probably easy for most people, but I enter in the repository and I didnt understand, if you can help me more, I will be very grateful: c and, much text in English confuses me

[Bernd Wechner]

I can't see why not. What would prevent that? It's a python script. Nothing to do you running out server side in response to requests and delivering responses based on our containing the results.

[Alexander]

I'll try, I'm a bit of a newbie to this

[0xAungkon]

Same just learning new things .

[Alexander]

I want to develop an AI chatbot in my university on a web that can help each student that need help of any information or protocol of the university.

[0xAungkon]

use python fastapi

[Mathieu Parent]

Hi! Great article.
Im using this to make visit reports of customer meetings from audio, emails and documents to give the full picture.

Is there a way to create a Python pipeline with pre-made prompts and get the responses saved in a txt file? Basically an automation of that Private Gpt.

Merci

[Ashutosh Sharma]

This is very interesting, I will give it a try and post the results.

[Nelson Figueroa]

This is pretty fun to play around with, thanks for sharing!

[ARYAN SINGH]

ive read somewhere that it only works on CPU becasue of llama cpp , cant we run it in gpu linux environment

[Erica]

Can you make it search your Google Drive?

[Andy Piper]

This code relies on local files in one of a set of understood formats (text, markdown, etc). You could extend it to pull down your Google Drive content, or sync it to a local directory first.