DEV Community

Cover image for Run Deepseek locally using Docker!

Run Deepseek locally using Docker!

Savvas Stephanides on February 10, 2025

About If you follow AI news even a little bit, you've probably heard of Deepseek. The new AI app built in China which is supposed to be ...
Collapse
 
giuseppebaldi profile image
giuseppebaldi

Thank you for this, it was a fun, fairly easy thing to follow along with. I am, however, having one problem. I am getting a 403 Forbidden error when I try to access the web page. I followed the instructions exactly with the exception of using port 3003 instead of 3001, another container was already using 3001. I've installed this on a NAS and am trying to access the page from my laptop, I do not have issues accessing other containers. What might be causing this? Is there something I need to add/change in the docker-compose.yaml?

Collapse
 
savvasstephnds profile image
Savvas Stephanides

Hello Guiseppe! Thank you for reading! This might be because of Nginx not getting the contents of your webpage. Are you sure your docker-compose had the correct directory as a volume in the nginx container?

Collapse
 
giuseppebaldi profile image
giuseppebaldi

I used the docker-compose exactly as you've shown it with the only change being port 3003 instead of 3001. What wasn't clear for me was what the file structure should be exactly. Currently, I have a folder called deepseek, inside that is the docker-compose.yaml, the ollama-models folder (which does have the blobs, manifests, etc. inside it after starting the container) and the web folder (with the html, js & css files inside). Is this correct or does the web directory go elsewhere?

For reference, I am able to see "ollama is running" at port 11434.

Thread Thread
 
savvasstephnds profile image
Savvas Stephanides

Check this repository to check how the project structure should be:

github.com/SavvasStephanides/local...

Thread Thread
 
giuseppebaldi profile image
giuseppebaldi

I put this on the back burner for a while but finally got back into it today and figured it out, it was a permissions issue. I had to run sudo chown 101:101 -R '/path/to/files' to get it to finally load the page.

I am having another issue though, pushing the Ask! button does nothing. Looking at my browser console I am getting CORS errors about "No 'Access-Control-Allow-Origin' header is present on the requested resource."

Thread Thread
 
giuseppebaldi profile image
giuseppebaldi

The CORS error I was able to fix by setting the environmental variable OLLAMA_ORIGINS to * in my docker-compose.yaml file. The 'Ask!' button not doing anything I was able to fix by running 'ollama run deepseek-r1:7b' in a terminal, letting that load then refreshing my browser pages.

Collapse
 
zyntax profile image
John Bradshaw

Everything seemed to install OK, and clicking on the links in between steps all showed what was expected. The only issue is nothing happens when I type something in and click "Ask!".

Collapse
 
savvasstephnds profile image
Savvas Stephanides

Hello John! Maybe your browser’s console might give you a hint as to why this happens

Collapse
 
giuseppebaldi profile image
giuseppebaldi

If not, I finally did get it to work. It seems to be an issue with ollama not downloading the model, per this page. In a terminal I ran 'ollama run deepseek-r1:7b' it took a bit to load but once it did everything started working. I hope this helps anyone who runs into the same issue.

Collapse
 
giuseppebaldi profile image
giuseppebaldi

Were you ever able to fix this? I am having the same issue.

Collapse
 
zyntax profile image
John Bradshaw

No. I ended up moving to OpenAI and ollama docker containers and it's working fine.

Collapse
 
dillibabu_k profile image
Dilli K

Thanks for the great tuts & resources. The steps provided are working fine.
And I'm curious about how to input docs in the prompt & ask.

Collapse
 
savvasstephnds profile image
Savvas Stephanides

I'm sure the Ollama docs will have what you need.

Collapse
 
christopher_meeker_36615d profile image
Christopher Meeker

I have followed the instructions but the "web" container is unable to access the "ollama" container.

They are both running on a remote server from my web browser.

Collapse
 
christopher_meeker_36615d profile image
Christopher Meeker

Also, if I try to curl the localhost:11434 from inside the "web" container it is unable to connect but if I curl ollama:11434, I get the "Ollama is running" response.

Collapse
 
savvasstephnds profile image
Savvas Stephanides

Yeap. That's how networking works in Docker. If you need to curl to a container within another container, you can use the name of the container you wish to curl to: ollama:11434 in this case.

Collapse
 
christopher_meeker_36615d profile image
Christopher Meeker

If I send a POST directly from the "web" container with the correct data, it simply times out.
Tried moving the containers to an x86_64 server with the same outcome.