DEV Community

Cover image for Run Deepseek locally using Docker!

Run Deepseek locally using Docker!

Savvas Stephanides on February 10, 2025

About If you follow AI news even a little bit, you've probably heard of Deepseek. The new AI app built in China which is supposed to be ...
Collapse
 
giuseppebaldi profile image
giuseppebaldi •

Thank you for this, it was a fun, fairly easy thing to follow along with. I am, however, having one problem. I am getting a 403 Forbidden error when I try to access the web page. I followed the instructions exactly with the exception of using port 3003 instead of 3001, another container was already using 3001. I've installed this on a NAS and am trying to access the page from my laptop, I do not have issues accessing other containers. What might be causing this? Is there something I need to add/change in the docker-compose.yaml?

Collapse
 
savvasstephnds profile image
Savvas Stephanides •

Hello Guiseppe! Thank you for reading! This might be because of Nginx not getting the contents of your webpage. Are you sure your docker-compose had the correct directory as a volume in the nginx container?

Collapse
 
giuseppebaldi profile image
giuseppebaldi •

I used the docker-compose exactly as you've shown it with the only change being port 3003 instead of 3001. What wasn't clear for me was what the file structure should be exactly. Currently, I have a folder called deepseek, inside that is the docker-compose.yaml, the ollama-models folder (which does have the blobs, manifests, etc. inside it after starting the container) and the web folder (with the html, js & css files inside). Is this correct or does the web directory go elsewhere?

For reference, I am able to see "ollama is running" at port 11434.

Thread Thread
 
savvasstephnds profile image
Savvas Stephanides •

Check this repository to check how the project structure should be:

github.com/SavvasStephanides/local...

Thread Thread
 
giuseppebaldi profile image
giuseppebaldi •

I put this on the back burner for a while but finally got back into it today and figured it out, it was a permissions issue. I had to run sudo chown 101:101 -R '/path/to/files' to get it to finally load the page.

I am having another issue though, pushing the Ask! button does nothing. Looking at my browser console I am getting CORS errors about "No 'Access-Control-Allow-Origin' header is present on the requested resource."

Thread Thread
 
giuseppebaldi profile image
giuseppebaldi •

The CORS error I was able to fix by setting the environmental variable OLLAMA_ORIGINS to * in my docker-compose.yaml file. The 'Ask!' button not doing anything I was able to fix by running 'ollama run deepseek-r1:7b' in a terminal, letting that load then refreshing my browser pages.

Collapse
 
zyntax profile image
John Bradshaw •

Everything seemed to install OK, and clicking on the links in between steps all showed what was expected. The only issue is nothing happens when I type something in and click "Ask!".

Collapse
 
savvasstephnds profile image
Savvas Stephanides •

Hello John! Maybe your browser’s console might give you a hint as to why this happens

Collapse
 
giuseppebaldi profile image
giuseppebaldi •

If not, I finally did get it to work. It seems to be an issue with ollama not downloading the model, per this page. In a terminal I ran 'ollama run deepseek-r1:7b' it took a bit to load but once it did everything started working. I hope this helps anyone who runs into the same issue.

Collapse
 
giuseppebaldi profile image
giuseppebaldi •

Were you ever able to fix this? I am having the same issue.

Collapse
 
zyntax profile image
John Bradshaw •

No. I ended up moving to OpenAI and ollama docker containers and it's working fine.

Collapse
 
dillibabu_k profile image
Dilli K •

Thanks for the great tuts & resources. The steps provided are working fine.
And I'm curious about how to input docs in the prompt & ask.

Collapse
 
savvasstephnds profile image
Savvas Stephanides •

I'm sure the Ollama docs will have what you need.

Collapse
 
christopher_meeker_36615d profile image
Christopher Meeker •

I have followed the instructions but the "web" container is unable to access the "ollama" container.

They are both running on a remote server from my web browser.

Collapse
 
christopher_meeker_36615d profile image
Christopher Meeker •

Also, if I try to curl the localhost:11434 from inside the "web" container it is unable to connect but if I curl ollama:11434, I get the "Ollama is running" response.

Collapse
 
savvasstephnds profile image
Savvas Stephanides •

Yeap. That's how networking works in Docker. If you need to curl to a container within another container, you can use the name of the container you wish to curl to: ollama:11434 in this case.

Collapse
 
christopher_meeker_36615d profile image
Christopher Meeker •

If I send a POST directly from the "web" container with the correct data, it simply times out.
Tried moving the containers to an x86_64 server with the same outcome.