About
If you follow AI news even a little bit, you've probably heard of Deepseek. The new AI app built in China which is supposed to be a lot better than anything else out there. You've also probably heard horror stories about apps from China collecting information on their users.
If this sounds like you, you're probably looking for a better alternative. And there is: You can build a Deepseek client on your local system and run it without any access to the Internet. What's even better is that you can do this in Docker so you don't need anything else installed. Sound good? If yes, read on!
What we're building
We're going to use Docker to create a web interface where you can ask questions to Deepseek.
To do this, we're going to create two apps:
- A Deepseek endpoint built with Ollama
- A simple static website which calls the endpoint and gets answers to questions.
When we're building multiple apps that communicate with each other, we can do this easily in Docker using Docker Compose.
Steps
🏃♂️ In a hurry? Just clone this repository and follow the instructions in the README file!
Step 1: Create docker-compose file
To work with Docker Compose, we first need to create a new file in our project's directory. Create a new directory for your project if you haven't do so already and in that, create a new file called docker-compose.yml.
💡 A docker-compose.yml file is the place where we will be describing our two apps so we can run them.
Step 2: Add first app
Now that we created our Docker Compose file, let's define our first app, the Deepseek Ollama app. In Docker Compose, apps are called "services", but it's essentially the same thing.
The Deepseek Ollama app is where we're going to point to to ask questions to Deepseek and get answers.
💡 What is Ollama? Ollama is a lightweight framework where you can easily run open source frameworks, like Deepseek or Llama, on your computer locally.
To add the Deepseek endpoint to your docker-compose.yml file, add this to the file:
services:
ollama:
image: ollama/ollama
volumes:
- ./ollama-models:/root/.ollama
ports:
- 11434:11434
🤔 What is going on here?
- This part tells Docker Compose to create a container called "ollama"
- The container will be based on the image (basically a blueprint) called
ollama/ollama. This is an image which comes with Ollama preinstalled so you don't have to do anything else! - The
volumespart essentially saves all the data from the models we're going to install later on your local hard drive. Within the container, all this data is saved in/root/.ollamabut will disappear once the container is shut down. Not exactly what we want. So whatever is stored in the container's directory, will be permanently stored inollama-modelsdirectory in your project's root. - When the Ollama app gets up and running, it will run on the
11434port within the container, but not on your machine, which means you won't be able to access it from your browser. To fix this, theportspart pairs the two ports.
Now that we added the Ollama endpoint, we can now run it so we can install Deepseek. To do this, just run this Docker Compose command in your terminal:
docker compose up -d ollama
Now go to your browser and check that the Ollama app is running, by pointing your browser to this address:
Your browser should show the "Ollama is running" text like so:
Step 3: Install Deepseek
Now that Ollama is up and running, we can now go ahead and install our Deepseek model. To do this just run this Docker Compose command:
docker compose exec ollama ollama pull deepseek-r1:7b
🤔 What is going on here?
- The
execcommand from Docker Compose essentially runs any command within a given container, in this case theollamacontainer. The above line runs (executes) the commandollama pull deepseek-r1:7bwhich installs the Deepseek model. The basic structure of theexeccommand is as follows:docker compose exec <containerName> <command>.
This command will take a few minutes to install (depending on the size of the model) but once this is done, it should populate the new ollama-models directory with the files needed for the Deepseek model.
💡 The Deepseek model comes in lots of different sizes. For this example, I've chosen 7b (which means 7 billion parameters) but you can choose more or less depending on the capabilities of your system. You can see the full list here.
Step 4: Create a website
Now that we have our Deepseek app up and running, we can create a web interface to ask questions. We're going to create a simple site with HTML, CSS and Javascript. This is what we're creating:
And here's how:
HTML
The HTML is going to define a simple page with a text box, a button to send the question and a space for the answer.
Create a new directory called web and inside that create a new file called index.html. Paste this HTML inside the file:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>My AI</title>
<script src="showdown.min.js"></script>
<script src="ollama.js"></script>
<link rel="stylesheet" href="style.css" />
</head>
<body>
<h1>🤖 Local Deepseek</h1>
<textarea name="" id="question"></textarea>
<button onclick="run()">Ask!</button>
<div id="answer">
</div>
</body>
</html>
🤔 What is going on here?
- In the
<head>part of the HTML, you'll notice that we're linking to astyle.cssWe'll create this file next in order to style our website. - You'll also notice two Javascript files,
ollama.jsandshowdown.min.js.ollama.jswill be where we talk to Deepseek andshowdown.min.jsuses Showdown, a Javascript library for converting Markdown (what we'll be getting back from Deepseek) to HTML.
CSS
To style our page, create a new file called style.css and paste the CSS below:
body{
width: 600px;
margin: auto;
}
#question{
display: block;
width: 100%;
padding: 9px;
font-size: 15px;
}
#answer{
font-family: Arial, Helvetica, sans-serif;
font-size: 15px;
margin-top: 30px;
line-height: 1.5;
}
#answer #think{
border-left: 3px solid #eee;
padding-left: 9px;
color: #aaa;
font-style: italic;
}
Javascript
Now we're going to create the Javascript to talk to Deepseek and give us answers to our questions. Create a new file called ollama.js and paste this:
const converter = new showdown.Converter()
async function run(){
let prompt = document.querySelector("#question").value
const response = await fetch("http://localhost:11434/api/generate", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "deepseek-r1:7b",
prompt: prompt,
stream: true
})
})
const reader = response.body.getReader()
const decoder = new TextDecoder()
let compiledResponse = ""
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value, { stream: true });
let chunkJson = JSON.parse(chunk)
compiledResponse += chunkJson.response
compiledResponse = compiledResponse.replace("<think>", `<div id="think">`)
compiledResponse = compiledResponse.replace("</think>", `</div>`)
document.querySelector("#answer").innerHTML = converter.makeHtml(compiledResponse)
}
}
🤔 What is going on here?
- We're creating a Javascript function called
run(). - Within the function, we're going to get the text from the text box in our HTML using
document.querySelector("#question").valueand store it in a variable calledprompt. - Then we're going to use the built in
fetch()function to send aPOSTrequest tohttp://localhost:11434/api/generatewhich includes our prompt. The response is stored in a variable calledresponse. Since we've setstream: true, we're going to get a response in small chunks. - To get each chunk individually, we're going to run
response.body.getReader()to get the reader and initialise a newTextDecoderwithnew TextDecoder(). - We're introducing a new empty string
let entireResponse = ""which we're going to be appending each chunk of the response. - Finally, the
whileloop is going to run until it runs out of chunks. For each chunk, we're going to get the response, add it to theentireResponse, process it, and show it to the webpage usingdocument.querySelector("#answer").innerHTML = entireResponseAsHtml.
🤔 Why are we processing the response?
The response comes back from Deepseek looking like this:
<think>
The user says hello so I should say hello back
</think>
**Hello! How are you doing?**
When we process the file, we replace <think> with <div id="think"> and </think> with </div>. This way we can style it however we like.
entireResponse = entireResponse.replace("<think>", `<div id="think">`)
entireResponse = entireResponse.replace("</think>", `</div>`)
We're also then converting the entire response from Markdown into HTML, using the ShowdownJS library:
let entireResponseAsHtml = converter.makeHtml(entireResponse)
Import showdown
Finally, we need to add ShowdownJS to our project. To do this, simply download this file and add it to the web directory of your project.
At the end of all this, the web directory should look like this:
📝 index.html
📝 ollama.js
📝 showdown.min.js
📝 style.css
Step 5: Add web page to Docker Compose
Once you're done with the creation of your website, add your app to the Docker Compose file like so:
web:
image: nginx:1.27.3-alpine
volumes:
- ./web:/usr/share/nginx/html
ports:
- "3001:80"
Your entire docker-compose.yml file should look like this:
services:
ollama:
image: ollama/ollama
volumes:
- ./ollama-models:/root/.ollama
ports:
- 11434:11434
web:
image: nginx:1.27.3-alpine
volumes:
- ./web:/usr/share/nginx/html
ports:
- "3001:80"
Run the website
Now that we've created our website and added to our docker-compose.yml file, we can run it with this command:
docker compose up -d web
Give it one or two seconds and then point your browser to this URL:
If you see this, we're good to go:
Let's test it!
Let's give our AI app a go! Write a question in the text box and click the Ask button.
Deepseek should soon start responding. First with some "thinking"...
This is how it looks on Video:
And that's it! You are now running Deepseek locally using just Docker!
Now every time you want to run your AI app, just run this command from your project directory in your terminal:
docker compose up -d
And that's it!
Any questions? Let me know here!
Top comments (13)
Thank you for this, it was a fun, fairly easy thing to follow along with. I am, however, having one problem. I am getting a 403 Forbidden error when I try to access the web page. I followed the instructions exactly with the exception of using port 3003 instead of 3001, another container was already using 3001. I've installed this on a NAS and am trying to access the page from my laptop, I do not have issues accessing other containers. What might be causing this? Is there something I need to add/change in the docker-compose.yaml?
Hello Guiseppe! Thank you for reading! This might be because of Nginx not getting the contents of your webpage. Are you sure your docker-compose had the correct directory as a volume in the nginx container?
I used the docker-compose exactly as you've shown it with the only change being port 3003 instead of 3001. What wasn't clear for me was what the file structure should be exactly. Currently, I have a folder called deepseek, inside that is the docker-compose.yaml, the ollama-models folder (which does have the blobs, manifests, etc. inside it after starting the container) and the web folder (with the html, js & css files inside). Is this correct or does the web directory go elsewhere?
For reference, I am able to see "ollama is running" at port 11434.
Check this repository to check how the project structure should be:
github.com/SavvasStephanides/local...
Since you are encountering a 403 Forbidden error when trying to access the web page, here are a few possible causes and solutions:
Check the Web Server Configuration
If the container is running a web server (like Nginx or Apache), check the configuration files to ensure that access is allowed from external sources.
If you are using Nginx, check the nginx.conf or site configuration file to confirm that it is not restricting access based on IP.
Verify Port Binding in docker-compose.yaml
Ensure that the port is correctly mapped in docker-compose.yaml. If you are running the service on port 3003, make sure your YAML file includes:
ports:
Some NAS devices have built-in firewall rules that restrict access to certain ports. Ensure that your NAS allows traffic on port 3003.
You may need to create a rule to allow external access.
Verify File and Folder Permissions
A 403 error can also be caused by permission issues. Ensure that the web server has the necessary permissions to access the files and directories it is serving.
Examine Logs for More Details
Run docker logs to check if there are any error messages related to access permissions.
Thanks for the great tuts & resources. The steps provided are working fine.
And I'm curious about how to input docs in the prompt & ask.
I'm sure the Ollama docs will have what you need.
Everything seemed to install OK, and clicking on the links in between steps all showed what was expected. The only issue is nothing happens when I type something in and click "Ask!".
Hello John! Maybe your browser’s console might give you a hint as to why this happens
I have followed the instructions but the "web" container is unable to access the "ollama" container.
They are both running on a remote server from my web browser.
Also, if I try to curl the localhost:11434 from inside the "web" container it is unable to connect but if I curl ollama:11434, I get the "Ollama is running" response.
Yeap. That's how networking works in Docker. If you need to curl to a container within another container, you can use the name of the container you wish to curl to:
ollama:11434in this case.If I send a POST directly from the "web" container with the correct data, it simply times out.
Tried moving the containers to an x86_64 server with the same outcome.