DEV Community: Kirby Angell

VirtualBox: Using host's Ollama from guest VMs.

Kirby Angell — Fri, 18 Apr 2025 17:00:28 +0000

In this article we'll be looking at using locally hosted AI in a VirtualBox environment. At the end you'll have a VirtualBox Guest running VSCode/VSCodium with the Continue extension using the VirtualBox host's Ollama service. Why would you want to do this? I don't know, I'm not you, but it is a situation I needed to solve. I have a host machine running Windows, but the work I do is almost exclusively for Linux projects. Rather than dual boot I've opted to keep Windows as the host and setup VirtualBox guest virtual machines.

I am assuming you already have VirtualBox installed and your Guest VM running VSCode and now want to install Ollama and access it from the guest. The host will be providing the Ollama service. I've tried in the past to have Ollama running in a guest and exposing the host GPU for it to use. It is possible to make this work, but doing so always leaves me feeling empty and unfulfilled due to what I perceive as the inefficiency in such a setup. Also its usually a pain to setup. This is easy-peasy so lets get started.

Install Ollama

On Windows installing Ollama couldn't be easier. You can download the installer here: https://ollama.com/download/windows

Find OllamaSetup.exe in your Downloads folder and run it. Click the Install button and !boom! done.

The setup program not only installs Ollama, but starts the background service automatically.

Next we need to install one or more models. There are tons of choices here. Which ones you use are largely user and task specific. You can find a list of the models Ollama supports and instructions to install them at https://ollama.com/library. The models I am using, llama3 and qwen2.5-coder can be installed using these commands from a terminal window:

ollama pull llama3.1:8b
ollama pull qwen2.5-coder:latest

These two have very good performance on my system with an AMD Radeon RX 7700S with 8 gigabytes of VRAM.

Accessing Ollama from the Guest VM

This is our environment. Host running Ollama and Guest wanting to access it. The Guest networking is being provided by VirtualBox's "NAT" network. If you've left everything as default then as seen from the Guest, the Host's IP address is 10.0.2.2. The Ollama service runs on port 11434. So you can access it with the URL "http://10.0.2.2:11434" in the Guest just to make sure it is working:

Since we're inside the Guest VM lets take care of getting Continue added to VSCode. To do that click the Extensions button on the left side of the VSCode window.

In the Search field type "Continue" then click the install button for the extension. Once it is installed you're ready to configure it to use your local Ollama service. To do that you need to edit a text file inside the Guest VM. This is where the file is for Linux and Windows guests:

Linux: *~/.continue/config.yaml*
Windows: *%USERPROFILE%\\.continue\\config.yaml*

Regardless of the Guest OS here is what to put in that file, replacing the Local Assistant section entirely.

name: Local Assistant
version: 1.0.0
schema: v1
models:
  - name: chat
    provider: ollama
    model: llama3.1:8b
    apiBase: http://10.0.2.2:11434
    type: chat
  - name: autocomplete
    provider: ollama
    model: qwen2.5-coder:latest
    apiBase: http://10.0.2.2:11434
    type: autocomplete
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

After you save the file Continue will probably pick up the changes automatically. If it doesn't click on Extensions again and see if Continue has a "reload" button displayed and click that if so. Otherwise if it still hasn't picked up the changes just restart VSCode.

Conclusion

If everything has gone well then you can highlight some code and press CTRL-L and Continue will open a pane to let you ask questions about the block of code. Above I'm asking it why my overly complex Hello World program isn't working and Continue is pointing out my obvious mistake and offering to fix it. Fantastic.

Incidentally you can use the same URL to expose the Host's Ollama service to other applications and services running in your Guest. For instance here is a Docker Compose file that will run the Open WebUI service inside of your Guest:

version: '3.8'
services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://10.0.2.2:11434
    volumes:
      - open-webui:/app/backend/data
    restart: always
    deploy:
      mode: replicated
      replicas: 1

volumes:
  open-webui:

Have fun with your locally hosted AI coding assistant!

Private Docker Registry

Kirby Angell — Mon, 10 Feb 2025 20:18:19 +0000

Recently, as of this post, Docker Hub has raised their rates for their "Teams" platform. We were mostly using Docker Hub to host private images and made little use of "Teams". I can't say if it was an unreasonable rate hike or not, but it was enough for me to look at our usage compared to the new price. We have private images to load into containers on our servers. The software to host a Docker registry, the service Docker Hub provides, is itself available as a Docker image. So we have our own servers, why not our own registry?

I setup a registry for our company to use for its private images and here's how you can too. As a bonus I've included a simple web based UI you can use to examine what images are being hosted in your private registry.

In this post I will:

Create a Docker compose file for our registry; and
Configure the registry to proxy requests to Docker Hub for public images; and
Create an NGINX configuration with SSL and password support to act as a proxy to our registry; and
Demonstrate how to push and pull images from our private registry.

You will need to know how to do the following tasks that are not covered in this post:

Create, edit and deploy Docker containers from a compose YML file; the compose file can also be used in Portainer and probably other Docker management tools; and
Familiarity with NGINX configuration or some other proxy tool as it is not a good idea to directly expose your docker registry to the Internet.

compose.yml

Here is the compose.yml file we'll use to spin up our registry and UI. I'll go over some configuration you'll want to do for your particular environment below.

services:
  registry-ui:
    image: joxit/docker-registry-ui:main
    restart: always
    ports:
      - 80:80
    environment:
      - SINGLE_REGISTRY=true
      - REGISTRY_TITLE=Docker Registry UI
      - DELETE_IMAGES=true
      - NGINX_PROXY_PASS_URL=http://registry-server:5000
      - SHOW_CONTENT_DIGEST=true
      - SHOW_CATALOG_NB_TAGS=true
      - CATALOG_MIN_BRANCHES=1
      - CATALOG_MAX_BRANCHES=1
      - TAGLIST_PAGE_SIZE=100
      - REGISTRY_SECURED=false
      - CATALOG_ELEMENTS_LIMIT=1000
    container_name: registry-ui

  registry-server:
    image: registry:2
    restart: always
    ports:
      - "5000:5000"     
    volumes:
      - /var/lib/docker-registry:/var/lib/registry
    environment:
      - REGISTRY_STORAGE_DELETE_ENABLED=true
      - REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io
      - REGISTRY_PROXY_USERNAME= 
      - REGISTRY_PROXY_PASSWORD= 
    container_name: registry-server

registry-ui

The principal change you'll need to make to this service is the port you want the container to listen on.

    ports:
      - 80:80

If you want the UI to be available on a different port change the port number on the left side of the colon. If you want it to listen on port 8080 then you would change the ports specification to "8080:80".

You may also want to edit these options:

      - REGISTRY_TITLE=Docker Registry UI
      - DELETE_IMAGES=true
      - NGINX_PROXY_PASS_URL=http://registry-server:5000

Here you can change the title of the web page displayed. I like to have an easy way to delete images so I have "DELETE_IMAGES" enabled, but you can set it to "false" if you don't want this feature. "NGINX_PROXY_PASS_URL" should be set to the name of your registry service ("registry-server" in my example) and the port it is listening on (5000).

You can find more information on the UI here.

registry-server

For the registry service, like the UI, you set the port you want the registry to listen on.

   ports:
      - "5000:5000"

    volumes:
      - /var/lib/docker-registry:/var/lib/registry

You could use a persistent Docker volume to store the registry images, but instead I map a directory manually. This is just personal preference.

Environment

    environment:
      - REGISTRY_STORAGE_DELETE_ENABLED=true
      - REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io  
      - REGISTRY_PROXY_USERNAME= 
      - REGISTRY_PROXY_PASSWORD=

The first option turns on the ability to delete images from the registry. Simple enough, set it to "false" if you don't want your registry to allow this.

The rest of this section is entirely optional. If included your private docker registry will use the public Docker Hub registry to attempt to load any images you ask for that aren't in your registry. Doing this allows you to use the same process to load images for your containers whether they are your private images or public images.

You probably won't need to change "REGISTRY_PROXY_REMOTEURL"; it points to the Docker Hub registry as it is.

Docker Hub does throttle requests:

If 10 per hour isn't enough for your setup then you can create a free tier Docker Hub account and then configure your private registry to pull using those credentials. That will give you 40 pulls per hour from Docker Hub.

      - REGISTRY_PROXY_USERNAME= 
      - REGISTRY_PROXY_PASSWORD=

"REGISTRY_PROXY_USERNAME" must be set to the user name of your Docker Hub account. "REGISTRY_PROXY_PASSWORD" can be set to your Docker Hub password but I don't recommend that. Instead go to your Account Settings and click "Personal Access Tokens".

When you create a new token make sure you copy to the clipboard right then. You won't be able to see it later. Paste that token into the "REGISTRY_PROXY_PASSWORD" field.

A benefit of having your private registry act as a proxy for the Docker Hub registry is that your private registry will cache those images. So if you have a lot of containers to spin up that rely on public images only 1 pull from Docker Hub will be made for the 1st container and all the rest will get the cached version.

Start the Registry

You start the registry like you would any other docker compose project. From the directory where you have the compose.yml file:

# Depending on which version of docker you are using the command
# is:
docker compose up 
# or
docker-compose up

You will need to be a member of the "docker" group or use "sudo" to start the containers.

If your containers start without errors, then you can CTRL-C to kill them and restart them in the background by adding "-d".

docker compose up -d
# or
docker-compose up -d

Proxy the Registry Using NGINX

I recommend setting up NGINX or some other web proxy in front of your docker registry especially if you are already using it for other services you host. There are so many variations to NGINX configurations that I can't give you a stock, "this will work", for your setup. But this basic configuration works for me:

# Upstreams
upstream docker-registry {
   server 127.0.0.1:5000;
}


server  {
  listen 10000 ssl;
  server_name    myserver.com;

  ssl_protocols TLSv1.3 TLSv1.2;

  chunked_transfer_encoding on;
  gzip off;

  client_max_body_size 0;  
  proxy_buffering off;  
  proxy_request_buffering off;  
  proxy_max_temp_file_size 0;  

  #access_log /var/log/nginx/docker-registry-access.log;
  error_log /var/log/nginx/docker-registry-error.log;

  ssl_certificate /etc/letsencrypt/live/myserver.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/myserver.com/privkey.pem;

  location / {

        auth_basic "Registry Realm";
        auth_basic_user_file /etc/docker-registry-htpasswd;  

        proxy_pass http://docker-registry/;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
  }
}

This file goes in "/etc/nginx/sites-available/docker-registry", with a symbolic link in "/etc/nginx/sites-enabled/":

sudo ln /etc/nginx/sites-available/docker-registry /etc/nginx/sites-enabled/

You will need to change a few lines:

server  {
  listen 10000 ssl;
  server_name    myserver.com;
  ssl_certificate /etc/letsencrypt/live/myserver.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/myserver.com/privkey.pem;

Replace the listen port with the port you want your registry to run on. Replace "myserver.com" in all 3 places with the domain name you are using. If you are not using Let's Encrypt then replace the whole path with where you keep your SSL certificate.

    auth_basic "Registry Realm";
    auth_basic_user_file /etc/docker-registry-htpasswd;

This section adds password protection to your registry. It is using a standard htpasswd file. To create this file follow these steps (Debian based only):

sudo apt update
sudo apt install apache2-utils
htpasswd -c /etc/docker-registry-htpasswd username

Replace "username" with whatever you want for a name. You will be prompted for a password. You can add additional users by repeating the command without the "-c".

After you've create the file and made the necessary changes, you can test your ngix configuration sanity using this command:

sudo nginx -t

If the configuration is acceptable you'll see messages like these:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

If the NGINX configuration test works you can restart NGINX using this command:

sudo systemctl restart nginx

Using the Registry

To begin using your private registry you will need to login which creates the "~.docker/config.json" file. From the command line:

docker login myserver.com:PORT

Replacing "myserver.com" and "port" with your values. You will prompted for the user name and password.

Now we're finally ready to use the registry! First we'll need to push an image to it:

docker push myserver.com:port/yourimage:tag

Pull:

docker pull myserver.com:port/yourimage:tag

Those two commands work for your private registry, but if you want to pull a public image from Docker Hub you will have to adjust the normal image path to include "/library/" like this:

docker pull myserver.com:port/library/publicimage:tag

"/library" is the base path for all Docker Hub images.

Summary

In this post I've shown how to configure a private Docker registry. The registry uses Docker Hub as a backing store for public images. NGINX is used to provide SSL encryption and also password protect the private registry.

Packet Capture From Inside Docker

Kirby Angell — Sun, 10 Mar 2024 19:26:42 +0000

For an existing project I needed a better way of capturing network traffic than what I had been using. Before, I used Python to setup a capture using Impacket. I would have to keep track of the ports the process used and then filter the pcap data based on those ports to get just the process I was interested in. That method had a big problem though. The port data had to be collected from the connections themselves which meant until the connection was setup, I wouldn't see any of the traffic.

Overview

Looking for a new solution what I wanted was something like strace but instead of recording library function calls I want the network packets being sent and in pcap format. I found what I was looking for in an AskUbuntu post.

A lot has changed in 13 years since that post was written, but how Linux does networking, at least from the administrators side is still valid. A response to the question has the answer which is to use some iptables magic to create an isolated network namespace. That author points out that it is fairly complicated though. It might also be hard to "wrap up" into a usable tool.

But it wasn't! At least not for Jonas Danielsson who followed up with a Github project for the tool nsntrace. So that's what I'm going to use in this article where I'll show you how to capture all of the network traffic into a pcap file for a process spawned inside of a Docker container.

Docker Configuration

Here is the basic Dockerfile configuration we will be using for this project:

# Use Debian as the base image
FROM debian:buster

# Update package lists and install necessary packages
RUN apt-get update && apt-get install -y \
    curl \
    build-essential \
    git \
    automake \
    docbook-xml  \
    docbook-xsl \
    iptables \
    libnl-route-3-dev \
    libpcap-dev \
    pkg-config \
    xsltproc \
    && rm -rf /var/lib/apt/lists/*

RUN update-alternatives --set iptables /usr/sbin/iptables-legacy
RUN update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

WORKDIR /
RUN git clone https://github.com/nsntrace/nsntrace.git
WORKDIR /nsntrace
RUN ./autogen.sh
RUN ./configure
RUN make install

The actual one for my project involves quite a bit more, but this has the essentials we'll need to get nsntrace working. The package list comes from the nstrace build instructions plus a few extras to facilitate the build and testing. Debian as of the Buster release started using nftables instead of iptables. Luckily Debian provides an easy way to switch back to iptables using the update-alternatives command in the Dockerfile.

We build the container using this command:

docker build -t test .

Running

sudo docker run \
    --privileged \
    -v /tmp:/tmp -it test nsntrace \
    <process to run>

For nstrace to do the things it needs to capture the traffic, such as adding network interfaces and manipulating the firewall rules it needs the "--privileged" flag. I'm not a big fan of giving the container these capabilities and would like to narrow it down. With SYS_ADMIN and NET_ADMIN nstrace works, but displays an error message "failed to set namespace name". As of the time of this publiciation I haven't found a Docker run flag that will give it just the ability to control network namespaces.

I've chosen to map the host's /tmp directory to the container's /tmp directory so we can get the packet captures back out of the container to examine. Here I use the new container to capture the traffic curl generates accessing my company website:

$ sudo docker run --privileged -v /tmp:/tmp -it test nsntrace -o /tmp/test.cap curl https://alertra.com
Starting network trace of 'curl' on interface eth0.
Your IP address in this trace is 172.19.179.255.
Use ctrl-c to end at any time.

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://www.alertra.com/">here</a>.</p>
</body></html>
Finished capturing 33 packets.
$

The pcap file is placed in /tmp/test.cap which we can examine with Wireshark:

Here we see everything the process did on the network. The DNS queries, the setup and teardown of the TCP connection, and the TLS negotiation. We even see the local network ARP packets!

Conclusion

Obviously this is a simple example and if you just want to capture packets coming or going on a port then there are easier ways to do it like tcpdump and Wireshark. But if you really need to see all the network traffic at a process level, then nsntrace works great and can be run from within a Docker container if need be.