DEV Community

Matthieu SB
Matthieu SB

Posted on • Edited on

Self-Hosting Wikipedia (and other sites ...) with Kiwix

#selfhosting #docker #wkipedia #staticwebapps

In this article we'll be deploying Kiwix using docker and docker compose. You can look at the github repositories and contribute to its many projects (server, mobile, browser and desktop apps).

Here is an example of what it looks like:

Image description

Disclaimer

ℹ️ This article is helpful if you want to self-host an OFFLINE version of different sites (Wikipedia, Wikitionnary, Stack Exchange, DevDocs ...) in your home, or on a portable server to bring it with you (It should work on a Raspberry-Pi).

⚠️ This article is not for you if you want to host and edit your own version of Wikipedia, but there should be resources on MediaWiki out there. You also don't need this if you just want to read .zim files directly on your local computer or phone, kiwix client apps are largely enough.

Setup Instructions

This installation was tested on ubuntu 24.04, both server and desktop, but it should work on any system with docker installed. Most of this tutorial is a step-by-step guide largely inspired by joinboiser github repository

Pre-requisites

  • A system with docker installed
  • Your favorite command line
  • Your favorite editor

Create the project structure and empty files:

mkdir -p kiwix/zims
mkdir -p kiwix/scripts

cd kiwix

touch Dockerfile
touch docker-compose.yml
touch scripts/provision.sh
touch scripts/entrypoint.sh

chmod +x scripts/provision.sh
chmod +x scripts/entrypoint.sh
Enter fullscreen mode Exit fullscreen mode

Get static sites data

Kiwix uses .zim files, you can find a nice repository on the official kiwix downloads repository.

💡 To begin I would advise using tiny zim files to try out your setup, a full english wikipedia with images can take around 100GB and takes a while to download

For this example, we'll be downloading the following (From the kiwix folder:

wget https://download.kiwix.org/zim/devdocs/devdocs_en_ansible_2025-01.zim -O zims/devdocs-ansible-en_2025-01.zim
wget https://download.kiwix.org/zim/devdocs/devdocs_en_typescript_2025-01.zim -O zims/devdocs-typescript-en_2025-01.zim
Enter fullscreen mode Exit fullscreen mode

If you still want to download wikipedia, I am currently using this version (🚨 Very long to download):

wget https://download.kiwix.org/zim/wikipedia/wikipedia_en_all_maxi_2024-01.zim -O wikipedia-en_2024-01.zim
Enter fullscreen mode Exit fullscreen mode

Setting up scripts

Add the following content to the empty scripts:

provision.sh

#!/bin/bash

TGTDIR="${1:-/usr/local/bin}"

VERSION="${VERSION:-3.1.2-4}"
MACHINE="${MACHINE:-$(uname -m)}"

URL="${URL:-"https://download.kiwix.org/release/kiwix-tools/kiwix-tools_linux-${MACHINE}-${VERSION}.tar.gz"}"

DLPATH=${DLPATH:-/tmp}
FILE="$DLPATH/${URL##*/}"

# Install kiwix-tools from binary tarball
set -x -e
wget -O "$FILE" "$URL"
tar -xvf "$FILE" -C "$TGTDIR" --strip-components 1
rm -f "$FILE"
Enter fullscreen mode Exit fullscreen mode

entrypoint.sh

#!/bin/bash

ZIMDIR=${ZIMDIR:-/zims}
LIBRARY_XML=/library.xml

for f in "$ZIMDIR"/*.zim; do
  if [[ -f "$f" ]]; then
    ( set -x; kiwix-manage "$LIBRARY_XML" add $f )
  fi
done

kiwix-serve --port 8080 --library "$LIBRARY_XML"
Enter fullscreen mode Exit fullscreen mode

Setting up Dockerfile

Add this to your empty dockerfile:

FROM ubuntu:24.04

ENV TGTDIR=/usr/local/bin

RUN env LANG=C LC_ALL=C apt update
RUN env DEBIAN_FRONTEND=noninteractive LANG=C LC_ALL=C apt-get -y full-upgrade
RUN env DEBIAN_FRONTEND=noninteractive LANG=C LC_ALL=C apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install wget

VOLUME /zims

COPY ./scripts/*.sh $TGTDIR/

RUN provision.sh

EXPOSE 8080
ENTRYPOINT ["entrypoint.sh"]
Enter fullscreen mode Exit fullscreen mode

💡 For testing purposes, you can comment the following line as it can take a while to execute: RUN env DEBIAN_FRONTEND=noninteractive LANG=C LC_ALL=C apt-get -y full-upgrade

Setting up the docker-compose file

Add this to your empty docker-compose:

services:
  kiwix:
    build: .
    container_name: kiwix
    ports:
      - '8080:8080'
    volumes:
      - ./zims:/zims:ro
    networks:
      - kiwix_net

networks:
  kiwix_net: 
    driver: bridge
Enter fullscreen mode Exit fullscreen mode

Your kiwix folder should now look like this:

kiwix
├── docker-compose.yml
├── Dockerfile
├── scripts
│  ├── entrypoint.sh
│  └── provision.sh
└── zims
   ├── devdocs-ansible-en_2025-01.zim
   └── devdocs-typescript-en_2025-01.zim
Enter fullscreen mode Exit fullscreen mode

Launching and testing

To launch our kiwix server, just execute:

docker compose up --build
Enter fullscreen mode Exit fullscreen mode

The service should now be available at <server-hostname-or-ip>:8080. Use localhost:8080 if you deployed locally.

If you only added devdocs mentioned above, you should see something like this:

Image description

To stop the service:

docker compose stop
Enter fullscreen mode Exit fullscreen mode

If you add new zim files, you just need to restart the service, a docker compose restart should suffice

[OPTIONAL] Using a systemd service

This will help you if you want to enable the service on boot and administer it through systemd.

First, create the log folders and unit file with:

sudo mkdir -p /var/log/kiwix
touch kiwix.service
Enter fullscreen mode Exit fullscreen mode

Add and adapt the following to your kiwix.service file:

[Unit]
Description=Kiwix
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=<YOUR USER>
ExecStart=/usr/bin/docker compose -f <path-to-kiwix-service>/docker-compose.yml up --build
ExecStop=/usr/bin/docker compose -f <path-to-kiwix-service>/docker-compose.yml stop
StandardOutput=append:/var/log/kiwix/kiwix.log
StandardError=append:/var/log/kiwix/kiwix-err.log

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Copy it to systemd configs:

sudo cp kiwix.service /etc/systemd/system
Enter fullscreen mode Exit fullscreen mode

And then start and enable the systemd service:

# Reload configurations
sudo systemctl daemon-reload
# Enable start on boot and start service immediatly
sudo systemctl enable kiwix.service && sudo systemctl start kiwix.service
Enter fullscreen mode Exit fullscreen mode

This should do it ! Have fun with your custom static sites.

Top comments (0)