DEV Community

Cover image for Building a Whisper.cpp Container Image
Sanskriti Harmukh for Vultr

Posted on with Aashish Chaurasiya Originally published at docs.vultr.com

Building a Whisper.cpp Container Image

Whisper.cpp is a high-performance, lightweight inference implementation of OpenAI's Whisper automatic speech recognition (ASR) model. It's written in plain C/C++ with no external dependency packages, and it performs speech recognition with support for both CPU and GPU-based systems. This guide walks through cloning the Whisper.cpp repository and building container images for CPU-only and GPU-powered systems, then pushing the result to a container registry. By the end, you'll have a working Whisper.cpp container image tagged and pushed to a registry, ready to deploy.


1. Prerequisites

Before you begin:

  • Deploy a GPU-enabled server with NVIDIA GPU drivers, CUDA, and Docker preinstalled.
  • Access the server using SSH.
  • Start the Docker service.
$ sudo systemctl start docker
Enter fullscreen mode Exit fullscreen mode
  • Add the non-root user to the Docker group. For example, linuxuser:
$ sudo usermod -aG docker linuxuser
Enter fullscreen mode Exit fullscreen mode
  • Switch to the user:
$ su - linuxuser
Enter fullscreen mode Exit fullscreen mode

2. Set Up the Server

1. Create a new directory to store your Whisper C++ project files:

$ mkdir whisper-project
Enter fullscreen mode Exit fullscreen mode

2. Switch to the directory:

$ cd whisper-project
Enter fullscreen mode Exit fullscreen mode

3. Clone the Whisper.cpp project repository using Git:

$ git clone https://github.com/ggerganov/whisper.cpp
Enter fullscreen mode Exit fullscreen mode

4. List all directory files and verify that a new whisper.cpp directory is available:

$ ls
Enter fullscreen mode Exit fullscreen mode

Output:

whisper.cpp
Enter fullscreen mode Exit fullscreen mode

5. Switch to the project directory:

$ cd whisper.cpp
Enter fullscreen mode Exit fullscreen mode

6. List all hidden files and verify that a .devops directory is available:

$ ls -a
Enter fullscreen mode Exit fullscreen mode

Output:

.  .devops         cmake           examples        ggml-backend.h       ggml-cuda     ggml-impl.h       ggml-metal.m      ggml-quants.c  ggml-vulkan.h    .gitmodules  models  .......
Enter fullscreen mode Exit fullscreen mode

The .devops directory includes the following Dockerfile resources:

  • main.Dockerfile: Contains the build context for CPU systems.
  • main-cuda.Dockerfile: Contains the build context for GPU systems.

Use these resources in the next sections to build a CPU or GPU system container image.

3. Build a Whisper.cpp Container Image for CPU Systems

Use main.Dockerfile, which contains all necessary dependencies for CPU-based systems.

1. Copy the main.Dockerfile from the .devops directory to the main whisper.cpp project directory:

$ cp .devops/main.Dockerfile .
Enter fullscreen mode Exit fullscreen mode

2. Build a new container image using main.Dockerfile with all files in the project directory. Replace whisper-image with your desired image name:

$ docker build -f main.Dockerfile -t whisper-image .
Enter fullscreen mode Exit fullscreen mode

Wait for the build process to complete. When successful, your output should look like the one below.

[+] Building 117.7s (12/12) FINISHED                                                                                                                                                                                 docker:default
 => [internal] load build definition from main.Dockerfile                                                                                                                                                                      0.0s
 => => transferring dockerfile: 446B                                                                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                                                                3.1s
 => [internal] load .dockerignore
Enter fullscreen mode Exit fullscreen mode

3. View all Docker images on the server and verify that the new whisper-image is available:

$ docker images
Enter fullscreen mode Exit fullscreen mode

Output:

REPOSITORY      TAG       IMAGE ID       CREATED         SIZE
whisper-image   latest    d54bd7163db5   4 minutes ago   608MB
Enter fullscreen mode Exit fullscreen mode

4. Build a Whisper.cpp Container Image for GPU Systems

The main-cuda.Dockerfile resource contains the build context for GPU systems compatible with the latest NVIDIA CUDA drivers.

1. Copy main-cuda.Dockerfile to the main project directory:

$ cp .devops/main-cuda.Dockerfile .
Enter fullscreen mode Exit fullscreen mode

2. Build a new container image whisper-gpu-image using the main-cuda.Dockerfile with all files in the working project directory:

$ docker build -f main-cuda.Dockerfile -t whisper-gpu-image .
Enter fullscreen mode Exit fullscreen mode

3. View all Docker images on the server and verify that the new Whisper.cpp GPU image is available:

$ docker images
Enter fullscreen mode Exit fullscreen mode

Output:

REPOSITORY          TAG       IMAGE ID       CREATED          SIZE
whisper-gpu-image   latest    47be5b1566fd   31 seconds ago   2.76GB
Enter fullscreen mode Exit fullscreen mode

To run the Whisper.cpp GPU container image, verify that your target host includes the minimum or higher CUDA version referenced by the ARG CUDA_VERSION= directive within main-cuda.Dockerfile.

4. Verify the target CUDA version:

$ cat main-cuda.Dockerfile | grep CUDA_VERSION=
Enter fullscreen mode Exit fullscreen mode

Output:

ARG CUDA_VERSION=12.3.1
Enter fullscreen mode Exit fullscreen mode

5. Upload the Whisper.cpp Container Image to a Container Registry

1. Open your cloud provider's console and navigate to your container registry to view the registry access credentials. Copy the Registry URL, Username, and API Key.

2. Switch to your server terminal session and log in to your container registry. Replace your-registry.example.com, exampleregistry, exampleuser, registry-password with your actual registry details:

$ docker login https://your-registry.example.com/exampleregistry -u exampleuser -p registry-password
Enter fullscreen mode Exit fullscreen mode

3. Tag the Whisper.cpp container image with your desired registry tag. For example, your-registry.example.com/exampleregistry/whisper-gpu-image:

$ docker tag whisper-gpu-image your-registry.example.com/exampleregistry/whisper-gpu-image
Enter fullscreen mode Exit fullscreen mode

4. View all Docker images on the server and verify that the new tagged image is available:

$ docker images
Enter fullscreen mode Exit fullscreen mode

Output:

REPOSITORY                                                     TAG       IMAGE ID       CREATED          SIZE
whisper-gpu-image                                              latest    47be5b1566fd   6 minutes ago    2.76GB
your-registry.example.com/exampleregistry/whisper-gpu-image    latest    47be5b1566fd   6 minutes ago    2.76GB
Enter fullscreen mode Exit fullscreen mode

5. Push the tagged image to your container registry:

$ docker push your-registry.example.com/exampleregistry/whisper-gpu-image
Enter fullscreen mode Exit fullscreen mode

When successful, your output should be similar to the one below:

The push refers to repository [your-registry.example.com/exampleregistry/whisper-gpu-image]
a42c530a140b: Pushed
445c20a8fa7a: Pushed
8ceb9643fb36: Pushed
latest: digest: sha256:70645eed0095268b6c11b051083cb83c41c1991dc088b0d0485ccc50aa4cf4af size: 2838
Enter fullscreen mode Exit fullscreen mode

6. Open your container registry's management panel and verify that the new repository is available.

Next Steps

  • Push the image to a container registry and wire it into your transcription pipeline.
  • Benchmark CPU vs. GPU inference latency for your target audio workloads.
  • Automate rebuilds via CI so new whisper.cpp releases stay current.
  • Experiment with different Whisper model sizes for accuracy/speed tradeoffs.

For the full guide with additional tips, visit the original article on Vultr Docs.

Top comments (0)