DEV Community

Héctor Pascual
Héctor Pascual

Posted on

3 1

Docker in Docker, managing existant volumes

Introducing the context, I am working with a docker container that runs jenkins, inside it there are some other docker services running aswell, brought up during some jenkins job execution.

As an existant volume I refer to a folder containing data that I will mount to the container, for instance :

$ docker run ... -v /folder/with/data:/var/data ...

The command I use to run jenkins is the following one :

$ docker run \                                        
  -d \               
  -u  \
  -p 8080:8080 \
  -v /data/jenkins_test:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  shimmi/jenkins

As you can see the docker socket, which is the main entry point for docker API is mapped as a volume into the container, that means, that the docker engine will be "shared" (it will be the same docker socket) and volumes, images and containers will be shared inside and outside the container.

This is a relevant point that caused me a lot of headache trying to figure why once I run a container inside jenkins, the path specified as existant volumes when creating containers were not found or empty.

The problem was that the path I was specifying was the correct for the jenkins container but Docker was trying to search for it in the host that runs the jenkins container (my machine filesystem) not in the jenkins container itself.

About volumes and docker-compose

Specifically the services I bring up are specified in a docker-compose file, in order to use existant volumes from the host you have to declare a top level volumes option and set the external property as true :

volumes:
  <myVolume>:
    external: true

And now, if this is an existant volume in your docker host you should be able to use it.

Creating volumes from existing data folders

In my case I wanted to create a volume and "link" it to some folder with data, not an empty volume, in order to do that I followed these steps :

$ docker volume create <myVolume>
$ docker volume inspect <myVolume> 
[
    {
        "CreatedAt": "0001-01-01T00:00:00Z",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/<myVolume>/_data",
        "Name": "<myVolume>",
        "Options": {},
        "Scope": "local"
    }
]
# See the mountpoint is the path we will link
$ sudo rm -rf /var/lib/docker/volumes/<myVolume>/_data
$ sudo ln -s <MY_FOLDER_WITH_DATA> /var/lib/docker/volumes/<myVolume>/_data

That way, you can bring up containers inside another containers (docker in docker) managing volumes with data in an easy way.

Bash script

In order to automate the task of creating a volume and the symlink I wrote a bash script :

#!/bin/bash

# This script creates a docker volume and links it to a existing folder that contains data.

# Arg1 : volume name
# Arg2 : existant folder with data to create the volume

ARG1=$1
ARG2=$2

function create_volume {
    echo '$1 = ' $ARG1
    echo '$2 = ' $ARG2
    docker volume create $ARG1
    if [ $? -ne 0 ]
      then
        echo "Script failed"
        exit 1
    fi
    echo "Volume created!"
    sudo rm -rf /var/lib/docker/volumes/$ARG1/_data
    sudo ln -s $ARG2 /var/lib/docker/volumes/$ARG1/_data
    if [ $? -ne 0 ]
      then
        echo "Script failed"
        exit 1
    fi
    echo "Folder linked"
    echo "Script executed succesfully"
}

if [ $# -eq 0 ]
  then
    echo "No arguments supplied, specify the volume name and the path of the folder"
    exit 1
fi

if [ $# -eq 1 ]
  then
    echo "Not enough arguments supplied, specify the volume name and the path of the folder"
    exit 1
fi

EXISTS="$(docker volume ls -f name=$ARG1 | grep -w ARG1)"
if [ -z "$EXISTS" ]; then 
    read -p "A volume with this name already exists. `echo $'\n> '`Do you want to delete it and create it again linked to the specified path (y/n) ?" -n 1 -r
    echo
    if [[ $REPLY =~ ^[Yy]$ ]]
    then
        if [ -d "/var/lib/docker/volumes/$ARG1/_data" ]; then
            unlink /var/lib/docker/volumes/$ARG1/_data
            echo "_data folder unlinked!"
        fi
        docker volume rm $ARG1
        if [ $? -ne 0 ]
          then
            echo "Please stop the containers using this volume before launching the script"
            exit 1
        fi
        create_volume
    fi
else 
    create_volume
fi

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Immerse yourself in a wealth of knowledge with this piece, supported by the inclusive DEV Community—every developer, no matter where they are in their journey, is invited to contribute to our collective wisdom.

A simple “thank you” goes a long way—express your gratitude below in the comments!

Gathering insights enriches our journey on DEV and fortifies our community ties. Did you find this article valuable? Taking a moment to thank the author can have a significant impact.

Okay