Docker in Docker, managing existant volumes

Introducing the context, I am working with a docker container that runs jenkins, inside it there are some other docker services running aswell, brought up during some jenkins job execution.

As an existant volume I refer to a folder containing data that I will mount to the container, for instance :

$ docker run ... -v /folder/with/data:/var/data ...

The command I use to run jenkins is the following one :

$ docker run \                                        
  -d \               
  -u  \
  -p 8080:8080 \
  -v /data/jenkins_test:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  shimmi/jenkins

As you can see the docker socket, which is the main entry point for docker API is mapped as a volume into the container, that means, that the docker engine will be "shared" (it will be the same docker socket) and volumes, images and containers will be shared inside and outside the container.

This is a relevant point that caused me a lot of headache trying to figure why once I run a container inside jenkins, the path specified as existant volumes when creating containers were not found or empty.

The problem was that the path I was specifying was the correct for the jenkins container but Docker was trying to search for it in the host that runs the jenkins container (my machine filesystem) not in the jenkins container itself.

About volumes and docker-compose

Specifically the services I bring up are specified in a docker-compose file, in order to use existant volumes from the host you have to declare a top level volumes option and set the external property as true :

volumes:
  <myVolume>:
    external: true

And now, if this is an existant volume in your docker host you should be able to use it.

Creating volumes from existing data folders

In my case I wanted to create a volume and "link" it to some folder with data, not an empty volume, in order to do that I followed these steps :

$ docker volume create <myVolume>
$ docker volume inspect <myVolume> 
[
    {
        "CreatedAt": "0001-01-01T00:00:00Z",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/<myVolume>/_data",
        "Name": "<myVolume>",
        "Options": {},
        "Scope": "local"
    }
]
# See the mountpoint is the path we will link
$ sudo rm -rf /var/lib/docker/volumes/<myVolume>/_data
$ sudo ln -s <MY_FOLDER_WITH_DATA> /var/lib/docker/volumes/<myVolume>/_data

That way, you can bring up containers inside another containers (docker in docker) managing volumes with data in an easy way.

Bash script

In order to automate the task of creating a volume and the symlink I wrote a bash script :

#!/bin/bash

# This script creates a docker volume and links it to a existing folder that contains data.

# Arg1 : volume name
# Arg2 : existant folder with data to create the volume

ARG1=$1
ARG2=$2

function create_volume {
    echo '$1 = ' $ARG1
    echo '$2 = ' $ARG2
    docker volume create $ARG1
    if [ $? -ne 0 ]
      then
        echo "Script failed"
        exit 1
    fi
    echo "Volume created!"
    sudo rm -rf /var/lib/docker/volumes/$ARG1/_data
    sudo ln -s $ARG2 /var/lib/docker/volumes/$ARG1/_data
    if [ $? -ne 0 ]
      then
        echo "Script failed"
        exit 1
    fi
    echo "Folder linked"
    echo "Script executed succesfully"
}

if [ $# -eq 0 ]
  then
    echo "No arguments supplied, specify the volume name and the path of the folder"
    exit 1
fi

if [ $# -eq 1 ]
  then
    echo "Not enough arguments supplied, specify the volume name and the path of the folder"
    exit 1
fi

EXISTS="$(docker volume ls -f name=$ARG1 | grep -w ARG1)"
if [ -z "$EXISTS" ]; then 
    read -p "A volume with this name already exists. `echo $'\n> '`Do you want to delete it and create it again linked to the specified path (y/n) ?" -n 1 -r
    echo
    if [[ $REPLY =~ ^[Yy]$ ]]
    then
        if [ -d "/var/lib/docker/volumes/$ARG1/_data" ]; then
            unlink /var/lib/docker/volumes/$ARG1/_data
            echo "_data folder unlinked!"
        fi
        docker volume rm $ARG1
        if [ $? -ne 0 ]
          then
            echo "Please stop the containers using this volume before launching the script"
            exit 1
        fi
        create_volume
    fi
else 
    create_volume
fi