Transferring data across docker or podman volumes

Jeet Parekh — Wed, 26 May 2021 14:05:30 +0000

I often find myself starting temporary containers to transfer data between the local file-system and container volumes. Or sometimes, between volumes. And the entire process is a bit tedious. Starting a temporary container, mounting the volume, mounting the local file-system, and finally transferring the data.

So I made a python script which makes this easy. I call it cvcp - container volume cp (copy). It actually does not use cp though. It uses rsync for the data transfer. So it's necessary to have a container image with rsync installed.

You can find the script at https://github.com/jeet-parekh/cvcp. The repository also contains a Dockerfile which will build an image with rsync installed.

Using cvcp

Download the file cvcp from the repository and place it inside your preferred path. I prefer ~/bin.

Then, to use cvcp:

cvcp SOURCE DESTINATION

Path inside a container volume should be written as <volume_name>:<path_inside_volume>.

Either SOURCE or DESTINATION can be a local path or a path inside a container volume. Note however that both SOURCE and DESTINATION cannot be local paths.

If you use cvcp without any options, it will pull a rsync image from Docker Hub, and will use docker with root (sudo docker).

The following command line options are available to customise the behaviour:

--docker and --podman
- The container engine to use.
- The default is docker.
--rootless
- Use the container engine without root (without sudo).
--image
- The container image to be used.
- Note that the image needs to have rsync installed. Read the next section for more details.
--rsync-opts
- Options to pass to rsync.
- Note that, you need to pass all the options as a quoted string with a space at the beginning. For example, ' --verbose'.
- Using this option will clear the default rsync options used by cvcp, so make sure to pass all the options required.
- The default options are ' -a --progress'.
--no-exec
- Print the command and exit.

Using a custom image

To use a custom image with cvcp, use the --image argument.

To permanently change the default image that cvcp uses, change this line in the cvcp script file:

RSYNC_IMAGE = "image_name"

It is necessary for your custom image to have rsync installed. You can use the Dockerfile inside the same repository as a base to build the image.

# docker
sudo docker build -t localhost/rsync .

# podman
podman build -t localhost/rsync .

DEV Community: Jeet Parekh

Transferring data across docker or podman volumes

Using cvcp

Using a custom image