DEV Community

loading...

Transferring data across docker or podman volumes

Jeet Parekh
Solutions Architect | DevOps Engineer | Data Engineer
Updated on ・2 min read

I often find myself starting temporary containers to transfer data between the local file-system and container volumes. Or sometimes, between volumes. And the entire process is a bit tedious. Starting a temporary container, mounting the volume, mounting the local file-system, and finally transferring the data.

So I made a python script which makes this easy. I call it cvcp - container volume cp (copy). It actually does not use cp though. It uses rsync for the data transfer. So it's necessary to have a container image with rsync installed.

You can find the script at https://github.com/jeet-parekh/cvcp. The repository also contains a Dockerfile which will build an image with rsync installed.


Using cvcp

Download the file cvcp from the repository and place it inside your preferred path. I prefer ~/bin.

Then, to use cvcp:

cvcp SOURCE DESTINATION
Enter fullscreen mode Exit fullscreen mode

Path inside a container volume should be written as <volume_name>:<path_inside_volume>.

Either SOURCE or DESTINATION can be a local path or a path inside a container volume. Note however that both SOURCE and DESTINATION cannot be local paths.

If you use cvcp without any options, it will pull a rsync image from Docker Hub, and will use docker with root (sudo docker).

The following command line options are available to customise the behaviour:

  • --docker and --podman

    • The container engine to use.
    • The default is docker.
  • --rootless

    • Use the container engine without root (without sudo).
  • --image

    • The container image to be used.
    • Note that the image needs to have rsync installed. Read the next section for more details.
  • --rsync-opts

    • Options to pass to rsync.
    • Note that, you need to pass all the options as a quoted string with a space at the beginning. For example, ' --verbose'.
    • Using this option will clear the default rsync options used by cvcp, so make sure to pass all the options required.
    • The default options are ' -a --progress'.
  • --no-exec

    • Print the command and exit.

Using a custom image

To use a custom image with cvcp, use the --image argument.

To permanently change the default image that cvcp uses, change this line in the cvcp script file:

RSYNC_IMAGE = "image_name"
Enter fullscreen mode Exit fullscreen mode

It is necessary for your custom image to have rsync installed. You can use the Dockerfile inside the same repository as a base to build the image.

# docker
sudo docker build -t localhost/rsync .

# podman
podman build -t localhost/rsync .
Enter fullscreen mode Exit fullscreen mode

Discussion (0)