DEV Community

Major
Major

Posted on

I Brought rsync to kubectl

#kubernetes #devops #kubectl

Over the last few months, I've been working with Kubernetes pods in my development environment. One thing I frequently need to do is transfer files from my local machine into a development pod.

The usual choice is kubectl cp:

kubectl cp ./local-dir my-dev-pod:/tmp/remote-dir
Enter fullscreen mode Exit fullscreen mode

It works, but I wanted something better for a development workflow.

When I'm making changes locally and syncing them to a pod repeatedly, I don't want to transfer everything every time. I want rsync's incremental synchronization that is only the files that have changed should be transferred.

So I started wondering:

Could I make rsync work with kubectl?

The rsync -e option

It turns out that rsync has exactly the mechanism I needed: the -e option.

The -e option allows rsync to specify the remote shell command it should use for connecting to the remote machine.

For example, I created a small rsh.sh script:

#!/bin/bash

POD="$1"
shift

kubectl exec -i "$POD" -- "$@"
Enter fullscreen mode Exit fullscreen mode

Then I could tell rsync to use this script as its remote shell:

rsync -avzP -e ./rsh.sh \
  ~/major1201/local-dir/ \
  my-dev-pod:/tmp/remote-dir/
Enter fullscreen mode Exit fullscreen mode

And it worked.

sending incremental file list
./
1.txt
              4 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=2/4)
b/
b/2.txt
              4 100%    3.91kB/s    0:00:00 (xfr#2, to-chk=0/4)

sent 233 bytes  received 65 bytes  198.67 bytes/sec
total size is 8  speedup is 0.03
Enter fullscreen mode Exit fullscreen mode

The important part wasn't the transfer itself. It was that rsync was doing the synchronization while kubectl exec provided the connection to the pod.

The flow looked like this: rsync -> rsh.sh -> kubectl exec -> kubernetes Pod

And that's the idea behind kubectl-rsync

Once I had the proof of concept working, I packaged the idea into a kubectl plugin.

Now the same operation can be written as:

kubectl rsync \
  ~/major1201/local-dir/ \
  my-dev-pod:/tmp/remote-dir/
Enter fullscreen mode Exit fullscreen mode

Under the hood, kubectl-rsync acts as an rsync transport shim. It connects rsync to the pod through kubectl exec, allowing rsync to handle the file synchronization.

This means I can use rsync features such as incremental transfers, compression, progress reporting, and partial transfers while still using my existing Kubernetes authentication and kubectl workflow.

Install it with Krew

If you already have Krew installed, you can install the plugin with:

kubectl krew install rsync
Enter fullscreen mode Exit fullscreen mode

Try it out

The project is open source:

https://github.com/major1201/kubectl-rsync

If you work with Kubernetes pods and frequently move files between your local machine and development environments, I'd love to hear what you think.

If you find it useful, consider giving the project a star on GitHub.

Top comments (0)