DEV Community

kohei_wada
kohei_wada

Posted on Originally published at wada-dev.com

Notes on the tar + ssh pipe technique

#linux #devops #bash

I can never quite recall the exact command when I actually need it, so I'm parking it here for copy-paste.

TL;DR (for the impatient) 

# Pull from remote to local (with sudo)
ssh host "sudo tar cz /var/log/nginx" | tar xzv --strip-components=3

# Push from local to remote
tar cz . | ssh host "tar xz -C /path/to/dest"
Enter fullscreen mode Exit fullscreen mode

What happened

I needed to pull logs off an old CentOS server.

Tried rsync:

-bash: rsync: command not found
Enter fullscreen mode Exit fullscreen mode

The box is EOL and its repos are dead, so yum install rsync won't work either.

Tried scp:

scp: /var/log/nginx/access.log: Permission denied
Enter fullscreen mode Exit fullscreen mode

The log directory needs root, and scp gives you no way to slip a sudo in there.

This is exactly where the tar + ssh pipe earns its keep.

When to reach for it

  • rsync isn't installed (old servers, minimal containers)
  • scp can't read the file because of permissions (sudo is required)
  • You don't want to leave a temp file on the remote
  • You want to compress on the fly to save bandwidth

Basic patterns

Remote → local 

ssh host "tar cz /path/to/dir" | tar xz
Enter fullscreen mode Exit fullscreen mode

The remote /path/to/dir gets extracted into your current local directory.

When sudo is required 

ssh host "sudo tar cz /var/log/nginx" | tar xzv --strip-components=3
Enter fullscreen mode Exit fullscreen mode

--strip-components=3 chops off the first three path components of /var/log/nginx/... so it extracts as nginx/....

Choosing the extraction target 

ssh host "tar cz /path/to/data" | tar xz -C ./backup/
Enter fullscreen mode Exit fullscreen mode

Local → remote

Send a directory and extract it on the other side:

tar cz . | ssh host "tar xz -C /path/to/dest"
Enter fullscreen mode Exit fullscreen mode

Save as an archive file instead:

tar cz directory | ssh host "cat > backup.tar.gz"
Enter fullscreen mode Exit fullscreen mode

Variations

Show progress 

tar cz /data | pv | ssh host "tar xz -C /backup"
Enter fullscreen mode Exit fullscreen mode

Slotting pv into the pipe gives you a live transfer rate.

Fan out to multiple servers at once 

tar cz /deploy | tee >(ssh server1 "tar xz -C /app") \
                      >(ssh server2 "tar xz -C /app") > /dev/null
Enter fullscreen mode Exit fullscreen mode

Gotchas

Counting --strip-components

It strips N leading path components.

/var/log/nginx/access.log
  │    │    │
  1    2    3  ← strip-components=3 leaves you with nginx/access.log
Enter fullscreen mode Exit fullscreen mode

Where -C goes

It belongs on the extracting side (right-hand side of the pipe).

# OK
ssh host "tar cz /path" | tar xz -C ./dest/

# Pointless — `-C` on the compression side doesn't change where it lands
ssh host "tar cz -C /path ." | tar xz
Enter fullscreen mode Exit fullscreen mode

Compression options

Option Format When to use
z gzip Usually all you need
j bzip2 When you care about ratio over speed
J xz Best ratio, slowest

scp vs rsync vs tar + ssh

Situation Pick
One-off small file scp
Recurring sync / incremental rsync
No rsync available / sudo required tar + ssh

Wrap-up

  • tar + ssh is the fallback when rsync isn't available
  • It can grab files that are only readable through sudo
  • --strip-components lets you trim the path depth
  • Streaming transfer — no temp file left behind

This post was written with AI assistance and reviewed by the author.

Top comments (0)