DEV Community

Cover image for Mastering SCP: A Comprehensive Guide to Secure File Transfers and Advanced Features
Richard Chamberlain
Richard Chamberlain

Posted on

Mastering SCP: A Comprehensive Guide to Secure File Transfers and Advanced Features

SCP (Secure Copy Protocol), much like SSH, is a cornerstone technology for managing remote servers. While SSH enables access to a remote server's command line, SCP extends this functionality by allowing the transfer of files and directories to and from a remote server’s file system. The majority of SCP commands follow a straightforward syntax:

# Put a file on a remote server
scp <file name> <user>@<remote server>:<path to put file>

# Get a file from a remote server
scp <user>@<remote server>:<file with full path> <where to put it locally>
Enter fullscreen mode Exit fullscreen mode

This simple approach is commonly used, but SCP offers a host of additional features that can make it even more versatile.
Here’s a table of contents for your polished article:

Table of Contents

  1. Introduction
  2. Moving a Directory
  3. Bandwidth Limiting
  4. Compression
  5. Verbose Mode
  6. Ports, Proxies, and SSH Keys
  7. Preserving File Attributes
  8. Conclusion

Each section heading in the article can be directly linked using the corresponding anchor tags. Let me know if you'd like help formatting the article with these links or making further adjustments!

Moving a Directory

Sometimes, transferring entire directories—complete with their contents—is necessary. SCP supports this with the -r option, which recursively copies entire directories. The commands for this are similarly intuitive:

# Put a directory and its contents on a remote server
scp -r <directory> <user>@<remote server>:<path to put the directory>

# Get a directory from a remote server
scp -r <user>@<remote server>:<directory with full path> <where to put it locally>
Enter fullscreen mode Exit fullscreen mode

Bandwidth Limiting

A less commonly used but critical feature of SCP is its ability to limit bandwidth usage. This is particularly useful when transferring large files during business hours, as excessive network traffic can degrade performance for others. The -l option allows you to specify a bandwidth limit in kilobits per second, which can help mitigate such issues. For example, to limit SCP to 5 Mbps:

BITS_TO_BYTES=8
KILO_TO_MEGA=1024
SPEED_IN_MEGA=5
scp -l $((${BITS_TO_BYTES}*${KILO_TO_MEGA}*${SPEED_IN_MEGA})) <file name> <user>@<remote server>:<path to put file>
Enter fullscreen mode Exit fullscreen mode

Additionally, SCP includes the -B option to adjust the buffer size, which determines how much data is read and written in a single operation. The default is 32 KB, but larger sizes can improve performance on high-speed networks, while smaller sizes may be better for low-speed or high-latency networks.

Compression

If network performance is a concern, compression can also be employed using the -C option. This compresses the data during transfer, which can speed up file transfers over slower connections:

scp -C <file name> <user>@<remote server>:<path to put file>
Enter fullscreen mode Exit fullscreen mode

Verbose Mode

When troubleshooting SCP issues, verbose mode is invaluable. The -v option provides detailed output of what SCP is doing, with additional levels of verbosity available using -vv or -vvv. A common approach to debugging involves first testing connectivity with SSH, as SCP issues are often rooted in SSH configuration problems:

scp -v <file name> <user>@<remote server>:<path to put file>
Enter fullscreen mode Exit fullscreen mode

Ports, Proxies, and SSH Keys

SCP supports advanced options like specifying a custom port (-P), using proxy servers (-o), and providing SSH keys (-i). While these can be included in the SCP command, a more efficient approach is to define them in your .ssh/config file. This simplifies commands, documents your setup for future reference, and applies the same settings to both SCP and SSH. Here’s an example configuration:

Host remote
    HostName remote.example.com
    User myuser
    ProxyJump firewall_jump           ## Proxy server - defined in another SSH config
    Port 2222                         ## Port
    IdentityFile ~/.ssh/keys/auth_key ## SSH keys
Enter fullscreen mode Exit fullscreen mode

With this configuration, the SCP command becomes much simpler:

scp <file name> <remote>:<path to put file>
Enter fullscreen mode Exit fullscreen mode

Transferring files between two remote hosts can also benefit from this approach. Define both hosts in your .ssh/config, and the SCP command is just as straightforward:

scp <remote from server>:<file name> <remote to server>:<path to put file>
Enter fullscreen mode Exit fullscreen mode

Preserving File Attributes

When transferring files, preserving attributes such as ownership and permissions can be crucial. The -p option ensures these attributes remain intact, making it ideal for tasks like backups, file system migrations, and compliance:

scp -p <file name> <user>@<remote server>:<path to put file>
Enter fullscreen mode Exit fullscreen mode

Conclusion

This guide covered some of the most useful features of SCP, demonstrating its flexibility beyond basic file transfers. If you need to achieve something not listed here, I recommend checking the manual pages (man scp) or reaching out—I’d be happy to help!

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Engage with a sea of insights in this enlightening article, highly esteemed within the encouraging DEV Community. Programmers of every skill level are invited to participate and enrich our shared knowledge.

A simple "thank you" can uplift someone's spirits. Express your appreciation in the comments section!

On DEV, sharing knowledge smooths our journey and strengthens our community bonds. Found this useful? A brief thank you to the author can mean a lot.

Okay