Pipes and Redirection: How >, >>, 2> and | Really Work

Four small symbols move text between commands and files. Once you see the three streams behind them, shell one-liners stop looking like magic.

Adapted from the Command Line Essentials Companion Guide.

---

At some point you copy a one-liner from a tutorial and it works, though you couldn't say why:

sort fruits.txt | uniq -c | sort -rn > counts.txt 2> errors.txt

It reads like line noise, but it's five small ideas in a row, and once you know them you can read almost any shell one-liner and write your own. The ideas all come from one fact: every command you run has three channels for text, and the shell lets you rewire them.

The three streams

Every program starts with three connections already open:

• Standard input (stdin): where it reads from. By default, your keyboard.
• Standard output (stdout): where its normal results go. By default, your terminal screen.
• Standard error (stderr): where its error messages go. Also your terminal screen by default.

Output and errors look identical on screen, which hides the fact that they're separate channels. That separation is what makes the rest of this work. Redirection changes where a stream points, and a pipe connects one command's output to the next command's input.

Sending output to a file: > and >>

> sends stdout to a file instead of the screen. If the file exists, it's replaced:

$ echo one > out.txt
$ echo two > out.txt
$ cat out.txt
two

The first line was overwritten. To add to the end instead, use >>:

$ echo three >> out.txt
$ cat out.txt
two
three

Use > when you want a fresh file each time, and >> for something like a log you keep adding to.

Sending errors somewhere: 2>

Errors go to stderr, which is stream number 2 (stdout is 1, stdin is 0). > only captures stdout, so errors still appear on screen. To capture them, redirect stream 2:

$ ls nope fruits.txt > found.txt 2> errors.txt
$ cat found.txt
fruits.txt
$ cat errors.txt
ls: cannot access 'nope': No such file or directory

The file that exists went to found.txt. The error for the one that doesn't went to errors.txt. To throw errors away entirely, send them to /dev/null, a special file that discards everything written to it:

ls nope fruits.txt 2> /dev/null

Combining both streams: 2>&1

To send errors to the same place as normal output, use 2>&1, which reads as "point stream 2 at wherever stream 1 is pointing right now":

$ ls nope fruits.txt > both.txt 2>&1
$ cat both.txt
ls: cannot access 'nope': No such file or directory
fruits.txt

The order matters, and this is the most common mistake with it. Redirections are applied left to right. In the version above, stdout goes to the file first, and then stderr is pointed at that same place. If you swap them:

$ ls nope fruits.txt 2>&1 > only.txt
ls: cannot access 'nope': No such file or directory
$ cat only.txt
fruits.txt

Now stderr was pointed at the screen (where stdout was at that moment), and only afterward was stdout moved to the file. The error still lands on your screen. Put > file first, then 2>&1.

Connecting commands: |

A pipe takes the stdout of the command on its left and feeds it to the stdin of the command on its right, with no file in between. That lets you chain small tools, each doing one job:

$ sort fruits.txt | uniq -c | sort -rn
      3 apple
      2 banana
      1 cherry

Read it left to right: sort the lines so duplicates sit together, count each run of duplicates, then sort by that count, highest first. None of these tools knows about the others. They each just read stdin and write stdout, and the pipe does the connecting.

One catch: a pipe only carries stdout. Errors from the left-hand command skip the pipe and still print on screen. If you want them to go through, merge them first:

$ ls nope fruits.txt | wc -l
ls: cannot access 'nope': No such file or directory
1
$ ls nope fruits.txt 2>&1 | wc -l
2

Saving and viewing at once: tee

Redirecting to a file hides the output from your screen. tee copies its input to both the screen and a file, which is useful when you want to watch a long command and keep a record:

$ echo hi | tee t.txt
hi

Two mistakes worth knowing about ahead of time

Redirecting a file onto itself. The shell opens, and empties, the output file before the command even starts. So this wipes your data:

$ sort fruits.txt > fruits.txt
$ wc -c fruits.txt
0 fruits.txt

The file was emptied first, so sort had nothing to read. Write to a different file, then rename it if you want to replace the original.

Using sudo with a redirect. If you run sudo echo text > /etc/somefile and get "Permission denied", it's because the > is handled by your shell, which isn't running as root. Only the echo got elevated, and it was never the part that needed it. Pipe into tee instead, which runs under sudo and does the writing: echo text | sudo tee /etc/somefile.

A habit that makes it click

When you see a one-liner you don't understand, read it in pieces rather than as a whole. Run the first command alone and look at what it prints. Then add the next | and see what that stage changes. Because every stage just takes text in and sends text out, you can build the pipeline up one step at a time, checking each result before moving on. That's also how the people who write these one-liners build them in the first place.

---

If you'd like more posts like this sent straight to your inbox, subscribe to the newsletter.

Prefer to dig in yourself? The Command Line Essentials repo on GitHub has more free examples and exercises.