Quick intro to Bash I/O redirections, *here documents* and *here strings*

Note: this article was originally published on my personal blog. Check it out here.

If you have done a bit of scripting in Bash, you may know that you can redirect the input and output of the commands to other commands or files. This article introduces the basic of I/O redirections and explains some of the syntax that may seems cryptic at first.

• stdin, stdout and stderr
• Controlling the program input and output
  • Controlling the input
    • Here documents and here strings
  • Controlling the output
    • Controlling stdout
    • Controlling stderr
    • A note on the pipe operator

stdin, stdout and stderr

In Bash every command or script has three files¹ opened when running:

1. stdin (STanDart INput): the input of the data fed to the program
2. stdout (STanDart OUTput): the location where the program will output anything it needs to output
3. stderr (STanDart ERRor): the location where the program will output any error occured during the run

Each of them get assigned to a specific file descriptor (ie. a number assigned to the file used by the OS to handle it): O for stdin, 1 for stdout and 2 for stderr. These numbers allow you to control and use them in your program.

Controlling the program input and output

Controlling the input

Depending on the programm that you want to run, there might be different ways to control on their input is provided: some are built to accept files, some will read only from stdin and others may accept both.

When it comes to directly providing files to the program, I'll let you read their manpage to know how to do so.

When your program is launched directly in a terminal, its stdin is attached to your keyboard, providind you with a way to input the data.

If you want to use something else as the input, there are a few ways:

• Using the < operator: this operator allows to use the content of a file as the input data stream.

$ cat >input-file <<EOF
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10
Line 11
EOF
$ grep "^Line 1" <input-file
Line 1
Line 10
Line 11

(See section below about here docs to understand the syntax of the first command.)

You can also use the 0< operator, which is equivalent to <. The 0 here is a reference to the file descriptor 0 attached to stdin, but because a program only has one stdout, it can be abreviated from 0< to <.

• Using the pipe | operator: the pipe operator is a special operator that combines input and output redirection to allow chaining commands. See below for more info.

• Using here documents and here strings

Here documents and here strings

Here documents (or heredoc) are a special form of input redirection allowing to input a multiline string to a program. The syntax is the following:

COMMAND <<DELIMITOR
...
...
DELIMITOR

DELIMITOR is used to control when to start and stop the input. It can be any string without spaces, and should then be choosed wiselly so that it won't apear in the input:

$ cat <<ThisIsADelimitor
Hello
There
ThisIsADelimitor
Hello
There
$ cat <<1
Hello
There
1
Hello
There

Heredoc allows for parameter expansion as well:

$ export FIRSTNAME='Obi-Wan'
$ export LASTNAME='Kenobi'
$ cat <<EOF
Hello $FIRSTNAME $LASTNAME
EOF
Hello Obi-Wan Kenobi

If you have played a bit with the heredocs, you may have noticed that tabs included in the input were kept:

$ cat <<EOF
        hello
there
EOF
    hello
there

Fortunately you can strip them by prefixing the delimitor with a dash:

$ cat <<-EOF
        hello
there
EOF
hello
there

This will strip all leading tabs, allowing the input to be more readable visually. Note though that this strips only leading tabs, not spaces, so be careful, especially if you copy/paste the example above.

Here strings

Here strings are a more consise version of the here documents. They only allow a single line input, and thus don't need the use of a string delimitor. Like the heredocs, it allows for parameter expansion:

$ cat <<EOF > file
This is
a
multiline
file
EOF
$ export TITLE="Title of the file"
$ export FILE="file"
$ cat - $FILE <<<$TITLE > $FILE.new
$ cat $FILE.new
Title of the file
This is
a
multiline
file

Controlling the output

Like for the input, there are a few ways to redirect the output of a program, but unlike the first, a program has two type of possible outputs, stdout and stderr, so there are ways to control which output gets redirected.

When the program is launched directly in a terminal, its stdout and stderr are attached to the terminal, so everything the program outputs gets printed on your screen.

Controlling stdout

• Using the > operator: redirects stdout to the provided filename, creates it if needed, otherwise overwrite it.

$ ls -la > list-files.txt

• Using the >> operator: same as >, but appends to the end of the file if it already exists

$ ls -lat | head -n3 >> most-recents-files.txt

Controlling stderr

• Using the 2> operator: same as > for stdout, but for stderr

$ export ERRORFILE=script.errors
$ bad_command1 2>$ERRORFILE
$ cat $ERRORFILE
zsh: command not found: bad_command1

• Using the 2>> operator: same as >> for stdout, but for stderr

$ export ERRORFILE=script.errors
$ bad_command1 2>>$ERRORFILE
$ cat $ERRORFILE
zsh: command not found: bad_command1

• 2>&1: redirects stderr to stdout

• &> filename: redirects both stdout and stderr to filename

A note on the pipe operator

The pipe operator | is special operator that allows to chain multiple commands by binding the stdout of one to the stdin of the following:

$ ls -la | wc -l
15

The example above would be roughly the same as the following script:

#!/bin/bash
filename=temp
ls -la >$filename
exec 3<$filename
wc -l <&3
3<&-
rm -rf $filename

Sources

• Advanced Bash-Scripting Guide: Chapter 20. I/O Redirection
• Advanced Bash-Scripting Guide: Chapter 19. Here Documents
• Advanced Bash-Scripting Guide: Chapter 19.1. Here Strings

---

1. While they may use regular files as input or output these are actually more like data streams, but by convention the UNIX and Linux world treat them like files. ↩