Handy Sed One-Liners for Text Manipulation and Processing

FILE SPACING:

Double space a file:

sed G

Double space a file which already has blank lines, output file should contain no more than one blank line between lines of text:

sed '/^$/d;G'

Triple space a file:

sed 'G;G'

Undo double-spacing (assumes even-numbered lines are always blank):

sed 'n;d'

Insert a blank line above every line which matches "regex":

sed '/regex/{x;p;x;}'

Insert a blank line below every line which matches "regex":

sed '/regex/G'

Insert a blank line above and below every line which matches "regex":

sed '/regex/{x;p;x;G;}'

NUMBERING:

Number each line of a file (simple left alignment):

sed = filename | sed 'N;s/\n/\t/'

Number each line of a file (number on left, right-aligned):

sed = filename | sed 'N; s/^/     /; s/ *\(.\{6,\}\)\n/\1  /'

Number each line of file, but only print numbers if line is not blank:

sed '/./=' filename | sed '/./N; s/\n/ /'

Count lines (emulates "wc -l"):

sed -n '$='

TEXT CONVERSION AND SUBSTITUTION:

Convert DOS newlines (CR/LF) to Unix format:

sed 's/.$//'

Convert Unix newlines (LF) to DOS format:

sed "s/$/`echo -e \\\r`/"

Delete leading whitespace (spaces, tabs) from front of each line:

sed 's/^[ \t]*//'

Delete trailing whitespace (spaces, tabs) from end of each line:

sed 's/[ \t]*$//'

Delete both leading and trailing whitespace from each line:

sed 's/^[ \t]*//;s/[ \t]*$//'

Insert 5 blank spaces at beginning of each line (make page offset):

sed 's/^/     /'

Align all text flush right on a 79-column width:

sed -e :a -e 's/^.\{1,78\}$/ &/;ta'

Center all text in the middle of 79-column width:

sed  -e :a -e 's/^.\{1,77\}$/ & /;ta'

Substitute (find and replace) "foo" with "bar" on each line:

sed 's/foo/bar/'

Substitute "foo" with "bar" EXCEPT for lines which contain "baz":

sed '/baz/!s/foo/bar/g'

Reverse order of lines (emulates "tac"):

sed '$!G;h;$!d'

Join pairs of lines side-by-side (like "paste"):

sed '$!N;s/\n/ /'

If a line ends with a backslash, append the next line to it:

sed -e :a -e '/\\$/N; s/\\\n//; ta'

Add a blank line every 5 lines:

gsed '0~5G'

SELECTIVE PRINTING OF CERTAIN LINES:

Print first 10 lines of file (emulates behavior of "head"):

sed 10q

Print first line of file (emulates "head -1"):

sed q

Print the last 10 lines of a file (emulates "tail"):

sed -e :a -e '$q;N;11,$D;ba'

Print only lines which match regular expression (emulates "grep"):

sed -n '/regexp/p'

Print only lines which do NOT match regexp (emulates "grep -v"):

sed '/regexp/!d'

Print paragraph if it contains AAA:

sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;'

Print section of file from regular expression to end of file:

sed -n '/regexp/,$p'

SELECTIVE DELETION OF CERTAIN LINES:

Print all of file EXCEPT section between 2 regular expressions:

sed '/Iowa/,/Montana/d'

Delete duplicate, consecutive lines from a file (emulates "uniq"):

sed '$!N; /^\(.*\)\n\1$/!P; D'

Delete all blank lines from a file:

sed '/^$/d'

Delete all consecutive blank lines from file except the first:

sed '/./,/^$/!d'

Delete all trailing blank lines at end of file:

sed -e :a -e '/^\n*$/{$d;N;ba' -e '}' 

SPECIAL APPLICATIONS:

Remove nroff overstrikes (char, backspace) from man pages:

sed "s/.`echo \\\b`//g"

Get Usenet/e-mail message header:

sed '/^$/q'

Get Usenet/e-mail message body:

sed '1,/^$/d'

Add a leading angle bracket and space to each line (quote a message):

sed 's/^/> /'

Delete leading angle bracket & space from each line (unquote a message):

sed 's/^> //'

Remove most HTML tags:

sed -e :a -e 's/<[^>]*>//g;/</N;//ba'

Extract multi-part uuencoded binaries:

sed '/^end/,/^begin/d' file1 file2 ... fileX | uudecode

SORTING PARAGRAPHS OF FILE ALPHABETICALLY:

sed '/./{H;d;};x;s/\n/={NL}=/g' file | sort | sed '1s/={NL}=//;s/={NL}=/\n/g'

OPTIMIZING FOR SPEED:
Substitution executes more quickly if the "find" expression is specified before the "s/.../.../" instruction:

sed '/foo/ s/foo/bar/g' filename