DEV Community 👩‍💻👨‍💻

Do Hoang
Do Hoang

Posted on

Learning Sed Is Beneficial For Linux Users

One of the most important command line utility in Linux is sed which is a stream editor.
A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline).

Essentially what it can do is it allows you to search for String of text or specific pattern and then replace it pattern with whatever you tell sed to replace with, so it is really kind of search and replace function.

Basic sed substitution 's'

sed 's/find/replace/' <oldfile >newfile
Enter fullscreen mode Exit fullscreen mode

So we are telling sed that go search for the word find then replace it with the word replace and what it does is it searches every lines for its first instance of the word find. So it not gonna replace every instances of find, only the first time it appears on each line from oldfile and then writes it to newfile.

Sometimes you want to replace every find in oldfile you just need to add global substitution g for every instances find.

sed 's/find/replace/g' <oldfile >newfile
Enter fullscreen mode Exit fullscreen mode

But What if you just want to take a file, change it then write to that same file, you could use -i flag.
It will replace all occurrences of a string in a file, overwriting the file (i.e. in-place):

sed -i 's/find/replace/g' filename
Enter fullscreen mode Exit fullscreen mode

You are not often doing this as far taking input from a file and then taking the output and write to new file. Probably most of the time, you take the output from other shell command and pipe it to sed command. For example:

echo "Thank Do Hoang" | sed 's/Ho/W/'
Thank Do Wang
Enter fullscreen mode Exit fullscreen mode

Other basic usage:

  • To replace only on lines matching the line pattern:
sed '/line_pattern/s/find/replace/' filename
Enter fullscreen mode Exit fullscreen mode
  • Delete lines matching the line pattern:
sed '/line_pattern/d' filename
Enter fullscreen mode Exit fullscreen mode
  • Print the first 11 lines of a file:
sed 11q filename
Enter fullscreen mode Exit fullscreen mode
  • Apply multiple find-replace expressions to a file:
sed -e 's/find/replace/' -e 's/find/replace/' filename
Enter fullscreen mode Exit fullscreen mode
  • Replace separator / by any other character not used in the find or replace patterns, e.g. #:
sed 's#find#replace#' filename
Enter fullscreen mode Exit fullscreen mode

What if I want to search for multiple pattern?

cat /etc/shells | sed -e 's/usr/u/g' -e 's/bin/b/g'
Enter fullscreen mode Exit fullscreen mode

Sed is not picky about the symbol that you use for the separator in the
command, if you dont want to use the /, you can use pretty much any others kind of special symbol that you can think of as long as it is not part of the search pattern.

cat /etc/shells | sed -e 's|usr|u|g' -e 's#bin#b#g'
Enter fullscreen mode Exit fullscreen mode

You can also print specific line that match the search pattern ( kind of grep )

cat /etc/shells | sed -n '/usr/p'
Enter fullscreen mode Exit fullscreen mode

The cool trick about sed is to go and delete extra spaces at the end of the line, for example

sed -i 's/ *$//' test.sh
Enter fullscreen mode Exit fullscreen mode

It will search for every lines that has spaces at the end of it and replace it with nothing (basically deleting them).
How about tabs?

sed -i 's/[[:space:]]*$//' test.sh
Enter fullscreen mode Exit fullscreen mode

empty lines?

cat test.sh | sed '/^$/d'
Enter fullscreen mode Exit fullscreen mode

First the ^ is the begining of the line then the $ symbol is the end of the line ( find every pattern that match the begining of the line, and the end of the line and there is nothing between it - so that means that line is empty ).

You can also replace all the lowercase characters with uppercase characters for fun 🙃

sed 's/[a-z]\/U&/g' test.sh // uppercase
sed 's/[a-z]\/L&/g' test.sh // lowercase
Enter fullscreen mode Exit fullscreen mode

C'est tout, merci kn 🙃

Top comments (2)

Collapse
 
epsi profile image
E.R. Nurwijayadi

Good article. Thank you for posting.

Sed can also be utilized to solve simple data structure, such as flatten data or something similar.

To help more beginner, I have made a working example of sed with source code in github.

🕷 epsi.bitbucket.io/lambda/2021/02/2...

First the data source, as CSV like, as an example case. Then a few sed label.

Utilizing sed to flatten array

🙏🏽

Thank you for posting this general introduction.

Collapse
 
netteyoder profile image
NetteYoder

What are the basics of using the Said Stream Editor to manipulate text in Linux? best work you have done here . Spell for breakup

Update Your DEV Experience Level:

Settings

Go to your customization settings to nudge your home feed to show content more relevant to your developer experience level. 🛠