Linux's commands and tricks I'm using in my daily job as a developer

This is not a post from the series of those describing the cd command. It's just a list of commands and tricks I'm using (almost) every day.

Port forwarding

Sometimes I have to connect to database and of course I prefer to use my GUI manager (JetBrains DataGrip).
So, if security policy exist in your company and your database's port is not exposed you can execute

ssh -L{port on your PC}:localhost:{database's port} root@{server IP}

The command below will open port 3308 on your laptop and everything will be forwarded to 192.168.1.2:3306

ssh -L3308:localhost:3306 root@192.168.1.2

localhost means that database is listening on 192.168.1.2. You can type for example 192.168.3.77 and everything will be forwarded to .3.77 server via .1.2.

Edit file in VIM without sudo, but save with sudo

Have you ever edited some configs file and forgot to sudo? Me too... There is a trick to save the file anyway, just type in VIM:

:w !sudo tee %

Explanation

Go to beggining/end of line in terminal

If you wrote a very long command in the terminal it may take a long time before you return to the begging of the line to add missing sudo. And back to the end to add some parameters.
Press crtl + a to move to the begging and crtl + e to the end of the line in terminal.

ll

Save few days in a year by typing ll instead of ls -la. Works on most Linux servers.

Execute command you executed in the past

Last command

To execute last command over again you can of course press ↑ (arrow up) key. But you can also type !!. So executing last command as a root is very easy

sudo !!

To run the last command that started with apt type !apt

Search history

To find the command that contains /tmp you have executed in the past press ctrl + r and type /tmp. Press ctrl + r again for next result.
To show all commands or to search using regular expression use

history | grep "/tmp"

Agree for everything

To say yes for each question you can use application called yes

yes | yum install curl

use yes no to say no and discard.

---

WARNING

As @patricnox notices in the comment - using yes may do unexpected things. You can accidentally install 10 GB of dependencies or other things you don't want to do.

---

Run a long-lasting process in the background and close the terminal

If you run a script that will end in 3 days, you don't have to wait with the terminal window open to end. You can run it using nohup command

nohup wget http://large-files.com/10gb-super-movie.avi &

wget works in the background, output is saved to nohup.out file in working directory.

Checking who has stolen your favourite port

It's really annoying when you are trying to run nginx but you can't because there is already apache running and port 443 is busy.
So, how to determinate which process is listening on port 80:

$ netstat -tulpn | grep 80
tcp6       0      0 :::80                 :::*                   LISTEN     10177/java

10177 is a pid you are looking for. Now execute

ps aux | grep 10177

for more details.

Reading logs

Everyone knows that less is a very good way to read a logs files. But you can also read gziped logs without extracting!

less /var/log/my-app/my-app.log.2015.12.14.gz

Live reading

tail -f /var/log/my-app/my-app.log | grep ERROR

The command above will show only new lines that contains ERROR.

Sort process

Show top 3 processes sorted by CPU usage

ps aux --sort=-pcpu | head -n 4

Show top 3 processes sorted by memory usage

ps aux --sort=-rss | head -n 4

Executing command every X seconds

To print command's output every X seconds you can use watch command. For example to create clock run

watch -n 1 date

Quiet mode

A lot of standards commands has quiet or silent mode. Very useful when you are creating some bash script. In most of the cases just add -q or -s (read --help or man or check on StackOverflow)

zip -q archive.zip big-file.jpg

But sometimes (practically always with in-house scripts) you have to ignore the output (send to /dev/null)

./very-verbose.sh 1>/dev/null

Create log files for scripts executed by crontab

0 22 * * 1-5 /opt/scripts/send-report.sh 2>/var/log/scripts/report-error.log

So next time when your script will fail you won't lose the reason

Comments

[simonced]

Those are great tips.

In the terminal, one trick I like is going back to previous folder with

cd -

(that is a minus sign)

Say, you are in /var/logs and you cd /data/dev/myproject, then you can go back to logs with cd -.

Useful in some situations only, but quite handy nonetheless.

[Charles Banas]

If you're editing config files with vim and need to use that sudo tee hack, stop.

You should be using sudoedit. It safely copies the config to /tmp and runs $EDITOR as your regular user with your user config, and only overwrites the file you're editing if you actually save. This is far more desirable than running your editor as root.

[Maxime Moreau]

Very good point, I think that we should never use sudo vim.

[Patryk]

alt + . is a great one too, in bash.

$ mkdir -p /tmp/some/nested/directory
$ cd <alt+.> # types the LAST argument - /tmp/some/nested/directory

Re: netstat, it still works, of course, and you can continue using it, but it is deprecated in favour of ss.

Re: sshing into a DB server as root, no offence, but I don't believe your company actually has a security policy :D (hopefully it was just an example).

[Mateusz Jarzyna]

Re: sshing into a DB server as root, no offence, but I don't believe your company actually has a security policy :D (hopefully it was just an example).

Yup, it was a 'funny' example

[Guillermo Tobar]

For years I looked for this, and finally found it. Thank you very much for your contribution Patryk.

[PatricNox]

Working with drush, the last one is very neat!

I have a concern regarding "Yes". I find it a bad idea to use this since what if you encounter a new tool? Maybe you install something and Yes makes it install, or maybe skip, dependencies that's crucial to have/not have in the project

(Vague example scenario, but you get the point!)

[Ben Sinclair]

A lot of commands take a -y flag too.

If you're particularly concerned about it doing something you don't want, you can use an expect script instead. It's kind of like selenium for the command line.

[PatricNox]

Yep! I tend to use the y flag.

[Mateusz Jarzyna]

Thanks, PatricNox! I've added little warning in the post

[Mateusz Jarzyna]

Yup, you are right. yes program may do unexpected thing, but sometimes you do know the script very well and you know all the questions.
Maybe I should change the example and add some warning

[Thomas H Jones II]

Last command
To execute last command over again you can of course press ↑ (arrow up) key. But you can also type !!. So executing last command as a root is very easy

While !! is great, the overall ! method becomes really powerful if you understand that you're neither limited to re-executing just the last command or re-executing the command relatively unmodified.

• Previously executed ssh host1.my.domain and now need to connect to host2.my.domain? Execute either of ^host1^host2^ or !!:s/host1/host2
• Related to the latter, if you execute history and see that you previously connected to host1 with the 23rd command in your history-buffer, you could do !23:s/host1/host2
• Have a command in your history that had a repeating string that you'd like to substitute all values for? !43:gs/orig_string/new_string

Agree for everything

I would be suuuuuuuper leery about developing habits around the yes command. Great sadness can ensue from habituating to its use ...especially if you have privileged access to a system. For commands that implement a built-in auto-yes feature (e.g. your yum does via the -y flag). Even there, I'd tend to avoid auto-yes except in the very specific context of scripted routines where you've validated the the auto-acknowleged behavior always acts the way you expect and need it to.

It's also worth noting that not every command you use will understand accepting a yes from piped <STDIN>

Run a long-lasting process in the background and close the terminal

There are a non-trivial number of commands that don't react well to being backgrounded. You might think, "lemme background this thing and let it go about its business" only to come back minutes/hours/days later to find that it's done nothing. Basically, whenever you background a command, it's always a good idea to run the jobs command immediately afterwards to verify that it's in a running state. You may, instead, find that your backgrounded command is in a stopped state.

Even better than using shells' built-in job-control for long-running tasks may be to use a terminal-multiplexer like screen or tmux. They're also great if you're connecting to a system over crappy links (remote in, fire up screen, kick off your tasks ...even if your connection dies, stuff keeps running and you can re-connect and re-attach to your session to finish things up).

[João Paulo dos Santos Portela]

Yeah! I was also going to suggest tmux instead of nohup.

[the-harry]

Great tips! I think it worth to mention the !$ operator in addition to !!.
Where !$ is the argument of last command.
So if you:

mkdir foo

You can enter the folder typing:

cd !$

[mgfrobzz]

If you have to telecommute, and your company's vpn connection is sh*tty, and they don't allow you to ssh in, reverse ssh tunneling is your friend:
thegeekstuff.com/2013/11/reverse-s...

[Jack Kingsman]

Also a good way to get your security team grumpy with you ;) Outbound SSH from prod sets off about a dozen alarm bells for us. Talking to your manager or DevOps team member is another option for addressing rough inbound access :)

[Patrick Hanford]

Awesome list.

I vote to add ctrl + r for searching your previous commands to this list!

Testing Gunicorn or Prometheus or something with a long pipeline? Run it once then ctrl + r and type some keywords to find it again without having to type it all out or mash the up arrow.

[Vinay Hegde]

If you've a common SSH login with sudo (shouldn't be the case mostly), do be careful with !! as you could end up running a potentially dangerous command in a hurry.

You could use sudo lsof -i :80 to identify which app is taking up a port instead of netstat & ps

Also zless to view gzipped logs without extraction & wget -c URL to retry downloads in case of issues.

[iamnielsjanssen]

For me 'screen' is unmissable when working in terminal applications. It is like having multiple windows! ctrl-a then 'c' creates new screen, ctrl-a 'space' switches through screens.

[Keith Sloan]

Agreed. I tend to map my F-keys to swich between next, prev and list screens.
bindkey -k k3 prev
bindkey -k k4 next
bindkey -k k5 windowlist

[Jon Austin]

generally use tmux nowadays

[Araslanov Eugene]

This also works in git

  git checkout -

[Luan Fonseca]

A lot of git commands works with the -

Like:

(my-branch*) $ git checkout develop
Switched to branch 'develop'
Your branch is up-to-date with 'origin/develop'.

(develop*) $ git checkout my-branch
Switched to branch 'my-branch'

(my-branch*) $ git merge -
Already up-to-date.

[Gergely Polonkai]

ll is actually a convenient alias set on most systems. Aliases are really powerful in and of themselves and worth a separate article (in fact, there’s a lot out there…)

[SerjP]

A "nice" command with 'yes' is:

yes >/dev/sdaX &

where X is 1, 2, etc
Wipes your hdd with "yes".
:)))

[Vladimir Nikolic]

Your title is misleading kind of.
Says commands you use in daily job, implies you are using them every day? Or that you use those command in your daily job as developer. If second whats your night job?
I dont see any command i would run daily.
Sorry to say but i dont see any of these commands so useful for daily work.
Hard to believe you would be executing ssh -L{port on your PC}:localhost:{database's port} root@{server IP} daily (or almost) instead of creating an alias, same as for the other longer command you mention.
Or this:
0 22 * * 1-5 /opt/scripts/send-report.sh 2>/var/log/scripts/report-error.log
Really? Almost daily?
Cant take this seriously :)

[Sreeni]

CTRL - R is best to scroll through history