How to Protect Your Server From Hackers

This post was originally posted in my personal blog.

It is not a hard deal to make your server secure, but when a lot of routines comes, It is possible to forget to do this. In my case, ssh server was hacked in two weeks after I bought it. One morning my mail had a couple of the abuses from third-side people said "something" on my server tried to hack their servers. So, I should solve the problem quickly.

How to find the vulnerability

In my case it was simple. I executed next command

cat /var/log/auth.log |  grep Accepted

and it returns me a list of successful authorization to my server. From the all returned lines I found one IP that is not my own. So, In my case, the SSH was a source of vulnerability.

How to protect server

Briefly about what I needed to do immediately after buying the server.

• update && upgrade the all packages on the server;
• Install ufw - plain firewall;
• close all server's ports besides SSH, HTTP(s) ports;
• Install and config fail2ban utility. It helps to analyze the /var/log/auth.log and ban some IPs if they make some wrong activity;
• change sshd config to accept the authorization only by private key.

What to do?

If you were hacked, your server is infected, and you need to know how to research and clean it. The best way - recreating the VPS. That was my case. I had the server at hetzner. From their dashboard, it is possible to recreate (drop and create new) VPS with the same IP in one click. So, I did. After that on my local PC, was generated SSH keys with an ssh-keygen utility (is a part of standard OpenSSH package). The command bellow same for Linux and MacOS.

ssh-keygen

It creates the pairs of keys in the ~/.ssh directory. After that running

 ssh-copy-id you_user@your_server_id

will upload your "just created" public key to the server. Next step, log in to the server and edit the config file for sshd:

nano /etc/ssh/sshd_config

In the config make changes for PasswordAuthentication variable

PasswordAuthentication no

This instruction close the possibility to connect with the password (only connection with private key accepted)

Installing and tuning ufw and fail2ban

I used ubuntu on server, so installation is

apt install ufw fail2ban

next step open only ssh, https port on server so:

ufw allow ssh
ufw allow 80
ufw allow 443

and enable the ufw:

ufw enable

Next step is configuring the fail2ban utility

# make a copy of default config (this copy will overload default params according to manual)
cp  /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
nano /etc/fail2ban/jail.local

in there find "banaction = " and set ufw as a value. After that reload fail2ban

fail2ban-client reload

According to this simple config, any three wrong attempts from particular IP get to access to ssh port will ban this IP for 10 minutes. Personally, I changed the ban time for 7 days.
How to check the status:

fail2ban-client status sshd

will return in my case

Status for the jail: sshd
|- Filter
|  |- Currently failed: 1
|  |- Total failed: 6
|  `- File list:   /var/log/auth.log
`- Actions
   |- Currently banned: 1
   |- Total banned: 2
   `- Banned IP list:   187.109.168.150

You can see, that one IP already blocked by the firewall. Same things possible to see with ufw report:

ufw status
Status: active

To                         Action      From
--                         ------      ----
Anywhere                   REJECT      187.109.168.150           
80/tcp                     ALLOW       Anywhere                  
22                         ALLOW       Anywhere                  
443                        ALLOW       Anywhere           

The fail2ban can be configured to send reports to your email if some IP has been banned.

Comments

[Padde]

While fail2ban is very useful you still get a lot of automated authentication attempts. I additionally added port knocking using the knock daemon on the Server: so in regular state even the ssh port is blocked by the firewall. to temporarily unlock the ssh port you have to knock on a few ports in a configurable order. only then the ssh port is opened for a few seconds foryou to connect to. e.g. knock youserver.com 18754 26557 28864 && ssh [e-mail] to login

this completely took away the failed login attempts in my logs.

[Vitaliy Kolesov]

Thank you for knock utility. Never used it before.

[Glenn Carremans]

Great post! This is also my default workflow when I setup a new VPS.

Just a couple of tips:

Just like SSH has an alternative syntax you can do the same for port 80 and 443.
ufw allow http & ufw allow https

You can also run the command: ufw app list
This will show a list of available applications that you can add to your UFW firewall so that if the port config has changed for example this will also be updated in your UFW.
My UFW rules:

To                         Action      From
--                         ------      ----
Nginx Full                 ALLOW       Anywhere
OpenSSH                    ALLOW       Anywhere
Nginx Full (v6)            ALLOW       Anywhere (v6)
OpenSSH (v6)               ALLOW       Anywhere (v6)

If you want your server to accept IPv6 you will need to enable this in your UFW config.

sudo nano /etc/default/ufw

And then change this: IPV6=yes
UFW reboot required after this change of course.

For fail2ban I have 3 jails enabled: sshd, sshd-ddos and nginx-botsearch
Current status for my sshd jail:

Status for the jail: sshd
|- Filter
|  |- Currently failed: 2
|  |- Total failed: 28259
|  `- File list:    /var/log/auth.log
`- Actions
   |- Currently banned: 0
   |- Total banned: 78
   `- Banned IP list:

[Vitaliy Kolesov]

Thanks to you, I pay attention to sshd-ddos!

In front of the nginx on my server I have traefik inside the docker container. I will try to research how to make them work together.

[Gergely Polonkai]

It feels really strange that neither the article nor anyone in the comments mention one of the most important thing:

Keep your software up to date

You can devise a firewall as hard to open as the most sophisticated safe, but if the back of the safe is missing, you did nothing. Security vulnerabilities happen, not just in the applications you use, but even in the operating system. Keep everything up to date, and you are one step closer to being protected.

[Vitaliy Kolesov]

Completely agree with your advice. I forgot to mention it in the article, I changed the post with your advice. Thank you!

[Alice-sos]

Hi,

I hope to get your consent to translate and shared with Chinese developers, I will indicate the source and author.

[Vitaliy Kolesov]

Surely, you can do it!

[Alice-sos]

Chinese link：nextfe.com/protect-server-from-hac... （中文）

[Alice-sos]

Thanks!

[LuisPa]

Great post!

Thanks for sharing.

[southcarolina803]

Thank you !

[Juanjo Salvador]

Great post! Really needed!

[Vitaliy Kolesov]

I set up a weak root password (it was the same as I used to have on my local PC) and forgot to close password access to the server in /etc/ssh/sshd_config.