DEV Community: Himadri Ganguly

Scraping external URL metrics using Python and exporting to Prometheus

Himadri Ganguly — Wed, 28 Apr 2021 16:05:31 +0000

The sample application in Python producing Prometheus format metrics at the endpoint "/metrics", is collected by Prometheus and a dashboard in Grafana is used to display the metrics. You can build the Docker image and deploy the application in Kubernetes.

https://github.com/himadriganguly/sample_external_url

Configure SSH Server With Key-Based And Two Factor Authentication

Himadri Ganguly — Tue, 29 Dec 2020 19:03:01 +0000

SSH or secure shell is an encrypted protocol used to connect to a server. I think all of us used it for connecting to servers but have you used two-factor authentication with SSH to make the process more secure? Most of you might have not done this till now. So in this tutorial we will go step by step to make that magic happen on Ubuntu and CentOs server (LOL).

Introduction

We all know what SSH is, but to be on the same page let's discuss a little bit about Secure Shell a.k.a SSH.

SSH or Secure Shell is a cryptographic network protocol for operating network services securely over an unsecured network. Typical applications include remote command-line, login, and remote command execution, but any network service can be secured with SSH. - From Wikipedia

So from the definition, we can understand that the main focus of this protocol is securely connecting to services over the untrusted network using a client-server architecture. When data is transferred it is secured and not in plain-text that transfers through the wire, so even if someone eavesdropping on your network they will not able to get the data. It came into existence to replace the Telnet protocol which transfers data in plain-text over the network. The standard port for SSH server is 22 and the client connects to the server on this port.

There are a lot of ways SSH user authentication can be done but in this article, we will see how to use the combination of SSH Key-Based And Two Factor Authentication. In SSH Key-Based Authentication two cryptographic keys are generated one public and one private key. The public key is transferred to the SSH server and the private key is retained by the client which is later used to prove the identity of the client. The private key should be kept securely so that it doesn't get exposed to any untrusted parties.

Create SSH Keys

The first step of configuring SSH Key-Based Authentication is creating the SSH keys. To do this we will go to our Linux terminal and use the tool ssh-keygen to generate the keys. In most Linux distros this tool is present by default but if not, please Google it to find the installation procedure.

Generating SSH key pair



$ ssh-keygen

This will prompt for the name of the file which will be stored in the default location at ~/.ssh, within the home directory of the user. The default name of the public key is id_rsa.pub and the private key is id_rsa.



Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):

Next, it will prompt for the passphrase if you want to encrypt your private key in your local system. This passphrase will protect the private key even the key is exposed to any malicious user, and it will prompt for the password to decrypt it when using this private key to connect to the SSH server. If you don't want to use a password just hit the ENTER key which will not encrypt the private key. For this demo, I am not using any passphrase so pressing ENTER and ENTER again to confirm no password.



Enter passphrase (empty for no passphrase):

After this, it will generate the Keys



Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa
Your public key has been saved in /home/username/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:Vv6kq8aYMxH4f+0zP7zltoCDaoE14uHqhtOoVqcvOKE username@hostname
The key's randomart image is:
+---[RSA 3072]----+
|                 |
|                 |
|     .    .      |
|    .o.o o       |
|    o.=.S . .    |
| . . =oo  .+.    |
|. B +  *...+.o  .|
|EB *  =.= ..= +o.|
|+ =.o..=.o...+o=o|
+----[SHA256]-----+

If you now list the directory ~/.ssh you can see that two files are created



$ ls ~/.ssh
id_rsa  id_rsa.pub

Transfer Public Key To SSH Server

Transferring the public key to the SSH server can be done in two ways we will see both the methods. I have created two virtual servers -

Ubuntu Server 20.04 - 192.168.99.105
CentOs 8 - 192.168.99.104

Manually Copy SSH Public Key To Server

We will copy the SSH public key manually to the Ubuntu server.

We will cat the content of the public key



$ cat ~/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL9LHP61fH98RJ0hAhAdqTTd6dQPJ0H07vHrMzsa8S/bxxRgfd850+aE8mpHcuRnf1I+UMCKpkLZvhUJRBKXW9SmHu6skhXw0mY+TsLZFM3MjziX6vr+el4gbPZ6XROklGbY6BH3axmGz4uaaO/3vKYDGCdOSodm3azQ7ewPgJtOFNGV3BEsTuQh/Q2rmoO/a31FArntvQrljuuYt3Fd9jHTdjd82TjRUoqPR4MugJiB2I8iiVL4kY/VjZrkYiG47rtzOYBNohsQm1knI7Sg0KggR2s27hpvZvQl7guVLAGaF5Rif8vQPQkXIKUdOR46pIjvNAdt2y3HwNd36Uskr3jFmtd/Nf8mAweooKMdP9eNLX9GZX4h49jLJJXL6d3GmnnmUAA3LFauL5BObAKdftILWzGe3KrBroNtTED5nRAWYl8+EVLTDN7sWVE8UhqxH1gEx4gpbQ1dFmqlb9fcgg9349vlzG9bi/lhkIEFgrUUObh7SLJwAjXPEVGqgooHE= username@hostname

Copy the whole content and login to the server i.e in this case Ubuntu server and paste the content in this file ~/.ssh/authorized_keys



username@ubuntu-test-server:~$ mkdir ~/.ssh
username@ubuntu-test-server:~$ vi ~/.ssh/authorized_keys

Now go to your local console and check the connection to the server using SSH Key-Based Authentication.



$ ssh username@192.168.99.105

If everything works as expected will be able to log into the remote Ubuntu server console.

Copy SSH Public Key Using SSH Copy Id

Another way we can copy the SSH public key to the server is by using the tool called ssh-copy-id which is by default included with the SSH package. But for this to work you should be able to login to the server using SSH username and password.

We will use this method to copy the key to the CentOs server.



$ ssh-copy-id username@192.168.99.104

If this is the first time you are connecting to the server computer will ask you to confirm the identity of the server.



/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/username/.ssh/id_rsa.pub"
The authenticity of host '192.168.99.104 (192.168.99.104)' can't be established.
ECDSA key fingerprint is SHA256:ZLd3hAk7HENCuKG+T6yEEIKQdSZefSNICP6cmjv/O1E.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Type yes and hit ENTER to continue. Now the public key will be copied to the server. It will ask for the SSH password for the particular user, to do the operation.



/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
username@192.168.99.104's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'username@192.168.99.104'"
and check to make sure that only the key(s) you wanted were added.

If you go to the CentOs server and cat the file ~/.ssh/authorized_keys you will get the exact content of your local id_rsa.pub file.



[username@centos-test-server ~]$ cat ~/.ssh/authorized_keys

Now from your local console, check the connection to the server using SSH Key-Based Authentication.



$ ssh username@192.168.99.104

If we have done above steps correctly we will be able to get the remote shell of the CentOs server.

Disable SSH Password-Based Authentication

As we are able to SSH both the servers using Key-Based Authentication, so now we can disable the Password-Based Authentication in both the SSH server.

In order to disable password-based authentication in SSH you have to edit the file /etc/ssh/sshd_config in sudo mode and change the line PasswordAuthentication from yes to no. The process is the same for both the server.



[username@centos-test-server ~]$ sudo vi /etc/ssh/sshd_config

Edit line



# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication no # Change this line from yes to no
#PermitEmptyPasswords no

Save the file and exit. Next restart SSH daemon.



[username@centos-test-server ~]$ sudo systemctl restart sshd

Enable Two Factor Authentication In SSH

To use this feature we will be using two tools

If you want to know what is Pluggable Authentication Module a.k.a PAM you can check it out at

Let's setup each server one by one.

Ubuntu Server

First, we have to install the Google Authenticator PAM module on the server.



$ sudo apt update && sudo apt-get install libpam-google-authenticator -y

Next, we have to instruct PAM to use the Google Authenticator module with SSH. To do so we have to edit the PAM file sshd and add the line auth required pam_google_authenticator.so nullok at the end of the file, also comment the line @include common-auth in the file.



username@ubuntu-test-server:~$ sudo vi /etc/pam.d/sshd

Add the line



# Standard Un*x password updating.
@include common-password
auth required pam_google_authenticator.so nullok # Add this line

The nullok at the end of the line indicates that the OTP is optional while login in using SSH if OTP is not set up for that user. After you tested everything you can remove the nullok from the line to make 2FA mandatory.

Edit line



# Standard Un*x authentication.
#@include common-auth # Comment out this line

Next, we have to instruct SSH service to use this 2FA, to do so we have to edit the file /etc/ssh/sshd_config and change the line ChallengeResponseAuthentication from no to yes.



username@ubuntu-test-server:~$ sudo vi /etc/ssh/sshd_config

Edit line



# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication yes # Change the line from no to yes

Add another line



AuthenticationMethods publickey,keyboard-interactive # Add this line

Restart SSH service



username@ubuntu-test-server:~$ sudo systemctl restart sshd

Next, we will generate the time-based token using Google Authenticator and add it to our Authy App.



username@ubuntu-test-server:~$ google-authenticator

It will ask to confirm if you want to generate time-based authentication token. Type y and hit ENTER.



Do you want authentication tokens to be time-based (y/n) y

It will generate the QR code which you can scan in Authy App.

Open Authy App
Click on the 3-dots on the top right corner
Click on Add Account
Click on Scan QR Code button and scan the code

Next, it will ask to update the Google Authenticator file in your home directory. You must do this otherwise this 2FA will not work.



Do you want me to update your "/home/username/.google_authenticator" file? (y/n) y

It will ask if the token is used only once in every 30 seconds. This will help to mitigate the man-in-the-middle attack by destroying the key once used.



Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y

It will ask if you want to change the time skew. By default, it will generate 3 valid code in a 1:50 minutes rolling window before it time out. Unless there is an issue we should stick to the default.



By default, a new token is generated every 30 seconds by the mobile app.
In order to compensate for possible time-skew between the client and the server,
we allow an extra token before and after the current time. This allows for a
time skew of up to 30 seconds between authentication server and client. If you
experience problems with poor time synchronization, you can increase the window
from its default size of 3 permitted codes (one previous code, the current
code, the next code) to 17 permitted codes (the 8 previous codes, the current
code, and the 8 next codes). This will permit for a time skew of up to 4 minutes
between client and server.
Do you want to do so? (y/n) n

Now there is another beautiful feature of rate-limiting, which will block a remote user after 3 unsuccessful failed attempts.



If the computer that you are logging into isn't hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.
Do you want to enable rate-limiting? (y/n) y

We are done with the setup of our Ubuntu server with both Key-Based and 2FA Authentication.

Let's test if it is working or not. So exit from the Ubuntu server and run the ssh command from local terminal.



$ ssh username@192.168.99.105

Now it will prompt for verification code.



Verification code:

After providing the 2FA authentication code from the Authy App we logged into the Ubuntu server shell.

CentOs Server

To install Google Authenticator PAM in CentOs server we have to install the epel repository first.



[username@centos-test-server ~]$ sudo yum install epel-release -y

Now install Google Authenticator PAM and qrencode (which is used to display the QR code).



[username@centos-test-server ~]$ sudo yum install google-authenticator -y && sudo yum install qrencode -y

Next, we have to instruct PAM to use Google Authenticator module before login into SSH.



[username@centos-test-server ~]$ sudo vi /etc/pam.d/sshd

Add the line to the end of the file



session    include      postlogin
auth required pam_google_authenticator.so nullok secret=/home/${USER}/.ssh/.google_authenticator

The line is the same as we have done previously only difference is the part secret=/home/${USER}/.ssh/.google_authenticator which specified the location of the .google_authenticator config file which have to be moved from ~/.google_authenticator to ~/.ssh/.google_authenticator location due to SELinux security context, otherwise, it will not work.

Comment out the line



#auth       substack     password-auth # Comment this line
auth       include      postlogin

Before moving forward we have to move the file .google_authenticator to the ~/.ssh/ folder



[username@centos-test-server ~]$ mv ~/.google_authenticator ~/.ssh/

Next, we have to instruct SSH service to use this 2FA, to do so we have to edit the file /etc/ssh/sshd_config and change the line ChallengeResponseAuthentication from no to yes.



[username@centos-test-server ~]$ sudo vi /etc/ssh/sshd_config

Edit line



# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication yes # Change this line from no to yes

Add another line



AuthenticationMethods publickey,keyboard-interactive

Restart SSH service



[username@centos-test-server ~]$ sudo systemctl restart sshd

Generating a time-based token using the Google Authenticator and adding it to Authy App is the same as above.

After completing all the steps if everything works fine we can log in to the CentOs server using Key-Based and Two Factor Authentication.

Conclusion

Hope you like this article. In this article, I tried to explain as much as possible and in a simple step by step method, how we can create SSH key based authentication and implement 2FA authentication using Google Authenticator.

Thank you.

CKA & CKAD Completed

Himadri Ganguly — Mon, 21 Dec 2020 12:19:19 +0000

Very happy to end this year 2020 with a positive, meaningful, and knowledgeful ending while completing the two eminent certifications in the DevOps field - Certified Kubernetes Administrator (CKA) and Certified Kubernetes Application Developer (CKAD) from Cloud Native Computing Foundation in association with The Linux Foundation.

Always hope for the best and keep trying to achieve whatever we want in difficult times as well. Even if we fail there will be some learning which can be rectified in the next journey.

Another thing like to share with those who are going for the exam of K8s, just don't sit for the exam only to get a good position in a career or for a change in a better job role, this will definitely not help you if you don't love this technology, clear out the concept and try hands-on with K8s even if that topic is not included in the exam. The main point is that if you don't love the technology you can't work with it in long run and after some time get bored which will affect your career as well.

KubeAcademy is also a very good platform developed by VMware to learn Kubernetes. All the courses are free and taught by experienced persons. You check it out at - https://kube.academy/.

I like to wish those who are preparing for the exam or learning K8s all the best.

BIG THUMPS UP AND LOTS OF LOVE TO Google FOR SHARING SUCH A WONDERFUL TECHNOLOGY WITH THE OPENSOURCE COMMUNITY AND EASILY ACCESSIBLE TO EVERYONE. 👍❤️❤️👍

Keep learning and keep sharing knowledge. Power of OPEN SOURCE.

Got My CKAD Certificate

Himadri Ganguly — Sun, 06 Dec 2020 07:37:21 +0000

Very excited to complete Part 1 of the Kubernetes certification journey.

Today received my certificate from The Linux Foundation for the completion of my Certified Kubernetes Application Developer (CKAD) certification.

I like to thank my parents for their love and support without them nothing is possible and to all those who love and support me.

This made my DAY.

YaraSilly2 Open Source Threat Intelligent Project For Contribution In HacktoberFest

Himadri Ganguly — Wed, 30 Sep 2020 12:08:49 +0000

What is Yara Silly2?

A semi-automatic handy tool to generate YARA rules from sample virus files ( WIP ) for Malware Analyst, inspired by the DIFF function of VirusTotal Premium Account.

You can find some sample virus files at https://github.com/YARA-Silly-Silly/sample-malwares

You can find the project repo at https://github.com/YARA-Silly-Silly/yarasilly2

Why you should contribute?

The following are the rewards and benefits for contributors to this project.

If you are an active contributor we will add your image and Github handle in our Readme file.
This project will give a good understanding of the malware analysis world from the Blue Team perspective.
You can get to learn how YARA rules work and details regarding the rules.
Lear to use Python in the world of Threat Identification.

How you can contribute?

You don't have to be a coder to contribute to this project you can contribute by providing suggestions to improve the application or any other better way you want to involve in the project.

Want to provide suggestions? You can post it here - https://github.com/YARA-Silly-Silly/yarasilly2/issues/3

Are you a developer and want to try your hands dirty by adding new features to the application or make the existing code better? Check out the link - https://github.com/YARA-Silly-Silly/yarasilly2/blob/master/CONTRIBUTING.md

About Us

Yara Silly Silly is maintained by:

Himadri Ganguly	GitHub	Twitter	LinkedIn

Krishnendu Paul	GitHub	Twitter	LinkedIn	Website

YaraSilly2 - YARA Rule Generator

Himadri Ganguly — Sat, 26 Sep 2020 23:51:10 +0000

A Semi-automatic handy tool to generate YARA rules from sample virus files ( WIP ) for Malware Analyst, inspired by the DIFF function of VirusTotal Premium Account. It generates all the artifacts from samples ( Static Analyzer ) needed to create good Yara Rules, with less false positive.

Features:-

It uses a fuzzy match to find similar malware files based on tolerance the percentage set by the user.
String dump from the files in binary mode and find the occurrences of the files and take those string that occurs the specified number of times set by the user.
Eliminate all the blacklist strings that are provided from the match occurrences criteria.
Generate YARA rule and also prints the MD5 hash of the files within the meta section of YARA rules.

You can find the project repo at https://github.com/YARA-Silly-Silly/yarasilly2