DEV Community: gvelrajan

How to configure and setup SSH certificates for SSH authentication

gvelrajan — Thu, 10 Mar 2022 05:53:27 +0000

SSH public key based authentication has several drawbacks and operational challenges that could potentially compromise your organization’s SSH access security. We discussed the drawbacks in detail in our earlier blog on: Tightening SSH access using short-lived SSH certificates.

SSH certificate based authentication addresses most of these security problems while simplifying certificate and key management.

In this article, we’ll discuss how to configure and set up SSH certificates for SSH access to your servers and cloud resources.

Overview:

Here is an overview of what we are about to discuss in this article.

We’ll be creating two SSH Certificate Authorities(CA) - a host CA and a user CA, to sign the SSH host certificates and user certificates respectively. (Note: You can create just a single CA to sign both user and host certificates. But it is better to keep them separate, so that you could rotate them separately when the situation warrants.)
We’ll show how to configure the host and user machines to trust certificates issued by these host and user CAs.
We’ll be creating an SSH host certificate for each host and sign it using the host CA’s private key.
We’ll be creating an SSH user certificate for each user and sign it using the user CA’s private key.
Finally, we’ll discuss the benefits and drawbacks of using SSH certificate based authentication.

Step 1 - Creating CA certificates

Before we could create the CA certificates, we needed an SSH key pair to work with. We’ll be using the ssh-keygen tool to generate an SSH key pair, as usual. We’ll be executing the following set of commands on the server designated to be the CA.

User CA SSH key pair:

# ssh-keygen -t rsa -b 4096 -f ~/.ssh/ssh_user_ca

Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/ssh_user_ca
Your public key has been saved in /root/.ssh/ssh_user_ca.pub
The key fingerprint is:
SHA256:ShpCF11/fYjujAVDYivNKpNM6QdVwh8Z9sX00PIspdo 
The key's randomart image is:
+---[RSA 4096]----+
|    .oooBo.o+.   |
|     +o*o* .o=o. |
|  . = ..+.= o*+ .|
| . = o o.  =o o. |
|  . B + S  oo.   |
|   . B .  .=Z    |
|    . .   . o    |
|                 |
|                 |
+----[SHA256]-----+

Host CA SSH key pair:

# ssh-keygen -t rsa -b 4096 -f ~/.ssh/ssh_host_ca
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/ssh_host_ca
Your public key has been saved in /root/.ssh/ssh_host_ca.pub
The key fingerprint is:
SHA256:ShpCF11/fYjujAVDYivNKpNM6QdVwh8Z9sX00PIadsfj 
The key's randomart image is:
+---[RSA 4096]----+
|    .oooBo.o+.   |
|     +o*o* .o=o. |
|  . = ..+.= o*+ .|
| . = o o.  =o o. |
|  . B + S  oo.   |
|   . B .  .=F    |
|    . .   . o    |
|                 |
|                 |
+----[SHA256]-----+

We are requesting an RSA type key using the -t flag and the key length to be 4096 bits using the -b flag. The longer the key, the harder it is to crack.

It is highly recommended to provide a passphrase for the private key. If an unauthorized user gets hold of the private key with no passphrase, they will be able to log in to any server you’ve configured with the associated public key.

The above command would have generated an SSH pair - public key and a private key, in the default .ssh folder in your home directory. For example: /home/bob/.ssh/

Step 1a - Signing the CA’s Host Certificate

Now, using the following command, let’s create the host CA’s certificate using the host public key and sign it using the corresponding host private key generated in the previous step.

# ssh-keygen -s ~/.ssh/ssh_host_ca -I my-ca -h -n my-ca.example.com -V +52w ~/.ssh/ssh_host_ca.pub

The -s specifies the host private key to be used for signing the certificate. The -h option is used for generating a host type certificate.

The -n option above sets the FQDN (Fully Qualified Domain Name) of the host as the principals in the certificate.

The -V option sets the validity period of the certificate.

In the example above, the certificate will be valid from today and expires one year (52 weeks) from now.

You can verify the details of the certificate using the following command:

# ssh-keygen -Lf ~/.ssh/ssh_host_ca-cert.pub
/root/.ssh/ssh_host_ca-cert.pub:
        Type: ssh-rsa host certificate
        Public key: RSA-CERT SHA256:7sCdBjn0...
        Signing CA: RSA SHA256:7sCdBjn0... (using rsa-sha2-512)
        Key ID: "my-ca"
        Serial: 0
        Valid: from 2022-02-12T10:09:00 to 2023-02-11T10:10:29
        Principals: 
                my-ca.example.com
        Critical Options: (none)
        Extensions: (none)

Copy the CA’s host certificate to the /etc/ssh/ folder in the CA server.

Step 2 - Distributing and trusting the CA certificates

We need to copy the user CA certificate to all the hosts in the organization. This is to make the hosts trust the user certificate’s signing authority.

Similarly we need to copy the host CA certificate to all the user’s machines. This is to make the ssh clients in the user’s machine trust the host certificate’s signing authority.

Step 2a - Making hosts to trust user CA certificate

# scp ~/.ssh/ssh_user_ca.pub root@host1.example.com:/etc/ssh/

Next edit the SSH server config file at /etc/ssh/sshd_config and make the TrustedUserCAKeys directive to point to the user CA public key (NOT the user CA certificate) we just copied over.

TrustedUserCAKeys /etc/ssh/ssh_user_ca.pub

Restart the host to make the config change to take effect.

# systemctl restart sshd

Step 2b - Making clients to trust the host CA certificate

For this, you need to copy the contents of the host CA certificate and append it to the /etc/ssh/ssh_known_hosts file using the @cert-authority directive as shown below.

# cat /etc/ssh/ssh_host_ca-cert.pub
ssh-rsa AAAAHHNzaC1yc2EtY2....  root@example.com

Now copy this content to the SSH global config file ssh_known_hosts in all the user systems.

# nano /etc/ssh/ssh_known_hosts
@cert-authority *.example.com ssh-rsa AAAAHHNzaC1yc2EtY2....  root@example.com

The *.example.com wildcard entry above instructs the user systems to trust the host certificates received from any hosts in the example.com domain, signed by the host CA.

Now that we have the host and user CA certificates in place, we are ready to issue SSH certificates to hosts and user machines using them. Let’s see how to issue the host certificate first.

Step 3 - Creating SSH Host Certificates

To create the host certificate, we needed an SSH key pair to work with. Let’s create one as usual using the ssh-keygen command. Login to the host machine that needs the certificate and execute the below command.

# ssh-keygen -t rsa -b 4096 -f ~/.ssh/ssh_host

The above command will generate a public key (ssh_host.pub) and a private key (ssh_host) in the .ssh folder in the home directory (/root/.ssh/).

You should retain the private key safely within the host system. You should never copy or transfer the file to any other location.

We’ll copy just the host public key the CA server using the scp command as shown below:

# scp /root/.ssh/ssh_host.pub root@my-ca:/root/tmp/

Next, create the host certificate from the host public key and sign the certificate using the host CA’s private key.

# ssh-keygen -s ~/.ssh/ssh_host_ca -I host1@example.com -h -n host1.example.com -V +52w /root/tmp/ssh_host.pub

Once signed, copy the host SSH certificate (ssh_host-cert.pub) to the host machine using the scp command. It is safe to copy SSH certificates around because they are public objects and can be shared with anyone.

# scp /root/tmp/ssh_host-cert.pub  root@host1.example.com:/etc/ssh/

Now configure the host to use the signed host certificate for any connections coming from the SSH clients. Edit the /etc/ssh/sshd_config file and set the HostCertificate directive to point to the host certificate as shown below:

# nano /etc/ssh/ssh_config
…
HostCertificate  /etc/ssh/ssh_host-cert.pub
…

Finally, restart the SSH server for the configuration changes to take effect.

# systemctl restart sshd

Step 4 - Creating SSH User Certificates

First, let’s create an SSH key pair for the user certificate as usual.

# ssh-keygen -t rsa -b 4096 -f ~/.ssh/ssh_user

Next, we should copy the user public key to the CA server for signing it.

# scp ~/.ssh/ssh_user.pub  /root/tmp/.

# ssh-keygen -s ~/.ssh/ssh_user_ca -I bob@example.com  -n root -V +4w /root/tmp/ssh_user.pub

Secure copy the signed user certificate back to the user machine.

# scp /root/tmp/ssh_user-cert.pub bob@user1.example.com:~/.ssh/.

Next edit the ssh client config file /etc/ssh/ssh_config(global config file) or ~/.ssh/config file (local config file) in the user’s home directory and add the following line to it, so that SSH client will be able to automatically pick up the user’s private key file and the certificate during authentication.

IdentityFile ~/.ssh/ssh_user

Note that the IdentityFile directive is made to point to the user’s private key and not the public key or the certificate.

Try login to the host machine from the user machine to verify that everything is set properly and working just fine

$ ssh bob@host1.example.com

Delete any existing public key for this host from the ~/.ssh/known_hosts file in the user machine, if required, to prevent the SSH client from complaining about the host key has changed.

The host public key in the ~/.ssh/known_hosts file is irrelevant for SSH certificate based authentication because it uses the host CA certificate to validate the signature in the host certificate.

Benefits of using SSH certificate based authentication:

SSH certificates have a validity period, so user and host certificates expire eventually sometime in the future.
User certificates need not be copied to all the host machines.
Short-lived SSH certificates can be issued to users to access privileged resources.
No more weird messages (about recognizing and accepting the host key fingerprint) shown by the SSH client when login to the host machines for the first time.
When a security compromise is detected, invalidate the old CA, create a new CA and issue new certificates to hosts and users. This is called as certificate rotation.

Drawbacks of using SSH certificate based authentication:

Certificates need to be rotated periodically with proper planning and resource allocation
Short-lived user certificates are great for privileged access to production resources. However, issuing user certificates more frequently and copying them to the user machines is still a laborious process.

What is BastionXP Bastion Host Solution

BastionXP Bastion Host solution with built-in Public Key Infrastructure(PKI) and Certificate Authority(CA), automates and simplifies the certificate creation, signing and distribution process at scale.

It also automates the ability to rotate certificates when situation warrants. BastionXP Bastion Host PKI/CA also issues short-lived user certificates to end users to access cloud resources or on-prem servers. Moreover, the solution works seamlessly with OpenSSH server and clients.

So when a privileged user leaves an organization, you don’t need to clean up the public keys or certificates in your host machines because we don’t copy them to the host machines in the first place under the SSH certificate based authentication method.

Moreover, the privileged user’s short-lived certificates would have expired by the time he/she leaves the office on the last day of work.

Learn more about BastionXP Bastion Host solution and start your free trial here:
https://www.bastionxp.com

How to configure and setup SSH public keys, the right way

gvelrajan — Thu, 10 Mar 2022 05:01:46 +0000

Mostly, users login to their servers using SSH passwords. SSH password based authentication is not safe for enterprise use cases because passwords are usually short in length, contain dictionary words, and can be easily guessed. Using SSH public key based authentication is safe for enterprise deployments.

In this article we’ll discuss how to configure and setup SSH public key based login for a linux server.

Note: There are several security problems & operational challenges associated with using SSH public key based authentication and SSH key management tools. We discuss this in great detail in our next article here: Tightening SSH access using short-lived SSH certificates.

Step #1 Create the SSH keys

When you install SSH, it comes with a bunch of handy tools. One of them is named ssh-keygen. We’ll use ssh-keygen to create SSH public keys.

$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/bob/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/bob/.ssh/id_rsa
Your public key has been saved in /home/bob/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:ShpCF11/fYjujAVDYivNKpNM6QdVwh8Z9sX00PIspdo 
bob@localhost
The key's randomart image is:
+---[RSA 4096]----+
|    .oooBo.o+.   |
|     +o*o* .o=o. |
|  . = ..+.= o*+ .|
| . = o o.  =o o. |
|  . B + S  oo.   |
|   . B .  .=E    |
|    . .   . o    |
|                 |
|                 |
+----[SHA256]-----+

We are requesting a RSA type key and the key length should be 4096 bits. The longer the key, the harder it is to crack it.

It is highly recommended to provide a passphrase for the private key, when prompted. If an unauthorized user gets hold of the private key with no passphrase, they will be able to log in to any server you’ve configured with the associated public key.

The above command would have generated an SSH pair - public key and a private key, in the default .ssh folder in your home directory. For example: /home/bob/.ssh/

Public Key: /home/bob/.ssh/id_rsa.pub
Private Key: /home/bob/.ssh/id_rsa

You should neither share the private key with anyone nor copy it to your online storage or emails or chat window.

Step 2: Configure the server to use the public key

Next, we need to copy the SSH public key to the server. We’ll use a tool named ssh-copy-id that is part of the ssh toolkit.

$ ssh-copy-id bob@server-name

Once the above command completes, you can login to your server using the SSH public key. If you have set up a passphrase during the key generation, the ssh-copy-id command will ask for it. Alternatively, you can copy the SSH public key manually to the server. You need to copy-paste or append the public key to the ~/.ssh/authorized_keys file on the server.

$ cat /home/bob/.ssh/authorized_keys 
#Bob's key
ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB... bob

Step 3 Verify SSH login using the key

Verify if you can login to the server using the ssh key generated above. Run the SSH client command as shown below:

ssh -i /home/bob/.ssh/id_rsa bob@server-name

The -i option is used to specify the identity file or the SSH private key file. You may be prompted for the passphrase to unlock the private key, if you provided one during the step #1 above.

When you login using the SSH key for the first time, you’ll see a weird message on the console, something like this:

The authenticity of host 'server-name (10.1.1.1)' can't be established.
ED25519 key fingerprint is SHA256:LKVs+g5kgmIXsYkU2Fl8XmNOmW4Rz1/AoXBLdRoanzM.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?

This is a catch22 situation where you can neither deny nor agree with the message. If you know this is your host or server, then say yes and continue. The SSH public key of the host will be stored in the ~/.ssh/known_hosts file in your client machine for future reference. Next time when you SSH login, you’ll not see the message again.

If you prefer not to input the SSH key file location and the username everytime you use the ssh command, you could setup the SSH config in your user machine as shown below:

$ cat ~/.ssh/config

Host host1
   HostName host1.example.com
   User bob
   IdentityFile ~/.ssh/id_rsa

Description of the fields used in the above config file:

Host — the nickname for the host.
HostName — the IP address or domain of the remote server.
User — the username associated with the account in the server.
IdentityFile — the location of your SSH key to be used for login to this host.
If you prefer to use a different private key for each host, then you could setup your config file as shown below:

$ cat ~/.ssh/config

Host host1
   HostName host1.example.com
   User bob
   IdentityFile ~/.ssh/id_rsa_host1

Host host2
   HostName host2.example.com
   User bob
   IdentityFile ~/.ssh/id_rsa_host2

Host host3
   HostName host3.example.com
   User bob
   IdentityFile ~/.ssh/id_rsa_host3

Here after you could SSH into any host with a simple command like:

$ ssh [host-nickname]

For example, to login to host1:

$ ssh host1

Step 4: Disable password based authentication, if required.

Now that you have set up your SSH keys for login, it is the right time to completely disable password based login in your SSH server.

Note: Make sure that all other users of the server also have setup SSH public key based login successfully, before you turn-off the password based SSH login. Because once you disable password based authentication, any user still using password for SSH login wouldn’t be able to do so after this.

Edit the /etc/ssh/sshd_config file in the server using any editor of your choice, say nano or vim. Change the setting named PasswordAuthentication: to no as shown below.

$ nano /etc/ssh/sshd_config
…
PasswordAuthentication: no
…

If you see any ‘#’ at the beginning of the line, make sure you delete it. # is used to comment out a configuration. You need to uncomment it to make the configuration effective. Save the change and restart the SSH server daemon using the following command as sudo user:

 $ sudo systemctl restart sshd

Verify sshd server is up and running.

 $ sudo systemctl status sshd

Problems at Scale:

Now that you have set up one user for SSH public key based authentication, you as an admin of your organization, should repeat the same procedure to set up SSH public key based authentication for the remaining users.

If you have more than one server in your organization(which would most likely be the case), then you need to copy each user’s SSH public key to the authorized_keys file in all the servers. This becomes a M * N problem set.

SSH public key setup

It is not possible to do this task manually with a scaled infrastructure. Some sort of a software or automation tool is required to complete the task.

It is highly recommended that a user doesn’t reuse the same SSH public key to login to all SSH servers. If an unwanted user were to get hold of the private key, the potential attack surface becomes large.

There are some additional problems & operational challenges associated with using SSH public key based authentication. We discuss this in detail in our next article here: Tightening SSH access using short-lived SSH certificates

This article was originally published at: https://www.bastionxp.com/blog/how-to-configure-and-setup-ssh-public-keys-the-right-way/

Tighten SSH access using short-lived SSH certificates

gvelrajan — Thu, 10 Mar 2022 04:14:28 +0000

SSH access using public private key based authentication has several drawbacks that could potentially compromise your organization’s SSH access security. SSH certificate based authentication addresses most of these security problems while simplifying certificate distribution. BastionXP PKI/CA solution automates and simplifies the SSH certificate management while offering additional security services such as issuing short-lived SSH certificates, SSH session recording, monitoring, logging and tracking functionalities for auditing purposes.

Traditionally, enterprises use SSH public private key based authentication for SSH login to servers - both in-house and in the cloud.

Usually, IT admins create, copy and set up the SSH public private keys for each host (host key) when the VM or server is brought into service. Similarly, IT admins create, copy and set up the SSH public private keys for each user (user key) who needs to gain access to the VM or server. Sometimes, they even use a homegrown or third party tool to create and distribute SSH public private keys.

Problem:

Setting up a SSH public private key requires so much time and effort. The admins need to meticulously create, copy and set up a public private key for each user (and also for each host) without leaving any relic anywhere.

For example, the user public key needs to be appended to the host machine’s ~/.ssh/authorized-keys file and the user private key needs to be copied to a folder (~/.ssh/) in the user’s device such as laptop or desktop. Note that the user public key needs to be copied to each and every host that the user needs access to.

As a result, the server and user onboarding process is usually laborious, slow and cumbersome. At scale, the problem only gets magnified and the key distribution become really messy.

Here are some security issues associated with using SSH public private key based authentication for SSH login:

Copy of an SSH key pair left undeleted in emails or local folders or shared online storages, while copying to the host or the user’s machine.
SSH keys don’t have an expiry date or validity period stamped in them. So they can be used perpetually. The technology doesn’t force the admins or users to rotate the keys periodically. As a result, there is no real urge to renew SSH keys periodically. Secondly, rekeying and distributing the keys to servers and user machines at scale requires significant planning and execution. This builds up the inertia against rotating the SSH keys periodically.
Some users use the same SSH key pair to login to multiple host machines, potentially increasing the attack surface for any unwanted user who gains access to such an SSH key.
When users connect to a server for the first time, weird cryptic messages are shown to users to identify the host key fingerprint and accept it, so that the host key is stored permanently in the ‘known_hosts’ file (~/.ssh/known_hosts) for future references. Mostly, users blindly accept the message and continue with the login anyway, without bothering to know the authenticity of the host they are trying to log in. The only way to address this security problem is to pre-populate the “known_hosts” file with the host public keys of all known hosts that the user would potentially login to. In large organizations, this list is huge and changes over time, which means the ‘known_hosts" file on each user’s machine should be kept updated as well. As a result, most IT teams don’t follow this approach.
SSH keys don’t have the hostname or username stamped in them. As a result, any host that is setup with the SSH key could pretend to be the actual host.
When users leave the organization, SSH keys assigned to them are usually not deleted from the “authorized_keys” file in all the host machines.

Solution:

The solution to all the above security problems is to use SSH certificates for SSH login and authentication.

How SSH Certificate based authentication works:

Each organization sets up a Public Key Infrastructure (PKI) to issue SSH certificates to its hosts and users. Certificate Authority(CA) is the prime component of a PKI. For SSH certificate based authentication, two CAs need to be created - a Host Certification Authority(Host CA) and a User Certification Authority (User CA). Host certificates are signed by the Host CA and the user certificates are signed by the User CA.

A host certificate and the corresponding private key needs to be stored in the host machine only. Similarly, a user certificate and the corresponding private key needs to be stored in the user machine only. User certificates need not be copied to the host machines and host certificates need not be copied to user machines, unlike the SSH public private key based authentication. Advantages of SSH certificate based authentication over SSH public private key based authentication:

It is sufficient to copy just the user CA certificate to the host machines and the host CA certificate to the user machines. All the users’ certificates need not be copied to all the host machines and all the host machines' certificates need not be copied to all the user machines.

SSH certificates have an expiry date and validity period stamped in them. So, all SSH certificates will eventually expire sometime in the future. The need to rotate SSH certificates periodically is built into the technology.

An organization’s security policy could decide the lifetime of a certificate issued to its hosts and users.

SSH certificates can be issued to a specific identifiable entity within an organization. So SSH certificates have the host or user name stamped in them to identify who owns the certificate. Role based access control (RBAC) is possible with SSH certificates.

When a user connects to a server for the first time, no weird cryptic messages will be displayed on the screen to identify the fingerprint and trust the host key. The authenticity of the host the user is trying to connect to will be automatically verified using the Host CA Certificate stored locally in the user’s machine.

Drawbacks of SSH certificate based authentication:

Though, SSH certificate based authentication addresses most of the security problems in the SSH public private key based authentication, it still has the following caveats in it when SSH certificates are managed manually or using any home-built tools:

Rotating or renewing certificates takes time and effort to distribute them. Anyone who gets hold of an SSH certificate could login to a host without any issues.
Short-lived user certificates are highly secure because it reduces the duration of the damage any unwanted user who holds such a user certificate could cause. However, short-lived certificates require more frequent rotations and therefore more effort from the admin teams.
Long-lived certificates have the same inherent security risk as the SSH public private key based authentication but requires reduced effort from the admin teams.

BastionXP Bastion Host PKI/CA Solution:

BastionXP Bastion Host PKI/CA solution is an Identity Based Infrastructure Access Management solution. BastionXP has a built-in Public Key Infrastructure (PKI) and Certificate Authority(CA) that uses Single-Sign On (SSO) with Multi-Factor Authentication (MFA) to automatically generate and distribute short-lived user SSH certificates to end user machines.

User identity is verified using IAM software such as Microsoft Azure Active Directory, AWS IAM or Okta or other IAM solutions. User roles and groups defined in the IAM will be used to generate SSH certificates for the user.

User SSH certificates expire in 8 hours by default and can be configured to suit an organization’s security policy, to any value like 4 weeks or 30 secs or so.

Once users use the SSH certificate to login to a host, they can continue with that SSH session for any longer even if the certificate expires. But, if they exit from the SSH session and need re-login, the users need to SSO login again using their MFA credentials to generate a new SSH certificate.

Because user SSH certificates are short-lived, no certificate cleaning or housekeeping is required in the host or user machines, when a user leaves an organization or when teams get dismantled. The user certificate becomes invalid the next day.

Components of the BastionXP Solution:

BastionXP solution has three components:

SSH Bastion Host with PKI/CA
BastionXP SSH Server,
BastionXP SSH client/authentication utility

Note: BastionXP Bastion Host solution can be configured to work as a pure play solution or work along with OpenSSH server and OpenSSH client software.

The ** BastionXP bastion host** has a builtin auth server which does the PKI/CA functionality. It can also perform SSH proxy or Jump Host functionality to connect to hosts that exist safely behind it, protected from the internet.

The ** BastionXP SSH server (sshd)** implementation runs on each host server, talks to the auth server in the BastionXP bastion host, downloads a host certificate from the auth server to the host and has the ability to record SSH sessions for auditing and playback purposes.

The ** BastionXP SSH client** (a.k.a authentication CLI utility) allows users to authenticate using SSO and MFA based authentication mechanism (Okta/MS 365/G-Suite) to download a short-lived SSH certificate from the auth server(CA ) to the user’s laptop or desktop. The BastionXP SSH client also does the regular SSH client functionalities such as SSH sessions with hosts, agent forwarding, connect to target hosts via a proxy jump host etc.

BastionXP can be used in any brownfield deployments where OpenSSH servers and clients are already in use and is preferred over the BastionXP SSH server and client software. In such a scenario, BastionXP Bastion Host’s SSH proxy (Jump Host) functionality can be disabled and BastionXP would simply run in the PKI/CA only mode.

BastionXP could be deployed in three different ways. We’ll discuss each one of them in detail below.

BastionXP Deployment Option #1 - CA Only Mode

In the CA only mode of deployment, BastionXP Bastion Host will function only as a CA server to manage the SSH certificates. BastionXP Bastion Host’s SSH proxy mode (or Jump Host) functionality will be disabled. An OpenSSH server (already in place) will perform the Jump Host functionality.

SSH host certificates need to be generated by an admin using a BastionXP provided API executed from the host machines running OpenSSH server. Usually, SSH host certificates have a very long validity period. For example: 4 weeks, 26 weeks, or 52 weeks. This is a configurable value and can be set according to an organization’s security policy.

Users need to download, install and use a simple CLI tool provided by BastionXP to login into their SSO provider using MFA and download the short-lived user SSH certificates from the BastionXP CA to their laptops or a desktops. Users need to use this CLI tool only once everyday when they begin their work in the morning because the short-lived user certificate will be valid for 8 hours by default (This validity period is a configurable value).

BastionXP Deployment Option #2 - SSH Jump Host + CA Mode

In this mode, BastionXP Bastion Host will replace an OpenSSH server functioning as a Bastion Host or Jump Host. BastionXP Bastion Host will function as the SSH proxy server or Jump Host, in addition to performing the CA functionality for SSH certificate management. However, the hosts residing behind the Jump Host will still run the OpenSSH server and the user machines will continue to use the OpenSSH client. BastionXP Bastion Host can interoperate seamlessly with OpenSSH servers running in the host machines and the OpenSSH clients running the user machines.

BastionXP Deployment Option #3 - Pure Play Mode

In the pure play mode, BastionXP software will run everywhere in the network as an SSH server, SSH client and as an SSH Jump Host/CA server. The diagram below explains this mode of operation.

BastionXP Bastion Host Solution with SSH Certificate Management

Conclusion:

SSH public private key based authentication is prone for security breaches as it doesn’t have an expiry date and user or host identity associated with the keys. SSH certificate based authentication addresses several drawbacks of SSH public private key based authentication. BastionXP Bastion Host PKI/CA solution automates and simplifies the SSH certificate management while offering additional security services such as short-lived SSH keys, SSH session recording, monitoring, logging and tracking functionalities for auditing purposes

Free Trial

Start your free trial of BastionXP Bastion Host solution here.

To know more about BastionXP Bastion Host solution and request for a demo, please contact us at: support@bastionxp.com

This article was originally published at: https://www.bastionxp.com/blog/tightening-ssh-access-using-short-lived-ssh-certificates/

How to develop and test a local microservice with remote microservices

gvelrajan — Fri, 08 Jan 2021 14:10:47 +0000

In this article, we'll discuss how to develop, debug, and test a microservice locally on your laptop, without having to run all the other microservices of an app also on your laptop.

The solution presented in this article could also be used to debug, fix, and test a problem reported by your customer in your live production cluster by routing accesses to your microservice (that has the bug) to the one running in your laptop with the fix.

Problem Statement:

Let's say your team is focussed on developing and testing a front-end ReactJS microservice for your company's application that has like 25 other microservices such as Spring Boot, Elastic Search, Redis, Postgres, Grafana, and so on, in it.

As a dev, you would usually try to run all of these microservices in your laptop so that you could develop,build and test changes to your front-end ReactJS App natively on your laptop. But running all these 25 microservices on your laptop will make it into a toaster, because you are pushing your laptop cpu, memory and power consumption to their maximum capacity throughout the day.

Workarounds Used Today

Many try to workaround this problem by upgrade their RAM or disk or fan module or even the laptop itself.

This adds more costs to your organization. Sometimes it may make you wonder, "Was it really a smart decision to convert your monolithic applications into microservices, in the first place?"

Don't worry, you are not left alone. Many companies that have begin their journey into the microservices world, for the benefits it offers, have faced the exact same problem. And, fortunately, SocketXP has a solution to address this problem.

Wall Power Sockets Analogy

Let me ask you this question. Would you run a huge nuclear power reactor in your house just because you needed some electricity to run your TV, Air Conditioner and Washing machine? The answer would be "no". You'd just install a couple of sockets on your wall, run a power cable to the energy company in your city and hook up your electrical equipments to those wall sockets. Neat and simple isn't it?

Then why do we try to run all our microservices(both 800 pound gorillas and tiny little mice ones) on that poor little laptop on your lap, all day long, just because you need to focus on developing and testing one front-end microservice? You could very well install software "wall sockets" on your laptop and have your ReactJS front-end microservice talk to 25 other microservices running remotely in a cloud (or some on-prem cluster perhaps?) via the software "wall sockets" And vice-versa.

Our SocketXP Microservice Remote Access solution would help create those software "wall-sockets" in your laptop, for all your remote microservices.

How SocketXP Microservice Remote Access Solution Works?

You need to download and run a SocketXP proxy agent (also available as Docker Container) on your laptop and on your remote cluster where other N-1 microservices of your application reside.

SocketXP proxy agent will create "tcp or http" sockets on your laptop (one for each of the microservice running in a remote cluster) and run a bidirectional TLS tunnel between your laptop and your remote cluster. We'd also adjust DNS records (for those remotely running microservices) in your laptop to route to the localhost IP address. So that, any requests from your front-end microservices to these 25 other microservices, would be routed to these local sockets.

Your front-end microservice would continue to connect to other microservices of the app as usual, without requiring any modifications to your code or configurations.*

Docker Container Demo

For this exercise, we are going to run Postgres DB as a Docker Container in a remote server and make a python based backend microservice (under development in my laptop) access the remote DB via the SocketXP proxy.

Postgres DB is accessible via the hostname postgres and port 5432

Here is the simple backend microservice written in python.

$cat backend.py 
import psycopg2

con = psycopg2.connect(database="postgres", user="gvelrajan", password="pa$$word123", host="postgres", port="5432")

print("Database opened successfully")

cur = con.cursor()
cur.execute('''CREATE TABLE STUDENT
      (ADMISSION INT PRIMARY KEY     NOT NULL,
      NAME           TEXT    NOT NULL,
      AGE            INT     NOT NULL,
      COURSE        CHAR(50),
      DEPARTMENT        CHAR(50));''')

print("Table created successfully")

cur.execute("INSERT INTO STUDENT (ADMISSION,NAME,AGE,COURSE,DEPARTMENT) VALUES (34231, 'Dave', 19, 'Information Technology', 'ICT')");
cur.execute("INSERT INTO STUDENT (ADMISSION,NAME,AGE,COURSE,DEPARTMENT) VALUES (34232, 'Gary', 18, 'Electrical Engineering', 'ICT')");
cur.execute("INSERT INTO STUDENT (ADMISSION,NAME,AGE,COURSE,DEPARTMENT) VALUES (34233, 'Abel', 17, 'Computer Science', 'ICT')");

print("Student records added to the table successfully")

con.commit()
con.close()

The python backend microservice connects to the DB using the hostname 'postgres' and port '5432'

Run the Postgres DB Docker Container

First create a docker volume

$ docker volume create postgres

Next run the Postgres DB container and mount the postgres volume in the host machine to the directory /var/lib/postgresql/data inside the container.

$ docker run --name postgres --restart unless-stopped --net appnet --mount source=postgres,target=/var/lib/postgresql/data -e POSTGRES_USER=gvelrajan -e POSTGRES_PASSWORD=pa$$word123 -e POSTGRES_DB=postgres -d postgres:latest

Run SocketXP Proxy Agent container in a Cluster/Server

First go to SocketXP Portal. Signup for a free account and get your authtoken there.

Use your authtoken in the below config file.

$ cat config.json
{
    "authtoken": "<your-auth-token-goes-here>"
    "tunnel_enabled": true, 
    "tunnels" : [{ 
        "destination": "tcp://postgres:5432", 
        "iot_device_id": "postgres-mservice-cluster1", 
    }] 
}

Store the config.json file in a local directory(/home/gvelrajan/config/config.json) and map the directory into the SocketXP docker container at /data as shown in the command below to start the docker container.

docker run --name socketxp --restart unless-stopped --net appnet -d  -v /home/gvelrajan/config:/data expresssocket/socketxp:latest

Execute the following command to check the logs for success status or any errors:

docker logs socketxp

It would have something like this:

Connected.
TCP tunnel [gvelrajan-5bz4y6hi] created.

Now go to the SocketXP Portal's Tunnel Page. Check the name of the tunnel created and its status.

Run SocketXP Proxy Agent on your Laptop

First download the SocketXP Proxy Agent in your laptop from the SocketXP download page. SocketXP Proxy Agent is available for all OS versions - Windows/Mac/Linux.

Use the same SocketXP authtoken you used in the previous section to authenticate the proxy agent with SocketXP Cloud Gateway.

$ cat config.json
{
    "authtoken": "<your-auth-token-goes-here>"
    "tunnel_enabled": true, 
    "tunnels" : [{ 
        "destination": "tcp://postgres:5432", 
        "iot_device_id": "postgres-mservice-cluster1",
        "iot_slave": true
    }] 
}

$ socketxp --config config.json

The above config requests socketxp agent to run in slave mode and create a local proxy TCP socket in your laptop at 127.0.0.1:5432. Also it requests socketxp agent to create a secure TLS connection to the postgres-mservice-cluster1 proxy device (running in your remote cluster or server) we created in the previous section.

Note:
Although it is called TCP socket, we run TLS on top of it. So your data is securely transmitted over the internet end-to-end.

Next, update the DNS record in your laptop with the following command:

$ echo "127.0.0.1    postgres" | sudo tee -a /etc/hosts > /dev/null

We need to add the above hostname mapping for our postgres microservice so that any local requests to the microservice will be routed to our SocketXP proxy agent listening at IP 127.0.0.1.

Run the python backend microserver

We are all set to run the python backend microservice under development in our laptop.

$ python backend.py
Database opened successfully
Table created successfully
Student records added to the table successfully

Let's validate this success by running an SQL query in our remote postgres database.

$ docker exec -it postgres psql -U gvelrajan postgres
postgres=# \c postgres
postgres=# \d
          List of relations
 Schema |  Name   | Type  |   Owner   
--------+---------+-------+-----------
 public | student | table | gannygans
(1 row)
postgres=# select * from student;
 admission | name | age |                       course                       |                     department                     
-----------+------+-----+----------------------------------------------------+----------------------------------------------------
     34231 | Dave |  19 | Information Technology                             | ICT                                               
     34232 | Gary |  18 | Electrical Engineering                             | ICT                                               
     34233 | Abel |  17 | Computer Science                                   | ICT                                               
(3 rows)

(END)

Awesome! We are able to develop/test our local python backend service and make it to seamlessly connect with the remote postgres DB microservice.

Make a remote NGINX microservice locally accessible

Now suppose, in addition to the postgres db microservice, we also need to make an NGINX microserice locally accessible from our laptop, use the following config.json file to setup the SocketXP Proxy Agent in the remote server/cluster.

$ cat config.json
{
    "authtoken": "<your-auth-token-goes-here>"
    "tunnel_enabled": true, 
    "tunnels" : [
      { 
        "destination": "tcp://postgres:5432", 
        "iot_device_id": "postgres-mservice-cluster1", 
      },
      { 
        "destination": "tcp://nginx:80", 
        "iot_device_id": "nginx-mservice-cluster1", 
      },
    ] 
}

Next, go to your laptop, and update the config.json file to create a local Proxy for the nginx microservice, as shown below.

$ cat config.json
{
    "authtoken": "<your-auth-token-goes-here>"
    "tunnel_enabled": true, 
    "tunnels" : [
      { 
        "destination": "tcp://postgres:5432", 
        "iot_device_id": "postgres-mservice-cluster1",
        "iot_slave": true 
      },
      { 
        "destination": "tcp://nginx:80", 
        "iot_device_id": "nginx-mservice-cluster1", 
        "iot_slave": true 
      },
    ] 
}

$ socketxp --config config.json

The above config will create a TCP listening socket at port 80 in your laptop. Make sure port 80 is free in your laptop and not used by any other application.

Next, add the follow DNS entry into your /etc/hosts file, so that any local DNS resolution requests to access your ngnix service will be routed to the SocketXP Proxy Agent listening on IP 127.0.0.1.

$ echo "127.0.0.1    nginx" | sudo tee -a /etc/hosts > /dev/null

Now open up a browser in your laptop and point it to: http://nginx:80 or simply http://nginx. It will open up the NNGINX home page as shown below.

We are looking for beta customers to evaluate our SocketXP Microservice Remote Access Solution. Please write to us at: support@socketxp.com

This article was originally published at: https://www.socketxp.com/blog

How to SSH into a Kubernetes Worker Node

gvelrajan — Thu, 31 Dec 2020 07:29:25 +0000

Kubernetes is a very popular and widely deployed container management and orchestration platform, preferred by devops engineers worldwide today.

Usually Kubernetes clusters and their worker nodes are not exposed to the public Internet but the apps running in them are.

In this article, I’ll discuss how to configure SocketXP lightweight VPN solution to remote SSH access your private Kubernetes cluster worker nodes in your on-prem cloud or private cloud or public cloud (AWS, MS Azure, GCP, Digital Ocean etc.) or multi-cloud.

Note: We at SocketXP are looking for beta customers to evaluate and provide feedback for our Kubernetes Remote Access Solution. Please feel free to reach out to us at: support@socketxp.com

Prerequisites:

You are expected to have a working Kubernetes Cluster with atleast one worker node in it. And you could reach those worker nodes now.

Overall Strategy -- In a nutshell

We'll install SocketXP agent in your worker nodes and configure it to function as an SSH server. SocketXP agent will also establish a secure TLS VPN connection with the SocketXP Cloud Gateway. You could then, remote SSH into your Kubernetes worker nodes from the SocketXP Cloud Gateway Portal using your browser. No SSH client is required to SSH into your worker nodes.

Excited? Let's get started!

Step 1: Download and Install

Download and install the SocketXP agent on your Kubernetes Worker Node.

Step 2: Get your Authentication Token

Use the following command to authenticate you node with the SocketXP Cloud Gateway using the auth token.


 bash
$ socketxp login <your-auth-token-goes-here>

Step 3: Create SocketXP TLS VPN Tunnel for Remote SSH Access

Use the following command to create a secure and private TLS tunnel VPN connection to the SocketXP Cloud Gateway.


 bash
$ socketxp connect tcp://localhost:22  --iot-device-id "kube-worker-node-001"  --enable-ssh --ssh-username "test-user" --ssh-password "password123"

TCP tunnel [test-user-gmail-com-34445] created.
Access the tunnel using SocketXP agent in IoT Slave Mode

Where TCP port 22 is the default port at which the SocketXP agent would listen for SSH connections from any SSH clients. The "--iot-device-id" represents a unique identifier assigned to the Kubernetes worker node within your organization. It could be any string value but it must be unique for each of your worker node.

Security Info:
SocketXP does not create any public TCP tunnel endpoints that can be connected and accessed by anyone in the internet using an SSH client. SocketXP TCP tunnel endpoints are not exposed to the internet and can be accessed only using the SocketXP agent (using the auth token of the user) or through the XTERM terminal in the SocketXP Portal page.

SocketXP also has the option to setup and use your private/public keys to remote SSH into your worker nodes.

You could now remote SSH into your Kubernetes worker node by clicking the terminal icon as shown in the screenshot below.

Next, you'll will be prompted to provide your SSH login and password.

Once your credentials are authenticated with your SSH server you'll be logged into your device's shell prompt.

The screen capture below shows the "htop" shell command output from an SSH session created using the XTERM window in the SocketXP Portal page.

Configuring SocketXP agent to run in slave mode

This is an alternate method for SSH into your private worker node from a remote location using the SocketXP Remote SSH Access solution.

If you don't want to access your IoT device or RPi from the browser(SocketXP Portal) and you want to access it using an SSH client (such as PuTTy) installed on your laptop or desktop, follow the instructions below.

First download and install the regular SocketXP agent software on your accessing device (such as a laptop running Windows or Mac OS). Next, configure the agent to run in slave mode using the command option "--iot-slave" as shown in the example below. Also, specify the name of the private TCP tunnel you want to connect to, using the --tunnel-name option.


 bash
$ socketxp connect tcp://localhost:3000 --iot-slave --tunnel-name test-user-gmail-com-34445

Listening for TCP connections at:
Local URL -> tcp://localhost:3000
Accessing the IoT device from your laptop

Why this is important?:
SocketXP IoT Agent when run in Slave Mode acts like a localproxy server. It proxies all connections to a user-specified local port (10111 in the example above) in your laptop/PC to the SocketXP Cloud Gateway using a secure SSL/TLS tunnel. Also the SocketXP Agent authenticates itself with the SocketXP Cloud Gateway using your auth token. This ensures that only legitimate, authenticated users are permitted to access your private worker nodes. SocketXP ensures Zero-Trust security on all connected devices.

Now you can SSH access your Kubernetes Worker Node using the above SocketXP local endpoint, as shown below.


 bash
$ ssh -i ~/.ssh/test-user-private.key test-user@localhost -p 3000

Tip:
You can also use PuTTY SSH client to remote SSH into your device using the same parameters show above. Similarly, you can use PuTTY or FileZilla to perform SFTP actions such as file upload and file download to your private Kubernetes Worker Nodes.

Note: We at SocketXP are looking for beta customers to evaluate and provide feedback for our Kubernetes Remote Access Solution that includes Worker Node/Pod SSH access/Microservice Remote Access/Database Remote access. Please feel free to connect with us at: support@socketxp.com

This article was originally published at: https://www.socketxp.com/blog

Remote Access Kubernetes Dashboard

gvelrajan — Fri, 04 Dec 2020 16:59:45 +0000

Kubernetes is a popular cluster and container management/orchestration platform widely used in pulic and private clouds. Kubernetes Dashboard is a web-based Kubernetes user interface. You can use Dashboard to deploy containerized applications to a Kubernetes cluster, troubleshoot your containerized application, and manage the cluster resources.

SocketXP TLS VPN

SocketXP TLS VPN solution (a lightweight VPN) provides secure remote access to private Kubernetes Clusters in your private cloud or public cloud. SocketXP also provides a secure public URL to access your local private applications including Kubernetes Dashboard.

SocketXP agent is available as a docker container in the SocketXP DockerHub Repository. Run the SocketXP Docker container as a standalone container (as explained in the below sections) in your Kubernetes cluster to setup remote access to your Kubernetes Dashboard.

Deploying the Dashboard UI:

Follow the instructions in the Kubernetes Open Source Project pageon how to deploy and setup the Dashboard UI in your Kubernetes Cluster.

https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

As explained in the Kubernetes official documentation, you need to run the kubectl CLI utility in proxy mode to access your dashboard in a web browser.

Kubernetes Dashboard Login from Outside Network

What is the approach to access Kubernetes Dashboard login from outside network?

Overall Strategy

Here is the overall strategy to setup remote access to your Kubernetes Dashboard:

Deploy SocketXP VPN agent Docker container in your K8 cluster.
Install the kubectl CLI utility locally on your laptop.
Setup the kubectl config file in your laptop with SocketXP Public URL, K8 SSL Certs, and Key.
Remote access your private Kubernetes cluster from your laptop using the kubectl CLI utility.
Run kubectl in proxy mode in your laptop.
Access your Kubernetes dashboard in a web browser via the local kubectl proxy.

SocketXP Agent Docker Container Deployment:

First go to SocketXP Portal. Signup for a free account and get your authtoken there. Use the authtoken to create a Kubernetes secret as shown below.

kubectl create secret generic socketxp-credentials --from-literal=authtoken=[your-auth-token-goes-here]

Verify that the secret socketxp-credentials got created.

$ kubectl get secrets
NAME                   TYPE                                  DATA   AGE
default-token-5skb7    kubernetes.io/service-account-token   3      4h
socketxp-credentials   Opaque                                1      4h
$

We'll use the below config.json file to configure the SocketXP agent Docker container. In this example, we are trying to create a secure public web URL and a TLS VPN tunnel to the Kubernetes API server.

$ cat config.json
{ 
    "tunnel_enabled": true, 
    "tunnels" : [{ 
        "destination": "https://kubernetes.default", 
        "protocol": "tls", 
        "custom_domain": "", 
        "subdomain": "" 
    }], 
    "relay_enabled": false, 
}

Next create a Kubernetes configmap to store the above SocketXP agent configuration file.

kubectl create configmap socketxp-configmap --from-file=/home/test-user/config.json

Verify that the socketxp-configmap got created.

$ kubectl describe configmaps socketxp-configmap
Name:         socketxp-configmap
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
config.json:
----
{ "tunnel_enabled": true, "tunnels" : [{ "destination": "https://kubernetes.default", "protocol": "tls", "custom_domain": "", "subdomain": "" }], "relay_enabled": false }

Events:  <none>

Now that we have created the authtoken secret and the configmap needed by the SocketXP agent, it's time to launch the SocketXP Docker container expresssocket/socketxp:latest as a Kubernetes Deployment.

Here is the deployment.yaml file we'll use to create a standalone SocketXP agent deployment.

$cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: socketxp
  labels:
    app: socketxp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: socketxp
  template:
    metadata:
      labels:
        app: socketxp
    spec:
      containers:
      - name: socketxp
        image: expresssocket/socketxp:latest
        env:
          - name: AUTHTOKEN
            valueFrom:
              secretKeyRef:
                name: socketxp-credentials
                key: authtoken
        volumeMounts:
        - name: config-volume
          mountPath: /data
      volumes:
        - name: config-volume
          configMap:
            # Provide the name of the ConfigMap containing the files you want
            #to add to the container
            name: socketxp-configmap

Note:
We have created a separate volume named config-volume and mounted it under /data directory inside the container, so that the socketxp-configmap will be available as a config.json file under the /data directory in the running container.

Next, check if the pods are created from the deployment and running.

$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
socketxp-75cb4dd7c9-bhxfp   1/1     Running   0          4s
$

Now you can retrieve the SocketXP Public URL created for your Kubernetes API server from the SocketXP Portal Page at: https://portal.socketxp.com/#/tunnels (or) from the pod logs as shown below.

$ kubectl logs socketxp-75cb4dd7c9-bhxfp
...
...

Login Succeeded.
User [] Email [test-user@gmail.com].

Connected.
Public URL -> https://test-user-fn4mda420.socketxp.com

You can now use the above SocketXP Public URL to access the Kubernetes Cluster's API server remotely using a kubectl utility.

Local kubectl installation

Install the kubectl CLI utility locally on your laptop to remote access your Kubernetes cluster. Follow the instructions here to download and install kubectl on your laptop:

https://kubernetes.io/docs/tasks/tools/install-kubectl/(opens new window)

After you have installed the kubectl CLI utility, overwrite the kubectl config file located at $HOME/.kube/config in your laptop with the one from your cluster's master node($HOME/.kube/config).

Next, update the API server URL in your kubectl config file to use the SocketXP Public URL https://test-user-fn4mda420.socketxp.com, as shown below.

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /Users/test-user/.minikube/ca.crt
    server: https://test-user-fn4mda420.socketxp.com 
  name: minikube
contexts:
- context:
    cluster: minikube
    user: minikube
  name: minikube
...
...

Please ensure that you also copy the client certificate, CA certificate and private key files or authtoken from your Kubernetes cluster's master node to your laptop in the appropriate folder as specified in the kubectl config file.

Verify that the config works fine, using the following command:

kubectl config view

Now remote access or remote manage your private Kubernetes Cluster from your laptop by executing any kubectl command locally:

$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
socketxp-75cb4dd7c9-bhxfp   1/1     Running   0          1h

Note:
When you create more than one replica of the SocketXP agent pod using the deployment, each pod would be assigned a unique SocketXP Public URL. This is because each SocketXP agent pod running in the Kubernetes Cluster will fetch a new Public URL from the SocketXP Cloud Gateway.

Run kubectl in proxy mode

To remote access your Kubernetes Dashboard, run the kubectl CLI utility in proxy mode in your laptop as shown below:

$ kubectl proxy  
Starting to serve on 127.0.0.1:8001

Let this command continue to run in the foreground.

Remote access Kubernetes Dashboard:

Now you can remote access your Kubernetes Dashboard from your laptop using the following local URL via the kubectl proxy. Kubectl will make Dashboard available at:

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ (opens new window).

Now you can view or manage your k8 resources.

Note: This article was originally published at: https://www.socketxp.com/iot

Configure local kubectl to access remote Kubernetes cluster

gvelrajan — Wed, 02 Dec 2020 09:40:36 +0000

Kubernetes is a very popular and widely deployed container management and orchestration platform, preferred by devops engineers worldwide today.

Usually Kubernetes clusters are not exposed to the public Internet but the apps running in them are.

In this article, I’ll discuss how to configure a local kubectl to remote access your Kubernetes cluster or minikube running in a server in your lab or private cloud or public cloud (AWS, MS Azure, GCP, Digital Ocean etc.).

Prerequisites:

You are expected to have a basic understanding on:

How to configure and setup a Kubernetes cluster or minikube
How to run a Docker container as a Kubernetes deployment and service
What kubectl and kubeadm tools are and how they are used for Kubernetes cluster, pod management and orchestration.
You have a working Kubernetes Cluster or Minikube setup already.

Overall strategy — In a nutshell

The kubectl CLI utility talks to the Kubernetes cluster via the cluster’s API server. As long as we could make the cluster’s API server accessible from your laptop, we could access or manage your remote Kubernetes cluster or minikube through a local kubectl instance installed on your laptop.

Enabling secure remote access to the cluster’s API server over the public internet is key here. We’ll use SocketXP VPN solution (SSL/TLS tunnels) to provide secure remote access to the cluster’s API server. SocketXP VPN solution has a free plan for beginners.

Setup a Kubernetes cluster or Minikube

To begin with setup a Kubernetes cluster or Minikube instance on your laptop. The aim of this article is not to teach you how to setup a Kubernetes cluster or Minikube. So let’s jump straight into our task at hand, that is, configure local kubectl to remote access Kubernetes cluster or minikube.

Install, Setup and Configure Kubectl for remote access to Kubernetes cluster

Follow the below instructions to setup and configure kubectl locally on your laptop for remote access to your Kubernetes cluster or minikube.

Step #1 — Install and Setup local Kubectl

Install the kubectl CLI utility on your laptop (Mac/Windows/Linux version) from the Kubernetes project’s public repository. Instruction on how to install and setup kubectl are described here in detail.

Step #2 — Copy the kubectl config file

Now go to your Kubernetes cluster’s master node or minikube that you have setup in the previous section and copy the kubectl config file from there to your laptop.

Usually the kubectl config file is stored at: $Home/.kube/config in the master node of your remote Kubernetes cluster. This is the config file used by the kubectl utility installed in your remote cluster’s master node.

Note: kubectl is one of the utilities installed in any Kubernetes cluster or minikube during a cluster setup.


 sh
$ cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
 certificate-authority: /home/test-user/.minikube/ca.crt
 server: https://192.168.99.100:8443
 name: minikube
contexts:
- context:
 cluster: minikube
 user: minikube
 name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
 user:
 client-certificate: /home/test-user/.minikube/profiles/minikube/client.crt
 client-key: /home/test-user/.minikube/profiles/minikube/client.key

The above kubectl config file was captured from a remote server running minikube cluster.

Copy this kubectl config file to your laptop and replace any existing config file at $HOME/.kube/config in your laptop.

Step #3 — Copy the SSL certificates and private key

Next, you should copy the SSL certificates and private key used by the kubectl utility installed in the master node of your remote Kubernetes cluster, to your local laptop.

Copy the CA certificate (ca.crt), client certificate (client.crt) and client private key (client.key) files from your remote Kubernetes cluster to your local laptop.

The location of these files in the master node of your remote cluster is specified in the kubectl config file you copied in Step#2 (look for the bold text fields in the config file above).

You could download these certificate and key files to any directory in your local laptop, as long as you update their full path in the appropriate fields in your local kubectl config file (again look for the bold text fields in the config file shown above).

Step #4 — Install and setup SocketXP agent

For the locally installed kubectl instance to remote access your Kubernetes cluster’s API server running at https://cluster-ip-address:8443, you need to setup a public we URL for the API server, so that you could access and manage the cluster from anywhere in the internet.

SocketXP SSL/TLS VPN tunnels provide a secure, private and lightweight communication channel and a public URL to remote connect to your private Kubernetes cluster’s API server over the internet. Moreover, SocketXP VPN solution is free (checkout the “Tunnel Free Plan” here).

Follow the instructions here to download and install SocketXP agent docker container on your Kubernetes cluster or minikube as a standalone container deployment.

Standalone Container Deployment:

First go to SocketXP Portal. Signup for a free account and get your authtoken there. Use the authtoken to create a Kubernetes secret as shown below.


 sh
$ kubectl create secret generic socketxp-credentials --from-literal=authtoken=[your-auth-token-goes-here]

Verify that the secret socketxp-credentials got created.


 sh
$ kubectl get secrets
NAME                   TYPE                                  DATA   AGE
default-token-5skb7    kubernetes.io/service-account-token   3      4h
socketxp-credentials   Opaque                                1      4h
$

We’ll use the below config.json file to configure the SocketXP agent Docker container. In this example, we are trying to create a secure public web URL and a TLS VPN tunnel to the Kubernetes API server.


 json
$ cat config.json
{ 
    "tunnel_enabled": true, 
    "tunnels" : [{ 
        "destination": "https://kubernetes.default", 
        "protocol": "tls", 
        "custom_domain": "", 
        "subdomain": "" 
    }], 
    "relay_enabled": false, 
}

Next create a Kubernetes configmap to store the above SocketXP agent configuration file.


 sh
kubectl create configmap socketxp-configmap --from-file=/home/test-user/config.json

Verify that the socketxp-configmap got created.


 sh
$ kubectl describe configmaps socketxp-configmap
Name:         socketxp-configmap
Namespace:    default
Labels:       <none>
Annotations:  <none>
Data
====
config.json:
----
{ "tunnel_enabled": true, "tunnels" : [{ "destination": "https://kubernetes.default", "protocol": "tls", "custom_domain": "", "subdomain": "" }], "relay_enabled": false }
Events:  <none>

Now that we have created the authtoken secret and the configmap needed by the SocketXP agent, it’s time to launch the SocketXP Docker container expresssocket/socketxp:latest as a Kubernetes Deployment.

Here is the deployment.yaml file we'll use to create a standalone SocketXP agent deployment.


 yaml
$cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: socketxp
  labels:
    app: socketxp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: socketxp
  template:
    metadata:
      labels:
        app: socketxp
    spec:
      containers:
      - name: socketxp
        image: expresssocket/socketxp:latest
        env:
          - name: AUTHTOKEN
            valueFrom:
              secretKeyRef:
                name: socketxp-credentials
                key: authtoken
        volumeMounts:
        - name: config-volume
          mountPath: /data
      volumes:
        - name: config-volume
          configMap:
            # Provide the name of the ConfigMap containing the files you want
            #to add to the container
            name: socketxp-configmap

Note:
We have created a separate volume named config-volume and mounted it under /data directory inside the container, so that the socketxp-configmap will be available as a config.json file under the /data directory in the running container.

Next, check if the pods are created from the deployment and running.


 sh
$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
socketxp-75cb4dd7c9-bhxfp   1/1     Running   0          4s
$

Now you can retrieve the SocketXP Public URL created for your Kubernetes API server from the SocketXP Portal Page at: https://portal.socketxp.com/#/tunnels (opens new window) or from the pod logs as shown below.


 sh
$ kubectl logs socketxp-75cb4dd7c9-bhxfp
...
...
Login Succeeded.
User [] Email [test-user@gmail.com].
Connected.
Public URL -> https://test-user-fn4mda420.socketxp.com

Step #5 — Update the API server URL

You can now use the above SocketXP Public URL to access the Kubernetes Cluster’s API server remotely using a kubectl utility or directly using your custom application.
If you are using a locally installed kubectl utility from your laptop to remotely access the Kubernetes, then update the API server URL in the kubectl config file located at $HOME/.kube/config to use the SocketXP Public URL https://test-user-fn4mda420.socketxp.com


 yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /Users/test-user/.minikube/ca.crt
    server: https://test-user-fn4mda420.socketxp.com 
  name: minikube
contexts:
- context:
    cluster: minikube
    user: minikube
  name: minikube
...
...

Please ensure that you also copy the client certificate, CA certificate and private key files from your Kubernetes cluster’s master node to your laptop in the appropriate folder as specified in the kubectl config file.
Verify that the config works fine, using the following command:


 sh
kubectl config view

Step #6 — Access your Kubernetes cluster remotely from your laptop

Next, you could execute any kubectl commands such as ‘kubectl get pod’ or ‘kubectl get service’ from your laptop and the remote API server should respond back with the status of your pods running in your remote Kubernetes cluster or minikube.


 sh
$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
socketxp-75cb4dd7c9-bhxfp   1/1     Running   0          1h

Hope that was easy and straight forward to setup it up.

Advantages of SocketXP SSL/TLS VPN over other VPN solutions:

SocketXP SSL/TLS VPN is a L4 VPN (unlike L2 or L3 VPNs such as MACsec or IPsec, respectively). So remote access to only one specific application in a private network is allowed (unlike L2 or L3 VPNs which permit access to an entire private network).
SocketXP SSL/TLS VPN tunnels, like any VPN software, supports client authentication via TLS client authentication. So only a client application (kubectl instances in this case) with a valid TLS client certificate could access or talk to the remote server (Kubernetes cluster API server in the example above). No rogue user or app from the internet could access the server application made accessible via a SocketXP public URL.
SocketXP TLS tunnels are extremely lightweight unlike OpenVPN or other IPsec VPN softwares in the market, but it provides the exact same level of security (using the same SSL encryption technology) provided by OpenVPN or other IPsec VPN softwares. SocketXP uses the same encryption technology (SSL encryption) used by banks, financial institutions and Governments to securely transfer confidential data over the public internet.
SocketXP assigns you a unique public URL for your server application with random strings in it, that eliminates any guess work for the random Public URL uniquely assigned to you. This adds an additional level of security, in the first place.
SocketXP TLS VPN solution enables app-to-app communication only and not network-to-network communication. This drastically reduces the scope for any attack surface. The traffic from the internet over the VPN cannot go beyond the private IP:port boundary.
Moreover, SocketXP VPN Cloud Gateway is an online SaaS service that eliminates the need to run any VPN server in your private cloud or the need to run a VPN client software on your access devices such as laptops.
And it’s free. Can it get any better than this? Checkout the “Tunnel Free Plan” here.

Have a question or comment, leave it below. Alternatively, you could write to us at: support@socketxp.com

Note: This article was originally published at: https://www.socketxp.com/blog

Autoupdate Kubernetes Deployment on GitHub Webhook

gvelrajan — Fri, 10 Jul 2020 12:11:39 +0000

Last week I discussed how to autoupdate a Docker Compose deployment using SocketXP on receiving a GitHub push or release webhook. This week I'll discuss how to autoupdate a Kubernetes workload or deployment on receiving a GitHub Webhook when a new version of the app is released.

Prerequisites

For understanding and following the instructions in this demo you need to have a basic understanding and some working knowledge of the following tools.

GitHub
Docker
Kubernetes Cluster
Helm
SocketXP

Continuous Deployment Strategy

Our goal is to upgrade our demo app in a GitOps fashion, meaning we don't want to upgrade whenever a new checkin happens into our app git repo but when a change or release is created in our Devops template or script git repo.

In this demo, I'll show you how to create a separate git repo for the helm chart of our app. Whenever a new release is tagged on the master branch for this helm chart git repo we'll upgrade the helm chart release in our Kubernetes Cluster, which in turn will upgrade our http-server app in our Kubernetes cluster.

Demo App

We'll use the same demo app we used in our last week's demo. I'm displaying it again over here for convenience.

$cat http-server.js 
var http = require('http');
var port = 8080
var handleRequest = function(request, response) {
  console.log('Received HTTP request for URL: ' + request.url);
  response.writeHead(200);
  response.end('Hello World!, v1.0');
};
var server = http.createServer(handleRequest);
server.listen(port);

Also the Dockerfile used to build our http-server app from our last week's demo.

FROM alpine:latest
RUN apk update && apk add nodejs
RUN mkdir -p /usr/src/app
COPY ./http-server.js /usr/src/app
WORKDIR /usr/src/app
EXPOSE 8080 
CMD ["node","http-server.js"]

Refer to our last week's demo article on how to build a version 1.0 of the docker image and publish it to a DockerHub registry.

Kubernetes Cluster

Install Kubernetes Cluster or a Minikube Cluster following the instructions here:

Check the nodes and the pods available.

$kubectl get nodes
NAME       STATUS   ROLES    AGE     VERSION
minikube   Ready    master   3h45m   v1.18.3

$kubectl get pods
No resources found in default namespace.
$

Helm

I have already created an helm chart for this demo and is available at: https://github.com/socketxp-com/kubernetes-webhook-autoupdate-helm-chart

But if you want to create your own helm chart follow the instructions below to install helm and create a helm chart.

$ brew install helm

$ helm create http-server-chart
$ ls
http-server-chart
$ cd http-server-chart
$ ls
Chart.yaml  charts      templates   values.yaml
$

Now edit the values.yaml file and update the repository and tag fields to your app's docker image name and tag. Also update templates such as deployment.yaml or service.yaml under the templates folder, if required.

Commit the changes to your chart into a git repository, so that the changes/versions can be tracked. Also this will help in upgrading the corresponding workloads in a Kubernetes cluster, in a GitOps fashion.

Run SocketXP on the master node

Download and install SocketXP agent on the master node of your Kubernetes cluster's master node. You can follow the SocketXP download instructions here.

Next jump into the root directory of the git cloned chart repo and execute the below command. The repo already has the files "update-kube.sh" and "filter-rules.sh" from the last week's demo. The only change is in the update-kube.sh file to execute a "helm upgrade" command instead of "docker-compose up -d" command.

$ cat update-kube.sh
#!/bin/bash

git pull
helm upgrade http-server http-server-chart

Execute the SocketXP relay command.

$ socketxp relay http://localhost:8443 --exec ./update-kube.sh --filter filter-rules.json

connected.
Public URL -> https://webhook.socketxp.com/ganeshvelrajan-rmzlayq9

Now pick up this SocketXP Public Webhook URL and go to your GitHub Webhook Settings page and paste this URL in the Public Webhook URL text box. Make sure you set the content type of GitHub webhook to "application/json". Also set the secret with your own secret. This secret will be used to validate the sender when a GitHub webhook is received at your Kubernetes Cluster master node.

Install Helm Chart

Install the helm chart manually using the following command and bring up the http-server app in the kubernetes cluster. We need to do this only once when we bring up the app for the very first time. Thereafter, we'll automate the upgrade process using SocketXP.

$ls
LICENSE         filter-rules.json   update-kube.sh
README.md       http-server-chart
$helm install http-server http-server-chart/
NAME: http-server
LAST DEPLOYED: Fri Jul 10 11:59:39 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services http-server-http-server-chart)
  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

Let's follow the instructions in the above output and copy paste the above commands.

$export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services http-server-http-server-chart)
  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
$  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
$echo http://$NODE_IP:$NODE_PORT
http://192.168.99.100:32194
$

Let's try to access our http-server application at the above mentioned URL.

$curl http://192.168.99.100:32194
Hello World!, v1.0

Great. Our app is running version 1.0 as expected.
Next verify the helm release version currently deployed.

$helm ls
NAME        NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                   APP VERSION
http-server default     1           2020-07-10 11:59:39.651311 +0530 IST    deployed    http-server-chart-0.1.0 1.16.0

Begin Demo - Upgrade the app and the chart

Now let's edit our http-server app like we did in the last week's demo and push a docker container image version 2.0 to DockerHub. Please refer to our last week's demo article if you forgot the instruction for doing so.

Next let's edit the helm-chart and upgrade the image version and the chart version to 2.0 and 0.2.0 respectively.

$vim http-server-chart/values.yaml
# Default values for http-server-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

image:
  repository: gvelrajan/http-server
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "v2.0"
...
...

...
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.2.0
...
...

Commit the changes and push to GitHub.

 
$git add .
$git commit -m "image version and chart version updated"
[master 1765b2e] image version and chart version updated
 4 files changed, 35 insertions(+), 2 deletions(-)
 create mode 100644 filter-rules.json
 create mode 100755 update-kube.sh
$git push
Counting objects: 7, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 839 bytes | 839.00 KiB/s, done.
Total 7 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/socketxp-com/kubernetes-webhook-autoupdate-helm-chart.git
   2f12153..1765b2e  master -> master

As soon as we do git push for the chart into its online repo, GitHub will trigger a webhook notification for the git push event. SocketXP will receive the webhook, validate the signature and try to match it with the fiter rules. Since this is not a release webhook, SocketXP will throw a mismatch error and not further process this webhook.

Connected.
Public URL -> https://webhook.socketxp.com/ganeshvelrajan-rmzlayq9

Webhook received:
POST / HTTP/1.1
Host: webhook.socketxp.com
Accept: */*
Accept-Encoding: gzip
Content-Length: 9397
Content-Type: application/json
User-Agent: GitHub-Hookshot/f2f2346
X-Github-Delivery: 4aba8354-c279-11ea-980e-6cd6cfda12a3
X-Github-Event: push
X-Hub-Signature: sha1=14e1f0a3d368d9b5dadd7d41cd575af2f45065ce

{"ref":"refs/heads/master","before":"2f12153ef944b939bc81712e1d047d281d2deabe","after":"1765b2ecd58ad3c6d3fec15c7b3198063b543405","repository":{"id":278398971,"node_id":"MDEwOlJlcG9zaXRvcnkyNzgzOTg5NzE=","name":"kubernetes-webhook-autoupdate-helm-chart",
...
]}}

#Rule: Webhook signature matched.
#Rule: ref in payload didn't match
Webhook received didn't match with the filter rules.
Command not executed.

Release version 0.2.0 of the chart

So this proves that our http-server app will not be upgraded on receiving just any ad-hoc wehbook notifications from GitHub. It gets upgraded only on receiving an official release webhook for the app's helm chart.

Now, let's go ahead and create an official release version 0.2.0 of the helm chart for our app. We do this in the GitHub release page for our app's helm-chart repo.

[ Note: But before doing this make sure you update the print string in our http-server app to print "v2.0". Also build a new Docker container image for the upgraded app and tag it as v2.0. Refer to our last week'd demo if you need to know how to perform these actions]

SocketXP will receive the release webhook from GitHub. It will process the webhook, validate the signature and match the ref field in the webhook payload with the filter rules.

Webhook received:
POST / HTTP/1.1
Host: webhook.socketxp.com
Accept: */*
Accept-Encoding: gzip
Content-Length: 8725
Content-Type: application/json
User-Agent: GitHub-Hookshot/f2f2346
X-Github-Delivery: b08b5ec8-c27a-11ea-8774-72206af92c06
X-Github-Event: push
X-Hub-Signature: sha1=d6fefb9919e8b322226f6dcd485bd2f2591e80c8

{"ref":"refs/tags/v0.2.0","before":"0000000000000000000000000000000000000000","after":"1765b2ecd58ad3c6d3fec15c7b3198063b543405","repository":{"id":278398971,"node_id":"MDEwOlJlcG9zaXRvcnkyNzgzOTg5NzE=","name":"kubernetes-webhook-autoupdate-helm-chart","full_name":"socketxp-com/kubernetes-webhook-autoupdate-helm-chart",
...
...

]}}

#Rule: Webhook signature matched.
#Rule: ref in payload matched.
All rules matched.

Executing command:  [./update-kube.sh]
From https://github.com/socketxp-com/kubernetes-webhook-autoupdate-helm-chart
 * [new tag]         v0.2.0     -> v0.2.0
Already up to date.
Current branch master is up to date.
Release "http-server" has been upgraded. Happy Helming!
NAME: http-server
LAST DEPLOYED: Fri Jul 10 12:42:09 2020
NAMESPACE: default
STATUS: deployed
REVISION: 2
NOTES:
1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services http-server-http-server-chart)
  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

Let's follow the instructions above output and copy paste the above commands in a separate terminal.

$  export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services http-server-http-server-chart)
$  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
$  echo http://$NODE_IP:$NODE_PORT
http://192.168.99.100:32194

Now, let's access the URL using curl.

$curl http://192.168.99.100:32194
Hello World!, v2.0

Also confirm the kubernetes workload(pod) has the correct versions.

$kubectl describe  pods
Name:         http-server-http-server-chart-5947946f5f-vb5cn
Namespace:    default
Priority:     0
Node:         minikube/192.168.99.100
Start Time:   Fri, 10 Jul 2020 02:30:00 +0530
Labels:       app.kubernetes.io/instance=http-server
              app.kubernetes.io/name=http-server-chart
              pod-template-hash=5947946f5f
Annotations:  
Status:       Running
IP:           172.17.0.5
IPs:
  IP:           172.17.0.5
Controlled By:  ReplicaSet/http-server-http-server-chart-5947946f5f
Containers:
  http-server-chart:
    Container ID:   docker://7deb29b8786bc474374be82b27c33858cad8ec4b86754c29ad6467bf41f4ecc9
    Image:          gvelrajan/http-server:v2.0
    Image ID:       docker-pullable://gvelrajan/http-server@sha256:2a9ee82f76758758c77659260091380b20e3b3d095efdc6480d66e47c5842476
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 10 Jul 2020 02:30:03 +0530
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:      http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:    
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from http-server-http-server-chart-token-rmppc (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  http-server-http-server-chart-token-rmppc:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  http-server-http-server-chart-token-rmppc
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  10h   default-scheduler  Successfully assigned default/http-server-http-server-chart-5947946f5f-vb5cn to minikube
  Normal  Pulled     10h   kubelet, minikube  Container image "gvelrajan/http-server:v2.0" already present on machine
  Normal  Created    10h   kubelet, minikube  Created container http-server-chart
  Normal  Started    10h   kubelet, minikube  Started container http-server-chart
$

Also let's check the helm release version currently deployed.

$helm ls
NAME        NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                   APP VERSION
http-server default     2           2020-07-10 12:42:09.288038 +0530 IST    deployed    http-server-chart-0.2.0 1.16.0     
$

Everything confirms that we are have successfully upgraded our app to version 2.0 by upgrading the helm chart release version to 0.2.0

Rolling back to the previous working version

Suppose, the upgrade to version 2.0 of the app introduced some severe bugs or problems that is unacceptable by any user, we would want to quickly go back to the previous working version, so that customers are not affected by our upgrade.

Helm makes the process of rollback very simple. Let's do it. We'll rollback to revision #1 of the helm chart from revision #2.

$helm rollback http-server 1
Rollback was a success! Happy Helming!
$helm ls
NAME        NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                   APP VERSION
http-server default     3           2020-07-10 12:53:04.755063 +0530 IST    deployed    http-server-chart-0.1.0 1.16.0

Curl the app's web URL and verify if the rollback is successful.

$curl http://192.168.99.100:32194
Hello World!, v1.0

Verify the pods running in the Kubernetes cluster have the right version. Notice that the pod running the old version of our http-server app is terminating and the pod running the new version is up and running.

$kubectl describe  pods
Name:                      http-server-http-server-chart-5947946f5f-vb5cn
Namespace:                 default
Priority:                  0
Node:                      minikube/192.168.99.100
Start Time:                Fri, 10 Jul 2020 02:30:00 +0530
Labels:                    app.kubernetes.io/instance=http-server
                           app.kubernetes.io/name=http-server-chart
                           pod-template-hash=5947946f5f
Annotations:               
Status:                    Terminating (lasts 10h)
Termination Grace Period:  30s
IP:                        172.17.0.5
IPs:
  IP:           172.17.0.5
Controlled By:  ReplicaSet/http-server-http-server-chart-5947946f5f
Containers:
  http-server-chart:
    Container ID:   docker://7deb29b8786bc474374be82b27c33858cad8ec4b86754c29ad6467bf41f4ecc9
    Image:          gvelrajan/http-server:v2.0
    Image ID:       docker-pullable://gvelrajan/http-server@sha256:2a9ee82f76758758c77659260091380b20e3b3d095efdc6480d66e47c5842476
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 10 Jul 2020 02:30:03 +0530
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:      http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:    
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from http-server-http-server-chart-token-rmppc (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  http-server-http-server-chart-token-rmppc:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  http-server-http-server-chart-token-rmppc
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  10h   default-scheduler  Successfully assigned default/http-server-http-server-chart-5947946f5f-vb5cn to minikube
  Normal  Pulled     10h   kubelet, minikube  Container image "gvelrajan/http-server:v2.0" already present on machine
  Normal  Created    10h   kubelet, minikube  Created container http-server-chart
  Normal  Started    10h   kubelet, minikube  Started container http-server-chart
  Normal  Killing    10h   kubelet, minikube  Stopping container http-server-chart


Name:         http-server-http-server-chart-bdd75ff4-7jmbx
Namespace:    default
Priority:     0
Node:         minikube/192.168.99.100
Start Time:   Fri, 10 Jul 2020 02:40:56 +0530
Labels:       app.kubernetes.io/instance=http-server
              app.kubernetes.io/name=http-server-chart
              pod-template-hash=bdd75ff4
Annotations:  
Status:       Running
IP:           172.17.0.4
IPs:
  IP:           172.17.0.4
Controlled By:  ReplicaSet/http-server-http-server-chart-bdd75ff4
Containers:
  http-server-chart:
    Container ID:   docker://239aadd09eb54185b3aa559d5ba3a89f2cb99a30c5b340270b834f9e25de24cb
    Image:          gvelrajan/http-server:v1.0
    Image ID:       docker-pullable://gvelrajan/http-server@sha256:07e70e778e1002519064503ece0d334291f4fbb7a7196b894f605ff5e3076e18
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 10 Jul 2020 02:40:57 +0530
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:      http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:    
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from http-server-http-server-chart-token-rmppc (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  http-server-http-server-chart-token-rmppc:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  http-server-http-server-chart-token-rmppc
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  10h   default-scheduler  Successfully assigned default/http-server-http-server-chart-bdd75ff4-7jmbx to minikube
  Normal  Pulled     10h   kubelet, minikube  Container image "gvelrajan/http-server:v1.0" already present on machine
  Normal  Created    10h   kubelet, minikube  Created container http-server-chart
  Normal  Started    10h   kubelet, minikube  Started container http-server-chart
$

That's all folks!! I hope you were able to follow the instructions and auto deploy your app in Kubernetes using SocketXP and Helm chart in a GitOps fashion.

Conclusion

SocketXP and Helm are two great tools to setup a full-automated Continuous Deployment(CD) pipeline for your app running in your Kubernetes Cluster. Though you might want to expose your app ( a front-end microservice) running in your Kubernetes Cluster to the internet, you wouldn't want to and shouldn't expose your Kubernetes Cluster itself to the internet.

When you are not exposing the Kubernetes Cluster to the internet, it may be challenging to receive webhook notifications from GitHub, Gitlab, DockerHub, GCR and other online version control and artifact repositories. SocketXP solves this problem by quickly creating a secure webhook relay tunnel between the online registry and your Kubernetes Cluster, so that webhooks could reach your private Kubernetes Cluster.

SocketXP is a freemium online webhook relay service, which has free and paid plans for developers, businesses and enterprise customers. Sign up for your free SocketXP account here.

This article was originally published at:
https://www.socketxp.com/webhookrelay/autoupdate-kubernetes-deployment-on-github-webhook/

Docker Compose Auto Update on GitHub Webhook

gvelrajan — Sat, 04 Jul 2020 07:28:48 +0000

In the last week's demo, I showed how to setup a Continuous Deployment(CD) pipeline for a native NodeJS application.

In this article, I'll show you how to build a GitOps style CD pipeline that keeps a Docker Compose deployment in sync with a docker-compose YAML file hosted on a git repository.

What's new in this demo?

We have added some cool new features to SocketXP agent, such as:

ability to filter incoming webhooks on user specified filter rules
ability to execute a command or a script file when an incoming webhook matches with the user specified rules

Secondly, and more importantly, we'll be running our NodeJS web app inside a Docker container and bring it up using a Docker Compose YAML file.

Prerequisites

Here are the prerequisites for this demo.

GitHub
SocketXP Account
SocketXP Agent Installation
Docker Engine, Docker Compose and DockerHub Account

Repository with scripts that I used for this article can be found here: https://github.com/socketxp-com/docker-compose-autoupdate-on-github-webhook.

Demo App

We'll use the same nodejs http server that we used in the last week's demo.

$cat http-server.js 
var http = require('http');
var port = 8080
var handleRequest = function(request, response) {
  console.log('Received HTTP request for URL: ' + request.url);
  response.writeHead(200);
  response.end('Hello World!, v1.0');
};
var server = http.createServer(handleRequest);
server.listen(port);

Dockerfile

Here is the Dockerfile we'll use to package the above nodejs app into a Docker container.

FROM alpine:latest
RUN apk update && apk add nodejs
RUN mkdir -p /usr/src/app
COPY ./http-server.js /usr/src/app
WORKDIR /usr/src/app
EXPOSE 8080 
CMD ["node","http-server.js"]

Let's build the Docker image using the following Docker command.

$docker build -t gvelrajan/http-server:v1.0 .
Sending build context to Docker daemon  56.83kB
Step 1/7 : FROM alpine:latest
 ---> f70734b6a266
Step 2/7 : RUN apk update && apk add nodejs
 ---> Using cache
...
Successfully built b6b06a819b63
Successfully tagged gvelrajan/http-server:v1.0
$

Next, let's push this Docker image to DockerHub image registry, so that it can be downloaded by the production team to deploy it.

$docker push gvelrajan/http-server:v1.0
The push refers to repository [docker.io/gvelrajan/http-server]
3b8d0f6b058e: Layer already exists 
947da554eff8: Layer already exists 
cc403a71dcfb: Pushed 
3e207b409db3: Layer already exists 
v1.0: digest: sha256:07e70e778e1002519064503ece0d334291f4fbb7a7196b894f605ff5e3076e18 size: 1154

Docker Compose

Docker Compose is a very helpful tool to bring up various services of an application as Docker containers using a simple single command. In our demo example we have just a single service - the http web service, packaged as a Docker container.

Here is the docker-compose yaml file we'll use to bring up our http-server docker container in production.

$cat docker-compose.yml 
version: '3'
services:
  web:
    image: "gvelrajan/http-server:v1.0"
    ports:
      - "8080:8080"
$

Now, bring up the app container by executing the following command.

$docker-compose up -d
Creating network "docker-compose-autoupdate-on-github-webhook_default" with the default driver
Creating docker-compose-autoupdate-on-github-webhook_web_1_fd2bb2cedec9 ... done

Check if the docker container is up and running, and listening on the correct port(8080).

$docker ps
CONTAINER ID        IMAGE                        COMMAND                 CREATED             STATUS              PORTS                    NAMES
899ab99abedf        gvelrajan/http-server:v1.0   "node http-server.js"   2 seconds ago       Up 1 second         0.0.0.0:8080->8080/tcp   docker-compose-autoupdate-on-github-webhook_web_1_64dc034e55ac
$

Curl the web server

Verify that our web app is running and serving content on port 8080.

$curl http://localhost:8080
Hello World!, v1.0

Our initial deployment is very successful. But this was a manual deployment. Every time a newer version of the app is released we have to manually download and deploy the application. This is not what we wanted. Our goal for this demo is to build a Continuous Deployment(CD) pipeline for our nodejs based web service running as a docker container.

Let's begin by setting up various stages in the CD pipeline, like we did in the last week's demo.

Auto deployment strategy

Our auto deployment strategy has changed and is slightly different from the one used in the last week's native app demo. Here is the auto deployment strategy for this demo:

When a new version of our app is available, we'll update the docker-compose.yml file in the GitHub and release a new version of our app in the GitHub. GitHub will trigger a webhook for this release event. We'll listen for this webhook event and automatically do a git pull on the docker-compose.yml file. We'll use the new docker-compose.yml to re-spin our nodejs web service docker container in production.

Note: GitHub will trigger webhooks for all sorts of push events. We don't want to react to all those webhooks and auto deploy our containers on every GitHub push. We want to auto deploy only when a new release is made.

SocketXP Webhook Relay Service

We'll use SocketXP Webhook Relay Service to relay webhooks from GitHub to our local production server (where the nodejs app is running as a docker container). SocketXP can also filter GitHub webhooks based on user specified rules in JSON format. We'll discuss more about it in the next section. Additionally, SocketXP can execute a script or a command, if an incoming webhook matches with the user specified rules.

Before creating webhook rules, it is better to study the webhook in detail and pick up a few fields in the webhook payload to match on. For this we need to make GitHub trigger a release webhook and capture the webhook and study it.

Webhook Capture and Study

First, let's setup the SocketXP agent to relay any webhooks from the internet to our local web service.

$ socketxp relay http://localhost:8080

Connected.
Public URL -> https://webhook.socketxp.com/ganeshvelrajan-1awy5t0h

Pick up the above SocketXP Public URL and head to your GitHub repository. Setup the webhook settings for your GitHub project as shown below. Setup your secret and copy paste the SocketXP Public URL in the webhook URL text box. Also set the content-type to "application/json" in the drop-down box. Save the changes.

Triggering GitHub Release Webhook

Next, create a new release for your GitHub project in the release settings page and create a new tag for your release, as shown below.

GitHub will trigger a webhook for this release event in your project. We can view details of the triggered webhook either in the GitHub Webhook Settings page or in the SocketXP Portal Webhook Logs page.

Request URL: https://webhook.socketxp.com/ganeshvelrajan-1awy5t0h
Request method: POST
Accept: */*
content-type: application/json
User-Agent: GitHub-Hookshot/e509757
X-GitHub-Delivery: ca631384-bd0e-11ea-8ba6-ea05e17356b6
X-GitHub-Event: push
X-Hub-Signature: sha1=c902f69547901dc0658ba54eaee5cde6c0a5ef00

{
  "ref": "refs/tags/v1.0",
  "before": "0000000000000000000000000000000000000000",
  "after": "b5595d43b2c681257e0bae638f9de814df968901",
  "repository": {
    "id": 276817731,
    "node_id": "MDEwOlJlcG9zaXRvcnkyNzY4MTc3MzE=",
    "name": "docker-compose-autoupdate-on-github-webhook",
    "full_name": "socketxp-com/docker-compose-autoupdate-on-github-webhook",
    "private": false,
    ...
    ...
}

In the above webhook capture, we see the signature field in the header that encodes the secret we set in the GitHub webhook settings page. Verifying this signature using the secrete we provided, authenticates the validity of the sender.

Secondly, in the above webhook capture, the payload contains the release tags ("refs/tags/v1.0") in the "ref" field.

Let's try to filter our incoming webhook based on these two fields in the webhook.

SocketXP Webhook Filter-Rules JSON File

Here is how the SocketXP filter-rules JSON file should look like to match the two fields we picked up in the webhook above.

$cat filter-rules.json 
{
  "and":
  [
    {
      "match":
      {
        "type": "payload-hash-sha1",
        "secret": "asdfasdf23421dsaf",    
        "parameter":
        {
          "source": "header",
          "name": "X-Hub-Signature"
        }
      }
    },
    {
      "match":
      {
        "type": "value",
        "value": "refs/tags",
        "parameter":
        {
          "source": "payload",
          "name": "ref"
        }
      }
    }
  ]
}

Make sure you update the "secret" field in the filter-rules file above with your GitHub webhook secret.

SocketXP Exec Command Option

Here is the script we'd like to execute to automatically update our Docker containers running in our production or test server when a new release is made in the GitHub project.

$cat update-docker.sh 
#!/bin/bash

git pull
docker-compose up -d

Let's kill our SocketXP agent and re-run it with the two new arguments: "filter-rules.json" file and the "update-docker.sh" script file, as shown below.

socketxp relay http://localhost:8080 --exec update-docker.sh --filter filter-rules.json

Connected.
Public URL -> https://webhook.socketxp.com/ganeshvelrajan-1awy5t0h

Note that the SocketXP Webhook Public URL has not changed despite killing and re-executing the command. This is because we are requesting a public URL for a webhook relay service again and that too for the same local service endpoints (http://localhost:8080). This nice feature of SocektXP saves us from updating the GitHub Webhook URL settings again.

Begin Demo

Now that the CD pipeline is all set, let's upgrade our NodeJS web app and release a version 2.0 of the artifact. We also need to update the docker-compose.yml file to reflect the updated app version.

App Upgrade

Here is a new version of our app (for the sake of this demo, we have simply updated the version string printed to 2.0).

$cat http-server.js 
var http = require('http');
var port = 8080
var handleRequest = function(request, response) {
  console.log('Received HTTP request for URL: ' + request.url);
  response.writeHead(200);
  response.end('Hello World!, v2.0');
};
var server = http.createServer(handleRequest);
server.listen(port);

Build Push New App Container

Build a new docker container containing this new version of our app.

$docker build -t gvelrajan/http-server:v2.0 .
Sending build context to Docker daemon  69.63kB
Step 1/7 : FROM alpine:latest
 ---> f70734b6a266
Step 2/7 : RUN apk update && apk add nodejs
...
...
Successfully tagged gvelrajan/http-server:v2.0

Push the docker image to DockerHub.

$docker push gvelrajan/http-server:v2.0
The push refers to repository [docker.io/gvelrajan/http-server]
4a303822279a: Pushed 
947da554eff8: Layer already exists 
cc403a71dcfb: Layer already exists 
3e207b409db3: Layer already exists 
v2.0: digest: sha256:2a9ee82f76758758c77659260091380b20e3b3d095efdc6480d66e47c5842476 size: 1154

Update the docker-compose file with the new docker image version.

$cat docker-compose.yml 
version: '3'
services:
  web:
    image: "gvelrajan/http-server:v2.0"
    ports:
      - "8080:8080"

Git Commit and Release Webhook

Perform git commit and push the changes to the GitHub repository. Create new release version tag "version 2.0" for your project in GitHub.

This event will make GitHub trigger a webhook to SocketXP agent.

Verify Webhook Received

Check the console where SocketXP agent is running. It should have received the webhook from GitHub and applied the filters on it.

$ socketxp relay http://localhost:8080 --exec update-docker.sh --filter filter-rules.json

Connected.
Public URL -> https://webhook.socketxp.com/ganeshvelrajan-1awy5t0h

Webhook received:
POST / HTTP/1.1
Host: webhook.socketxp.com
Accept: */*
content-type: application/json
User-Agent: GitHub-Hookshot/e509757
X-GitHub-Delivery: fa44ca8e-bd31-11ea-9def-3e0cd2844448
X-GitHub-Event: push
X-Hub-Signature: sha1=d32c79c8de3282e07400711c4e7d3d6542ef79eb

{
  "ref": "refs/tags/2.0",
  "before": "0000000000000000000000000000000000000000",
  "after": "b5595d43b2c681257e0bae638f9de814df968901",
  "repository": {
    "id": 276817731,
    "node_id": "MDEwOlJlcG9zaXRvcnkyNzY4MTc3MzE=",
    "name": "docker-compose-autoupdate-on-github-webhook",
    "full_name": "socketxp-com/docker-compose-autoupdate-on-github-webhook",
    "private": false,
    ...
    ...
}

#Rule: Webhook signature matched.
#Rule: ref in payload matched.
All rules matched.

Executing command: [update-docker.sh]
...
...

Check if the docker container has been upgraded and running version 2.0.

$docker ps
CONTAINER ID        IMAGE                        COMMAND                 CREATED              STATUS              PORTS                    NAMES
e7ef0e98cd45        gvelrajan/http-server:v2.0   "node http-server.js"   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp   docker-compose-autoupdate-on-github-webhook_web_1_64dc034e55ac

Verify the upgraded app

Curl the web service and verify if the app has been upgraded.

$curl http://localhost:8080
Hello World!, v2.0

Our web app prints "v2.0", denoting that it got updated, automatically!

Conclusion

Setting up Docker Compose to auto deploy on GitHub push or release webhook notification is a great way to set up Continuous Deployment(CD) pipeline. The challenge is in receiving, processing and filtering GitHub webhooks in the production servers or in the internal test servers.

SocketXP is a great tool to receive, store, forward, filter and take actions based on webhooks. SocketXP Webhook Relay Service takes care of bulk of the work involved in setting up an automated CD pipeline. So that, developers can focus on their core work, while leaving the rest to SocketXP.

Moreover, SocketXP has a free tier for developers. And our paid plans start at just $1.99/month.

This article was originally published at:
https://www.socketxp.com/webhookrelay/docker-compose-auto-update-github-webhook

How to receive Stripe webhooks on your localhost for development and testing

gvelrajan — Fri, 26 Jun 2020 06:26:51 +0000

Testing Stripe webhooks locally on your localhost machine is easy as long as you have the right set of tools to create a publicly reachable URL for your localhost app. In this article, I'll demonstrate how to receive and test stripe webhooks locally on your localhost using a simple demo app.

What is Stripe Webhook

Stripe is one of the major online payment gateway and is a great alternative to paypal. It is a preferred choice for SaaS, e-commerce businesses as well as developers like us. Stripe is popular because it has an easy to use APIs, SDKs in many different programming languages(Ruby, NodeJS, Go, Python, Java etc.) and excellent documentation.

Stripe, like other online payment platforms, utilizes webhooks to inform the seller’s application about customers, product subscriptions, payment cards and other events occuring in the Stripe payment processing system. Receiving and processing Stripe webhooks are critical for building subscription (recurring payments) based payment gateway application where your backend system needs to track subscription status,

What is the problem

While it is easy to receive and process Stripe webhooks in production systems (because production systems are mostly customer facing public servers with public IPs), it is extremely challenging to receive and process Stripe webhooks on your local machine during development and testing. Your development and test machines are present in the local network. Your payment gateway application under development runs on your localhost at some port 8080. Stripe online cannot reach your localhost app.

In this article, I’ll demonstrate how to receive and test Stripe webhooks locally on a laptop using a demo application running on a localhost network.

What you’ll learn from this article:

Build a simple application to receive and process Stripe webhooks for subscription change events.
We will use SocketXP webhook proxy service to relay Stripe webhooks to the demo app run locally
We will use Stripe’s webhooks testings dashboard to simulate subscription change events.
We will use the SocketXP’s webhooks dashboard to monitor and replay the cached webhooks as they pass through the reverse proxy tunnel.

Before we get started

Here are some of the prerequisites for this demo.

Stripe account. You can register at https://dashboard.stripe.com/register.
SocketXP account. You can register at https://portal.socketxp.com
SocketXP agent installation instructions can be found here.
And ofcourse, Golang installed on your system to try the Go app. Install it from https://golang.org/doc/install/source

The demo application

Our demo application is written Go and it simply does the following:

The app listens on localhost port 8080 and handles any incoming HTTP requests for the subdirectory path “/stripe”.
It first verifies the signature of the Stripe webhook notification using the Stripe library method “webhook.ConstructEvent()” to validate the authenticity of the webhook source. For this you need to set an environmental variable named “STRIPE_SECRET” with your stripe secret retrieved from the Stripe Webhooks Dashboard.
It processes 3 different Stripe webhook events related to customer subscription.
It prints select details from the webhooks received such as CustomerID, Quantity and Status of the subscription.
It throws an error if the select details are missing in the webhooks received.

Download the demo application from GitHub: https://github.com/socketxp-com/stripe-webhook-demo

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "os"

    "github.com/stripe/stripe-go/webhook"
)

// application server port 
const port = ":8080"

func main() {
    secret := os.Getenv("STRIPE_SECRET")
    if secret == "" {
        fmt.Println("Required STRIPE_SECRET env variable is missing")
        os.Exit(1)
    }

    // incoming stripe webhook handler
    http.HandleFunc("/stripe", func(resp http.ResponseWriter, req *http.Request) {
        body, err := ioutil.ReadAll(req.Body)
        if err != nil {
            resp.WriteHeader(http.StatusBadRequest)
            return
        }

        // validating signature
        event, err := webhook.ConstructEvent(body, req.Header.Get("Stripe-Signature"), secret)
        if err != nil {
            resp.WriteHeader(http.StatusBadRequest)
            fmt.Printf("Failed to validate signature: %s", err)
            return
        }

        switch event.Type {
        case "customer.subscription.created":
            // subscription create event
            customerID, ok := event.Data.Object["customer"].(string)
            if !ok {
                fmt.Println("customer key not found in event.Data.Object")
                return
            }

            subStatus, ok := event.Data.Object["status"].(string)
            if !ok {
                fmt.Println("status key not found in event.Data.Object")
                return
            }

            quantity, ok := event.Data.Object["quantity"]
            if !ok {
                fmt.Println("quantity key not found in event.Data.Object")
                return
            }

            fmt.Printf("customer %s subscription created, quantity (%f), current status: %s \n", customerID, quantity, subStatus)

        case "customer.subscription.updated":
            // subscription update event
            customerID, ok := event.Data.Object["customer"].(string)
            if !ok {
                fmt.Println("customer key not found in event.Data.Object")
                return
            }

            subStatus, ok := event.Data.Object["status"].(string)
            if !ok {
                fmt.Println("status key not found in event.Data.Object")
                return
            }

            quantity, ok := event.Data.Object["quantity"]
            if !ok {
                fmt.Println("quantity key not found in event.Data.Object")
                return
            }

            fmt.Printf("customer %s subscription updated, quantity(%f), current status: %s \n", customerID, quantity, subStatus)
        case "customer.subscription.deleted":
            // subscription deleted event 
            customerID, ok := event.Data.Object["customer"].(string)
            if !ok {
                fmt.Println("customer key not found in event.Data.Object")
                return
            }

            subStatus, ok := event.Data.Object["status"].(string)
            if !ok {
                fmt.Println("status key not found in event.Data.Object")
                return
            }

            quantity, ok := event.Data.Object["quantity"]
            if !ok {
                fmt.Println("quantity key not found in event.Data.Object")
                return
            }

            fmt.Printf("customer %s subscription deleted, quantity(%f), current status: %s \n", customerID, quantity, subStatus)
        default:
            fmt.Printf("Unknown event type received: %s\n", event.Type)
        }
    
    })

    fmt.Printf("Listening for Stripe webhooks on http://localhost%s/stripe \n", port)
    // starting the http server
    log.Fatal(http.ListenAndServe(port, nil))

}

Downloading and executing the demo application:

Assuming that you have already installed Golang on your machine, execute the below commands to download, set the STRIPE_SECRET environmental variable and run the demo application.

$ git clone https://github.com/socketxp-com/stripe-webhook-demo.git
$ ls
stripe-webhook-demo
$ cd stripe-webhook-demo/
$ ls
README.md main.go
$ export STRIPE_SECRET=”whsec_…”
$ go run main.go

The demo app listens on “localhost:8080”. We cannot use this local address for registering with Stripe to receive webhooks for development and testing purposes. We need a public IP address or domain name to receive Stripe webhooks. And that’s where the SocketXP webhooks proxy and relay service comes in handy.

What is SocketXP

SocketXP is a reverse proxy tunneling service that relays webhook notifications from online web services such as Stripe, Paypal, GitHub, DockerHub etc to applications running in localhost network behind NAT and Firewall.

SocketXP is a freemium SaaS service that significantly reduces the development and testing cycle for software teams, resulting in cost savings.

How SocketXP webhook proxy and relay service works

Install the SocketXP agent on your localhost machine where your app(webhook receiver) is developed and tested. It could be installed on your laptop or your office LAN server. The SocketXP agent creates a secure SSL/TLS tunnel to the SocketXP Cloud Gateway. The SocketXP Cloud Gateway creates a unique secure (HTTPS) public endpoint for each user. The user can use the SocketXP public endpoint (URL) to register with online webhook sender applications such as Stripe, Paypal etc.

When the webhook sender sends a webhook message on the SocketXP public endpoint(URL), SocketXP Cloud Gateway will validate the URL and then forward the webhook to the corresponding registered user's SocketXP agent via a secure (SSL/TLS) tunnel. The SocketXP agent would in turn forward the webhook message to the receiver application running in the localhost. Learn more about how SocketXP solution works here.

SocketXP Cloud Gateway also caches the webhooks locally for your auditing and analysis needs. Secondly, and more importantly, you could resend failed webhook notifications from the SocketXP dashboard without requesting the sender application to resend the webhook. This feature is extremely useful during development and testing cycle.

Download Install and Login

Download and install the SocketXP agent for your OS version and Platform, and authenticate the SocketXP agent to the SocketXP Cloud Gateway by following the instructions here.

Create SocketXP public webhook relay endpoint

$socketxp relay http://localhost:8080/stripe

Connected.
Public URL -> https://webhook.socketxp.com/test-user-gmail-com-45803

Use this SocketXP public webhook endpoint (URL) to register with Stripe Webhook Testing Dashboard, explained in the next section.

Sending test webhooks from Stripe dasboard

Toggle the “view test data” button on the side navigation bar to put Stripe on a test mode.

Select the “webhooks” under the head “Developers” on the side navigation bar or go to https://dashboard.stripe.com/test/webhooks

[caption id="attachment_908" align="aligncenter" width="1920"] Stripe Webhooks Testing Dashboard[/caption]

Click the “Add endpoints” button on the right hand side window.

Add the SocketXP public webhook URL to the “Endpoint URL” text box. Choose the Stripe webhook events “customer.subscription.created”, “customer.subscription.updated”, “customer.subscription.deleted” from the “Events to send” drop down menu box right below it. Finally, click the “Add endpoint” button below to save the endpoint.

Now click open the endpoint you just added. And click the “Send test webhook” button on the top. Choose the Stripe webhook events, you wish to send and click the “Send test webhook” button.

Our demo app should receive webhook events from the Stripe webhook dashboard.

Demo app output:

The demo app receives webhooks from the SocketXP agent running locally on the same machine.

$go run main.go
Listening for Stripe webhooks on http://localhost:8080/stripe
customer cus_00000000000000 subscription created, quantity(1.000000), current status: active
customer cus_00000000000000 subscription updated, quantity(1.000000), current status: active
customer cus_00000000000000 subscription deleted, quantity(1.000000), current status: canceled

Conclusion

Stripe is a great payment gateway for developers to integrate because of its easy to use APIs and SDKs. Developing, integrating and testing your backend application with Stripe webhooks dashboard on your local machine could be a nightmare. SocketXP webhook proxy service and the record/replay option will significantly reduce the development and testing time (and ofcourse, the cost) for your software team.

This article was originally published at:
https://www.socketxp.com/webhookrelay/testing-stripe-webhooks-locally-stripe-webhooks-localhost/

Auto Deploy from GitHub to Local Server on Git Push

gvelrajan — Fri, 26 Jun 2020 06:12:55 +0000

As developers (oh yeah, as devops guys), it's a no-brainer to automate our workflows to make life simple and easy for us. In this article, I'll demonstrate how to automatically deploy a nodejs web app from GitHub to a production or test server on git push. You can follow the same instructions to deploy any application written in any language.

Tools used in this demo

We use quite a few tools in this demo. Each one has a specific role to play in the Continuous Deployment(CD) pipeline we are about to setup. Each of these tools are briefly explained below.

Nodemon

Monitors any changes in your node.js application and automatically restarts the server. Though Nodemon is typically used in development for quick re-spins, we'll use it in production in this demo to simply explain you the overall concept.

Webhook

Incoming webhooks handler which can execute shell commands.

SocketXP

Enables us to receive webhooks from the internet and relay it into your local network without exposing the app to the internet. SocketXP eliminates the need to own a public IP.

Demo app

We'll use the following simple nodejs web application for this demo.

var http = require('http');
var port = 8080
var handleRequest = function(request, response) {
  console.log('Received HTTP request for URL: ' + request.url);
  response.writeHead(200);
  response.end('Hello World!, v1.0');
};
var server = http.createServer(handleRequest);
server.listen(port);

You can find the source code used in this demo at the following github repo:
https://github.com/socketxp-com/webhook-autoupdate

# Git clone the demo app

First clone the demo app from GitHub.

$ git clone https://github.com/socketxp-com/webhook-autoupdate.git

# Install Nodemon

Change the working directory to the "webhook-autoupdate" git folder you just cloned above. Install nodemon globally and run it.

$ cd webhook-autoupdate

$ npm install -g nodemon
$nodemon
[nodemon] 2.0.4
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node http-server.js`

Curl the demo web app

Access the NodeJS web app running at http://localhost:8080 using curl.

$curl http://localhost:8080
Hello World!, v1.0

# Install Webhook app

To handle push notifications (or webhooks) from github (or any webhook sender application for that matter), you don't need to implement a webhook handler yourself. There are apps available to offload the job. One such app is Webhook app.

Using the Webhook app easily create HTTP endpoints (webhooks handler) on your server, which you can use to execute configured commands. Webhook app also allows you to specify rules which must be satisfied in order for any hook to be triggered.

Download the Webhook app (choose one appropriate for your OS and Platform) from the github repo release folder

$wget https://github.com/adnanh/webhook/releases/download/2.7.0/webhook-darwin-amd64.tar.gz
$ls
webhook-darwin-amd64.tar.gz
$tar -xvf webhook-darwin-amd64.tar.gz 
x webhook-darwin-amd64/
x webhook-darwin-amd64/webhook
$ls
webhook-darwin-amd64        webhook-darwin-amd64.tar.gz
$cd webhook-darwin-amd64
$ls
webhook
$

You'll find a "hooks.json" file in the "webhook-autoupdate" GitHub repository that you cloned in the previous section. Update the hooks.json file with the correct directory names and secret value, as shown by the marked lines below.

[
  {
    "id": "webhook",
    "execute-command": "/Users/gannygans/webhook-autoupdate/update-nodejs.sh",   // <<<=== Update this field to your directory path
    "command-working-directory": "/Users/gannygans/webhook-autoupdate",    // <<<=== Update this field to your directory path
    "response-message": "I got the payload!",
    "response-headers":
    [
      {
        "name": "Access-Control-Allow-Origin",
        "value": "*"
      }
    ],
    "pass-arguments-to-command":
    [
      {
        "source": "payload",
        "name": "head_commit.id"
      },
      {
        "source": "payload",
        "name": "pusher.name"
      },
      {
        "source": "payload",
        "name": "pusher.email"
      }
    ],
    "trigger-rule":
    {
      "and":
      [
        {
          "match":
          {
            "type": "payload-hash-sha1",
            "secret": "adsFgGdf1T23423",    // <<<<==== Update the secret 
            "parameter":
            {
              "source": "header",
              "name": "X-Hub-Signature"
            }
          }
        },
        {
          "match":
          {
            "type": "value",
            "value": "refs/heads/master",
            "parameter":
            {
              "source": "payload",
              "name": "ref"
            }
          }
        }
      ]
    }
  }
]

Use this hooks.json file to kickstart the webhook app.

$./webhook -hooks hooks.json -verbose
[webhook] 2020/06/25 16:27:42 version 2.7.0 starting
[webhook] 2020/06/25 16:27:42 setting up os signal watcher
[webhook] 2020/06/25 16:27:42 attempting to load hooks from hooks.json
[webhook] 2020/06/25 16:27:42 os signal watcher ready
[webhook] 2020/06/25 16:27:42 found 1 hook(s) in file
[webhook] 2020/06/25 16:27:42   loaded: webhook
[webhook] 2020/06/25 16:27:42 serving hooks on http://0.0.0.0:9000/hooks/{id}

Note: The above webhook URL "http://0.0.0.0:9000/hooks/{id}" is a localhost url. Github cannot send notifications to this IP address because it is not a publicly reachable IP address. Github needs a publicly reachable public URL to send push notifications.

This is where SocketXP solution comes in handy. We can use the SocketXP Secure Reverse Proxy Tunneling service to relay webhooks notifications from Github to the localhost Webhook app.

# Install SocketXP agent

Download and install SocketXP agent on your system following the instructions here - SocketXP Downaload and Install.

After SocketXP agent is installed, request the agent to relay the github webhook to your local webhook URL as shown below.

$ socketxp relay http://0.0.0.0:9000/hooks/webhook

Connected.
Public URL -> https://webhook.socketxp.com/ganeshvelrajan-0er32gfj

Now copy the above SocketXP Public URL and use it to update the webhook settings in the Github page of your project repository as shown below. Also make sure you select the payload content type to "applicatin/json" in the dropdown box. Save the changes.

Begin Auto Deploy Demo

Let's begin the demo by committing a code change into Github.

Git commit and push

Commit a code change into your Github project. I have bumped up the version string in my http-server.js" to version 2.0 and committed the code changes.

$git commit http-server.js -m "minor fix"
[master cae0d29] minor fix
 1 file changed, 1 insertion(+), 1 deletion(-)
$git push
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 286 bytes | 286.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/socketxp-com/webhook-autoupdate.git
   f4f1516..cae0d29  master -> master

SocketXP webhook relay agent

Check if the SocketXP agent has received the webhook notifications from Github.

Webhook received:
POST /hooks/webhook HTTP/1.1
Host: webhook.socketxp.com
Accept: */*
Accept-Encoding: gzip
Content-Length: 8077
Content-Type: application/json
User-Agent: GitHub-Hookshot/3ea2330
X-Github-Delivery: b55dd9f6-b6d3-11ea-9b25-2831f872cb2c
X-Github-Event: push
X-Hub-Signature: sha1=43b9f9114ecf5c3da6e306976f95766f52c6bdd9

{"ref":"refs/heads/master","before":"714706d5f6f411581ec7abf1653c4a1e3cd6d53b","after":"49d0ec0d1310917338baf1dcda5e3bc99b2f2644","repository":{"id":274867773,"node_id":"MDEwOlJlcG9zaXRvcnkyNzQ4Njc3NzM=","name":"webhook-autoupdate","full_name":"socketxp-com/webhook-autoupdate","private":false,"owner":{"name":"socketxp-com","email":"64076796+socketxp-com@users.noreply.github.com",  ... }

Check the webhook app

Check if the webhook app has received the webhook notification from the SocketXP agent and processed the webhook notification.

$./webhook -hooks hooks.json -verbose
[webhook] 2020/06/25 16:27:42 version 2.7.0 starting
[webhook] 2020/06/25 16:27:42 setting up os signal watcher
[webhook] 2020/06/25 16:27:42 attempting to load hooks from hooks.json
[webhook] 2020/06/25 16:27:42 os signal watcher ready
[webhook] 2020/06/25 16:27:42 found 1 hook(s) in file
[webhook] 2020/06/25 16:27:42   loaded: webhook
[webhook] 2020/06/25 16:27:42 serving hooks on http://0.0.0.0:9000/hooks/{id}
[webhook] 2020/06/25 16:35:00 [0fd63b] incoming HTTP POST request from 127.0.0.1:57435
[webhook] 2020/06/25 16:35:00 [0fd63b] webhook got matched
[webhook] 2020/06/25 16:35:00 [0fd63b] webhook hook triggered successfully
[webhook] 2020/06/25 16:35:00 [0fd63b] 200 | 18 B | 531.245µs | webhook.socketxp.com | POST /hooks/webhook
[webhook] 2020/06/25 16:35:00 [0fd63b] executing /Users/gannygans/webhook-autoupdate/update-nodejs.sh (/Users/gannygans/webhook-autoupdate/update-nodejs.sh) with arguments ["/Users/gannygans/webhook-autoupdate/update-nodejs.sh" "49d0ec0d1310917338baf1dcda5e3bc99b2f2644" "socketxp-com" "64076796+socketxp-com@users.noreply.github.com"] and environment [] using /Users/gannygans/webhook-autoupdate as cwd
   057cadf..069d282  master     -> origin/master
Updating 057cadf..069d282
Fast-forward
 http-server.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Current branch master is up to date.

[webhook] 2020/06/25 16:35:02 [0fd63b] command output: Already up to date.
Current branch master is up to date.

[webhook] 2020/06/25 16:35:02 [0fd63b] finished handling webhook

Check Nodemon has restarted the web server

$nodemon
[nodemon] 2.0.4
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node http-server.js`
[nodemon] restarting due to changes...

Curl the web server

$curl http://localhost:8080
Hello World!, v2.0

The hello-world application displays "v2.0" in the output, which proves that the auto deployment has worked as we expected.

That's all folks!

You can receive webhooks from any online service to your local server using SocketXP. SocketXP eliminates the need to own a public IP address to receive webhook notifications from online services. Moreover, SocketXP has a free-tier for developers.

Turnkey Solution for Raspberry Pi Remote SSH Access Over the Internet

gvelrajan — Wed, 24 Jun 2020 06:47:59 +0000

Imagine managing few thousands of Raspberry Pi devices at your work or at your customer site behind firewall or NAT. How would you configure, setup, access or even debug them from remote locations?

SocketXP provides turnkey IoT solution that can bring up hundreds or thousands of RPi devices online for remote SSH connectivity in less than a minute(limited only by your ability to copy paste a command).

Not just that, it is highly secure (uses SSL/TLS tunneling - a lightweight VPN alternative solution)

How the SockeXP turnkey solution for Raspberry Pi Remote SSH Access over the Internet

Just 2 simple steps.

Step #1

Copy a single command from SocketXP Portal Page as shown below.

Note: The above command has a unique auth-token assigned just for you and your devices.

Paste the above command on your Raspberry Pi device terminal. Paste this command on as many number of devices as you want to bring up for remote connectivity. Just one command for all your RPi devices.

Step #2

Check if your device is up and connected on the SocketXP portal page as shown below.

Click the terminal icon next to your RPi device.

Provide your RPi SSH credentials in the next screen.

Boom... and there you are, now inside your RPi SSH shell.

Now type any command in the shell. Say for example "htop".

How hard was this? Easy-Peasy isn't it?

Let me know what you think about this single-touch installation and our SocketXP Raspberry Pi remote SSH connectivity solution in general.

BTW, did I say that it's completely free for adding upto 4 RPi devices?

Got 5 or more devices? SocketXP service is just 20 Cents per device for one full month of remote connectivity ? And that too backed by a 14-day money back guarantee!!

Read more about the full solution here:
https://www.socketxp.com/iot/remote-access-iot-ssh-over-the-internet/