DEV Community: Amritanshu Pandey

Setup NFS Storage Class on Kubernetes

Amritanshu Pandey — Tue, 28 Sep 2021 13:00:36 +0000

Create the following resources in order to setup an NFS storgae provisioner in a Kubernetes cluster. This allows creating Kubernetes volumes on an NFS server.

Since the code required for creating various K8S resources for NFS provisioner is split into
multiple files, I prefer to just clone this gist and using kubectl to create all resources
in one go

$ git clone https://gist.github.com/amritanshu-pandey/8ab00179c98720cbc28d8bb0c7064426
$ cd 8ab00179c98720cbc28d8bb0c7064426
$ kubectl apply -f *.yaml

Note: This article was originally posted at https://blog.amritanshu.in/posts/setup-nfs-storage-class-on-kubernetes/

Create self signed certificates for Kubernetes using cert-manager

Amritanshu Pandey — Sat, 13 Feb 2021 21:37:22 +0000

Install Cert manager in Kubernetes

Read this for up-to-date instructions: https://cert-manager.io/docs/installation/kubernetes/

# Kubernetes 1.16+
$ kubectl apply —validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.0.2/cert-manager.yaml

Create a keypair secret

In this step create a new k8s secret that contains the TLS CA cert and key that is used by cert manager to issue new certificates. As a prerequisite, we need a CA certificate and associated key encoded in base64.

apiVersion: v1
kind: Secret
metadata:
  name: ca-key-pair
  namespace: default
data:
  tls.crt: <tls-key-base64-encoded>
  tls.key: <tls-key-base64-encoded>

Create an issuer

Issuers are used by Cert manager to issue new certificates

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: ca-issuer
  namespace: default
spec:
  ca:
    secretName: ca-key-pair

Create certificates

This creates new certificate using the issuer and CA key pair created earlier. In the following example, the certificate is stored as k8s secret k8s-xps-lan in default namespace.

apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: k8s-xps-lan
  namespace: default
spec:
  secretName: k8s-xps-lan
  issuerRef:
    name: ca-issuer
    # We can reference ClusterIssuers by changing the kind here.
    # The default value is Issuer (i.e. a locally namespaced Issuer)
    kind: Issuer
  commonName: k8s.xps.lan
  organization:
  - XPS.LAN
  dnsNames:
  - gitlab.xps.lan
  - minio.xps.lan
  - registry.xps.lan
  - k8s.xps.lan
  - kibana.xps.lan
  - elastic.xps.lan

In a separate post, we will see how this certificate can be used by ingress-nginx and other applications.

How to trigger Gitlab pipelines using REST API

Amritanshu Pandey — Wed, 16 Dec 2020 11:15:40 +0000

Gitlab pipelines are the CI(Continuous Integration) build job(s), that are usually triggered when a new commit is pushed to a git repository. Pipelines can be used to test, compile and deploy the code or to automate certain tasks. In addition to changes in git repositories, pipelines can also be triggered through other mechanisms as well. For instance:

Manually through the GitLab UI
Based on a schedule
Using GitLab's REST API

In this article, I will explain how to trigger a GitLab pipeline using the GitLab's REST API.

Create pipeline trigger

Navigate to project settings -> CI/CD -> Pipeline triggers
Create a new trigger and make note of the token. Token is used to authenticate the REST calls to trigger the pipeline.
Also make note of the URL displayed in the same page that must be used to trigger the pipeline. For e.g. https://git.xps.lan/api/v4/projects/14/trigger/pipeline

Trigger pipeline using `curl`

This curl command can be used to trigger the pipeline for project with id 14 for branch or tag specified as ref in the curl command

curl -X POST \
     -F token=<token-noted-in-previous-step> \
     -F "ref=<ref-name-like-branch-or-tag>" \
     https://git.xps.lan/api/v4/projects/14/trigger/pipeline

Sometime it might be required to pass on some variables to the pipeline while triggering the same. The variables passed while triggering the pipeline are available as environment variables at run time and takes precedence over all other variables.

Following command will provide a variable named RUN_NIGHTLY_BUILD with value as true to the pipeline. Any number of variables can be provided like this.

curl -X POST \
     -F token=<token-noted-in-previous-step> \
     -F "ref=<ref-name-like-branch-or-tag>" \
     -F "variables[RUN_NIGHTLY_BUILD]=true" \
     https://git.xps.lan/api/v4/projects/14/trigger/pipeline

Using Powershell Invoke-WebRequest

For the users on Windows, it might be bit easier to use Powershell and Invoke-WebRequest, rather than using curl on Windows or WSL.

Invoke-WebRequest https://git.xps.lan/api/v4/projects/14/trigger/pipeline `
    -Method POST `
    -Body @{
         "token"="<token-noted-in-previous-step>"
         "ref"="<ref-name-like-branch-or-tag>"
         "variables[RUN_NIGHTLY_BUILD]"="true"
     }

Install and manage multiple versions of python using Pyenv

Amritanshu Pandey — Sun, 31 May 2020 05:47:35 +0000

YouTube:

systemd-journald

Amritanshu Pandey — Sun, 01 Mar 2020 01:46:10 +0000

This article was originally published on my blog at - https://blog.amritanshu.in/posts/systemd-journald/

journald is a part of the systemd suite of softwares that provide variety of system components for Linux based
operating systems. journald is a daemon responsible for recording logs generated by Linux kernel and other
applications installed on the system.

Since its inception and later mass adoption by majority of the Linux distributions, systemd has remained a
controversial topic among the Linux enthusiasts and sysadmins. Now there are both valid and not-so-sound arguments
against systemd; but the fact that almost every major Linux distribution today uses systemd speaks for its merit
and is a reason good enough for us to understand it better.

I for one quite like the convenience and coherent ecosystem of utilities and commands that systemd provides and
in this series of web posts I want to share some of the systemd related tid-bits and the use-cases I solve using
systemd utilities.

This first post is about the log collection and management utility provided by the systemd i.e. systemd-journald.

What is `systemd-journald`

systemd-journald is a daemon to collect event logs into its data store in binary format. This daemon can be
configured by modifying the configuration file that by default is stored at /etc/systemd/journald.conf.

Configuration files for all systemd daemons (including journald) are in INI format and are stored under
/etc/systemd/. After modifying the INI files it is necessary to restart the corresponding daemon using systemctl

How to restart journald

systemctl restart systemd-journald

Detailed reference for the journald config can be found at the freedesktop website here - https://www.freedesktop.org/software/systemd/man/journald.conf.html#

Few facts about the journal daemon and its config that stand out for me are:

journald can be configured to store the logs in either the persistent file system under the default path of
/var/log/journal/ or in a volatile file system under /run/log/journal by default. Logs stored in the
persistent file system are stored permanently depending upon the retention factors specified in config like total
size, age etc. Logs stored in the volatile file system also obey similar retention logic, but are removed upon
reboot.
journald can be configured to forward the logs to syslog. This the default behavior for most Linux distributions.
For e.g. in fedora journald logs are also stored in syslog format in plain text in various files under /var/log.

   $ sudo tail -3 /var/log/messages

   Feb 28 11:49:03 xpsws NetworkManager[773]: <info>  [1582850943.6441]
   Feb 28 11:49:03 xpsws systemd[1]: Starting Network Manager
   Feb 28 11:49:03 xpsws systemd[1]: Started Network Manager

What is `journalctl`

Journalctl is a command line interface to query the logs stored by journald in its binary store.

When invoked without any parameters, it displays all the logs in default syslog format. Following are a few more
examples of the common journalctl commands that highlight its powerful features.

Common commands

To follow all the logs on stdout (analogous to tail -f /var/log/syslog)

$ journalctl -f

To display logs pertaining to a service (systemd unit) and display only last 12 log entries

$ journalctl -u <servicename> -n 10

To display logs in a different format e.g. json

$ journalctl -u <servicename> -n 2 -o json

Notable supported formats: json, json-pretty, short, short-iso, verbose

To display the amount of disk used by journald logs

$ journalctl --disk-usage

Archived and active journals take up 24.0M in the file system.

A detailed reference guide for the journalctl cli can be found here - https://www.freedesktop.org/software/systemd/man/journalctl.html