DEV Community: Yankee Maharjan

Build CLI blazingly fast with python-fire 🔥

Yankee Maharjan — Mon, 05 Jun 2023 02:37:07 +0000

Command line applications are developers best friend. Want to get something done quickly? Just a few keystrokes and you already have what you are looking for.

Python is the first language many developers pick if they need to hack together something quickly. But what we scrap together is not a CLI in its entirety most of the time, you need to manage flags, parse the arguments, chain sub-commands, and many more which is a hassle, thus results in multiple small and unmanaged scripts.

In todays article we are going to put an end to this and see how we can build reasonably feature rich CLI in mere minutes without any fancy decorators or anything.

Create and activate a virtual environment

python -m venv venv

source venv/bin/activate

# Install python-fire 🔥
pip install fire

Your first sub-command

Our CLI application is going to be an aggregation of bunch of tools, so we will just call it tools CLI.

With python-fire you can use either function or class to create your subcommands. But I find working with classes more intuitive and manageable. Our first command is going to be a sub-command that shows us the UTC time.

We will create a new method utc() which will be our sub-command and we have an argument called pretty which will be the flag for our sub-command which prints UTC date time in more human readable format. This argument already has a default value so this is not a required flag.

# tools.py

from datetime import datetime
import fire

class Tools:
    def utc(self, pretty: bool = False):
        """
        Get UTC date time
        """
        utc_time = datetime.utcnow()

        if pretty:
            ## strftime format codes:
            # https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
            print(utc_time.strftime("%B %d :: %H:%M %p"))
        else:
            print(utc_time)

Next we need to run this file as a script so at the end of our file we need to add the following:

if __name__ == "__main__":
    fire.Fire(Tools)

Now we have our CLI ready! Let’s run it!!

python tools.py utc
python tools.py utc --pretty

# For help message
python tools.py

# For sub-command help message
python tools.py utc --help

It’s a bit overwhelming to type through all of these overheads python tools.py <command>; well it’s not even like a CLI I was hoping for 🤷‍♂️.
You might have questions like:

How can I invoke it from any location I want?
I want to name it to something I find intuitive, how do I do that?

Well for that you would want to create a distribution for it.

Package for Command Line 📦

First we need to revamp our program a little to accommodate packaging:

# REMOVE THIS CHUNK
if __name__ == "__main__":
    fire.Fire(Tools)

# ADD THIS CHUNK
def run():
    fire.Fire(Tools)

Now let’s create a setup.py file to manage our packaging/distribution. You can use this file as a reference to create your own CLI:

# setup.py
"""Package setup"""
import setuptools

# Development Requirements
requirements_dev = ["pytest", "black", "mypy", "flake8", "isort"]

setuptools.setup(
    name="tools_cli",
    version="0.0.1",
    author="Yankee Maharjan",
    url="https://yankee.dev/build-cli-blazingly-fast-with-python-fire",
    description="Collection of handy tools using CLI",
    license="MIT",
    packages=setuptools.find_packages(exclude=["dist", "build", "*.egg-info", "tests"]),
    install_requires=["fire"],
    extras_require={"dev": requirements_dev},
    entry_points={"console_scripts": ["to = tools:run"]},
)

The line you need to focus here is the entry_points, which describes the entry point to our program as a console script.

entry_points={"console_scripts": ["to = tools:run"]},

Here to is the name of our CLI, you can name it to anything you like. If you want to name it to brr it will go like this:

entry_points={"console_scripts": ["brr = tools:run"]},

tools:run represent the name of our module followed by the function it needs to run. Console Scripts always require a function to run hence the modification we did earlier.

Feels like a CLI 💆‍♂️

Now let’s install our CLI in an editable mode within our virtual environment. This is like hot reloading for your CLI, whatever changes you make is reflected instantly.

Inside your project directory run the following command.

pip install -e .

Now you can use your CLI using the command to or whatever you put on the console_scripts

to utc
to utc --pretty
to utc --help

This is pretty neat!

Now how do I make sure I can run it from any location I want?

Deactivate your virtual environment:
```
deactivate
```
Install the project in editable mode again on your global site-packages:
```
pip install -e .
```

Now this is done, you will have your CLI accessible throughout the system. But note that if you make any changes to the main CLI logic, it will be reflected instantly.

Bonus: Nested Commands ➿

If you have made it this far, then you are set to make your own CLI and get done with most of the use cases. But if you want to see some more then stick around for a bit.

Let’s add other commands to our tool, first a sub-command called leap() that validates if the given year is leap or not, lastly a sub-command called pw() to generate a strong password.

...
import calendar
import string
import secrets

class Tools:
    def utc(self, ...):
        ...

    def leap(self, year:int): # required: since no default value here
        """
        Check if given year is leap or not
        """
        print(calendar.isleap(year))

    def pw(self, len: int = 16):
        """
        Generate strong password
        """
        alphabet = string.ascii_letters + string.digits + string.punctuation
        pwd_length = len

        pwd = ""
        for i in range(pwd_length):
            pwd += "".join(secrets.choice(alphabet))

        print(pwd)

def run():
    fire.Fire(Tools)

Now run the commands

to leap 2022
to pw
to pw --len 22
to pw 25

Sometimes there are related commands that you want to group together, like in our case we can group utc and leap under something like datetime or dt for short. Basically what we want to do here is nested commands.

Let’s group our commands. We will move our leap() and utc() method inside of a new class called DateTime.

...

class DateTime:
    def utc(self, pretty: bool = False):
        """
        Get UTC time
        """

        from datetime import datetime

        utc_time = datetime.utcnow()

        if pretty:
            print(utc_time.strftime("%B %d :: %H:%M %p"))
        else:
            print(utc_time)

    def leap(self, year:int): # required: since no default value here
        """
        Check if given year is leap or not
        """
        print(calendar.isleap(year))

class Tools:
    def __init__(self):
        self.dt = DateTime()

    def pw(self, ...):
        ...

Sub-command is determined by whatever we put as variable name when instantiating the new DateTime class. Here we have named it as dt, but you can name it as datetime, dtt or whatever you want.

Now, we have more organized sub-commands for our CLI. If you want to run commands that are related to date time you can do so using to dt <command-name> ; for example:

to dt utc
to dt leap 2025

Password command will be normal:

to pw
to pw --len 30

Conclusion

Using python-fire makes the process of creating CLIs really easy and intuitive because you are using nothing but Python functions and classes. I hope this mini rundown of the tool and how to package it for your daily use has been helpful.

Streamline Github Actions Releases with Github CLI

Yankee Maharjan — Tue, 14 Feb 2023 04:08:37 +0000

GitHub Actions is the default choice for CI/CD in many Open Source and Enterprise projects. It gets regular feature updates and is flexible enough to get most of the things done.

One of the powerful feature it provides is the ability to re-use Actions, which is available on Marketplace and we can use it from any GH repos. Most common Actions that I see being used almost everywhere are related to:

creating a Release
generating Release Notes for the release
and uploading any Artifacts for the release

Well, I was doing the same until recently. Playing around with the GitHub CLI I figured everything I mentioned above could be done with it effortlessly! ✨ Not to mention it is flexible and provides options to move things our way. ++ It's already installed on the GitHub Actions runner.

Enough talk, now let's see it in Action 🥁

Creating a Release

GitHub CLI provides a built-in sub-command to create a release. 🌟

For a basic example: we want to create a release whenever we push a tag 🏷️

steps:
  - uses: actions/checkout@v3
  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/}

gh release create <tag> : create a new release with the latest tag pushed
-t <tag>: Set title of the release; same as the pushed tag

Apart from general releases, we can also create draft releases and prerelease with the following flags.

-d : draft release
-p: prerelease

Generate Release Notes

Now this one is pretty easy. We just have to pass the --generate_notes flag to the above command and it will be generated for us.

steps:
  - uses: actions/checkout@v3
  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/} --generate-notes

If we want more flexibility then it also provides options to generate release notes from a file or from STDIN with the -F flag.

Upload artifacts with the Release

Now this gets more interesting. You won't believe me if I say it.

To upload artifacts to our release all we have to do is pass the file or directory names to the same command!

I am taking an example of my repository where I build browser extension artifacts for Chrome and Firefox:

permissions: write-all
name: Create GH Release
steps:
  - uses: actions/checkout@v3
  - name: Build Artifacts
    run: ./release.sh

  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/} 12_ft_chrome.zip 12_ft_firefox.zip --generate-notes

permissions: write-all: For uploading artifacts, we will need to assign additional permissions to our GITHUB_TOKEN.
./release.sh: Builds Release Artifacts
12_ft_<browser>.zip: are the artifacts generated by the above script

If we want a separate step to upload our artifacts, we can do so with the following change:

permissions: write-all
name: "Create GH Release"
steps:
  - uses: actions/checkout@v3
  - name: "Build Artifacts"
    run: ./release.sh

  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/} --generate-notes

  - name: Upload Artifact to Release
    run: gh release upload  ${GITHUB_REF#refs/*/} 12_ft_chrome.zip 12_ft_firefox.zip

Conclusion

We don't have to dabble around with multiple third-party actions, all of it can be done from a single command using GitHub CLI. This is more manageable, and effortless.

Hope this has been helpful! ✨

Full reference to the above Action YAML 🚀

6 Tools to Run Kubernetes Locally

Yankee Maharjan — Thu, 05 Aug 2021 08:00:07 +0000

Kubernetes is a big and complicated technology and it clearly requires some time and dedication to wrap your head around. There is no vendor lock-in meaning it runs the same no matter which managed cloud platform you use it on. This means using it locally will be no different from using it on the cloud.

There are multiple tools for running Kubernetes on your local machine, but it basically boils down to two approaches on how it is done:

running it from a single binary package
running it as a container using Docker in Docker (DinD)

Kubernetes Marketplace

alexellis / arkade

Open Source Marketplace For Developer Tools

arkade - Open Source Marketplace For Developer Tools

arkade is how developers install the latest versions of their favourite CLI tools and Kubernetes apps.

With arkade get, you'll have kubectl, kind, terraform, and jq on your machine faster than you can type apt-get install or brew update.

With over 120 CLIs and 55 Kubernetes apps (charts, manifests, installers) available for Kubernetes, gone are the days of contending with dozens of README files just to set up a development stack with the usual suspects like ingress-nginx, Postgres, and cert-manager.

arkade - Open Source Marketplace For Developer Tools

View on GitHub

Before we move on to talk about all the tools, it will be beneficial if you installed arkade on your machine. It will help you get these tools with a single command.

curl -sLS https://get.arkade.dev | sudo sh

Features?

All of the tools listed here more or less offer the same feature, including but not limited to:

Multi-Node cluster
Persistent volumes
Networking
Certificates
Bare-metal support
Dashboard
Kubernetes Versions
Add-ons
Cross-platform
Tracks upstream Kubernetes

Single package binary

k3s

k3s is a lightweight Kubernetes distribution from Rancher Labs. It is specifically targeted for running on IoT and Edge devices, meaning it is a perfect candidate for your Raspberry Pi or a virtual machine.

It comes with a single binary of mere <40 MB and takes as low as 500 MB of RAM.

You can bootstrap k3s quickly using k3sup !

  arkade get k3sup

k0s k0s is the latest entry on the block. By name, you might think it's a more stripped version of k3s, but it's an entirely different distribution from an entirely different company, called Mirantis. Contrary to the name, it comes in a larger binary of 150 MB+.

It can be run as a binary or in DinD mode. k0s takes security seriously and out of the box, it meets the FIPS compliance. Although, a new distribution, k0s has reached the production-ready status, so there wouldn't be an issue for development usage.

  arkade get k0s

Microk8s

MicroK8s is a Kubernetes distribution by Canonical, the company behind Ubuntu. You already saw this coming; it can only be installed using snap. It comes with loads of add-ons baked in like Fluentd, Grafana and Prometheus.

If you are on Ubuntu or its derivatives that uses snap you'll feel right at home using MicroK8s.

  sudo snap install microk8s --classic

Docker in Docker (DinD)

Running Docker inside of Docker (Inception anyone?) is a popular way of bootstrapping Kubernetes. The isolating nature of Docker makes running a multi-node cluster a breeze on a single machine, and also ensures the running instance does not affect the machine itself.

As the title suggests, you need to have Docker installed on your machine to go this route.

minikube Despite running on top of Docker and similar container technologies, minikube is really flexible in how it operates and supports multiple virtualization drivers making it adaptable to different computing environments. These include KVM2, Virtualbox, Podman, Hyperkit, Hyper-V and many more.

  arkade get minikube

KinD Kubernetes in Docker (KinD) is similar to minikube but it does not spawn VM's to run clusters and works only with Docker. KinD for the most part has the least bells and whistles and offers an intuitive developer experience in getting started with Kubernetes in no time.

  arkade get kind

k3d k3d is basically running k3s inside of Docker. It provides an instant benefit over using k3s on a local machine, that is, multi-node clusters. Running inside Docker, we can easily spawn multiple instances of our k3s Nodes.

  arkade get k3d

Conclusion

Either you choose a single binary package or DinD approach, Kubernetes has made itself pretty accessible. For new learners, the barrier to entry is low, and the feedback loop is instantaneous.

I hope this article has been helpful in deciding which tool to use for running your local Kubernetes instance.

Practical Guide to Git Worktree

Yankee Maharjan — Mon, 12 Apr 2021 17:45:00 +0000

Git has a solution to all of our problems, you just need to know where to look. As developers, context switching is a part of the job that you need to account for in more than a few occasions.

Problem Statement

Imagine this; you are working on a feature where you have made bunch of changes to files that are not yet commited, and suddenly you need to work on a hot fix or a more priority feature. There are two ways you can tackle this:

Using Git Stash workflow
Using Git Worktree (You are here! 📍)

Git worktree to the rescue 🌳

Git worktree helps you manage multiple working trees attached to the same repository.

In short, you can check out multiple branches at the same time by maintaining multiple clones of the same repository.

OK back to our problem! Update changes? New Feature? Hot Fix? Whatever it is, you need to change to a different branch and work on it without any changes to your current work directory.

Let’s say it’s a new feature, your workflow would look like this:

create an replica of your project and switch to a new branch
create a new feature
push it
back to previous working directory

Create worktree

Let’s say the name of your feature is feature-x and you want the branch with the same name. You can create additional worktree on the same directory or move it to a desired path, I prefer the later.

git worktree add command creates a worktree along with a branch that is named after the final word in your path.



git worktree add <PATH>

# Create feature-x directory and branch with the same name.
git worktree add ../feature-x

Named Branch

If you want to give you branch a unique name then you can use the -b flag with the add command.



git worktree add -b feature-xyz ../feature-xyz

Track remote branch

Let’s say you want to switch to a new branch that is tracking the branch at remote, where you want to push changes to.



git worktree add -b <branch-name> <PATH> <remote>/<branch-name>

git worktree add -b feature-zzz ../feature-x origin/feature-zzz

Create worktree with local branch



git worktree add <PATH> <branch-name>

git worktree add ../feature-xz feature-xz

View the list of worktrees with git worktree list

Switching Worktrees

As much easy it is to create a worktree, it is equally difficult to navigate back and forth between them if they are spread across. You have to git worktree list and then copy the path navigate to the worktree of your choice. To minimize this friction, I have built a small tool that let’s you switch between worktrees just with their partial or complete directory name.

Download wt CLI tool for faster switching between worktrees.

With this I can simply switch between my worktrees.

You can wt list which is equivalent to git worktree list to see the list of your worktrees. Now to move to feature-x worktree directory, I can just use wt feature-x to cd into that directory to continue with the work. To go back to my main worktree directory I can just wt -.

Remove Worktrees

Now that you have created a new worktree, switched to it and made your changes and pushed it. To remove the worktree, we can run:



git worktree remove <name-of-worktree>

git worktree remove feature-x

Conclusion

Git worktree is a handy feature that let's you context switch in your project to try out things on a completely different environment, without modifying your main work directory. This might come handy occasionally but it's pretty neat being able to do so without breaking a sweat.

I hope this guide has been helpful, if you have any queries or corrections, feel free to reach out to me.

Streamline your projects using Makefile

Yankee Maharjan — Sat, 26 Dec 2020 16:17:53 +0000

make is one of the tools that we use heavily for streamlining tasks on our projects. It has proven to be helpful specifically for streamlining the development process, repeating mundane tasks with custom CLI like subcommands and mainly onboarding new team members.

With a set of rules in Makefile, you can get up and running in no time, keeping the process sane and saving time and effort for everyone in the team. We'll be going through the basics to some interesting stuffs we can do with Makefile.

There are two pieces to this equation, one is the make CLI tool and the next is the Makefile . The basics is make reads the rules from the Makefile and executes them. What I will be showing today is just a small part of what make is capable of.

Writing Makefile

If you have worked with YAML files before then you will feel right at home writing Makefiles.

Anatomy of Rules

Every Makefile consists of rules with the anatomy of:

target: dependencies
    recipe

target: target can be an executable, object or just a name for an action that we want to carry out. We will be using targets purely with the placeholder name for the rule. Be mindful about the name, as it should resonate with the action we want to perform with no confusion whatsoever.
dependencies: Dependencies are the rules that needs to be executed, in order for the current rule to work.
recipe: recipe is the meat of the Makefile, it is the action that we want to perform with our target name. Make sure to put a tab character at the start of every recipe line (just like YAML). You can also replace the tab character with anything you want using the .RECIPEPREFIX variable.

Next we will be looking into some examples on how to make use of Makefile. These examples will be based on setting up development environments.

Basic Rules

A basic rule where you just want to put some alias is straight forward.

Let’s say you have a python project and you want to hand it over to a new team member. How do you streamline the setup process. Maybe it can look something like this.

Note:
# is for comments.

@ symbol is to disable printing the recipe to stdout.
Test without the @ symbol at the beginning of the recipe.

:= is the expansion operator which prevents using subsequent value with the same variable name.

SHELL variable determines the default shell to execute the recipe.

SHELL :=/bin/bash

.PHONY: format check

venv: # setup a virtual environment
    @python3 -m venv venv

setup: # install dev dependencies
    @pip install -e .[dev]
    @echo -e "\nInstalling pre-commit hook..."
    @pre-commit install

format: # format code using black
    @black .

check: # check for formatting using black
    @black --check --diff -v .

test: # run pytest
    @pytest -vvv

You can do something similar with your existing project.

Now to get up and running, all you have to do is:

$ make venv 

$ . venv/bin/activate 

$ make setup

$ make format 

# and so on

Rules with Dependencies

Taking the reference from the example above, suppose we want to print out the output of check target every time we run the format target. So how do we create that dependency? It’s plain simple, we just have to update the format target to look something like this:

format: check # run the formatter on files.
 @black .

We have added the dependency of check to the right of the target, just like showcased on the anatomy Anatomy of Rules section.

Variables

We can also define variables if we have some piece of command for repeated use. For this example we will be taking the reference of the Django management command.

Variables are normally written with all caps and uses := to assign variable name to a value. Variables can be accessed using either $() or ${} syntax.

DJANGO_MANAGE := python manage.py
run: 
 @${DJANGO_MANAGE} runserver

show: 
 @${DJANGO_MANAGE} showmigrations

migrate: 
 @${DJANGO_MANAGE} migrate

Also your SHELL environment variables are converted in to Makefile environment variables, so you can directly make use of them while creating your rules.

Example:

In our shell we can export an environment variable called INFO.

$ export INFO="Run make help to show all the available rules."

And now in the Makefile we can refer to it as any variable.

info: # show project info
    @echo ${INFO}

Default target

If you just run make on your command line nothing is going to happen. But we can change that by using the .DEFAULTGOAL special variable and assigning the target we want to run by default.

.DEFAULT_GOAL := run

Now, next time you run make it is going to run the Django server by default.

Self documenting

Now we have bunch of targets on our Makefile and we also called this combo as a custom mini CLI app. Wouldn’t it be great, if we could have a help command similar to a real CLI app? Say no more, thanks to the blog from Victoria Drake we have the script to do so.

Just create a help target and assign it as a .DEFAULT_GOAL. With this, all the comments we have been writing on our target gets converted into a nice help message.

.DEFAULT_GOAL := help 
help: # Show this help
 @egrep -h '\s#\s' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?# "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

Include other Makefiles

We can separate out Makefiles based on the tasks they perform and include them into the main Makefile. We usually have separate Makefile managed for environment variables, Docker and Kubernetes. This offloads all the tasks from project set up to Deployment to the Makefile.

I will show a brief example of each of the file just to give an example:

Note: Since make runs each recipe on a new instance of the shell, we can lazy evaluate the variables using ?= meaning, they are initialized only when referenced for a single shell instance.

Makefile
Root makefile composed of other Makefiles.

SHELL :=/bin/bash
APP_ROOT := $(PWD)
TMP_PATH := $(APP_ROOT)/.tmp
VENV_PATH := $(APP_ROOT)/.venv

export ENVIRONMENT_OVERRIDE_PATH ?= $(APP_ROOT)/env/Makefile.override

-include $(ENVIRONMENT_OVERRIDE_PATH)
include $(APP_ROOT)/targets/Makefile.docker
include $(APP_ROOT)/targets/Makefile.k8s

Environment Variables
Makefile.override
Makefile containing just the essential environment variables.

STAGE ?= <stage>
SERVICE_NAME ?= <service-name>
AKS_RESOURCE_GROUP ?= <resource-group>
AKS_CLUSTER_NAME ?= <cluster-name>
REGISTRY_URL ?= <registry-url>
AZ_ACR_REPO_NAME ?= <repo-name>

Docker
Makefile.docker
Makefile containing docker rules.

export GIT_COMMIT ?= $(shell cut -c-8 <<< `git rev-parse HEAD`)
export BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD)

export DOCKER_BUILD_FLAGS ?= --no-cache
export DOCKER_BUILD_PATH ?= $(APP_ROOT)
export DOCKER_FILE ?= $(APP_ROOT)/Dockerfile

export TARGET_IMAGE ?= $(REGISTRY_URL)/$(AZ_ACR_REPO_NAME)/$(SERVICE_NAME)
export TARGET_IMAGE_LATEST ?= $(TARGET_IMAGE):$(BRANCH)-$(GIT_COMMIT)

acr-docker-login:
    az acr login --name $(AZ_ACR_REPO_NAME)

docker-build:
    docker build $(DOCKER_BUILD_FLAGS) -t $(SERVICE_NAME) -f $(DOCKER_FILE) $(DOCKER_BUILD_PATH)

docker-tag:
    docker tag $(SERVICE_NAME) $(TARGET_IMAGE_LATEST)

docker-push: acr-docker-login
    docker push $(TARGET_IMAGE_LATEST)

Kubernetes

Makefile.k8s
Makefile containing rules for Kubernetes.

export OVERLAY_PATH ?= $(APP_ROOT)/k8s/overlays/$(STAGE)/

define kustomize-image-edit
    cd $(OVERLAY_PATH) && kustomize edit set image api=$(1) && \
    cd $(APP_ROOT)
endef

kubectl-apply:
    kustomize build $(OVERLAY_PATH)
    kustomize build $(OVERLAY_PATH) | kubectl apply -f -

update-kubeconfig:
    az aks get-credentials --resource-group $(AKS_RESOURCE_GROUP) --name $(AKS_CLUSTER_NAME)

aks-deploy: update-kubeconfig
    $(call kustomize-image-edit,$(TARGET_IMAGE_LATEST))
    make kubectl-apply

aks-delete: update-kubeconfig
    kubectl delete namespace $(STAGE)-api

kustomize-edit:
    $(call kustomize-image-edit,$(TARGET_IMAGE_LATEST))

Now we have orchestrated all these Makefiles, it is easier to keep track of all the rules and makes working with Makefiles sane, if you are doing a lot with it.

Conclusion

So with the use of Makefile we can streamline a lot of redundant tasks in our projects without having to remember overwhelmingly long and varying commands.

It increases the productivity of the whole team; with easier project setup and redundant tasks outsourced to the Makefile with intuitive target names, leaving the devs to focus on more serious tasks at hand.

How Rolling and Rollback Deployments work in Kubernetes

Yankee Maharjan — Sun, 25 Oct 2020 08:42:27 +0000

Kubernetes has been used heavily on production for the past few years. It offers a plethora of solutions for orchestrating your containers using its declarative API. One of the prominent feature of Kubernetes is its resilience with the ability to perform Rolling and Rollback Deployments.

Primer

Deployments

Deployment is one of the mechanisms for handling workloads (applications) in Kubernetes. It is managed by Kubernetes Deployment Controller.

In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state.

In case of deployment here, the desired state we want to achieve is for the pods. Everything is declarative in K8s, so the desired state is written as a spec in Deployment manifest file.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

If any of our pod instances should fail or update(a status change), the Kubernetes system responds to the difference between manifest spec and status by making a correction, i.e. matching the state of Deployment as defined on the spec.

Deployment under the hood

Deployment is an abstraction over ReplicaSet. Under the hood, Deployment creates a ReplicaSet which in turn creates pods on our cluster. As per the name, ReplicaSet is used for managing the replicas of our pods.

In summary, Controller reads the Deployment spec, forwards the pod configuration to ReplicaSet and then it creates the pods with proper replicas.

Deployment > ReplicaSet > Pods

Rolling Deployment

Kubernetes promises zero down time and one of the reasons behind it is Rolling Deployments. With Rolling Deployments, Kubernetes makes sure that the traffic to the pods are not interrupted when updated pods are being deployed.

Rollback Deployment

Rollback Deployment means, going back to the previous instance of the deployment if there is some issue with the current deployment.

Hands on 🙌

Let’s get our hands-on with Kubernetes and see how these deployment strategies are carried out.

Setup KinD

We will setup a single node Kubernetes cluster on our local machine using KinD (Kubernetes in Docker). Make sure you have Docker running.

Install KinD

We will create a K8s cluster using:

$ kind create cluster

With our cluster ready we are ready for Deployments.

For the purpose of this tutorial, we won’t be touching any YAML files and will be fully utilizing the power of kubectl CLI tool.

Create Deployment

Let’s create our first Deployment using the nginx image.

$ kubectl create deployment test-nginx --image=nginx:1.18-alpine

Like mentioned earlier, a Deployment creates a ReplicaSet followed by Pods. You can check these newly created resources using:

$ kubectl get deploy,rs,po -l app=test-nginx

List newly created Deployment, ReplicaSet and Pod

You can check to make sure the Replicaset was created by our Deployment using:

$ kubectl describe rs <replica-set-name>

Check ReplicaSet created by Deployment

You can do the same with Pods:

$ kubectl describe po <pod-name>

Scale Deployment

Let’s scale the deployment to have 3 instances of nginx pods.

$ kubectl scale deploy test-nginx --replicas=3

Creating 3 replicas of nginx pods

Now that we have significant number of pods on our cluster. Let’s try out the Deployment strategies.

Hands-on: Rolling Update Deployment 🍥⏩

Let’s say you were having some issues with v18 of nginx and the v19 fixes it for you. You need to rollout a new update of nginx image to your pod.

By updating the image of the current pods (state change), Kubernetes will rollout a new Deployment.

$ kubectl set image deploy test-nginx nginx=nginx:1.19-alpine

After we set the new image, we can see the old pods getting terminated and new pods getting created.

Rolling update, new pod replacing old ones

We can see Kubernetes at work, making sure the pods are maintained properly. The last of the old pod doesn’t get terminated until the complete replicas for the new pods are created. The old pods also have a grace period which makes sure the traffic it is serving isn’t disconnected for certain time until the requests can be safely routed to the newly created pods.

We successfully updated all our pods to use nginx v19.

$ kubectl describe deploy test-nginx

Pods updated to nginx v19

Hands-on: Rollback Deployment 🍥⏪

Let’s assume the new nginx update has even more problems than the last one and now you realized how blissful life was with the old version. Time for a rollback to the previous version of nginx.

But how do we do that? You might have noticed that there are now two ReplicaSets. It’s due to the same Deployment pattern we discussed earlier, we update our Deployment, it creates a new ReplicaSet which creates new Pods. Kubernetes holds history of up to 10 ReplicaSet by default, we can update that figure by using revisionHistoryLimit on our Deployment spec.

These history are tracked as rollouts. Only the latest rollout is active.

By now, we have made two changes to our Deployment test-nginx so the rollout history should be two.

$ kubectl get rs 

$ kubectl rollout history deploy test-nginx

$ kubectl rollout history deploy test-nginx --revision=1

Rollout history for test-nginx Deployment

Alright let’s get to rolling back our update to the previous rollout. We want to rollback to the stage where we were using nginx v18 which is rollout Revision 1.

$ kubectl rollout undo deploy test-nginx --to-revision=1

Rollout undo pod states

Like the Rolling Update Deployment, the Rollback Deployment terminates the current pods and replaces them with the pods containing the spec from Revision 1.

If you check the rollout history once again, you can see that the Revision 1 has been used to create the latest pods tagging it with Revision 3. There is no point in maintaining the same spec repeated for multiple revisions, so Kubernetes removes the Revision 1 since we have the latest Revision 3 of the same spec.

Revision history after rollout undo

Now we are back on the nginx v18. with the Rollback Deployment. You can check that out by:

$ kubectl describe deploy test-nginx

Check rollback nginx version

Conclusion 🚀

Kubernetes makes it easy to control the Deployment of our applications with these strategies. This was just a rundown of how Rolling and Rollback Update Deployments work from a ground level. In real-life, we rarely do all these steps manually, since we hand it down to our CI/CD pipeline like ArgoCD.

Thank you for reading. Hope this guide has been helpful for you to understand how Rolling and Rollback Deployments work in Kubernetes. If you have any corrections, suggestions, or feedback feel free to DM me on Twitter or comment down below.

Setting up multi-node Kubernetes cluster locally with K3s and Multipass

Yankee Maharjan — Sat, 24 Oct 2020 15:12:04 +0000

There are a lot of tools that allow you to setup a local Kubernetes cluster in no time. But with a full-blown K8s running on your local machine, you will soon hit a wall if you want to play with multi-node cluster.

For tackling this very issue, we will be looking into how we can setup a lightweight Kubernetes cluster using K3s and Multipass for our VMs.

If you are interested in what makes K3s so light, you can watch the talk on K3s under the hood.

Prerequisites

Installing Multipass

Multipass is a command line tool to orchestrate virtual Ubuntu instances on your local machine. With the CLI, you can spawn VMs within minutes allocating optimal CPU, memory and disk space.

Download Multipass

Installing K3sup (ketchup)

We talked about K3s but it can be daunting to set up if you just want a quick hands on with Kubernetes. To abstract all that we will be installing a handy CLI called k3sup. It will bootstrap Kubernetes cluster on our VMs using K3s within a minute! 🚀

k3sup uses SSH under the hood, so make sure you have SSH service on your machine.

Install k3sup

Creating Virtual machines

Nodes are just virtual machines on Kubernetes working in sync. For our cluster we will be creating 3 virtual machines, each with 1CPU, 1GB RAM, 2GB Storage.

Generating keys

If you have SSH installed and already have ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub , you can skip this part and move to **creating config file **section.

Create Private/Public key:

$ ssh-keygen

This will create the aforementioned files on your system. Copy the contents of ~/.ssh/id_rsa.pub

$ cat ~/.ssh/id_rsa.pub

Creating config file

Create a file called multipass.yaml and place your public key in ssh-rsa.

# multipass.yaml
ssh_authorized_keys:

  - ssh-rsa <add-your-public-key>

This config file makes sure the public key is stored on the virtual machine once it’s created. Let’s create our VMs with proper names based on their roles we will be assigning (master/worker).

$ multipass launch --cpus 1 --mem 1G --disk 2G --name master-node --cloud-init multipass.yaml

$ multipass launch --cpus 1 --mem 1G --disk 2G --name agent-master --cloud-init multipass.yaml

$ multipass launch --cpus 1 --mem 1G --disk 2G --name agent-worker --cloud-init multipass.yaml

$ multipass ls

In K3s terms, a master node is called the server and the rest of the nodes are called agents. Agents are simply the nodes that gets added to the master node; they can be another master node or a worker node.

Adding K8s sauce with k3sup

Now that we have our VMs ready, let’s install Kubernetes on them. First we will be creating a master node to setup a control plane.

Adding a master node

We will need the IP and username of our virtual machine to SSH into and install Kubernetes. Run multipass ls and take note of IP of master-node . All the usernames for VMs are ubuntu by default.

$ k3sup install --ip <IP> --user ubuntu --k3s-extra-args "--cluster-init"

We are passing the --k3s-extra-args "--cluster-init" to make sure this node is prepared to connect with another master node, else it might cause errors.

Once installed it downloads the kubeconfig file on the directory where you invoked your command. You can set environment variable KUBECONFIG with path to recently downloaded kubeconfig file.

# mac / linux
$ export KUBECONFIG=<path-to-kubeconfig>

# windows
$ setx KUBECONFIG <path-to-kubeconfig>

Now you can kubectl get nodes to view the nodes on your cluster.

Adding a second master node (HA)

Multi master setup on local machine is an overkill for most use cases but for the purpose of this tutorial we are going to add it anyway. To join a new master node in the cluster we use the join command.

Here, all we have to do is introduce the server IP and username (previously created) that this node should connect to . And additionally pass the --server flag specifying this will be a server node (master).

$ k3sup install --ip <IP-of-agent-master> --user ubuntu --server-ip <IP-of-master-node> --server-user ubuntu --server

And that’s about it. Now you can kubectl get nodes again, and you’ll see two master nodes that is Highly Available (HA).

Adding a worker node

Finally, we have to setup our worker node where we will be deploying our applications and services.

For this grab the IP of agent-worker and pass on the same command as before minus the --server flag.

$ k3sup join --ip <IP-of-agent-worker> --user ubuntu --server-ip <IP-of-master-node> --server-user ubuntu

Now if you kubectl get nodes , you will find two master nodes and a single worker node composing your multi node cluster setup. You can create new VMs and add more master or worker nodes depending on how you plan to utilize your cluster.

Conclusion

Thank you for reading. Hope this guide has been helpful for you to setup and play with multi-node Kubernetes cluster. If you have any corrections, suggestions, or feedback feel free to DM me on Twitter or comment below.

Happy hacking! 🚀

Mastering Git Stash Workflow

Yankee Maharjan — Sun, 23 Aug 2020 12:03:10 +0000

Git is a powerful tool that makes up for a lot of use cases on our development workflow. One such case is to isolate the changes of a certain branch to itself. Let me elaborate.

Suppose you are working on a branch called admin-dashboard implementing ,well, an administrative dashboard. But you are not done yet, and the project manager wants a quickfix for the login implementation. Now you want to switch to the login branch and fix the issue but don’t want to carry the changes you are doing on the admin-dashboard branch. Well this is where git stash comes in.

Git stash what?

Git stash allows you to quickly put aside your modified changes to a LIFO (Last In First Out) stack and re-apply them when feasible. We will be doing a rough walkthrough to see this in action.

Git stash walkthrough 🚶‍♂️

Initiate git on an empty directory. Add a file named add.py and put the following code.

# add.py 

def add(a, b):
    return a + b

Let’s add the file to git and commit it .

git add add.py && git commit -m "Add function"

Next let’s create and checkout to a new branch called mul and create a file called mul.py

git checkout -b mul

Add the following code to the file.

# mul.py 

def mul(a, b): 
    return a * b

Add the file to git and commit it.

git add mul && git commit -m "Mul function"

Now suppose, we need to update the mul function to take in a third argument and you just edited the function like so:

# mul.py

def mul(a, b, c): 
    return a * b

Take note that, we still haven’t updated the return value with c. While we were making our changes, the project manager called-in to update the add function immediately with a third argument. Now you can’t waste a second on the mul function which you haven’t completed. What are you going to do?

If you try to checkout to master where add function resides, git won’t let you because you have unfinished changes that hasn’t been committed.

Stashing Changes 📤

Well this is the situation you should be using the git stash command. We want to put away the changes on our current branch so that we can come back to it later.

Stash the file with a message on the mul branch.

git stash save "Multiply function"

Now if you git status, the working directory will be clean and you can jump into the master branch for the changes. We can view the items in our stash using git stash list. We can view the diff of the items on our stack using git stash show

Now you are in master branch modifying the add function, and comes an even higher priority job to include a subtraction function.

Note:
Project Managers in real life doesn’t put forward tasks on this manner.

Your add function looks pretty much like the mul function now.

# add.py 

def add(a, b, c): 
    return a + b   # couldn't include `c` due to a priority task.

Stash it with a message.

git stash save "Add function third argument"

Now let’s suppose we have completed our task for subtraction branch and we want to continue working on other branches.

Let’s move to the master branch first to complete our changes for addition.

There are two ways we can re-apply those changes:

git stash pop
applies the top most change stored on the stack and removes it from the stack.
git stash apply <item-id>
applies the stash based on the index provided, keeps the applied item intact on the stack.

Popping stashed changes 🍾

Like I mentioned earlier, stash follows the LIFO convention. The latest item we save are always on the top. And when we use pop, always the top most change is applied to the current branch. Run the following command on master.

git stash pop

Complete the add function and commit it.

Applying stashed changes 📥

Next, checkout mul branch. We can use pop here as well since there is only one item remaining on the stack. But let’s see how git stash apply works.

git stash apply stash@{0}

When we apply from the stash, the item still remains on the stack.
Complete the changes here and commit it.

Create a new branch with stashed changes

Let’s do something fun. Let’s say we want to add a divide function on a new branch. Well it is kind of similar to our multiple function so why not utilize the item in stash and create a divide function with it?

We can do that with the git stash branch command. It takes the <item-id> and a branch name, then applies those changes to that branch.

git stash branch <branch-name> <item-id>

git stash branch divide stash@{0}

Now we can rename the file and change the function to perform division.

Viewing the stashed changes

Sometimes we tend to forget what changes we stashed. What we forget can range from which files were stashed to what changes on the files were stashed.

To view a list of files that were stashed, we can run:

# EXAMPLE:
# git stash show stash@{<stash-id>}

git stash show stash@{2}

To get a diff view of changes on the files that were stashed, we can run:

# EXAMPLE:
# git stash show -p stash@{<stash-id>}

git stash show -p stash@{2}

Clearing the stack 🧹

Now that the stash has served its purpose, we can clear it. There is again two ways of doing it:

git stash clear
wipes the whole stack clean.
git stash drop <item-id>
removes the item from stack based on provided id.

Conclusion 🚀

Git stash is a powerful tool that comes handy in many situations. Hope this article helped you in understanding and implementing the concept in your project. If you have any suggestions or feedback, let’s talk on the comment section or you can @ me on Twitter.

Follow me on GitHub

Deploy your Serverless Python function locally with OpenFaas in Kubernetes

Yankee Maharjan — Sat, 15 Aug 2020 10:27:36 +0000

We have come a long way in building distributed applications. From monoliths to Microservices we have been able to decouple and scale our applications with optimal use of the underlying hardware resources. But now we are moving towards more lightweight approach to deploy our applications; Serverless! With the inception of Function as a Service (FaaS), application modularization has been taking an ascent in the world of cloud computing.

In this blog, we will be exploring development to deployment of our Serverless function locally.

The beauty of OpenFaas and KinD is that you don’t need to deploy your functions to cloud to test them, you can run them locally!

Prerequisite:

You just need to have Docker installed on your machine. The rest we will setup as we go along. We will be installing a few tools so bear with me. 🐻

OpenFaas? 🐬

OpenFaas is an awesome tool/framework developed by Alex Ellis with the Open Source community that helps to deploy burstable functions and microservices to Kubernetes without repetitive, boiler-plate coding.

It was built with developer experience in mind creating a low barrier of entry to the Kubernetes ecosystem. It comes with it’s own cli-tool called the faas-cli which makes a clever use of templates and makes pushing docker images and deployment of containers a breeze. And once deployed it takes care of auto-scaling and auto-provisioning based on the network traffic.

With all the overhead being handled by OpenFaas, all you have to do is focus on writing your function. But in our case we have some setting up to do 👨‍🏭

Properties of functions:

Here are few things to keep in mind when you are creating a function.

A rule of thumb is to make your function stateless.
Function should be atomic (single responsibility).
It should be idempotent.
Trigger to the function could be TCP or event, however effect must be an event.

Let’s get started! 🚀

Install arkade

arkade will be our one stop tool to install and deploy apps and services to Kubernetes. Think of this tool as a package manager like apt or pacman. We will be needing a bunch of CLI utilities to get up and running.

Make sure you install this if you have never set up a Kubernetes tooling in your system.

$ curl -sLS [https://dl.get-arkade.dev](https://dl.get-arkade.dev) | sudo sh

Just to make things clear:

use arkade get to download cli tools and applications related to Kubernetes.
use arkade install to install Kubernetes applications using helm charts or vanilla yaml files.

Install kubectl

kubectl is a command line tool that talks to the Kubernetes Master Controller API for performing actions on our cluster.

$ arkade get kubectl

# follow the instructions on the screen.

Creating a local Kubernetes cluster 🧱

We will be deploying our function on a local Kubernetes cluster so let’s set that up. We will be using a tool called KinD(Kubernetes in Docker) for this.

Unlike Minikube which requires VM; KinD utilizes Docker container to deploy your cluster. It is comparatively faster.

Install KinD

Our goal is to keep everything local so we will be creating a local Docker registry (Dockerhub for your local machine). KinD provides a shell script to create a Kubernetes cluster along with local Docker registry enabled.

#!/bin/sh
set -o errexit

# create registry container unless it already exists
reg_name='kind-registry'
reg_port='5000'
running="$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)"
if [ "${running}" != 'true' ]; then
  docker run \
    -d --restart=always -p "${reg_port}:5000" --name "${reg_name}" \
    registry:2
fi

# create a cluster with the local registry enabled in containerd
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:${reg_port}"]
    endpoint = ["http://${reg_name}:${reg_port}"]
EOF

# connect the registry to the cluster network
docker network connect "kind" "${reg_name}"

# tell https://tilt.dev to use the registry
# https://docs.tilt.dev/choosing_clusters.html#discovering-the-registry
for node in $(kind get nodes); do
  kubectl annotate node "${node}" "kind.x-k8s.io/registry=localhost:${reg_port}";
done

kind-with-registry.sh from KinD website

Once you have the file on your local machine. Run the following command to make it executable.

$ chmod +x kind-with-registry.sh

Now you can run the script to create your local Kubernetes cluster with local Docker registry.

$ ./kind-with-registry.sh

To make sure the kubectl context is set to the newly created cluster; run:

$ kubectl config current-context

If the result is not kind-kind; then run:

$ kubectl config use kind-kind

Make sure your cluster is running:

$ kubectl cluster-info

Make sure Docker registry is running.

$ docker logs -f kind-registry
OR 
$ docker ps -l

Deploying OpenFaas to KinD cluster

Now that we have our local Kubernetes cluster up and running lets deploy our OpenFaas services to support our functions.

$ arkade install openfaas

Make sure OpenFaas is deployed. It might take a minute or two.

$ kubectl get pods -n openfaas

Templates📂

Before we move to creating our function, it is essential to understand the concept of templates in OpenFaas.

What are templates?

Templates are basically a wrapper for your functions. Same template can be used for multiple functions.

OpenFaas already provides a variety of templates for different programming languages to start with.

There are two type of templates based on the webserver :

classic watchdog which uses template format from here.
of-watchdog which uses template format from here.

Watchdogs are basically webservers to proxy request to our functions.

We will be using the templates that supports the latest of-watchdog. You can read more about OpenFaas watchdog.

The most important piece in the templates directory is the Dockerfile, it drives everything from your watchdog to how and where your function gets placed in the template. You can also create your own template based on the need of your function.

Pull the template from store:

# list all the templates 
$ faas-cli template store list

# pull python3-flask template from store
$ faas-cli template store pull python3-flask

This will pull all the templates for python from OpenFaas store to your template directory of your project folder. We will be using the python3-flask-debian template. You can ignore rest of the templates or delete them.

Openfaas template structure.

Shown above is the basic template structure for all languages.

Template structure explanation:

index.py is an entrypoint to our Flask application.
requirements.txt on the root project directory contains dependencies for our Flask application. It contains Flask and waitress (WSGI).
template.yml contains the deployment specific configurations.
requirements.txt inside of the function directory is for the function specific dependencies.
Dockerfile contains the instructions to build image for our function.

Building our function 🔨

Now it’s time to create our Serverless function.

We’ll keep the scope really small and build a simple dictionary function that returns the meaning of words you query. For this we will be using the PyDictionary module.

Since we have our template in place, we can utilize it to scaffold our function. You may have noticed the function directory and template.yml file inside our python3-flask-debian template folder earlier. We can leverage that to create our new function using the faas-cli.

$ faas-cli new <function-name> --lang <template-name>

$ faas-cli new pydict --lang python3-flask-debian

This will create a folder with the function name(pydict) along with the contents we saw earlier inside the templates function folder, plus a yaml file with the function name(pydict).

Let’s add our dependency to the requirements.txt file of function folder.

PyDictionary==2.0.1

Update the handler.py with the following code.

from PyDictionary import PyDictionary

dictionary = PyDictionary()

def handle(word):     
    return dictionary.meaning(word)

Our function is complete. ✔

Openfaas YAML configuration 📄

A Kubernetes guide won’t be complete without talking about YAML files. They are necessary to describe the state of our deployments. faas-cli has already created a yaml file for our function, namely, pydict.yml

This file will tell the faas-cli about the functions and images we want to deploy, language templates we want to use, scaling factor, and other configurations.

contents of pydict.yml

YAML file breakdown

gateway: determines the IP address to connect to the OpenFaas service. Since we are testing it locally the gateway provided by default is fine.
Note: if we were deploying on managed cloud: we will take the IP address of gateway-external
Under functions: we can register different Serverless functions we want to deploy along with their configuration.
lang: name of the language template the function uses.
handler: location of our Serverless function.
image: fully qualified docker registry URL along with image name and tag.

Since we are using Local registry we have to change the image tag to incorporate it. Change value of image to the following:

Change image tag to use local container registry

Port-forward to Localhost ⏩

Now, we need to port-forward the OpenFaas gateway service to our localhost port. Remember the gateway on our yaml file?

To check the gateway of OpenFaas service; run:

$ kubectl get service -n openfaas

$ kubectl port-forward -n openfaas svc/gateway 8080:8080

Although the the cluster is deployed locally, internally Kubernetes manages it’s own IP addresses. We are port forwarding to access the service inside the Kubernetes cluster from our local machine. Once that is done, open a new terminal window, and prepare for take off!

Build ➡Push ➡Deploy 🚀

We have most of the setup ready; now we can build our image, push it to the registry, and deploy it to Kubernetes.

Build our image

$ faas-cli build -f pydict.yml

Push our image to the local registry

$ faas-cli push -f pydict.yml

Now to deploy the function to the cluster; we need to login to faas-cli Let’s generate credentials for authentication.

Generate password:

$ PASSWORD=$(kubectl get secret -n openfaas basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode; echo)

Check if password exists:

$ env | grep PASSWORD

Now, login to faas-cli :

$ echo -n $PASSWORD | faas-cli login --username admin --password-stdin

Time for deployment:

$ faas-cli deploy -f pydict.yml -g http://127.0.0.1:8080

Check if our deployment has been successful.

$ kubectl get pods -n openfaas-fn

Testing our function 👨‍🔬

Test using CLI

We can invoke our function from CL I to get the result.

$ echo "brevity" | faas-cli invoke pydict

Test using Openfaas Portal

OpenFaas also comes with a neat UI portal to invoke our function.

Navigate to localhost:8080 in your browser. A prompt will ask you for username and password. Use admin for username and for password check the password we generated earlier. In case you forgot, you can always repeat the earlier command, i.e.

PASSWORD=$(kubectl get secret -n openfaas basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode; echo) && echo $PASSWORD

OpenFaas UI

Build more functions 🔨

Now that you have your cluster deployed with OpenFaas, you can create function with ease and deploy them. If you want to create a new function, all you have to do is:

scaffold a new function using faas-cli new
write your function
faas-cli up -f <config-yaml-file>

faas-cli up is a shortcut for build, push and deploy command.

Conclusion

Hope this has been a helpful guide in learning how to use OpenFaas for Serverless deployment. If you have any queries or suggestions, let’s talk in the comment section or you can DM me on Twitter.

Check out example functions on GitHub

Understanding Callable in Python

Yankee Maharjan — Tue, 11 Aug 2020 18:35:38 +0000

Functions and classes are the most common things we use in our daily development. We invoke them, pass them around and yet never wonder what makes them so amazing. Well the short answer is callable protocol; for the long answer keep reading!

Definition 📢

Objects defining the __call__ method is known as callable. Or basically callable is anything you can call using the parenthesis () and pass arguments to it. Yes I am basically talking about a function.

`call` method 🤙

__call__ is one of the most interesting dunder method in Python. It is what most of the built-in functions make use of. If we peek into the type of some of these built-in functions then we often see the result as a class.

>>> range
<class 'range'>

>>> zip 
<class 'zip'>

>>> int
<class 'int'>

You get the gist. But how is a class acting as a function? I mean you can just zip(iterable1, iterable2) and you get your result without further invoking any other methods of the class.

Just imagine using zip like if it were a normal class.

processor = ['Intel', 'Ryzen', 'Apple Silicon']
year = [2018, 2019, 2020]

zipped = zip(processor, year)
zipped.start()

This definitely isn’t intuitive to work with. So how does it work?

Using `call` method 🔨

Well behind the scenes it’s because of the __call__ method. This method is used in classes if we want the instance of the class to be callable.

What do I mean by that? Let’s take an example of a class that prints the square of a number.

class Square:

    def __call__(self, num):
            print(num * num)

>>> sq = Square() # create an instance 
>>> sq(5) # invoke
25

This is similar to executing:

>>> sq = Square()
>>> sq.__call__(5)
25

But with the __call__ dunder method we don’t have to do that. We can directly invoke the instance like a normal function.

You may have noticed, we don’t need to use the __init__ constructor to pass the value. Since the class instance acts like a function, then it can take arguments like a function without a need of a constructor.

Testing callable 👨‍🔬

A class or a function is callable by default; but the instance is callable with __call__ dunder method only. We can check this by using the callable() function.

Take for example a class without a __call__ method.

    class Random:
        pass

    >>> callable(Random)
    True

    >>> r = Random()
    >>> callable(r)
    False

On the contrary, let’s take our Square class from earlier example.

    >>> callable(Square)
    True

    >>> callable(sq)
    True

Conclusion 🚀

Next time when you’re creating a class for something and it needs to return a value in an instant; you can make use of the __call__ method.

I hope this blog has been a help to understand an underlying concept involved in functions and classes. If you have any queries or suggestions, let’s discuss them in the comment section.

Faster Git workflow with Git Aliases

Yankee Maharjan — Sat, 25 Jul 2020 15:39:06 +0000

Git is an intricate part of our development workflow. There are handful of commands that you keep repeating day in and day out. I always relied on the command suggestions, or packages on top of my shell that gave access to handy git aliases.

But usually you had to stick with the aliases decided by the package creators . Although most of the aliases included are unofficially globally accepted like ga for git add and so on.

But guess what? You don’t have to rely on any third party packages ; you can create your own with the aliases you prefer!

There are two ways of creating Git aliases: ✌

with the git config utility 💻

This is the preferred way of adding aliases as git gives you an option to do so.

Suppose you feel tired of repeating the commit command every now and then. It would be nice if we could create an alias to write our commits faster.

Say no more! 🎉

git config --global alias.c "commit -m"

The above command follows the following syntax:

git config --global alias.<alias> "<command>"

Now we can use the c alias to represent commit -m .

git c "Update readme with social links"

editing the .gitconfig file 📝

It would be tedious to add multiple aliases with the git config command, so there’s an easier alternate approach.

All the alias you create is saved to the .gitconfig file sitting in your home directory. We can open the file and add our aliases following the TOML formatting. Make sure you do not modify anything in the file apart from adding the [alias] table and its contents.

Open the file using your favorite editor.

vim ~/.gitconfig 
# or
code ~/.gitconfig

Start adding your alias ✍️

...
[alias]
        st = status
        c = commit -m
        a = add
        cb = checkout -b

You can use the above aliases as follows:

git st # git status
git c "hello world" # git commit -m "hello world"
git a hallucination.py # git add hallucination.py 
git cb multi-stage-build # git checkout -b multi-stage-build

There’s also a handy command: git config --list to view the contents of the file including other git configurations.

Bonus 🛸

Git alias with parameters!

We can take our alias a step further by using shell script to add parameters.

Where do we benefit from the parameters?
Let’s take an example of adding a git remote to our local repository. There are basically two variables in the command that I can pass as a parameter, namely, the remote name and project name.

git remote add <remote-name> git@github.com:yankeexe/<project-name>.git

We can represent the above abstraction using an anonymous bash function f()as follows:

[alias]
        ra = "!f() { git remote add $1 git@github.com:yankeexe/$2.git; };f"

The $1 and $2 is the order in which the parameter will be used by the function. ! represents a shell script; our anonymous function is f() which we have invoked immediately at the end.
Let's use our alias:

git ra origin demo-project

Here origin will be used in $1 since it’s the first parameter we are passing and $2 will be demo-project.
The above command will translate to:

git remote add origin git@github.com:yankeexe/demo-project.git

Conclusion 🚀

I hope this article has been of help in improving your Git workflow. If you have any queries or suggestions, let’s discuss in the comment section.

Understanding Iterators and Iterables in Python

Yankee Maharjan — Fri, 17 Jul 2020 03:06:57 +0000

Iterator and Iterable are general terms that gets thrown around a lot in Python. But what do they mean? Are they the same? We will try to understand what these objects in Python are and debunk some misunderstandings.

Along the line we will also see a high-level overview of how for loop and range function might be working under the hood. FYI these concepts as a whole is also known as the Iterator Protocol.

So let’s get started 🔨

Iterable ➰

In general terms, anything that we can loop over is an iterable. I think the name itself gives it away. Most of the data structures in Python is an iterable like List, Tuple, Dict and so on.

You don’t have to believe me if I say a List is an iterable, you can see for yourself. We will be using the built-in dir() function to analyze the properties and methods that a list object consists.

fruits = ['apple', 'banana', 'mango'] # Create a list. 
print(type(fruits)) # make sure it is a List object. :D
print(dir((fruits))

Checking for iter method in an object. 🔎

You can find the __iter__ method in the list, which means it is an iterable. All the objects that are iterable has the method __iter__. If you ever get confused, this handy knowledge might help you identify an iterable.

Iterators ⏭

Now that we know what iterables are and how to identify them, let’s get our feet wet with iterators.

In general, iterator is an object that makes iteration possible. Sounds vague but let me explain. While iterables are the object that we can iterate over; iterator is the implementation that keeps track of the state of the iterable like what value has been returned, what value to return next and also returns the value accordingly.

So how do we use it ?
Remember the __iter__ method we analyzed in our iterable example? Well, if we invoke the __iter__ method of our iterable, it returns an iterator object.

Identifying iterator type and its methods. 🔎

fruits = ['apple', 'banana', 'mango']
fruity = fruits.__iter__()
print(type(fruity))

Again, you don’t have to believe what I say, you can check for yourself if it is an iterator using the dir() method. Like iterables, iterators can be identified using the __next__ method.

print(dir(fruity))

So what does next_ do?

Next returns the item in a list, one at a time.

Differences?

You might have noticed the iterator also consists of the __iter__ method, so we can say that:

Every iterator is also an iterable but not every iterable is an iterator.

Hands on 🙌

Implementing for loop manually

For loop makes use of the concepts of iterator and iterables to return value. Let’s see how we can implement the for loop ourselves.

fruits = ['apple', 'banana', 'mango']
fruity = fruits.__iter__() # alternative:  iter(fruits)
fruity.__next__() # alternative: next(fruity)

You may notice that our next method only returns a single value. What if we want the next item in the list? Well, you call the same method again.

But did you notice, when calling the method again, it doesn’t return value from the start of the list, but carries on to return the next item? This is because iterator is maintaining the state of the items and knows exactly which item to return next.

Manual implementation of for loop ➿

So in our example, we have to call the next function on our iterator object three times to return all the values in the list. But what if we run it four times? Well there is nothing to return as we have already exhausted our list by returning everything. So the iterator throws an StopIteration error.

Well this was a bit redundant isn’t it? What if our list had 1000s of items? We already know the internals so let’s move ahead and automate the process. We are not going to invoke next() for each item 🤷‍♂️.

For that we can make use of class to implement them automatically. For this you need some knowledge of OOP, but I will try my best to explain it using code comments.

Abstracting for loop implementation using a class.📦

We will create a class that will somewhat act like a for loop and return all the elements of an iterable.

class Loop:
    def __init__(self, some_iterable):  # constructor
        self.some_iterable = some_iterable
        self.index = 0

    def __iter__(self):
        return self  # next method is in the same class so return iterator here.

    def __next__(self):
        while self.index < len(self.some_iterable):
            try:
                index = self.index
                self.index += 1
                print(self.some_iterable[index])
            except StopIteration:
                break

Custom loop in action ⚡

Since we are already peeking inside how loop works, lets see how we can implement the range() function using these concepts.

Abstracting range() function implementation 🏹

The concept is pretty similar, we need to print out a series of number one after another taking into consideration the steps to increase.

class Range:
    def __init__(self, start, end, step=1):
        self.start = start
        self.end = end
        self.step = step

    def __iter__(self):
        return self

    def __next__(self):
        while self.start <= self.end:
            try:
                current = self.start
                self.start += self.step

                print(current)
            except StopIteration:
                break

Custom range in action ⚡

Conclusion 🙏

I hope this article has been of help to make things clear about Iterable and Iterator. If you have any suggestions or questions let’s talk in the comment section.

Another implementation of Iterator protocol is the generator functions which we will cover in the future blog.