DEV Community: Dunith Danushka

How To Structure a Perfect Technical Tutorial?

Dunith Danushka — Thu, 04 Jan 2024 13:25:13 +0000

A technical tutorial is a detailed instructional guide that teaches individuals how to perform a specific task or learn a particular skill in a specialized field. These tutorials provide step-by-step instructions, explanations, and examples to help users understand and implement the concepts being taught.

Creating technical tutorials is a regular part of your job if you are an educator, content creator, subject matter expert, or a DevRel person like me. Maintaining a consistent structure across tutorials offers several benefits to creators as well as readers.

In this post, I will dissect the structure of a typical tutorial that society would potentially reward as "good." I will lay out the key sections of a tutorial, their goals, and what information should go into them. Whether you are a beginner or an experienced creator, use this post as a guiding light to structure your next tutorial.

What makes a good tutorial?

A good tutorial often goes beyond a typical instruction list and touches upon several factors.

In my opinion, a tutorial qualifies as "good" if it meets the following criteria.

Clarity of Explanation: Clear explanations are crucial for user comprehension. A good tutorial should present concepts in a straightforward and understandable manner, avoiding unnecessary complexity. Users should be able to follow the content without confusion.
Structural Organization: A logical and well-organized structure is vital for user navigation and comprehension. Headings, subheadings, and a clear flow of information contribute to a seamless learning experience. Users should be able to follow the progression of the tutorial easily.
Completeness: The tutorial should cover all necessary aspects of the topic without leaving critical information gaps. It should strike a balance between providing enough depth to facilitate understanding and avoiding unnecessary details that might overwhelm users.
Engagement: Engaging content keeps users interested and motivated. A good tutorial employs a variety of content types, such as text, images, videos, and interactive elements. The writing style should be engaging, making the learning experience enjoyable.
Usability: Usability refers to how easy it is for users to navigate and follow the tutorial. Clear instructions, a user-friendly layout, and well-defined steps contribute to usability. Users should be able to access and implement the information without encountering unnecessary obstacles.

Beginner creators might take a while to master all these criteria at once. But they can certainly start at some point and work iteratively to make tutorials better over time.

I can help you by providing a starting point - a template that helps you structure your tutorials better. This template divides a tutorial into several sections and guides you to put the right information in the right places.

The key sections of a tutorial include:

Introduction - Essential summary of the tutorial
Overview - The problem you are trying to solve with the tutorial
Prerequisites - What the reader must have before getting started.
Execution - Step-by-step instructions
Validation - How should the reader verify the tutorial outcome?
Tearing down - Cleaning up resources
Conclusion - Takeaways

Let me dive deep into each section next.

The seven sections of a good tutorial

Introduction

This section introduces the tutorial and briefly explains what to expect in the content. After reading the introduction, the reader can decide whether to continue or quit reading, saving him a few minutes in his life.

The hook and the summary are two elements you can include in the introduction to persuade the reader to continue reading the rest of the content.

The hook: A few powerful sentences explaining why the reader should read the tutorial. In most cases, this can be the problem you are trying to solve, summarized in 2–3 sentences. The closer the hook gets to the reader's problem, the more readers continue reading.

Summary of the article (TLDR): What can the reader expect in the tutorial? What's the outcome, and what are the takeaways?

Make sure to keep the introduction short and sweet to lure in more readers to the content. Ideally, you should limit it to 3–4 short paragraphs.

The Overview

This is where you should describe two things in particular.

Problem statement: What problem are you trying to solve with the tutorial? What benefits does it provide? Spend a paragraph or two mentioning these points.

Solution architecture: How are you trying to solve the problems? What's the high-level solution architecture would look like? Mention the technology components you will use in the tutorial and highlight the data flow, connectivity, and user interactions among them.

I often find it easy to pick a fictitious use case and build your narrative around that. For example, to build a data pipeline from a database to a data lake, you can think of an online store use case.

Include visuals as much as possible in this section to provide more clarity to the reader. I highly recommend including a high-level solutions architecture diagram capturing the tutorial scenario. That way, the reader can quickly capture what you are trying to do, even if you miss some instructions.

Prerequisites

You should mention what tools, technologies, and components the reader must have beforehand to complete the tutorial.

For example, the reader must have installed something like a Git CLI, a database, or a build tool in the local workstation or should create a SaaS account before proceeding with the tutorial. Use a bulleted list to mention them.

Furthermore, if certain tools are specific to a platform, highlight them as well. For example, some Linux build tools don't work in Windows, causing the reader to install an alternative, like WSL, before starting the tutorial.

Finished code repo: Not all readers are made equal. Some readers are impatient; they want to see how the completed solution would look like, so they copy code from it :) You can help these readers by pushing the completed solution code into a public Git repo and sharing its link here.

Execution

Now that we have set the stage so far, we've come to the meat of the tutorial. This is where you provide step-by-step, Detailed guidance on each stage of the process or task.

Be sure to provide clear instructions in the order of their dependency. Add screenshots and diagrams as needed - the more, the merrier.

After each instruction, provide the reader the means to verify whether they have executed that properly. For example, if the instruction is about creating a table, the reader can verify the operation by running an SQL SELECT statement against the table.

Validation

Once the reader finishes following instructions, you should provide a way of verifying the end-to-end solution.

Method of integration testing: Provide a sample data set to verify the connectivity, data flow, and expected output of the solution the reader has built. You can include a screenshot or a visual portraying the final solution if the instructions were followed correctly.

Known issues and challenges: Sometimes, certain steps can't be completed due to technical challenges or limitations. Therefore, state any known issues and inform the reader about possible workarounds.

Tearing down

This is an optional step where you instruct the reader to destroy the environment and clean things up to save costs. For example, you can ask the user to turn off an EC2 instance after following the tutorial to reduce the cloud bill.

Conclusion

At last, we come to the end of the tutorial. This is where we wrap things up.

Collecting the thoughts: Reiterate what you have accomplished in the tutorial. Spend the first paragraph of the conclusion wrapping things up.

Taking it to the next level: The tutorial might have shown a thin slice of a solution that addresses a simple problem. Suggest to the reader how to extend the solution to implement a real-world solution and take it to production. You don't have to be detailed here, but highlight a few potential steps necessary as a guideline.

Call to action: Dedicate the last paragraph to insert your CTA, driving satisfied readers to your marketing funnel. After putting so much time and effort into creating the tutorial, you may lure the reader into the next step of your lead funnel ;)

Summary

Technical tutorials can be packaged and presented in various formats, including written articles, video tutorials, interactive online courses, or a combination of these mediums. You can use this post as a template to structure your tutorials, regardless of the format they are presented.

As a rule of thumb, try to cut your content at the 2000 word mark because most readers, including me, have a short attention span, and it isn't easy to navigate and follow a long document. Use a tutorial series if the content goes beyond 2000 words.

I hope you learned something new today. If you are interested in reading posts about DevRel, writing, and productivity, subscribe to my newsletter The Tributary

How To Set a Static IP Address in macOS Using Command Line

Dunith Danushka — Sat, 25 Sep 2021 07:02:05 +0000

Imagine you have an Apple device like a Macmini, MacBook, or an iMac in a remote location. You only have access to it via SSH and no Desktop GUI interaction is allowed.

What if you wanted to change the IP address of the device? For example, how do you assign a static IP address instead of DHCP?

Command line is your option in that case.

Let's see how to get it done...

Make sure you are in the sudoers list

To make changes like network interface configurations, you need to be a user with super user privileges.

Follow the below commands

First, get a list of all your network services.

networksetup -listallnetworkservices

That command will result in an output like below.

>networksetup -listallnetworkservices
An asterisk (*) denotes that a network service is disabled.
Apple USB Ethernet Adapter
USB 10/100/1000 LAN
Wi-Fi
Bluetooth PAN
Thunderbolt Bridge

Then use the desired service name in the following command:

networksetup -setmanual SERVICE IP SUBNET ROUTER

For example, to change the IP address of my Wi-Fi interface, I would use a command like this:

networksetup -setmanual Wi-Fi 192.168.1.2 255.255.255.0 192.168.1.1

In the above example, 192.168.1.2 is my new IP address. 192.168.1.1 is the address of the default gateway, which is the router most of the time.

Setting it back to DHCP

To set it back to DHCP, use the following command:

networksetup -setdhcp SERVICE

Keep DHCP with a manual IP

To keep DHCP services but only use a manually designated IP address, then use the following command:

networksetup -setmanualwithdhcprouter SERVICE IP

Confirm settings are permanent

Restart the device sudo shutdown -r now and see if the new address persists.

If everything goes well, it should persist.

Redis Sorted Sets Explained

Dunith Danushka — Tue, 16 Mar 2021 17:14:09 +0000

Before diving in, let's try to understand Sorted Sets by deconstructing the name.

As per the above, Sorted Set is a Set data structure that doesn't allow duplicate members. At the same time, its members are ordered in ascending order. By combining both, we can define a Sorted Set as an ordered collection of unique members.

Sorted Set is similar to the Set data structure in Redis. Members can be a list of non-repeating strings. The only difference is that each member is associated with a score, a floating-point number that provides a sorting order for the Sorted Set. Members are always sorted from the smallest to the greatest score.

The fitness leaderboard

The best way to understand a Sorted Set is to picture it as a table with score and member columns. The following is an imaginary fitness leaderboard where the member column represents the players while the score represents the number of steps they had walked so far.

The leaderboard has been sorted in ascending order, based on the scores. Scores can be duplicated, but the members can not.

If two members have the same score, Redis sorts them based on members' lexicographical order. For example, Jennifer and Peter have the same score of 690. But Redis makes sure to put Jennifer before Peter because J comes before P.

You can also notice the Rank on the right side. Rank is an implicit attribute assigned to each member, based on the magnitude of its score. It's a zero-based index. The member with the smallest score will be assigned the rank of 0.

Rank is useful when you are retrieving members from the sorted set.

Benefits and use cases

Redis guarantees that a Sorted Set is always sorted. There is no additional sorting required for the client who retrieves the stored members back. That saves CPU cycles in the client and also reduces the code complexity.

Sorted Sets allow you to add, remove, or update elements in a speedy way (in a time proportional to the logarithm of the number of elements). Since elements are taken in order and not ordered afterward, you can also get ranges by score or rank (position) quickly. Accessing the middle of a sorted set is also very fast, so you can use Sorted Sets as a smart list of non-repeating elements where you can quickly access everything you need.

That makes Sorted Sets an excellent choice of implementing real-time, low-latency leaderboards, priority queues, and secondary indexes.

So far, we have covered the basics of Sorted Sets. Let's look at some interesting operations we can perform on them.

Operations

We can perform operations on Sorted Sets to add, remove, increment, and retrieve members. Usually, these operations start with the letter 'Z.'

Let's use the fitness leaderboard example we discussed above to demonstrate these operations. Operations are performed using the Redis CLI. Note that I'm not going to cover all available Sorted Set operations here. I would instead glance through the operations that matter when building a leaderboard-like application. For a full list of operations, you can always refer to Sorted Sets documentation.

Add new players and their steps to the game

When a new player joins, he/she should be added to the leaderboard with the number of steps.
The operation ZADD adds one more element to the Sorted Set. Below is the format of the command.

ZADD <key> <score> <member>

key is the name of the Sorted Set. The following command adds Lester with 790 steps to the players. In return, it gives you the number of elements added to the set, which is 1 in this case.

127.0.0.1:6379> ZADD players 790 Lester
(integer) 1
127.0.0.1:6379>

You can add any number of elements along with their scores as follows. Note that the elements are stored according to their score, not based on the order they've added.

127.0.0.1:6379> ZADD players 980 Mary 850 Alice
(integer) 2
127.0.0.1:6379>

After adding new players, the structure of players would look like this.

Increment/decrement player steps

When players perform their daily walks, their step count needs to be updated on the leaderboard.

You can use the ZINCRBY command to increment the score of a given member by any arbitrary value.

ZINCRBY <key> <increment> <member>

The following command increments Tom's steps by 10.

127.0.0.1:6379> ZINCRBY players 10 Tom
"570"

You can use a negative value to decrement a score. The following command decrements Kendra's steps by 5.

127.0.0.1:6379> ZINCRBY players -5 Kendra
"395"

Retrieve top 10 players by score

Now we need to retrieve only the top 10 players to display them on a leaderboard.

ZREVRANGE command makes that possible by returning only a specified set of members based on their reverse order score.
The command takes the following format.

ZREVRANGE <key> <start> <stop> [WITHSCORES]

ZREVRANGE returns members in reversed-rank order, with scores ordered from high to low. You have to specify starting and ending rank index positions with start and stop parameters.

Let's get the top 10 players by the steps they had walked.

127.0.0.1:6379> ZREVRANGE players 0 9
1) "Mary"
2) "Alice"
3) "Lester"
4) "Frank"
5) "Peter"
6) "Jennifer"
7) "Barbara"
8) "Tom"
9) "Kendra"
10) "Dave"

There is one additional parameter: WITHSCORES, which returns the score for each member.

127.0.0.1:6379> ZREVRANGE players 0 9 WITHSCORES
1) "Mary"
2) "980"
3) "Alice"
4) "850"
5) "Lester"
6) "790"
7) "Frank"
8) "740"
9) "Peter"
10) "690"
11) "Jennifer"
12) "690"
13) "Barbara"
14) "650"
15) "Tom"
16) "570"
17) "Kendra"
18) "395"
19) "Dave"
20) "340"

The ZRANGE command does the opposite to this. It returns members in rank order, with scores ordered from low to high.

The following command returns the first ten players who had walked the least number of steps.

127.0.0.1:6379> ZRANGE players 0 9
1) "Dave"
2) "Kendra"
3) "Tom"
4) "Barbara"
5) "Jennifer"
6) "Peter"
7) "Frank"
8) "Lester"
9) "Alice"
10) "Mary"

Retrieve the rank and the score of an individual player

Now we need to find the score and the rank of a given player. Sometimes, that can be displayed on the player's profile.

ZRANK and ZREVRANK commands return the rank of a member, which is a 0 based index position.

The format of the command is:

[ZRANK|ZREVRANK] <key> <member>

The following command returns 0 for Dave, who's currently at the bottom of the leaderboard.

127.0.0.1:6379> ZRANK players Dave
(integer) 0

Similarly, ZREVRANK for Mary should return 0. Because it returns the rank in the reverse order, with scores sorted from low to high.

127.0.0.1:6379> ZREVRANK players Mary
(integer) 0

ZSCORE provides the current score associated with the member. The format of the command is:

ZSCORE <key> <member>

This returns Peter's score, which is 690.

127.0.0.1:6379> ZSCORE players Peter
"690"

Get the player count based on the score

How can we find the number of players who had walked between a given range of steps? For example, how many players had walked more than 500 but less than 700 steps?

The command ZCOUNT will count the members between the min and the max score. That is inclusive. The format of the command is:

ZCOUNT <key> <min> <max>

The following command returns the number of players who had walked between 500 and 700 steps.

127.0.0.1:6379> ZCOUNT players 500 700
(integer) 4

Remove players from the game

When players leave the game, we need to remove them from the leaderboard.

The ZREM command removes one or more members from a Sorted Set. The command format is:

ZREM <key> <member> [<member> …]

We can remove Barbara from the leaderboard as she decided to leave the game because she was tired.

127.0.0.1:6379> ZREM players Barbara
(integer) 1

Finally, after all these operations, our leaderboard looks like this.

Conclusion

Due to its natural ordering and efficiency in accessing stored elements, Sorted Sets are getting increasingly popular for building applications like real-time leaderboards. They typically store more than a million elements in memory and provide low-latency access to real-time dashboards.

Implementing the same functionality with a traditional database would always be challenging. Even if you manage to do that, feeding that to a dashboard will be a recipe for disaster.

Thus, an in-memory data structure such as Redis shines well here.

References

https://redis.io/topics/data-types

Production-grade Ballerina Microservices on Azure AKS - Part 2: Continuous Integration With Azure DevOps

Dunith Danushka — Mon, 17 Aug 2020 15:05:17 +0000

Introduction

In our previous tutorial, we built a simple Ballerina microservice that returns a JSON array of Todo list items. There, we did everything manually from building the project, generating a Docker image, and pushing it to Azure Container Registry and deploying it into Azure Kubernetes Service.

Although that provided you with a simple starter project, you need a production-grade continuous integration (CI) and continuous deployment (CD) pipelines to make the shipping of developer changes to produce more reliable, repeatable, and frequent.

In the scope of this tutorial, we are going to build a continuous integration pipeline with Azure Pipelines to automate the build and verification process for the TodoService we've implemented in the previous tutorial. During the tutorial, you will:

Create a project and a pipeline in Azure Pipelines.
Connect a GitHub repo to the pipeline
Create a private container registry in Azure Container Registry (ACR) to store the TodoService Docker image
Create an Azure AD service principal to authenticate the pipeline against ACR
Create a Docker registry service connection to connect the pipeline to ACR
Code the pipeline to build the TodoService project, generate a Docker image and push it to ACR

If you have missed the first part of the series, you can find it here.

Prerequisites

A GitHub account, where you can create a repository. If you don't have one, you can create one for free.
An Azure DevOps organization. If you don't have one, you can create one for free.
An Azure account. If you don't have one, you can create one for free.
Installation of Docker and Azure CLI in your local workstation.

Azure Pipelines basics - pipelines, stages, agents, jobs, and tasks

Azure Pipelines is a cloud service that you can use to automatically build and test your code project and make it available to other users. It works with just about any language or project type.

Azure Pipelines combines continuous integration (CI) and continuous delivery (CD) to constantly and consistently test and build your code and ship it to any target.

In order to complete the tutorial, you need a fair knowledge about the key concepts of Azure Pipelines. Especially with build stages, build agents, and various pipeline tasks that'll be used throughout the tutorial.

I suggest you visit the official Azure pipelines documentation that covers the basic concepts before we dive deeper.

1. Get the code

Fork the following repository that contains our initial TodoService implementation.

https://github.com/dunithd/ballerina-on-aks/

2. Sign in to Azure Pipelines and create a project

Sign in to Azure Pipelines. After you sign in, your browser goes to https://dev.azure.com/my-organization-name and displays your Azure DevOps dashboard.

Within your selected organization, create a project. If you don't have any projects in your organization, you see a Create a project to get started screen. Otherwise, select the Create Project button in the upper-right corner of the dashboard.

2.1 Connect and select repository

Navigate to your project.
Go to Pipelines, and then select New Pipeline.
Walkthrough the steps of the wizard by first selecting GitHub as the location of your source code.
You might be redirected to GitHub to sign in. If so, enter your GitHub credentials.
When the list of repositories appears, select the repository you just forked above.
You might be redirected to GitHub to install the Azure Pipelines app. If so, select Approve and install.
Select Starter pipeline. In the Review tab, replace the contents of azure-pipelines.yml with the following snippet -

trigger:
- master

resources:
- repo: self

variables:
  vmImageName: 'ubuntu-latest'

stages:
  - stage: Build
    displayName: Build stage
    jobs:
    - job: Build
      displayName: Build
      pool:
        vmImage: $(vmImageName)
      steps:
      - script: |
          wget https://dist.ballerina.io/downloads/1.2.7/ballerina-1.2.7.zip
          unzip -d /opt ballerina-1.2.7.zip
          /opt/ballerina-1.2.7/bin/ballerina build todo_service.bal
        displayName: 'Download and install Ballerina SDK'

Select Save and run, after which you're prompted for a commit message as Azure Pipelines adds the azure-pipelines.yml file to your repository. After editing the message, select Save and run again to see the pipeline in action.

The above pipeline definition starts with a trigge expression that tells the pipeline to run when something has been committed to the master branch of your GitHub repository fork. There's only one stage defined here with one job and one step. The script step executes the given bash commands to fetch and unzip the Ballerina distribution into the build agent which has the latest version of Ubuntu as specified in the variable vmImageName. The last bash command builds the local TodoService project with the ballerina command. That'll produce a Dockerfile and corresponding Kubernetes artifacts.

For the sake of simplicity, I'm directly downloading the Ballerina distribution into the build agent and running the build from there. But Azure pipeline provides an option to execute a job within a container image of your choice, enabling you to use a standard Ballerina Docker image for the build job.

Once the build job completes, you should see an output similar to this.

Now we have a Dockerfile for the project which needs to be built with Docker to produce a container image and subsequently push it to an Azure Container Registry(ACR) instance. Let's do that with the Docker task in a later section.

Before that, we need to configure the Azure environment.

3. Setup the Azure environment

In this section, you'll configure your Azure environment to create a resource group and an ACR instance using the Azure CLI tool. If you have already performed these steps in the Part 1, you can skip this section.

3.1 Install Azure CLI tool

I’ll be executing almost all the commands related to Azure configurations with Azure CLI. If you haven’t installed it already in your local machine, see this guide to get it done. Note that you need to have Azure CLI version 2.0.53 or later to complete this tutorial. You can verify the version by running az — version command.

3.2 Create a resource group

Once installed locally, the Azure CLI tool can be accessed with az command. Type the following command in a terminal to log in with your Azure account credentials.

az login

Once you are successfully logged in to your Azure account, let’s create a resource group.

An Azure resource group is a logical group in which Azure resources are deployed and managed. When you create a resource group, you are asked to specify a location. This location is where resource group metadata is stored.

The following example creates a resource group named myResourceGroup in the eastus location.

az group create --name myResourceGroup --location eastus

You’ll get an output like below when your resource group created successfully.

{
   "id":"/subscriptions/<guid>/resourceGroups/myResourceGroup",
   "location":"eastus",
   "managedBy":null,
   "name":"myResourceGroup",
   "properties":{
      "provisioningState":"Succeeded"
   },
   "tags":null
}

3.3 Create an Azure Container Registry(ACR) instance

Azure Container Registry (ACR) is a private registry for container images. A private container registry lets you securely build and deploy your applications and custom code. Let's create an ACR instance so that once we build the TodoService Docker image, it can be pushed there.

The following command creates a container registry named dunithd and attaches it to the resource group that I have created previously. Note that the name dunithd is just a name given by me. You may choose your own.

az acr create --resource-group myResourceGroup --name dunithd --sku Basic

The registry name must be unique within Azure and contain 5–50 alphanumeric characters. The Basic SKU is a cost-optimized entry point for development purposes that provides a balance of storage and throughput.

Once created, you can refer to your container registry as <your_name>.azurecr.io. In my case, it is dunithd.azurecr.io.

Note: I chose Azure Container Registry (ACR) for this tutorial since it is well integrated with the Azure ecosystem. But Azure pipelines support a wide variety of container registries including Docker Hub, Google Container Registry (GCR), or any Docker-based registry.

4: Create a Service Principal to authenticate with ACR

In the previous tutorial, you manually built the container image and pushed it to ACR with Azure CLI. But when building a fully automated build pipeline, that has to be done unattended. Hence, the pipeline we are building here should be given necessary access permission to log in to ACR and push the built image.

There are several ways to authenticate with an Azure container registry, each of which is applicable to one or more registry usage scenarios. One option is to log in with az acr login where it provides an interactive shell for developers and testers to push and pull images. Also, another recommended option is to use an Azure AD service principal to perform unattended push/pull operations from CI/CD pipelines. For more information on all available options, please refer to this link.

4.1 Why we need a service principal?

Azure AD service principals provide access to Azure resources within your subscription. You can think of a service principal as a user identity for a service, where "service" is any application, service, or platform that needs to access the resources. You can configure a service principal with access rights scoped only to those resources you specify. Then, configure your application or service to use the service principal's credentials to access those resources. A good example would be creating a service principal for continuous integration systems like Azure DevOps or Jenkins to push/pull images from the container registry. That way, the pipelines are granted minimal and necessary privileges, but nothing more.

In the context of this tutorial, you'll use a service principal to provide container image docker push and pull access to your container registry.

4.2 Create a service principal

To create a service principal with access to your container registry, run the following script in the Azure CLI. The script is formatted for the Bash shell.

Before running the script, update the ACR_NAME variable with the name of your container registry. The SERVICE_PRINCIPAL_NAME value must be unique within your Azure Active Directory tenant. If you receive an "'http://acr-service-principal' already exists." error, specify a different name for the service principal.

Notice that I've provided acrpush as the --role as it grants me both push and pull access to the registry. For more information on RBAC, please refer to this.

After you run the script, take note of the service principal's ID and password. These credentials will be used by the Docker service connection in the next section.

#!/bin/bash

# Modify for your environment.
# ACR_NAME: The name of your Azure Container Registry
# SERVICE_PRINCIPAL_NAME: Must be unique within your AD tenant
ACR_NAME=dunithd
SERVICE_PRINCIPAL_NAME=acr-service-principal

# Obtain the full registry ID for subsequent command args
ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv)

# Create the service principal with rights scoped to the registry.
# Default permissions are for docker pull access. Modify the '--role'
# argument value as desired:
# acrpull:     pull only
# acrpush:     push and pull
# owner:       push, pull, and assign roles
SP_PASSWD=$(az ad sp create-for-rbac --name http://$SERVICE_PRINCIPAL_NAME --scopes $ACR_REGISTRY_ID --role acrpush --query password --output tsv)
SP_APP_ID=$(az ad sp show --id http://$SERVICE_PRINCIPAL_NAME --query appId --output tsv)

# Output the service principal's credentials; use these in your services and
# applications to authenticate to the container registry.
echo "Service principal ID: $SP_APP_ID"
echo "Service principal password: $SP_PASSWD"

5: Create a Docker Registry service connection to ACR

In Azure DevOps, a Service Connection defines a connection to an external system. For example, you may need to connect to your Microsoft Azure subscription, to a different build server or file server, to an online continuous integration environment, or to services you install on remote computers.

A service connection eliminates the need for hard coding external system credentials in your pipeline YAML itself. Instead, Azure DevOps allows you to configure connection details in the portal UI and refer to that from the pipeline at a later stage by using its name.

Let's create a service connection to the ACR we created earlier. Then use service principal credentials as the username and password.

In Azure DevOps, open the Service connections page from the project settings page.
Choose + New service connection and select the type Docker Registry.
Fill in the parameters for the service connection. Use the service principal ID ($SP_APP_ID) as the Docker ID and service principal password (SP_PASSWD) as the Docker Password. I gave acr-service-connection as the connection name.
Choose OK to create the connection. Use the following as a reference.

We'll be using this service connection in the next section.

6: Build and push the project with Docker task

This is the final section in the tutorial where we build the checked out TodoService inside the build agent to produce a container image and push it to ACR with the credentials configured in the previous sections. The build stage uses the Docker task to build and push the image to the container registry.

Replace the content of your pipeline YAML with the following.

trigger:
- master

resources:
- repo: self

variables:
  dockerRegistryServiceConnection: 'acr-service-connection'
  imageRepository: 'todo-service'
  dockerfilePath: '**/Dockerfile'
  tag: 'latest'
  vmImageName: 'ubuntu-latest'

stages:
  - stage: Build
    displayName: Build stage
    jobs:
    - job: Build
      displayName: Build
      pool:
        vmImage: $(vmImageName)
      steps:
      - script: wget https://dist.ballerina.io/downloads/1.2.7/ballerina-1.2.7.zip
      - script: unzip -d /opt ballerina-1.2.7.zip
      - script: /opt/ballerina-1.2.7/bin/ballerina build todo_service.bal
      - script: mv docker/Dockerfile .
      - task: Docker@2
        displayName: Build and push the image to Azure container registry
        inputs:
          command: buildAndPush
          repository: $(imageRepository)
          dockerfile: $(dockerfilePath)
          containerRegistry: $(dockerRegistryServiceConnection)
          tags: |
            $(tag)

Select Save and run to commit the new changes into your repository. That will trigger a new build for the pipeline. As your pipeline runs, select the build job to watch your pipeline in action.

If everything goes well, you should see something similar to this in the pipeline build results.

7. Verify the pipeline execution

Congratulations! You've just completed building a build pipeline for the TodoService project. Now let's verify whether it works as we expected.

To verify, clone the forked GitHub repo into your local workstation and make any change to the todo_service.bal file. Then push the changes back to your fork to trigger a build in the Azure pipeline. That build will checkout the latest source code from the master branch of your fork, build it, and push the built container to your ACR instance.

Login to your ACR instance with the following command to see the newly built container. Make sure to replace the name accordingly.

az acr login --name dunithd

To list the images, type.

az acr repository list --name dunithd --output table
------------ 
todo-service

If you got the above result, you are done!

8. Clean up resources

Once you are done with the tutorial, you may want to delete the resources in your Azure account. Use the following command for that.

az group delete --name myResourceGroup --yes --no-wait

What's next?

In this tutorial, we built a continuous integration pipeline with Azure DevOps to automate the build process for a Ballerina microservice.

Surely, this is more than enough to carry out production-grade build tasks. But There can be a couple of tweaks that can be done to the pipeline such as adding tasks for unit testing and vulnerability scanning etc.

We will not stop it from here. There's a Continuous Deployment part that has to be considered as well. That is, the build pipeline should be altered to deploy the container image to an Azure Kubernetes Service (AKS) environment automatically, run tests there, and promote till the production environment.

Let's explore that in the next post...

Production-grade Microservices on Azure AKS with Ballerina — Part 1: The Basics

Dunith Danushka — Sat, 16 May 2020 03:07:01 +0000

In this multipart tutorial series, you will learn how to write a basic Microservice with Ballerina programming language, deploy it to Azure Kubernetes Service (AKS) and make it production-ready with features available in the Azure platform.

The first installment of this series focuses on getting the simplest things done with the bare minimum effort. At the end of this post, you'll learn the following:

Objectives

Write a simple Ballerina microservice that returns a JSON array of Todo list items.
Generate a Docker image with Docker annotations built into Ballerina.
Push the generated image to Azure Container Registry (ACR).
Create a Kubernetes cluster in Azure Kubernetes Service (AKS) and deploy the above container image into that.
Test the microservice with Postman.

As we dive deep into the series, I'll show you how to make this microservice production-ready to have features like authentication, auto-scaling, load balancing, monitoring, and integrating with a CI/CD pipeline.

Before you begin

This tutorial assumes a basic understanding of Ballerina programming language. But we are not going to explore all the bells and whistles that come with Ballerina. I'll limit the scope to the implementation of a basic REST service. For a primer on Ballerina basics, see https://ballerina.io/learn/

To complete this tutorial, you need to install Ballerina on your local machine. Use https://ballerina.io/downloads/ to get it done.

In addition to that, you need a fair amount of knowledge on Microservices, Docker, and Kubernetes along with a local Docker development environment running Linux containers. Docker provides packages that configure Docker on a Mac, Windows, or Linux system.

Finally, you'll need an account on Azure. It gives you $200 credit to explore Azure for 30 days. If you don't have an account already, I suggest you register for a one.

Why Ballerina

Ever since the Microservices concept conceived, Java-based frameworks like Spring Boot had been the developer's first choice for writing Microservices. But here I'm going to take a little detour from that and introduce you to Ballerina, an open-source programming language purpose-built for optimizing microservices development.

The major reason I fell in love with Ballerina is it comes with all batteries included for Microservices development ' you don't need to borrow anything from the outside.

As a developer, my justification for Ballerina is you'll get more things done with less effort.

Part 1: Run Todo Service locally

1.1 Code walkthrough

In this section, let's create a Ballerina service called TodoService and run it locally.

Open up a text editor of your choice (Visual Studio Code in my case) and create a file called todo_service.bal file inside a directory of your choice. Then add the following code to it.

import ballerina/log;
import ballerina/http;

listener http:Listener todosListener = new(9090);

@http:ServiceConfig {
    basePath: "/todos"
}
service TodoService on todosListener {

    @http:ResourceConfig {
        methods: ["GET"],
        path: "/" 
    }
    resource function getAll(http:Caller caller, http:Request request) {
        //return some mock Todo objects as a JSON array
        json todos = [
            {
                "id" : 1,
                "name" : "Create the Todo Service",
                "status" : "OPEN"
            },
            {
                "id" : 2,
                "name" : "Create an AKS Cluster",
                "status" : "OPEN"
            }
        ];
        //respond to the caller
        var responseResult = caller->respond(todos);
        if (responseResult is error) {
            log:printError("Error responding back to client.", responseResult);
        }
    }
}

In the above code, TodoService represents an HTTP service that is bound to /todos context. It listens for incoming HTTP connections with TodosEP endpoint which is bound to port 9090. It has got only one resource method getAll() that returns an in-memory array of Todo objects formatted as a JSON array.

1.2 Run TodoService

Now we have the basic Ballerina Microservice code in place. Let's try to run it locally just to make sure that our code works.

Open up a terminal and navigate to the place where you save the todo_service.bal file. Then execute below command in the terminal.

ballerina run todo_service.bal

If you have installed Ballerina on your local machine successfully, the ballerina command will be added to your *PATH * variable automatically. When you run a Ballerina service, it will spring up a lightweight web server under the covers and bind it to a port specified with the listener configuration. In our case, the port is 9090.

You will see an output similar to below when everything is properly in place.

A GET request to http://localhost:9090/todos will give you the below response in Postman.

Part 2: Set up the Azure environment

In this section, you'll configure your Azure environment using Azure CLI to deploy above the Todo service.

2.1 Install Azure CLI and kubectl tool

I'll be executing almost all the commands related to Azure configurations with Azure CLI. If you haven't installed it already in your local machine, see this guide to get it done. Note that you need to have Azure CLI version 2.0.53 or later to complete this tutorial. You can verify the version by running az --version command.

In order to point to the Kubernetes cluster in AKS and manage it from the local machine, you are going to need the kubectl command-line tool. If you haven't installed it already, see this guide to get it done.

2.2. Create a resource group

Once installed locally, the Azure CLI tool can be accessed with az command. Type the following command in a terminal to log in with your Azure account credentials.

az login

Once you are successfully logged in to your Azure account, let's create a resource group.

The following example creates a resource group named myResourceGroup in the eastus location.

az group create --name myResourceGroup --location eastus

You'll get an output like below when your resource group created successfully.

{
   "id":"/subscriptions/<guid>/resourceGroups/myResourceGroup",
   "location":"eastus",
   "managedBy":null,
   "name":"myResourceGroup",
   "properties":{
      "provisioningState":"Succeeded"
   },
   "tags":null
}

2.3 Create an Azure Container Registry

Azure Container Registry (ACR) is a private registry for container images. A private container registry lets you securely build and deploy your applications and custom code.

Let's create a container registry now so that it'll be useful in the later parts of this tutorial. The following example creates a container registry named dunithd and attaches it to the resource group that I have created previously.

az acr create --resource-group myResourceGroup --name dunithd --sku Basic

The registry name must be unique within Azure and contain 50 alphanumeric characters. The Basic SKU is a cost-optimized entry point for development purposes that provides a balance of storage and throughput.

Once created, you can refer to your container registry as dunithd.azurecr.io

Part 3: Generate Kubernetes artifacts with annotations

Now we have a basic Ballerina service that works and we have configured our Azure environment accordingly. Then let's see how to generate Kubernetes artifacts for the Todo service.

Ballerina offers a convenient way of generating corresponding Kubernetes artifacts automatically with annotations built into the language. The Kubernetes builder extension offers native support for running Ballerina programs on Kubernetes with the use of annotations that you can include as part of your service code. Also, it will take care of the creation of the Docker images, so you don't need to explicitly create Docker images prior to deployment on Kubernetes.

For more information, see How to Deploy Ballerina Programs in the Cloud.

To add annotations, open the todo_service.bal file and overwrite its content with the below code.

import ballerina/log;
import ballerina/http;
import ballerina/kubernetes;

@kubernetes:Service {
    name : "todos",
    serviceType: "LoadBalancer"
}
listener http:Listener todosListener = new(9090);

@kubernetes:Deployment {
    name : "todos",
    image : "dunithd.azurecr.io/todo-service:v1"
}
@http:ServiceConfig {
    basePath: "/todos"
}
service TodoService on todosListener {

    @http:ResourceConfig {
        methods: ["GET"],
        path: "/" 
    }
    resource function getAllTodos(http:Caller caller, http:Request request) {
        //return some mock Todo objects as a JSON array
        json todos = [
            {
                "id" : 1,
                "name" : "Create the Todo Service",
                "status" : "OPEN"
            },
            {
                "id" : 2,
                "name" : "Create an AKS Cluster",
                "status" : "OPEN"
            }
        ];
        //respond to the caller
        var responseResult = caller->respond(todos);
        if (responseResult is error) {
            log:printError("Error responding back to client.", responseResult);
        }
    }
}

Here, @kubernetes:Deployment is used to specify the Docker image name, which will be created as part of building this service. Notice that I have put the private Docker registry name created in **Part 2 **in the image name as below.

image : dunithd.azurecr.io/todo-service:v1

The @kubernetes:Service {} annotation will create a Kubernetes service that will expose the Ballerina service running on a Pod. For simplicity, I have put LoadBalancer as the service type. Then Azure will automatically provision a load balancer resource for me with an external IP address. I will show you how to configure NGINX as an Ingress controller in a later tutorial.

Now let's use the below command to build the Todo_service.bal file. This will also create the corresponding Docker image and the Kubernetes artifacts using the Kubernetes annotations that you have configured above.

You can use the docker images command to verify that the Docker image specified in the @kubernetes:Deployment was created.

The Kubernetes artifacts related to your service will be generated in addition to the .jar file. Notice the docker and kubernetes directories created in your working directory.

Part 4: Push Docker image to Azure container registry

Now we have Todo service as a Docker image in our local repository. This needs to be pushed to the Azure container registry (ACR) so that our Kubernetes cluster can pull it from there.

4.1 Login to container registry

In a previous step, we created a container registry in Azure. In order to use it, you must first login. In the Azure CLI, type the below command with the name you have given as the registry name before (dunithd in my case).

az acr login --name dunithd

The command returns a Login Succeeded message once completed.

4.2 Push the image to ACR

Use the docker push command to push to the local image to your ACR as follows.

docker push dunithd.azurecr.io/todo-service:v1

This command may take a few minutes to complete.

The following command returns a list of images that have been pushed to your ACR instance

    > az acr repository list --name dunithd --output table

    ------------ 
    todo-service

The following command returns the tags for the todo-service image we just pushed.

    > az acr repository show-tags --name dunithd --repository todo-service --output table 

    -------- 
    v1

Part 5: Create Kubernetes cluster and deploy artifacts

So far, we have tested the todo-service locally, generated a Docker image, and Kubernetes artifacts out of it and pushed the local image to the Azure container registry. What's left is to create a Kubernetes cluster in AKS and deploy the generated artifacts into that.

5.1 Create a cluster of two nodes

The following command creates a Kubernetes cluster of two nodes named myAKSCluster in the resource group named myResourceGroup which we have created in a previous step.

Here, the AKS cluster needs to access Azure Container Registry (ACR) instance to pull the todo-service:v1 image you pushed earlier. For that, Azure automatically creates an Azure Active Directory service principal and grants the right to pull images from the ACR instance. If needed, there's an option to use a managed identity instead of a service principal for easier management.

After a few minutes, the deployment completes and returns JSON-formatted information about the AKS deployment.

5.2 Configure kubectl to connect to your cluster

The following command configures the kubectl tool to connect to your AKS cluster. It fetches credentials for the AKS cluster named myAKSCluster in the myResourceGroup and creates an entry in your *~/.kube *directory.

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

Verify the connectivity to your cluster by running kubectl get nodes command as follows.

5.3 Deploy generated Kubernetes artifacts

Now we are at the final moment of our tutorial. The only thing left is to deploy the Kubernetes artifacts generated in a previous step.

From your working directory where todo_service.bal resides, execute the below command.

kubectl apply -f ./kubernetes

If everything goes fine, it should produce an output like below.

Behind the scenes, kubectl has created a deployment and a service for our Todo service. You can verify the service as follows.

Notice the external IP address assigned our todos service. That is because I have set the service type to LoadBalancer when configuring the service annotation.

Now our service is accessible from outside the cluster. Here's the output I got when I called the todo service with the external IP using Postman.

See, how easy it is to spin up a Kubernetes cluster and deploy your Microservice into that!

Cleaning up

As this tutorial is the last part of the series, you may want to delete the AKS cluster. As the Kubernetes nodes run on Azure virtual machines (VMs), they continue to incur charges even if you don't use the cluster. The following command removes the resource group, container service, and all related resources.

az group delete --name myResourceGroup --yes --no-wait

What's next?

Well, I guess the tutorial was a bit long. But it covers almost every step that is essential to write a Microservice with Ballerina and getting that deployed into AKS. For an absolute beginner on Ballerina and AKS, this tutorial would be an ideal place to start.

We merely scratched the surface here. When I said ÔÇ£production-gradeÔÇØ in the title, there's a lot more to come in this series.

Hence, expect the following posts as follow up items to this article in the future.

Using Azure DevOps to build a CI/CD process for our Todo service and perform rolling updates
Configure NGINX as an ingress controller and use TLS with Let's Encrypt
Secure Todo service with Azure AD and JWT authentication
Access databases from Todo service
Configure Service-to-Service communication with Istio (Todo service will be calling another service via Istio)
Configure Azure Monitoring and logging

If there's any additional topic you want me to discuss in this series, put them as a comment or tweet me.

DEV Community: Dunith Danushka

How To Structure a Perfect Technical Tutorial?

What makes a good tutorial?

The seven sections of a good tutorial

Introduction

The Overview

Prerequisites

Execution

Validation

Tearing down

Conclusion

Summary

How To Set a Static IP Address in macOS Using Command Line

Make sure you are in the sudoers list

Follow the below commands

Setting it back to DHCP

Keep DHCP with a manual IP

Confirm settings are permanent

Redis Sorted Sets Explained

The fitness leaderboard

Benefits and use cases

Operations

Add new players and their steps to the game

Increment/decrement player steps

Retrieve top 10 players by score

Retrieve the rank and the score of an individual player

Get the player count based on the score

Remove players from the game

Conclusion

References

Production-grade Ballerina Microservices on Azure AKS - Part 2: Continuous Integration With Azure DevOps

Introduction

Prerequisites

Azure Pipelines basics - pipelines, stages, agents, jobs, and tasks

1. Get the code

2. Sign in to Azure Pipelines and create a project

2.1 Connect and select repository

3. Setup the Azure environment

3.1 Install Azure CLI tool

3.2 Create a resource group

3.3 Create an Azure Container Registry(ACR) instance

4: Create a Service Principal to authenticate with ACR

4.1 Why we need a service principal?

4.2 Create a service principal

5: Create a Docker Registry service connection to ACR

6: Build and push the project with Docker task

7. Verify the pipeline execution

8. Clean up resources

What's next?

Production-grade Microservices on Azure AKS with Ballerina — Part 1: The Basics

Objectives

Before you begin

Why Ballerina

Part 1: Run Todo Service locally

1.1 Code walkthrough

1.2 Run TodoService

Part 2: Set up the Azure environment

2.1 Install Azure CLI and kubectl tool

2.2. Create a resource group

2.3 Create an Azure Container Registry

Part 3: Generate Kubernetes artifacts with annotations

Part 4: Push Docker image to Azure container registry

4.1 Login to container registry

4.2 Push the image to ACR

Part 5: Create Kubernetes cluster and deploy artifacts

5.1 Create a cluster of two nodes

5.2 Configure kubectl to connect to your cluster

5.3 Deploy generated Kubernetes artifacts

Cleaning up

What's next?

Resources