DEV Community: Boris Zaikin

Internal Developer Platform In Plain English

Boris Zaikin — Wed, 08 Jun 2022 13:59:54 +0000

Pros and Cons of Using Internal Development Platform in the Cloud Computing World

Introduction

Internal Development Platform (IDP) as a brand new word in the DevOps and product development. In this article, I'm explaining:

What exactly IDP is
How to involve it in your organization
An overview of the existing IDP platforms on the market, their pros, and cons
An example of IDP Architecture based on the Azure Cloud

What is an internal developer platform?

As a software architect, I design the DevOps processes and spend much time selecting the DevOps toolset, building pipelines, and communicating with the DevOps team to implement some procedures for the development team. We can solve this with the internal development platform (IDP).

An internal developer platform(IDP) is an additional layer between development and DevOps teams. The main idea behind the IDP is to remove the dependency between DevOps and developers.

Let's look at what functions and components you should include in the IDP. The IDP platform should provide the following:

RBAC management
Infrastructure management
Configuration management
Deployment management

Let's have a look at what is the IDP from insights and how it is connected with DevOps tools and cloud providers.

IDP and DevOps

As seen in the previous example, IDPs are like frameworks built on top of DevOps tools, resources, and cloud providers. Some developer platforms may include an option to work with numerous DevOps tools and cloud providers, like in the diagram above. Other IDPs focused on specific ones.

The diagram below shows the example of the IDP components and how they combine DevOps tools and cloud providers.

Figure 1

Let's see what options of developer platforms are on the market. For example, Gimplet.IO is an open-source IDP that focuses primarily on operating Kubernetes deployments. Gimlet contains CLI to manage charts, deployments, secrets, and GitOps.

The Gimlet doesn't contain application configuration management which makes it partial IDP. I like the idea when IDP focuses on specific platforms. For example, Kubernetes. It makes it more stable than other platforms that try to cover all well-known platforms and cloud providers. Covering too many platforms can be suitable for marketing. However, having the proper support of multiple platforms and cloud providers requires significant effort as they have different APIs, standards, and even naming conventions.

Another good example is an Upbound. Upbound is a powerful IDP that allows you to deploy and build your application in the Kubernetes cluster. It supports GCP, AWS and the Goolge Cloud. I like how an Upbound team supports the documentation and provides architecture examples and architecture references. Below you can see Azure Reference Architecture for Kubernetes.

Figure 2

There are products like Backstage that made good progress in supporting different platforms. Backstage is an open-source platform made by Spotify and supported by Cloud Native Computing Foundation (CNCF). The Backstage is a platform that is intended to build developer portals. It includes the following components:

Backstage Software Catalog for managing microservices, libraries, data pipelines, websites, and ML models.
Backstage Software Templates for quickly creating new projects.
Backstage TechDocs for generating project documentation

The DevOpsBox is an example of a platform that supports numerous DevOps platforms, and Cloud Providers offer application configuration management, infrastructure orchestration, environment management, and adequate RBAC. However, many cloud providers and tools are still in the implementation phase ' for example, Azure and Google Cloud.

And last but not least is a Humanitec. It is a platform that connects multiple GitOps tools and Cloud platforms. I like the concept that Humanitec brings. For example, you can connect one or several Kubernetes clusters over Azure and AWS. Connect Datadog and Grafana. You can even reuse your existing Terraform and Pulumi scripts. You can operate all this infrastructure via CLI, API, and developer-focused UI. Developers can quickly provision Kubernetes clusters, spinup databases, and VM instances without writing pipelines, manifests, and Terraform scripts.

They don't have to wait for the DevOps team when they provide this one. However, DevOps specialists still should spend time on:

Selecting DevOps tools
Migrating existing scripts, pipelines, and manifests
Connect an environment

Figure 3

Humanitec also supports the PlatformCON community around IDP and DevOps topics.

Example of internal development platform

Many enterprise companies like Spotify are building their own IDP. They already have a massive DevOps code base that includes scripts and templates, pipelines for the most available platforms, and cloud providers.

A few years ago, I was working for an enterprise company with 500+ ongoing projects, and they needed to set up a team for the new project quickly. All projects need to provide infrastructure ASAP. The developer had no time to do DevOps stuff. The company asked to build Internal Developer Platform (IDP) to provision and manage infrastructure. The idea is simple. When a new project appears, the algorithm should be the following:

Onboard a development team.
The support engineers provision all required infrastructure.

Figure 4

As you can see, the architecture is primarily based on the Azure stack. However, you can easily extend it to use in AWS, GoogleCloud, or any other cloud provider. The IDP can provide the following resources and applications:

Azure DevTestLabs, VMs, and Extensions
Kubernetes Clusters (AKS)
Databases
App Service

A developer portal is a simple ReactJS application that uses an internal API. The API is based on Azure API management and contains several functions representing a single API call. For example, to trigger deployment or receive deployment status. The API layer is connected to the Azure DevOps API. The Azure DevOps contains YAML pipelines, Terraform, Bash, and PowerShell scripts.

Pros and Cons

An IDP provides you with the following benefits:

Developer's productivity. Developers can deliver the code and focus less on DevOps tasks.
DevOps teams freedom. An IDP reduces the infrastructure provisioning workload and focuses on building baseline configuration and templates.

IDP can bring value to the product. However, there are some drawbacks involving IDP. Below I've listed some of them:

Costs and complexity. Building a developer platform requires companies to provide additional resources to build, support, and keep it updated. We can partially solve this issue by buying the external development platform. I will describe a few existing platforms below.
Culture change. Involving a developer platform requires the development and DevOps team to change their culture and way of working. It may end up with the teams will lose time.

The list can be endless and may require its own article. In the article: Why companies that build internal dev platforms wouldn't do it again you can find a lot more points against building IDP internally.

Conclusion

Here I've tried to clarify what is an internal developer platform and how you and your organization can benefit from involving IDP. Also, I've added points about why using IDP may be not reasonable. Also, I've shared the architecture example that demonstrates how an organization can build its IDP and save time on managing several DevOps teams across the company.

CI/CD for Cloud-Native Applications"

Boris Zaikin — Wed, 20 Apr 2022 14:35:08 +0000

Define continuous integration and continuous delivery, review the steps in a CI/CD pipeline, and explore DevOps and IaC tools used to build a CI/CD process.

Continuous integration (CI) and continuous delivery (CD) are crucial parts of developing and maintaining any cloud-native application. From my experience, proper adoption of tools and processes makes a CI/CD pipeline simple, secure, and extendable. Cloud native (or cloud based) simply means that an application utilizes cloud services. For example, a cloud-native app can be a web application deployed via Docker containers and uses Azure Container Registry deployed to Azure Kubernetes Services or uses Amazon EC2, AWS Lambda, or Amazon S3 services.

In this article, we will:

Define continuous integration and continuous delivery
Review the steps in a CI/CD pipeline
Explore DevOps and IaC tools used to build a CI/CD process

An Overview of CI/CD

The continuous integration process is when software engineers combine all parts of the code to validate before releasing tested applications to dev, test, or production stages. CI includes the following steps:

Source control - Pulls the latest source code of the application from source control.
Build - Compiles, builds, and validates the code or creates bundles and linting in terms of JavaScript/Python code.
Test - Runs unit tests and validates coding styles.

Following CI is the continuous delivery process, which includes the following steps:

Deploy - Places prepared code into the test (or stage) environment.
Testing - Runs integration and/or load tests. This step is optional as an application can be small and not have a huge load.
Release - Deploys an application to the development, test, and production stages.

In my opinion, CI and CD are two parts of one process. However, in the cloud-native world, you can implement CD without CI. You can see the whole CI/CD process in the diagram below:

Figure 1

The Importance of CI/CD in Cloud-Native Application Development

Building a successful cloud-native CI/CD process relies on the Infrastructure-as-Code (IaC) toolset. Many cloud-native applications have integrated CI/CD processes that include steps to build and deploy the app and provision and manage its cloud resources.

How IAC Supports CI/CD

Infrastructure as Code is an approach where you can describe and manage the configuration of your application's infrastructure. Many DevOps platforms support the IaC approach, integrating it directly into the venue for example, Azure DevOps, GitHub, GitLab, and Bitbucket support YAML pipelines. With the YAML pipeline, you can build CI/CD processes for your application with infrastructure deployment. Below, you can see DevOps platforms that can easily be integrated with IaC tools:

Azure Resource Manager, Bicep, and Farmer
Terraform
Tekton
Pulumi
AWS CloudFormation

Building a Successful Cloud-Native CI/CD Process

In many cases, the CI/CD process for cloud-native applications includes documenting and deploying infrastructure using an IaC approach. The IaC approach allows you to:

Prepare infrastructure for your application before it is deployed.
Add new cloud resources and configuration.
Manage existing configurations and solve problems like "environment drift".

Environment drift problems appear when teams have to support multiple environments manually. Drift can lead to an inconsistent environment setting that causes application outages. A successful CI/CD process with IaC relies on what tool and platform you use.

Let's have a look into the combination of the most popular DevOps and IaC tools.

Azure DevOps

Azure DevOps is a widely used tool to organize and build your cloud-native CI/CD process, especially for Azure Cloud. It supports UI and YAML-based approaches to building pipelines. I prefer using this tool when building an automated CI/CD process for Azure Cloud. Let's look at a simple YAML pipeline that creates a virtual machine (VM) in Azure DevTest Labs:

    . . . . . .
    jobs:
    - deployment: deploy
      displayName: Deploy
      pool:
        vmImage: 'ubuntu-latest'
      environment: ${{ parameters.environment }}

      strategy:
        runOnce:
          deploy:
            steps:
            - checkout: self
            - task: AzurePowerShell@4
              displayName: 'Check vm-name variable exists'
              continueOnError: true
              inputs:
                azureSubscription: ${{ parameters.azSubscription }}
                scriptType: "inlineScript"
                azurePowerShellVersion: LatestVersion
                inline: |
                  $vm_name="$(vm-name)"
                  echo $(vm-name) - $vm_name
                  #if ([string]::IsNullOrWhitespace($vm_name))
                  {
                      throw "vm-name is not set"
                  }

            - task: AzureResourceManagerTemplateDeployment@3
              displayName: 'New deploy VM to DevTestLab'
              inputs:
                deploymentScope: 'Resource Group'
                azureResourceManagerConnection: ${{ parameters.azSubscription }}
                subscriptionId: ${{ parameters.idSubscription }}
                deploymentMode: 'Incremental'
                resourceGroupName: ${{ parameters.resourceGroup }}
                location: '$(location)'
                templateLocation: 'Linked artifact'
                csmFile: templates/vm.json
                csmParametersFile: templates/vm.parameters.json
                overrideParameters: '-labName "${{parameters.devTestLabsName}}"
                                     -vmName ${{parameters.vmName}} -password ${{parameters.password}}
                                     -userName ${{parameters.user}}
                                     -storageType
    . . . . . .

To simplify the pipeline listing, I've shortened the example above. As you can see, the pipeline code can also be generic; therefore, you can reuse it in multiple projects. The pipeline's first two steps are in-line PowerShell scripts that validate and print required variables. Then this pipeline can be integrated easily into Azure DevOps, as shown in the image below:

Figure 2

The last step creates the VM in Azure DevTest Labs using Azure Resource Manager (ARM) templates and is represented via the JSON format. This step is quite simple: It sends (overrides) variables into the ARM scripts, which Azure uses to create a VM in the DevTest Labs.

AWS CloudFormation

AWS CloudFormation is an IaC tool from the AWS Cloud stack and is intended to provision resources like EC2, DNS, S3 buckets, and many others. CloudFormation templates are represented in JSON and YAML formats; therefore, they can be an excellent choice to build reliable, cloud-native CI/CD processes. Also, many tools like Azure DevOps, GitHub, Bitbucket, and AWS CodePipelines have integration options for CloudFormation.

Below is an example of what an AWS CloudFormation template may look like, which I've shortened to fit in this article:

    {
      "AWSTemplateFormatVersion" : "2010-09-09",
      "Parameters" : {
         "AccessControl" : {
         "Description" : " The IP address range that can be used to access the CloudFormer tool. NOTE:
    We highly recommend that you specify a customized address range to lock down the tool.",
          "Type": "String",
          "MinLength": "9",
          }
      },
          "Mappings" : {
            "RegionMap" : {
            "us-east-1" : { "AMI" : "ami-21341f48" },
            }
      }
      "Resources" : {
         "CFNRole": {
         "Type": "AWS::IAM::Role",
         "Properties": {
         "AssumeRolePolicyDocument": {
                "Statement": [{
                "Effect": "Allow",
                "Principal": { "Service": [ "ec2.amazonaws.com" ] },
                "Action": [ "sts:AssumeRole" ]
                }]
          },
          "Path": "/"
          }
          },
      }
    }

CloudFormation templates contain sections including:

Parameters - You can specify input parameters to run templates from the CLI or pass data from AWS CodePipeline (or any other CI/CD tool).
Mappings - You can match the key to a specific value (or set of values) based on a region.
Resources - You can declare the resources included, answering the "what will be provisioned" question, and you can adjust the resource according to your requirement using a parameter.

AWS CloudFormation templates look similar to Azure ARM templates as they do the same work but for different cloud providers.

Google Cloud Deployment Manager

Google Cloud (GC) offers Cloud Deployment Manager, the all-in-one tool that includes templates that describe resources for provisioning and templates to build CI/CD pipelines. The templates support:

Let's explore a deployment of the VM using Jinja templates ' it looks common to YAML:

    resources:
    - type: compute.v1.instance
      name: {{env["project"]}}-deployment-vm
      properties:
         zone: {{properties["zone"]}}
         machineType:https://www.googleapis.com/compute/v1/projects/{{env["project"]}}/zones/
    {{properties["zone"]}}/machineTypes/f1-micro
         disks:
         - deviceName: boot
         type: PERSISTENT
         boot: true
         autoDelete: true
         initializeParams:
         sourceImage: https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/family/
    debian-9
         networkInterfaces:
         - network: https://www.googleapis.com/compute/v1/projects/{{env["project"]}}/global/networks/
    default
         accessConfigs:
         - name: External NAT
         type: ONE_TO_ONE_NAT

The example above is similar to ARM and CloudFormation templates as it describes resources to deploy. In my opinion, the YAML/Jinja 2.10.x format works better than the JSON-based structure because:

YAML increases readily and can read much faster than JSON
Teams can find and fix errors in YAML and Jinja faster than in JSON
YAML pipelines (with small adaptations) can be reused on many other platforms

You can find an extended version of this example in the GitHub gist.

Tekton and Kubernetes

Tekton, supported by the CD Foundation (part of the Linux Foundation), is positioned as an open-source CI/CD framework for cloud-native applications based on Kubernetes. The Tekton framework's components include:

Tekton Pipelines - Most essential and are intended to build CI/CD pipelines based on Kubernetes Custom Resources.
Tekton Triggers - Provide logic to run pipelines based on an event-driven approach.
Tekton CLI - Built on top of the Kubernetes CLI and allows you to run pipelines, check statuses, and manage other options.
Tekton Dashboard and Hub Use web-based graphical interfaces to run pipelines and observe pipeline execution logs, resource details, and resources for the entire cluster (see dashboard in Figure 3).

Figure 3

I like the idea behind Tekton Hub that you can share your pipelines and other reusable components. Let's look at a Tekton Pipeline example:

    apiVersion: tekton.dev/v1beta1
    kind: Pipeline
    metadata:
      name: say-things
    spec:
      tasks:
        - name: first-task
          params:
            - name: pause-duration
              value: "2"
            - name: say-what
              value: "Hello, this is the first task"
          taskRef:
            name: say-something
        - name: second-task
          params:
            - name: say-what
              value: "And this is the second task"
          taskRef:
            name: say-something

The pipeline code above is written in the native Kubernetes format/manifests and represents a set of tasks. Therefore, you can build native cross-cloud CI/CD processes. In the tasks, you can add steps to operate with Kubernetes resources, build images, print information, and many other actions. You can find the complete tutorial for Tekton Pipelines here.

Terraform, Azure Bicep, and Farmer

Terraform is a leading platform for building reliable CI/CD processes based on the IaC approach. I will not go in depth on Terraform as it requires a separate article (or even book). Terraform uses a specific language that simplifies building CI/CD templates. Also, it allows you to reuse code parts, adding dynamic flavor to the CI/CD process. Therefore, you can build templates that are much better than ARM/JSON templates. Let's see a basic example of Terraform template code:

    terraform {
      required_providers {
      }
      backend "remote" {
        organization = "YOUR_ORGANIZATION_NAME"
        workspaces {
          name = "YOUR_WORKSPACE_NAME"
        }
      }
    }

The basic template above contains the resources, providers, and workspace definition. The same approach follows the Azure Bicep and Farmer tools. These Terraform analogs can drastically improve and shorten your code. Let's look at the Bicep example below:

    param location string = resourceGroup().location
    param storageAccountName string = 'toylaunch${uniqueString(resourceGroup().id)}'

    resource storageAccount 'Microsoft.Storage/storageAccounts@2021-06-01' = {
      name: storageAccountName
      location: location
      sku: {
        name: 'Standard_LRS'
      }
      kind: 'StorageV2'
      properties: {
        accessTier: 'Hot'
      }
    }

The Bicep code above deploys storage accounts in the US region. You can see that the JSON ARM example acts the same. In my opinion, Terraform and Azure Bicep are shorter and much easier to understand than ARM templates. The Farmer tool can also show impressive results in readability and type-safe code:

    // Create a storage account with a container
    let myStorageAccount = storageAccount {
        name "myTestStorage"
        add_public_container "myContainer"
    }

    // Create a web app with application insights that's connected to the storage account.
    let myWebApp = webApp {
        name "myTestWebApp"
        setting "storageKey" myStorageAccount.Key
    }

    // Create an ARM template
    let deployment = arm {
        location Location.NorthEurope
        add_resources [
            myStorageAccount
            myWebApp
        ]
    }

    // Deploy it to Azure!
    deployment
    |> Writer.quickDeploy "myResourceGroup" Deploy.NoParameters

Above, you can see how to deploy your web application to Azure in 20 lines of easily readable and extensible code.

Pulumi

The Pulumi framework follows a different approach to building and organizing your CI/CD processes: It allows you to deploy your app and infrastructure using your favorite programming language. Pulumi supports Python, C#, TypeScript, Go, and many others. Let's look at the example below:

    public MyStack()
    {
        var app = new WebApplication("hello", new()
        {
            DockerImage = "strm/helloworld-http"
        });

        this.Endpoint = app.Endpoint;
    }

This part of the code deploys the web app to the Azure cloud and uses Docker containers to spin up the web app. You can find examples of how to spin up the web app in an AKS cluster as well as from the Docker container here

Conclusion

Building a cloud-native CI/CD pipeline for your application can be a never-ending story if you don't know the principles, tools, and frameworks best suited for doing so. It is easy to get lost in various tools, providers, and buzzwords, so this article aimed to explain what CI/CD cloud-native applications are and walk you through the widely used tools and principles of building reliable CI/CD pipelines. Having this guide helps you to feel comfortable while designing CI/CD processes and choosing the right tools for your cloud-native application.

This is an article from DZone's 2022 DevOps Trend Report.
For more:
Read the Report

Common Performance Management Mistakes

Boris Zaikin — Tue, 21 Dec 2021 16:20:10 +0000

How to Avoid Pain Points Introduced by Cloud-Based Architectures

Editor's Note: The following is an article written for and published in DZone's 2021 Application Performance Management Trend Report

Performance in any cloud-distributed application is key to successful user experience. Thus, having a deep understanding of how to measure performance and what metric IO pattern to use is quite important. In this article, we will cover the following:

Performance analysis checklist and strategies
Performance issue detection and tools
Performance anti-patterns and how to mitigate issues

Performance Monitoring in the Cloud Checklist

When improving the performance of your application or infrastructure architecture, use the following checklist:

Introduce logging tools in your application and infrastructure components
Find anti-patterns and bottlenecks of your code and infrastructure
Set up monitoring tools to gather CPU, memory, and storage utilization
Adjust monitoring tools to gather info about events between base components
Do not log everything identify important events to log

Most Common Application Performance Anti-Patterns

The most important item from the performance checklist is identifying anti-patterns. This will save valuable time once you already know the weak spots.

Nosy Neighbor

Imagine you have a microservice that is deployed as a Docker container, and it is eating more CPU and memory than other containers. That can lead to outages since other services might not receive enough resources. For example, if you use Kubernetes, it may kill other containers to release some resources. You can easily fix this by setting up CPU and memory limits at the very beginning of the design and implementation phases.

No Caching

Some applications that tend to work under a high load do not contain any caching mechanisms. This may lead to fetching the same data and overusing the main database. You can fix this by introducing a caching layer in your application, and it can be based on a Redis cache or just a memory cache module. Of course, you don't need to use caching everywhere, since that may lead to data inconsistency. Sometimes, you can improve your performance by simply adding output caching to your code. For example:

namespace MvcApplication1.Controllers
{
  [HandleError]
  public class HomeController : Controller
  {

     [OutputCache(Duration=10, VaryByParam="none")]
     public ActionResult Index()
     {
         return View();
     }
   }
}

Above, I've added the output cache attribute to the MVC application. It will cache static content for 60 seconds.

Busy Database

This issue is often found in modern microservices architectures, when all services are in a Kubernetes cluster and deployed via containers but they all use a single database instance. You can fix this problem by identifying the data scope for each microservice and splitting one database into several. You can also use the database pools mechanism. For example, Azure provides the Azure SQL elastic pool service.

Retry Storm

Retrying storms and the issues they cause usually occur in microservice or cloud-distributed applications; when some component or service is offline, other services try to reach it. This often results in a never-ending retry loop. It can be fixed, however, by using the circuit breaker pattern. The idea for circuit breaker comes from radio electronics. It can be implemented as a separate component, like an auto switch. When the circuit runs into an issue (like a short circuit), then the switch turns the circuit off.

Example: Cloud Performance Monitoring Architecture That Results in an Outage

To check the performance of your architecture, you need to run load tests against your application. Below, you can see an architectural example that is based on Azure App Services and Azure Functions. The architecture contains the following components:

Separated into four stages: Dev, Test, Staging, and Production.
Login and monitoring are implemented with Prometheus and Grafana
Loading tests are implemented with Azure DevOps and JMeter

Here, you can find an example of how to set up a JMeter load test in Azure DevOps.

Figure 1

This is an excerpt from DZone's 2021 Application Performance Management Trend Report.
For more: Read the Report

Fixing the Issue

As you can see in the diagram, everything looks good at first glance. Where do you think there could be an issue?

You can see the fixed architecture of the diagram in Figure 2.

Figure 2

First of all, you should never run load testing against the Production stage as it may (and often will) cause downtime when you run an excessive load test. Instead, you should run the test against the Test or Staging environments.

You can also create the replica of your production environment explicitly for load test purposes. In this case, you should not use real production data, since that may result in sending emails to real customers!

Next, let's look at an application that should be architected under a high load.

Example: High-Load Application Architecture on AKS

Imagine that we have a popular e-Commerce application that needs to survive Black Friday.

The application architecture should be based around the following components:

Azure Kubernetes Services (AKS) with Kubernetes Cluster Autoscaler are used as the main distributed environment and mechanism to scale compute power under load.
Istio's service mesh is used to improve cluster observability, traffic management, and load balancing.
Azure Log Analytics and Azure Portal Dashboards are used as a central logging system.

Figure 3

In the figure above, you can see that the AKS cluster contains nodes that are represented as virtual machines under the hood. The Autoscaler is the main component here. It scales up the node count when the cluster is under high load. It also scales the node down to a standard size when the cluster is under normal load.

Istio provides the data plane and control plane, assisting with the following:

Load balancing
TLS termination
Service discovery
Health checks
Identity and access management
Configuration management
Monitoring, logs, and traffic control

Please note that the architecture includes the stages Dev, Test, Staging, and Production, of course. The formula for the highly available Kubernetes cluster is having separate clusters per stage. However, for Dev and Test, you can use a single cluster separated by namespaces to reduce infrastructure costs.

For additional logging reinforcement, we used Azure Log Analytics agents and the Portal to create a dashboard. Istio contains a lot of existing metrics, including those for performance and options to customize them. You can then integrate with a Grafana dashboard.

Lastly, you can also set up a load test using Istio. Here is a good example.

Top Open-Source Application Monitoring Tools

Let's start off by listing some of the most popular open-source application performance tools:

Apache Sky Walking is a powerful, distributed performance and log analysis platform. It can monitor applications written in .NET Core, Java, PHP, Node.js, Golang, LUA, C++, and Python. It supports cloud integration and contains features like performance optimization, slow service and endpoint detection, service topology map analysis, and much more. See the feature map in the image below:

Figure 1

Scouter is a powerful tool that can monitor Redis, Nginx, Kafka, MongoDB, Kubernetes, and other sources. It can monitor CPU, memory network and heap utilization, active users, active services, and more.
GoappMonotr is a tool that provides performance monitoring for Golang applications.
Pinpoint is a performance monitoring tool for Python, Java, and PHP applications. It can monitor CPU, memory, and storage utilization. You can integrate it into your project without changing a single line of code.
Code Speed is a simple APM tool. It can be installed into your Python application to monitor and analyze the performance of your code.

There are tools like Datadog that have community licenses or trials. Also, if you are using Azure, you can enable Azure AppInsights with low cost or no cost at all, depending on bandwidth.

Conclusion

In this article, we've dug into common performance mistakes and anti-patterns when working with distributed cloud-based architectures, introducing a checklist to consider when building applications that will face heavy loads. Also, we explored the open-source tools that can help with performance analysis, especially for projects on limited budgets. And, of course, we've covered a few examples of highly performant applications and an application that may have performance issues so that you, dear reader, can avoid these common performance mistakes in your cloud architectures.

Originally published athttps://dzone.com.

Want to boost your cloud and software architecture skills? Enroll on my hands-on courses:

Building Enterprise applications with multitenancy

Building Event Driven and Microservices Architecture in Azure

2 Widespread Attacks on Your Containerized Environment and 7 Rules to Prevent it.

Boris Zaikin — Fri, 23 Jul 2021 13:39:30 +0000

How to Apply Common Security Principles and What tool to Choose to Prevent Attacks on Your Docker Containers and Kubernetes clusters.

1. Man-in-the-Middle attack

The MITM attack is widespread in the Kubernetes and Docker. This attack includes additional malicious parts between the component that sends data and the component that receives this data. It can be a fake container, service, middleware, or even a human. For example:

CVE-2020–8554 — vulnerability that allows attackers to obtain access to the cluster by creating ClisterIPs service.
Siloscape — malware inside windows containers. The Silocape creates a backdoor to the whole Kubernetes cluster, including sensitive data and CPU, GPU, and resources

2. Cryptojacking attack

This attack allows an attacker to run the malicious code to use the CPU, GPU, and Memory of the PC for mining cryptocurrencies. Example:

CVE-2018–15664 — gives access to the docker system with root permission.

Introduction

The container itself is a small OS that can be susceptible to attack with malicious code. In this article, we talk about

What you need to do to secure your app in the container.
Security tools.
Security rules, policies.
How to apply common security principles to prevent attacks.

I consider using Docker and Kubernetes as two current leaders in container engines and orchestration.

Securing Docker Containers Principles and Rules

1. Use Principle of Least Privileges

This principle means that you should not execute containers using admin users. You should create users that have admin access and can only operate with this particular container. You can also make groups add users there. You can read more about it in “Isolate containers with a user namespace”. Below is an example of how to create the user and group.

First of all, you should use official verified and signed images. To find and check images you can use docker trust inspect.

For example docker trust inspect — pretty google/apigee-mart-server:1.3.6.

Docker Content Trust (DCT) can help with digital signatures.

It allows verification of images and publishers during runtime. This process is based on (Docker Content Trust Keys)[https://docs.docker.com/engine/security/trust/#docker-content-trust-keys], which generates several keys during the first interaction with DCT.

2. Setup Resource Limits

You should set up Memory and CPU limits for your docker container because docker has a container that does not have this option by default. This principle is a way to prevent DoS attacks. For example, you can set up a Memory limit to prevent your container from consuming all memory. The same applies to CPU limits.

There is also an option to set up resource limits on a Kubernetes level. I cover this topic in the Kubernetes Security section.

3. Secure and Understand Docker Networking

This principle is essential to understand Docker’s networking principles. You should understand what Docker Network Drivers are, for example:

bridge
host
overlay

By default, one container network stack does not have access to another container. However, if you configure bridged or host to accept traffic from any other containers or external networks, you can create a potential security backdoor for an attack. You can also disable inter container communication using just set flag — icc=false within docker daemon.

4. Add Container Security Monitoring Architecture

Security monitoring is essential for the detection of malicious code and attacks on your containers.

With a proper monitoring tool, you should be able to detect the issues.

The tool allows you:

Build a real-time dashboard.
Set up alerts to send you messages via Email, SMS, or even your preferable chat platform.

To find the vulnerability in Docker containers, you can use

docker scan command.
CAdvisor, which is also a pretty powerful tool to monitor your container. Moreover, you can also run it in the Kubernetes cluster.

Docker scan or Caadvisor is a simple solution that applies to only one particular container. For more complex scenarios, for example, when you run 50+ containers in the Kubernetes, you need complex monitoring tools such as:

Prometheus and Grafana. Prometheus is a logging component that “scrapes” information from your container and puts it into a data source. The data sources can be SQL, NoSQL data storage. Also, Prometheus has an Alert Manager component. It allows users to create rule-based alerts. Grafana is a framework that helps to build complex UI dashboards. The dashboards can be easily configured to get data from Prometheus.
Datadog, which is a comprehensive, all-in-one monitoring tool that contains logging component subsystems and sidecars. It also includes a complex and interactive UI framework.
Azure Log Analytics. It is a complex tool that monitors your containers, especially in Azure Container Registry, Azure Kubernetes Services. It is a handy option if you have your container solution under Azure Cloud Services as it supports it out of the box.

I have listed some pretty popular solutions. However, there are many others in the market worth mentioning, like Sysdig, Sematext, Dynatrace.

I like to “cook” using the combination of Prometheus + CAdvisor + Grafana.

Prometheus is a powerful and open-source option for monitoring CPU, GPU, Memory, Images, and other metrics.
CAdvisor is quite good at detecting vulnerabilities.
Grafana is good at building, configuring dashboards and alerts, and imports all components together.

Also, I use the security tool Kube-bench that covers vulnerabilities scanning only. The Kubebench brings an additional layer to your cluster security monitoring. There are plenty of security tools available for Kubernetes.

5. Avoid putting Sensitive data in Docker Images.

This principle is quite important to move all sensitive data out of the container. You can use different options to manage your secrets and other sensitive data:

Docker secret allows you to store your secrets outside of the image.
If your run docker containers in Kubernetes, you can use Secrets to store your passwords, certificates, or any other sensitive data
Use cloud-specific storage for sensitive data — for example, Azure Key Vault or AWS Secret Manager.

6. Involve Vulnerability Scanning Tools

Vulnerability scanning tools are the essential part of detecting images that may have security holes. Moreover, you can also integrate properly selected tools into the CI/CD process. Below I have listed some scanning tools:

Dagda uses a static analysis approach to find viruses, malware, and fake sub-images and trojans. It is based on Red Hat Security Advisories (RHSA) libraries of existing vulnerabilities databases.
Trivy can detect complex vulnerabilities with high accuracy for OS like Alpine Linux and RHEL/CentOS , Debian, Ubuntu, and others. It is quite powerful, opensource, and free. You can run Trivi in standalone or client/server modes. Therefore you can add it to your CI/CD process.
Clair is used for static analysis of your images. It supports images that are based on the Open Container Initiative (OCI). You can build your services for scanning images that can be based on Clair API. Clair uses CVE databases to detect vulnerabilities.

You can find more in the Containers Trend Report article.

Use Secured Docker Registries

To protect your images, you can create an additional security layer and use images from protected registries like:

Harbor, which is an open source registry with integrated vulnerability scanning. It is based on security policies that apply on docker artifacts.
Quay, which scans your images for vulnerability. It is an image registry powered by RedHat. Quay also offers a standalone image repository that you can install and use internally in your organization. Below you can see how it scans for vulnerabilities.

But what if you are already using other registries like Azure Container Registry or Docker Hub? You can find how to do it in the Containers Tend Report article.

Securing Kubernetes

Kubernetes security is a big and important topic. The main security rules are:

Networking and Network Policies. You should understand how the Kubernetes networking model works. It will help you set up proper network communication between pods, and pretend creating open ports or direct access to the nodes. The Network Policy can help you to organize this communication.
Secure Ingress and Egress traffic to your pod. Here you can also use Network Policies. You can use strategy to deny all Egress and Ingress traffic and then start opening. You can also use service mesh like Istio. It adds additional service layers, automates traffic and helps with monitoring. However, you should be careful to use the service mesh as it may add additional complexity.
Transport Layer Security. You should enable TLS if it is not opened. TLS should be used for communication between Kubernetes cluster services.
Restrict Access To Kubernetes Dashboard.
Use RBAC and follow the principle of least privilege.
Restrict access to Kubelet. You should enable the authentication and authorization to use this tool. Only admins should have access to Kubelet.

The security principles that are mentioned in the docker container section

Detect Kubernetes Configuration and Security Issues

To find security and misconfiguration in kubernetes, you can use the following tools:

Kube-bench. It is one of the most powerful tools in the section. Its security check is based on CIS Kubernetes Benchmark.
Kubeaudit
Kubesect.io

Conclusion

Security Topic is very important and complex, especially in the Docker and Kubernetes world. This article contains the most important recommendations to be taken into account. You can find more information about Containers Trends in the Containers Tend Report.

Need to Build Event-Driven Architecture in the Azure? Easy with Azure Event Grid and Azure function!

Boris Zaikin — Wed, 24 Mar 2021 11:48:52 +0000

Building and supporting infrastructure in a large organization can be a challenging task due to automation of all deployment steps, for example, creating cloud environments for a department with a set of virtual machines that contain preinstalled software, utilities, and services that may have their own dependencies like database and storage, connection to log analytics and other log services. Utilities and services may have external dependencies as well. Therefore you need to have a mechanism to check and ensure that required connections are established, dependencies are installed, proper automation runbook is triggered. The mechanism should alert the administrator of error, perform a backup.

To build this mechanism, I will use the following Azure resources.

Event Grid is a key component that acts as a main event processor and contains:

Topics are azure resources that represent components that generate events.
Subscriptions/endpoint is azure resource that handles the event.

The relation between publisher components, events, topic, and subscriber/endpoint is shown in the diagram below.

Event grid contains:

Dead Letter Queue and retry policy — if message not able to reach the Endpoint, then you should also configure the retry policy
Event filtering — the rule allows the event grid to deliver specific event types to the endpoint point. For example: when you create a new VM in the topic container (resource group or subscription). The Event Grid will catch the event and deliver it to the endpoint.

Queue Storage Account

Queue Storage uses as a main event storage. When the event is generated via Event Grid, the final destination will be a queue.

Azure Function

Functions are used as microservice that contains logic to validate required resources and connections. Each function may also contain a database for storing configuration or state management data.

After a rather high-level architecture description, I will provide more details on it below.

Architecture

As you have noticed, an event grid is linked with a subscription listening on the events related to new virtual machine creation. It is necessary to add filtering here. Otherwise, the event grid will generate messages whenever any resource is created in a subscription or the target resource group. All events delivered are delivered in the storage queue. In my project, I’m using three queues:

The main queue is a destination for all messages from the event grid.
Retry queue receives all messages which failed during the first steps of validation and were scheduled for future retry.
Succeeded queue is used for all successfully processed messages. In my project, I used this queue for future statistics and reporting.

Also, a storage account is linked to Azure Log Analytics to synchronize all logs and alerts. For example, log analytics will log this as an error alert administrator if there are more messages in the retry queue than expected.

The next component is the Azure function app contains several azure functions with validation, message processing, and logic to trigger the runbook.

Validation Function is linked with the main event queue. When the event grid sends a message to the main queue, the function is automatically triggered.

Retry Function is based on a timer trigger and will be run constantly to check failed messages intended to retry (in retry event queue).

API Function(HTTP trigger) is intended to trigger (re-run) the whole process from runbook, Admin UI, etc.

The whole functions workflow is described below.

Architecture Code Base

The function app is written on PowerShell (Powershell Class) and uses the principles of OOP. This approach allows us to build modular well-supported code and add or remove functions at any time.

Why did I opt for PowerShell and not c# or JavaScript? The main reason is that a solution based on PS can be supported not only by developers but also by cloud administrators and system engineers.

Conclusion

In this article, I described building event-driven architecture to manage the virtual machine, related utilities, and components.

You can reuse the presented solution in the following scenarios:

Key Vaults and SSL certificate management (check certificate expiration time, log and inform, update certificate automatically)
Create custom logic to build cloud expense reports
Cloud resources backup, check availability and log (using Log Analytics or other tools)
Resource clean up management
Container management solution

- Here you can find Source Code

- And here the complete tutorial article.

Bonus

As this topic quite useful and popular in the world of Cloud and Software architecture. Moreover, these approaches solve a lot of business problems. Taking this into account, I’ve decided to create a practical, interactive course. It is based on my experience building EDA architecture for enterprise customers.

You can play with code, test architecture, and run pipelines directly in the course! Below you can find a link and enroll in the course.

Building Event Driven and Microservices Architecture in Azure

Building a Bot as an Alternative to Classic User Interface. Why is it better?

Boris Zaikin — Mon, 22 Mar 2021 09:39:34 +0000

My team and I were working on an open-source customer project in the aerospace domain that provided scientific data for students and university data scientists. After several attempts to build a classic Web UI application, we moved to a simplified Bot Flow. And this is why and how.

The project has a short and straightforward data response. It contains only essential data like:

Coordinates.
Event time frames.
Source type, etc.

However, some messages have links to the detailed report that is a classic Web UI. Having a chat with a bot is more natural as you feel like you are communicating with a person.

Azure Healthcare Bot

Here is another good example of how bot service can be convenient in our daily life. Recently Microsoft has launched the Healthcare Bot Service.

Introduction

In this article, I build a simplified version of a Bot that aggregates data from NASA API and provides the user with a simple workflow and short data feedback. Also, I use Azure Language Recognition (LUIS) service to improve and simplify the conversation flow.

The NASA Bot contains the following functions:

Gather data from the NASA API.
Aggregate and simplify data payload.
Recognize the user input with Azure Language Recognition (LUIS). We will go into the LUIS below.

Before we dive deep into the details, let’s examine why the Bot is better than a classic UI.

Using the bot approach may significantly improve the user experience if you have:

Simplified and short data payload.
Intuitive user interaction flow that looks like a Q&A dialog.
No complex UI.
Focus on content.

UI is better than a Bot if you have:

Complex data payload.
Complex UI forms (Admin Portals, Editors, Maps, News Portals, Calendars).
Complex data visualization (Dashboards, Charts).

The best option may be to combine both approaches and use all benefits from a Bot and a classic UI.

Why NASA API?

I have chosen NASA API because of:

A granular and quite simple API design.
Well-documented API calls.
Easy and fast registration process.
API key request.

NASA API is divided into categories or sub APIs, like:

Space Weather Database Of Notifications, Knowledge, Information (DONKI)
Mars Weather Service API
Two-line element data earth-orbiting objects at a given point in time (TLE API)
Satellite Situation Center

NASA API with other calls.

Why Azure Bot Service?

Azure Bot Service, alongside the Bot Framework, provides you with an opportunity to build, test, store your Bot. Bot Framework includes tools, SDK, AI services, and templates that allow you to make a bot quite quickly.

In my case, I have created a project with Visual Studio using the template and installed a toolset, SDK, and templates. I have used .NET Core and C#. However, this framework also allows using JavaScript and Python.

What is LUIS?

LUIS is a language understanding service that allows you to build and train your Bot to understand a natural language. I have included LUIS in my Bot to provide people with information from NASA API using different natural language construction.

More information about LUIS.

For the current project, I have trained LUIS manually using the LUIS Portal. Below you can see that I added language sentences to train the LUIS to recognize user feedback and set the score. The LUIS portal also contains an option that allows you to test your data set and deploy it to different stages.

Of course, a manually trained model is suitable for small and middle-level bot projects. You should consider using pre-trained models that you quickly find in JSON format or set up for the production application.

You can also improve language understanding by adding QnA (Question and Answers) and language generation templates.

NASA Bot Architecture Overview

The NASA Bot architecture represents an N-tire structure based on an MVC pattern. The Bot includes API Controllers, Models, and Services.

The NASA bot consists of the following main components.

Nasa bot class is a component that contains the conversation flow event handlers like: MembersAdded and MessageActivity. And it also serves as an entry point to start a conversation flow.
External API aggregation service joins logic related to language recognition and fetches data from NASA API.
NasaApi library subproject is a wrapper of Nasa API calls. It contains logic to obtain, parse data from API and build a proper response for the business layer.

You can find code examples and the NASA Bot project here.

Below is the list of other bot components with a link to the GIT repository:

API (and BotController) is an endpoint to the Bot, and when you create your Bot with Visual Studio or VS Code, it creates a project that looks almost like a Web API project. The Web API is based on an MVC pattern.
LanguageRecognitionService connects the Bot to the LUIS API and triggers the language recognition process.
ObservationTypeModel contains all fields related to LUIS Response, including maximum intent score calculation logic.

Conversation workflow

Below, I have added several screenshots that demonstrate an interaction with the Bot. As you see, here I used Bot Framework Emulator.

Conclusion

The article demonstrates the main advantages that bot services can bring in comparison to UI interfaces. In the next article, I will extend the Bot by adding more advanced features.

The link to the git repository where you can clone the project.

What is next

In the next article, I will upgrade the Bot by adding:

QnA maker and models
Unit Tests
Other APIs
Improved language recognition (Involve language generation)
Bot deployment across multiple channels (Telegram, Slack)

Please feel free to suggest your ideas on what else should be added to the Bot or covered in the next article. Looking forward to hearing from you in the comments.

Easy and Fast Adjustment of Kubernetes CPU and Memory

Boris Zaikin — Wed, 17 Feb 2021 13:39:12 +0000

Assigning and Managing CPU and Memory resources in the Kubernetes can be tricky and easy at the same time. Having done this task for numerous customers, I have decided to create a framework zone. I will show you what Kubernetes resources and limits are and how to manage them.

The framework contains the following steps.

Infographic Guide shows what algorithms to follow to determine and assign the resources and limits.
Code templates allow applying those algorithms with minimal adaptation.
Algorithms and tools to gather the metrics about resource consumption and set up the limits.
Links to the official documentation where you can quickly grab some examples and read more detailed information.

What this article doesn’t contain.

My goal here is simplicity. So you won’t find detailed descriptions of how resources, limit ranges, and quotas work. There are plentiful articles written about it, as well as Kubernetes thorough documentation. Instead, here you will find information on how to quickly start adjusting Kubernetes resources in your projects.

Basics Guide on CPU and Memory Resources

Two cheat-sheets explaining what resources are in terms of Kubernetes, measurement units, the resource state workflow, and some rules on how to apply it.

Basics Guide on CPU and Memory Resources

Be aware that the CPU calculation formula is not applied to every setup or project. It is used as a starting point in the process of detecting and assigning the resource and limits.

Kubernetes deployment YAML template includes container, resource, and limits definitions.

More detailed information and code snippets:

Resources Quota Guide

Two quota templates that contain quotas definition for persistent volume claims and resources in the backed namespace.

You can use kubectl “apply” command to set the quota constraints for a namespace.

kubectl apply -f resource-quota.yaml — namespace backend

Detailed explanation of quotas, metrics, and how to use quota expressions and code samples.

https://kubernetes.io/docs/concepts/policy/resource-quotas/

Limit Ranges Guide

Two limit range code templates to limit CPU and Memory resources for the containers, and to limit the storage (Persistent Volume Claim). To set this constraint, you can also use kubectl apply -f path/to/file.yaml -n namespace

apiVersion: v1
kind: LimitRange
metadata:
  name: backend-limit-range
spec:
  limits:
  - default:
      memory: 110Mi
      cpu: 500m
    defaultRequest:
      memory: 20Mi
      cpu: 100m
    type: Container

--
apiVersion: v1
kind: LimitRange
metadata:
  name: backend-storage-limits
spec:
  limits:
  - type: PersistentVolumeClaim
    max:
      storage: 5Gi
    min:
      storage: 2Gi

More detailed information and code snippets.

Tools and Frameworks that Helps with Resource and Limits Routine

Popeye scans your cluster for potential issues with configuration, resources, and network holes and generates detailed reports with all issues.
Goldilocks scans pods for resource limits and creates reports with recommended resources.
Kube-advisor simple tool, from the Azure team, that scans pods for missing resources and limits requests.
K9s+benchmark — provides a command-line interface (CLI) that allows you to easily manage, monitor, and even benchmark your cluster in your favorite terminal software

Some useful free and open-source tools, which are easy to set up. You can also combine these tools with Datadog, Grafana+Prometeus, Azure Monitor to improve resource and limits monitoring.

Algorithm

Setup Resource Requests. Get information about CPU and Memory usage of specific Application/Container.
Setup Resource Limits. Setup Resource Requests. Run a load test to detect CPU and Memory of a container under the high load..
Monitor containers CPU and Memory usage.
Monitor persistent storage usage.
Check if you can apply resource limit using limit ranges (if you have similar containers, storage set up)
Apply limit range for storage if needed. Or use Quota (it is not recommended to apply Quota for production environment)

Conclusion

I have shared with you the framework I created and use on a daily basis while working with the Kubernetes requests and limits. I hope you will find it useful. Any ideas on how to make it better?

Please feel free to share your feedback and suggestions in the comments.

6 Easy Ways to Manage and Harden VM Images in Azure

Boris Zaikin — Wed, 23 Sep 2020 15:24:36 +0000

Managing VM Images may be a nightmare. Here are 6 simple approaches on how to seamlessly build, share, test, and copy images in Azure. And as a bonus, you also get how to build image notifications based on Event-Driven-Architecture.

Problem

Whether you are managing 100 Virtual Machines or 1000+ that build and harden VM images, do you manually manage your images? If you do, you know that it is quite expensive and leads to hard-to-detect errors and potential security vulnerabilities.

Here is how I approach this issue for my customers. My procedure of creating image management for Azure.

1.Building an image using the Azure Image Builder

The Azure Image Builder is a service that allows you to create custom images with Azure CLI. The image creation based on the JSON template, example below.

{
    "type": "Microsoft.VirtualMachineImages/imageTemplates",
    "apiVersion": "2019-05-01-preview",
    "location": "<region>",
    "dependsOn": [],
    "tags": {
        "imagebuilderTemplate": "ubuntu1804",
        "userIdentity": "enabled"
            },
        "identity": {
            "type": "UserAssigned",
                    "userAssignedIdentities": {
                    "<imgBuilderId>": {}

                }
                },
    "properties": {

        "buildTimeoutInMinutes" : 80,

        "vmProfile": 
            {
            "vmSize": "Standard_D1_v2",
            "osDiskSizeGB": 30
            },

        "source": {
            "type": "PlatformImage",
                "publisher": "Canonical",
                "offer": "UbuntuServer",
                "sku": "18.04-LTS",
                "version": "latest"

        },
        "customize": [
            {
                "type": "Shell",
                "name": "RunScriptFromSource",
                "scriptUri": "https://raw.githubusercontent.com/danielsollondon/azvmimagebuilder/master/quickquickstarts/customizeScript.sh"
            }          

        ],
        "distribute": 
            [

            ]
        }
    }

I have to cut the original template as it is quite long. Here you can find full example.

The ARM template is quite simple it contains following properties:

Identity section is required, you have to create Managed Identity for image builder to have an access to create and edit images
VmProfile is to set up VM configuration plan
Source allows you to specify base image parameters. I use the latest Ubuntu Server 18.04 LTS from Canonical
The customization section allows you to specify VM hardening scripts.

Here you can see the full list of the Image Builder options.

Advantages

Templates and the process itself is easy to understand and it can be easily integrated with Azure DevOps.

Disadvantages

The image builder is still in the review therefore it is not recommended using it in the production. Set up can be a bit difficult in comparison to Hashicorp Packer. (explain how to work with the Packer in the next section).

2. Building an image using the Hashicorp Packer

Hashicorp Packer is a multi-platform solution that allows building custom images based on JSON templates. The JSON templates are well-structured and based on an easy-to-understand object model. The JSON templates have just three root objects: Communicators, Builders, Provisioners, and Post-Processors.

Here you can find the JSON template.

JSON template creates the VHD image of Ubuntu with a preinstalled NGINX web server and other updates. Here you can find a lot of other templates.

To set up packer you can use Chocolatie package manager:

choco install packer -y

You can run the JSON template using the following command:

packer build <path/your/template.json>

Before your run the template you need to create Management Identity or Service Principal with proper permissions, For example:

az ad sp create-for-rbac -n "ImageContributor" 
 --role contributor `    
 --scopes /subscriptions/<subscription_id> `     
 --sdk-auth > az-principal.auth

This command automatically creates JSON file with clientId, clientSecret, tenantId, subscriptionId and other fields.

Advantages

The Packer and JSON templates simple to understand. They allow you to quickly set up the environment and start building images. There is also a strong community. Plus, the Packer supports multiple cloud providers.

Disadvantages

I have not found any serious disadvantages. However, while building an image, the packer always removes the disk that is required in some operations with images. For example, for copying an image.

3. Sharing images

To share images in Azure, you may use the Shared Image Gallery. It can:
- create image definition
- keep versions of an image
- share an image

For example, you can share an image across your Azure subscriptions, resource groups, and tenants.

In the current scenario, you can create a user group or a single user/service principal, assign contributor rights only for this shared image gallery.

az ad sp create-for-rbac -n "ImageContributor" 
 --role contributor `    
 --scopes /subscriptions/<subscr-id>/resourceGroups/sig-we-rg/providers/Microsoft.Compute/galleries/testsig

As a result, users from the group can create the Virtual Machine based on the Ubuntu image from this Shared Image Gallery in the different subscriptions.

az vm create `
  --resource-group myResourceGroup `
  --name UbuntuVM`
  --image "/subscriptions/<subscr-id>/resourceGroups/sig-we-rg/providers/Microsoft.Compute/galleries/testsig/images/ubuntu-server-image-def/versions/1.0.0" \
  --admin-username azureuser \
  --generate-ssh-keys

Advantages

The Azure Shared Image gallery easily allows you to build, share, manage, and customize images within your organization. SIG has Azure CLI, so you can easily automate image distribution.

Disadvantages

The image remains in a shared access gallery. Thus, it physically stays there. So, when you share it across subscriptions, you cannot change or remove it independently for each subscription.

4. Copying images

By copying images, you can deliver images from one subscription to another or from one resource group to another. All copied images are independent of each other. You can copy images using Azure Image Copy extension, or implement manual copying using Go. Let us have a look at the examples below.

Copying image with Az Copy Extension

To copy images between subscriptions, I use Az Image Copy extension. It creates a new image (from the source image) in Resource Group A.

Install copy extension:

az extension add --name image-copy-extension

Copy command:

az image copy --source-resource-group Azure-Resource-Group-B `
              --source-object-name image-version-1.0 `
              --target-resource-group Azure-Resource-Group-A `
              --target-location westeurope `
              --target-subscription 111111-2222-2222-0000-0000000 ` # Subscription A
              --cleanup

Important. Packer removes the disk automatically after the managed image is created.

Advantages

Az Image Copy extension is simple to use and automate.

Disadvantages

An image must contain a managed disc, otherwise the copy process fails with ‘Resource not found’ error message.

Copy image manually

If Az Copy Image Extension does not work for you, you can create a VHD image in the storage account and copy it to another destination storage account.

The process workflow:

Create 2 Storage accounts, Source and Destination
Create the VHD image in the Source storage account. You can use the Packer script from the first section.
Generate Shared Access Signature for the VHD Source image. Here you can find an example of how to do it with Azure CLI.
And copy the image. To demonstrate I use the following Go script. You can also use the AzCopy tool, here the example of how to do it.

You can find source code here.

Advantages

The current image coping workflow is fully custom. Therefore, it can be changed at any time. Also, it can be useful when Az Image Copy extension does not work you. For example, when it removes managed disk while creating an image.

Disadvantages

You have to implement all workflow steps including:

- creating VHD
- generating and managing SAS token
- copying an image
- cleaning up
- converting an image

5. Image and Disk Converting

The operations to convert a VHD image to a Managed Disk, or a Managed Disk to a VHD image. This is useful when you need to distribute images across your organization with Azure Shared Image Gallery in the image copy process and when you need you to spin up a new VM.

Also, it is useful when you have some legacy VHD and need to install updates, set up automatic backups, use availability sets, and availability zones.

Here is a list of advantages of the Managed Image.

Converting VHD image to the Managed Disk

# Provide the subscription Id where Managed Disks will be created
$subscriptionId = '00000-00000-0000-0000-0000000'

# Provide the name of your resource group where Managed Disks will be created. 
$resourceGroupName ='HBI'

# Provide the name of the Managed Disk
$diskName = 'image-test-disk'

# The Disk It should be greater than the VHD file size.
$diskSize = '130'

# Storage LRS. Options: Premium_LRS, Standard_LRS, etc
$storageType = 'Premium_LRS'

# Set Azure region (e.g. westus, westeurope), Ensure that location of the VHD image (alongside with Storage Account), 
# and future Managed Disk is the same.
$location = 'westeurope'

# Set URI of the VHD file (page blob) in a storage account.
$sourceVHDURI = 'https://imagesstorage.blob.core.windows.net/test-image-disk.vhd'

# Set the Resource Id of the Source storage account where VHD file is stored. You can avoid it if VHD in the same subscription
$storageAccountId = '/subscriptions/subscription-id/resourceGroups/test-rg/providers/Microsoft.Storage/storageAccounts/imagesstorage'

#Set the context to the subscription Id where Managed Disk will be created
Select-AzSubscription -SubscriptionId $SubscriptionId

$diskConfig = New-AzDiskConfig -AccountType $storageType -Location $location -CreateOption Import -StorageAccountId $storageAccountId -SourceUri $sourceVHDURI

New-AzDisk -Disk $diskConfig -ResourceGroupName $resourceGroupName -DiskName $diskName

Convert Managed Disk to Managed Image

$rgName = "HBI"
$location = "westeurope"
$imageName = "boriszn-test-imagefromdisk"


# Get disk 
$disk = Get-AzDisk -ResourceGroupName 'HBI' -DiskName 'image-test-disk' 
$diskId = $disk.Id

# Create Managed Image Config with Managed Disk Info 
# OS Types (Linux, Windows)
$imageConfig = New-AzImageConfig -Location $location
$imageConfig = Set-AzImageOsDisk -Image $imageConfig -OsState Generalized -OsType Linux -ManagedDiskId $diskId

# Create the image.
$image = New-AzImage -ImageName $imageName -ResourceGroupName $rgName -Image $imageConfig

Advantages

With both cmdlets, you can deliver an image to your Azure Shared Image across different subscriptions and resource groups.

Disadvantages

The conversion process can be complicated because some images can be outdated, and the conversion process may fail.

6.Testing images

To test images, you can simply spin-up the new VM from the image gallery and use an image definition

az vm create `
   --resource-group container-image-rg `
   --name vm-test `
   --image "/subscriptions/<subscription-id>/resourceGroups/container-image-rg/providers/Microsoft.Compute/galleries/test-gallery/images/image-test-def" `
   --generate-ssh-keys

Advantages

The command and process itself quite simple.

Disadvantages

It does not check whether specific software and services are installed correctly in the JSON configuration. So this logic has to be implemented separately.

The Images Pub/Sub subsystem concept (Bonus)

Managing images across several subscriptions or resource groups can be difficult, especially when you constantly producing new versions of images, or you have some automated process to spin-up new virtual machines.

In this case, you need to build a notification system for notifying different components when a new image was created, a new version or image definition appears in the Azure Shared Image Gallery. You can easily create it using Azure Event Grid with filters.

You can see how it can be done with the event grid, Queue, Webhook, and Azure Service Bus to deliver messages to the target component here.

Conclusion

That’s it. Based on these ways you can easily set up images Hardening and management pipeline in the Azure DevOps.

CI/CD as a Code for .NET Core application and Kubernetes using Azure DevOps +YAML.

Boris Zaikin — Mon, 25 May 2020 15:51:52 +0000

Building CI/CD process for the .NET Core application could be complicated especially when you are dealing with Kubernetes and Docker and if you need to include code style analysis, unit test, and code coverage report.

In this article I am going to explain the process of building a simple CI/CD pipeline for existing .net core application to moving it to Azure Kubernetes Services using Azure DevOps. The final Pipeline will be easy to understand and reusable.

Deployment Architecture

Deployment process includes the following steps: fetching the code from Git Repository, building application, restoring NuGet packages, running unit test, building unit test and code coverage reports, pushing docker image to registry and deploying to AKS cluster.

Azure Kubernetes Cluster setup

I have used a simple AKS cluster, with 3 nodes architecture that contains Ingress-nginx load balancer. You can also use Traefik.io, istio.io even Standard Azure Load Balancer.

Detailed description of how to setup AKS cluster including Ingress setup and deployment scripts can be found Here.

After deploying AKS, the main resource group it will look like in the image below.

Setup service connections

Before starting to configure the main pipeline steps the connection between Azure Container Registry(ACR) and Azure Kubernetes service needs to be granted by granting access of AKS service principal to ACR. RBAC service principal for Azure DevOps is created and everything is ready to push and pull docker images withing pipelines. Alternatively you can do it in Azure DevOps Service Connection which I will explain in the next session.

$AksResourceGroup = '<rg-name>'
$AksClusterName = '<aks-cluster-name>'
$AcrName = '<acr-name>'
$AcrResourceGroup = '<acr-rg-name>'

# Get the id of the AKS service principal
ClientID=$(az aks show --resource-group $AksResourceGroup --name $AksClusterName --query "servicePrincipalProfile.clientId" --output tsv)

# Get the ACR registry resource id
AcrId=$(az acr show --name $AcrName --resource-group $AcrResourceGroup --query "id" --output tsv)

# Create role assignment
az role assignment create --assignee $ClientID --role acrpull --scope $AcrId

# Create a specific Service Principal for our Azure DevOps pipelines to be able to push and pull images and charts of our ACR
$ registryPassword=$(az ad sp create-for-rbac -n $acr-push --scopes $AcrId --role acrpush --query password -o tsv)

Here you can find a detailed description of how to configure the connection between ACR and AKS.

Azure DevOps service connection with Azure Kubernetes Services and Azure Container Registry.

By Using Service Connection you can connect Azure DevOps to your, already deployed AKS cluster, Azure Container Registry, Docker Registry (Docker Hub), and many other services.

The creation of connection to ACR is quite easy, you just need to specify a connection name, a subscription, and a registry name and that’s it.

You can connect (and authenticate) your AKS cluster using Kubeconfig, Service Account and Azure Subscription. In my project I used Kubeconfig, as one of the fast option, because you need just find kubeconfig JSON, copy it and choose your Cluster context.

You can find KubeConfig it in the following directory (In the Windows): C:\Users\your_user_name\.kube\config Here is documentation how to find and work with KubeConfig also for Linux and Mac.
Pipeline steps

Finlay I’m moving to pipeline steps and the very first step is to use a script for restoring all NuGet dependencies/packages, building dot net core application, Running Unit Tests, building code coverage report. Also please notice that I use system variable $(Build.BuildNumber) as tag for the coverage report generation. At the end test results will be published as artifacts and Azure DevOps can build visualization analytics charts.

- script: |
    dotnet restore
    dotnet build ./src/DeviceManager.Api/ --configuration $(buildConfiguration)
    dotnet test ./test/DeviceManager.Api.UnitTests/ --configuration $(buildConfiguration) --filter Category!=Integration --logger "trx;LogFileName=testresults.trx"
    dotnet test ./test/DeviceManager.Api.UnitTests/ --configuration $(buildConfiguration) --filter Category!=Integration /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(System.DefaultWorkingDirectory)/TestResults/Coverage/
    cd ./test/DeviceManager.Api.UnitTests/
    dotnet reportgenerator "-reports:$(System.DefaultWorkingDirectory)/TestResults/Coverage/coverage.cobertura.xml" "-targetdir:$(System.DefaultWorkingDirectory)/TestResults/Coverage/Reports" "-reportTypes:htmlInline" "-tag:$(Build.BuildNumber)"
    cd ../../
    dotnet publish ./src/DeviceManager.Api/ --configuration $(buildConfiguration) --output $BUILD_ARTIFACTSTAGINGDIRECTORY

Below you can see Tests blade with statistic data and Test Results in the Tests section of the Azure DevOps.

Step 2 - Code coverage results

The next step is to publish code coverage results to DefaultWorkingDirectory. The whole process is based on Cubertura Report Generator, a .Net core library.

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: cobertura
    summaryFileLocation: $(System.DefaultWorkingDirectory)/TestResults/Coverage/**/*.xml
    reportDirectory: $(System.DefaultWorkingDirectory)/TestResults/Coverage/Reports
    failIfCoverageEmpty: false

The first statistic results are already available:

The detailed file by file report you can find on the Code Coverage tab. This report is based on generated xml reports.

Step 3 and 4 — Building Container and Pushing it to the ACR

After the previous steps have been completed the project is to be put into a container. For this I am going to use version 1 Docker step. You need to specify a path to a Docker file like I did it here, provide image name, for example, boriszn/devicemanagerapi and tag — 1.102.1, and the last step is to specify a container registry, for example, devicemanagerreg.azurecr.io.

For a tag creation I used semantic versioning and hard-coded some version numbers to simplify the Pipeline, however you can also use different approaches to build a tag, for example using variables that are required to run a pipeline or take then from git version.

- task: Docker@1
  displayName: 'Containerize the application'
  inputs:
    azureSubscriptionEndpoint: $(serviceConnection)
    azureContainerRegistry: $(containerRegistry)
    dockerFile: './src/DeviceManager.Api/Dockerfile'
    imageName: '$(fullImageName)'
    includeLatestTag: true

The next step is to push our dockerised app to the Azure Container Registry. Here I specified command and image to push. You should take into account that I pushed 2 images with the first image tag 1.58338.4 and the second one is tag: latest.

- task: Docker@1
  displayName: 'Push image'
  inputs:
    azureSubscriptionEndpoint: $(serviceConnection)
    azureContainerRegistry: $(containerRegistry)
    command: 'Push an image'
    imageName: '$(fullImageName)'

- task: Docker@1
  displayName: 'Push latest image'
  inputs:
    azureSubscriptionEndpoint: $(serviceConnection)
    azureContainerRegistry: $(containerRegistry)
    command: 'Push an image'
    imageName: '$(imageName):latest'

AKS deployment steps

For Azure Kubernetes Deployment you need to replace a build number in AKS deployment YAML file and display it in Azure DevOps after successful execution.

- task: PowerShell@2
  displayName: 'Replace version number in AKS deployment yaml'
  inputs:
    targetType: inline
    script: |
        # Replace image tag in aks YAML
        ((Get-Content -path $(aksKubeDeploymentYaml) -Raw) -replace '##BUILD_ID##','$(imageTag)') | 
        Set-Content -Path $(aksKubeDeploymentYaml)
        # Get content
        Get-Content -path  $(aksKubeDeploymentYaml)

After that run “Apply” command for AKS cluster.

Two important parameters here are a path to my cluster deployment YAML script (./deployment/aks-deployment.yaml) and a cluster name (device-manager-api-aks). Other parameters were explained in the previous sections.

- task: Kubernetes@1
  displayName: 'kubectl apply'
  inputs:
    kubernetesServiceEndpoint: $(kubernetesServiceEndpoint)
    azureSubscriptionEndpoint: $(serviceConnection)
    azureResourceGroup: $(azureResourceGroupName)
    kubernetesCluster: $(aksClusterName)
    arguments: '-f $(aksKubeDeploymentYaml)'
    command: 'apply'

That’s it! Below I’ve listed complete YAML pipeline that we configured in the previous few sections and that is easy to import to your Azure DevOps projects. If you have any questions reach out in the comments!

trigger:
- develop

pool:
  vmImage: 'ubuntu-latest'

variables:
  imageName: 'boriszn/devicemanagerapi'
  buildConfiguration: 'Release'
  fullImageName: '$(imageName):$(imageTag)'
  containerRegistry: devicemanagerreg.azurecr.io
  imageTag: '1.$(build.buildId).4'
  serviceConnection: 'az-connect'
  azureResourceGroupName: 'boriszn-rg-aks-devicemanager-api-we'
  aksClusterName: 'device-manager-api-aks'
  aksKubeDeploymentYaml: './deployment/aks-deployment.yaml'
  kubernetesServiceEndpoint: 'device-managerapi-aks-connect'

steps:
- script: |
    dotnet restore
    dotnet build ./src/DeviceManager.Api/ --configuration $(buildConfiguration)
    dotnet test ./test/DeviceManager.Api.UnitTests/ --configuration $(buildConfiguration) --filter Category!=Integration --logger "trx;LogFileName=testresults.trx"
    dotnet test ./test/DeviceManager.Api.UnitTests/ --configuration $(buildConfiguration) --filter Category!=Integration /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(System.DefaultWorkingDirectory)/TestResults/Coverage/
    cd ./test/DeviceManager.Api.UnitTests/
    dotnet reportgenerator "-reports:$(System.DefaultWorkingDirectory)/TestResults/Coverage/coverage.cobertura.xml" "-targetdir:$(System.DefaultWorkingDirectory)/TestResults/Coverage/Reports" "-reportTypes:htmlInline" "-tag:$(Build.BuildNumber)"
    cd ../../
    dotnet publish ./src/DeviceManager.Api/ --configuration $(buildConfiguration) --output $BUILD_ARTIFACTSTAGINGDIRECTORY
- task: PublishTestResults@2
  inputs:
    testRunner: VSTest
    testResultsFiles: '**/*.trx'

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: cobertura
    summaryFileLocation: $(System.DefaultWorkingDirectory)/TestResults/Coverage/**/*.xml
    reportDirectory: $(System.DefaultWorkingDirectory)/TestResults/Coverage/Reports
    failIfCoverageEmpty: false

- task: PublishBuildArtifacts@1

- task: Docker@1
  displayName: 'Containerize the application'
  inputs:
    azureSubscriptionEndpoint: $(serviceConnection)
    azureContainerRegistry: $(containerRegistry)
    dockerFile: './src/DeviceManager.Api/Dockerfile'
    imageName: '$(fullImageName)'
    includeLatestTag: true

- task: Docker@1
  displayName: 'Push image'
  inputs:
    azureSubscriptionEndpoint: $(serviceConnection)
    azureContainerRegistry: $(containerRegistry)
    command: 'Push an image'
    imageName: '$(fullImageName)'

- task: Docker@1
  displayName: 'Push latest image'
  inputs:
    azureSubscriptionEndpoint: $(serviceConnection)
    azureContainerRegistry: $(containerRegistry)
    command: 'Push an image'
    imageName: '$(imageName):latest' 

- task: PowerShell@2
  displayName: 'Replace version number in AKS deployment yaml'
  inputs:
    targetType: inline
    script: |
        # Replace image tag in aks YAML
        ((Get-Content -path $(aksKubeDeploymentYaml) -Raw) -replace '##BUILD_ID##','$(imageTag)') | 
        Set-Content -Path $(aksKubeDeploymentYaml)
        # Get content
        Get-Content -path  $(aksKubeDeploymentYaml)
- task: Kubernetes@1
  displayName: 'kubectl apply'
  inputs:
    kubernetesServiceEndpoint: $(kubernetesServiceEndpoint)
    azureSubscriptionEndpoint: $(serviceConnection)
    azureResourceGroup: $(azureResourceGroupName)
    kubernetesCluster: $(aksClusterName)
    arguments: '-f $(aksKubeDeploymentYaml)'
    command: 'apply'

Error handling and validation architecture in .NET Core

Boris Zaikin — Tue, 11 Dec 2018 17:05:47 +0000

In many projects error handling and validation is distributed across business logic, API controllers, data access layer in the form of conditions (“if-else” sequences). This leads to the violation of the Separation of Concerns Principle and results in “Spaghetti code”, like in the example below.

....
    if (user != null)
    {
        if (subscription != null)
        {
            if (term == Term.Annually)
            {
                // code 1
            }
            else if (term == Term.Monthly)
            {
                // code 2
            }
            else
            {
                throw new InvalidArgumentException(nameof(term));
            }
        }
        else
        {
            throw new ArgumentNullException(nameof(subscription));
        }
    }
    else
    {
        throw new ArgumentNullException(nameof(user));
    }
.....

In this article I describe the approach to splitting validation and error handling logic from the other application layers. The patterns and practices used below can be found in the Git Hub repository below.

DeviceManager.Api

Architecture overview

For simplicity I use N-tire architecture, however, explained approaches can be reused in CQRS, Event Driven, Micro Services, SOA, etc architectures.

Example architecture includes following layers:

Presentation Layer — UI / API
Business Logic Layer — Services or Domain Services (in case you have DDD architecture)
Data Layer / Data Access Layer

In the diagram below shows the components and modules which belong to different layers and contains presentation/API layer, business logic layer, data access, in the right side and related validation and error handling logic in the left side.

The validation and error handling architecture contains several components which I will describe in next few sections.

API validation level

API controllers may contain a lot of validation such as parameters check, model state check etc like on example below. I will use declarative programming to move validation logic out from API controller.

[HttpGet]
[SwaggerOperation("GetDevices")]
[ValidateActionParameters]
public IActionResult Get([FromQuery][Required]int page, [FromQuery][Required]int pageSize)
{
    if (!this.ModelState.IsValid)
    {
        return new BadRequestObjectResult(this.ModelState);
    }

    return new ObjectResult(deviceService.GetDevices(page, pageSize));
}

API controllers can be easily cleaned by creating validation model attribute. Example below contains simple model validation check.

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Filters;

namespace DeviceManager.Api.ActionFilters
{
    /// <summary>
    /// Intriduces Model state auto validation to reduce code duplication
    /// </summary>
    /// <seealso cref="Microsoft.AspNetCore.Mvc.Filters.ActionFilterAttribute" />
    public class ValidateModelStateAttribute : ActionFilterAttribute
    {
        /// <summary>
        /// Validates Model automaticaly 
        /// </summary>
        /// <param name="context"></param>
        /// <inheritdoc />
        public override void OnActionExecuting(ActionExecutingContext context)
        {
            if (!context.ModelState.IsValid)
            {
                context.Result = new BadRequestObjectResult(context.ModelState);
            }
        }
    }
}

Just add this attribute to the startup.cs

services.AddMvc(options =>
{             
   options.Filters.Add(typeof(ValidateModelStateAttribute));
});

To validate parameters of API action methods I will create an attribute and move validation logic. Logic inside attribute checks if parameters contains validation attributes and validates the value.

Now attribute can be added to the action method, if necessary. (examples below)

[HttpGet]                               
[SwaggerOperation("GetDevices")]                               [ValidateActionParameters]
public IActionResult Get(
[FromQuery, Required]int page, 
[FromQuery, Required]int pageSize)                               
{                                   
  return new ObjectResult(deviceService.GetDevices(page, pageSize));                               }

Business layer validation

Business layer validation consists 2 components: validation service and validation rules.

In the device validation services I’ve moved all custom validation and rule based validation logic from the service (Device service in the example below). This idea quite similar to using Guard pattern. Below the example of the validation service.

using System;
using DeviceManager.Api.Model;
using DeviceManager.Api.Validation;
using FluentValidation;

namespace DeviceManager.Api.Services
{
    /// <inheritdoc />
    public class DeviceValidationService : IDeviceValidationService
    {
        private readonly IDeviceViewModelValidationRules deviceViewModelValidationRules;

        /// <summary>
        /// Initializes a new instance of the <see cref="DeviceValidationService"/> class.
        /// </summary>
        /// <param name="deviceViewModelValidationRules">The device view model validation rules.</param>
        public DeviceValidationService(
            IDeviceViewModelValidationRules deviceViewModelValidationRules)
        {
            this.deviceViewModelValidationRules = deviceViewModelValidationRules;
        }

        /// <summary>
        /// Validates the specified device view model.
        /// </summary>
        /// <param name="deviceViewModel">The device view model.</param>
        /// <returns></returns>
        /// <exception cref="ValidationException"></exception>
        public IDeviceValidationService Validate(DeviceViewModel deviceViewModel)
        {
            var validationResult = deviceViewModelValidationRules.Validate(deviceViewModel);

            if (!validationResult.IsValid)
            {
                throw new ValidationException(validationResult.Errors);
            }

            return this;
        }

        /// <summary>
        /// Validates the device identifier.
        /// </summary>
        /// <param name="deviceId">The device identifier.</param>
        /// <returns></returns>
        /// <exception cref="ValidationException">Shuld not be empty</exception>
        public IDeviceValidationService ValidateDeviceId(Guid deviceId)
        {
            if (deviceId == Guid.Empty)
            {
                throw new ValidationException("Should not be empty");
            }

            return this;
        }
    }
}

In the rules I’ve moved all possible validation checks related to the view or API models. In the example below you can see Device view model validation rules. The validation itself triggers inside the the validation service.

The Validation rules based on FluentValidation framework which allows you you build rules in fluent format.

using DeviceManager.Api.Model;
using FluentValidation;

namespace DeviceManager.Api.Validation
{
    /// <summary>
    /// Validation rules related to Device controller
    /// </summary>
    public class DeviceViewModelValidationRules : AbstractValidator<DeviceViewModel>, IDeviceViewModelValidationRules
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="DeviceViewModelValidationRules"/> class.
        /// <example>
        /// All validation rules can be found here: https://github.com/JeremySkinner/FluentValidation/wiki/a.-Index
        /// </example>
        /// </summary>
        public DeviceViewModelValidationRules()
        {
            RuleFor(device => device.DeviceCode)
                .NotEmpty()
                .Length(5, 10);

            RuleFor(device => device.DeviceCode)
                .NotEmpty();

            RuleFor(device => device.Title)
                .NotEmpty();
        }
    }
}

Exception handling Middleware

The last turn I will cover errors/exception handling. I address this topic to the end as all validation components generate exceptions and the centralized component that handles them and provide proper JSON error object is required to have.

In the example below I’ve used .NET core Middleware to catch all exceptions and created HTTP error status, according to Exception Type (in ConfigurationExceptionType method) and build error JSON object.

Also Middleware can be used to log all exception in one place.

using System;
using System.Net;
using System.Threading.Tasks;
using DeviceManager.Api.Model;
using FluentValidation;
using Microsoft.AspNetCore.Http;
using Newtonsoft.Json;

namespace DeviceManager.Api.Middlewares
{
    /// <summary>
    /// Central error/exception handler Middleware
    /// </summary>
    public class ExceptionHandlerMiddleware
    {
        private const string JsonContentType = "application/json";
        private readonly RequestDelegate request;

        /// <summary>
        /// Initializes a new instance of the <see cref="ExceptionHandlerMiddleware"/> class.
        /// </summary>
        /// <param name="next">The next.</param>
        public ExceptionHandlerMiddleware(RequestDelegate next)
        {
            this.request = next;
        }

        /// <summary>
        /// Invokes the specified context.
        /// </summary>
        /// <param name="context">The context.</param>
        /// <returns></returns>
        public Task Invoke(HttpContext context) => this.InvokeAsync(context); 

        async Task InvokeAsync(HttpContext context)
        {
            try
            {
                await this.request(context);
            }
            catch (Exception exception)
            {
                var httpStatusCode = ConfigurateExceptionTypes(exception);

                // set http status code and content type
                context.Response.StatusCode = httpStatusCode;
                context.Response.ContentType = JsonContentType;

                // writes / returns error model to the response
                await context.Response.WriteAsync(
                    JsonConvert.SerializeObject(new ErrorModelViewModel
                    {
                        Message = exception.Message
                    }));

                context.Response.Headers.Clear();
            }
        }

        /// <summary>
        /// Configurates/maps exception to the proper HTTP error Type
        /// </summary>
        /// <param name="exception">The exception.</param>
        /// <returns></returns>
        private static int ConfigurateExceptionTypes(Exception exception)
        {
            int httpStatusCode;

            // Exception type To Http Status configuration 
            switch (exception)
            {
                case var _ when exception is ValidationException:
                    httpStatusCode = (int) HttpStatusCode.BadRequest;
                   break;
                default:
                    httpStatusCode = (int) HttpStatusCode.InternalServerError;
                  break;
            }

            return httpStatusCode;
        }
    }
}

Conclusion

In this article I covered several option to create maintainable validation architecture. The main goal of this article is to clean up business, presentation and data access logic. I would not recommend considering these approaches as “Silver bullets” as along with advantages they have several disadvantages.

For example:

Middleware — overrides existing response flow which good option for the API and may be disadvantage for Web solutions. You may need to have 2 middlewares for different solution types.

Source code

All examples can be found implemented in the ready-to-go framework here

Building Multi-tenant Web API using dot net core and best practices (Tutorial)

Boris Zaikin — Sun, 11 Nov 2018 15:18:06 +0000

Business needs to grow in order to be successful and handle an increasing number of clients and partners, and if a company is not ready to respond to this load then there is a big chance that opportunities can be missed. This brings a topic of scalability into the game, as one of the main requirements that a company should address. As one of the possible ways to address this requirement is to build a multi-tenant solution. And as this topic gains more on importance, lots of options are available to achieve this, for example using Microsoft Elastic database (elastic tools). However in particular cases, like the case I faced on my project, not all of the product requirements could be satisfied with the already available options. This brought me to the idea of gathering my experience on this topic and presenting it below.

As we all are aware there are two main approached to tackle applications scaling — horizontal and vertical. Horizontal scaling will bring you the benefit of scaling on the fly and will imply dealing with multiple databases as each tenant has its own database/shard. Vertical approach to scaling presumes having one database that serves several tenants.

In my article I will address the approach of horizontal scaling with a step-by-step guide on how to build a multi-tenant web API application.

If you would like to refresh some aspects of multi-tenant architecture or what are pros and cons it

brings to the project, then I recommend visiting these resources:

Why Cloud Architecture Matters: The Multi-Instance Advantage over Multi-Tenant
Why Multi-Tenant Application Architecture Matters in 2017
Design patterns for multi-tenant SaaS applications and Azure SQL Database

Architecture

Let’s briefly take a look at the architecture first. The example below is designed based on N-tire

architecture and has the following layers:

Presentation layer or web api
Service layer that will accommodate all the business logic
Data access layer that is implemented using UnitOfWork and Repository patterns. As ORM in this example I used Entity Framework Core.

The key component of tenant separation is ContextFactory that contains logic to get the tenant id from HTTP header, retrieve a tenant database name using DataBaseManager and replace a database name in the connection string. As a result a database context (Entity Framework context) is created.

The diagram below demonstrate this architecture.

Implementation

As you can see architecture is not that complicated here, and skimming through it, I’d suggest to
focus on the steps to implement it.

1. Create ContextFactory

As I mentioned before ContextFactory is key component of whole architecture. It construct Entity Framework context (in current example DeviceApiContext) with specific to tenant database

/// <summary>
/// Entity Framework context service
/// (Switches the db context according to tenant id field)
/// </summary>
/// <seealso cref="IContextFactory" />
public class ContextFactory : IContextFactory
{
    private const string TenantIdFieldName = "tenantid";
    private const string DatabaseFieldKeyword = "Database";
    private readonly HttpContext httpContext;
    private readonly IOptions<ConnectionSettings> settings;
    private readonly IDataBaseManager dataBaseManager;
    public ContextFactory(
        IHttpContextAccessor httpContentAccessor,
        IOptions<ConnectionSettings> connectionSetting,
        IDataBaseManager dataBaseManager)
    {
        this.httpContext = httpContentAccessor.HttpContext;
        this.settings = connectionSetting;
        this.dataBaseManager = dataBaseManager;
    }

    public IDbContext DbContext
    {
        get
        {
        var dbOptionsBuidler = this.ChangeDatabaseNameInConnectionString();
        // Add new (changed) database name to db options
        var bbContextOptionsBuilder = new DbContextOptionsBuilder<TeleServiceApiContext>();
        bbContextOptionsBuilder.UseSqlServer(dbOptionsBuidler.ConnectionString);
        return new DevicesApiContext(bbContextOptionsBuilder.Options);
        }
    }

    // Gets tenant id from HTTP header
    private string TenantId
    {
        get
        {
        if (this.httpContext == null)
        {
            throw new ArgumentNullException(nameof(this.httpContext));
        }
        string tenantId = this.httpContext.Request.Headers[TenantIdFieldName].ToString();
        if (tenantId == null)
        {
            throw new ArgumentNullException(nameof(tenantId));
        }
        return tenantId;
        }
    }

    private SqlConnectionStringBuilder ChangeDatabaseNameInConnectionString()
    {
        var sqlConnectionBuilder = new SqlConnectionStringBuilder(this.settings.Value.DefaultConnection);
        string dataBaseName = this.dataBaseManager.GetDataBaseName(this.TenantId);
        if (dataBaseName == null)
        {
        throw new ArgumentNullException(nameof(dataBaseName));
        }
        // Remove old DataBase name from connection string AND add new one
        sqlConnectionBuilder.Remove(DatabaseFieldKeyword);
        sqlConnectionBuilder.Add(DatabaseFieldKeyword, dataBaseName);
        return sqlConnectionBuilder;
    }
}

Source code of ContextFactory available Here

2. Add the data base manager

Database manager orchestrates all tenants metadata such as tenant database name, activation

status of tenants (activated/deactivated) and bunch of other properties. To demonstrate a base principle I used dictionary in the current solution. Later on dictionary should be replaced with more appropriate solutions, like SQL or NoSQL database that contains tenants metadata. This idea is similar to shard map manager that is used in Microsoft Elastic Tools. Also tenant metadata may includes fields to store database name, options to activate/deactivate tenants, even tenant styles for front-end application based on CSS/SASS/LESS file an on and on

/// <summary>
/// Contains all tenants database mappings and options
/// </summary>
public class DataBaseManager : IDataBaseManager
{
    /// <summary>
    /// IMPORTANT NOTICE: The solution uses simple dictionary for demo purposes.
    /// The Best "Real-life" solutions would be creating 'RootDataBase' with 
    /// all Tenants Parameters/Options like: TenantName, DatabaseName, other configuration.
    /// </summary>
    private readonly Dictionary<Guid, string> tenantConfigurationDictionary = new Dictionary<Guid, string>
    {
        {
            Guid.Parse("b0ed668d-7ef2-4a23-a333-94ad278f45d7"), "DeviceDb"
        },
        {
            Guid.Parse("e7e73238-662f-4da2-b3a5-89f4abb87969"), "DeviceDb-ten2"
        }
    };
    /// <summary>
    /// Gets the name of the data base.
    /// </summary>
    /// <param name="tenantId">The tenant identifier.</param>
    /// <returns>db name</returns>
    public string GetDataBaseName(string tenantId)
    {
        var dataBaseName = this.tenantConfigurationDictionary[Guid.Parse(tenantId)];
        if (dataBaseName == null)
        {
            throw new ArgumentNullException(nameof(dataBaseName));
        }
        return dataBaseName;
    }
}

Source code of DataBaseManager available Here

3. Add Unit of Work class (contains commit to specific context method)

UnitOfWork solves two tasks. It commits all changes made by Entity Framework in entities and dispose specific context.

/// <summary>
/// The Entity Framework implementation of UnitOfWork
/// </summary>
public sealed class UnitOfWork : IUnitOfWork
{
    /// <summary>
    /// The DbContext
    /// </summary>
    private IDbContext dbContext;
    /// <summary>
    /// Initializes a new instance of the <see cref="UnitOfWork"/> class.
    /// </summary>
    /// <param name="context">The object context</param>
    public UnitOfWork(IDbContext context)
    {
         this.dbContext = context;
    }
    /// <summary>
    /// Saves all pending changes
    /// </summary>
    /// <returns>The number of objects in an Added, Modified, or Deleted state</returns>
    public int Commit()
    {
        // Save changes with the default options
        return this.dbContext.SaveChanges();
    }
    /// <inheritdoc/>
    public int Commit(IDbContext context)
    {
        // Change context
        this.dbContext = context;
        // Save changes with the default options
        return this.dbContext.SaveChanges();
    }
    /// <summary>
    /// Disposes the current object
    /// </summary>
    public void Dispose()
    {
        this.Dispose(true);
        GC.SuppressFinalize(obj: this);
    }
    /// <summary>
    /// Disposes all external resources.
    /// </summary>
    /// <param name="disposing">The dispose indicator.</param>
    private void Dispose(bool disposing)
    {
        if (disposing)
        {
            if (this.dbContext != null)
            {
                this.dbContext.Dispose();
                this.dbContext = null;
            }
        }
    }
}

Source code of UnitOfWork available Here

4. Add Generic Repository class.

Repository will make changes in EF entities and Unit of Work will commit changes to tenants database. Be aware that EF making changes in memory, using Tracking Mechanism.

/// <summary>
/// Generic repository, contains CRUD operation of EF entity
/// </summary>
/// <typeparam name="T">Entity type</typeparam>
public class Repository<T> : IRepository<T>
    where T : class
{
    /// <summary>
    /// Used to query and save instances of
    /// </summary>
    private readonly DbSet<T> dbSet;
    /// <summary>
    /// Gets the EF context.
    /// </summary>
    /// <value>
    /// The context.
    /// </value>
    public IDbContext Context { get; }
    /// <summary>
    /// Initializes a new instance of the <see cref="Repository{T}" /> class.
    /// </summary>
    /// <param name="contextFactory">The context service.</param>
    public Repository(IContextFactory contextFactory)
    {
        this.Context = contextFactory.DbContext;
        this.dbSet = this.Context.Set<T>();
    }
    /// <inheritdoc />
    public void Add(T entity)
    {
        return this.dbSet.Add(entity);
    }
    /// <inheritdoc />
    public T Get<TKey>(TKey id)
    {
        return this.dbSet.Find(id);
    }
    /// <inheritdoc />
    public async Task<T> GetAsync<TKey>(TKey id)
    {
        return await this.dbSet.FindAsync(id);
    }
    /// <inheritdoc />
    public T Get(params object[] keyValues)
    {
        return this.dbSet.Find(keyValues);
    }
    /// <inheritdoc />
    public IQueryable<T> FindBy(Expression<Func<T, bool>> predicate)
    {
        return this.dbSet.Where(predicate);
    }
    /// <inheritdoc />
    public IQueryable<T> FindBy(Expression<Func<T, bool>> predicate, string include)
    {
        return this.FindBy(predicate).Include(include);
    }
    /// <inheritdoc />
    public IQueryable<T> GetAll()
    {
        return this.dbSet;
    }
    /// <inheritdoc />
    public IQueryable<T> GetAll(string include)
    {
        return this.dbSet.Include(include);
    }
    /// <inheritdoc />
    public bool Exists(Expression<Func<T, bool>> predicate)
    {
        return this.dbSet.Any(predicate);
    }
    /// <inheritdoc />
    public void Delete(T entity)
    {
        return this.dbSet.Remove(entity);
    }
    /// <inheritdoc />
    public void Update(T entity)
    {
        return this.dbSet.Update(entity);
    }
}

Source code of Repository available Here

5. Add tenant header operation filter.

TenantHeaderOperationFilter class will add tenant id field to all API calls (as HTTP header). In solutions which uses OIDS services e.g IdentityServer or auth0.com tenant can be injected to JWT token.

/// <summary>
/// Adds Tenant Id field to API endpoints
/// </summary>
/// <seealso cref="Swashbuckle.AspNetCore.SwaggerGen.IOperationFilter" />
public class TenantHeaderOperationFilter : IOperationFilter
{
    /// <summary>
    /// Applies the specified operation.
    /// </summary>
    /// <param name="operation">The operation.</param>
    /// <param name="context">The context.</param>
    public void Apply(Operation operation, OperationFilterContext context)
    {
        if (operation.Parameters == null)
        {
            operation.Parameters = new List<IParameter>();
        }
        operation.Parameters.Add(new NonBodyParameter
        {
            Name = "tenantid",
            In = "header",
            Description = "tenantid",
            Required = true,
            Type = "string",
        });
    }
}

This is how API will look like after filter applied.

Current example of service class (DeviceService.cs) contains functions to retrieving device by id and add new device for specific tenants. Source Code of service layer available Here

Conclusion

In this article I was explained how to build “ready to go” Multi tenant solution and given some suggestions how it can be used in your product/business. As I mentioned before this solution is ready to go so it can be used as “Boilerplate” project or partially.

Source code

The project’s source code available in Git repository here