DEV Community: Ivan Cvitkovic

Server Monitoring with Grafana and Prometheus

Ivan Cvitkovic — Fri, 05 Jan 2024 15:06:36 +0000

Server monitoring is crucial for maintaining the health and performance of your systems. In this blog post, we'll walk through a basic setup using Grafana and Prometheus to monitor your servers. Before diving into the configuration details, let's briefly outline the components we'll be using in this setup.

What is Grafana?

Grafana is an open-source platform for monitoring and observability. It allows you to create, explore, and share interactive dashboards, enabling you to visualize and understand your metrics, logs, and other data sources easily.

What is Prometheus?

Prometheus is an open-source monitoring and alerting toolkit designed for reliability and scalability. It collects and stores time-series data, making it an excellent choice for monitoring systems and generating alerts based on predefined conditions.

What is Node Exporter?

Node Exporter is a Prometheus exporter for hardware and OS metrics. It runs on the servers you want to monitor and collects various system-level metrics, such as CPU usage, memory usage, disk activity, and network statistics. Prometheus scrapes these metrics from Node Exporter, providing a centralized location for monitoring your infrastructure.

Requirements

Before we begin, ensure you have Docker installed on your system. Once Docker is set up, you can use the provided docker-compose.yml and prometheus.yml files to launch the monitoring stack.

docker-compose.yml

version: '3.8'

networks:
  monitoring:
    driver: bridge

volumes:
  prometheus_data: {}
  grafana_storage: {}

services:
  node-exporter:
    image: prom/node-exporter:latest
    container_name: node-exporter
    restart: unless-stopped
    expose:
      - 9100
    networks:
      - monitoring

  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
    expose:
      - 9090
    networks:
      - monitoring

  grafana:
    image: grafana/grafana:9.5.15-ubuntu
    container_name: grafana
    restart: unless-stopped
    ports:
     - '3000:3000'
    volumes:
      - 'grafana_storage:/var/lib/grafana'
    networks:
      - monitoring

prometheus.yml

global:
  scrape_interval: 1m

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 1m
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node'
    static_configs:
      - targets: ['node-exporter:9100']

This file serves as the configuration for Prometheus, specifying the intervals at which it scrapes metrics and targets it monitors. In our case, it targets the Node Exporter on port 9100 since we are running that container in our Docker Compose setup.

Setting Up Grafana and Prometheus

Create a Directory and Copy YAML Files:

mkdir server-monitoring
cd server-monitoring

Launch the Stack: Run the following command to start the monitoring services.

docker-compose up -d

This command will pull the necessary Docker images and launch the Grafana, Prometheus and Node Exporter containers.

Access Grafana Dashboard: Open your web browser and navigate to http://localhost:3000 (or IP address of your virtual machine). Log in using the default credentials (username: admin, password: admin).

After initial login you will be prompted to change the password.

Configure Prometheus as a Data Source:
In Grafana, add Prometheus as a data source by specifying the URL http://prometheus:9090.
Import Node Exporter Full Dashboard:
Grafana provides a rich collection of dashboards that can be imported to visualize various metrics. To import the Node Exporter Full dashboard, follow these steps:

From the left pane menu, select "Dashboards."
On the right side, click "New" and then select "Import" from the dropdown menu.
In the "Grafana.com Dashboard" section, enter the dashboard ID 1860 and click "Load."
Configure the Prometheus data source (if not configured already) by selecting it from the drop-down menu.
Finally, click "Import" to add the Node Exporter Full dashboard to your Grafana instance.

This dashboard (ID 1860) is specifically designed for Node Exporter metrics and provides a comprehensive view of your server's performance.

Conclusion

With Grafana and Prometheus, you now have a basic yet powerful server monitoring setup. Explore the imported Node Exporter Full dashboard and other Grafana features to gain insights into your system's performance. Feel free to customize the setup and dashboards to fit your specific monitoring needs.

Happy monitoring! 🚀

Azure Blob Storage as Terraform backend

Ivan Cvitkovic — Sun, 19 Feb 2023 17:00:17 +0000

Managing Infrastructure as Code can be challenging, especially when working within a team. Terraform is a powerful tool for managing infrastructure resources and we briefly described it in one of the previous blog posts, but it can be tricky to keep track of the current state of your infrastructure when working in a team environment.

This is where Terraform state comes in. Terraform state is a snapshot of your infrastructure that is stored as a file on your local machine. This file contains information about the resources you've created, their dependencies, and their current configuration.

The Terraform state file is essential for managing your infrastructure, as it allows Terraform to determine the changes that need to be applied to your resources. Without a proper state file, Terraform wouldn't be able to properly manage your infrastructure resources.

In essence it's just a JSON file which is kind of like a map that tells Terraform what it has already built, and what it still needs to build. It's crucial to keep the state file safe and up-to-date, because if Terraform doesn't know what it has already built, it might accidentally create duplicate resources or overwrite existing ones, which could cause all kinds of problems.

You can store the state file locally, but there is an issue with that approach because it's not easily shareable. If you're working within a team of engineers, it's important for everyone to have access to the same state file. But if it's stored locally, it can be difficult for others to get access to it. That's why we usually store state file remotely on services like AWS S3, HashiCorp Consul or Azure Blob Storage.

In this post we will demonstrate how to set up an Azure Blob Storage backend for your Terraform state file. For that we will need to create a resource group and storage account. Of course, you will need an Azure subscription. If you don't have one already, you can create a free account. We can create these resources via Azure portal, but since we are talking about Infrastructure as Code, let's use Terraform to create them as well. In your code editor create main.tf file with the following code:



terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.44.1"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "random_string" "resource_code" {
  length  = 5
  special = false
  upper   = false
}

resource "azurerm_resource_group" "tfstate" {
  name     = "tfstate"
  location = "West Europe"
}

resource "azurerm_storage_account" "tfstate" {
  name                     = "tfstate${random_string.resource_code.result}"
  resource_group_name      = azurerm_resource_group.tfstate.name
  location                 = azurerm_resource_group.tfstate.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  tags = {
    environment = "demo"
  }
}

resource "azurerm_storage_container" "tfstate" {
  name                  = "tfstate"
  storage_account_name  = azurerm_storage_account.tfstate.name
  container_access_type = "private"
}

Make sure you are logged in your Azure account via Azure CLI. Now when we have Terraform configuration we run terraform init and after that terraform apply to create those resources. To verify if resources have been provisioned go to Azure portal and navigate to the Resource groups section where you should see tfstate resource group with storage account tfstate followed by the 5 characters long random string.

Let's further explore the storage account containers, it should be empty without blobs. Why is that so?

Currently, our state is stored locally inside terraform.tfstate file and it keeps track of our resources which leads us to a chicken-egg problem when it comes to state management. So what is the solution?

Let's go and create backend.tf file with the following code:



terraform {
  backend "azurerm" {
    resource_group_name  = "tfstate"
    storage_account_name = "tfstate<RANDOM-STRING>"
    container_name       = "tfstate"
    key                  = "terraform.tfstate"
  }
}

Now run terraform init again, and if you are using the latest Terraform version you should receive a prompt similar to:



Do you want to copy existing state to the new backend?
 Pre-existing state was found while migrating the previous "local" backend to the newly configured "azurerm" backend. 
No existing state was found in the newly configured "azurerm" backend. 
Do you want to copy this state to the new "azurerm" backend? Enter "yes" to copy and "no" to start with an empty state.

Type yes and check tfstate container in Azure portal. Your Terraform state is now successfully stored remotely in Azure Blob Storage.

If you did not receive this prompt you can import local state manually using the command terraform state push terraform.tfstate

To verify if everything is setup correctly delete the local instance of terraform.tfstate file and run command terraform state list. You should receive output like this one:



azurerm_resource_group.tfstate
azurerm_storage_account.tfstate
azurerm_storage_container.tfstate
random_string.resource_code

Let's cover another topic before conclusion. One important aspect of Terraform state management is state locking. State locking prevents multiple Terraform instances from modifying the state file at the same time, which can cause issues and inconsistencies in your infrastructure.

So, how do we implement state locking when using Azure as a backend for our Terraform state file? The good news is that Azure Blob Storage supports state locking for Terraform using native capabilities. Azure Storage blobs are automatically locked before any operation that writes state. This pattern prevents concurrent state operations, which can cause corruption.

In other words, we don't need any additional configuration, this comes out of the box when using Azure Blob Storage for Terraform backend configuration.

Building multi-architecture Docker images

Ivan Cvitkovic — Tue, 23 Aug 2022 10:48:04 +0000

In the last few years, the need for multi-architectural container images has grown significantly. Let's say you develop on your local Linux or Windows machine with an amd64 processor and want to publish your work to AWS machines with a Graviton2 processor, or simply want to share your work with colleagues who use Macbooks with an M1 chip, you need to ensure that your image works on both architectures. This process is significantly facilitated by the advent of the Docker Buildx tool.

But what is Buildx actually? According to the official documentation Docker Buildx is a CLI plugin that extends the docker command with the full support of the features provided by Moby BuildKit builder toolkit. It provides the same user experience as docker build with many new features like creating scoped builder instances and building against multiple nodes concurrently. Buildx also supports new features that are not yet available for regular docker build like building manifest lists, distributed caching, and exporting build results to OCI image tarballs.

In our demo, we will show how to setup buildx on a local machine and build a simple Node.js application. You can find the complete source code on this GitHub repository.

Creating Node.js application

In the demo application, we created a web server using Node.js. Node.js provides extremely simple HTTP APIs so the example is very easy to understand even for non-javascript developers.

Basically, we define the port and then invoke the createServer() function on http module and create a response with a status code of 200 (OK), set a header and print a message on which architecture the program is running. We obtained the architecture of the CPU through the arch property of the built-in process variable. At the end we simply start a server listening for connections.

const http = require("http");

const port = 3000;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader("Content-Type", "text/plain");
  res.end(`Hello from ${process.arch} architecture!`);
});

server.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

If you want to test the app locally open the terminal in the working directory and run node server.js command.

In order to package the application in the form of a container, we have to write a Dockerfile. The first thing we need to do is define from what image we want to build from. Here we will use the version 16.17.0-alpine of the official node image that is available on the Docker Hub. Right after the base image we will create a directory to hold the application code inside the image.

FROM node:16.17.0-alpine
WORKDIR /usr/src/app

To put the source code of our application into a Docker image, we'll use a simple copy command that will store the application code in the working directory.

COPY . .

Application is listening on port 3000 so we need to expose it and then finally start the server.

EXPOSE 3000
CMD ["node", "server.js"]

Setup Buildx and create the image

The easiest way to setup buildx is by using Docker Desktop, because the tool is already included in the application. Docker Desktop is available for Windows, Linux and macOS so you can use it on any platform of your choice.

If you don't want to use Docker Desktop you can also download the latest binary from the releases page on GitHub, rename the binary to docker-buildx (docker-buildx.exe for Windows) and copy it to the destination matching your OS. For Linux and macOS that is $HOME/.docker/cli-plugins, for Windows that is %USERPROFILE%\.docker\cli-plugins.

In the code below you can see the setup for macOS:

ARCH=amd64 # change to 'arm64' if you have M1 chip
VERSION=v0.8.2
curl -LO https://github.com/docker/buildx/releases/download/${VERSION}/buildx-${VERSION}.darwin-${ARCH}
mkdir -p ~/.docker/cli-plugins
mv buildx-${VERSION}.darwin-${ARCH} ~/.docker/cli-plugins/docker-buildx
chmod +x ~/.docker/cli-plugins/docker-buildx
docker buildx version # verify installation

After installing buildx we need to create a new builder instace. Builder instances are isolated environments where builds can be invoked.

docker buildx create --name builder

When new builder instance is created we need to switch to it from the default one:

docker buildx use builder

Now let's see more informations about our builder instance. We will also pass --bootstrap option to ensure that the builder is running before inspecting it.

docker buildx inspect --bootstrap

Once we have made sure which platforms our builder instance supports, we can start creating the container image. Buildx is very similar to the docker build command and it takes the same arguments, of which we will primarily focus on --platform that sets target platform for build. In the code below we will sign in to Docker account, build the image and push it to Docker Hub.

docker login # prompts for username and password

docker buildx build \
 --platform linux/amd64,linux/arm64,linux/arm/v7 \
 -t cvitaa11/multi-arch:demo \
 --push \
 .

When the command completes we can go to Docker Hub and see our image with all the supported architectures.

It's time to test how the image works on different machines. First we will run it on Windows (Intel Core i5 CPU which falls under amd64 architecture) with the command:

docker run -p 3000:3000 cvitaa11/multi-arch:demo

Let's navigate to the web browser to localhost:3000 and chech the response.

Now let's switch to Macbook Pro with M1 chip and run the same command.

Open the web browser and again go to the localhost:3000:

We see that our container image runs successfully on both processor architectures, which was our primary goal.

Introduction to Terraform

Ivan Cvitkovic — Tue, 15 Feb 2022 18:57:22 +0000

Infrastructure as Code

Almost every software engineer has experienced that provisioning servers, operating systems, storage, and other infrastructure components is a demanding and time-consuming process. Provisioning is usually done manually, and when we take into account that in addition to the development environment, the staging and production workloads should also be deployed, we come to a large amount of work. In such a large task, human error can easily occur, which means that only a small configuration error can cause problems and disable the proper operation of the application.

To avoid potential outages we come to the concept of Infrastructure as Code. IaC is the managing and provisioning of infrastructure through code instead of through manual processes. Using the infrastructure as code allows configuration files on virtual machines, disks, network and other components to be stored together with the application code within source control management system. Storing them within the SCM allows you to version and track changes over time. This approach helps you to avoid undocumented, ad-hoc configuration changes.

Deploying your infrastructure as code also means that you can divide your infrastructure into modular components that can then be combined in different ways through automation. Automating infrastructure provisioning means that there is no need for manual interventions and thus eliminates the human error factor. There are several IaC tools and one of the most popular is Terraform.

Terraform basic concepts

Terraform is HashiCorp's infrastructure as code tool. It lets you define resources and infrastructure in human-readable, declarative configuration files, and manages your infrastructure's lifecycle. It lets you define both cloud and on-premise resources trough configuration language called HCL, which looks like simplified JSON (JavaScript Object Notation). Terraform can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features.

But how does it all actually work under the hood? Terraform creates resources on cloud platforms and on-premises based on their application programming interfaces (APIs), while communication with APIs takes place through providers. Providers are plugins that interact with various platforms and manage their resources, serving as a logical abstraction of an upstream API.

Together with the community, Terraform has written more than 1800 providers, including those for major cloud providers like Azure, AWS and Google Cloud Platform. There are also providers for Docker, Kubernetes, Helm and more. You can find the full list at the Terraform Registry.

As in everything else, the best way to learn Terraform is through practice, but keep in mind that creating resources on cloud providers can cost you money. If you no longer need certain resources after exercise, simply delete them and clean up your environment. Also, for practice you can use various free tiers on platforms like Microsoft Azure and Amazon Web Services or some provider to create resources locally on your development machine. There is a lot of beginner friendly examples, like this one, on HashiCorp Learn website which can help you get started.

Demo: Terraform Docker provider

The complete source code for this demo is on GitHub, and you can find the repository here.

We'll start with the main.tf file. Inside that file is placed terraform {} block which contains all the settings, including the required providers Terraform will use to provision your infrastructure components. Each provider consists of source attribute which defines where is provider actually located. By default it will be installed from the Terraform Registry, but optionally you can pass hostname parameter. In this example configuration, the docker provider's source is defined as kreuzwerker/docker.

The version attribute is optional, but it's highly recommended to use it so that Terraform does not install a version of the provider that does not work with your configuration. Each module should at least declare the minimum provider version it is known to work with, using the >= version constraint syntax. If the version is not passed, Terraform will download and install the latest one. Although this is not the case here, keep in mind that you can define multiple different providers in your configuration block and use them together.

terraform {
  required_providers {
    docker = {
      source = "kreuzwerker/docker"
    version = ">= 2.13.0" }
  }
}

Under the provider block we don't have much stuff, it just configures the specified provider, in this case docker.

The resource section is the most important part of every Terraform project. Each resource block describes one or more infrastructure components, which can be a physical or virtual object. Before the configuration block there are two parameters that define the resource type and its name. In this case we declare the Docker image and Docker container where both resources are named nginx.

resource "docker_image" "nginx" {
  name         = "nginx:latest"
  keep_locally = false
}

resource "docker_container" "nginx" {
  image = docker_image.nginx.latest
  name  = var.container_name
  ports {
    internal = 80
    external = 8080
  }
}

In the docker_container resource, under image parameter we are referencing previously declared docker_image. In the ports section we publish local port 80, which is the default for Nginx, to port 8080 on our local machine. For the container name, instead of some hard-coded string, we decided to use value assigned to a variable container_name.

You can name your configuration files however you want because Terraform loads all files in the current directory ending in .tf, but we choose to follow the naming convention and name the file variables.tf. Variables file is quite simple, each variable is declared by keyword and name. Inside the configuration block we define variable type, and can optionally pass default value or sensitive flag which limits Terraform UI output when the variable is used in configuration.

variable "container_name" {
  type = string
}

When variables are declared in your configuration, they can be set in a number of ways. They can be passed individually with the -var command line option, as environment variables in TF_VAR_ format or inside variable definitions files (.tfvars). We used the latter option to set a name for our Docker container:

container_name = "nginx-example"

Now we are ready to deploy our infrastructure, and to do that we need to initialize a working directory containing Terraform configuration files with terraform init command. This command performs several different initialization steps in order to prepare the current working directory for use with Terraform which include downloading and installating the provider.

After the initialization process is complete we can create a new execution plan and then apply it. The easiest way to achieve this is by using terraform apply command. Execution plan is displayed in the command line and user is prompted to apply that plan. You should get similar output:

$ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # docker_container.nginx will be created
  + resource "docker_container" "nginx" {
      + attach           = false
      + bridge           = (known after apply)
      + command          = (known after apply)
      + container_logs   = (known after apply)
      + entrypoint       = (known after apply)
      + env              = (known after apply)
      + exit_code        = (known after apply)
      + gateway          = (known after apply)
      + hostname         = (known after apply)
      + id               = (known after apply)
      + image            = (known after apply)
      + init             = (known after apply)
      + ip_address       = (known after apply)
      + ip_prefix_length = (known after apply)
      + ipc_mode         = (known after apply)
      + log_driver       = (known after apply)
      + logs             = false
      + must_run         = true
      + name             = "nginx-example"
      + network_data     = (known after apply)
      + read_only        = false
      + remove_volumes   = true
      + restart          = "no"
      + rm               = false
      + security_opts    = (known after apply)
      + shm_size         = (known after apply)
      + start            = true
      + stdin_open       = false
      + tty              = false

      + healthcheck {
          + interval     = (known after apply)
          + retries      = (known after apply)
          + start_period = (known after apply)
          + test         = (known after apply)
          + timeout      = (known after apply)
        }


Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

docker_image.nginx: Creating...
docker_image.nginx: Still creating... [10s elapsed]
docker_image.nginx: Still creating... [20s elapsed]
docker_image.nginx: Still creating... [30s elapsed]
docker_image.nginx: Still creating... [40s elapsed]
docker_image.nginx: Still creating... [50s elapsed]
docker_image.nginx: Still creating... [1m0s elapsed]
docker_image.nginx: Still creating... [1m10s elapsed]
docker_image.nginx: Still creating... [1m20s elapsed]
docker_image.nginx: Creation complete after 1m23s [id=sha256:c316d5a335a5cf324b0dc83b3da82d7608724769f6454f6d9a621f3ec2534a5anginx:latest]
docker_container.nginx: Creating...
docker_container.nginx: Creation complete after 1s [id=0e3fda53befd697a51a3b15a615c236c62f65469f4a02450c1899289f998f128]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Now, navigate to your web browser and check the localhost:8080, you should see Nginx web server up and running.

When you are done with the exercise, resources can be deleted with a simple command terraform destroy.

Packaging Java apps with Maven and GitHub Actions

Ivan Cvitkovic — Sat, 20 Nov 2021 15:27:39 +0000

This post shows how to create workflows that package Java application with Maven and then store it as an artifact or publish to GitHub Packages.

In the previous post we described GitHub Actions and how they work, so if you need a quick reminder on the jobs, steps and syntax check it out.

Development environment

On the following link you can find repository with the source code. It's a simple Spring Boot application that returns students from the database. The application was bootstrapped using Spring Initializr. For the dependencies we added Spring Web which is used for building web applications using Spring MVC. This package also uses Apache Tomcat as the default embedded container. We also used Spring Data JDBC for persisting data in SQL with plain JDBC and PostgreSQL Driver that allows Java programs to connect to a Postgres database using standard, database independent Java code.

For local development, we can easily start Postgres instance with Docker by using the following command:



docker run -d --name postgresDB -p <port>:5432 -e POSTGRES_PASSWORD=<YourPassword>-v /postgresdata:/var/lib/postgresql/data postgres:latest

Spring Boot follows a layered architecture in which each layer communicates with the layer directly below or above it. We followed that practice and implemented Controller, Service and Repository inside our application and demonstrated dependency injection principles. Source code also contains StudentConfig class which simply inserts student in database.

After successfully setting up development environment and writing some code we decided to push our work into the source control management system, in this case GitHub. Now we need to build the code and publish it as a Maven package. This process can be done manually, but we would like to do it automatically when changes are merged into the main branch. In this way we will avoid manual tasks when publishing a new version.

Like most other things, this problem can be solved in several ways, and we will use two different approaches. Firstly, we will publish our package as an build artifact and make it available for download, while in the second approach we will publish package to the GitHub Packages Maven repository.

Storing workflow data as artifacts

In the main.yaml file first couple of lines tell us which events will start the workflow. Except push and pull request on the main branch we also added workflow_dispatch tag to enable manual workflow run.

Under the jobs section we defined build job that will be executed on the Ubuntu runner. First two steps checkout main branch from GitHub and set up JDK (Java Development Kit).

Next up, we build the project and set up a cache for Maven:



- name: Build Maven project
  run: |
    mvn -B package --file pom.xml -Dmaven.test.skip
    mkdir staging && cp target/*.jar staging

- name: Set up a cache for Maven
  uses: actions/cache@v2
  with:
    path: ~/.m2
    key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
    restore-keys: ${{ runner.os }}-m2

Once the build is completed, staging directory will contain produced .jar file. Each job in a workflow runs in a fresh virtual environment which means that once the build job is done we cannot access that environment and our .jar file is gone. That's where our last step comes in place which uploads artifacts from your workflow allowing you to share data between jobs and store data once a workflow is complete.



- name: Persist workflow data as artifacts
  uses: actions/upload-artifact@v2
  with:
    name: github-actions-artifact
    path: staging

By default, the artifacts generated by workflows are retained for 90 days before they are automatically deleted. You can adjust the retention period, depending on the type of repository. When you customize the retention period, it only applies to new artifacts and does not retroactively apply to existing objects.

Artifacts can be found under the Actions tab when you click on the desired workflow run.

Publishing to the GitHub Packages

You can configure Apache Maven to publish packages to GitHub Packages and to use packages stored on GitHub Packages as dependencies in a Java project.

Beside Maven, GitHub Packages offers different package registries for commonly used package managers like npm, NuGet, Gradle and RubyGems. It's also possible to store Docker and other OCI images. With all these features you can create end-to-end DevOps solutions and centralize your software development on GitHub.

In maven-publish.yaml file you can find workflow details for publishing package to the GitHub Packages. Just like in the previous solution we provide name and events that will trigger workflow run. Next, under the jobs section we selected Ubuntu runner as environment for the publish job and defined permissions to read the content and write packages.



name: Publish package to GitHub Packages
on:
  push:
    branches: [main]
jobs:
  publish:
    runs-on: ubuntu-latest 
    permissions: 
      contents: read
      packages: write

In the following steps we checkout the main branch and set up JDK with java-version parameter. The last step is publishing the package and it needs personal access token for the authentication. PAT is a sensitive information and we don't want to store it as a plain text, so we defined the secret on the repository level and accessed it as an environment variable. For simplicity we skipped the tests during deployment.



- name: Publish package
  run: mvn --batch-mode deploy -Dmaven.test.skip
  env:
    GITHUB_TOKEN: ${{ secrets.TOKEN }}

Before running the workflow we also need one configuration change in the application source code. Inside pom.xml file we need to pass information about package distribution management.



<project ...>
  ...
  <distributionManagement>
    <repository>
      <id>github</id>
      <name>GitHub Packages</name>
      <url>https://maven.pkg.github.com/cvitaa11/java-spring-demo</url>
    </repository>
  </distributionManagement>
</project>

After pushing changes to the main branch workflow starts automatically and we can follow the log output under the Actions tab on the repository page. When all steps are completed we can see Maven package, ready for use, in our code repository under the Packages section.

With these two approaches, we have successfully solved the problem of automatic publish of Java applications as Maven packages and demonstrated use of GitHub Actions and GitHub Packages to create complete end-to-end development solution in one place.

Continuous integration with GitHub Actions

Ivan Cvitkovic — Tue, 12 Oct 2021 10:20:11 +0000

Continuous integration is cheap, but not integrating continuously can be very expensive. Nowdays, it's hard to imagine software development lifecycle without automated solutions that take care of all those repetitive tasks. There is a ton of different technologies that can solve our problems and in this post we will focus on GitHub Actions.

Understanding GitHub Actions

GitHub Actions are event-driven workflows that come right in your code repository. By saying event-driven, it means that you can execute series of commands when specific event occurs. For example, every time someone creates a pull request for a repository, you can automatically run a command that executes a software testing script. Aside from event trigger, procedure can also be scheduled.

Procedures are defined in YAML file called workflow. Workflow consists of one or more jobs that can be executed simultaneously or chronologically. Each job then uses steps to control the order in which actions are run. These actions are the commands that automate your software testing, building etc.

Event is the starting point in this cycle. It's an activity that triggers workflows and it can originate from GitHub when someone creates issue, pull request or merge changes to the specific branch. Also, it's possible to use repository dispatch webhook and trigger workflow when an external event occurs.

Job is collection of steps that are executed on the same runner. By default all jobs in workflow are running in parallel, but there are some situations where job depends on the result of the previous one so we can configure them to execute sequentially. For some simple tasks we can create workflows with only one job, but for more complex scenarios this is not recommended approach. In that case multi-job workflows are way to go.

Each step in a job represents an individual task that can be shell command or an action. Since every job is executed on the same runner it allows steps to share data with each other. For example first step of a job can build container image and then in the second step that image can be pushed to a container registry. Data can also be shared between jobs by storing workflow data as artifacts. Artifacts allow you to persist data after a job has completed, and share that data with another job in the same workflow. To use the data in another job you just need to download artifacts. Some of the common artifacts are log outputs, test results, binary or compressed files and code coverage results.

Per official documentation, actions are standalone commands that are combined into steps to create a job. Actions are the smallest portable building block of a workflow and you can create your own actions, or use actions created by the GitHub community. To use an action in a workflow, you must include it as a step.

Another important concept we have to mention are runners. In essence a runner is a server with GitHub Actions runner application installed on it. A runner listens for available jobs, runs one job at a time, and reports the progress, logs, and results back to GitHub. Hosted runners, provided by GitHub, are based on Ubuntu, Windows, and macOS, and each job in a workflow runs in a fresh virtual environment. Trough workflow you can install additional tools and binaries on a runner. If for some reason you need different operating system or require specific configuration you can use self-hosted runners where you have full control of the entire environment. However, GitHub does not recommend self-hosted runners for public repositories.
Windows and Ubuntu runners are hosted in Azure and subsequently have the same IP address ranges as the Azure datacenters. macOS runners are hosted in GitHub's own macOS cloud. Each Windows or Ubuntu runner has 2-core CPU, 7GB of RAM memory and 14 GB of SSD disk space. macOS runners have 3-core CPU, 14GB of RAM memory and 14GB of SSD disk space.
In terms of pricing, on a free plan you have 2000 automation minutes per month which is enough for learning and some smaller projects. You can find more about pricing plans on the following link.

Demo

In the following demo we have .NET REST API application. Our task is to run unit tests against the solution and if they succeed package the application as the container image and push it to the container registry. Of course, we want to do all of that in an automated way when push or pull request on the master branch occurs.

Source code can be found here. As you can see, workflow is placed inside .github/workflows directory as YAML file.

First couple of lines are used to define workflow name and specify events that will trigger the workflow run.

name: CI master

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

Next, workflow_dispatch section allows us to run the workflow manually using Actions tab on GitHub or trough CLI.

The jobs section is the main part which defines tasks and controls their order of execution.

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Test the solution 
        run: dotnet test

First attribute is the name of the job followed by the runs-on parameter which specifies the infrastructure environment, in this case ubuntu-latest. Under the steps segment we define actual tasks. The dotnet test command is used to execute unit tests in a given solution. The dotnet test command builds the solution and runs a test host application for each test project in the solution. The test host executes tests in the given project using a test framework, for example: MSTest, NUnit, or xUnit, and reports the success or failure of each test. If all tests are successful, the test runner returns 0 as an exit code. Otherwise if any test fails, it returns 1 and the workflow will be stopped.

Obviously, if any test fails we don't want to build and package our application so the second job depends on the results of the first one.

  build:
    needs: test
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Build container image
        run: |
          docker build -f ./dotnet-ci/Dockerfile -t ${{secrets.registry}}/${{github.repository}}:${{ github.run_number }} .
      - name: Container registry login
        uses: docker/login-action@v1.10.0
        with:
          registry: ${{secrets.registry}}
          username: ${{secrets.username}}
          password: ${{secrets.password}}

      - name: Push image to container registry
        run: |
          docker push ${{secrets.registry}}/${{github.repository}}:${{ github.run_number }}

Attribute needs will make this job to wait for the test to finish and just if test succeeds, workflow will start the build job on a new runner.

Workflow needs access to some sensitive informations like login credentials and we definitely don't want to store it as plain text. That's where secrets come in place, they are definied on a repository level and can only be created by the repository owner. Secret value is not visible, they can only be modified or deleted. For container registry we will use GitHub Packages and authenticate with username and PAT (Personal Access Token) as password. This way we will have all the assets in one place, our code repository.

Each container image needs to have a tag, and we want that value to be unique. Thankfully, GitHub provides us with number of environment variables and we decided to tag image with run_number. That is a unique number for each run of a particular workflow in a repository. This number begins at 1 for the workflow's first run, and increments with each new run. This number does not change if you re-run the workflow run. Alternative options would be to tag image with commit SHA, run_id or use semantic versioning, but for simplicity we chose run_number.

Now we can take a look at GitHub web console under Actions tab.

As you can see, under Summary we have a nice overview of all the jobs in our workflow and and their interrelationship.

Click on the single job will give us list of all the steps of which the job is composed. We can also click on any step and get the entire log output which is really helpful when it comes to debugging.

Conclusion

Ever since Microsoft acquired GitHub they have been adding new features and functionalities that made GitHub not only great source control managment tool, but also very mature DevOps platform. GitHub Actions are very practical and easy to use which makes them a great choice for smaller projects and getting started with continuous integration. However, for projects at a large scale they are still not as powerful as Azure Pipelines or some other enterprise solutions from cloud providers.

Running Dapr on Kubernetes

Ivan Cvitkovic — Tue, 07 Sep 2021 10:53:46 +0000

The distributed application runtime, Dapr, is a portable, event-driven runtime that can run on the cloud or any edge infrastructure. It puts together the best practices for building microservice applications into components called building blocks.

Each building block is completely independent so you can use one, some, or all of them in your application. Building blocks are extensible, so you can also write your own.

Dapr supports a wide range of programming languages and frameworks such as .NET, Java, Node.js, Go and Python. That means you can write microservice apps using your favorite tools and deploy them literally anywhere.

Basically, building blocks are just HTTP or gRPC APIs that can be called from application code and use one or more Dapr components. They abstract some of the major challenges during development such as service-to-service communication, state managment, pub/sub, observability and more. Building blocks do not depend on underlying technology. This means if you need to implement, for example, pub/sub functionality you can use Apache Kafka, RabbitMQ, Redis Streams, Azure Service Bus or any other supported broker that interface with Dapr.

In this example we will show how to run Dapr on the Kubernetes cluster with two .NET applications. First one will send messages to Apache Kafka while the second one will read those messages and store them in Redis. Communication to Kafka and Redis will be realized using the Dapr Client, which means that we will not have any dependencies on NuGet packages like Confluent.Kafka or StackExchange.Redis.

Architecture diagram

Prerequisites

This demo requires you to have the following installed on your machine:

Kubernetes CLI kubectl
Kubernetes cluster, such as Minikube or Docker Desktop

Also, clone the repository and cd into the right directory:

git clone https://github.com/cvitaa11/dapr-demo
cd dapr-demo

Step 1 - Setup Dapr on your Kubernetes cluster

The first thing you need is an RBAC enabled Kubernetes cluster. This could be running on your machine using Minikube/Docker Desktop, or it could be a fully-fledged cluser in Azure using AKS or some other managed Kubernetes instance from different cloud vendor.

Once you have a cluster, follow the steps below to deploy Dapr to it. For more details, look here

$ dapr init -k
⌛  Making the jump to hyperspace...
ℹ️  Note: To install Dapr using Helm, see here: https://docs.dapr.io/getting-started/install-dapr-kubernetes/#install-with-helm-advanced

✅  Deploying the Dapr control plane to your cluster...
✅  Success! Dapr has been installed to namespace dapr-system. To verify, run `dapr status -k' in your terminal. To get started, go here: https://aka.ms/dapr-getting-started

The dapr CLI will exit as soon as the kubernetes deployments are created. Kubernetes deployments are asyncronous, so you will need to make sure that the dapr deployments are actually completed before continuing.

Step 2 - Setup Apache Kafka

The easiest way to setup Apache Kafka on your Kubernetes cluster is by using Helm package manager. To install Helm on your development machine follow this guide.
We will use Bitnamy Library for Kubernetes to launch Zookeper and Kafka message broker.

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release bitnami/kafka

Step 3 - Setup Redis

Just like Apache Kafka, easy way to spin up Redis on your Kubernetes cluster is by using Helm.

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install redis bitnami/redis

To verify the installation of Kafka and Redis run kubectl get all and you should see similiar output:

NAME                             READY   STATUS        RESTARTS   AGE
pod/my-release-kafka-0           1/1     Running       0          18m
pod/my-release-zookeeper-0       1/1     Running       0          18m
pod/redis-master-0               1/1     Running       1          11m
pod/redis-slave-0                1/1     Running       1          11m
pod/redis-slave-1                1/1     Running       1          11m

NAME                                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/kubernetes                      ClusterIP   10.96.0.1        <none>        443/TCP                      15d
service/my-release-kafka                ClusterIP   10.110.225.238   <none>        9092/TCP                     18m
service/my-release-kafka-headless       ClusterIP   None             <none>        9092/TCP,9093/TCP            18m
service/my-release-zookeeper            ClusterIP   10.99.95.252     <none>        2181/TCP,2888/TCP,3888/TCP   18m
service/my-release-zookeeper-headless   ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   18m
service/redis-headless                  ClusterIP   None             <none>        6379/TCP                     11m
service/redis-master                    ClusterIP   10.111.109.148   <none>        6379/TCP                     11m
service/redis-slave                     ClusterIP   10.111.66.85     <none>        6379/TCP                     11m

NAME                                    READY   AGE
statefulset.apps/my-release-kafka       1/1     18m
statefulset.apps/my-release-zookeeper   1/1     18m
statefulset.apps/redis-master           1/1     11m
statefulset.apps/redis-slave            2/2     11m

Step 4 - Create Dapr components in Kubernetes cluster

To deploy pub/sub and state store components make sure you are positioned in the right directory and then apply Dapr YAML manifests.

cd dapr-components
kubectl apply -f .\kafka.yaml
kubectl apply -f .\redis.yaml

Step 5 - Deploy .NET Core applications

Now when all prerequisites are ready we can deploy our apps. To deploy .NET Core publisher and consumer applications make sure you are positioned in the right directory and then apply Kubernetes manifests.

cd k8s
kubectl apply -f .\publisher.yaml
kubectl apply -f .\consumer.yaml

Each manifest contains Deployment object for the application and Service object for accessing the application through a browser.

Navigate to the localhost:8081/swagger and you will se our publisher app with a POST method on MessageController. This action sends a message to newMessage topic on Kafka pub/sub component. Communication between application and message broker is not performed directly. Dapr is running as a sidecar container inside publisher pod and handles the entire process of sending a message.

Our consumer application is running on localhost:9091 and is subscribed to newMessage topic on Kafka pub/sub component. When new message arrives it reads the content and trough the Dapr client saves it to Redis state store uneder the key message.

To test the entire process we can run Redis client pod and check if content is stored. First of all we will export password to REDIS_PASSWORD variable:

export REDIS_PASSWORD=$(kubectl get secret --namespace default redis -o jsonpath="{.data.redis-password}" | base64 --decode)

Then run the client with the following command:

kubectl run --namespace default redis-client --rm --tty -i --restart='Never' \
    --env REDIS_PASSWORD=$REDIS_PASSWORD \
   --image docker.io/bitnami/redis:6.0.12-debian-10-r3 -- bash

and connect using Redis CLI:

redis-cli -h redis-master -a $REDIS_PASSWORD

Now that you are connected to Redis you can use command HGETALL message that will return content of the message we sent to Kafka. With this we have confirmed that the whole process works.

If you want to find out more about Dapr, the best place to start is the official documentation.