DEV Community: Idan Elhalwani

Ostara Version 0.10.0 Has Been Released

Idan Elhalwani — Mon, 12 Jun 2023 08:48:00 +0000

Hello and welcome to the first major update of Ostara!

We've been working hard the last 3 weeks on our latest release and I'm happy to say it's finally here: we are super excited to release version 0.10.0 of Ostara and bring it one step closer out of Alpha. This release is full of new features and improvements to make your experience smoother than ever. Among the many awesome updates, the one we are most thrilled about is the newly introduced Metrics Notifications!

Before diving into the specifics, let's quickly brush up on Ostara.

A Quick Recap

If you're new around here or just need a quick memory jog, Ostara is your go-to solution for managing and monitoring Spring Boot Actuator-enabled microservices.

With a primary focus on simplicity and user-friendliness, Ostara provides you with real-time insights into the health and performance of your microservices, allowing you to monitor critical metrics such as CPU and memory usage.

For more information, see my previous article.

Metric Notifications

Managing microservices can get a tad complex, and that's where Ostara's new feature comes into play - Metric Notifications. Let's take a deep dive into this key update in version 0.10.0.

Metric Notifications in Ostara serves as your personalized alarm system, alerting you when something goes amiss with your metrics. This feature introduces a level of proactive monitoring that is set to redefine how you interact with your metrics data.

In the Metric Notifications interface, you can now create custom alert rules for your application metrics. Once these rules are in place, Ostara will monitor your instances for metric updates and evaluate them, notifying you if the value has crossed the threshold that was set.

You can define the metric you wish to monitor and set the trigger value. Ostara currently supports two types of rules:

Simple rule

Monitors a single metric and triggers when the metric crosses your defined trigger value.

Relative rule

Tracks the relative value between two metrics, triggering when this value crosses your defined trigger value. For instance, you can set a rule for free disk space on your instance, where the first metric is disk.free and the relative value is disk.total. You can set a trigger for when the relative value drops below 20%.

Now, let's talk about how Ostara evaluates these rules. Ostara provides three comparison operations:

Lower than '<': Triggers a notification if the metric value (or relative value) falls below the trigger value.
Greater than '>': Triggers a notification if the metric value (or relative value) exceeds the trigger value.
Between: Triggers a notification if the metric value (or relative value) falls within the set trigger values.

If this is all too confusing and you're just looking to play around with the new rules, we've also added Predefined Rules that will allow you to instantly create a variety of default rules to play around with;

For more information, visit Metric Notifications

All The Other Goodies in Version 0.10.0

We've managed to achieve quite a lot in this version, here are some of the other features you will encounter:

Application Health Notifications

This feature sends immediate alerts if your application's health status changes. It's designed to help you react quickly to potential issues, reducing manual monitoring.

Notification settings

With the addition of OS level notifications, we've also added the option to turn them off completely in Settings, or just their sound.

Global, App and Folder Dashboard

There are three new dashboards for you to explore - the Global Dashboard, the App Dashboard and the Folder Dashboard. These dashboards provide a concise overview of the status of your applications and folders.

Export/Import Configurations

For those who manage multiple systems, want a backup of your configuration, or want to share your configurations with your friends and colleagues, you can now export and import Ostara configurations.

Instance Shutdown

We've added support for the shutdown capability on Actuator, allowing you to shutdown your instances directly from Ostara;

And a lot more! You can find the full changelog at the end of this post.

Quick Start with Ostara

If you're ready to dive into Ostara 0.10.0, head over to our website, download the version for your OS, and follow the Quick Start guide. You'll be up and monitoring in no time!

Contributions

Ostara is in pre release so we would greatly appreciate any feedback, recommendations and/or requests as well as any contributions. We would also love to hear from other Spring Boot developers. Head to the repository to participate and keep an eye out for our upcoming Beta program!

Links

Changelog: https://docs.ostara.dev/getting-started/changelog

Ostara home: https://ostara.dev

Ostara repository: https://github.com/krud-dev/ostara

Ostara documentation: https://docs.ostara.dev/

Join us on Discord! https://discord.gg/8pUtGUYncD

Introducing Ostara

Idan Elhalwani — Tue, 16 May 2023 13:09:11 +0000

(A previous version of this article introduced Ostara under its previous name, Boost, the article has been amended to reflect this)

What is Ostara

Ostara is a modern tool for managing and monitoring actuator-enabled microservices that aims to make the process more user-friendly and straightforward.

One of our main goals with Ostara was to create a tool that just works out of the box, without needing anything besides a functioning Actuator API on the other end.

Ostara allows you to gain insights into the performance and health of your microservices by providing real-time monitoring of critical metrics such as CPU and memory usage.

One of the key benefits of Ostara is its simplicity. The tool is designed to be easy to use and requires minimal setup. All you need is a running instance of your microservice with Actuator enabled, and you can start monitoring and managing it with Ostara.

Why?

The first question to ask is why even use Ostara, when excellent tools such as Spring Boot Admin already exist?

As Spring Boot Admin users, we had to ask ourselves this question before embarking on the development journey of Ostara.

For all its greatness, one thing Spring Boot Admin that cannot be said about Spring Boot Admin is that it's plug and play or 'off-the-shelf'.

To setup Spring Boot Admin, we start by having to create Spring project for the server itself (In our case, which is true for many, also meant we have to setup a full CI/CD flow, complete with Helm charts and Docker images in order to create a full production flow)

If we needed additional features such as authentication, notifications and so on, we would need to add them directly to the project.

All of this, despite the infinite versatility it offers (because it is essentially a blank slate), raises the startup costs of Spring Boot Admin considerably and even more so when you remember that you have to continually maintain this project as with any other project within your code ecosystem, either due to security updates to unrelated dependencies you may have needed to add, or in order to update Spring to match your main project(s).

But aren't there ready-made images of Spring Boot Admin Server? A quick Google search will show a few (often outdated) pre-built ones that people in the community have created for different scenarios, including this official codecentric/spring-boot-admin Docker image which states:

This repository contains pre-build Docker Images containing basic and hence not production-ready builds of Spring Boot Admin.

Now onto the client, in order to register instances on Spring Boot Admin, you have to add the spring-boot-admin-starter-client dependency to your app and point it at the admin server. From personal production experience in mature projects, this isn't always a pleasant, straightforward experience.

If your project guards Actuator's endpoints with Spring Security, you also have to configure the client to send the server the credentials it will need when registering itself.

Alternatively, if you use Spring Cloud Discovery within your ecosystem you will be able to use that in order to discover instances, dependency free, but not hassle free, as you will again have to add this to your server's project and configure on both ends.

The second part is performance. Spring Boot Admin lacks in a lot of areas when it comes to performance. The simplest example is the Thread Dump page, where if you stick around long enough without refreshing, it will eventually crash your tab.

With Ostara, we aimed for a solution that just works performantly with your existing setups, no matter their size
or scale and without requiring any prior modifications in your code or environment with the only prerequisite being the presence of Spring Actuator.

Feature Overview

A robust and user-friendly UI for an ever-growing list of Actuator endpoints (Full list)
Health monitoring of your applications and their individual instances
Neatly organize your applications and instances into different folders, with color coding and unique icons
Modify the loggers of your applications and individual instances on the fly
Evict caches from applications and individual instances on the fly
View cache statistics
View HTTP request statistics
Bean dependency visualization
Spring Integration Graph visualization
Application Security built in, interact with your secured Actuator instances with no extra code

The Future

Some of our planned features for future releases:

Service discovery support
Notifications
Custom themes
A plugin ecosystem for custom Actuator endpoints
Support for non-official actuator endpoints, such as Togglz
Native support for non-Spring Actuator facades such as gohealth, express-actuator and pyctuator

Setting up Ostara

Setting up Ostara is super simple, simply head over and download the latest release for your OS and follow the Quick Start to create your first instance and begin monitoring with Ostara

Contributions

Spring Boot + Electron, a case study

Idan Elhalwani — Mon, 15 May 2023 11:45:08 +0000

Background

We recently finished writing our desktop app for Spring Actuator, Ostara. Initially, we decided to rely on Electron's main process and write our "backend" in Node. We quickly hit roadblock after roadblock, until a decision was made to ditch the backend entirely, rewrite it in Spring Boot and Kotlin in the JVM ecosystem, and have the renderer communicate with it via REST.

We did this for a multitude of reasons:

First and foremost, we wanted the versatility of being able to run the project on and off Electron, and possibly with a remote backend. Our initial Node implementation relied heavily on IPC between the main and renderer and we knew that, for the most part, had to go.
It's a mature and robust ecosystem, and especially one where we knew we would be able to make rapid progress
We would be able to leverage our own JVM libraries, namely ShapeShift and a yet-unreleased framework we use for CRUD operations with traditional ORMs and ODMs.

Ecosystem Overview

Ostara is based off of electron-react-boilerplate and uses electron-builder to package the application.

The daemon uses JDK 17 with Kotlin, Spring Boot and Gradle

Project Daemon

Before we set out to research and implement this daemon for Ostara, we laid out a list of requirements:

A user should not need to install Java.
No significant downgrades to the developer experience.
The daemon process should be robust and resilient.
Support two-way communication in a manner that is similar to IPC, but doesn't tie us to Electron.
The startup time for the daemon should be under 20 seconds at all times.

Let's break these down.

A user should not need to install Java

The rationale here is simple, right? The user installed your app, they don't really care about your app's dependencies, and they certainly don't want to be given (either automatically or otherwise) a shopping list before they can use it.

Now, this may sound like a silly argument to make when you realize Ostara's primary target audience is JVM developers who certainly have JDK on their workstation. But which JDK? Spring Boot developers come in all shapes and sizes, some will still run JDK 8 on their workstation while others will be on the cutting edge.

Overall, the goal of this requirement was to reduce our footprint and (visible) overhead as much as possible while allowing maximum versatility.

Now right off the bat, let me just say that an alternative to this process called jlink exists. We opted not to use it, so I will not elaborate on it further.

The app is packaged 4 times, once for Windows (x64), once for Linux (x64), and twice for Mac (ARM and x64). Each one of these has its own unique JDK.

The goal would be to download the respective JDK, where it would eventually find its way inside the finished package for each operating system and architecture.

An additional goal we set was to avoid bloating the repository with zip files that we can easily download on the spot and being forced to use Git LFS, so committing the JDKs was not an option.

To achieve the procurement part, we wrote jdkutil, a pure Python tool with no external dependencies.

The tool is very simple in its operation. Given a list of categorized URLs (In our case, this csv), it will download them, verify the checksum matches, and unzip them into a directory tree matching this pattern: {platform}/{arch}/jdk

As a visual example, for the four variants above we will receive the following tree:

jdks/
├── linux
│   └── x64
│       └── jdk
├── mac
│   ├── arm64
│   │   └── jdk
│   └── x64
│       └── jdk
└── windows
    └── x64
        └── jdk

Once the JDKs were ready in this format, all we had to do was add the following to the extraResources block of the build section within our package.json:

{
    "build": {
        "extraResources": [
            {
              "from": "jdks/${os}/${arch}",
              "filter": [
                "**/*"
              ]
            }
          ]
    }
}

When electron-builder runs, it will populate ${os} and ${arch} according to the operating system and architecture being built, so at the end of this process we should have an arch specific, OS specific folder called jdk in every package that we create, with the correct JDK for the designated platform.

No significant downgrades to the developer experience

While a developer is expected to have JDK 17 installed, if they are only working on the Electron side there should be no additional steps to run or perform.

Setup

A "frontend" developer shouldn't need to setup a working environment to work on the daemon, but they will still need to be able to run the current development version and build it from scratch.

Likewise, a "backend" developer should be able to run their daemon from their IDE of choice and be able to hook it up to the process.

Finally, in a packaged state we will have a jar as well as a custom JDK that we will need to use.

This lead us to split the way we start the daemon into two ways.

The first is the "Packaged" approach, where Electron already knows the location of a prebuilt JAR and simply runs the process.

The second is the "Remote" approach. In this approach, Electron doesn't actually start the process, but simply receives a predetermined port and address (Usually localhost) and creates a connection with the daemon over HTTP.

The remote approach covers both development use-cases. In order to achieve this, we split our start command into two:

    "start": "concurrently \"npm run start:daemon\"  \"ts-node ./.erb/scripts/check-port-in-use.js && npm run start:renderer\"",
    "start:thin": "ts-node ./.erb/scripts/check-port-in-use.js && npm run start:renderer"
}

A "frontend" developer in this case would run start, which would then in turn compile and run the daemon locally. Since the app isn't packaged, the app will automatically determine that it needs to use a Remote daemon with the default host and port.

A "backend" developer would run start:thin which essentially does everything except start the daemon, with the expectation that the daemon is already running. Like the previous example, the app will automatically determine that it needs to use a Remote daemon with the default host and port.

Implementing health checks

Since the Daemon, like any process had the possibility of crashing or freezing, the next step was to implement health checks. We opted for a simple polling HTTP health check from the Electron main process to the daemon.

For this process we created 3 events in the Electron main process:

daemon-ready
daemon-healthy
daemon-unhealthy

The flow created around these events is simple.

Upon app start, the user will be sent to the splash screen. Once the daemon-ready event is fired, the splash screen is closed and is replaced by the main screen. If at any point during this time we receive the daemon-unhealthy event, the user's screen is replaced with the following:

If, during this time, daemon-healthy is fired, then the screen returns to normal.

Support two-way communication in a manner that is similar to IPC, but doesn't tie us to Electron

To achieve two-way communication between the Electron renderer and the daemon, we opted to use Websockets with the STOMP protocol. This approach decoupled the communication between the renderer and the daemon from the Electron framework, allowing for greater flexibility in the future and the ability to break away from Electron entirely if needed

Conclusion

In conclusion, despite the unorthodox choice, we successfully developed a robust and resilient daemon for our Electorn app using Spring Boot, all while addressing our primary goals.

By doing so, we have created a flexible and scalable app that can run on and off Electron with the potential for remote backend functionality. Additionally, we have opened the door for further expansion and improvements in the future, should the need arise.

Creating an automatic Helm Repository with GitHub Actions

Idan Elhalwani — Wed, 10 Feb 2021 15:35:00 +0000

I needed a Helm repository to house my private charts, but at the same time I didn't want to have to deal with and maintain a server for it.

In my research I discovered that it is possible and quite easy to use a git repository for this purpose. Great! However, I feel that did not go far enough for my use. For once, I did not want to have to manually package and index the repository on every chart change and so I decided to expand on it.

The end result

At the end, we should have a Helm repository that updates itself on every change to the git repository structured as follows:

├── master
│   ├── charts/
│   │   └── example/
│   └── sync_repo.sh
└── repo
    ├── example-0.1.0.tgz
    └── index.yaml

The full example is available at my GitHub

The setup

For this, create a new git repository, clone it locally and create a new chart in the charts directory;

mkdir charts
cd charts
helm create example

Once that's done and committed, we will create an orphan branch to store the actual Helm repository. Why an orphan branch? On every commit to the repository, CircleCI will do its own commit to update the Helm repository.

While this step is optional, it prevents developers from having to pull-rebase every time they make a change to the repo, and has the added benefit of keeping the history of the master branch clean.
Create an orphan branch called repo as follows:

git checkout --orphan repo
git rm -rf . 
touch index.yaml
git add index.yaml
git commit -m 'Initial Commit'
git push -u origin repo

Now let's return to the master branch. Our next step is to create a script which our CI will use on every commit. The script will package all charts, and re-generate the index.yaml file.

Use your text editor of choice to create sync_repo.sh and add the following to it:

#!/bin/sh
mkdir -p repo
cd repo
helm package ../charts/*
helm repo index .

At last, we add the last piece of this puzzle and integrate GitHub Actions into this process.

Our action will pick off where our shell script left us off; It will clone the separate repo branch and copy the generated files to it, finally committing and pushing back the changes to origin.

For this, we'll create the following action at .github/workflows/sync_repo.yml.

name: Sync Helm Repo
on:
  push:
    branches: [ master ]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          ref: 'master'
          path: 'master'
      - uses: actions/checkout@v2
        with:
          ref: 'repo'
          path: 'repo'   
      - uses: azure/setup-helm@v1
      - name: Sync Helm Repo
        run: cd master && sh sync_repo.sh
      - name: Move repo folder
        run: |
          mv master/repo/* repo/
          cd repo
          git config user.email "idan@elhalwani.com"
          git config user.name "Idan Elhalwani"
          git add -A
          git diff --quiet && git diff --staged --quiet || git commit -m "Update repo [SKIP CI]" -m "${{ github.event.head_commit.message }}"
          git push

Let's break down the action;

# Move our generated files from the master branch folder to the repo branch folder
mv master/repo/* repo/
cd repo
# Configure git for when we're ready to commit
git config user.email "idan@elhalwani.com"
git config user.name "Idan Elhalwani"
# Add all new files and attempt to commit. If there is nothing new, the commit won't go through and nothing will be pushed
git add -A
git diff --quiet && git diff --staged --quiet || git commit -m "Update repo [SKIP CI]" -m "${{ github.event.head_commit.message }}"
git push

Conclusion

At this point, we have a fully functional and automatically updating Helm repository. To use it, get the raw URL for the repo branch.

In our case, we will add the repository as follows:

helm repo add example https://raw.githubusercontent.com/idane/helm-repo-example/repo