DEV Community: Amit

Real-time data visualization and alerts using Redis TimeSeries, Grafana and Slack.

Amit — Wed, 13 Dec 2023 06:37:01 +0000

In any non-trivial application, each part of the system generates a substantial amount of data. Whatever be the domain, we want to observe, monitor, and act on this data.

We will explore visualization of real-time data in Grafana and while Prometheus is usually what is associated with in such scenarios, we will look at Redis TimeSeries.
We will create a dashboard for a simple app that generates a continuous stream of data and store it in Redis as Time Series data and visualize in Grafana, and configure alerts via Slack.

Application and Data

The client is a simple .NET console app that sends memory usage data on the Windows client, with an introduced delay of 750 milliseconds. This data includes total system memory usage (which is what is visualized) and memory used by each process.

The server is a basic Node.js app, and for simplicity it uses WebSocket connection as we have a steady stream of data being received. This data is then passed on to the primary Redis instance which will accept the incoming writes.

Note:
The client and the server apps are developed to get the data flowing in and do not follow any best practices like retries, error handling, configuration, logging etc.
In the current dev setup, the Node.js server, Redis and Grafana all run on the same Mac.

Configuring Redis

Installing Redis on Mac is straightforward following the instructions here. We have 2 Redis instances running - one to accept the writes and the other, from which Grafana will pull the data for visualization. There is significant documentation on replication and persistence in Redis and for this example, the second instance acts as a read-only replica which is replicated asynchronously.

We use the Redis TimeSeries type which, as the name suggests,¬ stores time series data. The type expects the value to be numeric and that is what we have - a series of numbers representing the memory utilization, in Megabytes, at that point in time.

For every data point, the key is the current timestamp, and the memory utilization number is the value. At this point, we have the data in memory, in the format we need, which can be visualized on a dashboard.

Note: We have a single Windows client here, but we can have multiple clients with a label for each time series data entry to query on.

Grafana and the Dashboard

Grafana is an open-source data visualization and monitoring solution - it allows us to query, visualize, set alerts and more for the data at hand. In this example, Grafana is setup to run locally on the Mac. After installation, the webpage is available at http://localhost:3000 where you can sign-in with the default credentials of admin/admin (and change the password).

First order of business in Grafana is to set up a dashboard with the data source of our choice, in this case Redis. You can create a new dashboard and provide Redis as the data source for the dashboard.

The screenshot shows the dashboard with Redis as the source and TimeSeries as the type. There are several commands available but the one we are interested is the TS.GET to fetch the data. You can also notice on the right side the numerous ways to customize the visualization with tool-tips, legend, graph styles etc.

We also apply a Transformation on the data for visualization where it creates a new data frame for each pair of data being received.

Once the entire setup is up and running, we can see the data show up in the dashboard as a continuously updated graph of memory utilization over time.

Slack Alerts for High-Memory Usage

Now that we have the dashboard setup and we can visualize the data, it would be nice if we could do something other than just looking at the graph.
As it is system memory usage, let’s say that we want to be notified if total system memory usage crosses 7000 MB on the client machine. To achieve this, an alert rule is configured where Grafana will notify on a Slack channel.

The alert also allows configuring options like how long it should evaluate the alert condition, how often to check for the condition, silencing options and more.

Here as we are using Slack, we first need to configure it to allow an incoming webhook (This webhook allows POST to the provided URL and is not shared anywhere other than in the Grafana notification setting).
Once added, you should see the notifications in Slack when the alert condition is met.

Grafana supports notification templates which can be reused across different types of notification contact points and messages can be configured and formatted too.
It also supports a whole bunch of options when it comes to alerting other than Slack.

Here are the GitHub links for the Windows client app, and the Node.js service with the Redis configs and Grafana dashboard/alert rule export.

Cover image credit: Photo by Markus Winkler on Unsplash

Cross-Platform app with C# and Uno Platform - Part 2

Amit — Mon, 09 Aug 2021 18:10:00 +0000

In part 1, I covered a bit of intro to Uno Platform and the app built on it. I'll close with some thoughts on cross platform development, challenges and alternatives for Uno Platform here and wrap up.

Let's talk Cross Platform

Single codebase means simpler maintenance - a feature upgrade is a single change for all platforms. This also means simpler testing and build pipeline too, as you don't need one for every platform or the associated language used for that platform. It does make a good case for itself and seems pretty logical when you think about it, right? If your team is proficient in C# then you're good. If not, how long are you willing to wait, if you are willing to?

You can visit the technology showcase section of Uno Platform, React Native, Flutter and Xamarin and it looks like there are quite a few of them.
On the flip side, there's this article from Airbnb and one from Dropbox as well, where they went back to native.

So, does a single, shared codebase work for everyone or not? Only way to find out is probably by trying out a sample (and slightly complicated app) and see how it goes, or an internal non-customer facing app to get a feel of how it would work.

Third-Party UI Controls for Uno Platform

In the previous post, I mentioned Windows Community Toolkit for UI controls, but in .NET apps on Windows (WPF, UWP, ASP.NET Core etc.) you have a lot of options for third party controls. With Uno, the documentation lists Syncfusion and Infragistics. Now this app that I have created, is a cut-down version of the Web Assembly version that I created with Blazor. The main reason for this is I don't have those chart controls here for me.

With Syncfusion, it required adding their entire project locally to the solution to integrate it and there was no license specified at that time, so I skipped it. It seems to be resolved now though. Infragistics looks good too but they have a trial license which I didn't go for at that time.

I cannot comment on these as I haven't used them but they are available for Uno.

What about Mac?

I started out with the target to have the app run on Windows, Mac, Linux and Android. It worked fine on all except for the Mac. This is due to the way I have created the sample and let's see what that is. UWP allows us to set visual breakpoints and set visual state for each of those. As seen in this example -

<x:Double x:Key="MinWindowBreakpoint">0</x:Double>
<x:Double x:Key="MediumWindowBreakpoint">641</x:Double>
<x:Double x:Key="LargeWindowBreakpoint">1008</x:Double>

We can then specify the Data Template, to optimize the view for that size, and this gets triggered depending on the dimensions specified in the breakpoint. Think of this as a CSS media query, to make it easier to relate to.
In our case, I also have the command in the viewmodel that fetches data on startup, asynchronously. This data is being bound to the DataGrid which is present in the above Data Template. This somehow doesn't work on Mac, while it works fine on other platforms.
I have opened a GitHub issue for this and until then, I have the app in Mac without any data, to show that it does launch and also is responsive (visual adaptive breakpoints).

With Default Width

if I minimize the app horizontally, it gets to the small breakpoint and readjusts accordingly/

Reduced width to see the responsive UI

The Players

There are quite a few alternatives and I'll just list them here as I haven't tried all of them.

Flutter - Google's UI toolkit offering, using Dart.
React Native - Apps for Android and iOS, using React.
React Native for Windows and Mac - An interesting option to bring React Native apps to Windows and Mac!
Xamarin - Extending .NET to multiple platforms including watchOS and tvOS.
.NET MAUI - Evolution of Xamarin.Forms, currently in preview.
Electron - Cross platform desktop apps for Windows\Mac\Linux. VS Code is built with Electron.

What Next?

Probably one last port of this app to .NET MAUI, Microsoft's upcoming offering, for comparison. React Native for Windows and Mac looks good for experimenting too.

Choice is good.

Cover image credit

Cross-Platform app with C# and Uno Platform - Part 1

Amit — Mon, 09 Aug 2021 17:57:02 +0000

If you are developing an app, you want to reach as many users as you can and the easiest and the most obvious target is a web application. It is easily accessible irrespective of the platform users are on. However, not all applications\scenarios can be delivered through a browser and this is why we have desktops\laptops, wearables and mobile apps. So now you have to write software that supports these platforms as well. Will life be easier if you could write code just once that would work on all these platforms?

So, desktops\laptops and mobile - that's 2 categories?

Only 2 platforms (or form factors) - desktops\laptops can be Windows, Mac or Linux. Phones are either Android or iOS. This is not even considering the huge market of Tablets and Wearables. It is also not considering the various versions of those devices running various versions of the operating system on them.

via GIPHY

In this article, we will take a look at Uno Platform to develop our app for Windows, Mac, Linux and Android.

Note: We will discuss a bit more about cross-platform development, challenges and alternatives to the Uno Platform in part 2 of this article.

What is Uno Platform?

Uno Platform is an open source UI platform to build apps with a single codebase that can target a wide variety of platforms. You write code in C# and XAML to target all these platforms. It builds on existing capabilities provided by each platform internally, so that you can focus on developing your app. It does allow you to add\modify the app's behaviour based on where your app is running.

Source:Uno Platform

GitHub docs has a good summary on how Uno works for build, runtime and rendering across platforms.

Simple app with Uno

We will look at creating a simplified version of the Portfolio Tracker web app that I previously wrote about. The constraint I have imposed is, I shouldn't have to modify my service or database. It has to work with the existing set of APIs.
My requirement boils down to this -

Use this current available data and write an app that runs on Windows, Linux, Mac and Android.
Reuse as much code as possible from the previous Web Assembly project that I've created.

Note: Uno supports more platforms than the ones I'm targeting. It supports iOS, Tizen and Web Assembly as well. I cannot try out iOS and Tizen so I've excluded those and since I already tried out Web Assembly using Blazor I didn't include that as well.

I have used Visual Studio 2019 on Windows and Visual Studio 2019 for Mac. Uno Platform getting started guide for both platforms are available here.

Project structure

Here is the Visual Studio project structure for this app -

Uno Platform Visual Studio Project Structure (in Windows)

The Mac target project is unloaded as it is not supported on Windows.
The Shared project is only a list of files. They are shared by each target platform and built for each as well.
Each target project can have items specific to itself - fonts and image assets for each platform are couple of examples.
Any reference\NuGet package to be added, has to be added to all target platforms.
MainPage.xaml in the shared project is where the navigation is setup using the NavigationView. Also the default view is set in Main.xaml.cs and in this case, Dashboard is set as default.

The code

From the actual code perspective, it is writing a C#\XAML UWP app. We're working on the UI and we need UI controls. I will come back to this in part 2 to talk a bit more but I will mention the Windows Community Toolkit here. There is a section in the docs on how to use it, which is ported onto other platforms. I did include DataGrid, NavigationView, ProgressBar in the project and it worked fine.
I could not try out gesture support as I tried the app in the Android emulator and not on a physical device.
In a real production app, it may be the case that we will need platform specific behaviour. I didn't really have such a scenario for this app but I wanted to try it out anyway, so this is what I did - on Android I wanted the navigation menu to be the hamburger menu (3 stacked dashes), on the top left. For desktops\laptops, we have plenty of screen space so I did not want a hamburger menu and instead as menu items with the selected one being underlined.


#if __ANDROID__
    RootNavigationView.PaneDisplayMode = NavigationViewPaneDisplayMode.Auto;
#else
    RootNavigationView.PaneDisplayMode = NavigationViewPaneDisplayMode.Top;
#endif

Uno does support platform specific code both in C# and via XAML as well.

Couple of app screenshots here from Linux and Android - Linux was a virtual machine on Windows 10 running Kubuntu 18.04.

C# UI app in Linux

Android screenshot is from the emulator as I didn't root my phone to install this locally developed app. Yeah the app does not look great, thanks to my choice of painting it gray.

Same app running in the Android Emulator

How is Uno so far?

You can target a wide variety of platforms with a single codebase and that is neat. It includes Web Assembly as well so it covers the browser space too. In the app, the code in the folders under Shared project (Models, Interfaces, Services, Helpers etc.) are code that I've used as-is from the previous project. This is the biggest advantage I've seen writing this app with Uno. I have my service and database and I have all the code to fetch and handle it. The focus is only on building the UI. Unless we need\want to do something platform specific, Uno abstracts it from us.
XAML Hot Reload is super useful, especially if we're used to, say React development for example. Uno does have an active Discord channel where core members participate, which is encouraging to see.
There is also decent support for 3rd party libraries (I'm excluding UI controls here, which I talk about in part 2) including presentation frameworks like MVVM Light and theming options like Material UI as well.

Link to the GitHub repo for this app.

Cover image credit

Working with ASP.NET Core Blazor WebAssembly

Amit — Wed, 28 Apr 2021 15:00:05 +0000

In the previous part I went over the app details but didn't really talk about the development experience. I'll close with some thoughts on that here in this second part.

Dev experience - Visual Studio

I'm using Visual Studio 2019 Community Edition and have encountered some issues in debugging, which are already being tracked\planned for future -

Breakpoint limitations - specifically the OnInitializedAsync part. There are workarounds like this which are available.
No hot reload, yet - you need to re-launch the app for a code change. There is an option to debug with dotnet watch run though.
Sometimes locals\watch wouldn't work for me, but I can't specifically recall exactly when that occurred.

Working with Blazor

When creating the app, you get an option to select if you want to make it a PWA, which I did. I was able to install the app from the browser without any additional setting, which is pretty neat. The documentation does cover details about push notifications and service workers but I didn't try that out for this app.
It is also straightforward to have a responsive design and then have the appropriate styling\CSS to make those look better. The app is usable on a mobile form-factor, but it definitely should be better and that's not Blazor's fault, it's mine :)
I have published the app to IIS but here's a detailed blog on various deployment options.
Since it is a web app already running in a browser, you can take advantage of existing tools like Lighthouse to audit the app and look for ways to improve accessibility, performance, SEO etc.

Downloading Compressed Assets

Downloading Uncompressed Assets

Above screenshots are taken on Chrome on Mac, loading the app published to IIS on Windows 10 on a different laptop. They compare network data transfer that includes stylesheets, Radzen components, .NET dependencies etc and that's ~4.4 MB vs ~9.9 MB for uncompressed. That's a good compression ratio but it'll be good to see some data for complex apps with more data and dependencies and see how it performs. The second image also shows the app in an almost mobile form factor with the menu shifted up.

Third-party Controls

Blazor has a quite a few third party libraries\controls available. Here's one of the few lists which link many resources available for Blazor, in particular the libraries piece - https://github.com/AdrienTorris/awesome-blazor#libraries--extensions
You do have lot of free, paid or open source options. I tried Syncfusion Community License and they're good (with support too) . I used Radzen in the app and it worked well.
I couldn't try all available libraries and this isn't a sponsored post

What next?

The app itself, has a lot of to-do items. It needs multi-user support and login, unit tests, improvements so that it doesn't fetch data on every page switch, accessibility etc. It also should be served over https which is not the case in my local setup.
Try comparing with some other front end technology, like React, for development experience and performance.
Couple of things announced by Microsoft to explore building native\hybrid and cross platform apps using Explore Blazor Mobile Bindings and also .NET MAUI

Building a Web App with ASP.NET Core Blazor WebAssembly

Amit — Wed, 28 Apr 2021 14:45:27 +0000

There are various options available today to create a web app, and usually at work, you can't pick up every new framework or technology that pops up. However, we can always experiment with one (or more) and build a hobby project. I've tried creating one such app with Blazor WebAssembly and also write my first article along with it.

What is Blazor?

Blazor is an open source client side framework to build web apps using C# instead of JavaScript. It can be considered, functionally, as an alternative to React\Angular etc. to write web apps\SPAs, though it is new and the tooling is evolving.

Why Blazor WebAssembly?

JavaScript (and the libraries\frameworks around it) is the more popular choice when it comes to web apps. There were other options like Flash, Silverlight etc. though not standardized (and not relevant anymore).

With WebAssembly, things get bit more interesting. WebAssembly enables code written in programming languages (C#, C++, Rust, etc) other than JavaScript to run on the browser and it certainly does help that it is an open standard as well. Also, all the major browsers - Firefox\Chrome\Edge\Safari, support WebAssembly already.

Coming back to Blazor, it has two hosting models for the app to run either on a browser or on the server with ASP.NET Core. I want to try build a web app, in the traditional sense that has all client code downloaded in the browser and talks to some REST endpoints, and see how it compares to building such an app using say React.

Source: Microsoft Docs

The Node.js Effect

I would like to draw a parallel with Node.js - the availability of JavaScript as a runtime on the server gained significant traction. But why even talk about that here? The evolution of WebAssembly and its support for various programming languages (specifically C# in this context), to me, feels like something similar on the client side of things. Say you have a .NET desktop app or a web service, if there is a reusable piece of logic, it can then be used pretty much as is on the web now.
Of course it is too early to say, but given the browser support and growing tooling, it could get widespread adoption.

Simple app with Blazor WebAssembly

Coming to the actual web application, it is a simple mutual funds portfolio tracker. You pick the funds you want to track, enter some details and save it. The app displays basic info like % returns, value trend, top gainers, top losers, ability to add\delete mutual funds and has few graphs. The funds are randomly picked, I'm not an investment expert and this is not investment advice :)

Some details about how the app is structured -

The data is sourced from https://www.mfapi.in/
I've picked a few prominent funds from the above website and stored the historical data into a MySQL database. JSON result from the above API source is parsed and the historical data is stored for these funds.
The app itself is hosted in IIS on Windows 10
I also have an ASP.NET Core Web API for my REST endpoints, which interfaces between the app and the database. This repo has the scripts for creating tables and inserting some of these funds data into it. Here are couple of screenshots from the app - Dashboard View

Manage Portfolio View

Project Structure

This is the Visual Studio project structure for the web app -

Blazor WebAssembly Visual Studio Project Structure

The menu items are specified in the NavMenu.razor file which is included inside the sidebar div of the MainLayout.razor. This menu is responsive, if you resize the browser you can notice that it transforms to a hamburger menu.
All references for your services and third party components are specified in the _Imports.razor file.
Pages are the individual pages that are loaded, to which users are routed to based on the menu item that is clicked. Each page has the @page directive and that is the same value mapped in the NavMenu.
I've made some changes to the app.css in the wwwroot folder to handle styling
The services should preferably have an interface and the corresponding implementation for each, which I don't have as of now.

Also, the official documentation has some more details on the project structure.

Here are the links to the GitHub repo for the Blazor UI app and the repo for the Web API project

In the second part of this article, I go over some thoughts on development experience and conclude.