Redis Status Page, Redesigned

You might still remember my previous post when I created a status page using Blazor Server and Redis. In the previous post I mentioned one of the downsides of the implementation which is overeliance to LINQ and Redis.OM, caused by my naive approach to data modelling using Redis.

In this new version, I tried to maximize the use of Redis data structures (List, Set, and Hashes) to better store the service uptime data so it will be easier to store and query.

What's new?

The first version

In the first version, I use Redis.OM to store the data and LINQ to query and format the data. Using this approach the data storage and query quickly became complex, since I tried to model the data as it was a table.

Storing the data:

// save to redis using OM
var collection = _cnProvider.RedisCollection<MonitoringSnapshot>();
await collection.InsertAsync(new MonitoringSnapshot
{
    UnixTimestamp = timestamp.ToUnixSeconds(),
    ServiceName = serviceName,
    Healthy = healthy,
    Latency = latency
});

Not that bad, eh?

Querying the data:

// get collection
var collection = _cnProvider.RedisCollection<MonitoringSnapshot>();

// query all data
var query = collection
    .Where(x => x.UnixTimestamp > nowUnixEpoch)
    .ToList();
var timestamps = query
    .GroupBy(x => x.ServiceName)
    .First()
    .OrderBy(x => x.UnixTimestamp)
    .Select(x => DateTimeHelpers.FromUnixSeconds((long)x.UnixTimestamp))
    .ToList();
var serviceLatency = query
    .GroupBy(x => x.ServiceName)
    .ToDictionary(x => x.Key, y => y.OrderBy(i => i.UnixTimestamp).Select(p => p.Latency).ToList());

There's so many GroupBy and other commands, not only the querying process is a bit complex, but the stored data is also not easy to scan in RedisInsight.

The new version

In the new version, I tried to use Set, List, and Hash to store the data with my own key pattern. The result is much cleaner and easier data to query.

Storing the data:

// get today date
var timestampDate = timestamp.ToString("yyyy-MM-dd");

// save to redis using OM
var timestampKey = string.Format(TimestampKey, serviceName, timestampDate);
var latencyKey = string.Format(LatencyKey, serviceName, timestampDate);
var healthKey = string.Format(HealthKey, serviceName, timestampDate);

// get redis db
var db = _cn.GetDatabase();

// set current status
await db.HashSetAsync(ServiceLastStatusKey, serviceName, healthy);

// add to set
await db.SetAddAsync(ServicesSetKey, serviceName);

// add timestamp, health, and latency status
await db.ListRightPushAsync(timestampKey, DateTimeHelpers.ToUnixSeconds(timestamp));
await db.ListRightPushAsync(latencyKey, latency);
await db.ListRightPushAsync(healthKey, healthy);

Querying the data:

// query all data
var timestampKey = string.Format(TimestampKey, services.First(), nowDate);
var timestampValues = await db.ListRangeAsync(timestampKey);
var timestamps = timestampValues
    .Select(x => DateTimeHelpers.FromUnixSeconds(Convert.ToInt64(x)))
    .ToList();

// get latency data from all services
var latencyDict = new Dictionary<string, List<int>>();
foreach (var service in services)
{
    // get latency history
    var latencyKey = string.Format(LatencyKey, service, nowDate);
    var latencyHistory = (await db.ListRangeAsync(latencyKey))
        .Select(x => Convert.ToInt32(x))
        .ToList();

    // add to dict
    latencyDict.Add(service, latencyHistory);
}

Not only the query logic is much straightforward, the stored data is now much easier to read in RedisInsight. This is a major benefit since you can use other data visualization tools such as Grafana to visualize the collected data.

I've updated the code on my repository and you can simply clone and run it again!

fahminlb33 / RedisStatusPage

Status page for your microservice apps

RedisStatusPage

Status page for your next microservice backend apps!

Read on DEV.to

This app started as an idea to integrate to my other project, ritsu-pi, it's a Raspberry Pi home server project where you can deploy all kind of apps to a single RPi (or even a cluster). Redis Hackathon comes in just the right moment to give me an extra motivation to finish this project :)

This project is basically a Status Page (like Github Status, Azure Status, Atlassian Statuspage, or something similar) built on top of Blazor Server and Redis. Here you can define a "health check" and get reports when one of your service is down. Also it has a Discord webhook client that will send you a message when one of your service status has changed.

• Monitor HTTP/TCP service uptime
• Simple latency graph over time
• Incident report when one of the services status has changed
• …

View on GitHub