DEV Community: Danny Verpoort

An alternative or simpler way to event stream with or without Kafka

Danny Verpoort — Fri, 30 Apr 2021 14:08:32 +0000

Alright hear me out, I know I'm over-engineering this hobby project but I could use some advice.

In the Netherlands, emergency services receive their alerts via texts on pagers. I've bought a receiver that makes it possible to (legally!) intercept these messages and it's churning away gathering all the alerts.

I'm also into microservices and now I've got something that's steadily producing data that I can eventually show in a web interface. Now let's say that I would like to use an event bus to introduce the data captured from the alerts to the microservices which parse it and store some data (i.e. about which types of units, what kind of locations, etc.).

I've set up a cheap Kubernetes cluster in Google making use of GitLab CI/CD, Terraform, and some helm charts with KubeIP and Traefik. This is all running fine.

Now comes the challenging part. I would love to try to use Kafka to publish the events in my microservice network but geez does it become complicated there. I've found some Helm charts here which seem to be meant for development, testing, or proof of concept services but it seems to contain more than just Kafka and Zookeeper. Looking at the documentation for real production data it seems like an even more daunting task.

Now here's my question:

What's your advice for running an event bus in a Kubernetes environment for a hobby project where I'm not necessarily afraid to learn new stuff? I'd love to hear your advice for either a bare-bone Kafka approach or any other products you could advise. Keep in mind that the budget is pretty much non-existent because It's a hobby project.

That's enough making my life unnecessarily complicated for today! :)

My beef 🥩 with Javascript and ReactJS ⚛️

Danny Verpoort — Tue, 10 Nov 2020 19:54:03 +0000

So, this article has two purposes. The first purpose is for me to vent a little bit about my experience working with ReactJS. At the end of the day, the purpose is to hear back from the community and learn a thing or two about how to solve the problems I ran into and what I could have done better.

I recently started an open-source project based on GitHub's Markdown profile feature. The goal is to make it easy to include awesome features for your own GitHub profile. I started off with a NextJS project which I would export with GitHub Actions and then publish through Github Pages. During development, I ran into some hardship using the ReactJS framework feature. Let's have a look at the issues.

dannyverp / markdownprofile

Markdown Profiles is a project that generates markdown files for those that want an awesome Github profile.It provides several templates for you to use.

Interfaces

I've been working with object-oriented programming for a while so it's hard for me to let go of the habits I picked up there. However, to find out that Javascript doesn't do easily do interfaces was a terrible shock to me. I mean how can you adhere to SOLID principles if your language doesn't even feature interfaces?

The use case I had here was that I wanted to make it easy for other contributors to extend the markdown template website with new templates. I was going to create an interface that could be implemented and would contain all the methods that I use on the main page to render and update the templates. I ended up creating a template class which would simply chuck errors for methods that weren't implemented. A bit like such:

import React from 'react'
import {error} from "next/dist/build/output/log";

export class Template extends React.Component {
    render() {
        throw error("The render method is used to render the Template's form. Please implement a form!")
    }
}

export default Template

Methods, methods everywhere!

Why is it that everything seems to be migrating functions? What have classes ever done wrong to you? I mean how are you supposed to easily keep track of objects that were returned from an API. Aren't you supposed to have high cohesion in your code? I'd love to have my data objects all within my classes. I'll be eternally (or at least a while) indebted to someone who (links me an article that) thoroughly explains to me why functional components are better.

Tracking global states

So I've got a system where users need to log in. Once they're logged in I want to be able to access the current user throughout my entire web app. I couldn't for the life of me figure out how to create something like a singleton in my application to track one sole instance of that user object. I found some documentation but it was mostly "we don't advice you do this". Admittedly a singleton is somewhat of an anti-pattern but that doesn't mean it suddenly completely ceased to be useful. How would I go about solving the above use case?

Ok, it's also been great

I can keep on ranting for a while but these three things have so far been my biggest pain points. The experience has also been great compared to the old school jQuery way of doing things. Javascript has come a long way and I hope it will keep doing great things in the future. But it also bugs beyond belief. Now's your chance to bash me and prove me wrong!😊

Insights in Cyclomatic and Cognitive Complexity in Your Application

Danny Verpoort — Fri, 10 Apr 2020 10:55:26 +0000

Everybody who's ever been exposed to a legacy codebase knows the feeling of not having any idea where to get started. You start opening some random files, looking through the code, hoping to get a grasp on how this project works. You size up the project and try to pinpoint problem areas that are suitable for the first refactoring attempts.

I wished that developers included more documentation and diagrams so others can more easily get an oversight of how things work. This would help in preparing roadmaps and communicating the effort to clean up technical debt to product departments and management. Luckily enough I came across a toolset that helped me size up the cyclomatic and cognitive complexity in the codebase and coordinate the refactoring efforts.

Elasticsearch project complexity visualized

What is cyclomatic complexity?

Cyclomatic complexity is a quantitative indication of how difficult the code is to comprehend. If a program is drawn out as a graph with vertices (nodes) and edges then the following calculation results in the cyclomatic complexity:

M = E - N + 2P

Where M is the resulting complexity, E is the number of edges, N is the number of vertices and P is the number of possible exit points.
Let's take the following PHP code and calculate the cyclomatic complexity:

<?php
  public function getWords(int number): string
  {
    switch (number) {
      case 1:
        return "one";
      case 2:
        return "a couple";
      case 3:
        return “a few”;
      default:
        return "lots";
    }
  }

The above code can be translated in the following graph:

In the graph, there are 5 vertices, 7 edges. In simple cases likes these we consider the return points as a single string return. Throwing an exception would be an example where we can consider two exit points. This leads to the following cyclomatic complexity calculation:

M = E - N + 2P
E = 7
N = 5
p = 1
M = 7- 5+ 2 * 1= 4

To give you a general idea of how this quantitive data transfers into a feeling of complexity, here's how the Software Engineering Institute classed different levels of cyclomatic complexity:

1–10 a simple program, without much risk

11–20 more complex, moderate risk

21–50 complex, high risk

>50 an untestable program, very high risk

Where does cognitive complexity come in?

Now lay the code we've just seen beside the following code:

<?php
sumOfPrimes(int max): int {
  $total = 0;
  for (int i = 1; i <= max; ++i) {
    for (int j = 2; j < i; ++j) {
      if (i % j == 0) {
        continue;
      }
    }
    total += i;
  }
  return total;
}

Both of these code snippets have a cyclomatic complexity of 4 whilst this code is clearly much harder to understand. Cyclomatic complexity is simply bound to the number of lines of code and the number of control structures (if, while, switch) used. We miss a form of context to judge the complexity of the code.
SonarQube came up with the concept of cognitive complexity in order to introduce a more contextualized form of quantitive data on code complexity. The complexity is assessed based on these three basic rules:

Ignore structures that allow multiple statements to be readably * shorthanded into one.
Increment (add one) for each break in the linear flow of the code.
Increment when flow-breaking structures are nested.

If you would like to further understand the context of cognitive complexity, check out the whitepaper SonarQube published on the matter.

But how does this help us? Well following the rules of cognitive complexity the first code example boils down to a cognitive complexity of 1 whilst the second example is a complexity of 7. This shows the contrast between the Cognitive and Cyclomatic Complexities and how cognitive Complexity judges more based on the code's context.

Great, but how does this help me in large codebases?

So obviously we're not going to calculate the cognitive complexity of every method within our codebase. How does this concept help us determine the paths to take inside our project?

This is where the combination of SonarQube and the plugin SoftVis3d come into play. Using this plugin you can visualize your codebase based on the lines of code, issues, expected bugs, security vulnerabilities, cyclomatic complexity, cognitive complexity, etc. The code is sorted by directory and each file is represented as a bar chart. You could also consider this a house or flat in a neighborhood if you will.

If we pick the building height as the number of lines of codes and the floor space as cognitive complexity the most ideal situation would be a nice suburb with small houses and nice gardens. However, in some cases, you might end up with the New York City skyline.

Eventually, this helps you tear down the flats and replace them with small buildings where you have your concerns separated and your coupling kept to a minimum. If you're looking for a nice example to see the idea in action check out ElasticSearch's public SonarQube.

Conclusion

We've looked at the concept of cognitive complexity which gives us some insights into the structural complexity of our codebase. In order to fill the gap of not having contextualized quantitative data SonarQube has invented cognitive complexity which is a more in-depth analysis of the complexity. In the end, we plotted this data into SoftVis3D which gave us a visualized idea of the structure of our codebase.

At Takeaway we're always looking to improve the quality of our work. We manage our complexity and quality using SonarQube and the SoftVis3D plugin amongst other cool tools. Make sure to check out our careers page to discover Tech jobs at Takeaway.com!
If you've got any thoughts, questions or concerns reach out in the comment or get in touch on twitter.

Sources:

https://www.sonarsource.com/resources/white-papers/cognitive-complexity
https://resources.sei.cmu.edu/asset_files/Handbook/1997_002_001_16523.pdf
https://softvis3d.com/sonar/project/extension/softvis3d/overview_page?id=org.elasticsearch:elasticsearch