DEV Community

Jonas Brømsø
Jonas Brømsø

Posted on

Incidents Often Come in Pairs

On the first of September I followed up on a recent update of llvm. Release 21 has just made it and I found the time to update my clang diagnostic flag matrix.

I generated the matrix using my Perl tool, which crawls the documentation and generates a markdown table of all the available versions for comparison.

I noticed that the release was 21.1.0, a pattern I had noticed with 18.1.0 so my first generation needed to be handled as a special case just as 18.1.0 had been.

Next up was the proxy, which the links point to, since they are shortened URLs. Here I noticed that I had missed the same pattern for 19.1.0 and 20.1.0, so I had to patch the proxy for those as well. So I patched the proxy which is a serverless function and deployed it, I did a quick test and everything seemed fine.

Unfortunately I missed a bug I introduced and as I deployed an alarm was triggered, which I also missed.

The other day I received a notification that my monitor had been paused due too inactivity and yesterday I finally got around to investigate.

With the introduction of support for 21.1.0, 19.1.0 and 20.1.0 I had broken everything, hence the alarm. So I pulled up the code and had a look. To get a second opinion I asked my patient pair programmer to explain, the following contstruct I suspected was the culprit:

if majorlevel >= "18" {
Enter fullscreen mode Exit fullscreen mode

And yes.

This line is a conditional statement in Go that checks whether the value of majorlevel is greater than or equal to the string "18". The comparison uses the >= operator, which, when applied to strings in Go, compares them lexicographically (dictionary order), not numerically. This means "20" >= "18" is true, but "2" >= "18" is false, because "2" comes before "18" in string order.

Potential gotcha: If majorlevel is meant to represent a numeric value (such as a version number), comparing it as a string can lead to unexpected results. For example, "100" >= "18" is true, but "9" >= "18" is false, even though numerically 9 is less than 18. To avoid this, you should convert majorlevel to an integer before comparing:

major, err := strconv.Atoi(majorlevel)
if err == nil && major >= 18 {
    // ...
}
Enter fullscreen mode Exit fullscreen mode

This ensures the comparison is numeric, which is usually the intended behavior for version checks or similar logic.

Potential gotcha indeed.

So I restructured the code a bit and deployed a new version, which I have now tested and verified is working as expected, this time with a bit more patience.

Lessons learned:

  • Remember to test your changes, even if they seem trivial.
  • Be patient
  • Incidents often come in pairs

Things to pick up on:

  • Automated tests for the proxy
  • Keep learning Go
  • Perhaps a peer review by Copilot

Top comments (1)

Collapse
 
jonasbn profile image
Jonas Brømsø

I wrote up a brief postmortem for the status page