DEV Community: Georgios Kampitakis

[Boost]

Georgios Kampitakis — Tue, 04 Feb 2025 18:05:52 +0000

Tracking down high memory usage in Node.js

Georgios Kampitakis ・ Dec 14 '24

#node #performance #learning #javascript

[Boost]

Georgios Kampitakis — Sat, 14 Dec 2024 19:53:36 +0000

Tracking down high memory usage in Node.js

Georgios Kampitakis ・ Dec 14

#node #performance #learning #javascript

Tracking down high memory usage in Node.js

Georgios Kampitakis — Sat, 14 Dec 2024 19:28:57 +0000

In this article, I will share my approach of tracking down and fixing high memory usage in Node.js.

Context
Approach
- Understand the code
- Replicate the issue in isolation
- Capture profiles from staging services
- Verify the fix
Results
Conclusion

Context

Recently I got a ticket with the title "fix memory leak issue in library x". The description included a Datadog dashboard showing a dozen services suffering from high memory usage and eventually crashing with OOM (out of memory) errors, and they all had the x library in common.

I was introduced quite recently to the codebase (<2 weeks) which was what made the task challenging and also worth sharing.

I started working with two pieces of information:

There is a library that is used by all the services that is causing high memory usage and it involves redis (redis was included in the name of the library).
The list of the services that were affected.

Below is the dashboard that was linked to the ticket:

Services were running on Kubernetes and it was obvious that services were accumulating memory over time until they reached the memory limit, crash (reclaim memory) and start again.

Approach

In this section, I will share how I approached the task at hand,identifying the culprit of the high memory usage and later fixing it.

Understand the code

As I was fairly new to the codebase, I first wanted to understand the code, what the library in question did and how it was supposed to be used, hoping with this process it would be easier to identify the problem. Unfortunately, there was no proper documentation but from reading the code and searching how services were utilising the library I was able to understand the gist of it. It was a library wrapping around redis streams and exposing convenient interfaces for event production and consumption. After spending a day and a half reading the code, I was not able to grasp all the details and how the data flowed due to code structure and complexity (a lot of class inheritance and rxjs which I am not familiar with).

So I decided to put a pause on reading and try to spot the problem while observing the code in action and collect telemetry data.

Replicate the issue in isolation

As there was no profiling data available (e.g. continuous profiling) which would help me investigate further I decided to replicate the issue locally and try to capture memory profiles.

I found a couple of ways for capturing memory profiles in Node.js:

With no leads on where to look, I decided to run what I thought was the most "data intensive" part of the library, the redis streams producer and consumer. I built two simple services that would produce and consume data from a redis stream and I proceeded with capturing memory profiles and comparing the results over time. Unfortunately, after a couple of hours of producing load to the services and comparing the profiles I was not able to spot any difference in the memory consumption in any of the two services, everything was looking normal. The library was exposing a bunch of different interfaces and ways of interacting with the redis streams. It became clear to me it would be more complicated than what I expected to replicate the issue, especially with my limited domain-specific knowledge of the actual services.

So the question was, how can I find the right moment and conditions to capture the memory leak?

Capture profiles from staging services

As mentioned earlier the easiest and most convenient way to capture memory profiles would be having continuous profiling on the actual services that were being affected, an option I did not have. I started to investigate how to at least leverage our staging services (they were facing the same high memory consumption) that would allow me to capture the data that I needed with no additional effort.

I started searching for a way to connect Chrome DevTools to one of the running pods and capture heap snapshots over time. I knew the memory leak was happening in staging, so if I could capture that data I was hoping I would be able to spot at least some of the hotspots. To my surprise, there is a way to do just that.

The process for doing this

Enable the Node.js debugger on the pod by sending a SIGUSR1 signal to the node process on your pod.

kubectl exec -it <nodejs-pod-name> -- kill -SIGUSR1 <node-process-id>

More at about Node.js signals at Signal Events

If successful you should see a log from your service:

Debugger listening on ws://127.0.0.1:9229/....
For help, see: https://nodejs.org/en/docs/inspector

Expose the port that the debugger is listening on, locally by running

kubectl port-forward <nodejs-pod-name> 9229

Connect Chrome Devtools to the debugger you enabled on previous steps. Visit chrome://inspect/ and there you should see in the list of targets your Node.js process:

if not, then make sure your target discovery settings are properly set up

Now you can start capturing snapshots overtime (the period depends on the time required for the memory leak to happen) and compare them. Chrome DevTools provides a very convenient way to do this.

You can find more information about memory snapshots and Chrome Dev Tools at Record heap snapshot

When creating a snapshot, all other work in your main thread is stopped. Depending on the heap contents it could even take more than a minute. The snapshot is built in memory, so it can double the heap size, resulting in filling up entire memory and then crashing the app.

If you're going to take a heap snapshot in production, make sure the process you're taking it from can crash without impacting your application's availability.

From Node.js docs

So back to my case, selecting two snapshots for comparing and sorting by delta I got what you can see below.

We can see the biggest positive delta was happening on the string constructor which meant the service had created a lot of strings between the two snapshots but they were still in use. Now the question was where those were created and who was referencing them. Good thing that the snapshots captured contained this information as well called Retainers.

While digging into the snapshots and the never shrinking list of strings I noticed a pattern of strings that were resembling an id. Clicking on them I could see the chain objects that were referencing them - aka Retainers. It was an array called sentEvents from a class name that I could recognise from the library code. Tadaaa we have our culprit, an only growing list of ids that by this point I assumed were are never released. I captured a bunch of snapshots overtime and this was the single place that kept reappearing as a hotspot with a big positive delta.

Verify the fix

With this information, instead of trying to understand the code to its entirety, I needed to focus on the purpose of the array, when it got populated and when cleared. There was a single place where the code was pushing items to the array and another one where the code was popping them out which narrowed the scope of the fix.

It's safe to assume that the array was not emptied when it should. Skipping the details of the code, what was basically happening is this:

The library was exposing interfaces for either consuming, producing events or producing and consuming events.
When it was both consuming and producing events it needed to track the events that the process itself produced in order to skip them and not re-consume them. The sentEvents was populated when producing and cleared when while trying to consume it would skip the messages.

Can you see where this is going? ? When services were using the library only for producing events the sentEvents would still get populated with all the events but there was no code path (consumer) for clearing it.

I patched the code to only track events on producer,consumer mode and deployed to staging. Even with the staging load it was clear that the patch helped with reducing the high memory usage and didn't introduce any regressions.

Results

When the patch was deployed on production, the memory usage was drastically reduced and reliability of the service was improved (no more OOMs).

A nice side effect was the 50% reduction in the number of pods needed to handle the same traffic.

Conclusion

This was a great learning opportunity for me regarding tracking memory issues in Node.js and further familiarising myself with the tools available.

I thought it best not to dwell on the details of each tool as that would deserve a separate post, but I hope this is a good starting point for anyone that's interested in learning more about this topic or facing similar issues.

Memory leaks in Go

Georgios Kampitakis — Mon, 18 Mar 2024 17:00:44 +0000

In this post, we are going to have a look at

What is a memory leak
Why are memory leaks bad
Common causes for memory leaks in Go
Methods for identifying memory leaks
Investigate memory leaks

What is a memory leak

Let's understand what memory leak is. Reading from Wikipedia

A memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations in a way that memory which is no longer needed is not released.

Go has a garbage collector that does a very good job managing memory for us, tracking down memory that is no longer used and can be returned back to the system. Still there are some cases where we can end up with either memory leaks or our system needing excessive memory to work.

Why are memory leaks bad

Before looking on cases that can make a Go program "waste" memory let's first discuss why it can be bad having memory leaks and why we need to be mindful:

System reliability and stability is affected. The system might behave unpredictably and crash from OOM (out of memory) errors.
Inefficient resource utilization resulting in increased costs.
Performance degradation. As memory leaks accumulate the performance of the system may degrade, affecting responsiveness and efficiency. Memory leaks also create additional pressure on the garbage collector resulting also in increased CPU usage.
Difficult to track down and debug, especially for bigger systems with complex interactions.

Common causes for memory leaks in Go

In this section we will see some cases that can cause memory leaks. Some of those cases are specific to Go and some others more general.

Unbounded resource creation

Creating resources without a limit can be seens as a type of memory leak.

For example if we have a cache that only ever grows our service eventually will crash with OOM (out of memory) error.

The solution would be to restrict how many items can a cache hold (e.g. TTL).

This applies to many resources, like Goroutines (more about this later), http connections or open files. We should always be mindful to have limits on creating resources.

Something you should keep in mind regarding maps in Go. They don't shrink after elements are deleted runtime: shrink map as elements are deleted #20135

Long lived references

Keeping references to objects that your service no longer needs can result to memory leaks as the garbage collector sees the references and cannot free the memory. Some cases where you can keep references unintentionally is with global variables, never ending goroutines, maps or not resetting pointers.

There is a special case in Go for holding references unintenionally with reslicing a slice (this also applies to strings). In our example we have a function readDetails that opens a big file and returns only a portion of it, so we slice the data []byte and return it. In Go slices share the same underlying memory block (Go Slices: usage and internals). That means, that even if we are only interested in a very small subset of the data we are still keeping in memory, (referenced) the whole file.

The correct way here, would be to call bytes.Clone(data[5:10]) so that the data will no longer be referenced and subsequently collected by the garbage collector.

You can also read more information for Go Slices at

Goroutines

Go runtime is doing a great job in spawning and managing goroutines, a lightweight thread, but as mentioned on the 'Unbounded resource creation' section, (realistically) there is a limit of how many goroutines you can have at any time, bounded to the underlying system your service is running on.

More over, iniside Goroutines you can allocate or reference resources. So you need to make sure that your goroutines are properly terminated and memory is finally released.

Let's see the example below. We have a function that creates a new goroutine every second to execute a task, allocates a big data slice does some processing and then hangs forever. This code, has two problems

creates an unbounder number of goroutines
due to not termination of those goroutines resources allocated are never going to be released

Deferring function calls

Deferring a big number of functions can also cause a type memory of memory leak. The most common mistake is when you call defer inside a loop but the defer calls are pushed into a stack and only executed in lifo order at the end of the calling function.

In the example below we are processing files in a for loop and calling .Close on defer. The problem with below code is if we call processManyFiles with a lot of files we are only going to close all the files after we are done processing.

The correct way to handle this case would be to break opening and processing the file to a separate function so when called in the for loop each file we be closed before moving to the next one.

Not stopping `time.Ticker`

Update(14-08-2024): This is no longer true after the release of Go 1.23. Read more on timer changes in the release notes.

Finally, another common case for memory leak is the time.Ticker. As stated in the docs the resources are only released when stopping the ticker.

Correct way to handle a time.Ticker

Methods for identifying memory leaks

The most straightforward method to notice if your service is having a memory leak is to observe memory consumption over time.

In distributed systems, observability is the ability to collect data about programs' execution, modules' internal states, and the communication among components.

From Wikipedia

There are some memory patterns, that if you notice them on your service, you should get at least suspicious that something is wrong.

In the first image, we are seeing a memory graph of our service per pod and the memory is only growing. There are some big spikes in memory, the memory goes down but never back to the previous value. This is a good indication that something is kept into memory and never released. So we should investigate what exactly is using this memory and why.

On the second image, this pattern with the abrupt (cliff like) reduction in memory appears when our service reaches the system's memory limit and crashes with an out of memory error.

You need to keep in mind that in both cases just by noticing this pattern can't be enough to conclude that your service is having memory leak. On the 1st image your service can still be on the start phase where it needs to allocate memory for initialisation or "warming caches" and on the second there is no doubt this is an OOM error but it can be the case you have wrongly sized your service so there is not enough memory. In both cases you should look deeper to identify the causes.

Investigate memory leaks

So if you are suspecting a memory leak where you should look at? Where is the culprit? You could answer simply, take a look at your code, and that would be a valid answer. But in a complex system with external factors and interactions it's not that straightforward.

Profiling

Profiling can help,

Profiling tools analyze the complexity and costs of a Go program such as its memory usage and frequently called functions to identify the expensive sections of a Go program.
Profiling is useful for identifying expensive or frequently called sections of code. The Go runtime provides profiling data in the format expected by the pprof visualization tool. The profiling data can be collected during testing via go test or endpoints made available from the net/http/pprof package. Users need to collect the profiling data and use pprof tools to filter and visualize the top code paths.

From Go Diagnostics

The profile type we are interested in for investigating memory leaks is the heap.

Heap profile reports memory allocations samples used to monitor current and historical memory usage, and to check for memory leaks.

Go has two ways for capturing profiles, either via tests or enabling profiling over HTTP.

go test -cpuprofile cpu.prof -memprofile mem.prof -bench .

and

package main

import (
_ "net/http/pprof"
)

func main() { 
  ...
  http.ListenAndServe(":6060", nil)
}

We are not going to get into more details on how you can setup all this as it would be a whole new post you can read more at
net/http/pprof.

It's worth knowing that go tool pprof supports different types of visualisations for analysing the profiling data you have captured.

Listing of the most expensive calls as text.

Visualization of the most expensive calls as a graph.

Visualization of the most expensive calls as weblist.

But there is a problem with this process, it's too manual and not always so straightforward. When do you capture the profiles? Do you know under what conditions the memory leak happens is it on a dev environment or in production under specific load?

Continuous profiling

With Continuous Profiling we can collect metrics and data from our Services while serving real traffic.

"Continuous profiling is the process of collecting application performance data in a production environment and making the data available to developers"

There are monitoring platforms like Datadog where with very few steps you can onboard your service and provide tools to slice and dice on the submitted data to investigate memory leaks or other various performance issues.

and personally one of the best features, comparing profile to a previous time period or even a previous version.

The graphs are from Datadog.

The gopher art is from MariaLetta.

HTTP Connection churn in GO

Georgios Kampitakis — Sat, 21 Oct 2023 16:03:33 +0000

In this post, we are going to have a look at

What is connection churn
Common causes for connection churn in Go
Benefits of utilizing persistent connections
Methods for identifying connection churn and finally
Demo that will help us better understand how to fix it connection churn in Go.

What is connection churn

While searching for a definition of what connection churn is I came across with

A system is said to have a high connection churn when its rate of newly opened connections is consistently high and its rate of closed connections is consistently high

or in other words when a service frequently opens and closes network connections instead of efficiently utilizing persistent connections. Persistent connections enable multiple HTTP requests and responses to be sent and received over a single TCP connection.

Common causes for connection churn in Go

Not Reading and Closing HTTP body

One of the most common causes and easy to miss, is not reading Body to completion and not closing after done with it.

It may be counterintuitive to read the body especially when you don't care about the contents of it.

There is a linter that can at least catch the bodyclose.

Using the default HTTP client

Go HTTP defaults are not great for heavy use

Internally Go HTTP client is using an LRU cache for persistent connection pooling. Instead of closing a socket connection after an HTTP request, if the connection is marked as reusable, it will be put back into the pool and later if you try to make another request before the idle connection timeout (90 seconds by default) it will re-use the existing connection from the pull rather than creating a new one. This will keep the number of total socket connections low as long as the pool doesn't fill up. If the pool is full the client will create a new connection and use it.

What's the size of the connection pool?

Looking at the DeafaultTransport in HTTP Client MaxIdleConns sets the size of the pool to 100 connections, BUT there is a caveat. DefaultMaxIdleConnsPerHost is set to 2, what that means is even if the pool has a size of 100 connections, there is a cap of two open sockets per host. We will see this later in action with an example.

Benefits of utilizing persistent connections

From Wikipedia for HTTP persistent connection#Advantages.

Reduced latency in subsequent requests (no handshaking and no slow start).

An example of a production service, after fixing an issue that allowed reusing connections.

A drop in latency from ~1 sec to 400 ms

Reduced CPU usage and round-trips because of fewer new connections and TLS handshakes.

same production service as before.

TLS handshakes can be CPU-intensive by nature, so reducing the newly established connections and inherently spend less time in TLS handshakes reduces your service's CPU usage.

Enables HTTP pipelining of requests and responses.
Reduced network congestion (fewer TCP connections).
Errors can be reported without the penalty of closing the TCP connection.

Methods for identifying connection churn

Let's see some methods that can help us identify that a service is having a high connection churn.

High CPU usage on TLS

A good indicator that your service might be suffering from connection churn is when a high percentage of CPU is spent on TLS handshake.

The below CPU profile is an example of this phenomenon. ~38% of CPU time is spent on establishing TLS connections,a size comparable to the time spent running our service code.

same production service as before.

Monitoring

Monitoring platforms like Datadog provide tools for Network Monitoring.

An example below is a view from Network Performance Monitoring where a service is having high number of connections established and closed per second. This clearly shows a service suffering from connection churn.

same production service as before.

HTTP Trace

Another method is using httptrace which provides mechanisms to trace events within HTTP client requests.

An example usage where we print connection info, which includes info on whether the connection was Reused or not.

Netstat

Finally, we can use netstat which is a useful tool for inspecting connection statuses. In the command below we are running netstat every second and counting the number of all TCP connections at port 8080 that are in status time_wait.

The high number of time_wait statuses indicates the creation of lots of short-lived TCP connections.

From man netstat

TIME_WAIT
The socket is waiting after close to handle packets still in the network.

Demo

It's time for the demo with 2 examples in Go that can cause connection churn.

The setup is simple, we are going to run a server that will accept requests and a client sending requests from a specified number of goroutines (you can find the code below).

To observe the state of our connections, in two different shells, we are going to run netstat

watch -n 1 'netstat -n -p tcp | grep -i 8080 | grep -i time_wait | wc -l'

and

watch -n 1 'netstat -n -p tcp | grep -i 8080'

to see the list of connections in more detail.

Example-1

In the first example, we run the code as is, without reading or closing the body and with the default Go HTTP client configuration.

After around ~16k requests we are getting errors

Get "http://localhost:8080/message": dial tcp [::1]:8080: connect: connection refused

we can see the shell running netstat also having a high number of sockets at time_wait state.

Every 1.0s: netstat -n -p tcp | grep -i 8080

tcp4       0      0  127.0.0.1.49478        127.0.0.1.8080         TIME_WAIT
tcp4       0      0  127.0.0.1.49477        127.0.0.1.8080         TIME_WAIT
tcp4       0      0  127.0.0.1.49481        127.0.0.1.8080         TIME_WAIT
tcp4       0      0  127.0.0.1.49482        127.0.0.1.8080         TIME_WAIT
tcp4       0      0  127.0.0.1.49486        127.0.0.1.8080         TIME_WAIT
tcp4       0      0  127.0.0.1.49485        127.0.0.1.8080         TIME_WAIT
tcp4       0      0  127.0.0.1.49489        127.0.0.1.8080         TIME_WAIT
tcp4       0      0  127.0.0.1.49490        127.0.0.1.8080         TIME_WAIT
...

What's happening is, because we are not reading and closing any of the requests, eventually we run out of available ephemeral ports and we are not able to establish new connections.

We can fix this by closing and reading the request. After rerunning the code with the fix we can see on the netstat list with TPC connections only two connections as ESTABLISHED and the sockets with status time_wait remaining 0.

All our requests are handled by those two persistent connections successfully.

Every 1.0s: netstat -n -p tcp | grep -i 8080

tcp4       0      0  127.0.0.1.8080         127.0.0.1.61273        ESTABLISHED
tcp4       0      0  127.0.0.1.8080         127.0.0.1.61272        ESTABLISHED
tcp4       0      0  127.0.0.1.61273        127.0.0.1.8080         ESTABLISHED
tcp4       0      0  127.0.0.1.61272        127.0.0.1.8080         ESTABLISHED

Example-2

For the second example, we are increasing the number of goroutines from 2 to 3, with this change essentially doing 3 concurrent requests.

Running the code now with the new update, we are seeing a steady increase of sockets in time_wait status and two established connections as before.

Every 1.0s: netstat -n -p tcp | grep -i 8080

tcp4       0      0  127.0.0.1.8080         127.0.0.1.49213        ESTABLISHED
tcp4       0    102  127.0.0.1.49213        127.0.0.1.8080         ESTABLISHED
tcp4       0    128  127.0.0.1.8080         127.0.0.1.49198        ESTABLISHED
tcp4       0    102  127.0.0.1.49198        127.0.0.1.8080         ESTABLISHED
tcp4       0      0  127.0.0.1.8080         127.0.0.1.61787        TIME_WAIT
tcp4       0      0  127.0.0.1.8080         127.0.0.1.61751        TIME_WAIT
tcp4       0      0  127.0.0.1.61787        127.0.0.1.8080         
...

We are successfully reusing two of the connections but still creating new connections to close the short after.

The question of why is this happening can be answered by DefaultMaxIdleConnsPerHost. By increasing the concurrency to 3 requests, on every iteration we are able to reuse two of the connections (DefaultMaxIdleConnsPerHost is 2) but the 3rd connection can not be put into the idle pool due to http: putIdleConn: too many idle connections for host.

So the fix for this example would be adjusting MaxIdleConnsPerHost to something > 2. Rerunning the code after the fix we can see now 3 established connections and zero sockets at time_wait state.

Every 1.0s: netstat -n -p tcp | grep -i 8080

tcp4       0    128  127.0.0.1.8080         127.0.0.1.52440        ESTABLISHED
tcp4       0      0  127.0.0.1.8080         127.0.0.1.52439        ESTABLISHED
tcp4       0      0  127.0.0.1.8080         127.0.0.1.52438        ESTABLISHED
tcp4       0    102  127.0.0.1.52440        127.0.0.1.8080         ESTABLISHED
tcp4       0    102  127.0.0.1.52439        127.0.0.1.8080         ESTABLISHED
tcp4       0    102  127.0.0.1.52438        127.0.0.1.8080         ESTABLISHED

Appendix

Jul 3 '22 Comments: Answers: 3

In MaxConnsPerHost and MaxIdleConnsPerHost in GO HTTP package, is "Host" a domain(i.e., yahoo.com) or an IP address? The actual meaning of "Host" will affect my settings for the connection pool.

Open Full Question

The graphs are from Datadog.

The demo was inspired by Tuning the Go HTTP Client Settings for Load Testing.

The gopher art is from MariaLetta.

Advent of Code: Investigating performance improvements in Go

Georgios Kampitakis — Sat, 26 Aug 2023 21:16:55 +0000

I often enjoy trying to solve previous years' Advent of Code puzzles. If you don't know what Advent of Code is make sure you check it out.

TLDR: Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.

This post is about puzzle 2021/day-6 I was trying to solve that's not very interesting as a puzzle but I found it to be a really good example for exploring how to deep dive into your code's performance, find potential bottlenecks and improve it with tools that Go provides.

Each puzzle on Advent of Code contains two parts. You need to solve the 1st to move to the 2nd one and usually the 2nd is a more complicated or demanding variation of the 1st. This is the approach we are going to follow on this post as well.

Tools

The tools we are going to use are

Benchmarks for measuring and analyzing the performance of go code with help from go tool pprof.
Benchstat computes statistical summaries and compares benchmarks.

Part 1

I am not going to repeat the problem here for brevity, but if you read the puzzle carefully you can probably come up with a trivial solution as I did the 1st time.

func parseInput(s string) []int {
    data := strings.Split(strings.Split(s, "\n")[0], ",")
    fish := make([]int, len(data))

    for i, d := range data {
        fish[i] = MustAtoi(d)
    }

    return fish
}

func MustAtoi(s string) int {
    n, err := strconv.Atoi(s)
    if err != nil {
        panic(err)
    }
    return n
}


func Solve(input string, days int) int {
    fish := parseInput(input)

    for i := 0; i < days; i++ {
        tempFish := []int{}

        for j := range fish {
            if fish[j] == 0 {
                fish[j] = 6
                tempFish = append(tempFish, 8)
            } else {
                fish[j]--
            }
        }

        fish = append(fish, tempFish...)
    }

    return len(fish)
}

The approach for solving it is:

parsing the input and turning it into a list of numbers and then iterating for the number of days
for each day I decrease each fish's timer, if the timer reaches zero resets to 6
and on a temporary list add a new fish with timer 8 that at the end of the day is going to be appended on the main list.

That's pretty much it, successfully solves the Part 1 and it's fast enough.

ok advent-of-code/2021/go/day-6 0.226s

The purpose of this post though is not to just solve the puzzle.

Before updating the code we need a benchmark of the current implementation as a base

Note: Always measure before optimizing.

var sink int

func BenchmarkSolution(b *testing.B) {
    res := 0
    input := util.ReadFile("input.txt")

    b.ResetTimer()
    b.ReportAllocs()

    for i := 0; i < b.N; i++ {
        res = Solve(input, 80)
    }

    sink = res
}

Benchmarks

We can run the benchmark with

go test ./... -benchmem -run=^$ -v -bench ^Bench -memprofile naive.out -count=5 | tee naive.txt

Which will produce output similar to the following one:

pkg: advent-of-code/2021/go/day-6
BenchmarkSolution
BenchmarkSolution-10                 235           5210983 ns/op        31653556 B/op       1025 allocs/op
BenchmarkSolution-10                 231           5086091 ns/op        31653547 B/op       1025 allocs/op
BenchmarkSolution-10                 225           5146135 ns/op        31653549 B/op       1025 allocs/op
BenchmarkSolution-10                 231           5191141 ns/op        31653571 B/op       1025 allocs/op
BenchmarkSolution-10                 229           5104518 ns/op        31653558 B/op       1025 allocs/op
PASS
ok      advent-of-code/2021/go/day-6    8.847s

Note: the flag -memprofile memory profiling is enabled which can help to identify memory-related bottlenecks.

You can use the interactive CLI interface for navigating the profiling data

go tool pprof naive.out
# and then list the data for Solve function with 
list Solve

From this view we can dig into profiling data collected from the benchmark and it's easy to spot the 3 big offenders that allocate memory.

Reducing unnecessary memory allocations leads to improved performance (e.g memory allocation overhead, garbage collection cache friendly behaviour).

Normally while investigating performance issues, you run the benchmarks against every code change to verify your changes and that you actually improving your code, but for brevity we are going to discuss all the improvements and then run the benchmark to see the final result.

Improvements

The first offender is parseInput which is allocating an N sized integer slice, where N is the input of the puzzle. Instead of slice of integers []int we can use a slice of bytes []byte so we can store the same information with less memory
(and as a bonus we skip parsing from string to integer). We can do this due to the puzzle's constraints, input numbers are from 0 to 8.

The second offender can be seen if we do list parseInput

data := strings.Split(strings.Split(s, "\n")[0], ",") is as well allocating space that we throw right after. Instead, it's better parsing char by char.

Here is the updated parseInput function:

func parseInput(s string) []byte {
    fish := make([]byte, 0, len(s)/2)

    for i := 0; i < len(s); i++ {
        if s[i] == '\n' {
            break
        }
        if s[i] == ',' {
            continue
        }
        fish = append(fish, s[i])
    }

    return fish
}

The third offender is the tempFish slice. On every iteration, we are populating the slice only to throw it on the next step and create a new one and we are keep reallocating memory. We can reuse the same slice across iterations by just clearing the slice instead of creating a new one.

The updated code for the Solve function

func Solve(input string, days int) int {
    fish := parseInput(input)
    tempFish := []byte{}

    for i := 0; i < days; i++ {
        for j := range fish {
            if fish[j] == '0' {
                fish[j] = '6'
                tempFish = append(tempFish, '8')
            } else {
                fish[j]--
            }
        }

        fish = append(fish, tempFish...)
        tempFish = tempFish[:0]
    }

    return len(fish)
}

Results

Rerunning the benchmarks

 go test . -benchmem -run=^$ -v -bench ^Bench -count=5 -memprofile improved.out | tee improved.txt

generates the following, much improved, output

pkg: advent-of-code/2021/go/day-6
BenchmarkSolution
BenchmarkSolution-10                 388           2945361 ns/op         2193282 B/op         45 allocs/op
BenchmarkSolution-10                 410           2950991 ns/op         2193280 B/op         45 allocs/op
BenchmarkSolution-10                 409           2917432 ns/op         2193278 B/op         45 allocs/op
BenchmarkSolution-10                 404           2968872 ns/op         2193277 B/op         45 allocs/op
BenchmarkSolution-10                 409           2917535 ns/op         2193281 B/op         45 allocs/op
PASS
ok      advent-of-code/2021/go/day-6    7.725s

the benchmark is already looking better but benchstat can help to answer "by how much"

benchstat naive.txt improved.txt 
name         old time/op    new time/op    delta
Solution-10    5.20ms ± 7%    2.94ms ± 1%  -43.51%  (p=0.008 n=5+5)

name         old alloc/op   new alloc/op   delta
Solution-10    31.7MB ± 0%     2.2MB ± 0%  -93.07%  (p=0.008 n=5+5)

name         old allocs/op  new allocs/op  delta
Solution-10     1.02k ± 0%     0.04k ± 0%  -95.61%  (p=0.008 n=5+5)

So the new solution is 43.51% faster than the old one and requires 93.07% less memory to do the same job.

Part 2

For the second part, the puzzle is running for 256 days. Even with the previous improvements though, at least on my machine, I can't get a result (after waiting for 5 minutes I gave up).

It's time to think the solution a bit more carefully and use a more efficient algorithm, as the solution is O(days*fish) and the number of fish increases on every iteration. We need to find a way to keep the number of fish constant.

An example input of the puzzle

3,4,3,1,2,2,1,1 ....

We know that a fish can have a timer from 0 to 8 and notice all fish that have the same timer are "synchronized".This means we can group all the fish that have the same timer and change the O(days*fish) to O(days*8) which is equal to O(days).

We can achieve this by using a map, where the keys are the timer values, hence 0-8, and the values are the number of fish that have this as their timer. That would turn the above input to the following structure

{
  0: 0,
  1: 3,
  2: 2,
  3: 2,
  4: 1,
  5: 0,
  6: 0,
  7: 0,
  8: 0
}

and the size of the map will be constant regardless of the number of fish. Every day we are increasing the timers per group instead of one fish at a time.

We are keeping the findings from Part 1 and the new adjusted code

func Solve(input string, days int) int {
    fish := parseInput(input)
    tmpFish := make(map[uint8]int, 9)

    for i := 0; i < days; i++ {
        resetCount := 0

        for k, v := range fish {
            if k == 0 {
                resetCount += v
            } else {
                tmpFish[k-1] = v
            }
        }

        tmpFish[8] += resetCount
        tmpFish[6] += resetCount

        fish, tmpFish = tmpFish, fish
        clear(tmpFish)
    }

    counter := 0
    for _, v := range fish {
        counter += v
    }

    return counter
}

func parseInput(s string) map[uint8]int {
    fish := make(map[uint8]int, 9)

    for i := 0; i < len(s); i++ {
        if s[i] == '\n' {
            break
        }
        if s[i] == ',' {
            continue
        }
        fish[s[i]-'0']++
    }

    return fish
}

Results

This solution returns a result - if you recall 5 minutes were not enough to get a result previously - running the benchmark against it produces the following output:

pkg: advent-of-code/2021/go/day-6
BenchmarkSolution
BenchmarkSolution-10               34027             33479 ns/op            2373 B/op         85 allocs/op
BenchmarkSolution-10               35720             33435 ns/op            2374 B/op         85 allocs/op
BenchmarkSolution-10               35521             33486 ns/op            2374 B/op         85 allocs/op
BenchmarkSolution-10               35900             33392 ns/op            2373 B/op         85 allocs/op
BenchmarkSolution-10               35878             33553 ns/op            2373 B/op         85 allocs/op
PASS
ok      advent-of-code/2021/go/day-6    7.960s

Running the benchstat with the previous solution reveals even more impressive results than before improving the performance of the code up to 98.86% and using 99.89% less memory.

benchstat improved.txt new_algo.txt
name         old time/op    new time/op    delta
Solution-10    2.94ms ± 1%    0.03ms ± 0%  -98.86%  (p=0.008 n=5+5)

name         old alloc/op   new alloc/op   delta
Solution-10    2.19MB ± 0%    0.00MB ± 0%  -99.89%  (p=0.008 n=5+5)

name         old allocs/op  new allocs/op  delta
Solution-10      45.0 ± 0%      85.0 ± 0%  +88.89%  (p=0.008 n=5+5)

Bonus point

What if I told you there is one more one-line improvement that could give an increase in the code's performance by another 95.90% and reach zero memory allocations?

benchstat  new_algo.txt new_algo_array.txt 
name         old time/op    new time/op    delta
Solution-10    33.5µs ± 0%     1.4µs ± 2%   -95.90%  (p=0.008 n=5+5)

name         old alloc/op   new alloc/op   delta
Solution-10    2.37kB ± 0%    0.00kB       -100.00%  (p=0.008 n=5+5)

name         old allocs/op  new allocs/op  delta
Solution-10      85.0 ± 0%       0.0       -100.00%  (p=0.008 n=5+5)

You can try changing the data structure from map[uint8]int to []int, the rest of the code will still work. Why is that you might be wondering or you might have already guessed but I leave it as an exercise, use the profiling data and the answer is there.

hint: You can run benchmarks with -cpuprofile instead of -memprofile and check the top entries with go tool pprof

Feel free to post the answer in the comments 📝

Conclusion

The code for this puzzle was trivial, but the approach of measuring, finding bottlenecks, improving code and repeating until being satisfied with the results will be the same even in production setups. Learn to leverage the tools that the language and the ecosystem provide and you won't be disappointed.

The gopher art on the image is from MariaLetta.

Kubernetes: Dynamic Admission Control

Georgios Kampitakis — Sat, 13 May 2023 18:33:44 +0000

This is a demo for Kuberentes Dynamic Admission Control, exploring various concepts, capabilities and constraints, it's following A Guide to Kubernetes Admission Controllers which contains a lot of information around Admission Controllers and Kubernetes.

What is Dynamic Admission Control

Borrowing the definion from the Kubernetes page

In addition to compiled-in admission plugins, admission plugins can be developed as extensions and run as webhooks configured at runtime.

Admission webhooks are HTTP callbacks that receive admission requests and do something with them. You can define two types of admission webhooks, validating admission webhook and mutating admission webhook.

You can read some of the use cases for admission controllers.

In the demo we will focus on validating admission webhooks, but most of the principles apply on mutating as well.

Demo

We are going to run the demo locally on minikube.

Prerequisites

You will need some tools installed for following along

Clone code from Github
minikube
kubectl
docker
go (optional if you want to play with webhook server code)

Setting up minikube

We are going to spin up a minikube cluster by running

# start kubernetes cluster locally
minikube start

# enable needed addons
minikube addons enable metrics-server
minikube addons enable registry

We are setting up a local image registry so we don't rely on pushing/pulling the webhook server container image on a remote registry e.g. docker hub.

# run local registry
docker run --rm -it --network=host alpine ash -c "apk add socat && socat TCP-LISTEN:5000,reuseaddr,fork TCP:$(minikube ip):5000"

All the components for running and testing the webhook now should be ready.

Building and Deploying

The next step is to build the Webhook Docker image and deploy it alongside with the Admission Webhook configuration.

You will need to build and push the container to your local registry by running:

make docker-image-local

then you are ready to deploy the Validating Admision Webhook in your local minikube cluster with:

make deploy

The make deploy command does a couple of actions

Generates TLS Certificates, since a webhook must be served via HTTPS, we need proper certificates for the server. [ More ]
Creates a dedicated namespace for putting the Webhook Server Deployment and the ValidatingWebhookConfiguration. This useful if you want to skip your Webhook from "validating itself".
Creates a Kubernetes TLS secret with the generated key and certificate so the Webhook server can access.
Creates a Service for reaching the Webhook Server Pods at port 443.
Creates the Webhook Server Deployment.
Finally after everything is in place creates the ValidatingWebhookConfiguration.

The Webhook Server

The Webhook Server we are deploying is a simple Go HTTP server that accepts AdmissionReview POST requests at /validate, verifys the payload is correct.

Then checks the resource passed for review, in the demo case we are only care about POD creation, if it contains a team label set up. You can configure rules for what review requests are directed to your Webhook in ValidatingWebhookConfiguration object. More details on Matching requests: rules.

If this label team: <value> is missing the Webhook Server will return either a warning that the lable is missing or a failure that prevents the POD from scheduling depending on the ALLOW_SCHEDULING env var.

if _, exists := pod.Labels["team"]; !exists {
    admissionReviewResponse.Response.Allowed = allowScheduling
    if allowScheduling {
        admissionReviewResponse.Response.Warnings = []string{
            "Team label not set on pod",
        }
    }
    admissionReviewResponse.Response.Result = &metav1.Status{
        Status:  "Failure",
        Message: "Team label not set on pod",
    }

    log.Printf("Team label not set on pod: %s\n", pod.Name)
}

In the warning case you will see this message

Warning: Team label not set on pod

Also it's a good practise to skip validating namespaces that are used by Kubernetes, e.g. kube-public and kube-system, as you might cause a disruption of you cluster if you deny scheduling of resources there.

You can experiment with the Webhook Server code do changes and redeploy it. You can build a new docker image and push it to the local registry we set up earlier with

make docker-image-local

and then restart the deployment for creating new PODS with the new image

kubectl rollout restart deploy/webhook-server

Verifying everything Works

Okay let's check that everything works and see the Validating Webhook in practise.

First we can check that the Webhook Server is scheduled and healty. You can do it with

kubectl get pods -n webhook-demo

and the output should be similar to this

NAME                              READY   STATUS    RESTARTS   AGE
webhook-server-79c79b5877-d624n   1/1     Running   0          11m
webhook-server-79c79b5877-dpf4b   1/1     Running   0          11m

Then if we try to schedule a simple POD it should fail to schedule with the error

kubectl apply -f deployment/busy-box.yaml

Error from server: error when creating "deployment/busy-box.yaml": admission webhook "webhook-server.webhook-demo.svc" denied the request: Team label not set on pod

You can add a team label in the pod spec and try again successfully.

Observations

Namespace Selector

You might have noticed the code in the Webhook Server contains logic for skipping validation for a list of namespaces.

But ValidatingWebhookConfiguration supports natively limiting requests that reach to your Webhook Server depending on namespaceSelector.

I couldn't find a way to whitelist namespaces by their name. For using namespaceSelector you can add this to ValidatingWebhookConfiguration.

namespaceSelector:
  matchExpressions:
    - key: webhook-skip
      operator: DoesNotExist

and then mark all namespaces you want to skip with this label

kubectl label ns webhook-demo webhook-skip=true

Reliability and availability

Dynamic Admission Controller can be on the critical path, depending how you your Webhook configured e.g. can block various actions on Kubernetes resources.

This is a non exhaustive list of things you can do to increase the reliability and the availability of your admission controller.

The Dynamic Admission Controller is backed by an HTTP Server and a Deployment. This means all concepts for making a stateless deployment more reliable in Kubernetes ( if it is hosted in Kubernetes )
- Use load balancer in front of your Webhook server that provides High Availability
- Use Rolling Updates
- Pod Disruption Budget
- Rollback to an earlier Deployment if the current state is not stable
- Autoscalling for matching increasing demand [ HPA ]
Avoiding deadlock in self hosted webhooks
Avoiding operating on the kube-system namespace
It is recommended that admission webhooks should evaluate as quickly as possible (typically in milliseconds), since they add to API request latency. It is encouraged to use a small timeout for webhooks.
Setting Failure Policy depending on your needs.
And last but not least always monitoring and observing your Webhooh Server status and the Kube API server status. The API Server exposes Prometheus metrics from the /metrics endpoint e.g. apiserver_admission_webhook_admission_duration_seconds_sum,apiserver_admission_webhook_rejection_count.

Summary

Dynamic Admission Controllers provide great benefits for security and governance across your Kubernetes Cluster. This demo can help you get started building and deploying a Validating Admission Webhook.

Resources

Go Snaps: The Latest Updates and Improvements

Georgios Kampitakis — Mon, 27 Feb 2023 21:33:16 +0000

More than a year after the previous post about Go Snaps and there have been some improvements and a lot bug fixes.

Introduction

go-snaps is a Go library that makes snapshot testing in Go easier.

Snapshot testing is an important part of the unit testing process. It simplifies writing assertions for complex or dynamic output which can be time consuming and can catch regressions or unintended changes that otherwise might went unnoticed.

Updates to go-snaps

The improvements and updates in the library can be put into 3 categories, visual changes, new features and bug fixes.

Visual Changes

Visual changes are always the one that make the more difference in the experience of using the library. So I focused on making the diff output better more easy to distinguish where the regression is happening in a long output.

An example diff

or with long diffs

The second important visual change was on the the reporting of the library and the summary of the tests.

An example when Go Snaps identifies snapshots that are no longer used there is a message prompting cleaning them and showing which tests are marked for deletion.

Finally now the library supports NO_COLOR.

New Features

Two new big features landed on Go Snaps on is complimentary to the other.

It's snaps.MatchJSON and property Matchers.

MatchJSON method as the name indicates is for capturing snapshots for JSON structured data. It can accept a valid json in form of string or []byte or whatever value can be passed successfully on json.Marshal.

func TestJSON(t *testing.T) {
  type User struct {
    Age int
    Email string
  }
  snaps.MatchJSON(t, `{"user":"mock-user","age":10,"email":"mock@email.com"}`)
  snaps.MatchJSON(t, []byte(`{"user":"mock-user","age":10,"email":"mock@email.com"}`))
  snaps.MatchJSON(t, User{10, "mock-email"})
}

JSON will be saved in snapshot in pretty format for better readability.

Having a method that accepts structured data also enabled building another feature that can be useful in snapshot testing, property matchers.

MatchJSON can accept a list of arguments called Matchers. Matchers are functions that can act as property matchers and test values on a specific path of the JSON.

Currently Go Snaps has two builtin matchers

match.Any
match.Custom

match.Any can be convenient when some fields are dynamic and you have different snapshots on every run, e.g. containing a date or time field.

t.Run("should ignore fields", func(t *testing.T) {
    value := fmt.Sprintf(
        `{"user":"mock-user","age":10,"nested":{"now":["%s"]}}`,
        time.Now(),
    )
    snaps.MatchJSON(t, value, match.Any("nested.now.0"))
})

the resulting snapshot will be

{
 "age": 10,
 "nested": {
  "now": [
   "<Any value>"
  ]
 },
 "user": "mock-user"
}

so you can see we are ignoring the dynamic value there.

On the other hand, match.Custom allows you to build your own matcher that either validates or mutates a specific path.

t.Run("struct marshalling", func(t *testing.T) {
    type User struct {
        Name  string `json:"name"`
        Email string `json:"email"`
        Keys  []int  `json:"keys"`
    }

    u := User{
        Name:  "mock-user",
        Email: "mock-user@email.com",
        Keys:  []int{1, 2, 3, 4, 5},
    }

    snaps.MatchJSON(t, u, match.Custom("keys", func(val interface{}) (interface{}, error) {
        keys, ok := val.([]interface{})
        if !ok {
            return nil, fmt.Errorf("expected []interface{} but got %T", val)
        }

        if len(keys) > 5 {
            return nil, fmt.Errorf("expected less than 5 keys")
        }

        return val, nil
    }))
})

Custom matcher provides a lot of expressiveness and can support very complex use cases. In the future I am thinking of adding support for common used Matchers like type, or Date.

If you are using a Matcher and you think it should added in the library feel free to open an issue or contribute.

For more information around Matchers.

Bug Fixes

As more people start using the library noticed some edge cases ( or not that edge 😅 ) that the library was either not handling gracefully or was misbehaving.

I am not going into much detail but I will share the list of pull requests if you are interested.

fix: update isSkippedFile, replace file absolute match w/ regex match by @gparasyris in #48
fix: permission denied when running from test suite by @gkampitakis in #44
fix: issue with updating snapshots with expand chars, fixes #30
fix: issue with marking non child tests as skipped v0.3.0...v0.3.1
fix: issue with newlines when removing obsolete snaps v0.3.0...v0.3.1
fix: ignore end chars --- inside snapshot in #26

Conclusion

The latest changes and updates to go-snaps improved and simplified using snapshot testing into unit testing workflow. Alongside the bug fixes it makes the library more reliable for people to use it and rely on catching issues and potential regressions.

If you liked or found go-snaps useful you can leave a ⭐️.

Go Unit Test Coverage

Georgios Kampitakis — Fri, 20 Jan 2023 18:21:05 +0000

Go has really good support for unit testing out of the box. With the go test CLI and the testing std library you can do almost *everything.

One of the niceties it provides is code coverage. You can run your tests with a flag -cover or you can even have to collect the coverprofile and spin up an interactive UI and check your code and what branches are covered with

# run tests and create a coverprofile
go test ./... -coverprofile=cover.out
# open the interactive UI to check the Coverage Repor
go tool cover -html=cover.out

You should see something similar to this for your code

The red colour indicates that those branches are not covered at all by your unit tests and then with shades of green, you can see how much your code is covered.

What is this post about then

All this is good, what Go provides right away is at least amazing. But what I have found missing, is coverage threshold.

What is coverage threshold?

If you are coming from js ecosystem you might have encountered this in jest. With a coverage threshold, if your tests are less than a specified percentage, even if they are successful they will fail.

There is a long debate about whether high code coverage or what percentage of code coverage is the best. This post is not covering this debate. If you are interested I could do it in a separate post and share my thoughts.

Code coverage "tracking" can be useful, in many different scenarios. Adding a new feature, you write your unit tests but you miss covering some branches. If your coverage goes below the acceptable threshold you will be notified right away when running your tests. If someone is contributing to your team's project, it can show that you are expecting at least some basic unit tests for the code he wrote.

Coverage

So I took the opportunity and wrote a small util library, that does exactly that.

Given a threshold number if your package is below that, then it will fail your tests

You can find the code and the library in coverage. Have a look and tell me what you think.

* Of course some useful libraries can help you with boilerplate code like testify or go-snaps (shameless plug 😅)

Thanks for reading ❤️

Check file's git history even if renamed/moved

Georgios Kampitakis — Sun, 15 Jan 2023 19:19:25 +0000

From time to time, I find myself that I want to rename or move a file to a different folder, but I am reluctant. The reason is the git history of a file is more important than you might think, so moving it or renaming it, will mean you are going to lose the file history.

How to check a file history even if renamed/moved

git has a really useful flag that can help us.

You can check a file's history with git log and pass the --follow flag.

In practise for a project go-snaps, if you run

# passing --pretty=oneline for brevity 
git log --pretty=oneline internal/test/test.go

will show

<sha-id> chore: add examples for MatchJSON
<sha-id> feat: implement MatchJSON snapshot function (#49)

but running with the --follow flag

git log --pretty=oneline --follow internal/test/test.go

will result in

<sha-id>  chore: add examples for MatchJSON
<sha-id>  feat: implement MatchJSON snapshot function (#49)
<sha-id>  chore: change file structure
<sha-id>  fix: update deps and add more tests (#39)

There can be more things before moving a file as you can see in this example.

Why file history can be important you ask

In a commit ( or a series of commits ) there can be a lot of information that can explain decisions that were taken and why the code has evolved as it is right now. This information can be as valuable as the code itself so you can understand why I find --follow useful.

That was my TIL 😃

Go Test supports shuffling tests

Georgios Kampitakis — Sun, 08 Jan 2023 22:42:36 +0000

TLDR: Randomize the execution order of tests and benchmarks with -shuffle off,on,N flag.

As a general guideline, it's bad for unit tests to depend on each other and the order of them messing with the results.

A "trick" I have seen some codebases adopt to mitigate and catch issues with unit test results relying on order is instead of using arrays in table-driven tests using maps.


TestMyFunc(t *testing.T) {
    for name, _ := range map[string]struct{}{} {
        t.Run(name, func(t *testing.T) {
                ...
        })
    }
}

The reason behind this is, the order of map iteration is not guaranteed, so on different runs the tests might run in different order making it more difficult to rely on the order.

But as of Go 1.17, you can "Randomize the execution order of tests and benchmarks."

As go help testflag states

-shuffle off,on,N
        Randomize the execution order of tests and benchmarks.
        It is off by default. If -shuffle is set to on, then it will seed
        the randomizer using the system clock. If -shuffle is set to an
        integer N, then N will be used as the seed value. In both cases,
        the seed will be reported for reproducibility.

testing: add shuffle flag #10655

Snapshot testing in Go

Georgios Kampitakis — Mon, 03 Jan 2022 22:08:21 +0000

On my journey into learning Go, after some time spent learning the basic concepts I came across the testing. Go has a built-in testing command with go test and a testing package which combined can be used for unit testing.

It is really useful having native testing capabilities but Go only provides the minimum for writing unit tests. There are no assertions like Mocha or Jest for javascript ( Equal, toBe), repeated steps (e.g. beforeEach, afterEach) or mocking/spying and as you would expect by now there is no snapshot testing.

The community has built libraries on top of the standard testing package to provide some of the functionality like testify does.

Snapshot testing

Snapshot testing is mostly used when you want to test UIs, as stated in Jest Docs

Snapshot tests are a very useful tool whenever you want to make sure your UI does not change unexpectedly.

But snapshot testing can be useful in applications other than UI as well. Some examples could be

Database clients, where you want to diff multiple rows of data
Backend services, where you can diff API responses

and any application that can have a "state" and you would like to create a representation of it and compare it at a later time.

For more information around snapshot testing, there is a wide range of resources you could read for more specific applications and where it fits in unit testing.

Go snaps

So I decided to build a library that supports snapshot testing go-snaps and practice my Go skills.

It's simple to use

// example_test.go
package example

import (
  "testing"

  "github.com/gkampitakis/go-snaps/snaps"
)

func TestExample(t *testing.T) {
  snaps.MatchSnapshot(t ,"Hello World")
}

On the 1st run of your tests this will create a file __snapshots__/example_test.snap containing your snapshot


[TestExample - 1]
Hello World
---

On every other run, it will compare this snapshot with the parameters provided.

So if we update the call with

snaps.MatchSnapshot(t,
  "hello world",
  "another value",
  100,
)

our test will fail and return a message containing a diff view of the snapshot and params.

If we are sure this is not a regression in our code and we want to update the snapshots we can call again the go test with either flag -snaps.update=true or the env variable UPDATE_SNAPS=true.

Detecting obsolete snapshots

Apart from the standard snapshot functionality that go-snaps provides, it also can detect obsolete snapshots (snapshots that are no longer used). In order to enable this functionality you have to use TestMain and call the snaps.Clean().

func TestMain(t *testing.M) {
  v := t.Run()

  // After all tests have run `go-snaps` can check for not used snapshots
  snaps.Clean()

  os.Exit(v)
}

The reason for using TestMain, is because go-snaps needs to be sure that all tests have run so it can keep track of which snapshots were not called and mark them as obsolete.

Again you can remove obsolete tests and files by calling the -snaps.update=true or UPDATE_SNAPS=true.

Conclusion

In the repository, you can find examples and more information in README.

Hope you find go-snaps useful, any feedback is welcome.

Acknowledgments

Go Snaps used Jest Snapshoting and Cupaloy as inspiration.

Jest is a full-fledged Javascript testing framework and has robust snapshoting features.
Cupaloy is a great and simple Go snapshoting solution.

If you liked or found go-snaps useful you can leave a ⭐️.

DEV Community: Georgios Kampitakis

[Boost]

Tracking down high memory usage in Node.js

Georgios Kampitakis ・ Dec 14 '24

[Boost]

Tracking down high memory usage in Node.js

Georgios Kampitakis ・ Dec 14

Tracking down high memory usage in Node.js

Contents

Context

Approach

Understand the code

Replicate the issue in isolation

Capture profiles from staging services

Verify the fix

Results

Conclusion

Memory leaks in Go

What is a memory leak

Why are memory leaks bad

Common causes for memory leaks in Go

Unbounded resource creation

Long lived references

Goroutines

Deferring function calls

Not stopping time.Ticker

Methods for identifying memory leaks

Investigate memory leaks

Profiling

Continuous profiling

HTTP Connection churn in GO

What is connection churn

Common causes for connection churn in Go

Not Reading and Closing HTTP body

Using the default HTTP client

Benefits of utilizing persistent connections

Methods for identifying connection churn

High CPU usage on TLS

Monitoring

HTTP Trace

Netstat

Demo

Example-1

Example-2

Appendix

Advent of Code: Investigating performance improvements in Go

Tools

Part 1

Benchmarks

Improvements

Results

Part 2

Results

Bonus point

Conclusion

Kubernetes: Dynamic Admission Control

What is Dynamic Admission Control

Demo

Prerequisites

Setting up minikube

Building and Deploying

The Webhook Server

Verifying everything Works

Observations

Namespace Selector

Reliability and availability

Summary

Resources

Go Snaps: The Latest Updates and Improvements

Introduction

Updates to go-snaps

Visual Changes

New Features

Bug Fixes

Conclusion

Go Unit Test Coverage

What is this post about then

What is coverage threshold?

Coverage

Check file's git history even if renamed/moved

Not stopping `time.Ticker`