DEV Community: Shannon

Using closures to measure call stack depth in Go

Shannon — Fri, 23 Jan 2026 01:18:01 +0000

Overview

I recently started reading through Grokking Algorithms as a primer before starting to review more DS&A, and chapter 4 is about quicksort, which recursively uses a divide-and-conquer method to sort an array in ascending order.

The pictures in the book (one linked below) are really great at describing the call stack, and it got me thinking about easy ways to accomplish viewing how deep the call stack goes recursively when calling these functions.

<< full disclosure - this little snippet is more for myself to remember it later :-P >>

Basics

The idea around quicksort is that you pick a pivot and then split the arrays into smaller and larger numbers from there. Depending on where you pick your pivot, it has an effect on the overall speed of your algorithm.

In the above example, this essentially shows the worst case. The algorithm is already sorted, and it just traverses over the entire array every time. This would be the equivalent to O(n^2). Anyway, that isn't too important, and I more wanted to just document how you could use a closure in go to document this.

Zero index pivot

In go, it's pretty common to utilize anonymous functions, and this Go By Example snippet has a good example of doing that here.

Now let's take a look at an example of passing in a variable defined before the anonymous function to keep track of this when the array is already sorted:

zeroPivotMaxDepth := 1
var zeroPivot func([]int, int) []int

zeroPivot = func(nums []int, depth int) []int {
    if depth > zeroPivotMaxDepth {
        zeroPivotMaxDepth = depth
    }

    if len(nums) < 2 {
        return nums
    }

    pivot := nums[0]
    less := []int{}
    greater := []int{}
    for _, v := range nums[1:] {
        if v <= pivot {
            less = append(less, v)
        } else {
            greater = append(greater, v)
        }
    }

    sorted := zeroPivot(less, depth+1)
    sorted = append(sorted, pivot)
    sorted = append(sorted, zeroPivot(greater, depth+1)...)
    return sorted
}

sortedArray := zeroPivot([]int{1, 2, 3, 4, 5, 6, 7, 8}, zeroPivotMaxDepth)

fmt.Printf("sorted array: %v\n", sortedArray)
fmt.Printf("max stack depth: %d\n", zeroPivotMaxDepth)

As we can see, zeroPivotMaxDepth is instantiated before the anonymous function definition, and then it is referenced in the code. Finally, it is recursively passed in to the further sorted commands and incremented when it goes down a level.

Of note, I started the count at 1 to emulate the above example.

When I run this, I get a stack depth of 8:

➜ go run main.go                                                                                                                                     (current-context is not set)
sorted array: [1 2 3 4 5 6 7 8]
max stack depth: 8

And if I update the array to []int{1, 8, 3, 4, 6, 2, 5, 7}, we should see the stack depth change:

➜ go run main.go                                                                                                                                     (current-context is not set)
sorted array: [1 2 3 4 5 6 7 8]
max stack depth: 6

Middle index Pivot

Now let's do the same thing but instead we're going to use the middle pivot. This is essentially the best case for divide and conquer. I'll just show the updated code to keep it a little more concise, but note the algorithm changing based on the pivot location.

middlePivotMaxDepth := 1
var middlePivot func([]int, int) []int

middlePivot = func(nums []int, depth int) []int {
    if depth > middlePivotMaxDepth {
        middlePivotMaxDepth = depth
    }

    if len(nums) < 2 {
        return nums
    }

    pivotIndex := len(nums) / 2
    pivot := nums[pivotIndex]
    lower := []int{}
    higher := []int{}

    for i, v := range nums {
        if i == pivotIndex {
            continue
        }

        if v <= pivot {
            lower = append(lower, v)
        } else {
            higher = append(higher, v)
        }
    }

    sorted := []int{}
    sorted = append(sorted, middlePivot(lower, depth+1)...)
    sorted = append(sorted, pivot)
    sorted = append(sorted, middlePivot(higher, depth+1)...)

    return sorted
}

And when we run this, here's what we see below.

When it's already sorted:

➜ go run main.go                                                                                                                                     (current-context is not set)
sorted array: [1 2 3 4 5 6 7 8]
max stack depth: 4

And when it's not:

➜ go run main.go                                                                                                                                     (current-context is not set)
sorted array: [1 2 3 4 5 6 7 8]
max stack depth: 5

And that's it :-) Hope it was interesting!

Utilizing the Go 1.25 Flight Recorder with tracing middleware

Shannon — Tue, 06 Jan 2026 21:35:09 +0000

Overview
Utilizing a FlightRecorder
Code
Run it
Looking at the trace file
Wrap up

Overview

I recently came across this video about profiling request latency from Datadog, and I realized I hadn’t really ever seen or used this Go package. Profiling production services is not an easy task, and the concept of continuous profiling has become a lot more popular in recent years. See below, courtesy of this CNCF article about the topic.

Anyway, Go has some more recent functionality that allows this kind of tracing with low overhead, meaning it won’t consume 10-20% of your CPU but instead 1-2%. Additionally, it allows snippets of trace samples, which can consume significant disk and RAM if you’re recording for an extended period.

The purpose of me writing this article is to give more a layman’s approach and another example from the following two articles (tbh, I’d suggest reading those two articles over this one :-P)

I want to state up front I am not an expert on this and still have a lot of questions myself. The hope is to eventually see more examples of utilizing this in production and just digging into this more myself.

Utilizing a FlightRecorder

The FlightRecorder (FR) object is new in Go 1.25 that captures execution trace data with low overhead, as mentioned above. It takes two potential configuration fields, from a struct called FlightRecorderConfig. You can either choose an age or maximum bytes. The article above goes into this as well, but I’ll be choosing max age since I have a simulated estimate for how long the request will take.

It is quite straightforward to set up a new FR:

// Set up the flight recorder.
    fr := trace.NewFlightRecorder(trace.FlightRecorderConfig{
        MinAge: 5 * time.Second,
    })

Code

For this code, I decided to simulate the tracing as middleware because it feels like an appropriate, realistic example. I will get into some specific code details below this, but the pseudocode is essentially this:

curl /ping
HTTP server is called → middleware handler is called → ping handler is called
Generate randomized number between 0-2
If rand number == 0, simulate syscall blocking through sleeps and CPU constraints by spawning many goroutines but limiting the CPU to 1
if request > .5seconds, take a trace sample
Write trace to trace.[number-here].out

Code comments specific to this page are written as n_[comment]

package main

import (
    "bytes"
    "fmt"
    "log"
    "math/rand"
    "net/http"
    "os"
    "runtime"
    "runtime/trace"
    "sync/atomic"
    "time"
)

// app holds the flight recorder, snapshot count, and max snapshots to take
type app struct {
    fr *trace.FlightRecorder
    // 1_ limit the number of snapshots to take
    snapshotCount atomic.Int32
    maxSnapshots  int32
}

// doWork simulates work
func doBlockingWork() {
    for i := 0; i < 100; i++ {
        time.Sleep(10 * time.Millisecond) // sleep blocking syscall
    }
}

// doSchedulerWork simulates work that schedules goroutines
func doSchedulerWork() {
    // 2_ purposely cap the number of CPUs to 2
    runtime.GOMAXPROCS(2)
    for i := 0; i < 100; i++ {
        go func() {
            _ = fmt.Sprintf("hello")
        }()
    }
    time.Sleep(1 * time.Second)
}

// handlePing responds with a "pong" message
func (a *app) handlePing(w http.ResponseWriter, r *http.Request) {
    start := time.Now()

    // 3_ 1/3 chance of doing work
    if rand.Intn(3) == 0 {
        doBlockingWork()
        doSchedulerWork()
    }

    w.WriteHeader(http.StatusOK)
    fmt.Fprintf(w, "pong, duration=%s\n", time.Since(start))
}

// tracingMiddleware is a middleware that takes a snapshot of the trace if the request takes longer than 500ms
func (a *app) tracingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        // 4_ serve the request (the /ping endpoint)
        next.ServeHTTP(w, r)
        if time.Since(start) > 500*time.Millisecond {
            // 5_ take a snapshot of the trace if > 500ms
            a.takeSnapshot()
        }
    })
}

// takeSnapshot writes the trace to a file
func (a *app) takeSnapshot() {
    count := a.snapshotCount.Add(1)
    // 6_ limit the total number of snapshots. this could rapidly fill up disk space if not limited.
    if count <= a.maxSnapshots {
        filename := fmt.Sprintf("trace.%d.out", count)
        log.Printf("Taking trace %d/%d and saving to file %s", count, a.maxSnapshots, filename)
        var b bytes.Buffer
        _, err := a.fr.WriteTo(&b)
        if err != nil {
            log.Print(err)
            return
        }

        // write to a file for each snapshot
        if err := os.WriteFile(filename, b.Bytes(), 0o755); err != nil {
            log.Print(err)
            return
        }
    }
}

func main() {
    fr := trace.NewFlightRecorder(trace.FlightRecorderConfig{
        // 7_ set the minimum age of the trace to 5 seconds, which is roughly double the duration of a long request
        MinAge: 5 * time.Second,
    })
    fr.Start()
    defer fr.Stop()

    app := &app{
        fr:            fr,
        snapshotCount: atomic.Int32{},
        maxSnapshots:  2,
    }

    // run http server
    http.Handle("/ping", app.tracingMiddleware(http.HandlerFunc(app.handlePing)))
    log.Fatal(http.ListenAndServe(":5051", nil))
}

Let’s talk about each of the labeled comments now:

Basically, I’m showing that there should be some kind of hard cap for how many traces you’d want to take. If this is a production service, it could either be a massive trace or you run the risk of filling up disk space. If containerized, that disk space might share space with other containers and you run the risk of evictions, etc.
We’re capping the number of CPUs where all the goroutines cannot schedule automatically. This will essentially make the wall clock time be much longer than the execution time. We should be able to see it in the trace
Basically, just generate that this won’t happen every time. It’s a contrived example of some requests taking longer than others
All handlers have the ServeHTTP(w, r) method, which is how you call the middleware. In this case, our only other handler is handlePing but this is wired up in such a way you could call it on any Handler
This will only take a snapshot if the request is long. In a normal GET /ping it will take a very short amount of time
Basically the same as mentioned in number 1. If I was writing this in production, I might have it disabled by default and then add a --enable-tracing or something similar to start the continuous profiling
As mentioned above, the suggestion is double what you expect the duration to be. I expect it will be around 2-2.5 seconds, so we set it to 5

Run it

Let’s spin this thing up.

For the server: go run main.go

And in another terminal for your client: while true; do curl localhost:5051/ping; sleep 1; done

You should see something like this for your client side:

pong, duration=4.333µs
pong, duration=2.417µs
pong, duration=2.126059041s
pong, duration=1.167µs
pong, duration=1.458µs
pong, duration=2.140110958s
pong, duration=1.333µs
pong, duration=1.416µs
pong, duration=1.833µs

And then there should be some generated trace files now too (in the directory your server was running):

➜  tracing git:(main) ✗ ls trace.*                                                                                                                                                       (current-context is not set)
trace.1.out     trace.2.out

Looking at the trace file

We use the following command to look at the trace file: go tool trace trace.1.out. It should auto spin up a tab running on localhost, but the output of the command will also give a link.

I am not an expert by any means on the Chromium profiling UI, but let’s examine the goroutines spawned by doSchedulerWork function.

Goroutine analysis → main.doSchedulerWork.func1 → Execution time and Scheduled wait time are listed.

We can see here that the scheduled wait time is an order of magnitude greater than the execution time (nanoseconds vs. microseconds). Additionally, we can see all 100 of the goroutines spawned. From there, we can go back one screen and analyze the goroutines that took the longest amount of it. It is important to click in and see more details, as there's a breakdown of execution time, wall clock time, scheduling time, etc.

Wrap up

I hope this was an interesting read, and if you find more on this topic and using the package with some better examples, send them my way!

Utilizing OS-recommended caching in Go to store files on disk

Shannon — Sat, 27 Dec 2025 05:59:34 +0000

Overview
Local caching
API Scaffolding
Creating the HTTP client
Seeing it in action

Overview

I recently started writing a wrapper around an API that rate limits you after 60 requests/minute. This is pretty low in the grand scheme of software, but its responses effectively don't change after a single query. With that in mind, it seemed like a great time to implement a local cache that is stored on disk and platform agnostic.

Local caching

There's a bunch of great caching options that range from storing things in memory at startup to using Redis or other database services, but this tutorial will keep things straight forward: storing responses locally on disk.

To start, let's briefly look at one of go's methods for placing cache files, regardless of the operating system you may be using. os.UserCacheDir() in the os go package is what we'll use for this part. If we look at the function, it states UserCacheDir returns the default root directory to use for user-specific cached data. Users should create their own application-specific subdirectory within this one and use that. Going into that function, we can see there's specific logic for Mac, Windows, Linux, and more.

A quick peek at the function:

...
    switch runtime.GOOS {
    case "windows":
        dir = Getenv("LocalAppData")
        if dir == "" {
            return "", errors.New("%LocalAppData% is not defined")
        }

    case "darwin", "ios":
        dir = Getenv("HOME")
        if dir == "" {
            return "", errors.New("$HOME is not defined")
        }
        dir += "/Library/Caches"
...

As I'm using a Mac, I thought it would be interesting to show what that looks like. The command (ls $HOME/Library/Caches/slucas-caching-testing) generates the following response:

user_1.json user_2.json

API Scaffolding

I want to frame this from the perspective of querying an API, so we'll set up some scaffolding to generate responses for essentially GET /users/1, GET /users/2, and so on. I won't get super into detail on this because it's setting up an httptest server and just generating responses. Nevertheless, it's worth noting.

type Response struct {
    ID   int    `json:"id"`
    Name string `json:"name"`
}

// getResponseByIndex returns a specific response by index (0-3)
func getResponseByIndex(index int) ([]byte, error) {
    responses := make([]Response, 4)
    names := []string{"User1", "User2", "User3", "User4"}
    for i, name := range names {
        responses[i].ID = i + 1 // Use 1-indexed IDs
        responses[i].Name = name
    }

    return json.Marshal(responses[index])
}

// setupFakeServer creates a test HTTP server that serves stub data
func setupFakeServer() *httptest.Server {
    handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Set content type
        w.Header().Set("Content-Type", "application/json")

        // Extract the user ID from the path
        parts := strings.Split(r.URL.Path, "/")
        if len(parts) != 3 {
            http.NotFound(w, r)
            return
        }

        // Get integer of userID
        userID, err := strconv.Atoi(parts[2])
        if err != nil || userID < 1 || userID > 4 {
            http.NotFound(w, r)
            return
        }

        // Get the response corresponding to the user ID (convert to 0-indexed)
        jsonData, err := getResponseByIndex(userID - 1)
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }

        // Write the response
        w.WriteHeader(http.StatusOK)
        w.Write(jsonData)
    })

    return httptest.NewServer(handler)
}

Above, we're doing the following:

Setting up some mock responses through the Response data type
Creating a new handler func that splits out the path and looks for a specific user
Writes a JSON response

Creating the HTTP Client

Next up, we'll implement our client, which is just a wrapper around an HTTP client with a directory name.

type CacheClient struct {
    httpClient *http.Client
    cacheDir   string
}

And a constructor to set up both of these pieces.

// NewCacheClient creates a HTTP client with cache directory initialized
func NewCacheClient() (*CacheClient, error) {
    cacheDir, err := os.UserCacheDir()
    if err != nil {
        return nil, fmt.Errorf("failed to get cache dir: %w", err)
    }

    cacheDir = filepath.Join(cacheDir, "slucas-caching-testing")
    if err := os.MkdirAll(cacheDir, 0755); err != nil {
        return nil, fmt.Errorf("failed to create cache dir: %w", err)
    }

    return &CacheClient{
        httpClient: &http.Client{
            Timeout: 10 * time.Second,
        },
        cacheDir: cacheDir,
    }, nil
}

A couple of points to note about this function.

We're doing the suggested method from the UserCachedDir function: Users should create their own application-specific subdirectory within this one and use that by creating a directory for the cached files at startup.
We're explicitly setting up our struct and constructor to mention Cache in the name. It would be totally reasonable to also implement a normal client that has no caching. We'll be writing methods below to get a user, which is tied to the CacheClient struct.

Caching

First, we're setting up some methods to find, read, and write from the cache.

// getCachePath returns the cache file path for a given user ID
func (c *CacheClient) getCachePath(userID int) string {
    return filepath.Join(c.cacheDir, fmt.Sprintf("user_%d.json", userID))
}

// getCachedResponse retrieves a cached response for a user ID
func (c *CacheClient) getCachedResponse(userID int) ([]byte, error) {
    cachePath := c.getCachePath(userID)
    data, err := os.ReadFile(cachePath)
    if err != nil {
        if os.IsNotExist(err) {
            return nil, nil // Cache miss, not an error
        }
        return nil, fmt.Errorf("failed to read cache: %w", err)
    }
    return data, nil
}

// saveToCache saves a response to cache for a user ID
func (c *CacheClient) saveToCache(userID int, data []byte) error {
    cachePath := c.getCachePath(userID)
    return os.WriteFile(cachePath, data, 0644)
}

All of this is pretty straightforward, but the key parts are that we have a way to save json responses as a file on disk. It's a relatively naive approach where we aren't using a mutex to prevent concurrent writes to the same file or adding an expiration, but it's fine in this concise example.

And finally, we create our caching logic in a GetUser method!

// GetUser retrieves user data, checking cache first, then making HTTP request if needed
func (c *CacheClient) GetUser(baseURL string, userID int) (*Response, error) {
    // Check cache first
    cachedData, err := c.getCachedResponse(userID)
    if err != nil {
        return nil, err
    }
    if cachedData != nil {
        fmt.Printf("User %d found in cache\n", userID)
        var resp Response
        if err := json.Unmarshal(cachedData, &resp); err != nil {
        } else {
            return &resp, nil
        }
    }

    // Cache miss, make HTTP request
    url := fmt.Sprintf("%s/user/%d", baseURL, userID)
    httpResp, err := c.httpClient.Get(url)
    if err != nil {
        return nil, fmt.Errorf("failed to make request: %w", err)
    }
    defer httpResp.Body.Close()

    body, err := io.ReadAll(httpResp.Body)
    if err != nil {
        return nil, fmt.Errorf("failed to read response: %w", err)
    }

    // Save to cache
    if err := c.saveToCache(userID, body); err != nil {
        fmt.Printf("Warning: failed to save to cache: %v\n", err)
    }

    // Parse and return response
    var resp Response
    if err := json.Unmarshal(body, &resp); err != nil {
        return nil, fmt.Errorf("failed to parse response: %w", err)
    }

    return &resp, nil
}

I think this is all quite readable as well, but it never hurts to walk through step-by-step.

Check for a cached response. If it exists, return
Cache missed. Run GET request
Save to cache
Return JSON response

Seeing it in action

func main() {
    // Set up httptest server
    server := setupFakeServer()
    defer server.Close()

    fmt.Printf("Server running at: %s\n", server.URL)

    // Create a client with caching enabled
    client, err := NewCacheClient()
    if err != nil {
        fmt.Printf("Error creating client: %v\n", err)
        return
    }

    // Example: Get user 1 (will fetch from server and cache)
    fmt.Println("\nFirst request (should fetch from server):")
    user, err := client.GetUser(server.URL, 1)
    if err != nil {
        fmt.Printf("Error getting user: %v\n", err)
        return
    }
    fmt.Printf("User: %+v\n", user)

    // Example: Get user 1 again (should use cache)
    fmt.Println("\nSecond request (should use cache):")
    user, err = client.GetUser(server.URL, 1)
    if err != nil {
        fmt.Printf("Error getting user: %v\n", err)
        return
    }
    fmt.Printf("User: %+v\n", user)
}

I'll clean up my environment to simulate starting from scratch: rm $HOME/Library/Caches/slucas-caching-testing/*

And from here, we should see that the first result is pulled from the HTTP server, saved, and then our second request is gotten from the cache:

✗ ./caching                            (current-context is not set)
Server running at: http://127.0.0.1:60219

First request (should fetch from server):
User: &{ID:1 Name:User1}

Second request (should use cache):
User 1 found in cache
User: &{ID:1 Name:User1}

And that's it! I hope it was useful seeing a little contrived example. From here, we could put in mutexes, add expirations, pre-load certain data, add hit rates to know the most commonly queried endpoints. Productionizing something opens up a whole new can of worms!

I'm dropping the full code below as well.

All code

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/http/httptest"
    "os"
    "path/filepath"
    "strconv"
    "strings"
    "time"
)

type CacheClient struct {
    httpClient *http.Client
    cacheDir   string
}

type Response struct {
    ID   int    `json:"id"`
    Name string `json:"name"`
}

// getResponseByIndex returns a specific response by index (0-3)
func getResponseByIndex(index int) ([]byte, error) {
    responses := make([]Response, 4)
    names := []string{"User1", "User2", "User3", "User4"}
    for i, name := range names {
        responses[i].ID = i + 1 // Use 1-indexed IDs
        responses[i].Name = name
    }

    return json.Marshal(responses[index])
}

// setupFakeServer creates a test HTTP server that serves stub data
func setupFakeServer() *httptest.Server {
    handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Set content type
        w.Header().Set("Content-Type", "application/json")

        // Extract the user ID from the path
        parts := strings.Split(r.URL.Path, "/")
        if len(parts) != 3 {
            http.NotFound(w, r)
            return
        }

        // Get integer of userID
        userID, err := strconv.Atoi(parts[2])
        if err != nil || userID < 1 || userID > 4 {
            http.NotFound(w, r)
            return
        }

        // Get the response corresponding to the user ID (convert to 0-indexed)
        jsonData, err := getResponseByIndex(userID - 1)
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }

        // Write the response
        w.WriteHeader(http.StatusOK)
        w.Write(jsonData)
    })

    return httptest.NewServer(handler)
}

// NewCacheClient creates a HTTP client with cache directory initialized
func NewCacheClient() (*CacheClient, error) {
    cacheDir, err := os.UserCacheDir()
    if err != nil {
        return nil, fmt.Errorf("failed to get cache dir: %w", err)
    }

    cacheDir = filepath.Join(cacheDir, "slucas-caching-testing")
    if err := os.MkdirAll(cacheDir, 0755); err != nil {
        return nil, fmt.Errorf("failed to create cache dir: %w", err)
    }

    return &CacheClient{
        httpClient: &http.Client{
            Timeout: 10 * time.Second,
        },
        cacheDir: cacheDir,
    }, nil
}

// getCachePath returns the cache file path for a given user ID
func (c *CacheClient) getCachePath(userID int) string {
    return filepath.Join(c.cacheDir, fmt.Sprintf("user_%d.json", userID))
}

// getCachedResponse retrieves a cached response for a user ID
func (c *CacheClient) getCachedResponse(userID int) ([]byte, error) {
    cachePath := c.getCachePath(userID)
    data, err := os.ReadFile(cachePath)
    if err != nil {
        if os.IsNotExist(err) {
            return nil, nil // Cache miss, not an error
        }
        return nil, fmt.Errorf("failed to read cache: %w", err)
    }
    return data, nil
}

// saveToCache saves a response to cache for a user ID
func (c *CacheClient) saveToCache(userID int, data []byte) error {
    cachePath := c.getCachePath(userID)
    return os.WriteFile(cachePath, data, 0644)
}

// GetUser retrieves user data, checking cache first, then making HTTP request if needed
func (c *CacheClient) GetUser(baseURL string, userID int) (*Response, error) {
    // Check cache first
    cachedData, err := c.getCachedResponse(userID)
    if err != nil {
        return nil, err
    }
    if cachedData != nil {
        fmt.Printf("User %d found in cache\n", userID)
        var resp Response
        if err := json.Unmarshal(cachedData, &resp); err != nil {
        } else {
            return &resp, nil
        }
    }

    // Cache miss, make HTTP request
    url := fmt.Sprintf("%s/user/%d", baseURL, userID)
    httpResp, err := c.httpClient.Get(url)
    if err != nil {
        return nil, fmt.Errorf("failed to make request: %w", err)
    }
    defer httpResp.Body.Close()

    body, err := io.ReadAll(httpResp.Body)
    if err != nil {
        return nil, fmt.Errorf("failed to read response: %w", err)
    }

    // Save to cache
    if err := c.saveToCache(userID, body); err != nil {
        fmt.Printf("Warning: failed to save to cache: %v\n", err)
    }

    // Parse and return response
    var resp Response
    if err := json.Unmarshal(body, &resp); err != nil {
        return nil, fmt.Errorf("failed to parse response: %w", err)
    }

    return &resp, nil
}

func main() {
    // Set up httptest server
    server := setupFakeServer()
    defer server.Close()

    fmt.Printf("Server running at: %s\n", server.URL)

    // Create a client with caching enabled
    client, err := NewCacheClient()
    if err != nil {
        fmt.Printf("Error creating client: %v\n", err)
        return
    }

    // Example: Get user 1 (will fetch from server and cache)
    fmt.Println("\nFirst request (should fetch from server):")
    user, err := client.GetUser(server.URL, 1)
    if err != nil {
        fmt.Printf("Error getting user: %v\n", err)
        return
    }
    fmt.Printf("User: %+v\n", user)

    // Example: Get user 1 again (should use cache)
    fmt.Println("\nSecond request (should use cache):")
    user, err = client.GetUser(server.URL, 1)
    if err != nil {
        fmt.Printf("Error getting user: %v\n", err)
        return
    }
    fmt.Printf("User: %+v\n", user)

    // Example: Get user 2
    fmt.Println("\nRequest for user 2:")
    user2, err := client.GetUser(server.URL, 2)
    if err != nil {
        fmt.Printf("Error getting user: %v\n", err)
        return
    }
    fmt.Printf("User: %+v\n", user2)
}

Writing events to pods using client-go

Shannon — Thu, 27 Oct 2022 01:10:53 +0000

In the past, I've written controllers utilizing Kubebuilder, where the instantiation of an event recorder is basically done for you with a mgr.GetRecorder() method in the controller manager.

When writing a small binary with client-go, however, I found it surprisingly difficult. It took digging through kubernetes source code to find a couple examples, and I thought I'd document it here.

Scaffolding

Before running the code, let's spin up a pod in the default namespace kubectl run nginx --image=nginx --namespace default

We will be attaching sample events to this pod.

Code

Full code here since some of the packages are specifically named

package main

import (
    "context"
    "path/filepath"
    "time"

    log "github.com/sirupsen/logrus"
    corev1 "k8s.io/api/core/v1"
    v1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/runtime"
    "k8s.io/client-go/kubernetes"
    typedv1core "k8s.io/client-go/kubernetes/typed/core/v1"
    "k8s.io/client-go/rest"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/tools/record"
    "k8s.io/client-go/util/homedir"
)

func main() {
    kubeconfig := filepath.Join(homedir.HomeDir(), ".kube", "config")
    config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
    if err != nil {
        log.Info("connecting with config in-cluster")
        config, err = rest.InClusterConfig()
        if err != nil {
            log.Fatal(err)
        }
    }
    if err != nil {
        log.Fatal(err)
    }

    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        log.Fatal(err)
    }

    pod, err := clientset.CoreV1().Pods("default").Get(context.TODO(), "nginx", metav1.GetOptions{})
    if err != nil {
        log.Fatal(err)
    }

    scheme := runtime.NewScheme()
    _ = corev1.AddToScheme(scheme)

    // all the good events stuff is here
    eventBroadcaster := record.NewBroadcaster()
    eventBroadcaster.StartStructuredLogging(4)
    eventBroadcaster.StartRecordingToSink(&typedv1core.EventSinkImpl{Interface: clientset.CoreV1().Events("")})
    eventRecorder := eventBroadcaster.NewRecorder(scheme, v1.EventSource{})
    eventRecorder.Event(pod, corev1.EventTypeNormal, "is this thing on", "is this thing on")
    eventBroadcaster.Shutdown()
    time.Sleep(2 * time.Second)
}

Things worth noting:

You must add the corev1 scheme for the pod, as it is a corev1 object type. If you are using any other object type, make sure to add the appropriate scheme.
I added a time.Sleep() because of the asynchronous nature of this. Without some wait time, the process will exit too quickly and no events will populate.
eventRecorder.Eventf() is a lot more useful than my above example
If anyone understands the internal workings of k8s events better, please send some articles my way ;)

Seeing the events with kubectl

We can see it by querying events:

(⎈|kind-kind:default)learning/write-events-client-go » k get events -n default | grep "is this thing on"
71s         Normal   is this thing on   pod/nginx   is this thing on

But more interestingly, we can also see them attached to the pod object itself:

(⎈|kind-kind:default)learning/write-events-client-go » k describe pod nginx | grep -A 10 Events
Events:
  Type    Reason            Age   From               Message
  ----    ------            ----  ----               -------
  Normal  Scheduled         37s   default-scheduler  Successfully assigned default/nginx to kind-control-plane
  Normal  Pulling           37s   kubelet            Pulling image "nginx"
  Normal  Pulled            36s   kubelet            Successfully pulled image "nginx" in 1.112899376s
  Normal  Created           36s   kubelet            Created container nginx
  Normal  Started           36s   kubelet            Started container nginx
  Normal  is this thing on  10s                      is this thing on

Interfaces satisfying interfaces

Shannon — Thu, 13 Jan 2022 01:55:23 +0000

<< this post is mostly just a refresher for myself >>

Interfaces are often confusing for me. When I started parsing through some of the core/v1 kubernetes code, something stuck out to me: there are a lot of interfaces in here. Some only needing a single method to be satisfied.

Within this, a common theme was interfaces satisfying interfaces. This is a weak point of mine, so I crafted up a quick example to jog my memory.

Here's the code:

func main(){
    var a nested
    c := cat{}
    a = c
    a.Speak()
    d := dog{}
    a = d
    a.Speak()
}

type nested interface {
    random
}

type random interface {
    animal
}

type animal interface {
    Speak()
}

type cat struct{}
type dog struct{}

func (c cat) Speak() {
    fmt.Println("meow")
}

func (d dog) Speak() {
    fmt.Println("woof")
}

As you can see, there are three interfaces here: nested, random, and animal. However, only one of these has an actual method attached to it: animal. The other 2 are simply requiring other interfaces to be satisfied.

From there, I set up two object types, cat and dog, which both have the Speak() method. As you can see, it just prints out a statement.

Once in the main function, I set up a nested variable, which is an interface. This interface only requires an object that satisfies random which requires animal which requires any object type that has a method Speak(). In this case, bot the cat and dog objects do.

From there, I set the a variable to a dog or cat variable and call the methods.

This, however, will not work if we try to call a.Speak() before assigning an object that satisfies all of these conditions. If, for instance, I had the below code:

func main(){
    var a nested
    a.Speak()
}

It would cause a panic because the interface is currently <nil>.

Anyways, it was a good refresher on interfaces within Go for myself :)

Circumvent STDIN when installing packages with apt

Shannon — Mon, 20 Dec 2021 03:28:44 +0000

Did you know there's an entire Unix command dedicated to generating y over and over? Or that you can make your apt package manager be completely noninteractive when installing or updating packages?

I recently went through this process when installing some packages and man pages in an Ubuntu Docker image and decided to write up a quick blurb about it for reference.

Here is the Dockerfile I started with:

FROM ubuntu:latest

WORKDIR /app

RUN apt-get update && apt-get install -y \
    openssl \
    curl \
    git \ 
    vim \
    dnsutils \
    man-db

CMD ["bash"]

Unfortunately, this command fails because dnsutils tries to aggregate some information about your timezone region. Because of this, I wasn't able to just add a -y and call it good. After some Googling, I came across DEBIAN_FRONTEND=noninteractive, which forces zero interaction while updating or installing packages with apt. It will accept the default answer to all questions, which can be super useful in Dockerfiles, scripts, and I'm sure a million other things. So with this in mind, I updated the Dockerfile to include this as an environment variable:

FROM ubuntu:latest

WORKDIR /app

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y \
    openssl \
    curl \
    git \ 
    vim \
    dnsutils \
    man-db

CMD ["bash"]

Voila! First step solved.

However, I wanted one more thing in this Docker image by default, which is man pages. In order to get these, I need to run unminimize to expand all of the pages. Unfortunately, neither unminimize -y or unminimize --yes worked by default.

After some more Googling, I came across the yes Unix binary, which is incredibly straightforward: it just endlessly types y. For instance, you can just run yes in your terminal and watch it spam y endlessly.

In this case, I added RUN yes | unminimize to accept all of the defaults and run through the package installation, which is piping in y over and over to the command. And just like that, I have a working Docker image that can be spun up with some networking, man pages, and other stuff working by default!

Here is the final Dockerfile:

FROM ubuntu:latest

WORKDIR /app

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y \
    openssl \
    curl \
    git \ 
    vim \
    dnsutils \
    man-db

RUN yes | unminimize

CMD ["bash"]

And it could be run with something like this: docker build -t shannon:v0; docker run -it shannon:v0

Parse your escaped JSON with jq and fromjson

Shannon — Mon, 15 Nov 2021 23:49:15 +0000

Recently, I was looking at secrets manager in AWS and used the AWS CLI in order to read a secret in my terminal, such as: aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:[acct-id]:secret:testing/slack/slack_api_key-y8FHjv --profile [profile] --region us-west-2

This gives the following response:

{
    "ARN": "arn:aws:secretsmanager:us-west-2:[account-id]:secret:testing/slack/slack_api_key-y8FHjv",
    "Name": "testing/slack/slack_api_key",
    "VersionId": "a9a1fc45-80d5-45d0-8884-27d359b55ebd",
    "SecretString": "{\"SLACK_API_KEY\":\"itsasecret\"}",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": "2021-11-15T15:32:59.737000-08:00"
}

As you can see, the response gave me escaped JSON with \ in front of every " character for the actual contents of the secret. Because I wanted to use this programmatically, I needed to parse the JSON further with jq. However, I wasn't sure how to actually do this with escaped characters. Thankfully, jq has built-in functionality for this!

Within the manual, search for Convert to/from JSON. This will give a small example on how to use it. Similarly, I'm going to show how to use it with secrets manager below.

So, we have a JSON response right now that is parseable, so let's use jq to shrink this response some: aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:[account-id]:secret:testing/slack/slack_api_key-y8FHjv --profile [profile] --region us-west-2 | jq .SecretString

This now gives us:

"{\"SLACK_API_KEY\":\"itsasecret\"}"

Let's take it a step further and get rid of the escaped characters(for brevity's sake, I'm going to only show the jq pieces from here on): | jq '.SecretString | fromjson'

Here's my response:

{
  "SLACK_API_KEY": "itsasecret"
}

Now this is parseable too! Let's take only the value: | jq '.SecretString | fromjson | .SLACK_API_KEY'

Here's our response:

"itsasecret"

Finally, I want this to not have quotes around it, so I'll pass in the -r flag for jq: | jq -r '.SecretString | fromjson | .SLACK_API_KEY'

Voila! So how can we actually use this? Well, I could easily pass this in as an env var or variable into a script to utilize the value without hardcoding it into a file.

Unit testing Stdout in Go

Shannon — Fri, 24 Sep 2021 05:40:16 +0000

Figuring out how to unit test functions that print to the terminal can be something of a daunting task. Thankfully, it's actually pretty simple due to the magic of interfaces in Go!

This post is going to be short and sweet on how to test output by capturing it in a bytes.Buffer. Below, I have a function that accepts a string, prints out a statement, and then returns.

func SayHello(w io.Writer, name string) {
    fmt.Fprintf(w, "Hi, my name is %s\n", name)
}

The critical component of this function is the io.Writer interface variable being passed in. Because it is an interface, this function will accept any data type that implements the method Write(b []byte) (n int, err error).

This interface is implemented in the io package and the entire code can be seen here. Most importantly, here is the code below for the interface:

type Writer interface {
    Write(p []byte) (n int, err error)
}

So how do we print out this function to the terminal? We pass in os.Stdout, which has its own Write method! This can be seen here.

To print this out to our terminal, just add the following in the main() function.

func main() {
    SayHello(os.Stdout, "Shannon")
}

Now, we can move onto unit testing this specific function, which just outputs a name and greeting. In order to test it, we need to capture its output and then turn it into a string from a slice of bytes. To do this, we'll utilize the bytes.Buffer struct, which has a Write method to satisfy the Writer interface.

Here is the full test code in order to compare these:

func TestSayHello(t *testing.T) {
    tt := []struct {
        description string
        name        string
        expect      string
    }{
        {"Empty", "", "Hi, my name is \n"},
        {"Name1", "Shannon", "Hi, my name is Shannon\n"},
        {"Name2", "Shug", "Hi, my name is Shug\n"},
    }

    for _, tc := range tt {
        t.Run(tc.description, func(t *testing.T) {
            var output bytes.Buffer
            SayHello(&output, tc.name)
            if tc.expect != output.String() {
                t.Errorf("got %s but expected %s", output.String(), tc.expect)
            }
        })
    }
}

You'll notice that we are declaring a Buffer here: var output bytes.Buffer and then passing that buffer into the SayHello function. However, we are actually passing in the address of the buffer. This is due to the Buffer method for Write being a pointer receiver, which can be seen here.

And that's it! Run go test -v to see that all tests pass, and you have successfully captured the output of a command in order to unit test the output without actually returning the string value.

Testing Vault in Go

Shannon — Mon, 23 Aug 2021 17:41:39 +0000

Recently, I wrote some code for a CLI that reads all the secrets in a Vault path, and I went down the road of figuring out how to actually unit test some of my functions that are reaching out to Vault via API calls.

Importantly, my functions should not be reaching out to a real instance of Vault to do this testing. As a core tenet of unit testing, we shouldn't be reaching out to real APIs, live databases, etc. in unit tests. So, this meant I really had two options:

Mock out the methods that are communicating directly with the Vault API using interfaces and dependency injection.
Spin up a test Vault server, write secrets into it, and have the API calls run against this test server

Luckily, both were viable options! I am going to run through both options as a way to work through mocking out these methods, but the second method is clearly superior, in my opinion. Additionally, this is how Vault is actually being tested.

Mocking the Vault Client to test Read() and List() methods

First, let's set up some scaffolding below. All of this can go in main.go:

var (
    ErrSecretNotFound    = errors.New("no secret not found at given path")
    ErrVaultVarsNotFound = errors.New("VAULT_TOKEN and VAULT_ADDR environment variables must be set")
)

// 1
type Logicaler interface {
    List(path string) (*api.Secret, error)
    Read(path string) (*api.Secret, error)
}

// 2
type VaultClient struct {
    client Logicaler
}

// 3
func NewVaultClient() (*VaultClient, error) {
    token := os.Getenv("VAULT_TOKEN")
    vault_addr := os.Getenv("VAULT_ADDR")

    if token == "" || vault_addr == "" {
        return &VaultClient{}, ErrVaultVarsNotFound
    }

    config := &api.Config{
        Address: vault_addr,
    }

    client, err := api.NewClient(config)
    if err != nil {
        return &VaultClient{}, err
    }

    client.SetToken(token)

    return &VaultClient{
        client: client.Logical(),
    }, nil
}

// 4
func (v *VaultClient) ReadSecret(endpoint string) ([]string, error) {
    // 5
    secret, err := v.client.Read(endpoint)
    if err != nil {
        return []string{}, err
    }
    if secret == nil {
        return []string{}, ErrSecretNotFound
    }

    list := []string{}
    for _, v := range secret.Data["data"].(map[string]interface{}) {
        list = append(list, v.(string))
    }

    return list, nil
}

// 6
func (v *VaultClient) ListSecret(path string) ([]string, error) {
    // 7
    secret, err := v.client.List(path)
    if err != nil {
        return []string{}, err
    }
    if secret == nil {
        return []string{}, ErrSecretNotFound
    }

    list := []string{}
    for _, v := range secret.Data["keys"].([]interface{}) {
        list = append(list, v.(string))
    }

    return list, nil
}

Some notes on the code above to make it comprehensible:

We are setting up an interface that requires two methods to implicitly be satisfied: Read and List. It's idiomatic in Go to append er onto interfaces, so I opted for this method.
Create a VaultClient struct that hosts a single field: the interface Logicaler.
Create a new Vault client that establishes a real connection to a Vault server. You can think of this as a constructor.
Method of the VaultClient to read a secret in Vault
This part is important! It is the actual communication with Vault. Notice this is one of the two methods in the interface to be satisfied. When mocking, you want to search out the functions or methods that are communicating directly with the API to be mocked.
Same as 4. We are listing secrets in a Vault path.
Same as 5! This is the other method that must be mocked and satisfied in the interface.

Let's create some additional testing scaffolding in our main_test.go file before explaining how we're going to mock this out:

package main

import (
    "fmt"
    "testing"

    kv "github.com/hashicorp/vault-plugin-secrets-kv"
    "github.com/hashicorp/vault/api"
    vaulthttp "github.com/hashicorp/vault/http"
    "github.com/hashicorp/vault/sdk/logical"
    hashivault "github.com/hashicorp/vault/vault"
)

// 1
type MockVault struct{}

// 2
func (mv *MockVault) List(path string) (*api.Secret, error) {
    return &api.Secret{}, nil
}

func (mv *MockVault) Read(path string) (*api.Secret, error) {
    return &api.Secret{}, nil
}

// 3
func NewMockVaultClient() *VaultClient {
    return &VaultClient{
        client: &MockVault{},
    }
}

We're creating a Mock Vault client to emulate the actual client's methods (Read and List).
We're creating those methods (although they return nothing at the moment) to satisfy the Logicaler interface.
Create a mock Vault client were the MockVault struct is passed into the client field, where it is looking for a variable of type Logicaler.

With this code, check everything is compiling: go build.

Now that we have a basic structure, we can create some mock data to be returned in the *api.Secret object. Let's update the Read(path string) method to return mock data:

func (mv *MockVault) Read(path string) (*api.Secret, error) {
    return &api.Secret{
        Data: map[string]interface{}{
            "data": map[string]interface{}{
                "value": "fakedata",
            },
        },
    }, nil
}

The Data field is the actual contents of the secret data. Unfortunately, this is a bit tough to read because of the multiple nested map[string]interface{} objects. It is worth peeking at the definition of the *api.Secret object in the Vault package to see how this is defined.

We have some mock data being returned in our Read() method now, so let's write a test for this!

func TestReadSecret(t *testing.T) {
    vc := NewMockVaultClient()
    want := []string{"fakedata"}
    got, _ := vc.ReadSecret("")

    if !reflect.DeepEqual(want, got) {
        t.Errorf("got %v but want %v", got, want)
    }
}

As you can see, we are now creating a NewMockVaultClient() instead of the actual client. Then, we are calling the method ReadSecret(path string), which calls out to the method Read(path string). This method is implemented in the Logicaler interface and thus calls out to the mocked (mv *MockVault) Read(path string) method instead of the actual VaultClient! Finally, we're testing that the arrays are the exact same.

If this is confusing, try implementing the same thing with TestListSecret afterwards. Now, you can test this is working with go test --run TestReadSecret.

Spinning up a test Vault server in Go to test Read() and List()

Let's take a step back and think about the differences between mocking and using a real Vault test client. Because we are not mocking the client, we no longer need an interface to mock out the methods. We'll still be using the real methods but calling out to a test server spun up during the testing process. Thus, we can get rid of the interface and adjust some of our code.

Here is the scaffolding:

type VaultClient struct {
    // 1
    client *api.Client
}

func NewVaultClient() (*VaultClient, error) {
    token := os.Getenv("VAULT_TOKEN")
    vault_addr := os.Getenv("VAULT_ADDR")

    if token == "" || vault_addr == "" {
        return &VaultClient{}, ErrVaultVarsNotFound
    }

    config := &api.Config{
        Address: vault_addr,
    }

    client, err := api.NewClient(config)
    if err != nil {
        return &VaultClient{}, err
    }

    client.SetToken(token)

    return &VaultClient{
        // 2
        client: client,
    }, nil
}

func (v *VaultClient) ReadSecret(endpoint string) ([]string, error) {
    // 3
    secret, err := v.client.Logical().Read(endpoint)
    if err != nil {
        return []string{}, err
    }
    if secret == nil {
        return []string{}, ErrSecretNotFound
    }

    list := []string{}
    for _, v := range secret.Data["data"].(map[string]interface{}) {
        list = append(list, v.(string))
    }

    return list, nil
}

func (v *VaultClient) ListSecret(path string) ([]string, error) {
    // 4
    secret, err := v.client.Logical().List(path)
    if err != nil {
        return []string{}, err
    }
    if secret == nil {
        return []string{}, ErrSecretNotFound
    }

    list := []string{}
    for _, v := range secret.Data["keys"].([]interface{}) {
        list = append(list, v.(string))
    }

    return list, nil
}

I've marked a couple spots to note again, so let's walk through these.

We are now passing in an *api.Client from the Vault package instead of the Logicaler interface. Because we are no longer mocking anything, we don't need an interface.
See the client being passed in. We are no longer calling the Logical() method.
As you can see, the method calls look a bit different. Each time we use the client, we are running the Logical() method, which is used to return the client for logical-backend API calls.
Same as 3. You now need to call client.Logical().* to run the Vault commands.

So, we have our code that is calling out to Vault. Now we need to write some testing that spins up a testing Vault server to run this against. Let's create the scaffolding for the test cluster:

// CreateTestVault spins up a Vault server and tests against
// an actual Vault instance. Currently, this is only set up for kv v2
func createTestVault(t testing.TB) *hashivault.TestCluster {
    t.Helper()

    // CoreConfig parameterizes the Vault core config
    coreConfig := &hashivault.CoreConfig{
        LogicalBackends: map[string]logical.Factory{
            "kv": kv.Factory,
        },
    }

    cluster := hashivault.NewTestCluster(t, coreConfig, &hashivault.TestClusterOptions{
        // Handler returns an http.Handler for the API. This can be used on
        // its own to mount the Vault API within another web server.
        HandlerFunc: vaulthttp.Handler,
    })
    cluster.Start()

    // Create KV V2 mount on the path /test
    // It starts in cluster mode, so you just pick one of the three clients
    // In this case, Cores[0] is just always picking the first one
    if err := cluster.Cores[0].Client.Sys().Mount("test", &api.MountInput{
        Type: "kv",
        Options: map[string]string{
            "version": "2",
        },
    }); err != nil {
        t.Fatal(err)
    }

    return cluster
}

Note: most of this is grabbed from this GitHub issue. It's worth reading through in totality and shows multiple ways to set up test clusters.

The comments lay most of the details out, but the important bit is that we're setting up a test Vault cluster with the /test path mounted.

We'll take the testing of ReadSecret piece by piece, so here is the initial setup that creates the Vault cluster, makes a client, and waits some time post-mount with the new cluster.

func TestReadSecrets(t *testing.T) {
    cluster := createTestVault(t)
    defer cluster.Cleanup()
    vaultClient := cluster.Cores[0].Client // only need a client from 1 of 3 clusters

    _ = &VaultClient{
        client: vaultClient,
    }

    // time buffer required after new mount
    // https://github.com/hashicorp/terraform-provider-vault/issues/677#issuecomment-609116328
    // Code 400: Errors: Upgrading from non-versioned to versioned data. This backend will be unavailable for a brief period and will resume service shortly.
    time.Sleep(2 * time.Second)
}

Of note, we are passing in only a single Core of the client. In the cluster spinup, there are 3 cores. We are just choosing the first.

Next, we need to write in some fake data into the path /test in order to read the secrets later on. Let's write some data:

// set up sample data to write into vault
testData := []struct {
        path  string
        key   string
        value string
    }{
        {"test/data/test0", "key0", "data0"},
        {"test/data/test1", "key1", "data1"},
        {"test/data/test2", "key2", "data2"},
    }

    // write k/v data pairs into vault
    for _, v := range testData {
        _, err := vc.client.Logical().Write(v.path, map[string]interface{}{
            "data": map[string]interface{}{
                v.key: v.value,
            },
        })
        if err != nil {
            t.Fatal(err)
        }
    }

In the above code, we are creating a struct that will be looped over three times to write sample data into Vault.

Finally, we create a test table that reads secrets and confirms their validity.

testTable := []struct {
        name       string
        endpoint   string
        key        string
        want       []string
        vaultError error
    }{
        // 1
        {
            name:       "find a k/v match",
            endpoint:   "test/test0",
            key:        "key0",
            want:       []string{"data0"},
            vaultError: nil,
        },
        // 2
        {
            name:     "do not find a secret",
            endpoint: "test/test123", key: "test_0_key",
            want:       []string{"test_0_data"},
            vaultError: ErrSecretNotFound,
        },
    }

    for _, tc := range testTable {
        t.Run(tc.name, func(t *testing.T) {
            secrets, err := vc.ReadSecret(tc.endpoint)
            if err != tc.vaultError {
                t.Fatal(err)
            }

            // 3
            for i := 0; i < len(secrets); i++ {
                if secrets[i] != tc.want[i] {
                    assert.Equal(t, tc.want[i], secrets[i])
                }
            }
        })
    }

We're finding a valid match here. Thus, we shouldn't have any kind of error returned, so that's why vaultError is nil.
We're not finding a valid match here. However, Vault by default does not return an error when there isn't a match. It returns an empty secret object. However, we have logic built in to accommodate for this in vc.ReadSecret. Thus, we're checking for ErrSecretNotFound.
Instead of using reflect, we are looping over the secrets object. When an error is returned and there is no secret, this will simply not have a loop, as no comparison is needed.

Phew! That is a lot of code. But we now have a working iteration of a test Vault cluster being created, having secrets, written in, and then running our methods against it.

And for posterity, I'm going to post the entire main_test.go file below because there were so many additions. À très bientôt!

package main

import (
    "testing"
    "time"

    kv "github.com/hashicorp/vault-plugin-secrets-kv"
    "github.com/hashicorp/vault/api"
    vaulthttp "github.com/hashicorp/vault/http"
    "github.com/hashicorp/vault/sdk/logical"
    hashivault "github.com/hashicorp/vault/vault"
    "gotest.tools/assert"
)

// CreateTestVault spins up a Vault server and tests against
// an actual Vault instance. Currently, this is only set up for
// kv v2. Mostly copied from this github issue:
// https://github.com/hashicorp/vault/issues/8440
func createTestVault(t testing.TB) *hashivault.TestCluster {
    t.Helper()

    // CoreConfig parameterizes the Vault core config
    coreConfig := &hashivault.CoreConfig{
        LogicalBackends: map[string]logical.Factory{
            "kv": kv.Factory,
        },
    }

    cluster := hashivault.NewTestCluster(t, coreConfig, &hashivault.TestClusterOptions{
        // Handler returns an http.Handler for the API. This can be used on
        // its own to mount the Vault API within another web server.
        HandlerFunc: vaulthttp.Handler,
    })
    cluster.Start()

    // Create KV V2 mount on the path /test
    // It starts in cluster mode, so you just pick one of the three clients
    // In this case, Cores[0] is just always picking the first one
    if err := cluster.Cores[0].Client.Sys().Mount("test", &api.MountInput{
        Type: "kv",
        Options: map[string]string{
            "version": "2",
        },
    }); err != nil {
        t.Fatal(err)
    }

    return cluster
}

func TestReadSecrets(t *testing.T) {
    cluster := createTestVault(t)
    defer cluster.Cleanup()
    vaultClient := cluster.Cores[0].Client // only need a client from 1 of 3 clusters

    vc := &VaultClient{
        client: vaultClient,
    }

    // time buffer required after new mount
    // https://github.com/hashicorp/terraform-provider-vault/issues/677#issuecomment-609116328
    // Code 400: Errors: Upgrading from non-versioned to versioned data. This backend will be unavailable for a brief period and will resume service shortly.
    time.Sleep(2 * time.Second)

    testData := []struct {
        path  string
        key   string
        value string
    }{
        {"test/data/test0", "key0", "data0"},
        {"test/data/test1", "key1", "data1"},
        {"test/data/test2", "key2", "data2"},
    }

    // write k/v data pairs into vault
    for _, v := range testData {
        _, err := vc.client.Logical().Write(v.path, map[string]interface{}{
            "data": map[string]interface{}{
                v.key: v.value,
            },
        })
        if err != nil {
            t.Fatal(err)
        }
    }

    testTable := []struct {
        name       string
        endpoint   string
        key        string
        want       []string
        vaultError error
    }{
        // 1
        {
            name:       "find a k/v match",
            endpoint:   "test/data/test0",
            key:        "key0",
            want:       []string{"data0"},
            vaultError: nil,
        },
        // 2
        {
            name:     "do not find a secret",
            endpoint: "test/data/test123", key: "test_0_key",
            want:       []string{},
            vaultError: ErrSecretNotFound,
        },
    }

    for _, tc := range testTable {
        t.Run(tc.name, func(t *testing.T) {
            secrets, err := vc.ReadSecret(tc.endpoint)
            if err != tc.vaultError {
                t.Fatal(err)
            }

            // 3
            for i := 0; i < len(secrets); i++ {
                if secrets[i] != tc.want[i] {
                    assert.Equal(t, tc.want[i], secrets[i])
                }
            }
        })
    }
}

Creating your own Homebrew tap and grabbing binaries from S3

Shannon — Thu, 29 Jul 2021 20:24:16 +0000

Recently, I set up my own homebrew in order to easily distribute a kubectl plugin binary that I created. By creating my own homebrew, it allows me to easily download the binaries to any home dev environment without having to pull down the code from GitHub, compile it, and then add it somewhere in my $PATH in order to utilize it globally.

I ran into a few hiccups along the way, so I figured it's best to document it for next time (or if someone stumbles into this post)

Set up your own homebrew repo in GitHub

For my personal repo, I opted to name it homebrew-homebrew, as you can see here: https://github.com/lucassha/homebrew-homebrew.

The reason I chose this is because of the naming convention when tapping a new repository in brew. If, for instance, I named my repository homebrew-lucassha then I would have to run brew tap lucassha/lucassha. brew is specifically looking for repositories prepended with homebrew-.

Now, I'll tap my instance: brew tap lucassha/homebrew. You can then see all tapped repositories with just brew tap.

Add the formula to download from S3

Once you have the repo set up, you'll need to add a formula to it. Here is my specific example:

class ShowSecrets < Formula
    desc "CLI to show decoded Kubernetes secrets"
    homepage "https://github.com/lucassha/show-secrets"
    url "https://lucassha-show-secrets-releases.s3.us-west-2.amazonaws.com/releases/v0.0.1"

    # to get the latest SHA:
    # find latest release: aws s3api list-objects --bucket lucassha-show-secrets-releases | jq '.Contents[].Key'
    # download release: aws s3 cp s3://lucassha-show-secrets-releases/releases/v0.0.1 . 
    # get sha: shasum -a 256 v0.0.1
    sha256 "36018cff6708dea3587c086e366ff80bed7a2750ddda9aa7e8c1af8311b17649"

    depends_on "kubernetes-cli"

    def install
        bin.install "show-secrets"
        # add kubectl binary to make it a kubectl plugin
        bin.install "kubectl-show-secrets"
    end

    def caveats; <<-EOS
        This tool may be used as a standalone CLI or a kubectl plugin
        # example
        show-secrets -n default
        kubectl show secrets -n default
    EOS
    end
end

The important bits to highlight here are the URL and SHA. Specifically, the URL is from S3 instead of a GitHub .tgz file. Similarly, the SHA is tarball that brew is using and not the binary. You must create a tarball.

S3 Policies

Interestingly, I was surprised that a fully public bucket still needs a GetBucket policy in order to let an anonymous user download it. You can test this with a public bucket and just running curl against it. It will fail. Similarly, you'll get a 403 error when trying to install it via brew. Because of this, a bucket policy is needed. Here is the JSON I added for the public bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow Public Access to All Objects",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::lucassha-show-secrets-releases/*"
        }
    ]
}

Install the binary

Now, you should be able to install the package!
brew install lucassha/homebrew/show-secrets

This should create two binaries, which can be seen with this command: which *show-secrets

C'est tout!

Using reflect.DeepEqual to compare data structures in Go

Shannon — Wed, 14 Jul 2021 22:22:38 +0000

Intro

When I started using Go, I would compare two slices by writing a for loop and iterating over both to make sure they were equal. After seeing some sample code using the reflect package, I realized this is totally unnecessary. We can compare slices, maps, structs (anything!) simply by passing them into reflect.DeepEqual(x,y), which reports whether two variables are "deeply equal."

Slice/Array Example

Let's take this function that reverses a slice:

func ReverseSlice(s []string) []string {
    reversedSlice := make([]string, len(s))

    for i, j := len(s), 0; i > 0; i, j = i-1, j+1 {
        reversedSlice[j] = s[i-1]
    }

    return reversedSlice
}

Now, I have written a test that shows the usage or comparing two slices:

func TestReverseSlice(t *testing.T) {
    got := ReverseSlice([]string{
        "person1",
        "person2",
        "person3",
        "person4",
    })

    want := []string{
        "person4",
        "person3",
        "person2",
        "person1",
    }

    if !reflect.DeepEqual(got, want) {
        t.Errorf("got %q slice but expected %q", got, want)
    }
}

So, pretty straightforward! However, it's really important to note that the two arrays must be in the same order. They cannot have the same values but be in a different order. The function comments specifically state: Array values are deeply equal when their corresponding elements are deeply equal.

Map Example

Maps are a little different! By default, they are not going to have a specific order. For instance, a map[string]string is going to be constructed with a potentially random order.

So how are they compared with DeepEqual? They will both need to either be nil, or not nil, have the same length, and the same corresponding key/values.

In this case, I'm going to just use main() to showcase a few of these points with print statements:

func main() {
    m1, m2 := make(map[string]string), make(map[string]string)

    m1["person1"] = "person1"
    m1["person2"] = "person2"

    m2["person2"] = "person2"
    m2["person1"] = "person1"

    fmt.Printf("type=%T -- addr=%p -- len=%d\n", m1, &m1, len(m1))
    fmt.Printf("type=%T -- addr=%p -- len=%d\n", m2, &m2, len(m2))

    if reflect.DeepEqual(m1, m2) {
        fmt.Println("they are equal")
    } else {
        fmt.Println("they are NOT equal")
    }
}

Specifically, we're looking to compare the type, address, and length of the map. We'll need the type and length to be the same, but the addresses should be different (but not nil).

Once running, you should see output similar to this:

type=map[string]string -- addr=0xc0000b0018 -- len=2
type=map[string]string -- addr=0xc0000b0020 -- len=2
they are equal

I'd also encourage reading the full documentation for this function in the Go docs :)

Consuming and decoding JSON in Terraform

Shannon — Tue, 06 Jul 2021 17:01:05 +0000

Intro

HCL, the language of Terraform, has come a long way since 0.11. With its many new releases, the ability to consume JSON and create adaptive Terraform has become considerably easier with additions like for loops, arrays, and maps

Scenarios for consuming JSON in Terraform

Consuming JSON and then working with it is not always the easiest. In this post, I'll be showing how to consume JSON in 3 different ways:

Reading a file in the local directory
Reading in JSON from an AWS S3 bucket
Passing in JSON via environment variables

Why these 3 specific options? Well, all 3 are totally viable ways to consume JSON and are largely dependent on what your needs are. If the JSON is static, reading from S3 might be a good option. If you are running Terraform in a containerized environment, passing in as an environment variable may be a good option to differentiate between prod and testing. Or, you could even pass in a file location as an environment variable, which would then read a file from the local directory where the Terraform is being run.

How to consume the JSON in Terraform

To start, we are using a function called jsondecode(). Documentation here. This interprets a given string as JSON from the documentation. But, the most important part is that jsondecode maps JSON values to Terraform language values. Meaning, you can now use loops, store specific values in maps and arrays, or anything you can think of using HCL (the language of Terraform).

I am providing some sample JSON here that can be used for this post:

{
    "users": [
        {
            "user_name": "user_1",
            "role": "admin",
            "ssh_key": "ssh-rsa [shortened]"
        },
        {
            "user_name": "user_2",
            "role": "dev",
            "ssh_key": "ssh-rsa [shortened]"
        },
        {
            "user_name": "user_3",
            "role": "read_only",
            "ssh_key": "ssh-rsa [shortened]"
        },
        {
            "user_name": "user_4",
            "role": "dev",
            "ssh_key": "ssh-rsa [shortened]"
        }
    ]
}

Reading a file from your local directory

Now, create both a main.tf file and a users.json file that hosts the above JSON. Within the Terraform file, add the following block:

locals {
    # get json 
    user_data = jsondecode(file("${path.module}/users.json"))

    # get all users
    all_users = [for user in local.user_data.users : user.user_name]
}

output "users" {
    value = local.all_users
}

To break down above, we are:

Decoding the JSON into an HCL query-able local variable
Creating an array that loops through each user and attaches their username into the array
Outputs all users by using this local variable

To test it out, run a terraform init and then terraform plan! It should output something like this:

Changes to Outputs:
  + users = [
      + "user_1",
      + "user_2",
      + "user_3",
      + "user_4",
    ]

Passing in JSON via environment variables

Within Terraform, there are many ways to pass in an environment variable. In our case, I am simply going to create a terraform.tfvars file that will automatically hold environment variables. Here is the content of the file that can be added:

users = {
    "users": [
        {
            "user_name": "user_1",
            "role": "admin",
            "ssh_key": "ssh-rsa [shortened]"
        },
        {
            "user_name": "user_2",
            "role": "dev",
            "ssh_key": "ssh-rsa [shortened]"
        },
        {
            "user_name": "user_3",
            "role": "read_only",
            "ssh_key": "ssh-rsa [shortened]"
        },
        {
            "user_name": "user_4",
            "role": "dev",
            "ssh_key": "ssh-rsa [shortened]"
        }
    ]
}

It is essentially the exact same but has an HCL block that recognizes it as a Terraform variable.

Now, main.tf can be replaced with the following:

locals {    
    uniq_dev_roles = distinct(
      [for user in var.users.users : user.role]
    )
}

variable "users" {}

output "roles" {
    value = local.uniq_dev_roles
}

Now, we can run this Terraform in 2 different ways:

terraform plan -var-file="terraform.tfvars"
terraform plan

Why? Because in the documentation for input vars, the documentation states: load variable values from the given file, in addition to the default files terraform.tfvars... So you'll only need to pass in the file name if it's different. A common pattern could be something like this: prod.terraform.tfvars, staging.terraform.tfvars, testing.terraform.tfvars.

Things to note from this block:

We are now using another Terraform built-in function to expand how we are manipulating the JSON. In this case, it's showing the unique (or distinct) instances of dev roles, which in this case in admin, dev, and read_only.
The variable block for users is now where the user json is located. As you can see in the for loop, we are now looping over var.users.users. This is because we are no longer parsing a file and instead parsing a variable. In our case, we are not stating a type so Terraform is inherently inferring the types of the object. Because it interprets users as an array still, we are able to loop over it.

Reading in JSON from an AWS S3 bucket

Finally, our last example is reading in JSON as a data object from AWS. In this case, you'll need an AWS account and also to have uploaded this JSON from the examples above to somewhere in an S3 bucket for them to be referenced. However, the example is really not much different from the first.

Here is the complete main.tf file for a functional example (in my own local dev environment) of utilizing an S3 bucket:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

provider "aws" {
  region = "us-west-2"
  profile = "shannon" # change this to your own aws credentials or profile
}

# use a data object instead of a resource
data "aws_s3_bucket_object" "user_access_data" {
  bucket = "shannon-terraform"
  key    = "user_access/users.json"
}

locals {    
    user_data = jsondecode(data.aws_s3_bucket_object.user_access_data.body)
    users = [for user in local.user_data.users: user.user_name]
}

# create an IAM user for each user found in the JSON
resource "aws_iam_user" "user" {
    for_each = toset(local.users)
    name = each.value
}

So, we're no longer printing output and actually utilizing the JSON to loop over users and create an IAM user for each dev. Things to point out:

We're now using a data source in Terraform, which is just another type of resource that already exists and can be referenced in other resources. In our case, we're parsing the JSON we've already uploaded to S3.
We are utilizing for_each to loop over the users from the JSON in order to create an IAM user for each one. Because it's a list of strings, we'll specifically need to wrap it in toset(). Docs on how to use for_each can be found here.

Now, run terraform plan and you should have a Terraform plan for 4 new IAM resources!

Wrap up

I hope this was helpful in seeing how to work with JSON in Terraform. Once you are able to grep the data and turn it into HCL, the possibilities are limitless!

DEV Community: Shannon

Using closures to measure call stack depth in Go

Overview

Basics

Zero index pivot

Middle index Pivot

Utilizing the Go 1.25 Flight Recorder with tracing middleware

Table of Contents

Overview

Utilizing a FlightRecorder

Code

Run it

Looking at the trace file

Wrap up

Utilizing OS-recommended caching in Go to store files on disk

Table of Contents

Overview

Local caching

API Scaffolding

Creating the HTTP Client

Caching

Seeing it in action

All code

Writing events to pods using client-go

Scaffolding

Code

Seeing the events with kubectl

Interfaces satisfying interfaces

Circumvent STDIN when installing packages with apt

Parse your escaped JSON with jq and fromjson

Unit testing Stdout in Go

Testing Vault in Go

Mocking the Vault Client to test Read() and List() methods

Spinning up a test Vault server in Go to test Read() and List()

Creating your own Homebrew tap and grabbing binaries from S3

Set up your own homebrew repo in GitHub

Add the formula to download from S3

S3 Policies

Install the binary

Using reflect.DeepEqual to compare data structures in Go

Intro

Slice/Array Example

Map Example

Consuming and decoding JSON in Terraform

Intro

Scenarios for consuming JSON in Terraform

How to consume the JSON in Terraform

Reading a file from your local directory

Passing in JSON via environment variables

Reading in JSON from an AWS S3 bucket

Wrap up