Reducing Garbage Collector Pressure in Golang

Rodrigo Oliveira — Sun, 26 Jan 2025 18:09:40 +0000

In high-performance Go applications, excessive memory allocations and deallocations can significantly impact performance. This creates unnecessary pressure on the Garbage Collector (GC), resulting in higher latency and reduced efficiency. This article will teach you how to reduce GC pressure using object reuse techniques and the sync.Pool feature.

Inspiration for this article came from a post on LinkedIn by Branko Pitulic, which highlighted the importance of optimizing memory usage in Go applications.

1. Understanding the Problem

Go's Garbage Collector is responsible for automatic memory management. However, when an application frequently allocates and deallocates memory (especially in the heap), the GC has to work harder, leading to:

Increased CPU usage;
Execution pauses during GC cycles;
Performance bottlenecks in low-latency systems.

The goal is to reduce the number of objects allocated on the heap by promoting memory reuse.

2. Techniques to Reduce GC Pressure

2.1 Reusing Objects

Where possible, reuse objects instead of creating new ones. A common pattern is reusing slices and arrays.

Bad Practice:

func process() []byte {
    return make([]byte, 1024) // Creates a new slice every time.
}

Good Practice:

var buffer = make([]byte, 1024)

func process() []byte {
    return buffer // Reuses the existing slice.
}

Note: This approach works well in non-concurrent contexts where reuse is safe.

2.2 Using `sync.Pool`

The sync package provides the Pool type, which is an efficient structure for pooling objects, enabling reuse. This reduces memory allocations on the heap.

How `sync.Pool` Works:

Objects can be stored in the pool after use.
When a new object is needed, the pool is checked before allocating memory.
If the pool is empty, a new object is created.

Basic Example:

package main

import (
    "fmt"
    "sync"
)

func main() {
    // Creating a pool of objects.
    pool := sync.Pool{
        New: func() any {
            return make([]byte, 1024) // Creates a new 1 KB slice.
        },
    }

    // Retrieving an object from the pool.
    buffer := pool.Get().([]byte)
    fmt.Printf("Buffer length: %d\n", len(buffer))

    // Reusing the object by putting it back into the pool.
    pool.Put(buffer)

    // Retrieving another object from the pool.
    reusedBuffer := pool.Get().([]byte)
    fmt.Printf("Reused buffer length: %d\n", len(reusedBuffer))
}

In this example:

A sync.Pool is created with a New function to initialize objects.
Get is used to retrieve objects from the pool.
Put is used to return objects to the pool for reuse.

3. Best Practices for Using `sync.Pool`

Lightweight Objects: Pools are ideal for small or medium-sized objects. For large objects, storage costs may outweigh the benefits.
Concurrency: sync.Pool is safe for use in multiple goroutines, though performance may vary under heavy load.
Initialization: Always define a New function in the pool to ensure proper object creation.
Avoid Overusing Pools: Use pools only for objects that are frequently reused.

4. Common Use Cases

4.1 Buffer Pooling for Read/Write Operations

Applications with heavy read/write operations (e.g., HTTP servers or message processors) can reuse buffers efficiently.

Example:

package main

import (
    "bytes"
    "sync"
)

var bufferPool = sync.Pool{
    New: func() any {
        return new(bytes.Buffer)
    },
}

func process(data string) string {
    buffer := bufferPool.Get().(*bytes.Buffer)
    buffer.Reset() // Clear the buffer before use.
    defer bufferPool.Put(buffer)

    buffer.WriteString("Processed: ")
    buffer.WriteString(data)
    return buffer.String()
}

func main() {
    result := process("Hello, World!")
    println(result)
}

4.2 Struct Reuse

If your application frequently creates and discards structs, sync.Pool can help.

Example:

package main

import (
    "fmt"
    "sync"
)

type Request struct {
    ID      int
    Payload string
}

var requestPool = sync.Pool{
    New: func() any {
        return &Request{}
    },
}

func handleRequest(id int, payload string) {
    req := requestPool.Get().(*Request)
    defer requestPool.Put(req)

    req.ID = id
    req.Payload = payload

    fmt.Printf("Handling request ID=%d with payload=%s\n", req.ID, req.Payload)
}

func main() {
    handleRequest(1, "data1")
    handleRequest(2, "data2")
}

5. Final Considerations

Using sync.Pool can significantly improve application performance, particularly in high-throughput scenarios. However:

Avoid premature optimizations. Before adopting sync.Pool, ensure GC is a real bottleneck by analyzing performance with tools like pprof.
Combine pool usage with general best practices, such as reducing variable scope and using slices or arrays efficiently.

Understanding and applying these techniques will help you build more efficient and scalable systems in Go.

If you have questions or want more advanced examples, feel free to ask! 🚀

I completed Hacktoberfest

Rodrigo Oliveira — Mon, 01 Nov 2021 04:41:56 +0000

It's never too late to start. My first Hacktoberfest, after more than 10 years of career.

Contribute and unlock your domain

Rodrigo Oliveira — Thu, 14 Oct 2021 00:40:04 +0000

Unlock your first PR

Being productive while following the best software development practices has been one of the greatest challenges for developers. Today I can say that with HerbsJS this challenge becomes much easier.

Using and contributing to the project, in addition to participating in a vibrant community, I've been putting into practice building software that delivers value and focuses on what really matters: the domain.

The pillars of HerbsJS are Domain First, creating microservices in Node.js with DDD, Clean Architecture, Authorization and auditing available even for simple applications.

I invite you to break this barrier. Take advantage of Hacktoberfest, visit the HerbsJS website, and contribute to a project that in essence elevates the developer level and enables you to build quality software.

DEV Community: Rodrigo Oliveira

Reducing Garbage Collector Pressure in Golang

1. Understanding the Problem

2. Techniques to Reduce GC Pressure

2.1 Reusing Objects

Bad Practice:

Good Practice:

2.2 Using `sync.Pool`

How `sync.Pool` Works:

Basic Example:

3. Best Practices for Using `sync.Pool`

4. Common Use Cases

4.1 Buffer Pooling for Read/Write Operations

Example:

4.2 Struct Reuse

Example:

5. Final Considerations

I completed Hacktoberfest

Contribute and unlock your domain

Unlock your first PR

Links

DEV Community: Rodrigo Oliveira

Reducing Garbage Collector Pressure in Golang

1. Understanding the Problem

2. Techniques to Reduce GC Pressure

2.1 Reusing Objects

Bad Practice:

Good Practice:

2.2 Using sync.Pool

How sync.Pool Works:

Basic Example:

3. Best Practices for Using sync.Pool

4. Common Use Cases

4.1 Buffer Pooling for Read/Write Operations

Example:

4.2 Struct Reuse

Example:

5. Final Considerations

I completed Hacktoberfest

Contribute and unlock your domain

Unlock your first PR

Links

2.2 Using `sync.Pool`

How `sync.Pool` Works:

3. Best Practices for Using `sync.Pool`