DEV Community

Cover image for Go Slices and Subslices: Understanding Shared Memory and Avoiding `append()` Pitfalls
Patrick A. Noblet
Patrick A. Noblet

Posted on • Originally published at blog.noblet.tech

1

Go Slices and Subslices: Understanding Shared Memory and Avoiding `append()` Pitfalls

Introduction

Heya! Welcome back to my blog.😁 If you are here, you are probably new to Golang or experienced and want to explore the inner workings of slices. Let’s GO on the ride then.

Go is often praised for its simplicity and efficiency — "Go just gets the job done," as they say. For those of us coming from languages like C, C++, or Java, Go’s straightforward syntax and ease of use is refreshing. However, even in Go, certain quirks can trip up developers, especially when it comes to slices and subslices. Let's uncover these nuances to better understand how to avoid common pitfalls with append() and shared memory in slices.

What Are Slices in Go?

Typically, when you need a data structure to store a sequence of values, slices are the go-to option in Go. Their flexibility comes from the fact that their length is not fixed as part of their type. This feature overcomes the constraints of arrays, allowing us to create a single function that can handle slices of any size, and enabling slices to grow or expand as necessary.

While slices share some similarities with arrays such as being indexable and having a length, they differ in how they manage data. A slice serves as a reference to an underlying array, which actually stores the slice’s data. Essentially, a slice provides a view into some or all of the elements of this array. So, when you create a slice, Go automatically handles the creation of the underlying array that holds the slice’s elements/data.

Shared Memory in Slices

An array is a contiguous block of memory but what makes slices interesting is how they reference this memory. Let’s break down the anatomy of a slice:

type slice struct {
    array unsafe.Pointer // Pointer to the underlying array
    len   int           // Number of elements in the slice
    cap   int           // Capacity of the underlying array
}
Enter fullscreen mode Exit fullscreen mode

When you create a slice, it consists of three components:

  1. A pointer to the underlying array

  2. len The length of the slice (how many elements it contains)

  3. cap The capacity (how many elements it can contain before needing to grow)

Here's where things get interesting. If you have multiple slices derived from the same array, changes made through one slice will be visible in the other slices because they share the same underlying array.

Let's look at an example below:

package main

import "fmt"

func main() {
    // Create a slice with some initial values
    original := []int{1, 2, 3, 4, 5}

    // Create a subslice - both slices share the same underlying array!
    subslice := original[1:3]

    fmt.Println("Unmodified Subslice:", subslice)  // Output => Umodified Subslice: [2 3]

    // Modify the subslice
    subslice[0] = 42

    fmt.Println("Original:", original) // Output => Original: [1 42 3 4 5]
    fmt.Println("Modified Subslice:", subslice)  // Output => Modified Subslice: [42 3]
}
Enter fullscreen mode Exit fullscreen mode

Understanding Slice Capacity

Before we dive in further, let’s try to understand slice capacity cap(). When you take a subslice from an existing slice in Go, the capacity of the new subslice is determined by the remaining capacity of the original slice from the point where the subslice starts. Let’s break this down a little:

When you create a slice from an array, the slice's length is the number of elements it initially contains, and its capacity is the total number of elements it can contain before needing to grow.

Taking a Subslice

When you take a subslice from an existing slice:

  • The length of the subslice is the number of elements you specify.

  • The capacity is calculated as the capacity of the original slice minus the starting index of the subslice.

Let’s take a look at a detailed example:

func main() {
    // Original slice
    original := []int{1, 2, 3, 4, 5}

    // Create a subslice
    subslice := original[1:4] // Refers to elements 2, 3, 4

    fmt.Println("Subslice:", subslice)    // Output => Subslice: [2 3 4]
    fmt.Println("Length of subslice:", len(subslice)) // Output => Length of subslice: 3
    fmt.Println("Capacity of subslice:", cap(subslice)) // Output => Capacity of subslice: 4
}
Enter fullscreen mode Exit fullscreen mode
  • The original slice has 5 elements with a length and capacity of 5.

  • When you take subslice := original[1:4], it refers to the elements from index 1 to 3 (2, 3, 4).

  • The length of subslice is 4 - 1 = 3.

  • The capacity of subslice is 5 - 1 = 4 because it starts from index 1 and includes the elements to the end of the original slice.

The append() Gotcha!

Here's where developers often get caught off guard. The append() function in Go can lead to unexpected behaviour when working with subslices.

Unused Capacity Sharing

The capacity of the subslice includes the elements that are not part of its length but are within the capacity range of the original slice. This means that the subslice can access or modify those elements if it grows.

Let’s consider this example:

func main() {
    original := []int{1, 2, 3, 4, 5}
    subslice := original[1:3] // Refers to elements 2, 3

    fmt.Println("Original slice before append:", original) // Outputs => [1 2 3 4 5]
    fmt.Println("Subslice before append:", subslice)       // Outputs => [2 3]
    fmt.Println("Capacity of subslice:", cap(subslice))    // Outputs => 4

    // Appending to subslice within capacity
    subslice = append(subslice, 60, 70)

    // Printing after appending to subslice
    fmt.Println("Original slice after append:", original)  // Outputs => [1 2 3 60 70]
    fmt.Println("Subslice after append:", subslice)        // Outputs => [2 3 60 70]
}
Enter fullscreen mode Exit fullscreen mode
  • subslice originally refers to 2, 3 with a capacity of 4 (it can grow up to the end of the original slice).

  • When you append 60, 70 to subslice, it uses the remaining capacity of the original slice.

  • Both original and subslice reflect the changes because they share the same underlying array.

Surprised? The append() operation modified the original slice because there was enough capacity in the underlying array. However, if we exceed the capacity or append more elements than the capacity allows, Go will allocate a new array for the subslice, breaking the sharing with the original slice:

func main() {
    original := []int{1, 2, 3, 4, 5}
    subslice := original[1:3] // Refers to elements 2, 3

    // Appending beyond the capacity of subslice
    subslice = append(subslice, 60, 70, 80)

    fmt.Println("Original slice after large append:", original) // Outputs =>[1 2 3 4 5]
    fmt.Println("Subslice after large append:", subslice)       // Outputs => [2 3 60 70 80]
}
Enter fullscreen mode Exit fullscreen mode

In this case, append() created a new underlying array because the original capacity was exceeded.

Best Practices to Avoid Pitfalls

  • Be Explicit About Capacity
// Let's say we start with this
original := []int{1, 2, 3, 4, 5}
subslice := original[1:3]  // subslice points to original's backing array

// This is our solution:
newSlice := make([]int, len(subslice))  // Step 1: Create new backing array
copy(newSlice, subslice)               // Step 2: Copy values over
Enter fullscreen mode Exit fullscreen mode

The key benefits are:

i. make([]int, len(subslice)) creates a new slice with its own separate backing array. This is crucial - it's not just a new slice header, but a completely new array in memory.

ii. copy() then transfers just the values, not the memory reference. It's like photocopying a document rather than sharing the original.

  • Use Full Slice Expressions
// Specify the capacity limit to prevent sharing beyond what you intend
limited := original[1:3:3] // third number limits the capacity
Enter fullscreen mode Exit fullscreen mode
  • Consider Immutability when passing slices to functions that shouldn't modify the original data
func processSlice(data []int) []int {
    // Create a new slice with its own backing array
    result := make([]int, len(data))
    copy(result, data)
    // Process result...
    return result
}
Enter fullscreen mode Exit fullscreen mode

The key benefits are:

i. Data Protection: The original data remains unchanged, preventing unexpected side effects

ii. Predictable Behavior: Functions don't have hidden effects on their inputs

iii. Concurrent Safety: Safe to use the original data in other goroutines while processing

Remember:

  • Slices are references to underlying arrays

  • Subslices share memory with their parent slice

  • append() may or may not create a new backing array depending on capacity

  • When appending to a subslice with available capacity, it modifies the parent slice's data.

  • Use explicit memory management when you want to avoid sharing

  • When working with subslices, either:

1. Create a new slice with its own backing array using `make()` and `copy()`

2. Use full slice expressions with capacity limits: `parent[1:3:3]`
Enter fullscreen mode Exit fullscreen mode

Happy coding. And remember, with great power comes great responsibility, especially when it comes to shared memory! 😉


Congratulations on getting to the end of this article.

Do you find this resource helpful? Do you have a question or have you discovered a mistake or typo? Please leave your feedback in the comments.

Don't forget to share this resource with others who may also benefit from it. Follow me for more.

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay