Be careful when working with Slice in Go

Ulrich Wake — Mon, 26 Sep 2022 16:04:38 +0000

Go has array and slice data structure. Both of them are used to store sequence element with the same type. The difference is that array is fixed in length, while slice has flexible length.

Both array and slice have length and capacity. You can check the length with len and capacity with cap keyword.

In array, since it has fixed length, you can’t append new element (changing the length). On the other hand, you can append element to a slice even at the maximum slice capacity.

package main

import "fmt"

func main() {
    length := 0
    capacity := 3
    x := make([]int, length, capacity)

    // slice: [], length: 0, capacity: 3
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
}

In the code above, we create a slice with length 0 and capacity 3. Let’s add three elements and see the length and capacity.

package main

import "fmt"

func main() {
    length := 0
    capacity := 3
    x := make([]int, length, capacity)

    // slice: [], length: 0, capacity: 3
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", x, len(x), cap(x))

    x = append(x, 1, 2, 3)

    // slice: [1 2 3], length: 3, capacity: 3
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
}

So far so good. Now, let’s add new element to x and see what happens.

package main

import "fmt"

func main() {
    length := 0
    capacity := 3
    x := make([]int, length, capacity)

    // slice: [], length: 0, capacity: 3
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", x, len(x), cap(x))

    x = append(x, 1, 2, 3)

    // slice: [1 2 3], length: 3, capacity: 3
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", x, len(x), cap(x))

    x = append(x, 4)

    // slice: [1 2 3 4], length: 4, capacity: 6
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
}

As you can see, the length becomes 4 and the capacity is 6. When a slice has reached its maximum capacity, it will double its capacity if we append new element.

However, the more interesting thing is that the slice will allocate new memory when slice has exceeded its capacity. So, in your memory there will be a slice [1 2 3] with capacity 3 and a slice [1 2 3 4] with capacity 6. The variable x will point to the new array.

This is what happen in the memory when we do x = append(x, 1, 2, 3).

We have variable x that has attribute length and capacity with value 3 and 3 respectively. When we append new element (x = append(x, 4)), Go will create new sequence of the element and copying all the values from previous array.

You can see that the previous array is not referenced anymore and will be freed by Go Garbage Collector later.

We have understood how slice works in Go. Let’s look at the problem below.

package main

import "fmt"

func main() {
    x := make([]int, 3, 5)

    x = append(x, 1)
    y := append(x, 2)
    z := append(x, 3)

    fmt.Printf("slice x: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
    fmt.Printf("slice y: %v, length: %d, capacity: %d\n", y, len(y), cap(y))
    fmt.Printf("slice z: %v, length: %d, capacity: %d\n", z, len(z), cap(z))
}

Can you guess the output?

If we run the code, we will get the following result.

# The first three element is initialized with 0 because we set the slice to have length 3.
slice x: [0 0 0 1], length: 4, capacity: 5
slice y: [0 0 0 1 3], length: 5, capacity: 5
slice z: [0 0 0 1 3], length: 5, capacity: 5

Shouldn’t y is equal to [0 0 0 1 2]?
If you are thinking this way, then let’s take a look at the memory representation step by step to understand what's actually happening.

package main

import "fmt"

func main() {
    x := make([]int, 3, 5)

    x = append(x, 1)

    fmt.Printf("slice x: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
}

At this point, the memory representation is like below.

Then we append x with 2 and assign it to variable y

You can see that x and y point to the same array. The difference is that y has length 5 while x has length 4. Note that Go does not create new array because the current array has not exceeded its capacity.

Next, we append x with 3 and assign it to z.

Before appending x, we can clearly see that x only has length 4 with capacity 5. Therefore, when doing z := append(x, 3), Go will overwrite 2 with 3 because in the point of view x, the capacity is not exceeded yet. So, no new array is allocated. Therefore, z will also point to the same header with x and y.

To prove that, try to execute this code.

package main

import "fmt"

func main() {
    x := make([]int, 3, 5)

    x = append(x, 1)
    y := append(x, 2)
    z := append(x, 3)

    fmt.Printf("slice x: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
    fmt.Printf("slice y: %v, length: %d, capacity: %d\n", y, len(y), cap(y))
    fmt.Printf("slice z: %v, length: %d, capacity: %d\n", z, len(z), cap(z))

    // change the value of third element of slice x
    x[2] = 100
    fmt.Printf("slice x: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
    fmt.Printf("slice y: %v, length: %d, capacity: %d\n", y, len(y), cap(y))
    fmt.Printf("slice z: %v, length: %d, capacity: %d\n", z, len(z), cap(z))
}

You will see that the third element of x, y, and z is equal to 100. This proves that those three pointers point to the same header of the array.

slice: [0 0 100 1], length: 4, capacity: 5
slice: [0 0 100 1 3], length: 5, capacity: 5
slice: [0 0 100 1 3], length: 5, capacity: 5

Now, what happen if you append z with new element like below.

package main

import "fmt"

func main() {
    x := make([]int, 3, 5)

    x = append(x, 1)
    y := append(x, 2)
    z := append(x, 3)
    w := append(z, 4)

    fmt.Printf("slice x: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
    fmt.Printf("slice y: %v, length: %d, capacity: %d\n", y, len(y), cap(y))
    fmt.Printf("slice z: %v, length: %d, capacity: %d\n", z, len(z), cap(z))
    fmt.Printf("slice w: %v, length: %d, capacity: %d\n", w, len(w), cap(w))
}

As we have discussed before, because z has reached its maximum capacity, adding new element will make Go creates new array and w will point to that array. The capacity of w is twice bigger than z. Below is the output.

slice x: [0 0 0 1], length: 4, capacity: 5
slice y: [0 0 0 1 3], length: 5, capacity: 5
slice z: [0 0 0 1 3], length: 5, capacity: 5
slice w: [0 0 0 1 3 4], length: 6, capacity: 10

To prove that w points to another array, let's change the value of the third element again.

package main

import "fmt"

func main() {
    x := make([]int, 3, 5)

    x = append(x, 1)
    y := append(x, 2)
    z := append(x, 3)
    w := append(z, 4)

    x[2] = 100

    fmt.Printf("slice: %v, length: %d, capacity: %d\n", x, len(x), cap(x))
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", y, len(y), cap(y))
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", z, len(z), cap(z))
    fmt.Printf("slice: %v, length: %d, capacity: %d\n", w, len(w), cap(w))
}

Below is the output

slice: [0 0 100 1], length: 4, capacity: 5
slice: [0 0 100 1 3], length: 5, capacity: 5
slice: [0 0 100 1 3], length: 5, capacity: 5
slice: [0 0 0 1 3 4], length: 6, capacity: 10

You can see that the third element of w is still 1 not 100. This proves that w points to another array.

So, next time be careful when working with Go slice.

DEV Community: Ulrich Wake

Be careful when working with Slice in Go