Efficiently concatenating strings in go

In this article, we will look at different ways to concatenate strings in Go.

What happens when you concatenate strings with the +operator

Using the + is good enough for pieces of code that are not critical or when there are not many strings to join.

However, when we have many strings to join (for whatever reason), then performance starts to degrade.

Using bytes.Buffer

Using bytes.Buffer is one of two efficient ways to handle many string concatenations. We define a bytes.Buffer type and append to this type until we are done with concatenation.

A very simple example of bytes.Buffer looks like:

var s bytes.Buffer
for i := 0; i < 100; i++  {
  s.WriteString("somestring")
}
log.Printf("A super long string: %s", s.String())

Using strings.Builder

From Go 1.10 a strings.Builder type was added. It's designed for string concatenation and minimizes memory copying. This is the preferred method for implementing efficient string concatenation.

A simple example of strings.Builder looks like:

var s strings.Builder
for i := 0; i < concats; i++  {
  s.WriteString("somestring")
}
log.Printf("A super long string: %s", s.String())

Let's measure some performance!

In order to test the differences between the three methods, we can write a program that can measure some times for a given concatenation count. Thereafter, we'll be able to visualize some performance.

This is not meant to be a very strict performance benchmark. It's more to give an indicator on performance.

Benchmark test

If you happen to be unfamiliar with Go benchmarks, take a look at the testing package godoc for a really nice explanation on how it works.

Our benchmark test looks like this:

package concats

import (
    "bytes"
    "strings"
    "testing"
)

func BenchmarkStringConcatenation(b *testing.B) {
    benchmarks := []struct {
        name   string
        testFn func(int)
        param  int
    }{
        {"Regular 50", regularConcatentation, 50},
        {"Regular 100", regularConcatentation, 100},
        {"bytes.Buffer 50", bytesBuffer, 100},
        {"bytes.Buffer 100", bytesBuffer, 100},
        {"strings.Builder 50", stringBuilder, 100},
        {"strings.Builder 100", stringBuilder, 100},
    }
    for _, bm := range benchmarks {
        b.Run(bm.name, func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                bm.testFn(bm.param)
            }
        })
    }
}

func BenchmarkRegularConcatenation40(b *testing.B) {
    for i := 0; i < b.N; i++ {
        regularConcatentation(40)
    }
}

func BenchmarkBytesBuffer40(b *testing.B) {
    for i := 0; i < b.N; i++ {
        bytesBuffer(40)
    }
}

func BenchmarkStringBuilder40(b *testing.B) {
    for i := 0; i < b.N; i++ {
        stringBuilder(40)
    }
}

func bytesBuffer(concats int) {
    var s bytes.Buffer
    for i := 0; i < concats; i++ {
        s.WriteString("somestring")
    }
}

func regularConcatentation(concats int) {
    s := ""
    for i := 0; i < concats; i++ {
        s += "somestring"
    }
}

func stringBuilder(concats int) {
    var s strings.Builder
    for i := 0; i < concats; i++ {
        s.WriteString("somestring")
    }
}

Results

I do think that our benchmark results will be different on our individual machines. After running these benchmarks, we get the following output:

goos: darwin
goarch: amd64
pkg: concats
BenchmarkStringConcatenation/Regular_50-4             300000          4542 ns/op
BenchmarkStringConcatenation/Regular_100-4            100000         15084 ns/op
BenchmarkStringConcatenation/bytes.Buffer_50-4       1000000          1585 ns/op
BenchmarkStringConcatenation/bytes.Buffer_100-4      1000000          1561 ns/op
BenchmarkStringConcatenation/strings.Builder_50-4    2000000           733 ns/op
BenchmarkStringConcatenation/strings.Builder_100-4   2000000           735 ns/op
PASS

Looking at these results, string concatentation is much slower than both bytes.Buffer and strings.Builder. This will be consistent across different machines. When using strings.Builder, it runs at about twice as fast as bytes.Buffer on my machine. I do think that the time differences between these two styles will vary between machines.

Conclusion

In this post, we learnt about the different approaches to mass string concatenation in Go. We also managed to benchmark these techniques and briefly discussed the results.