DEV Community

Cover image for String and []byte in Go: The Conversions That Quietly Allocate
Gabriel Anhaia
Gabriel Anhaia

Posted on

String and []byte in Go: The Conversions That Quietly Allocate


You profile a hot path in a Go service. A request parser, a
log line formatter, something that runs a few hundred thousand
times a second. The pprof alloc view points at a line you would
never suspect:

key := string(b)
Enter fullscreen mode Exit fullscreen mode

One conversion. No make, no append, no obvious allocation.
And yet it sits near the top of the allocation profile, quietly
producing garbage the collector has to sweep. This is the tax
on string and []byte conversions, and most Go developers pay
it without knowing where it comes from or when they could skip it.

Why the copy exists at all

A Go string is immutable. A []byte is not. That single rule
forces the behavior.

When you write []byte(s), the runtime cannot hand you a slice
that points at the string's backing memory, because you could
then write into that slice and mutate a value the language
promises is immutable. Some other part of the program might be
holding the same string, expecting it to never change. So the
runtime allocates a fresh backing array and copies the bytes.

The reverse, string(b), has the same problem in the other
direction. If the string aliased the slice's memory, a later
write to the slice would silently change a string that already
shipped somewhere else. So again: allocate, copy.

s := "hello"
b := []byte(s)   // allocates len(s) bytes, copies
b[0] = 'H'       // legal, and s is untouched
s2 := string(b)  // allocates again, copies again
Enter fullscreen mode Exit fullscreen mode

Two conversions, two heap allocations, two copies. For a
five-byte string nobody cares. For a 4KB request body converted
on every call, the allocator and the garbage collector care a
lot.

The conversions the compiler skips

Here is the part that surprises people. The gc compiler knows
these copies are wasteful in specific shapes where the converted
value cannot escape and cannot be mutated. In those shapes it
optimizes the copy away entirely. You get the readable code and
zero allocations.

Map access keyed by string(b):

var counts = map[string]int{}

func bump(b []byte) {
    counts[string(b)]++   // no allocation
}
Enter fullscreen mode Exit fullscreen mode

The compiler sees that string(b) is used only to look up a
map key. The key is read, never retained, never mutated. So it
computes the hash and does the lookup against the byte slice
directly. No temporary string is built.

Comparisons:

if string(b) == "GET" {   // no allocation
    // ...
}
Enter fullscreen mode Exit fullscreen mode

Same reasoning. The string form lives only long enough to feed
==, so the compiler compares bytes in place.

switch on string(b):

switch string(b) {        // no allocation
case "GET", "POST":
    // ...
}
Enter fullscreen mode Exit fullscreen mode

Range over []byte(s):

for i, c := range []byte(s) {   // no allocation
    _ = i
    _ = c
}
Enter fullscreen mode Exit fullscreen mode

The range clause consumes the slice element by element and
never lets it escape, so the compiler iterates over the string's
bytes without materializing a new slice.

Concatenation into a comparison or append also frequently
folds away
, but the four above are the reliable, documented
wins. Learn them and you remove a surprising share of the
conversions in string-heavy code without touching readability.

What does not get optimized: storing the result. The moment
string(b) is assigned to a variable that outlives the
statement, put into a struct field, sent on a channel, or
returned, the compiler has to give you a real, independent
string. That means a copy.

key := string(b)          // allocates: key outlives the line
m[key] = 1
Enter fullscreen mode Exit fullscreen mode

Move the conversion into the index expression and the allocation
disappears:

m[string(b)] = 1          // no allocation
Enter fullscreen mode Exit fullscreen mode

The unsafe escape hatch

Sometimes you genuinely need a string view of bytes, or a
[]byte view of a string, on a path so hot that even the
optimized cases are not enough, and you can guarantee nobody
mutates the underlying memory. Go 1.20 added the honest tools
for this in the unsafe package: unsafe.String,
unsafe.StringData, unsafe.Slice, and unsafe.SliceData.

Bytes to string, no copy:

func unsafeString(b []byte) string {
    if len(b) == 0 {
        return ""
    }
    return unsafe.String(unsafe.SliceData(b), len(b))
}
Enter fullscreen mode Exit fullscreen mode

String to bytes, no copy:

func unsafeBytes(s string) []byte {
    if len(s) == 0 {
        return nil
    }
    return unsafe.Slice(unsafe.StringData(s), len(s))
}
Enter fullscreen mode Exit fullscreen mode

The word unsafe is not decoration. The returned string aliases
the slice's memory. If any code writes to b while that string
is alive, you have mutated an immutable value, and the behavior
of everything holding it is now undefined. The unsafeBytes
direction is worse: string literals often live in read-only
memory, so writing to the returned slice can crash the process
with a segfault.

Use this only when you own both sides of the lifetime and can
prove the bytes stay frozen. The standard library uses the exact
same trick internally in places like strings.Builder, which
returns its accumulated bytes as a string with no copy precisely
because it never lets you touch the buffer again after String().

Prove it, do not guess

Never take any of this on faith, including this post. The Go
toolchain tells you exactly what allocates. Two commands settle
every argument.

First, escape analysis. Ask the compiler what it decided:

go build -gcflags='-m' ./...
Enter fullscreen mode Exit fullscreen mode

Lines like string(b) does not escape confirm the optimized
path. ... escapes to heap confirms a real allocation.

Second, a benchmark with allocation reporting. This is the one
that ends debates:

func BenchmarkKeyed(b *testing.B) {
    data := []byte("hello world")
    m := map[string]int{}
    b.ReportAllocs()
    for i := 0; i < b.N; i++ {
        m[string(data)]++      // in-index conversion
    }
}

func BenchmarkStored(b *testing.B) {
    data := []byte("hello world")
    m := map[string]int{}
    b.ReportAllocs()
    for i := 0; i < b.N; i++ {
        k := string(data)      // stored first
        m[k]++
    }
}
Enter fullscreen mode Exit fullscreen mode

Run it:

go test -bench=. -benchmem
Enter fullscreen mode Exit fullscreen mode

b.ReportAllocs() prints allocs/op. The keyed benchmark
reports 0 allocs/op. The stored one reports 1 allocs/op plus
the bytes. Same map, same data, one difference: whether the
converted string had to live past the statement. The number in
that column is not an opinion, and it is the only thing worth
optimizing against.

What to actually do

You do not need to rewrite every conversion in your codebase.
Most of them run cold and the copy costs nothing you will ever
measure. The discipline is narrower than that.

On a path that pprof flags as hot: check whether the conversion
result is used and discarded in the same expression. If it is,
push the conversion into the map index, the comparison, or the
switch, and the compiler hands you the copy elimination for free.
If you truly need a persistent view and you can guarantee frozen
memory, reach for unsafe.String, comment why it is safe, and
confine it to one small function. Everywhere else, let the copy
happen and move on.

The trap was never that conversions are slow. It is that they
look free in the source and are not, and that the cases where
they are free look identical to the cases where they are not.
The compiler flags and -benchmem are how you tell them apart.


If this was useful

The copy-elimination rules here come straight out of how the gc
compiler models escape analysis and immutability, which is the
kind of thing The Complete Guide to Go Programming digs into
end to end: the memory model, how strings and slices are laid
out, and why the runtime makes the copies it makes. Hexagonal
Architecture in Go
is the companion for keeping this where it
belongs, so the unsafe fast paths live behind a boundary instead
of leaking into your domain code.

Thinking in Go — the 2-book series on Go programming and hexagonal architecture

Top comments (0)