DEV Community: Philip Pearl

Go interfaces, but at what cost?

Philip Pearl — Sat, 06 Jul 2019 18:02:11 +0000

There's a cost associated with using interfaces. What is that cost? Let's try and work out some of it.

Let's start with the basic overhead of calling a method via an interface. We'll define a very simple interface with a single method and a very simple implementation. We'll also mark the method so it isn't inlined by the compiler. We do this so that the call to get isn't completely removed in the direct case.

type getter interface {
    get() int
}

type zero struct{}

//go:noinline
func (z zero) get() int {
    return 0
}

We make a very simple benchmark, with two subtests. One calls get via a getter the other calls get on the concrete zero type directly.

func BenchmarkInterfaceCallSimple(b *testing.B) {
    var z zero
    var g getter
    g = z

    b.Run("via interface", func(b *testing.B) {
        total := 0
        for i := 0; i < b.N; i++ {
            total += g.get()
        }

        if total > 0 {
            b.Logf("total is %d", total)
        }
    })

    b.Run("direct", func(b *testing.B) {
        total := 0
        for i := 0; i < b.N; i++ {
            total += z.get()
        }

        if total > 0 {
            b.Logf("total is %d", total)
        }
    })
}

Here's the result.

BenchmarkInterfaceCallSimple/via_interface-8    4.63 ns/op
BenchmarkInterfaceCallSimple/direct-8           2.44 ns/op

So there's a small overhead from making a method call via an interface. So small it won't matter except in extreme cases. Are there any other issues?

Let's try something a little different. We'll create a very simple implementation of io.Reader that fills the buffer with zeros.

type zeroReader struct{}

func (z zeroReader) Read(p []byte) (n int, err error) {
    for i := range p {
        p[i] = 0
    }
    return len(p), nil
}

Our benchmark will follow a very similar structure to the previous one. We'll test calling our implementation via an io.Reader interface and directly.

func BenchmarkInterfaceAlloc(b *testing.B) {
    var z zeroReader
    var r io.Reader
    r = z

    b.Run("via interface", func(b *testing.B) {
        b.ReportAllocs()

        for i := 0; i < b.N; i++ {
            var buf [7]byte
            r.Read(buf[:])
        }
    })

    b.Run("direct", func(b *testing.B) {
        b.ReportAllocs()

        for i := 0; i < b.N; i++ {
            var buf [7]byte
            z.Read(buf[:])
        }
    })
}

Here's the results. Instead of the approximately 2ns overhead we now have closer to 20ns and 1 allocation. What's going on here?

BenchmarkInterfaceAlloc/via_interface-8     50000000        24.5 ns/op      8 B/op       1 allocs/op
BenchmarkInterfaceAlloc/direct-8            300000000       5.52 ns/op      0 B/op       0 allocs/op

In both cases we allocate a 7 byte buffer before each Read call. In the direct call case, the compiler knows what the implementation of the Read call is, so it can apply escape analysis and can tell that the buffer does not escape the stack and therefore can be allocated on the stack.

When calling via an interface the implementation is unknown at compile time, therefore the compiler must assume the buffer will escape, and therefore must allocate the buffer on the heap rather than the stack. Allocating on the heap takes longer (and in particular must grab a lock), and will cause more GC overhead.

I find it quite disappointing that any memory passed via any interface will always escape and always require heap memory. In the case of io.Reader the english definition of the interface strongly hints that the buffer should not escape.

Implementations must not retain p.

Similarly the json.Unmarshaler interface description implies the buffer should not actually escape.

UnmarshalJSON must copy the JSON data if it wishes to retain the data after returning.

Wouldn't it be nice if we could express this on the interface definition in a way the compiler could understand?

What’s all that memory for?

Philip Pearl — Tue, 17 Apr 2018 20:17:37 +0000

Perhaps it’s for storing strings?

Photo by Sandrachile . on Unsplash

If you actually want to use the memory on your computer with Go — really use it, with gigabytes of it allocated — then you may pay a big penalty for the Go garbage collector (GC). But there are things you can do about it.

The Go GC checks what parts of the memory you have allocated are still in use. It does this by looking at all the memory for references to other pieces of memory. If you’ve allocated millions of pieces of memory, then all that ‘looking’ necessarily takes some CPU time to do. So if you actually want to use the gigabytes of memory in your computer, you might want to be a little careful about how you do things.

How bad is it?

Imagine you have a desperate need to remember 100 million random 20 byte strings. What kind of overhead does the GC impose if you do this in a normal way?

Here’s some code to allocate those strings. This uses about 3.5 GB of RAM.

space := make([]string, 100*1000*1000)
for i := range space {
    space[i] = fmt.Sprintf("%20.20d", rand.Int())
}

So what impact does this have on GC? Well, one easy thing we can do to measure this is call the Go runtime to force GC, and measure how long that takes.

start := time.Now()
runtime.GC()
fmt.Printf("GC takes %s\n", time.Since(start))

How long does that take?

GC takes 730.93697ms

Oh. That’s quite a long time.

Well, it’s quite quick for looking at 100 million things (about 7ns a thing). But burning 700ms of CPU time each time the GC runs is definitely edging into the realm of “not ideal”.

And if we run the GC again, it takes approximately the same time again. We’re stuck with ~700ms of GC work every time the GC runs until we’re done with these strings.

How can we fix it?

Luckily for us the Go GC is so clever that it does not look at every piece of memory allocated. If it knows the memory does not contain any pointers, it does not look at it. Without pointers the memory cannot be referencing other pieces of memory, so the GC doesn’t need to look at it to determine which memory is no longer referenced and therefore can be freed.

If we can arrange things so we can store the strings without any pointers, we can save this GC overhead.

Oh, strings contain pointers?

Yes, strings contain pointers. The reflect package shows us what a string actually is.

type StringHeader struct {
    Data uintptr
    Len int
}

A string is a pointer to a piece of memory containing the bytes of the string, and a length of the string. So our slice of 100 million strings contains 100 million pointers and 100 million lengths. And 100 million separate allocations which hold the bytes for the strings.

Our solution

Instead of having 100 million separate allocations and 100 million pointers, we can allocate a single slice of bytes to contain all the bytes for all the strings, and make our own string-like objects that contain offsets into this slice.

We define a string bank to contain the string bytes.

type stringbank []byte

And this is our “banked” version of a string with offsets instead of pointers.

type bankedString struct {
    offset int
    len int
}

We can make a function to add a string to the string bank and return a bankedString. This copies the bytes from the string into our string bank, and saves the offset of the string and the length of the string.

func (sb *stringbank) store(in string) bankedString {
    offset := len(*sb)
    *sb = append(*sb, in…)
    return bankedString{
        offset: offset,
        len: len(in),
    }
}

This bankedString can then be used to retrieve the original string.

func (sb stringbank) get(in bankedString) string {
    return string(sb[in.offset : in.offset+in.len])
}

Storing our random strings needs just a little modification. We need to allocate a stringbank, which we make big enough to hold all our string data, and we keep a []bankedString instead of a []string.

sb := make(stringbank, 0, 20*100*1000*1000)
space := make([]bankedString, 100*1000*1000)
for i := range space {
    space[i] = sb.store(fmt.Sprintf("%20.20d", rand.Int()))
}

If we now time GC we get a marked improvement.

GC takes 108.166528ms

This is still quite a long time for GC, but if we run GC again we see a further big drop. The first run of the GC frees up temporary strings we’ve created (rather carelessly) while we build our slice of strings. Once this is done, the GC overhead is practically nil.

GC takes 348.923µs

Conclusions

I doubt it makes sense to do this kind of thing normally. It only really makes sense if you are going to keep the strings for the lifetime of your process as there’s no way to delete individual strings.

What does this say about other situations? Perhaps you don’t want to store a huge amount of data. Perhaps you’re building some kind of API service. Does this stuff apply? Well, if across all your goroutines and API handlers you use a significant amount of RAM then perhaps it does. If you can avoid using pointers here and there, perhaps some of your allocations will end up being pointer-free, and this may reduce the overall CPU usage of the GC. Which might make your program perform better, or cost less to run. Just make sure you measure things before and after any change to be sure you actually make an improvement.