Getting to Yes -- As Quickly as Possible

#go #beginners #linux

There was a great discussion a year ago about how fast gnu's version of "yes" is. If you're unfamiliar, yes outputs y indefinitely.

yes |head -5
y
y
y
y
y

The key takeaway was that write is expensive and writing page-aligned buffers is much faster. The is true across languages, so let's see how to do it properly in go.

If you're shocked or impressed by the results, let's see you do it in your language -- post your results in the comments.

First, our benchmark C code from last-year's post.

/* yes.c - iteration 4 */
#define LEN 2
#define TOTAL 4096 * 4
int main() {
    char yes[LEN] = {'y', '\n'};
    char *buf = malloc(TOTAL);
    int bufused = 0;
    while (bufused < TOTAL) {
        memcpy(buf+bufused, yes, LEN);
        bufused += LEN;
    }
    while(write(1, buf, TOTAL));
    return 1;
}

And now the throughput benchmark

$ gcc yes.c -o yes_c
$ ./yes_c |pv >/dev/null
...[3.21GiB/s] ...

So our Benchmark rate = 3.21 GB/s

Go Round 1 -- Captain Obvious

Hypothesis: this will be slow. I'm guessing 20% as fast as the benchmark.

package main

import (
    "fmt"
    "os"
)

func main() {
    for {
        fmt.Fprintln(os.Stdout, "y")
    }
}

$ go run yes.go |pv > /dev/null 
$ ...[1.07MiB/s] ...

Yikes! 1.07 MB/s or only 0.03% of our benchmark. 💩 Hypothesis confirmed!

Go Round 2 -- Page-aligned buffers

note on Darwin Pagesize X 4 produced the best results -- i'll leave that to the readers to guess why.

package main

import (
    "os"
)

func main() {
    buf := make([]byte, os.Getpagesize()*4)
    for i := 0; i < len(buf); i += 2 {
        buf[i], buf[i+1] = 'y', '\n'
    }
    for {
        os.Stdout.Write(buf)
    }
}

results:

$ go run yes.go |pv > /dev/null 
$ ...[3.15GiB/s] ...

Results : 3.15 GB/s or 98.13% of our benchmark. We have a winner 🏁

➡️ How fast can you "Get to Yes" in your favorite language?

Top comments (3)

Dmitry Yakimenko • Mar 30 '19 • Edited

I'll bet the slowness in the first Go snippet comes from using fmt.Fprintln overhead and the fact you're writing 1 byte. It has nothing to do with page alignment, but rather with the buffering. You're writing many more bytes at once in the second example, doesn't matter aligned or not. Make the buffer of weird length, like 13377 bytes and try. I bet it will be as fast.