Introduction to `bufio` in Go: Why Buffered I/O Matters

Peyman Ahmadi — Sat, 24 May 2025 07:25:51 +0000

(Part 1 of 7: Mastering Buffered I/O in Go)

Why Should You Care About `bufio`?

Imagine you’re filling a swimming pool. You have two choices:

The Spoon Method: Scoop water one spoonful at a time.
The Bucket Method: Dump entire buckets in one go.

Which is faster? Obviously, the bucket.

Now, replace "water" with "data," and you’ve got the difference between unbuffered I/O and buffered I/O in Go.

Every time your Go program reads a file or writes to a network socket, it makes a system call—a request to the operating system. These calls are slow. Like, "waiting-for-your-coffee-to-brew" slow.

bufio is your performance savior. It wraps I/O operations with an in-memory buffer, reducing system calls by batching data. Fewer calls = faster code.

In this article, we’ll explore:

✔ What buffered I/O really means (no jargon, promise).

✔ When to use bufio (and when to skip it).

✔ A real benchmark proving it’s not just hype.

Let’s get started!

Buffered I/O: The "Bucket" for Your Data

The Problem with Unbuffered I/O

When you read a file in Go using os.ReadFile or ioutil.ReadAll, each Read() call goes straight to the OS. If you’re reading a 1MB file byte-by-byte, that’s 1 million system calls!

🚫 Bad:

file, _ := os.Open("large.log")  
data := make([]byte, 1)  
for {  
    _, err := file.Read(data) // 1 byte per system call!  
    if err != nil {  
        break  
    }  
}

(Don’t do this. Your CPU will cry.)

The `bufio` Solution

bufio.Reader acts like a middleman, reading big chunks upfront and serving them from memory:

✅ Good:

file, _ := os.Open("large.log")  
reader := bufio.NewReader(file)  
for {  
    line, err := reader.ReadString('\n') // Reads in chunks!  
    if err != nil {  
        break  
    }  
    fmt.Print(line)  
}

(Fewer system calls, happier server.)

When Should You Use `bufio`?

1. Reading Large Files

Logs, CSVs, or any file bigger than a few KB.
Bonus: bufio.Scanner makes line-by-line reading trivial.

2. High-Frequency Small Writes (Like Logging)

Without buffering, writing 100 short log lines = 100 system calls. With bufio.Writer, it might be just 1 call.

3. Network I/O (HTTP, TCP, etc.)

Ever parsed an HTTP request? bufio.Reader is why Go’s net/http is so efficient.

Benchmark Showdown: Unbuffered vs. `bufio` (Real Numbers!)

Enough theory—let’s break out the stopwatch and see how bufio actually performs. I ran four tests on a 10MB file (filled with random data to prevent OS optimizations):

Unbuffered, 1-byte reads (The "Spoon Method")
Buffered, 1-byte reads (The "Bucket Method")
Unbuffered, 4KB reads (Chunking manually)
Buffered, 4KB reads (Letting bufio handle chunks)

Here’s the code (full repo here):

// BenchmarkUnbufferedRead: Painfully slow 1-byte reads
func BenchmarkUnbufferedRead(b *testing.B) {
    file, _ := os.Open("10mb.log")
    defer file.Close()
    buf := make([]byte, 1)
    for {
        if _, err := file.Read(buf); err == io.EOF {
            break
        }
    }
}

(Other benchmarks here.)

The Results (On My Intel Ultra 5 Laptop)

Method	Speed (ns/op)	Memory (B/op)	Allocs (per op)	Verdict
Unbuffered (1B)	6,676,623,112	120	3	🐌 Snail pace
Buffered (1B)	35,121,689	4,216	5	🚀 190x faster!
Unbuffered (4KB)	2,197,041	120	3	Decent, but manual work
Buffered (4KB)	2,292,039	4,224	4	Similar, but `bufio` handles edge cases

Key Takeaways

1-byte reads without buffering are a crime (6.6 seconds per run!).
bufio makes 1-byte reads viable (35ms vs. 6.6s—yes, really).
For 4KB chunks, manual and bufio are close—but bufio wins on:
- Code simplicity (no manual chunk sizing).
- Edge cases (e.g., partial reads, EOF handling).

Why Is Buffered 1B So Much Faster?

Every file.Read(buf) in the unbuffered version hits the OS. bufio.Reader does this:

Grabs a 4KB chunk from the OS upfront (default buffer size).
Serves your tiny 1-byte reads from memory.
Only goes back to the OS when the buffer is empty.

(Like filling a water glass from a pitcher instead of the tap each time.)

When to Skip `bufio`

If you’re already reading in large chunks (e.g., 4KB+), bufio adds minor overhead. But for most cases—especially line-by-line reading (ReadString/Scanner)—it’s worth it.

Try it yourself:

git clone https://github.com/peymanahmadi/bufio-benchmark
go test -bench=. -benchmem

(Share your results! Did bufio speed up your code? Let me know.)

What’s Next?

Now that you’re convinced bufio is magic, let’s dig deeper.

Next article: "Mastering bufio.Reader in Go" where we’ll cover:

How to tweak buffer sizes.
The difference between Read(), ReadString(), and ReadLine().
Handling edge cases (like partial reads).

Your Homework:

Replace one os.Read call in your code with bufio and measure the difference. Got a cool result? Tweet it at me! 🚀

Final Thought

Buffered I/O isn’t just an optimization—it’s how performant Go code is written. Whether you’re parsing logs or handling HTTP requests, bufio is your silent speed booster.

Next time you see a slow I/O operation, ask yourself: "Am I using a spoon instead of a bucket?"

Stay tuned for Part 2!

💬 Discussion Prompt:

Have you ever been burned by slow I/O in Go? Share your war stories below!

Series Index

Introduction to bufio (You’re here!)
Mastering bufio.Reader
bufio.Scanner Deep Dive
Efficient Writing with bufio.Writer
Advanced bufio Tricks
Common bufio Pitfalls
Real-World bufio Use Cases

DEV Community: Peyman Ahmadi