amir

Posted on May 31

The 1978 Paper Behind Go’s Concurrency Model

#go #concurrency #architecture

Every Go developer eventually hears this sentence:

Do not communicate by sharing memory; share memory by communicating.

At first, it sounds like a nice Go proverb.

But the more concurrent systems you build, the more you realize it is not just a slogan. It is a completely different way of thinking about software design.

When I first started working deeply with Go concurrency, I mostly thought about goroutines as “lightweight threads” and channels as “safe queues.” That mental model is useful at the beginning, but it is not the full story.

The real story goes much deeper.

Go’s concurrency model is heavily inspired by a paper from 1978: Communicating Sequential Processes, written by Tony Hoare.

This article is my attempt to explain that idea in a practical way: why shared-memory concurrency becomes painful, what CSP was trying to solve, and how Go turned that theory into something we can use every day in production systems.

The problem: shared state looks simple until it does not

Most concurrency problems start innocently.

You have multiple workers. They need access to the same data. So you put the data in memory and let the workers read and write it.

Something like this:

type Counter struct {
    value int
}

func (c *Counter) Inc() {
    c.value++
}

Then concurrency enters the system:

counter := &Counter{}

for i := 0; i < 1000; i++ {
    go counter.Inc()
}

Now the code is broken.

Multiple goroutines can read and write value at the same time. The final result becomes unpredictable. So we add a mutex:

type Counter struct {
    mu    sync.Mutex
    value int
}

func (c *Counter) Inc() {
    c.mu.Lock()
    defer c.mu.Unlock()

    c.value++
}

This fixes the data race.

But this is where the next problem starts.

Mutexes are not bad. They are necessary in many real systems. The issue is that shared state plus locks can easily become the default architecture.

And once that happens, your system becomes harder to reason about.

You start asking questions like:

Who owns this data?
Which goroutine is allowed to update it?
How long is this lock held?
Can this function call another function that also needs the same lock?
Can this code deadlock under load?
Why is p99 latency suddenly worse?

This is especially painful in backend systems with high-throughput pipelines, network services, log processors, container runtimes, or in-memory systems where thousands of operations happen concurrently.

The code may look safe because it has locks.

But safe does not always mean simple.

And simple is what keeps production systems maintainable.

The hidden cost of “just add a mutex”

A mutex protects memory, but it also serializes access.

That means one slow operation inside a lock can block many other goroutines.

I have seen patterns like this in real backend code:

mu.Lock()

user.Name = "Test User"

sendEmail(user)
callExternalAPI(user)
writeAuditLog(user)

mu.Unlock()

Technically, the shared data is protected.

Architecturally, this is a problem.

The lock is not only protecting the small mutation of user.Name. It is now protecting email sending, external API calls, logging, and anything else that happens inside that critical section.

That means every goroutine waiting for this lock must wait for the whole flow.

The service may have goroutines. It may look concurrent. But a big part of the request path has silently become sequential.

This is why concurrency bugs are not always about missing locks.

Sometimes the problem is too much locking.

Sometimes the real bug is ownership.

Tony Hoare’s idea: stop sharing memory directly

In 1978, Tony Hoare introduced Communicating Sequential Processes, usually called CSP.

The idea was very different from the classic shared-memory model.

Instead of many processes fighting over the same memory, CSP describes a system as independent sequential processes that communicate by sending messages.

The important parts are:

Each process has its own local state.
Processes do not casually share variables.
When they need to coordinate, they communicate explicitly.
Communication is part of the design, not an afterthought.

That sounds simple, but it changes how you design systems.

Instead of asking:

Which lock protects this data?

You start asking:

Which goroutine owns this data?

That is a much more powerful question.

Because if one goroutine owns the state, other goroutines do not need to mutate it directly. They send messages to the owner.

This is the mental shift behind Go channels.

Go did not just add concurrency. It gave concurrency a shape.

Go could have chosen the same path as many other languages:

threads
locks
shared memory
concurrency libraries
complex abstractions built on top

But Go made a different design decision.

It gave concurrency first-class language support with:

go for starting goroutines
chan for communication
select for coordinating multiple communication operations

This matters because concurrency is not treated as a library feature bolted onto the language.

It is part of the language’s design.

A goroutine is not exactly the same as a CSP process, and Go is not a pure CSP language. Go still allows shared memory, mutexes, atomics, and low-level synchronization when needed.

But the spirit of CSP is clearly there:

Build independent units of execution and make them communicate explicitly.

That is why Go feels so natural for network servers, infrastructure tools, distributed systems, streaming pipelines, and cloud-native software.

A simple CSP-style pipeline in Go

Let’s look at a practical example.

Imagine a small pipeline:

Generate jobs.
Process jobs.
Return results.

A shared-memory mindset might create a global queue, protect it with a mutex, and let workers pull from it.

A CSP-style mindset is different.

The data flows through channels.

package main

import (
    "fmt"
    "time"
)

type Job struct {
    ID int
}

type Result struct {
    JobID int
    Value string
}

func producer(out chan<- Job) {
    defer close(out)

    for i := 1; i <= 5; i++ {
        out <- Job{ID: i}
    }
}

func worker(in <-chan Job, out chan<- Result) {
    for job := range in {
        time.Sleep(100 * time.Millisecond)

        out <- Result{
            JobID: job.ID,
            Value: fmt.Sprintf("processed job %d", job.ID),
        }
    }
}

func main() {
    jobs := make(chan Job)
    results := make(chan Result)

    go producer(jobs)

    go func() {
        defer close(results)
        worker(jobs, results)
    }()

    for result := range results {
        fmt.Printf("job=%d result=%q\n", result.JobID, result.Value)
    }
}

There is no shared slice.
There is no global queue.
There is no explicit mutex.

The producer owns job creation.
The worker owns processing.
The main goroutine owns result collection.

The data moves through the system.

That is the important part.

We are not asking multiple goroutines to mutate the same object at the same time. We are designing a flow of ownership.

Scaling the pipeline with multiple workers

Now let’s make it more realistic.

A single worker is not enough for heavy workloads. We want multiple workers processing jobs concurrently.

package main

import (
    "fmt"
    "sync"
    "time"
)

type Job struct {
    ID int
}

type Result struct {
    WorkerID int
    JobID    int
    Value    string
}

func producer(out chan<- Job, count int) {
    defer close(out)

    for i := 1; i <= count; i++ {
        out <- Job{ID: i}
    }
}

func worker(workerID int, jobs <-chan Job, results chan<- Result) {
    for job := range jobs {
        time.Sleep(100 * time.Millisecond)

        results <- Result{
            WorkerID: workerID,
            JobID:    job.ID,
            Value:    fmt.Sprintf("processed job %d", job.ID),
        }
    }
}

func main() {
    const workerCount = 3
    const jobCount = 10

    jobs := make(chan Job)
    results := make(chan Result)

    go producer(jobs, jobCount)

    var wg sync.WaitGroup

    for i := 1; i <= workerCount; i++ {
        wg.Add(1)

        go func(workerID int) {
            defer wg.Done()
            worker(workerID, jobs, results)
        }(i)
    }

    go func() {
        wg.Wait()
        close(results)
    }()

    for result := range results {
        fmt.Printf(
            "worker=%d job=%d result=%q\n",
            result.WorkerID,
            result.JobID,
            result.Value,
        )
    }
}

This is one of the places where Go shines.

The design is still readable:

The producer sends jobs.
Workers receive jobs.
Workers send results.
The result channel closes when all workers finish.

We still use sync.WaitGroup, because Go is pragmatic. CSP-style design does not mean “never use the sync package.”

It means we use synchronization intentionally.

The channel handles data flow.
The wait group handles lifecycle coordination.

That separation is clean.

The real value: ownership becomes visible

The biggest advantage of CSP-style design is not that it removes every mutex.

The biggest advantage is that it makes ownership visible.

In shared-memory systems, ownership is often hidden.

You see a pointer passed around. You see a struct used in multiple places. You see a lock somewhere. But it is not always obvious who is responsible for the state.

With channels, ownership is easier to see.

For example:

jobs <- job

This line says:

I am sending this job to another part of the system.

And this line:

job := <-jobs

says:

I am receiving this job and now I am responsible for processing it.

That clarity matters.

In production systems, many bugs are not caused by complex algorithms. They are caused by unclear ownership.

Who closes this channel?
Who updates this state?
Who retries this job?
Who owns cancellation?
Who handles backpressure?

A channel-based design forces you to answer these questions earlier.

Backpressure is built into the model

Another underrated benefit of channels is backpressure.

An unbuffered channel forces the sender and receiver to synchronize:

jobs := make(chan Job)

If the producer sends faster than the worker receives, the producer blocks.

That is not a bug. That is backpressure.

Buffered channels allow some temporary queueing:

jobs := make(chan Job, 100)

Now the producer can get ahead by 100 jobs, but not forever.

This is powerful in real systems.

For example, imagine a log processing service:

HTTP requests receive logs.
A parser normalizes them.
A batcher writes them to storage.
A separate worker sends alerts.

Without backpressure, one fast layer can overwhelm a slower layer.

With channels, you can make pressure visible and controlled.

You can decide where the system should block, buffer, drop, retry, or shed load.

That is architecture, not just syntax.

Where channels are a bad fit

It is also important to be honest: channels are not magic.

Not every concurrency problem becomes better with channels.

Sometimes a mutex is simpler and faster.

For example, this is perfectly reasonable:

type Metrics struct {
    mu       sync.Mutex
    requests int64
    errors   int64
}

func (m *Metrics) IncRequests() {
    m.mu.Lock()
    m.requests++
    m.mu.Unlock()
}

For small, local, short-lived critical sections, a mutex is often the cleanest solution.

Channels can become messy when they are used only because they feel “more Go-like.”

Bad channel usage can create:

goroutine leaks
unclear lifecycle
blocked sends
blocked receives
complicated shutdown logic
over-engineered code

The point is not:

Always use channels.

The point is:

Use channels when communication and ownership are the core problem.

Use mutexes when protecting a small piece of shared state is the core problem.

Senior Go engineering is knowing the difference.

A practical rule I use

When I design concurrent Go code, I usually ask myself:

1. Is this state owned by one goroutine?

If yes, channels can be a great fit. Other goroutines can send commands or data to the owner.

2. Is this just a small shared counter or cache?

A mutex or atomic may be better.

3. Do I need backpressure?

Channels make this easier to model.

4. Do I need cancellation?

Then I design the channel flow together with context.Context.

5. Who closes the channel?

If I cannot answer this clearly, the design is not finished.

That last question is very important.

Channel ownership is not only about sending data. It is also about lifecycle.

Usually, the sender closes the channel. Receivers should not close a channel they do not own.

CSP thinking in modern backend systems

The reason I like CSP is that it maps well to real backend architecture.

A backend service is not just functions calling functions.

It is a system of flows:

requests flow into handlers
jobs flow into queues
events flow into processors
logs flow into pipelines
metrics flow into aggregators
commands flow into state owners
results flow back to clients

When you think in flows, Go becomes very natural.

This is also why Go became popular in infrastructure software. Tools like Docker, Kubernetes, and Terraform are written in Go not only because Go compiles to a static binary, but also because its concurrency model fits the kind of problems infrastructure software needs to solve.

Infrastructure software is full of concurrent work:

watching state
reconciling resources
handling network calls
streaming logs
scheduling tasks
managing timeouts
coordinating workers

CSP-style thinking gives these systems a clean structure.

Final thoughts

Tony Hoare’s CSP paper is almost 50 years old, but the idea still feels modern.

That is rare in software.

Many technologies become outdated quickly, but good mental models survive.

Go did not invent the idea of communicating sequential processes. But Go made the idea practical for everyday engineers.

That is the beauty of Go’s concurrency model.

It takes a deep computer science concept and gives us a small set of simple tools:

go func()
chan
select

The tools are simple.

The design thinking behind them is powerful.

For me, the biggest lesson is this:

Concurrency becomes easier when we stop thinking only about shared state and start thinking about ownership, communication, and flow.

That is the real value of CSP in Go.

Not just fewer mutexes.

Better architecture.

References

Tony Hoare — Communicating Sequential Processes (1978)
Rob Pike — Go Concurrency Patterns
Rob Pike — Concurrency Is Not Parallelism
Go Blog — Share Memory By Communicating

DEV Community