Introduction to Concurrency in Go

#beginners #go

Modern processors today are "fast" because they have multiple cores. Multiple cores are useful since we can split up big problems into smaller chunks which can be given to multiple cores concurrently.

However, many basic tutorials only expect you to use one processor core at a time for simplicity's sake.

This is because using multiple cores requires real threads. Using threads is hard because syncing concurrent work is challenging. Working concurrently is challenging because each thread (or chunk of work) will finish at a different time, and usually out-of-order. Some tasks may require you to combine the results of each thread, and be impacted by the order in which you combine the results. Additionally, choosing how many threads to spawn isn't a trivial exercise.

Go set out to make concurrency easy, even for beginners.

Instead of real threads, Go supplies goroutines, which are like lightweight threads. Go makes handling concurrency easy by providing tools built in to the language's stdlib itself. Additionally, Go handles creating the right number of real threads to handle the goroutines you spawn. Go provides you with ways to respond to the results of each goroutine, and to wait for all the goroutines to finish.

Waiting for goroutines is a simple operation using WaitGroups. WaitGroups are like counters of goroutines in progress. Here's an outline of how to use WaitGroups:

Initialize a WaitGroup
Add(1) to the WaitGroup
Spawn a goroutine
Call Done() on the WaitGroup when the goroutine finishes (to decrement the WaitGroup by one)
Wait() for the WaitGroup (execution will await the counter returning to 0).

But once you have spawned goroutines, how can we recombine the results?

One option is to pass the goroutine a pointer to a variable, and allow it to mutate that variable itself. In order to demonstrate passing pointers, I am using the "Count the divisors of a number) Codewars Kata as an example.

The task is, "Count the number of divisors of a positive integer n."

For example

divisors(4)  == 3  //  1, 2, 4
divisors(5)  == 2  //  1, 5
divisors(12) == 6  //  1, 2, 3, 4, 6, 12
divisors(30) == 8  //  1, 2, 3, 5, 6, 10, 15, 30

Below is a solution using pointers to pass a variable by reference to increment with the count of divisors of the number n.

Another option is to use channels. When using channels, goroutines communicate via messages rather than using a shared pointer. Go provides us with range operator which lets us iterate over an array, a slice, a string, a map, or a channel. Using a for loop on a channel lets us process each message as they are received.

for x := range ch {
 // do some work
}

After we're done with a channel, we should remember to close it to indicate we aren't waiting for other operations to finish. Otherwise, we may deadlock execution (one of the pesky multi-threading challenges we want to avoid).

By using channels, we can split up work and communicate back the results to the central handler. See the example below.

And in case you want to run the code above, here is an embedded repl.it for tweaking, forking, testing, and reviewing the Go concurrency patterns outlined above

Somewhat surprisingly, a simple loop without concurrency was the fastest, and I am still wondering why. One other selling point for Golang is it's built in support for testing and benchmarking. Below I wrote a benchmark for the functions above. I think the simple loop is fastest since it doesn't do any memory allocation, and perhaps due to compiler optimization. However,for a real workload, I still expect concurrency and parallelization to be the best for overall performance.

Great resources to continue with after this: