DEV Community: Sonu Kumar Saw

Mutex Lock in Golang

Sonu Kumar Saw — Fri, 04 Mar 2022 15:00:14 +0000

This article was originally posted at ioscript.org

While writing concurrent code in Golang, often programs face situations when two or more goroutines try to access and modify shared variables. Such conditions can lead to inconsistency in data stored in the shared variable. Such conditions are called Race conditions. You can learn more about race conditions from our previous article of this course, Race conditions in Golang.

Let's see one example of Race Condition:

package main

import (
    "fmt"
    "sync"
    "time"
)

var wg sync.WaitGroup

func main() {

    var counter int
    fmt.Println("Initial value: ", counter)

    // deploy 5 goroutines
    for i := 0; i < 5; i++ {

        wg.Add(1)

        go func() {
            defer wg.Done()
            //increment the counter 100 times
            for j := 0; j < 100; j++ {
                temp := counter
                time.Sleep(time.Microsecond * 1)
                temp += 1
                counter = temp
            }

        }()

    }

    wg.Wait()
    fmt.Println("Final value: ", counter)
}

Output

Initial value:  0
Final value:  102

As we can see in the output, there's definitely an inconsistency in our counter variable. The expected value of the counter is 500 and we go the 102. Let's find out where is the issue in the above code causing the Race condition. To find the race condition in our code, we can use the command go run -race main.go.

When we run/build our code with the -race flag on. It generates the following output.

Initial value:  0
==================
WARNING: DATA RACE
Read at 0x00c000126008 by goroutine 8:
  main.main.func1()
      ioScript/concurrency/main.go:25 +0xa8

Previous write at 0x00c000126008 by goroutine 7:
  main.main.func1()
      ioScript/concurrency/main.go:28 +0xce

Goroutine 8 (running) created at:
  main.main()
     ioScript/concurrency/main.go:21 +0xdb

Goroutine 7 (running) created at:
  main.main()
      ioScript/concurrency/main.go:21 +0xdb
==================
Final value:  101
Found 1 data race(s)
exit status 66

We can see in the logs that go runtime found 1 data race. A race condition was found due to lines 25 and 28. There is a read operation in line 25 and a write operation at 28. Goroutine 8 is reading from the counter variable and Goroutine 7 is trying to write to the counter variable.

We can solve this race condition if we somehow restrict the read and write operation of the shared variable to one goroutine at a time.

sync.Mutex

Mutex is a synchronization primitive which restricts one or more goroutine to enter the critical section while another goroutine is executing the critical section.

When one goroutine is executing the critical section of the code, mutex locks the section. Other goroutines need to wait till the execution is complete and the lock is released by the first goroutine.

Let's try to fix our above code using mutex lock.

package main

import (
    "fmt"
    "sync"
    "time"
)

var wg sync.WaitGroup

func main() {

    var counter int
    var mu sync.Mutex

    fmt.Println("Initial value: ", counter)

    // deploy 5 goroutines
    for i := 0; i < 5; i++ {

        wg.Add(1)

        go func() {
            defer wg.Done()
            //increment the counter 100 times
            for j := 0; j < 100; j++ {

                mu.Lock()
                temp := counter
                time.Sleep(time.Microsecond * 1)
                temp += 1
                counter = temp
                mu.Unlock()
            }

        }()

    }

    wg.Wait()
    fmt.Println("Final value: ", counter)
}

Output

ioscript@ioscript:concurrency$  go run -race main.go 
Initial value:  0
Final value:  500

So, let's break down what we have done here. We have declared a variable name mu which is of type Mutex provided by the sync package of golang. Mutex type provides two methods: Lock() and Unlock().

Lock( ): As the name suggests, it locks the mutex. If the lock is already in use, other goroutine blocks until the mutex is available.
Unlock( ): It unlocks the mutex. It throws a runtime error if the mutex is not locked when unlock was called.

In our code above, we have used mu.Lock() just before we are reading a value from the counter variable. Then we are incrementing the value and write the value to the counter variable again. Since read-write operations are complete, we are releasing the lock on mutex using mu.Unlock( ) so that other goroutines can acquire the lock again and execute the critical section.

When we run this code with the -race flag, we don't get any error message and the correct output is displayed in the code.

Conclusion

In this tutorial, we had a quick look into Race conditions and how they can create data inconsistency in our program. We then learn about Mutex Locks and how we can use them to solve race conditions in our code and ensure that our program always returns expected results.

Before You Leave

If you found this article valuable, you can support us by dropping a like and sharing this article with your friends. You can find more articles in this series here

You can sign up for our newsletter to get notified whenever we post awesome content on Golang.

You can also let us know what you would like to read next? Drop a comment or email @ sonukumarsaw@ioscript.org

Reference

Race Condition in Golang

Sonu Kumar Saw — Sat, 26 Feb 2022 19:19:50 +0000

This article was originally posted at ioscript.org

Writing a concurrent code is often a difficult task. often programmers working with concurrent code, face similar kinds of bugs. One of them is inconsistency while manipulating the data using concurrent goroutines.

Let's see an example.

package main

import (
    "fmt"
    "sync"
)

var msg string
var wg sync.WaitGroup

func foo() {
    defer wg.Done()
    msg = "Hello Universe"
}

func bar() {
    defer wg.Done()
    msg = "Hello Cosmos"
}

func main() {

    msg = "Hello world"

    wg.Add(2)
    go foo()
    go bar()

    wg.Wait()
    fmt.Println(msg)

}

Note: If you are not familiar with Goroutines and WaitGroups, I recommend checking out our course on Concurrency in Golang

In the above code example, we are deploying two goroutines named foo and bar. Both goroutines are updating a variable named message to "Hello Universe" and "Hello Cosmos" accordingly. When we run this code, It will deploy the two goroutines. These goroutines eventually update the value of msg and join back to the main goroutine. Since two goroutines are updating the same variable in the above code, we cannot determine the output of the fmt.Println(msg), i.e, It will either print "Hello Universe" or "Hello Cosmos". Since we cannot determine the value stored in the msg variable, this will be a serious bug in our code that can break our software.

Let's see another example with a real-life scenario. Michael and his wife have a joint bank account, they use the same account to make purchases. One fine day, Michael went to have coffee at Starbucks 🥤🥤. Meanwhile, her wife went shopping and purchased a beautiful dress 👗👗 for herself. They paid their bills simultaneously, But the bank software was not implemented correctly which resulted in an inconsistency in their balance amount at the bank.

Let's have a look at the bank software's code and see what went wrong.

package main

import (
    "fmt"
    "sync"
)

var bankBalance int
var wg sync.WaitGroup

func Purchase(purchaseAmount int) {
    defer wg.Done()

    value := bankBalance
    value = value - purchaseAmount
    bankBalance = value
}

func main() {

    //initially Michael and his wife had $1000 USD in their bank account
    bankBalance = 1000

    //they made bill payment simultaneously
    wg.Add(2)

    go Purchase(5)
    go Purchase(157)

    wg.Wait()

    //final amount in there bank account
    fmt.Println("Final amount : ", bankBalance)
}

When we run the above code, It can result in 995, 838 or 843. Final values completely depend on the order of execution of the Purchase goroutine.

The above case is called Race condition in programming.

Race Condition

A race condition occurs when two or more concurrent goroutines try to access and modify the same data. For example, if one goroutine tries to read a variable, meanwhile other goroutines are trying to update the value of the same variable.

Race condition mostly occurs, if the developer thinks the concurrent program is executing sequentially. In the above example of bank software code, the developer might have assumed go Purchase(5) will complete its execution first than go Purchase(157).

How to detect race conditions in GO ?

Go provides built-in support to tackle the Race condition issue. We can use the -race flag while compiling or building our code to detect the race conditions in our program.

Let's run our code above with the -race flag to detect where the race condition is actually happening.

sonukumarsaw@legion-5PRO:concurrency$ go run -race main.go
==================
WARNING: DATA RACE
Read at 0x0000005d4400 by goroutine 7:
  main.Purchase()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:14 +0x74
  main.main·dwrap·2()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:27 +0x39

Previous write at 0x0000005d4400 by goroutine 8:
  main.Purchase()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:16 +0x8c
  main.main·dwrap·3()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:28 +0x39

Goroutine 7 (running) created at:
  main.main()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:27 +0x90

Goroutine 8 (finished) created at:
  main.main()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:28 +0xd2
==================
Final amount : 838
Found 1 data race(s)
exit status 66

Well, there's a lot of things going on here. Let's go through it step by step. First of all, we Found 1 data race. Now let's see the most important lines

Goroutine 7 (running) created at:
  main.main()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:27 +0x90

Goroutine 8 (finished) created at:
  main.main()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:28 +0xd2

Above two message shows, two goroutines are deployed named Goroutine 7 at line 27, i.e, go Purchase(5) and Goroutine 8 at line 28, i.e, go Purchase(157). Also, Goroutine 8 has already completed its execution while Goroutine 7 is still running.

Now let's see the next part.

Read at 0x0000005d4400 by goroutine 7:
  main.Purchase()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:14 +0x74
  main.main·dwrap·2()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:27 +0x39

Previous write at 0x0000005d4400 by goroutine 8:
  main.Purchase()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:16 +0x8c
  main.main·dwrap·3()
      /home/sonukumarsaw/alpha/ioScript/concurrency/main.go:28 +0x39

It clearly states that Read at 0x0000005d4400 by goroutine 7: and Previous write at 0x0000005d4400 by goroutine 8:. This means Goroutine 7 is trying to read the same memory address 0x0000005d4400 which was earlier written by Goroutine 8.

Even though, we could detect the race condition and its cause. We can still see the correct amount in the output i.e, Final amount : 838.

Let's make some changes in our code to make sure, our code returns the incorrect amount.

package main

import (
    "fmt"
    "sync"
)

var bankBalance int
var wg sync.WaitGroup

func Purchase(purchaseAmount int) {
    defer wg.Done()

    value := bankBalance
    time.Sleep(1 * time.Second)                 // some delay in processing the calculation
    value = value - purchaseAmount
    bankBalance = value
}


func main() {

    //initially Michael and his wife had $1000 USD in their bank account
    bankBalance = 1000

    //they made bill payment simultaneously
    wg.Add(2)

    go Purchase(5)
    go Purchase(157)

    wg.Wait()

    //final amount in there bank account
    fmt.Println("Final amount : ", bankBalance)
}

We have added time.Sleep() to imitate a processing delay. Now when we run this program, we will never get the correct amount in the output.

How to fix the race condition ?

To avoid race conditions in our program, any operation in a shared variable, must be executed atomically. Shared variables are those variables that are shared among the goroutines.

We can execute a program atomically by locking the critical section, a section of the code where shared variables are being manipulated. Once, the execution of the critical section is completed by one goroutine, other goroutines can acquire the lock and complete its execution as well.

We can use go's built-in Mutex lock. Mutex lock is a part of the sync package. We will see how to fix our code using Mutex lock in our next article.

Before You Leave

If you found this article valuable, you can support us by dropping a like and sharing this article with your friends.

You can sign up for our newsletter to get notified whenever we post awesome content on Golang.

Reference

WaitGroups in Golang

Sonu Kumar Saw — Tue, 22 Feb 2022 08:17:59 +0000

This article was originally posted at ioscript.org

In the previous article, Introduction to Concurrency in Golang, of the series,Concurrency in Golang, we came across a problem where one or more goroutines might not necessarily finish execution before the main goroutine does, thus we were unable to see the message printed by the unfinished goroutine. We fixed the issue by adding time.Sleep() in the main goroutine to put it to sleep for a few seconds and thus increasing the probability of other goroutines finishing before the main does. Although our code worked in that particular case, most certainly it will not work for all cases. In this article, we will see, how we can make use of the **WaitGroup **synchronization primitive provided by Go.

Before we discuss WaitGroups let's have a look into Go's concurrency model fork-join.

Fork-Join Model

In concurrency, Fork-Join Model is a way of setting up and executing concurrent programs, such that execution branches off in designated points and joins or merges at a subsequent point and resumes the sequential execution. The word Fork refers to any point of time in the program runtime, it can create one or more child branches of execution to be run either concurrently or in some cases parallel to the parent. The word Join refers to the point in the future when concurrent execution branches
join back to the parent.

WaitGroups

Now, that we know more about the concurrency model, Fork-Join. Let's go through our code from the last article and try to fix it without putting the main goroutine to sleep.

package main

import "fmt"

func sayGreetings() {
    fmt.Println("Hello World!!")
}

func main() {
    go sayGreetings()
}

When we run the above code, there will be two possibilities. First, the sayGreetings goroutine prints Hello World! and complete its execution before the main goroutine does. In that case, both Fork and Join operations will happen. In the second case, the sayGreetings goroutine won't be able to complete its execution before the *main * goroutine does. In such a case, the Join operation will not happen.

To solve the above issue, we need to make sure that the Join operation happens in all the cases before the main goroutine completes its execution. We can use synchronization primitive WaitGroup provided by the sync package of Go's inbuilt library.

A WaitGroup **waits for a collection of goroutines to finish. We can think of **WaitGroup as a concurrent-safe counter. WaitGroup provides three methods attached to it.

    type WaitGroup

        func (wg *WaitGroup) Add(delta int)
        func (wg *WaitGroup) Done()
        func (wg *WaitGroup) Wait()

func (wg *WaitGroup) Add(delta int) method increments the counter by the integer passed in. Integer passed into the Add method means the number of new goroutines created.
func (wg *WaitGroup) Done() method decrements the counter by one. We call the Done method when a goroutine completes its execution.
func (wg *WaitGroup) Wait() method is used to block the execution until all the goroutines deployed have already completed their execution.

Let's see how to use WaitGroups in our previous code


package main

import (
    "fmt"
    "sync"
)

var wg sync.WaitGroup    // 1

func sayGreetings() {

    defer wg.Done()      // 3

    fmt.Println("Hello World!!")

}

func main() {

    wg.Add(1)           // 2  
    go sayGreetings()

    wg.Wait()           // 4
}

The above example will deterministically block the main goroutine until the sayGreetings goroutine completes its execution

Explanation

We created a global variable type WaitGroup from the sync package using var wg sync.WaitGroup.
In the main function we call wg.Add(1) just before we started a goroutine sayGreetings. This indicates that we have launched one goroutine
Here we call wg.Done() using defer keyword to make sure that wg.Done() is called just before we exit the sayGreetings goroutine closure. This indicates we have completed the execution of one goroutine.
Here we call wg.Wait(), which will block further execution of the main goroutine until all the goroutines have completed execution.

Let's see another example on WaitGroups where we launch multiple goroutines.

package main

import (
    "fmt"
    "sync"
)

var wg sync.WaitGroup

func sayGreetings(i int) {

    defer wg.Done()

    fmt.Printf("Hello from goroutine %v!!\n", i)

}

func main() {

    fmt.Println("Hello from the main goroutine !!")

    for i := 0; i < 5; i++ {

        wg.Add(1)
        go sayGreetings(i)

    }

    wg.Wait()

    fmt.Println("Bye!")
}

Output

Hello from the main goroutine !!
Hello from goroutine 4!!
Hello from goroutine 0!!
Hello from goroutine 1!!
Hello from goroutine 2!!
Hello from goroutine 3!!
Bye !!

We have modified the sayGreetings function to accept an integer value, in order to print the message. Also, we have made certain changes in the main function to use a for loop to deploy 5 goroutines to print the message.

It's a good practice to call Add() as closely as possible to the goroutine its tracking. I have called wg.Add(1) just before deploying the goroutine. But you will also find some codes in which wg.Add() is called just before the for loop, as shown in the below example. Both ways are correct. we just need to make sure that we use the correct integer value for the Add().


    const greetingsCount = 5

    wg.Add(greetingsCount)

    for i:=0; i<greetingsCount; i++ {

        go sayGreetings(i)

    }

    wg.Wait()

Conclusion

In this article, we learned about Go's concurrency model, WaitGroups, and the methods provided with it. We saw how we can use waitgroups to create a join point in our code and make sure all the goroutines deployed get executed completely before the main goroutine exits.

Before You Leave

If you found this article valuable, you can support us by dropping a like and sharing this article with your friends.

You can sign up for our newsletter to get notified whenever we post awesome content on Golang.

Reference

Check our ioscript.org for more such contents.

Introduction to Concurrency in Golang

Sonu Kumar Saw — Tue, 22 Feb 2022 08:11:38 +0000

This article was originally posted at ioscript.org

In computer science, often we write programs written as a linear path of code, for example, a program to read a JSON file or a program to calculate your bill. This type of program to one task at a time and follow a linear path to perform that task and finish. But sometimes we need to write a program that needs to execute multiple paths, in such conditions we use concurrent programming. For example, A web service has to connect multiple requests, process the data, and then respond to the requests. Since web services are built to handle concurrent requests, we don't have to wait for one request to complete and return the response in order to send another request to the web service.

Concurrency vs Parallelism

Programmers generally get confused by the word Concurrency with Parallelism, both the topics are related but not the same. Concurrency is the composition of independently executing processes, while Parallelism is the simultaneous execution of the programs.

Below diagram shows how Concurrency is different from Parallelism

The flow diagram on the left shows, the same processor is being used to complete two different tasks concurrently. The flow diagram on the right shows two processes being executed parallelly in two different processors.
You can learn more about this from the workshop by Rob Pike.

Concurrency in Go

Concurrency in Go has been built directly into Go's runtime. Also, Go has a built-in library which makes writing and maintaining concurrent go programs easier as compared to the other programming languages.

Goroutines

Concurrency in Go is powered by goroutines. Goroutines aren't OS threads, they are higher levels of abstraction known as coroutines. They are managed by the Go's runtime. Every Go program has at least one goroutine by default, which is created by the main function.

To create a goroutine we use the keyword go in front of any function. Here's a simple example of a goroutine.

package main

import "fmt"

func sayGreetings() {
    fmt.Println("Hello World!!")
}

func main() {
    go sayGreetings()
}

So, let's discuss what is happening in the code shared above. We have created a function names sayGreetings, which will be printing a message "Hello World". We have another function, main, which is calling the sayGreetings function as a goroutine. When we run this program, It will create two goroutines. The first one would be the main goroutine, and another one would be the goroutine created by sayGreetings function.

This program will compile successfully, however, there is a problem with this example: it's undetermined whether Hello World!! will be printed on the console or not. The goroutine for sayGreetings will be created, but, we can't determine if it will be executed by the time main goroutine executes completed and exits.

So, In order to solve above issue, we have to make sure that sayGreetings goroutine get executed before main goroutine completes its execution. We can make use of time package and put our main goroutine on sleep for 2 secs.

package main

import "fmt"

func sayGreetings() {
    fmt.Println("Hello World!!")
}

func main() {
    go sayGreetings()
    time.Sleep(2 * time.Second)
}

If we run above program, we will get the expected output i.e, Hello World!! in the console.

Even though we were able to print Hello World!! in the console, above code is not correct. By adding time.Sleep() we are just increasing the probability of main goroutine not completing its execution before sayGreetings. If we add another goroutine in our program which takes more than 2 seconds to complete its execution, then again we will get into same issue as before. To make sure that our code works deterministically we need to use Go's synchronization primitives such as Mutex locks, waitgroups, channels, etc.

We will explore more about Go's synchronization primitives in our next part of this series.

Before You Leave

If you found this article valuable, you can support us by dropping a like and sharing this article with your friends.

You can sign up for our newsletter to get notified whenever we post awesome contents on Golang.

Check our ioscript.org for more such contents.