A Tutorial on Modern Multithreading and Concurrency in C++

Written by Ryan Thelin, originally posted on Educative.io

In the modern tech climate, concurrency has become an essential skill for all C++ programmers. As programs continue to get more complex, computers are designed with more CPU cores to match. To efficiently design these programs, developers must write code that utilizes their multicore machines. This is accomplished through concurrency, a coding technique that ensures the use of all cores and maximizes a machine’s capabilities.

To get you familiar with concurrent programming and multithreading, I will walk you through all the definitions and real-world examples you need to know.

Here's what we'll cover today:

• What is concurrency?
• Methods of implementing concurrency
• Examples of multithreading
• C++ concurrency in action: real-world applications
• Resources

What is concurrency?

Concurrency occurs when multiple copies of a program run simultaneously while communicating with each other. Simply put, concurrency is when two tasks are overlapped. A simple concurrent application will use a single machine to store the program’s instruction, but that process is executed by multiple, different threads. This creates a kind of control flow, where each thread executes its instruction before passing to the next one.

This allows the threads to act independently and to make decisions based on the previous thread as well. Some issues can arise in concurrency that make it tricky to implement.

For example, a data race is a common issue you may encounter in C++ concurrency and multi-threaded processes. Data races in C++ occur when at least two threads can simultaneously access a variable or memory location, and at least one of those threads tries to access that variable. This can result in undefined behavior. Regardless of its challenges, concurrency is very important for handling multiple tasks at once.

History of C++ concurrency

C++11 was the first C++ standard to introduce concurrency, including threads, the C++ memory model, conditional variables, mutex, and more. The C++11 standard changes drastically with C++17. The addition of parallel algorithms in the Standard Template Library (STL) greatly improved concurrent code.

Concurrency vs. parallelism

Concurrency and parallelism often get mixed up, but it’s important to understand the difference. In parallelism, we run multiple copies of the same program simultaneously, but they are executed on different data. For example, you could use parallelism to send requests to different websites but give each copy of the program a different set of URLs.

These copies are not necessarily in communication with each other, but they are running at the same time in parallel. As we explained above, concurrent programming involves a shared memory location, and the different threads actually “read” the information provided by the previous threads.

Methods of Implementing Concurrency

In C++, the two most common ways of implementing concurrency are through multithreading and parallelism. While these can be used in other programming languages, C++ stands out for its concurrent capabilities with lower than average overhead costs as well as its capacity for complex instruction.
Below, we’ll explore concurrent programming and multithreading in C++ programming.

C++ Multithreading

C++ multithreading involves creating and using thread objects, seen as std::thread in code, to carry out delegated sub-tasks independently. Upon creation, threads are passed a function to complete, and optionally some parameters for that function.

Image from Medium, [C++] Concurrency by Valentina

While each individual thread can complete only one function at a time, thread pools allow us to recycle and reuse thread objects to give programs the illusion of unlimited multitasking. Not only does this take advantage of multiple CPU cores, but it also allows the developer to control the number of tasks taken on by manipulating the thread pool size. This ensures that the program uses the computer resources efficiently without overloading the system.

To better understand thread pools, consider the relationship of worker bees to a hive queen; the queen (the program) has a broader goal to accomplish (the survival of the hive) while the workers (the threads) only have their individual tasks given by the queen.

Once these tasks are completed, the bees return to the queen for further instruction. At any one time, there is a set number of these workers being commanded by the queen, enough to utilize all of its hive space without overcrowding it.

Parallelism

Creating different threads is typically expensive in terms of both time and memory overhead for the program; a cost which, when dealing with short duration, simpler functions, can sometimes not be worth it. For times like these, developers can instead use parallel execution policy annotations, a way of marking certain functions as candidates for concurrency without creating threads explicitly.

At its most basic, there are two marks that can be encoded into a function. The first is parallel, which suggests to the compiler that the function be completed concurrently with other parallel functions (however the compiler may overrule this suggestion if resources are limited). The other is sequential, meaning that the function must be completed individually.
Parallel functions can significantly speed up operations because they automatically use more of the computer’s CPU resources.

However, it is best saved for functions that have little interaction with other functions; ones which are neither dependent on other functions’ outcomes nor attempt to edit the same data. This is because while they are worked on concurrently, there is no way to know which will complete first, meaning the result is unpredictable unless synchronization such as mutex or condition variables are used.

Imagine we have two variables, A and B, and create functions addA and addB, which add 2 to their value. We could do so with parallelism, as the behavior of addA is independent of the behavior of the other parallel function addB, and therefore has no problem being completed concurrently.

However, if the functions both impacted the same variable, we would instead want to use sequential execution. Imagine that we instead had one which multiplied variable A by two, DoubleA, and another which added B to A, addBA. In this case, we would not want to use parallel execution as the outcome of this set of functions depends on which is completed first and would, therefore, result in a race condition.

While both multithreading and parallelism are helpful concepts for implementing concurrency in a C++ program, multithreading is more widely applicable due to its ability to handle complex operations. In the next section, we’ll look at a code example of multithreading at its most basic.

Multithreading Examples

In the following examples, we’ll look at some simple multithreaded programs designed to use a print function which we declare at the beginning.

Simple One-Thread example

Since all threads must be given a function to complete at their creation, we first must declare a function for it to be given. We’ll name this function print, and will design it to take int and string arguments when called. When executed, this code will simply report the data values passed in.

void print(int n, const std::string &str)  {  
    std::cout << "Printing integer: " << n << std::endl;  
    std::cout << "Printing string: " << str << std::endl;  
} 

In the next section, we’ll initialize a thread and have it execute the above function. To do this, we’ll have the main function, the default executor present in all C++ applications, initialize the thread for the print function.

After that, we use another handy multithreading command, join(), pausing the main function’s thread until the specified thread, in this case t1, has finished its task. Without join() here, the main thread would finish its task before t1 would complete print, resulting in an error.

int main() {
    std::thread t1(print, 10, "Educative.blog");
    t1.join();
return 0;
}

Multi-Thread Example

While the outcome of the single thread example above could easily be replicated without using multithreaded code, we can truly see concurrency’s benefits when we attempt to complete print multiple times with different sets of data. Without multithreading, this would be done by simply having the main thread repeat print one at a time until completion.

To do this with concurrency in mind, we instead use a for loop to initialize multiple threads, pass them the print function and arguments, which they then complete concurrently. This multithreading option would be faster one using only the main thread as more of the total CPU is being used.

Runtime difference between multithreading and non-multithreading solutions increasing as more print executions are needed.

Let’s see what a many-thread version of the above code would look like:

C++ concurrency in action: real-world applications

Multithreading programs are common in modern business systems, in fact, you likely use some more complex versions of the above programs in your everyday life.

One example could be an email server, returning mailbox contents when requested by a user. With this program, we have no way of knowing how many people will be requesting their mail at any given time. By using a thread pool, the program can process as many user requests as possible without risking an overload.

As above, each thread would execute a defined function, such as receiving the mailbox of the identifier passed in, void request_mail (string user_name).

Another example could be a web crawler, which downloads pages across the web. By using multithreading, the developer would ensure that the web crawler is using as much of the hardware’s capability as possible to download and index multiple pages at once.

Based on just these two examples, we can see the breadth of functions in which concurrency can be advantageous. With the number of CPU cores in each computer increasing by the year, concurrency is certain to remain an invaluable asset in the arsenal of the modern developer.

## Steps Forward and Resources

In this article, we merely scratched the surface of what’s possible with multithreading and concurrency in C++. As you continue to expand your learning, here’s some additional resources to guide you toward concurrency mastery!

Were some of the terms in this article unfamiliar? Want to refresh your knowledge of concurrency fundamentals? If so, check out our article summarizing everything you need to know to start your concurrency journey.

• Multithreading and Concurrency Fundamentals: Ready to begin tackling advanced C++ concurrency functions and applications? Explore Rainer Grimm’s course, full of insider tips, case studies, extensive sample code and more!

• Multithreading and Concurrency: Interested in multithreading and concurrency in other programming languages? Check out diverse courses in Java, Python, C++, and more.

• Top 5 Concurrency Interview Questions for Software Engineers breaks down the 5 most frequently asked concurrency and multithreading questions asked in an interview. Problems include: The ReaderWriter Problem, The Dining Philosopher, and more.

• Modern C++ Concurrency in Practice: Interested in concurrency but want to invest in a coding language other than C++? Jump into our full selection of concurrency courses, covering multiple languages and skill levels.
  Multithreading and Concurrency

• C++ Concurrency in Action: Practical Multithreading by Anthony Williams: an excellent guide to the new C++ standard. This book is for all C++ programmers looking to update their concurrency and multithreading skills.