DEV Community: Mateus Canello Ottoni

Multithreaded Programming

Mateus Canello Ottoni — Wed, 30 Jan 2019 22:48:20 +0000

Introduction

Many software developers consider multithreaded programming as an advanced (and scary) topic. Because understanding this matter is not essential to employ multithreading techniques throughout software development, some programmers would rather skip the low-level complexities and go straight to the usage.

However, take some time to comprehend a few concepts can help developers to better picture the aspects of this kind of programming hence improving the ability of writing and reading this sort of code.

In this article, I intend to present the reader with the ideas related to this topic without diving into any language or platform specificities but focusing on the general behavior.

It’s not an essay for pros. I’ll deliberately simplify a few things here and there for the sake of the big picture.

Thread

In computer science, a thread is no more than a stream of sequential instructions that a CPU will eventually process.

The operating system is usually responsible for managing the threads. Nevertheless, sometimes the platform where the application runs (JVM, for instance) implements its own threading system, which can or cannot rely on the hosting OS.

A multithreaded program or application are those that run more than one thread in its process. Which, in the real world, are, basically, all out there.

Applications that features a graphical interface often reserve a specific thread just to handle the user interface elements. This thread is frequently referred to as the main one. The goal is to avoid blocking or freezing the user interface while running the application inner logic. This way, the only responsibility of the UI thread is to handle UI interactions and updates, so that all the other threads take care of the dirty work.

By the other hand, too many threads within a single process might indicate a bottleneck somewhere – not necessarily related to the code, though: it may be a slow disk for an intense I/O system, for instance; or a high latency when performing web requests.

It’s also important to notice that the process will only truly terminate when all of its threads have finished. So, even when closing all the windows or screens (hence finishing the main thread), the process may still continue to run as long as at least one of its threads is still alive. Keep that in mind if that’s not the expected behavior.

Now, how many threads are too much? How many threads should be enough? Those are tough questions to answer. Theoretically, the maximum number of threads should be the number of available CPUs in the device in which the application is running, in order to avoid context switch (don’t worry, we’ll get there). Surely, if the application’s built upon a framework, the number of threads that the framework itself requires should also be taken into account.

A utopic assumption, as you may conclude as well. Open your task manager and check out how many threads are running right now on your computer. I guess it will be much more than the number of CPUs it has.

Therefore, the answer for the “right” number of threads is neither a fixed number nor even a mathematical formula; the application must have the number of threads that it needs.

Context Switch

Now, let’s dive on how the computer works (and, by “computer”, I mean “all kind of electronic devices that uses CPU”; just to be thorough).

The CPUs were designed to execute a sequence of instructions. Nevertheless, a single CPU can only process a single stream of instructions (thread) at a time. So, how to make a quadcore computer runs more than just four threads?

The OS along with the CPUs quickly iterates through all the threads, across all the processes, in order to make it seems that all the applications are running simultaneously. The OS distributes the threads among all the available CPUs and then each CPU will run all the instructions it’s able to in a fraction of seconds for each given thread. Switching between these threads means a pause in the execution of the current thread and a resume of the execution of the next thread.

“Context switch” is the name of the action of when a CPU changes from one thread to another.

This process of iterating through the threads has, obviously, a cost. CPU usage takes time. Some instructions and operations can run rapidly, others are slower. It also means electricity/battery consumption. That’s the reason in which an application with intensive processing can drain the battery of devices.

Thread Pool

You should be careful when creating a brand-new OS thread. It requires memory allocation and CPU instructions in order to set it up and also kill it down.

So, in order to better handle the usage of a thread and also avoid the creation of new ones, the operating systems or platforms reckon with a Thread Pool feature, which allows the application to take an already existing thread to use.

That’s a much more efficient way to handle multiple threads without dealing with its creation or destruction. Furthermore, the OSs know when a thread from the thread pool is not actively in use thus, they can automatically “skip” it during the threads’ iteration.

Unlike the regular thread, a thread-pool thread doesn't remain after the main thread is done, making it specifically useful for background operations that shall terminate along with the program.

Blocking Threads

You might have been told or read somewhere that you should never block a thread. People are willing to scream it aloud but seldom explain why.

When blocking a thread, you are forcing the CPU to attempt to run this thread even though it’s only waiting for something. In doing so, the code will slow down the performance not only of the application but also of the entire system. In short, it’s a waste of CPU time.

Although sometimes it seems easier to compel the thread to wait for something before moving on, it’s not recommended. This blocked thread forces the system to do nothing for a while when it could be working on something else.

Synchronous operations are, indeed, more straightforward to understanding and reading, but turning all the asynchronous operations into synchronous ones for the sake of maintainability is more than wrong. I know it might be painful to deal with asynchronous operations the right way in some languages or platforms, but it cannot turn into an excuse to not cope with.

Part of becoming a developer is to learn how to deal with both the nice and the awful portion of the technology with which you are working.

Asynchronous

Asynchronous methods or functions are procedures which depend on the response of something external. Querying a web service, reading a file, waiting for a user input interaction, all of these do not depend on the thread, do not depend on the CPU.

The whole point of implementing and utilizing asynchronous procedures in the right way is to let the CPU work on other threads, other processes, instead of wasting time doing nothing.

There are two kinds of asynchronous methods/functions: CPU-bound and I/O-bound.

An I/O-bound asynchronous code is a piece of code in which the CPU does not directly do anything. Drives that communicate with some specific hardware are responsible for running that code. It means that network request/responses and disk read/write operations, for instance, are naturally I/O-bound asynchronous actions as they rely on the hardware to work.

The reason most of I/O operations do not require a thread to wait for data is that the underneath protocol (low level) uses a queue structure that conveniently suits quite well into the async/await approach.

By the other hand, a CPU-bound asynchronous code is a code that does run on the CPU. Perform a time-consuming calculation or evaluate a heavy regular expression are examples of what types of operations can be put aside to work on a separate thread in an asynchronous manner.

It’s important to notice that just fitting in the async/await pattern doesn't necessarily mean that the code you wrote perform asynchronously.

If you write a for-statement and inside each iteration the program waits for an async method, you are just transforming asynchronous code into synchronous. Asynchronous code must be dispatched, so that other (synchronous) code can run. The CPU should only wait for the asynchronous operations to complete when it needs their results.

If you have a facade method that waits on each line and two or more of those lines don’t relate to each other, it’s not asynchronous either. In this case, the method should fire all the tasks that don’t relate to each other and, as they return, proceed to run the remaining tasks that depend on their respective outcome.

I hope that with those two examples you can realize how the asynchronous pattern is supposed to work.

Concurrency and Parallelism

Both concurrency and parallelism relate to the same concept: distribute the work in multiple units, in such a way that it doesn't compromise the final product but minimizing the total execution time.

Concurrent execution is the possibility of two (or more) tasks to run at the same time whereas parallel execution is the ability of those two (or more) tasks to run at the very same time.

Concurrency stands for the possibility. Parallelism is the reality.

Turning concurrency into parallelism requires more than one CPU. You might write concurrent code, but it will only actually run in parallel in the presence of multiple CPUs. A single-core device running a concurrent code can only execute it in a sequential way.

Notwithstanding how amazing the parallel execution may be, there are some resources, sometimes shared among all the threads, sometimes shared among all the processes, that will require some coordination in order to work properly. Writing to a stream (a file, or a console), for instance; in case you don’t want to corrupt the file, nor display randomly merged messages to the output, it’s necessary to coordinate their reading and writing operations.

CPU Caching

The tactic of caching data for the sake of performance has a long existence in computer science. It’s heavily applied in web environments as it also reduces the bandwidth consumption when surfing the Internet. What some people might not be aware of is that the CPUs employ this same technique as well.

The CPU may cache a variable’s value in order to improve the speed of the code execution.

The problem starts when a thread reads a variable’s value and stores it in its internal cache but then another thread, running in a different CPU, sets the value of the same variable: the first thread might keep using the outdated information, due to its cache.

It’s not common, though; and, certainly, doesn't happen to all kinds of data structures.

Furthermore, if you ever face this situation, know that it’s possible to specify to the CPU that it should not cache the value of some specific variable. The “volatile” keyword, when declaring the variable, should do the work. Nonetheless, the usage of this keyword should not be reckless. Analyze each case carefully because, even though the cache would store the obsolete information, it provides better performance for the application, overall.

Thread-Safe

It’s quite common to read the expression “thread-safe” related to asynchronous operations or to multithreading contexts. If different threads can access the same instance of a determined data structure and perform operations on it at the same time, it means it’s thread-safe.

Let’s say: a dictionary. A regular list of key/value pair in which the key must but unique. When working with multithreading, two (or more) threads can evaluate the same if-statement simultaneously, producing the same result. Now imagine that it is the statement that checks the existence of some key in the dictionary and the block of code that this if-statement protects should add a new (unique) key into the dictionary. The first thread that reaches the “add” method will run to completion, but the second one will throw an exception.

It happens because the dictionary structure of the example is not thread-safe. In order to work properly with multiple threads, it’s necessary to defer the entrance of the second thread into that code. In doing so, when the second thread attempts to evaluate the if-statement, it will produce a different result.

There are a few distinct ways to implement concurrency control and eventually you will find out the best technique for each occasion.

A solution for the aforementioned situation is to apply a lock around the checking and the adding statements. Locking a piece of code means that everything inside the lock block will run one thread at a time, and if other threads hit the begin of the block, they will have to wait until the thread that is inside the lock complete its execution.

The problem of applying a lock to certain blocks of statements is that it serializes the concurrent access to that specific part of the code, which is virtually the opposite of what you look for when using multiple threads. Of course, even in asynchronous methods, even in multithreading environments, there will be routines that require sequential or controlled execution. Therefore, it’s important to understand the code and only use a lock when it’s actually necessary. Otherwise, it just undoes part of all the working effort.

Conclusion

Multithreading is an undeniable part of modern software development. It’s supported by programming languages and platforms and goes all the way down to the operating system. Knowing how to work with multiple threads can definitely lead developers to build better applications.

Hence, I hope this article could have cast some light on this subject and helped you leverage your knowledge.

Building Better REST APIs

Mateus Canello Ottoni — Sun, 16 Jul 2017 01:25:44 +0000

This post was originally posted on Medium.

I’ve worked as a software developer for almost five years now, most of this time focused on developing backends for applications or other kinds of clients. I’ve learned a lot throughout this time and I want to share a bit of what I know in this text.

First Things First

One thing to point out before we start is the concept of an API. An API (Application Programming Interface) is, as its name says, a way to allow external parts to interact with some application, programmatically. It’s very helpful when some third-party software needs to handle data from your software, and, of course, you do not want to open your database to them. In cases like this, you simply can create a set of services, or operations, for them to use.

When someone builds an API that is accessible from some network (usually through the internet) we name it as Web API.

One of the most common standards for designing web APIs is the REST architecture. REST defines a set of properties (or rules) that constraints (for good reasons) the web API’s architecture. As long as you follow those properties, you will be fine. My tips will basically drive you through the right path for some of these properties.

The Simplistic (Yet Useful) Approach

You may think of a REST API as an answerer. Someone (a client) asks a question, the API listen to the question, thinks about the answer and then responds to it.

Thus, when designing a REST API, you must keep in mind this very simple flow:

the request comes in;
the API does whatever it has to do, as quickly as possible;
sends the response to the client.

Everything that doesn’t suit this approach, shall not stay inside the API. Therefore, you shall not employ an API for operations such as timed-recurring tasks, server-sent events or caching engine.

Besides that, you should pay attention to the “as quickly as possible” statement. It’s not there by chance. Web API shall not host long-running tasks. Web APIs communication is built on top of HTTP requests and responses, and it’s very likely that all the clients will set up a timeout for those requests.

How much time defines â€˜long’? Well, that’s up to you. But keep in mind that in the web world everything should be fast.

Designing Long-Running Tasks

Ok, I should not run a long-running task on my API, but I do have one to run. So, how can I design that?

In my opinion, the best design for long-running tasks is to receive the request for the long-running operation, store this intent somewhere (usually a database) and then return an answer to the requester with a URL where the client could consult the status (or the progress) of the task.

Somewhere else, there’s another software that periodically consults the storage/database, takes the tasks and run them, updating the entries while they’re being processed.

In a variation of this, the API can activate this external software to process the new request, but, of course, must not wait until it’s done.

Nonetheless, I wouldn’t recommend the API starting an external program because, well, it could fail. Misconfiguration, permission stuff, just keep away from things that may become a problem later.

Semantics Matters

HTTP is perhaps the most feared yet widely popular protocol in which the internet relies on. When it comes to REST APIs, I believe it’s heavy important to take all the advantages that this protocol allows.

HTTP is such a robust and complete protocol that it’d require an article to talk only about it. Well, this one is not about HTTP, so, I will try to simplify things here.

This protocol provides built-in features for – at least – the most frequent situations you may face while programming web applications. Do you need internationalization? HTTP has a header named “Accept-Language” to deal with this. Does the resource require authorization? HTTP has a header named “Authorization” to deal with this. Do you need to deliver compressed data to mobile clients? Wait for that, the HTTP has a header named “Accept-Encoding” to deal with this!

I barely scratched the surface so far. All the examples I mentioned is about headers, but HTTP has a lot of more fun.

With the HTTP verbs, you can accomplish different actions for the same endpoint. Let’s say you want to retrieve the user’s profile picture. You can GET it. Ok, but now the user wants to remove it. Well, you can DELETE it. Oh, what a shame, the user wants to upload a new profile picture! You can POST it. Damn it! The user wants to replace the picture. You can PUT it. That’s it. Literally it. Replace the “it” before each verb for the same URL, and you may deduce, just for looking at it, what does the method executes.

This is the “Semantics” I meant in the title of this section.

Using the built-in HTTP features, it’s simple to create requests and responses that contain meaningful data. If you keep adding proprietary headers or following the rule of “always returns success, even if it’s an error”, the processes of parsing and understanding your requests and your responses are much more expensive. By returning a “custom error object” along with a success status code, one must know its properties to grab the real error. Instead, if you just return, for instance, a 404 response, everyone will quickly notice it’s a not found resource.

Furthermore, remember, API is for programmatically using. Consequently, the much standard stuff you put on your API, easier it will be for other people to consume it.

The status code is another excellent feature of the HTTP. You can reply much information, just by sending the right status code. You could reply 404 for not found (as I mentioned), 400 for a bad request, 204 for success with no content, etc.

I will explain a little bit about my favorite status codes and when I apply each of them.

201 – Created At

In my opinion, this is the correct status code to return when you respond that a long-running task was accepted. Why? Because this response allows you to deliver a response-header named “Location”, and you should fill it with the location of the created resource. I fill this header with the absolute URL so the client can check the progress of the task.

204 – Success with No Content

Well, as the name says, everything worked just fine, but the API has no content to reply to the requester. I mention that because it’s my default response when implementing DELETEs. Well, if you requested for the API to delete something, it’s because you don’t want it anymore, right?

400 – Bad Request

What the hell is a bad request?! It’s a common question. Basically, it’s an invalid request. There are some data that is incorrect or missing, or the client requests for an action that cannot be done, etc. Really, anything that the API won’t be able to perform.

402 – Payment Required

Yes, you read it right. There’s a status code for that. I don’t really know if I use it correctly, but I use to return this whenever the authenticated user of the request lacks payment, subscription or something like this.

404 – Not Found

This one is easy, right? Well, but I usually grant another meaning for this status code: not found to you. When a user attempts to select an entry by id, I tend to respond with 404 when the entry exists but does not belong to the requesting user. So, the requester may never truly know if the entry exists or not. Of course, it’s not the right way to deal with this situation. By the book, you ought to return a 403 (forbidden), however, I particularly don’t feel comfortable using this approach.

401 – Unauthorized

You should return this status code whenever the endpoint requires authentication, but the client hasn’t sent it. Besides, you must fill a response-header named “WWW-Authenticate” with the expected authentication scheme. If the returned scheme stands for Basic Authentication, then the most popular browsers will automatically pop up a login dialog for the user inputs its username and password.

A Side Note About Basic Authentication

The Basic Authentication concatenates the username and password and submits it to the server behind a base64 string. It means that, once the request encounters the server, the server can decode the base64 string, retrieving the original username and password as plain-text. It also means that anyone between your client/requester and your server/API can decode that too. So please, please, just use Basic Authentication with SSL. Well, whenever you need to handle sensitive user data (e.g. geolocation, logins, registration forms, payment forms), you ought to communicate with the server through SSL. Seriously, it’s not that expensive.

Provide Significative URLs

Another interesting feature someone might deliver along with the API is significative URLs. Significative for those ones who don’t know your application. For example, when building the API, you could group the operations under the entity they are related to; so, everything related to customers will be under the /Customers URL. It’s helpful when tracking bugs between the client and the API.

Consistent Behavior

Some people think that Consistent Behavior means always return equal answer objects independently whether the request resulted in success or failure. No. Consistent Behavior stands for predictable response situations. Again, APIs are built to be consumed programmatically, hence when someone implements the client-side for that, it’s very important that all the possible responses are clear and well-documented. If you got a 400, read it that way; when receiving a 200, read it this way; etc.

I guess this is the substantial difference between SOAP and REST. In the SOAP architecture, you know right away the response structure, just by looking to its WSDL. In the REST architecture, you don’t have this ability, but it allows a much more flexible way to deal with the data. Although even without having a contract like a WSDL, I believe it’s important to keep some level of foreseeability – those are required information; these might be ignored; etc. It could benefit the development of both of sides – client and server.

Every Request is a Different Request

Unlike many other kinds of web applications, REST APIs are designed to be session-less. It means that each request must contain every necessary information for identification of the client, or user (or both) in addition to the action’s information itself.

Sometimes data length or data security are concerns, so one alternative is to establish a token-based identification. You log in the user using an API service that provides back a unique token to identify the user in the next requests. So, once logged in, the client sends this token in all the subsequent API calls instead of, for instance, always send the username and password.

A Side Note About Unauthorized Password Storing

When you create an API that provides user login you shall be aware of, if the client wants, it can store the password that the user has typed because the user typed it inside its software and not directly in the API.

So, in case you don’t intend to trust in the third-party app that’s consuming your API, you shall not write a login service in your API. The alternative is to bring the user to your site, so the user types his or her credentials inside your web application, which you have full control, and then calls back to the third-party informing the token for the user. This is how Facebook and Twitter accomplish their user authentication even though they have APIs for developers to use.

Another alternative consists in simply do not have your own user authentication. Once you don’t store any password for your users, you don’t need to worry about securing them.

Performance

As I mentioned before, APIs should respond to requests as fast as they can. You should understand that it’s not just the time inside your API that counts. You must to sum the network delay and consider that the client application also run some code when receiving the response from the API.

So, there are a few techniques you may employ to improve your performance, but I will focus on two of them: caching and multi-thread managing.

Caching

If you shall not keep a caching service inside your API because it doesn’t suit the API workflow, also there’s no need in going to the database every time to grab a list of available countries or currencies since they almost never change. Caching this data is important because keeping your code away from the database is a huge performance improvement you may have.

So, I will point out three ways of caching those data even when you are implementing an API. You must pick the right one for each specific situation.

Caching Service

A caching service is basically a software that only manages the cache. There are a few options available (some free, some paid) for you to take, or you can implement one yourself. I widely recommend you pick some already created alternative because they probably have thought in some situations you might forget.

When using a caching service, instead of going to the database, you will consult this service to retrieve the data you need. So, as you may observe, it’s the best approach to the situations when it’s necessary that all the requests run through your code.

Web Server Caching

Another option is to configure your web server to cache the responses for determined endpoints. This can prevent that all the requests touch your code. So, it’s important to you to notice that caching the responses through the web server might not run your code.

Client-Side Caching

The last option I present to you is the client-side cache. Utilizing the response-headers that relate to caching, you may set up the response information that your client can understand and, thus, avoid calling the API again. In this approach, the client won’t even perform a real request to the web server, so it’s definitely not running your code because it’s never even getting to the server in the first place.

Multi-thread Management

Multithread is such a complex subject it’d require an entire article just to talk only about this. So, as I did to the HTTP protocol, I will simplify things here.

REST APIs are naturally multi-thread, once they can execute multiple requests at the same time. Therefore, every time you put a thread to wait for something synchronously you are wasting CPU time because that thread could be being used to handle another request.

Many developers utilize asynchronous methods without really understanding what it does under the hood. Basically, every action that doesn’t run on the CPU could be performed asynchronously. What does run code besides CPU on a computer? Drivers: from disk read/write operations to keyboard inputs.

When you send a web request through the internet it requires CPU (because TCP utilizes CPU), but the biggest part of the process is done by your network card driver. Therefore, this could be accomplished asynchronously. When running an asynchronous code, the operating system knows that that thread is waiting for something and then it could use this thread to run some code that needs to run on the CPU, avoiding the creation of a new thread, thus avoiding wasting memory and time.

This is how utilizing asynchronous methods may help to improve scaling your API.

Fortunately, many of the most common server-side programming languages help you writing asynchronous methods by utilizing the async and await keywords. This way, it’s much easier to write some code that executes asynchronously and still looks like a synchronous one.

Conclusion

Well, those are some important stuff that I consider every web API developer should know or keep in mind when building REST APIs.

I hope this article helped you some way to become a better developer.

Thank you for reading.