Java 21: The Magic Behind Virtual Threads

#java #multitheading #virtualthread #javathread

To understand virtual threads very well, we need to know how Java threads work.

Quick Introduction to Java Threads (Platform Threads) and How They Work

First, let's review the relationship between the threads we've been creating (Java threads) and the OS threads. Whenever we create an object of type Thread, that object, among other things, contains the code that needs to execute and the start method. When we run that start method, we ask the OS to create and start a new OS thread belonging to our application's process, and ask the JVM to allocate a fixed-size stack space to store the thread's local variables from that point on. The OS is fully responsible for scheduling and running the thread on the CPU, just like any other thread.

So, in a sense, that Thread object inside the JVM is just a thin layer or wrapper around an OS thread.

From now on, we're going to call this type of Java thread a platform thread. As we've already seen, those platform threads are expensive and heavy, because each platform thread maps 1-to-1 to an OS thread, which is a limited resource, and it is also tied to a static stack space within the JVM.

Introduction to Virtual Threads

Virtual threads are a relatively newer type of thread that has been introduced as part of JDK 19.

Like platform threads, virtual threads contain, among other things, the code we want to execute concurrently, and the start method. However, unlike a platform thread, a virtual thread fully belongs and is managed by the JVM and does not come with a fixed-size stack.

The OS takes no role in creating or managing it and is not even aware of it. In fact, a virtual thread is just like any Java object allocated on the heap and can be reclaimed by the JVM's garbage collection when it is no longer needed. The consequence of those facts is that unlike platform threads, which are very expensive to create and heavy to manage, virtual threads are very cheap and fast to create in large quantities.

Now, a good question you may ask at this point is: if virtual threads are just Java objects, how do they actually run on the CPU?

The answer is, as soon as we create at least one virtual thread, under the hood, the JVM creates a relatively small internal pool of platform threads. Whenever the JVM wants to run a particular virtual thread, for example, thread A, it mounts it on one of the platform threads within its pool.

When a virtual thread is mounted on a platform thread, that platform thread is called a carrier thread. If the virtual thread finishes its execution, the JVM will unmount that thread from its carrier and make that platform thread available for other virtual threads. That virtual thread object now becomes garbage, which the garbage collection can clean up at any time. However, in certain situations, if thread A has not finished but is unable to make any progress at that time, the JVM will unmount it but save its current state on the heap.

It's worth pointing out that we as developers have very little control over the carrier threads and the scheduling of the virtual threads on them. It is something that the JVM manages for us under the hood.

Quick Demo

For demonstration purposes, we will create a Java thread to see the difference between a virtual thread and a platform thread.

Platform Thread (Java Thread)

Inside your IDE, create a class with a main method, like this:

public class ThreadDemo {
    public static void main(String[] args) {
        Thread thread = new Thread(() -> {
            System.out.println("Thread: " + Thread.currentThread());
        });
        thread.start();
    }
}

So, let's create a virtual thread:

public class VirtualThreadDemo {
    public static void main(String[] args) {
        Thread.startVirtualThread(() -> {
            System.out.println("Virtual Thread: " + Thread.currentThread());
        });
    }
}

As you can see when we run the program, the first thing we notice is the object we printed is of type VirtualThread, then we see that its ID is 24, and the name is "ForkJoinPool.commonPool-worker-1". This tells us a few things. First, it tells us that to schedule this and any future virtual threads, the JVM created an internal thread pool of platform threads, which is called ForkJoinPool.commonPool, and then the JVM mounted our virtual thread on one of those worker threads, which is called worker-1.

To make this easier to understand, let's create another virtual thread:

public class VirtualThreadDemo {
    public static void main(String[] args) {
        Thread.startVirtualThread(() -> {
            System.out.println("Virtual Thread 1: " + Thread.currentThread());
        });
        Thread.startVirtualThread(() -> {
            System.out.println("Virtual Thread 2: " + Thread.currentThread());
        });
    }
}

As you can see, we have two virtual threads, their IDs are 24 and 25, respectively. They ran on the same pool of carrier threads, which is called ForkJoinPool.commonPool, but because we ran them concurrently, each one was mounted on a different worker thread to be its carrier. The first one was mounted on worker-1, and the second on worker-2.

Now, to see the relationship between the number of virtual threads and the number of platform threads within that ForkJoinPool, let's increase the number of virtual threads from 2 to 20:

public class VirtualThreadDemo {
    public static void main(String[] args) {
        for (int i = 0; i < 20; i++) {
            Thread.startVirtualThread(() -> {
                System.out.println("Virtual Thread " + (i + 1) + ": " + Thread.currentThread());
            });
        }
    }
}

As you can see on the screen, we indeed created 20 new virtual threads, each with its own unique ID. However, based on their names, we can see that the JVM dynamically decided to create a pool of seven platform threads to be their carriers, and all those virtual threads were scheduled to run on this small pool of threads.