DEV Community

Ryan Zhi
Ryan Zhi

Posted on

Learning Virtual Threads in Java

What are Virtual Threads?

Virtual threads, introduced in Java 19 and officially released in Java 21 (September 2023), are lightweight threads that are managed by the JVM, not the operating system. They are similar to goroutines in Go.

Why Were Virtual Threads Introduced?

The main issue that virtual threads solve is improving CPU utilization for I/O-bound tasks. Traditional multi-threading in Java can lead to inefficient CPU usage when a thread is waiting for I/O operations, such as network communication or file reading. For example, if a thread is waiting for data from the network or a disk, it blocks and doesn't do anything else, causing idle CPU time.

Virtual threads address this by allowing threads to "yield" and do other work when blocked on I/O, releasing the CPU and allowing other tasks to execute. Once the I/O operation is complete, the virtual thread resumes. This greatly increases CPU utilization because the thread doesn't sit idle while waiting for I/O operations to complete.

However, virtual threads don't increase the actual number of available CPU threads. Instead, they improve the efficiency of the threads by allowing them to be used more effectively. For CPU-intensive tasks like mathematical calculations, virtual threads behave similarly to traditional threads. But if your workload involves significant I/O blocking, virtual threads can provide a notable performance boost.

How Do Virtual Threads Work?

Virtual threads are mapped to platform threads, which can be understood as platform threads being like electrical sockets that only allow one device to be plugged in. Virtual threads are like power strips, allowing multiple threads to share the same platform thread without blocking it when inactive. This metaphor, while not perfectly accurate, illustrates the idea of multiple virtual threads being managed on a single platform thread.

Key Concepts:

  • Scheduling Queue: Virtual threads are managed and scheduled by the JVM using a scheduling queue, which tracks all the suspended virtual threads. The queue ensures that threads are resumed in the order they were suspended.

  • Scheduler: The scheduler manages platform threads and assigns virtual threads to them when they are available. The default scheduler in Java uses a FIFO-based ForkJoinPool, which uses a "work-stealing" algorithm to maximize throughput.

Execution Flow:

  1. Creation: A virtual thread is created and placed in the scheduling queue.
  2. Mounting: The JVM's scheduler selects a platform thread and "mounts" a virtual thread onto it, allowing it to run.
  3. Execution: The virtual thread runs its task until it encounters a blocking operation.
  4. Unmounting: If the thread becomes blocked, it is "unmounted" from the platform thread, which is then free to execute other tasks.
  5. Suspension: The blocked thread is suspended by the JVM and placed back into the scheduling queue.
  6. Resumption: Once the blocking condition is cleared (e.g., I/O completes), the thread is resumed and mounted back onto a platform thread.
  7. Completion: The virtual thread continues execution until it completes, at which point it is garbage collected when there are no more references to it.

Continuations: The Core of Virtual Threads

The concept of continuations is crucial for virtual threads. A continuation is a data structure that holds the state of a task and allows it to be paused and resumed. This enables the JVM to suspend and resume tasks on virtual threads.

  • When a virtual thread needs to block (e.g., during I/O operations), the JVM uses a continuation to suspend the thread.
  • When the I/O completes, the JVM uses the continuation to resume the thread from where it was suspended.

The platform thread that is executing a virtual thread is also called the carrier thread. The JVM schedules virtual threads onto carrier threads in a way that allows for efficient resource usage. When a virtual thread is mounted onto a carrier thread, the thread's stack data (in the form of continuation frames) is copied into the carrier thread's stack.

Mounting and Unmounting Virtual Threads

  • Mounting: The process of assigning a virtual thread to a carrier platform thread. This involves copying the virtual thread's continuation stack frames to the carrier thread's stack, effectively transferring the virtual thread's execution to the platform thread.
  • Unmounting: When a virtual thread becomes blocked, it is "unmounted" from the platform thread, and its continuation data is often left on the heap (in memory) to be resumed later.

Key Takeaways:

  • Virtual threads provide lightweight task management within the JVM, enabling applications to handle large numbers of concurrent I/O-bound tasks efficiently.
  • They allow a single platform thread to carry out the work of many virtual threads, reducing the overhead typically associated with creating and managing threads.
  • Virtual threads are ideal for I/O-bound workloads (e.g., network requests, file I/O) but may not be as beneficial for CPU-bound tasks due to the limited number of platform threads available.
  • Continuations handle the suspension and resumption of tasks, providing a way for the JVM to manage the execution of virtual threads across platform threads.

Use Cases:

  • I/O-Intensive Tasks: Virtual threads are perfect for handling a large number of I/O operations, such as handling HTTP requests or processing file operations.
  • Concurrency in High-Load Applications: Virtual threads can help manage thousands or even millions of concurrent tasks without the memory and CPU overhead of traditional threads.
  • Async Programming: Virtual threads can simplify asynchronous programming by making it look and behave more like traditional synchronous code, improving readability and reducing complexity.

Summary

Virtual threads represent a powerful tool in Java for optimizing resource usage in highly concurrent, I/O-heavy applications. By efficiently handling blocking I/O operations and allowing threads to be more lightweight, they make it easier to scale applications while avoiding the overhead of managing a massive number of platform threads. However, virtual threads are not a silver bullet for every kind of task—particularly for CPU-bound tasks, where traditional threading models still hold an advantage.

Speedy emails, satisfied customers

Postmark Image

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Immerse yourself in a wealth of knowledge with this piece, supported by the inclusive DEV Community—every developer, no matter where they are in their journey, is invited to contribute to our collective wisdom.

A simple “thank you” goes a long way—express your gratitude below in the comments!

Gathering insights enriches our journey on DEV and fortifies our community ties. Did you find this article valuable? Taking a moment to thank the author can have a significant impact.

Okay