DEV Community: ThinkerQAQ

Concurrency Programming (1.5): The Operating System Layer — Threads, Schedulers, and Runtime Scheduling

ThinkerQAQ — Sun, 11 Oct 2026 13:24:01 +0000

0. From Language-Level Execution Units to the CPU
1. How Does Java Schedule Threads?
- 1.1 From Platform Threads to Virtual Threads
- 1.1.1 The Simplest Model: One Java Thread per OS Thread
- 1.1.2 The Problem: Resource and Scheduling Costs of Many OS Threads
- 1.1.3 The Solution: Let the JVM Schedule Lightweight Threads
- 1.1.4 The Result: Platform Threads and Virtual Threads
- 1.2 Execution Layers
- 1.3 From Java Thread Lifecycles to Their Implementation
- 1.3.1 Creation and Start: Created → Runnable
- 1.3.2 Waiting to Be Scheduled: Runnable
- 1.3.3 Running: Runnable → Running
- 1.3.4 Parking and Waiting: Running → Waiting
- 1.3.5 Wakeup and Resumption: Waiting → Runnable → Running
- 1.3.6 Completion and Cleanup: Running → Terminated
- 1.4 End-to-End Sequence Diagrams
- 1.5 Platform Threads vs. Virtual Threads
- 1.6 Returning to the Opening Question
2. How Does Go Schedule Goroutines?
- 2.1 From G-M to G-M-P
- 2.1.1 The Simplest Model: A Goroutine Runs on an OS Thread
- 2.1.2 The Problem: Many Goroutines, Fewer OS Threads
- 2.1.3 The Solution: Evolving from G-M to G-M-P
- 2.1.4 The Final Model: G, M, and P
- 2.2 Execution Layers
- 2.3 From a Goroutine's Lifecycle to Its Runtime Implementation
- 2.3.1 Creation: Created → Runnable
- 2.3.2 Waiting for Scheduling: Runnable
- 2.3.3 Execution: Runnable → Running
- 2.3.4 Parking and Waiting: Running → Waiting
- 2.3.5 Wakeup and Resumption: Waiting → Runnable → Running
- 2.3.6 System Calls and Preemption
- 2.3.7 Completion and Cleanup: Running → Dead
- 2.4 End-to-End Sequence Diagram
- 2.5 Returning to the Opening Question
3. How Does CPython Schedule Threads?
- 3.1 From Native OS Threads to the GIL and Free-Threaded Builds
- 3.1.1 The Simplest Model: One Python Thread per OS Thread
- 3.1.2 The Problem: Sharing Interpreter State Across OS Threads
- 3.1.3 The Default Solution: The GIL Restricts Interpreter Execution
- 3.1.4 The Next Step: Free-Threaded CPython
- 3.1.5 The Final Model: Thread, PyThreadState, and GIL
- 3.2 Execution Layers
- 3.3 From CPython Thread Lifecycles to Their Implementation
- 3.3.1 Creation and Start: Created → Runnable
- 3.3.2 Waiting for Scheduling: Runnable
- 3.3.3 Execution: Runnable → Running
- 3.3.4 Parking and Waiting: Running → Waiting
- 3.3.5 Wakeup and Resumption: Waiting → Runnable → Running
- 3.3.6 Completion and Cleanup: Running → Terminated
- 3.4 End-to-End Sequence Diagram
- 3.5 Returning to the Opening Question
4. What the Three Implementations Have in Common
- 4.1 How Does an Execution Unit Reach the CPU? — Establish Its Carrier
- 4.2 Who Selects What Runs Next? — Separate the Scheduling Layers
- 4.3 What Happens After a Wait? — Regain an Execution Opportunity

- 5. Next: Language Memory Models

0. From Language-Level Execution Units to the CPU

The previous article examined how the CPU executes count++ and how atomic instructions, cache coherence, and fences provide the hardware foundations for atomicity, visibility, and ordering.

This article moves one layer up, to the operating system. It asks one central question:

How do Java threads, Go goroutines, and CPython threads pass through their runtimes and the operating system to execute on a CPU?

Answering it requires understanding three things:

Execution mapping: How does a language-level execution unit map to an OS thread?
Scheduling responsibilities: What do the runtime scheduler and the operating system scheduler each control?
Waiting and resumption: When an execution unit waits, wakes, or resumes, what is suspended, and which threads can continue running?

Next, we will examine Java, Go, and CPython in turn. For each, we will trace the evolution of its threading model, then use execution layers, lifecycles, and sequence diagrams to answer these three questions. Finally, we will draw out the language-independent mechanisms they share and return to the opening question.

1. How Does Java Schedule Threads?

1.1 From Platform Threads to Virtual Threads

1.1.1 The Simplest Model: One Java Thread per OS Thread

When a program has only a few concurrent tasks, the most straightforward approach is to map each Java thread to an OS thread. Java creates the thread, and the operating system schedules the corresponding OS thread.

With relatively few threads, this one-to-one mapping is easy to understand and debug.

1.1.2 The Problem: Resource and Scheduling Costs of Many OS Threads

Suppose a service handles many requests, each on its own thread:

Request 1 → Thread 1 → OS Thread 1
Request 2 → Thread 2 → OS Thread 2
Request 3 → Thread 3 → OS Thread 3
...
Request N → Thread N → OS Thread N

Many request-handling tasks spend most of their time waiting:

Read from a database
Wait for a network response
Wait for a message
Wait for other I/O

If Java threads continue to map one-to-one to OS threads, two costs become significant:

Native-thread resources: Even while a task is waiting, the operating system must retain resources such as its kernel thread and native stack. These costs grow with the thread count.
Scheduling overhead: When many threads wake frequently and become Runnable, the operating system must schedule among more runnable threads. Context switches and scheduler work consume additional CPU time.

1.1.3 The Solution: Let the JVM Schedule Lightweight Threads

How can we address these costs?

In this setting, logical tasks spend most of their time waiting for I/O and relatively little time using the CPU. They do not need to occupy a dedicated OS thread for their entire lifetime. The key is to decouple logical execution units from OS threads.

Once they are decoupled, the JVM can introduce its own scheduler:

Logical Thread A ─┐
Logical Thread B ─┼─→ JVM Scheduler ─→ Carrier Thread 1 ⇔ OS Thread 1
Logical Thread C ─┤                  └→ Carrier Thread 2 ⇔ OS Thread 2
Logical Thread D ─┘

A Carrier Thread is a Platform Thread in the carrier role. The ⇔ symbol indicates its one-to-one native mapping to an OS thread, not a method call.

The JVM can manage many logical threads while the operating system schedules a smaller number of underlying OS threads.

Java calls this lightweight logical thread a Virtual Thread.

1.1.4 The Result: Platform Threads and Virtual Threads

A Platform Thread maps one-to-one to an OS Thread; the JVM Scheduler dispatches Virtual Threads onto Platform Threads acting as carriers, and the OS Scheduler schedules the underlying OS Threads.

1.2 Execution Layers

1.3 From Java Thread Lifecycles to Their Implementation

A Java thread's conceptual lifecycle is:

This is a conceptual lifecycle, not an exact mapping to Java's Thread.State. In particular, RUNNABLE covers both threads awaiting CPU time and threads currently running.

1.3.1 Creation and Start: Created → Runnable

A platform thread creates an OS thread at start(); a virtual thread submits its continuation to the JVM scheduler.

(1) Platform Thread

(2) Virtual Thread

1.3.2 Waiting to Be Scheduled: Runnable

A platform thread waits for OS scheduling; a virtual thread waits for JVM dispatch to a carrier thread.

(1) Platform Thread

(2) Virtual Thread

1.3.3 Running: Runnable → Running

A platform thread executes Thread.run() on its OS thread; a virtual thread mounts and runs its continuation on a carrier.

(1) Platform Thread

(2) Virtual Thread

1.3.4 Parking and Waiting: Running → Waiting

A waiting platform thread normally retains its OS thread; when unmounting is possible, a virtual thread releases its carrier.

(1) Platform Thread

(2) Virtual Thread

1.3.5 Wakeup and Resumption: Waiting → Runnable → Running

A platform thread resumes on the same OS thread; a virtual thread reenters the JVM scheduler and may resume on a different carrier.

(1) Platform Thread

(2) Virtual Thread

1.3.6 Completion and Cleanup: Running → Terminated

A platform thread cleans up its JavaThread and OS thread; a virtual thread terminates without terminating its carrier.

(1) Platform Thread

(2) Virtual Thread

1.4 End-to-End Sequence Diagrams

The following diagrams trace the native startup path for a platform thread and the essential mount, unmount, and resume operations for a virtual thread. The virtual-thread sequence assumes its carrier already has CPU time, so it does not repeat OS scheduling.

1.4.1 Platform Thread

1.4.2 Virtual Thread

1.5 Platform Threads vs. Virtual Threads

Their lifecycle differences can be summarized as follows:

Lifecycle phase	Platform Thread	Virtual Thread
Created → Runnable	The JVM creates an OS thread	The JVM submits a continuation to its scheduler
Runnable	Waits for the OS scheduler	The JVM scheduler dispatches it to a carrier; the OS scheduler schedules the carrier's OS thread
Running	The OS thread executes Java code directly	The virtual thread mounts on a Carrier Thread backed by an OS Thread
Waiting	The OS thread generally waits as well	The virtual thread can wait independently and release its carrier if it can unmount
Resume	The OS thread becomes runnable again	The virtual thread is resubmitted to the JVM scheduler
Terminated	JavaThread / OS thread exit	Its continuation completes and `afterDone()` moves it to TERMINATED

The central distinction is the execution relationship: platform threads remain associated with their OS threads, while the JVM dispatches virtual threads to carriers. When a virtual thread can unmount, its carrier becomes available for other tasks.

For implementation details, see OpenJDK's Thread.java, jvm.cpp, javaThread.cpp, and VirtualThread.java. For the virtual-thread design, see JEP 444.

1.6 Returning to the Opening Question

As Section 1.2, Execution Layers shows, a Platform Thread maps one-to-one to an OS Thread, while a Virtual Thread reaches an OS Thread through a Carrier Thread. Therefore, both thread models ultimately execute on OS Threads.

Section 1.3.2, Waiting to Be Scheduled and Section 1.3.3, Running show that the JVM Scheduler dispatches Virtual Threads to Carrier Threads, while the OS Scheduler schedules the underlying OS Threads. Therefore, the scheduling paths differ, but the OS Scheduler ultimately grants CPU execution time to both.

Section 1.3.4, Parking and Waiting and Section 1.3.5, Wakeup and Resumption show that a Virtual Thread can release its Carrier Thread when waiting permits unmounting, then reenter JVM scheduling when ready. Therefore, many waiting tasks need not each occupy an OS Thread for their entire lifetime.

Answering the opening question: Java retains the native-thread model of Platform Threads while allowing Virtual Threads to reuse Carrier Threads, so many concurrent tasks can ultimately execute on the CPU using a relatively small number of OS Threads.

2. How Does Go Schedule Goroutines?

2.1 From G-M to G-M-P

2.1.1 The Simplest Model: A Goroutine Runs on an OS Thread

For a small number of concurrent tasks, the simplest approach is:

Goroutine
    ↓
OS Thread
    ↓
Operating System Scheduler
    ↓
CPU Core

One Goroutine maps to one OS Thread; the Go runtime manages the Goroutine, while the operating system schedules its OS Thread.

At modest thread counts, this one-to-one model is easy to understand and debug.

2.1.2 The Problem: Many Goroutines, Fewer OS Threads

Suppose a service handles many requests, each on its own Goroutine:

Request 1 → Goroutine 1 → OS Thread 1
Request 2 → Goroutine 2 → OS Thread 2
Request 3 → Goroutine 3 → OS Thread 3
...
Request N → Goroutine N → OS Thread N

Many service tasks spend most of their time waiting for databases, network responses, or other I/O.

If Goroutines remained mapped one-to-one to OS Threads, they would face the same two costs as Java Platform Threads:

Native-thread resources: a waiting task would still retain a kernel thread, native stack, and other resources.
Scheduling overhead: many OS Threads repeatedly becoming runnable would increase context switching and OS scheduling work.

2.1.3 The Solution: Evolving from G-M to G-M-P

How can the runtime address this?

These logical tasks spend much of their time waiting for I/O rather than using the CPU. They therefore do not need to retain a dedicated OS Thread for their entire lifetime.

The solution is to decouple the logical execution unit from the OS Thread: the Go runtime schedules Goroutines so a limited number of OS Threads can be reused.

2.1.3.1 Step One: G-M with a Global Run Queue

Start with G, M, and a global run queue:

Here, G is a goroutine and M is an OS thread.

Here, runnable Gs share a global run queue, and multiple Ms fetch work from it.

As concurrency grows, two problems emerge:

Multiple M instances
  ↓
Access the same Global Run Queue
  ↓
More synchronization contention

And:

Scheduling state for G is centralized
        ↓
Poorer locality

To reduce contention on the global queue, some scheduling state and runnable work can be distributed across execution contexts:

What if each execution context had its own scheduling resources and local run queue?

2.1.3.2 Step Two: Introduce P and Local Run Queues

To reduce contention for the global queue, the Go runtime introduces P, with each P maintaining its own local run queue:

P holds the runtime resources needed to execute Go user code and maintains its own local run queue. The actual G-M-P scheduler also retains a global queue; goroutines are not permanently assigned to a particular P. We will examine other work sources next.

This gives us the defining constraint of G-M-P:

An M needs a P to execute Go user code.

Scheduling state is no longer concentrated in one queue contested by all M instances; each P maintains its own local run queue.

2.1.3.3 Third Question: What If a P's Local Queue Is Empty?

Local queues reduce contention but can leave work unevenly distributed: one P is busy while another has nothing queued. An M holding an idle P can use work stealing to obtain runnable goroutines from another P's queue.

The scheduler can also find work from other sources:

In the actual implementation, findRunnable() considers fairness, timers, the network poller, and other conditions when searching for runnable Gs. It does not rigidly follow the same sequence every time.

2.1.3.4 Fourth Question: What If M Blocks in a System Call?

Another constraint comes from blocking system calls:

If an M holding a P blocks in the OS, must the P become idle too?

If P remains permanently attached to that M, other goroutines cannot run using its scheduling resources while the M is blocked.

P and M therefore need to be separable:

G0 remains blocked in the system call on M0's OS thread. Meanwhile the runtime can reclaim P0 so M1 can run another runnable G. When M0 returns from the system call, it must try to acquire a P again before resuming Go user code. P and M must therefore be free to separate and recombine.

2.1.3.5 Fifth Question: What If One G Runs for Too Long?

Even without I/O, a goroutine that runs continuously for a long time can starve other runnable work of execution opportunities.

The runtime therefore also needs preemption:

G Running
    ↓
Preempt
    ↓
G Runnable
    ↓
Scheduler
    ↓
Another G gets CPU time

Modern Go supports preemption, including asynchronous preemption, to reduce the chance that a long-running goroutine blocks progress elsewhere. Preemption does not, by itself, guarantee strict fairness.

2.1.4 The Final Model: G, M, and P

2.1.4.1 G: Goroutine

G represents a goroutine: a logical execution unit managed by the Go runtime.

It holds:

Stack;
PC / scheduler context;
Scheduling state such as Runnable, Running, and Waiting.

go f() creates a G, not a new OS thread.

2.1.4.2 M: Machine

M represents a real OS thread.

M
↓
OS Thread
↓
OS Scheduler
↓
CPU

The OS scheduler schedules the OS thread represented by M onto a CPU.

Each M also has a special g0 goroutine. Its system stack is used by the runtime for scheduling, stack management, and other low-level work.

2.1.4.3 P: Processor

P represents the runtime resources and execution capacity an M must hold to run Go code.

It contains scheduler and allocator state, a local run queue, and runnext for preferential scheduling.

The number of P instances is determined by GOMAXPROCS:

GOMAXPROCS
    ↓
Number of P instances
    ↓
Maximum parallelism for Go user code

Their roles can be summarized in one sentence:

G is the task to run; M is the OS thread; P provides the runtime resources an M needs to execute Go code.

2.2 Execution Layers

The Go runtime scheduler and OS scheduler operate at different layers. Switching goroutines on the same M does not necessarily involve an OS thread context switch.

2.3 From a Goroutine's Lifecycle to Its Runtime Implementation

A goroutine's conceptual lifecycle is:

The diagram uses conceptual states such as Created and Waiting. The Go runtime uses more specific internal states, including _Grunnable and _Gwaiting.

2.3.1 Creation: Created → Runnable

go f() creates a G, marks it runnable, and enqueues it for scheduling.

2.3.2 Waiting for Scheduling: Runnable

The G is in a run queue, but an M holding a P has not yet selected it.

2.3.3 Execution: Runnable → Running

An M holding a P selects a runnable G through the runtime scheduler, then runs it through execute(G) and gogo. The OS scheduler independently schedules M's underlying OS thread.

2.3.4 Parking and Waiting: Running → Waiting

gopark() suspends only G, letting its M continue running other runnable goroutines while holding P.

2.3.5 Wakeup and Resumption: Waiting → Runnable → Running

When the wait ends, goready() requeues G so the scheduler can dispatch it to an available M.

2.3.6 System Calls and Preemption

A system call can block M while P becomes available to another M; preemption makes a long-running G eligible for rescheduling.

2.3.7 Completion and Cleanup: Running → Dead

When G's function returns, the runtime cleans up G, marks it dead, and M schedules other work.

2.4 End-to-End Sequence Diagram

This sequence follows one G through creation, execution, waiting, and termination. OS scheduling and goroutine switching are separate layers: the diagram shows M being scheduled onto a CPU once, but later goroutine switches on that same M need not trigger another OS scheduling event.

For implementation details, see Go's proc.go and runtime2.go.

2.5 Returning to the Opening Question

Section 2.1.4, The Final Model and Section 2.2, Execution Layers establish that G is a logical execution unit, M corresponds to an OS thread, and M needs P to execute Go code. Therefore, a G executes through an M holding a P, not as a dedicated OS thread.

Section 2.1.3.3, Work Sources, Section 2.3.2, Waiting for Scheduling, and Section 2.3.3, Execution show the Go scheduler selecting runnable Gs from queues, work stealing, or netpoll for an M, while the OS scheduler schedules the underlying OS thread. Therefore, the two schedulers decide which G runs and which OS thread receives CPU time, respectively.

Section 2.3.4, Parking and Section 2.3.5, Resumption show that M can run other Gs while one G waits. Section 2.3.6, System Calls and Preemption shows that another M may take P if a system call blocks the original M, and a long-running G may be preempted. Therefore, waiting or blocking need not leave the associated scheduling resources idle indefinitely.

Answering the opening question: Go uses G-M-P to schedule many goroutines on M instances holding P, while the OS scheduler runs their underlying OS threads on CPUs, allowing concurrent tasks to make progress on a limited number of CPU cores.

3. How Does CPython Schedule Threads?

3.1 From Native OS Threads to the GIL and Free-Threaded Builds

3.1.1 The Simplest Model: One Python Thread per OS Thread

For a small number of concurrent tasks, the execution relationship is straightforward:

Python Thread
      ↓
OS Thread
      ↓
Operating System Scheduler
      ↓
CPU Core

The OS scheduler decides which OS thread receives CPU time.

3.1.2 The Problem: Sharing Interpreter State Across OS Threads

For example, the OS might schedule two Python threads simultaneously on different CPU cores:

Python Thread A → OS Thread A → CPU Core A
Python Thread B → OS Thread B → CPU Core B

The operating system can schedule both threads in parallel. Inside CPython, however, the threads share interpreter state, including object internals, reference counts, and runtime data structures.

This raises a separate runtime question:

How can multiple OS threads safely access shared state while executing Python interpreter code?

The default GIL-enabled CPython build uses the GIL to protect shared interpreter state. Whether a thread can execute protected Python code therefore depends on two independent conditions: OS scheduling and the GIL.

OS Scheduler
Decides which OS Thread gets CPU time

GIL
Restricts which Thread can execute protected Python interpreter code

3.1.3 The Default Solution: The GIL Restricts Interpreter Execution

In a default GIL-enabled build, a thread must hold the GIL while executing interpreter code protected by it:

OS Thread
    ↓
OS Scheduler
    ↓
CPU Time

Python Thread
    ↓
Acquire GIL
    ↓
Python Interpreter Execution

Executing Python bytecode therefore requires two things: the OS thread must have CPU time, and the thread must hold the GIL.

If another thread already holds the GIL:

Thread A
  ↓
Holds the GIL
  ↓
Executes Python bytecode

Thread B
  ↓
Attempts to acquire the GIL
  ↓
Waits
  ↓
The GIL is released
  ↓
Competes to acquire it again

Their responsibilities remain distinct:

The OS scheduler governs OS Thread → CPU.
The GIL gates Thread → Python Interpreter Execution in a default CPython build.

3.1.4 The Next Step: Free-Threaded CPython

To allow multiple Python threads to execute Python code in parallel, the normal execution path must no longer depend on that global lock:

Python Thread A → OS Thread A → CPU Core A
Python Thread B → OS Thread B → CPU Core B

A free-threaded build does not rely on the GIL as a global execution lock on its normal path. Instead, finer-grained synchronization protects CPython's internal state, allowing multiple Python threads to execute code in parallel. What changes is interpreter synchronization; the OS scheduler still schedules the same kind of native threads.

CPython thus offers two principal execution models:

Some incompatible extensions may still cause a free-threaded runtime to enable the GIL.

3.1.5 The Final Model: Thread, PyThreadState, and GIL

3.1.5.1 Python Thread

Python programs create and manage threads through threading.Thread.

In CPython, _thread ultimately creates a real OS thread.

3.1.5.2 OS Thread

The OS scheduler determines when an OS thread runs on a CPU.

Unlike Go's G-M-P or Java virtual threads, CPython does not multiplex large numbers of threading.Thread objects onto a smaller pool of OS threads.

3.1.5.3 PyThreadState

PyThreadState stores the interpreter-side runtime state associated with a Python thread.

A thread's execution context in CPython consists of two parts:

OS Thread
    +
PyThreadState
    ↓
Execution context for this Thread inside the CPython Runtime

3.1.5.4 GIL

In the default GIL-enabled build, a thread must hold the GIL to execute protected Python interpreter code.

Even when its OS thread has CPU time, it must still satisfy the GIL requirement before executing the corresponding Python bytecode.

3.2 Execution Layers

3.3 From CPython Thread Lifecycles to Their Implementation

CPython relies on OS scheduling for threading.Thread. The default GIL-enabled build also requires GIL ownership to execute protected interpreter code; free-threaded builds use different synchronization mechanisms.

3.3.1 Creation and Start: Created → Runnable

Thread.start() creates an OS thread and its PyThreadState; CPU execution still depends on OS scheduling.

3.3.2 Waiting for Scheduling: Runnable

The OS thread underlying a Python thread waits for OS scheduling; the default build must also satisfy its GIL requirement.

3.3.3 Execution: Runnable → Running

The OS scheduler grants CPU time to the native thread; in the default GIL-enabled build, the thread acquires the GIL before executing Python code.

3.3.4 Parking and Waiting: Running → Waiting

Waiting for I/O, locks, or conditions may block the underlying OS thread. In the default build, operations that release the GIL also allow another thread to execute protected interpreter code.

3.3.5 Wakeup and Resumption: Waiting → Runnable → Running

When ready, the OS thread reenters OS scheduling; the default build must reacquire the GIL before resuming protected Python execution if it was released.

3.3.6 Completion and Cleanup: Running → Terminated

After Thread.run() returns, the runtime cleans up PyThreadState, the native thread exits, and its ThreadHandle reaches the completed state.

3.4 End-to-End Sequence Diagram

Here is the thread lifecycle in default GIL-enabled CPython:

For implementation details, see CPython's _threadmodule.c, pystate.c, and ceval_gil.c; for free-threading, see the official guide.

3.5 Returning to the Opening Question

Section 3.1.5, Final Model and Section 3.2, Execution Layers show that threading.Thread maps to a native OS thread, while PyThreadState records its interpreter state. Therefore, the OS scheduler directly grants CPU execution opportunities; CPython has no separate user-space thread scheduler like Go's G-M-P.

Section 3.1.3, GIL, Section 3.3.2, Waiting for Scheduling, and Section 3.3.3, Execution show that a thread in the default GIL-enabled build must acquire the GIL before executing protected Python interpreter code. Therefore, CPU time alone does not guarantee Python-code execution; the GIL is an execution constraint, not another thread scheduler.

Section 3.3.4, Parking and Waiting, Section 3.3.5, Resumption, and Section 3.1.4, Free-Threaded establish that a blocking operation can suspend an OS thread and release the GIL, whereas free-threaded builds change Python-code synchronization without changing the threading.Thread to OS-thread mapping. Therefore, the interpreter's concurrency constraints change, but the execution carrier does not.

Answering the opening question: CPython relies on the OS scheduler for native threads. The default build uses the GIL to limit parallel execution of Python code within one interpreter, whereas free-threaded builds permit multiple threads to execute Python code in parallel; neither model multiplexes large numbers of threading.Thread instances onto a few OS threads.

4. What the Three Implementations Have in Common

After examining Java, Go, and CPython, we can set aside their specific APIs and runtime names. Three questions remain: what carries an execution unit, who decides when it runs, and how does it resume after waiting?

4.1 How Does an Execution Unit Reach the CPU? — Establish Its Carrier

First, distinguish the execution unit defined by a language or runtime from the OS thread that the operating system schedules. They do not necessarily have a one-to-one relationship.

Suppose a program has three execution units, A, B, and C. The most direct approach gives each its own OS thread:

Execution Unit A → OS Thread 1
Execution Unit B → OS Thread 2
Execution Unit C → OS Thread 3

Tasks may, however, spend most of their time waiting for I/O. If every task permanently occupies an OS thread, native-thread resources must remain allocated even for waiting tasks.

Another approach separates tasks from underlying threads:

Runnable Task A ─┐
Runnable Task B ─┼─→ Runtime Dispatch → OS Thread 1
Runnable Task C ─┘                 (reused over time)

One OS thread can execute different tasks at different times. Models that do not require user-space scheduling can keep the direct one-to-one mapping. Both ultimately reach the same execution path:

Therefore, execution units can have different carriers, but the OS scheduler ultimately dispatches OS threads—not language-level tasks—to the CPU.

4.2 Who Selects What Runs Next? — Separate the Scheduling Layers

Establishing a carrier does not mean that a task immediately runs. Two different decisions remain:

Task selection: If a runtime manages lightweight runnable tasks, it chooses one to assign to an underlying thread.
Thread scheduling: The OS scheduler selects a runnable OS thread to receive CPU time.

Consider a single CPU core. OS threads T1 and T2 are both runnable, and a runtime has already assigned task A to T1:

Runtime: select Task A → assign to T1

OS Scheduler: T1 and T2 are both Runnable
                         ↓
                    select T2 first
                         ↓
                   CPU runs T2

Although the runtime has selected task A, it cannot make progress until T1 receives CPU time. Conversely, an OS thread receiving CPU time does not guarantee that a language-level synchronization condition is satisfied; such a condition is not another thread scheduler.

With direct one-to-one mapping, a separate runtime thread scheduler may be unnecessary, but OS scheduling still applies. Long-running work may also have to yield or be preempted so other tasks can make progress.

Therefore, runtime task dispatch and OS thread scheduling make choices at different levels; Runnable is not the same as Running.

4.3 What Happens After a Wait? — Regain an Execution Opportunity

Suppose task A is executing and then starts an I/O operation, while task B is already runnable. The central question is: can A's underlying OS thread run B while A waits?

Task A running → waits for I/O
                     │
                     ├─ Suspend only logical Task A
                     │       └─ OS thread can run Task B
                     │
                     └─ Block the underlying OS thread
                             └─ That thread waits; other OS threads can run

Either path is possible, depending on the execution model and whether the specific wait releases its carrier. Suspending a logical task does not necessarily block its OS thread; blocking one OS thread does not stop the entire process.

When the I/O completes, task A is not guaranteed to run immediately. It becomes eligible for scheduling again:

Waiting
   ↓ condition ready / wakeup
Runnable
   ↓ optionally queue in runtime and wait for a carrier
   ↓ OS thread receives CPU time
Running

If only the logical task was suspended, the OS thread could make progress on other tasks. If the OS thread itself was blocked, it must become runnable and receive CPU time again. Waiting and resumption determine whether execution resources can be reused; wakeup does not mean immediate execution.

Returning to the opening question: a language-level execution unit must first be associated with an OS thread. When present, a runtime scheduler selects the task; the OS scheduler assigns CPU time to its underlying thread. Waiting and resumption determine whether that thread can serve other tasks in the meantime. Together, these mechanisms explain how concurrent execution units reach the CPU.

5. Next: Language Memory Models

The previous chapter explained how the CPU executes instructions. This chapter addresses the missing link: how runtimes and operating systems give concurrent execution units CPU time and resume them after waiting.

With the execution path established, the next chapter turns to the Language Memory Model—the rules governing concurrent reads and writes, which provide the foundation for later discussions of Mutex, Atomic, Volatile, and other synchronization mechanisms.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Concurrency Programming (8): Read-Write Locks — From Language Rules to the CPU

ThinkerQAQ — Wed, 07 Oct 2026 11:16:25 +0000

0. What Does This Article Answer Next?
1. What Does Java ReadWriteLock Guarantee at the Language Level?
- 1.1 Atomicity
- 1.2 Visibility and Ordering
2. How Does the JDK Implement ReentrantReadWriteLock?
- 2.1 Layers
- 2.2 End-to-End Path from Write Lock to Read Lock
- 2.3 Guarantees at the Runtime / Language Implementation Layer
- 2.3.1 Atomicity
- 2.3.2 Visibility and Ordering
- 2.4 Guarantees at the Hardware Layer
3. How Does the Go Runtime Implement sync.RWMutex?
- 3.1 Layers
- 3.2 End-to-End Path from Write Lock to Read Lock
- 3.3 Guarantees at the Runtime / Language Implementation Layer
- 3.3.1 Atomicity
- 3.3.2 Visibility and Ordering
- 3.4 Guarantees at the Hardware Layer
4. CPython: Why Is There No Read-Write Lock in the Standard Library?
5. Common Pattern Across the Two Implementations
- 5.1 Who Gets In? — Atomically Maintaining Reader / Writer State
- 5.2 What Happens on Conflict? — Park / Wakeup
- 5.3 Why Can a Later Reader See an Earlier Writer's Data?
6. How ReadWriteLock and Mutex Relate
7. Next: AQS / AQLS — Java's Unified Synchronizer Framework

0. What Does This Article Answer Next?

The previous article ended by returning to Mutex: it protects an entire critical section, but it allows only one execution unit to enter at a time, regardless of whether that unit reads or writes.

If two threads only read shared state, they do not need to block each other:

Thread A                         Thread B

read counter                    read counter

A read-write lock refines Mutex by distinguishing Readers from Writers:

Read  + Read   → concurrent
Read  + Write  → exclusive
Write + Write  → exclusive

This article continues with the same two examples:

counter++ to see how the Write Lock provides Atomicity;
counter + ready to see how Write Lock → Read Lock establishes Visibility and Ordering.

As before, we start with the language-level rules, then trace how the JDK, HotSpot, and the CPU implement them.

1. What Does Java ReadWriteLock Guarantee at the Language Level?

1.1 Atomicity

Java's ReadWriteLock defines the Reader / Writer boundary directly:

“The read lock may be held simultaneously by multiple reader threads, so long as there are no writers.”

It also states:

“The write lock is exclusive.”

Reads may therefore proceed concurrently, while writes remain exclusive.

Use the same counter++ example:

ReentrantReadWriteLock rw = new ReentrantReadWriteLock();
Lock writeLock = rw.writeLock();

writeLock.lock();
try {
    counter++;
} finally {
    writeLock.unlock();
}

With two threads updating concurrently:

Initial: counter = 0

Thread A                         Thread B

writeLock.lock()
                                 writeLock.lock()
                                 wait
read counter -> 0
counter + 1 -> 1
write counter -> 1
writeLock.unlock()
                                 writeLock.lock() returns
                                 read counter -> 1
                                 counter + 1 -> 2
                                 write counter -> 2
                                 writeLock.unlock()

The final value is:

counter = 2

As with Mutex, Atomicity comes from serializing conflicting critical sections under the Write Lock, not from turning counter++ itself into an atomic instruction.

A Read Lock should protect reads only. Multiple Readers may enter together, so it cannot be used to protect shared writes.

1.2 Visibility and Ordering

ReadWriteLock also defines memory synchronization between the Write Lock and the associated Read Lock:

“A thread successfully acquiring the read lock will see all updates made upon previous release of the write lock.”

Use the same counter / ready example:

int counter = 0;
boolean ready = false;

ReentrantReadWriteLock rw = new ReentrantReadWriteLock();
Lock readLock = rw.readLock();
Lock writeLock = rw.writeLock();

// Thread A
writeLock.lock();
try {
    counter = 1;
    ready = true;
} finally {
    writeLock.unlock();
}

// Thread B
readLock.lock();
try {
    if (ready) {
        System.out.println(counter);
    }
} finally {
    readLock.unlock();
}

If B successfully acquires the Read Lock after A releases the Write Lock:

Thread A                              Thread B

counter = 1
    │
    │ program order
    ▼
ready = true
    │
    ▼
writeLock.unlock()
    │
    │ memory synchronization
    ▼
                                    readLock.lock() returns
                                        │
                                        │ program order
                                        ▼
                                    read ready == true
                                    read counter

By happens-before:

write counter
    ↓ happens-before
write ready
    ↓
writeLock.unlock
    ↓ memory synchronization
readLock.lock returns
    ↓ happens-before
read counter

So once B has acquired the subsequent Read Lock and observes ready == true, the earlier counter = 1 must be visible. This outcome is not allowed:

ready == true
counter == 0

That gives us two guarantees:

Visibility: writes completed by the Writer before releasing the Write Lock are visible to a later Reader that acquires the Read Lock;
Ordering: the Writer's writes, Write Lock release, Read Lock acquisition, and the Reader's reads form a happens-before order.

2. How Does the JDK Implement ReentrantReadWriteLock?

2.1 Layers

Start with the implementation stack:

Java Source Code
Example: readLock.lock() / writeLock.lock()
Role: declare read and write critical sections
        │
        ▼
JVM Bytecode
Example: invokeinterface Lock.lock / Lock.unlock
Role: invoke the ReadLock / WriteLock API
        │
        ▼
Runtime / Language Implementation
Example: ReentrantReadWriteLock.Sync / AbstractQueuedLongSynchronizer
       + Unsafe CAS / LockSupport
Role: implement Reader / Writer state, contention, queuing, and wake-up
        │
        ▼
OS Thread
Example: Runnable / Parked Java Thread
Role: block on contention and become runnable again after unpark
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Memory Ordering
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

2.2 End-to-End Path from Write Lock to Read Lock

Continue with counter / ready: Thread A writes under the Write Lock, then Thread B reads under the Read Lock.

The sequence diagram keeps only the cross-layer path. The Runtime / Language Implementation layer can be expanded into the Reader / Writer contention flow below:

Two counters are encoded in the synchronization state:

high 32 bits
  → shared count
  → number of Readers

low 32 bits
  → exclusive count
  → Writer reentrancy count

In JDK 25, ReentrantReadWriteLock.Sync extends AbstractQueuedLongSynchronizer. Readers use Shared mode; Writers use Exclusive mode; both ultimately contend on the same 64-bit state.

See OpenJDK 25 ReentrantReadWriteLock.java and AbstractQueuedLongSynchronizer.java.

2.3 Guarantees at the Runtime / Language Implementation Layer

2.3.1 Atomicity

Atomicity comes from the state transitions in the flowchart:

Writer
  → CAS exclusive count
  → exclusive write critical section after success

Reader
  → CAS shared count
  → multiple Readers may succeed independently

A Writer fails if Readers are present or another thread owns the Write Lock.

If the current thread already owns the Write Lock, acquisition is reentrant and increments the exclusive count.

A Reader fails if another thread owns the Write Lock; otherwise it increments the shared count with CAS.

The runtime therefore enforces:

Read + Read
  → multiple Shared acquires allowed

Read + Write
  → conflict

Write + Write
  → conflict

2.3.2 Visibility and Ordering

AbstractQueuedLongSynchronizer stores its state in a volatile long:

private volatile long state;

Its basic operations have these semantics:

getState()
  → volatile read

setState()
  → volatile write

compareAndSetState()
  → volatile read + volatile write

Releasing the Write Lock updates state. A later Reader reads and CASes the same state while acquiring the Read Lock:

Writer writes
    ↓
release Write Lock
    ↓
volatile / CAS state synchronization
    ↓
acquire Read Lock
    ↓
Reader reads

This is where the language-level Visibility and Ordering guarantees are carried by the Runtime / Language Implementation layer.

2.4 Guarantees at the Hardware Layer

At the hardware layer, the implementation reduces to the same three capabilities used throughout the series:

Atomicity
  → Atomic Instruction
  → x86-64: LOCKed CAS and other atomic RMW operations

Visibility
  → Cache Coherence
  → other CPU cores cannot keep using an invalidated old cache line indefinitely

Ordering
  → x86-64 Memory Ordering + LOCKed RMW / required ordering constraints
  → preserve the memory-access order required around lock-state transitions

compareAndSetState() ultimately uses Unsafe.compareAndSetLong, which HotSpot lowers to an atomic CAS on the target architecture.

Read-write locks add no new hardware primitive. They build the rule “multiple Readers or one Writer” on top of atomic state plus thread waiting / wake-up.

3. How Does the Go Runtime Implement sync.RWMutex?

Go's standard library directly provides sync.RWMutex.

It shares the same core Reader / Writer exclusion model as Java ReadWriteLock: multiple Readers may proceed concurrently, while a Writer must be exclusive. The complete API semantics are not identical.

The most important differences are:

	Java `ReentrantReadWriteLock`	Go `sync.RWMutex`
Read-lock reentrancy	Supported	Recursive `RLock` is not supported
Write-lock reentrancy	Supported	Not supported
Write → Read downgrade	Supported	Not supported
Read → Write upgrade	Not supported	Not supported
Fairness	Optional fair / nonfair mode	No fair mode

The Go Memory Model states:

“For any call to RLock, there exists an n such that the n'th call to Unlock ‘synchronizes before’ that call to RLock.”

3.1 Layers

Go Source Code
Example: rw.RLock() / rw.Lock()
Role: declare read and write critical sections
        │
        ▼
Go Implementation
Example: sync.RWMutex / sync.Mutex / sync/atomic / Runtime Semaphore
Role: implement Reader / Writer state, contention, waiting, and wake-up
        │
        ▼
Goroutine / Scheduler
Example: Runnable / Parked Goroutine
Role: park a contending Goroutine and schedule it again after wake-up
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Memory Ordering
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

The important distinction from Java is that runtime_SemacquireRWMutex* parks a Goroutine rather than directly blocking one fixed OS thread.

3.2 End-to-End Path from Write Lock to Read Lock

Continue with the same counter / ready example:

var counter int
var ready bool
var rw sync.RWMutex

Goroutine A writes first, then Goroutine B reads:

The sequence diagram keeps only the cross-layer path. The Go Implementation layer can be expanded into the Reader / Writer contention flow below:

The core state is:

w
  → serialize Writers

readerCount
  → >= 0: number of active Readers
  → < 0: a Writer is pending

readerWait
  → number of departing Readers the Writer is still waiting for

readerSem / writerSem
  → Reader / Writer waiting and wake-up

See Go's sync/rwmutex.go.

3.3 Guarantees at the Runtime / Language Implementation Layer

3.3.1 Atomicity

Atomicity comes from two core state operations in the flowchart.

A Reader enters through:

readerCount.Add(1)

That update must be atomic. Multiple Readers may increment the count independently.

A Writer first serializes against other Writers through the internal w Mutex, then executes:

readerCount.Add(-rwmutexMaxReaders)

to mark a pending Writer.

The runtime therefore enforces:

Read + Read
  → multiple Readers allowed

Read + Write
  → new Readers block once a Writer is pending

Write + Write
  → serialized by w Mutex

Unlike Java, Go does not encode every state in one state word. It combines a Mutex + Atomic Counter + Semaphore to implement the same Reader / Writer rules.

3.3.2 Visibility and Ordering

Go defines RWMutex synchronization at the API level; it should not be inferred from readerCount atomics alone.

The documented relations include:

Unlock
  → synchronized-before
  → the corresponding later RLock

RUnlock
  → synchronized-before
  → the corresponding later Lock

Using the same counter / ready example:

Goroutine A                           Goroutine B

counter = 1
ready = true
    │
    ▼
Unlock() ── synchronized-before ──► RLock() returns
                                        │
                                        ▼
                                    read ready == true
                                    read counter == 1

So once B acquires the Read Lock through this synchronization edge, A's writes under the Write Lock are visible to B in the required order.

Internally, the Mutex, runtime semaphores, and atomic state implement this path. The readerCount.Add operation alone should not be interpreted as synchronizing Readers with one another.

3.4 Guarantees at the Hardware Layer

At the hardware layer, the implementation reduces to the same three capabilities used throughout the series:

Atomicity
  → Atomic Instruction
  → x86-64: LOCK XADDL / LOCK CMPXCHGL and other atomic RMW operations

Visibility
  → Cache Coherence
  → other CPU cores cannot keep using an invalidated old cache line indefinitely

Ordering
  → x86-64 Memory Ordering + LOCKed RMW
  → provide the ordering constraints required by the synchronization path

readerCount.Add and readerWait.Add ultimately use the x86-64 atomic-add path. The internal w Mutex reuses the CAS / atomic machinery already covered in the Mutex implementation article.

So sync.RWMutex introduces no new hardware primitive. It combines atomics, a Mutex, and runtime semaphores to implement Reader / Writer semantics.

4. CPython: Why Is There No Read-Write Lock in the Standard Library?

Python is capable of implementing a Read-Write Lock. In fact, CPython had an early proposal to add RWLock to threading: Issue 8800.

The proposal never made it into the standard library. Looking at the long-running Issue 8800 discussion, at least two recurring obstacles stand out.

1. Under traditional CPython, the GIL limited the benefit of a Read-Write Lock

A CPython developer summarized the problem directly:

“since the GIL serializes everything anyway, this isn't likely to benefit many situations”

The basic trade-off was:

Main benefit of a Read-Write Lock
  → multiple Readers run in parallel

Traditional CPython
  → the GIL largely serializes execution of Python code

So even if multiple threads could acquire the Read Lock together, they usually could not execute Python code in parallel.

RWLock was not useless: I/O and C extensions that release the GIL could still benefit. But for general CPython threading, its advantage was much less direct than in Java or Go.

Free-threaded CPython changes this point. Without the GIL, multiple Python threads can truly execute in parallel, so concurrent Readers become substantially more valuable.

That changes the first part of the trade-off. But even if the benefit of RWLock becomes clearer, the standard library still faces the second problem: there was no consensus on the API and scheduling semantics.

2. Issue 8800 did not reach consensus on the API or scheduling policy

The discussion had two main classes of disagreement.

The first was the API shape:

one unified RWLock
or
two associated Shared / Exclusive Lock objects

One proposal argued that:

“having two different lock primitives ... would be much more flexible, pythonic”

Others preferred a single RWLock abstraction.

The second was the scheduling policy:

Reader priority or Writer priority?
Fair / FIFO?
May new Readers enter while a Writer waits?
Can a Read Lock be upgraded?
Can a Write Lock be downgraded?

Those choices directly affect RWLock behavior and API semantics, and Issue 8800 never converged on one standard design.

So the obstacle was not implementation feasibility. There was no consensus on which RWLock API and policy the standard library should permanently define.

The current threading standard library still provides Lock, RLock, Condition, Semaphore, Event, and related primitives, but no Read-Write Lock.

Applications that need RWLock semantics generally have two options:

Use a third-party / custom RWLock
        │
        └── obtain explicit Read Lock / Write Lock semantics

Build on threading.Lock + Condition
        │
        └── define Reader / Writer counts and scheduling policy yourself

The original Issue 8800 proposal itself implemented RWLock by composing a condition variable and a lock.

What Python lacks is therefore a standardized RWLock API and policy in the standard library, not the ability to implement Read-Write Lock semantics.

5. Common Pattern Across the Two Implementations

Strip away the APIs and compare what Java and Go must solve.

5.1 Who Gets In? — Atomically Maintaining Reader / Writer State

A Mutex only needs to answer whether the lock is held.

A ReadWriteLock must maintain:

Reader Count
+
Writer State

Java:

64-bit state
  → shared count
  → exclusive count

Go:

readerCount
+
Writer Mutex
+
readerWait

The encoding differs, but the admission rules are the same:

Reader
  → no conflicting Writer

Writer
  → no other Writer
  → no Reader

Those state transitions must be atomic. Otherwise competing execution units could observe stale state and enter incorrectly.

5.2 What Happens on Conflict? — Park / Wakeup

After a state check fails, a waiter cannot spin forever.

Both implementations reduce to:

check Reader / Writer state
        │
        ├── allowed
        │      └── update state and enter
        │
        └── conflict
               │
               ▼
           queue / wait
               │
               ▼
              Park
               │
               ▼
       relevant holder releases
               │
               ▼
            Wakeup
               │
               └── Retry

The parked unit differs:

Java
  → AQLS Sync Queue
  → LockSupport.park()
  → OS Thread Parked

Go
  → Runtime Semaphore
  → Park Goroutine
  → Scheduler runs another Goroutine

So the common pattern is Conflict → Park → Wakeup → Retry. Java parks a Thread; Go parks a Goroutine.

5.3 Why Can a Later Reader See an Earlier Writer's Data?

The final problem is the synchronization boundary:

Writer writes
    │
    ▼
Write Unlock [Release]
    │
    ▼
Read Lock [Acquire]
    │
    ▼
Reader reads

Java implements this boundary through AQLS volatile / CAS state transitions.

Go defines the semantics through RWMutex synchronized-before rules and implements them with its internal Mutex, runtime semaphores, and atomic operations.

Both implementations must guarantee the same result:

Writes before the Writer releases the lock precede reads performed after the later Reader acquires it.

6. How ReadWriteLock and Mutex Relate

Both protect critical sections. The difference is whether Readers and Writers are distinguished:

	Mutex	ReadWriteLock
Core problem	mutual exclusion for a critical section	distinguish read and write critical sections
Atomicity	entire critical section is exclusive	conflicting operations are exclusive; Readers may coexist
Visibility	Yes	Yes
Ordering	Yes	Yes
Read–Read	Exclusive	Concurrent
Read–Write	Exclusive	Exclusive
Write–Write	Exclusive	Exclusive
Typical use	general shared-state protection	read-heavy workloads with real contention in read-side critical sections

A compact model is:

Mutex
  → everyone uses one Exclusive entry

ReadWriteLock
  → Readers use the Shared entry
  → Writers use the Exclusive entry
  → only Read + Read is relaxed

ReadWriteLock is not a “stronger” Mutex.

It refines the exclusion boundary so non-conflicting Readers may proceed concurrently.

7. Next: AQS / AQLS — Java's Unified Synchronizer Framework

This article has already crossed the same path several times:

ReentrantReadWriteLock
        ↓
Sync
        ↓
AbstractQueuedLongSynchronizer

That is not merely an implementation detail of one lock. It belongs to the lower-level synchronizer framework used throughout Java's concurrency library.

The next article goes deeper into the AQS family:

AQS / AQLS
  → state
  → Exclusive / Shared
  → CLH-style Queue
  → acquire / release
  → park / unpark
  → Condition Queue

In JDK 25, ReentrantReadWriteLock specifically uses AbstractQueuedLongSynchronizer. Its core model is the same as AbstractQueuedSynchronizer, but the synchronization state is a long.

Go has no single framework directly equivalent to AQS.

Instead, Go synchronization primitives tend to compose lower-level mechanisms directly:

Atomic State
+
Mutex
+
Runtime Semaphore
+
Goroutine Scheduler

For example, sync.RWMutex directly combines w, readerCount, readerWait, readerSem, and writerSem rather than inheriting from a generic AQS-style synchronizer.

AQS / AQLS is therefore a major layer in Java's lock implementation stack and deserves its own article next.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Concurrency Programming (7): volatile — From Language Semantics to the CPU

ThinkerQAQ — Sat, 03 Oct 2026 12:24:28 +0000

0. What Does This Article Answer Next?
1. What Guarantees Does Java volatile Provide at the Language Level?
- 1.1 Visibility and Ordering
- 1.2 volatile Does Not Make Compound Operations Atomic
2. How Does HotSpot Implement volatile?
- 2.1 Layers
- 2.2 End-to-End Implementation Path for a volatile Write and Read
- 2.3 Guarantees at the Runtime / Language Implementation Layer
- 2.3.1 Visibility
- 2.3.2 Ordering
- 2.4 Guarantees at the Hardware Layer
3. Why Don't Go and CPython Have Java-style volatile?
- 3.1 Go: A Language Design Choice
- 3.2 CPython: Synchronization Remains a Runtime Responsibility
4. How volatile, Atomics, and Mutexes Differ
5. Next: From Mutexes to Read-Write Locks

0. What Does This Article Answer Next?

The previous two articles traced atomic operations from language-level semantics down to the CPU, showing how Atomicity, Visibility, and Ordering are implemented.

This article follows the same path for volatile.

volatile does not make a compound update such as counter++ atomic. It is better suited to publication and observation:

Thread A                         Thread B

counter = 1                     if (ready) {
ready = true                        print(counter)
                                }

The required guarantee is:

If Thread B observes ready == true, a subsequent read of counter must observe 1.

We first establish this guarantee in the Java Memory Model, then trace how HotSpot preserves it down to the CPU.

1. What Guarantees Does Java volatile Provide at the Language Level?

1.1 Visibility and Ordering

Declare ready as volatile:

int counter = 0;
volatile boolean ready = false;

// Thread A
counter = 1;
ready = true;

// Thread B
if (ready) {
    System.out.println(counter);
}

counter remains an ordinary field.

JLS §17.4.4 states:

“A write to a volatile variable v synchronizes-with all subsequent reads of v by any thread.”

So if Thread B's volatile read observes the value true written by Thread A, the relationship is:

Thread A                              Thread B

counter = 1
    │
    │ program order
    ▼
volatile ready = true ── synchronizes-with ──► read volatile ready == true
                                                   │
                                                   │ program order
                                                   ▼
                                               read counter

By transitivity of happens-before:

write counter
    ↓ happens-before
write volatile ready
    ↓ synchronizes-with
read volatile ready
    ↓ happens-before
read counter

Once B observes ready == true, the earlier counter = 1 write must be visible to B. The following outcome is therefore forbidden:

ready == true
counter == 0

That gives us two guarantees:

Visibility: writes completed by A before the volatile write become visible to B after B observes that write;
Ordering: the operations must be observed in a way that is consistent with the happens-before relationship.

1.2 volatile Does Not Make Compound Operations Atomic

Consider:

volatile int counter = 0;

counter++;

counter++ is still a read-modify-write operation:

volatile read
      ↓
     add
      ↓
volatile write

volatile constrains the visibility and ordering of the individual read and write, but it does not make the entire read → add → write sequence indivisible.

With multiple threads incrementing concurrently, updates can still be lost.

If you need an atomic RMW, use AtomicInteger, CAS, or a mutex.

2. How Does HotSpot Implement volatile?

2.1 Layers

As in the Mutex and Atomic articles, first establish the implementation stack:

Java Source Code
Example: volatile boolean ready
Role: declare a field with volatile memory semantics
        │
        ▼
JVM Bytecode
Example: getfield / putfield (or getstatic / putstatic)
Field metadata: ACC_VOLATILE
Role: perform ordinary field access; volatility is carried by field metadata
        │
        ▼
JVM Implementation (HotSpot)
Example: field->is_volatile() / MO_SEQ_CST / MemBar
Role: preserve and implement volatile memory-ordering semantics
        │
        ▼
x86-64 Hardware
Example: Load / Store / Cache Coherence / Fence
Role: provide the hardware basis for Visibility and Ordering

This differs from both synchronized and AtomicInteger:

synchronized
  → dedicated monitorenter / monitorexit bytecodes

AtomicInteger
  → ordinary invokevirtual
  → no Atomic-specific bytecode

volatile
  → still uses getfield / putfield
  → no dedicated volatile-load / volatile-store bytecode
  → the volatile property is recorded in ACC_VOLATILE field metadata

2.2 End-to-End Implementation Path for a volatile Write and Read

Continue with the same counter / ready example:

The key point is not a particular machine instruction. HotSpot must recognize that ready is volatile and preserve its memory semantics all the way to the target architecture.

2.3 Guarantees at the Runtime / Language Implementation Layer

2.3.1 Visibility

HotSpot distinguishes ordinary fields from volatile fields while parsing field accesses. In the current C2 path, volatile accesses carry memory-order information such as MO_SEQ_CST, while ordinary fields use unordered access semantics.

Conceptually:

ordinary field
  → ordinary load / store

volatile field
  → volatile load / store
  → preserve cross-thread observable memory semantics

HotSpot therefore cannot optimize a volatile access as if it were an ordinary field access—for example, by eliminating, merging, or moving it across synchronization boundaries that matter.

2.3.2 Ordering

HotSpot must also constrain reordering around volatile accesses.

In C2's IR, the typical structure includes:

volatile store
  ← ordering constraints such as MemBarRelease

volatile load
  → ordering constraints such as MemBarAcquire

Cases that require stronger ordering—such as volatile store → volatile load—also require the corresponding StoreLoad / volatile-barrier constraint.

These are IR nodes and compiler-ordering constraints, not a one-to-one promise that every barrier becomes a separate CPU fence instruction.

Relevant OpenJDK sources include parse3.cpp, memnode.hpp, and templateTable_x86.cpp.

2.4 Guarantees at the Hardware Layer

At the hardware layer, these guarantees reduce to the same two capabilities introduced earlier in the series:

Visibility
  → Cache Coherence
  → other CPU cores cannot keep using an invalidated old cache line indefinitely

Ordering
  → CPU Memory Ordering + Fence / equivalent ordering constraints
  → preserve the memory-access order required at volatile boundaries

x86-64 already provides relatively strong memory ordering, so many Acquire / Release constraints do not require a standalone hardware fence instruction.

When a StoreLoad fence is required, HotSpot's Linux x86 path can use:

lock; addl $0, 0(%rsp)

to provide a full-fence effect.

See OpenJDK orderAccess_linux_x86.hpp.

3. Why Don't Go and CPython Have Java-style volatile?

Go and Python do not expose Java's field-level volatile mechanism. The reasons are different, so consider them separately.

3.1 Go: A Language Design Choice

Effective Go summarizes one of Go's central concurrency design principles with a well-known rule:

“Do not communicate by sharing memory; instead, share memory by communicating.”

That reflects a language-design choice: synchronization should be expressed through explicit concurrency primitives rather than hidden in field modifiers.

Accordingly, Go does not provide a Java-style field-level volatile modifier. It expresses the same kinds of synchronization through channels, mutexes, and atomic operations.

For the use case closest to Java volatile, Go uses sync/atomic. The Go Memory Model states:

“This definition provides the same semantics as C++'s sequentially consistent atomics and Java's volatile variables.”

Use the same counter / ready example:

var counter int
var ready atomic.Bool

// Goroutine A
counter = 1
ready.Store(true)

// Goroutine B
if ready.Load() {
    fmt.Println(counter)
}

If ready.Load() observes the value written by ready.Store(true), the two atomic operations establish a synchronized-before relationship:

Goroutine A                          Goroutine B

counter = 1
    │
    │ sequenced-before
    ▼
ready.Store(true) ── synchronized-before ──► ready.Load() == true
                                                 │
                                                 │ sequenced-before
                                                 ▼
                                             read counter

So once B observes ready == true, the earlier counter = 1 write must be visible to B.

3.2 CPython: Synchronization Remains a Runtime Responsibility

Python does not expose Java-style volatile. The key reason is not syntax; synchronization has historically been a responsibility of the CPython runtime.

There are two main points.

1. Traditional CPython uses the GIL to cover many Visibility / Ordering concerns

The CPython C API documentation states:

“only a thread that holds the GIL may operate on Python objects or invoke Python’s C API.”

In traditional CPython, the GIL serializes many operations on Python objects.

That serialization also provides many of the practical Visibility and Ordering effects that Java code might otherwise obtain from volatile.

The abstractions are still different:

Java volatile
  → field-level language semantics

CPython GIL
  → runtime-level synchronization mechanism

2. Free-threaded CPython still uses runtime locks and atomics

Making the GIL optional did not introduce field-level volatile semantics.

PEP 703 instead says:

“This PEP proposes using per-object locks...”

The runtime also uses atomic operations internally.

So the implementation shifts from:

GIL

to:

Per-object lock + atomic

Synchronization therefore remains a runtime responsibility rather than becoming part of Python's field-level language semantics.

At the application level, publication and observation are expressed through synchronization objects. The documentation for threading.Event says:

“one thread signals an event and other threads wait for it.”

Use the same counter / ready example:

counter = 0
ready = threading.Event()

# Thread A
counter = 1
ready.set()

# Thread B
ready.wait()
print(counter)

Here, Event plays the synchronization role that ready played in the Java example:

Thread A                         Thread B

counter = 1
    │
    ▼
ready.set()  ───────────────►  ready.wait() returns
                                  │
                                  ▼
                              read counter

So Python application code uses Event / Lock for synchronization, while lower-level atomics and memory ordering remain implementation details of the CPython runtime.

4. How volatile, Atomics, and Mutexes Differ

The three abstractions operate at different granularities:

	`volatile`	Atomic	Mutex
Core problem	publish / observe shared state	one shared-state operation	a critical section
Atomicity	does not make compound RMW atomic	one atomic operation is indivisible	critical section is mutually exclusive
Visibility	Yes	Yes	Yes
Ordering	Yes	Yes	Yes
Typical use	initialization / configuration / stop flags; state publication	counters / Add / CAS / Swap	multi-step logic / multi-field updates

A compact way to view the relationship is:

volatile
  → establish Visibility / Ordering between a publication and a later observation

Atomic
  → make one shared-state operation indivisible
  → also provide the corresponding Visibility / Ordering

Mutex
  → use lower-level primitives such as atomics to arbitrate lock state
  → then protect an entire critical section

These are not simply stronger and weaker versions of the same tool. They solve different problems at different granularities.

5. Next: From Mutexes to Read-Write Locks

A Mutex allows only one execution unit to enter a critical section at a time.

If most operations only read shared state, making readers exclude one another is unnecessary.

The next article looks at read-write locks: multiple readers can proceed concurrently, while writers still retain exclusive access.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Concurrency Programming (6): Atomic Implementation — From Runtime to CPU

ThinkerQAQ — Thu, 01 Oct 2026 06:36:04 +0000

0. What Does This Article Continue to Answer?
1. How Does HotSpot Implement AtomicInteger?
- 1.1 Layers
- 1.2 A Complete AtomicInteger Implementation Path
- 1.3 Atomicity
- 1.4 Visibility and Ordering
2. How Does Go Implement sync/atomic?
- 2.1 Layers
- 2.2 A Complete atomic.Int64 Implementation Path
- 2.3 Atomicity
- 2.4 Visibility and Ordering
3. How Does CPython Implement Internal Atomics?
- 3.1 Layers
- 3.2 A Complete CPython Atomic Implementation Path
- 3.3 Atomicity
- 3.4 Visibility and Ordering
4. The Common Pattern Behind All Three Implementations
- 4.1 How Does a Single-variable Update Become Indivisible? — Atomic RMW
- 4.2 What Happens When a Conditional Update Fails? — CAS / Retry
- 4.3 Why Can Atomics Also Establish Visibility and Ordering?
5. Next: volatile

0. What Does This Article Continue to Answer?

The previous article described the atomicity, visibility, and ordering guarantees exposed by Atomic APIs at the language level. This article follows those guarantees down through the compiler and runtime to the CPU.

We continue to use the same two examples:

counter++

and:

counter = 1
ready = true

The implementation path can be summarized as:

Language API and memory model
        ↓
Compiler / Runtime
        ↓
CPU Atomic Instruction / Memory Ordering

1. How Does HotSpot Implement AtomicInteger?

1.1 Layers

Java API
Example: AtomicInteger.incrementAndGet() / compareAndSet() / get()
Role: declare single-variable Atomic operations
        │
        ▼
JDK Implementation
Example: AtomicInteger / Unsafe / VarHandle Memory Effects
Role: express Atomic RMW, CAS, and memory semantics
        │
        ▼
JVM Implementation (HotSpot)
Example: Intrinsic / Atomic Node
Role: map Atomic operations to the target architecture
        │
        ▼
x86-64 Hardware
Example: LOCK XADD / LOCK CMPXCHG / Load
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

1.2 A Complete AtomicInteger Implementation Path

Common Atomic operations can be expanded as:

1.3 Atomicity

On x86-64, an atomic addition can be simplified as:

mov  eax, 1
lock xadd dword ptr [counter], eax

XADD reads the old value and writes old + 1; the LOCK prefix makes the whole RMW indivisible with respect to competing cores.

CAS can be simplified as:

; EAX contains expected
; ECX contains newValue
lock cmpxchg dword ptr [counter], ecx
sete al

These snippets illustrate equivalent machine-level operations; they are not a claim about the exact full instruction sequence emitted by every JVM version.

For counter + ready, HotSpot must also constrain compile-time reordering according to the JMM and the memory effects of the specific Atomic method, then map those constraints onto x86-64 memory ordering. Looking only at a LOCK instruction and ignoring the compiler layer would miss part of the implementation.

1.4 Visibility and Ordering

Java Atomic methods carry explicit memory effects in addition to indivisible RMW. incrementAndGet() / getAndAdd() use the volatile read/write effects of VarHandle.getAndAdd, while get() has volatile-read semantics.

HotSpot must preserve those compiler-ordering constraints before x86-64 memory ordering, cache coherence, and locked RMW provide the hardware behavior.

Atomicity
  → Atomic Instruction
  → HotSpot / x86-64: LOCK XADD / LOCK CMPXCHG

Visibility
  → Cache Coherence
  → Typical x86-64 CPU: MESI-family (for example MESIF / MOESI)

Ordering
  → Compiler Ordering + Hardware Memory Ordering
  → HotSpot Memory Effects + x86-64 Locked RMW / Load rules

LOCK XADD / LOCK CMPXCHG are typical x86-64 mappings, not a promise that every JVM version or architecture emits the same instruction sequence.

See OpenJDK AtomicInteger.java, Unsafe.java, and HotSpot atomicAccess_linux_x86.hpp.

2. How Does Go Implement sync/atomic?

2.1 Layers

Go API
Example: atomic.Int64.Add() / CompareAndSwap() / Load()
Role: declare single-variable Atomic operations
        │
        ▼
Go Implementation (sync/atomic)
Example: atomic.Int64 / AddInt64 / CompareAndSwapInt64
Role: express Add, CAS, Load, and related Atomic semantics
        │
        ▼
Go Runtime Atomic
Example: internal/runtime/atomic
Role: implement target-architecture Atomic primitives
        │
        ▼
x86-64 Hardware
Example: LOCK XADDQ / LOCK CMPXCHGQ / Load
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

2.2 A Complete atomic.Int64 Implementation Path

Common Atomic operations can be expanded as:

2.3 Atomicity

On amd64, atomic addition can reach an XADD with the LOCK prefix:

LOCK
XADDQ AX, 0(BX)

CAS can reach:

LOCK
CMPXCHGQ CX, 0(BX)

So counter.Add(1) can use a hardware atomic-add operation directly rather than first being rewritten as a CAS loop.

For counter + ready, the compiler must also preserve the ordering required by the Go Memory Model. The amd64 implementation uses strongly ordered atomic instructions; other architectures can use different instruction and barrier combinations while preserving the same observable semantics.

2.4 Visibility and Ordering

The Go Memory Model defines synchronized-before for Atomic operations and requires all Atomic operations to behave as though they execute in some sequentially consistent order. The compiler and Runtime must preserve that observable behavior.

In the current non-race sync/atomic path, primitives enter internal/runtime/atomic; on amd64, Add and CAS then use locked RMW instructions.

Atomicity
  → Atomic Instruction
  → Go / amd64: LOCK XADDQ / LOCK CMPXCHGQ

Visibility
  → Cache Coherence
  → Typical x86-64 CPU: MESI-family (for example MESIF / MOESI)

Ordering
  → Compiler Ordering + Hardware Memory Ordering
  → Go Atomic semantics + amd64 Locked RMW / Load rules

See sync/atomic, internal/runtime/atomic, and atomic_amd64.s.

3. How Does CPython Implement Internal Atomics?

This section discusses only the Atomics used inside the current CPython Runtime. It does not turn them into a public application-level Python API.

3.1 Layers

CPython Runtime
Example: Reference Count / Runtime State / Lock State
Role: create internal shared-state update requirements
        │
        ▼
CPython Atomic Abstraction
Example: _Py_atomic_add_* / _Py_atomic_compare_exchange_* / _Py_atomic_load_*
Role: express Atomic operations and Memory Order
        │
        ▼
C Compiler Atomic Builtin
Example: GCC / Clang __atomic_*
Role: lower Atomic semantics to the target architecture
        │
        ▼
x86-64 Hardware
Example: LOCK XADD / LOCK CMPXCHG / Load
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

3.2 A Complete CPython Atomic Implementation Path

Common internal Atomic operations can be expanded as:

3.3 Atomicity

On Linux x86-64 with GCC or Clang, an atomic addition can typically compile to LOCK XADD, while CAS can typically compile to LOCK CMPXCHG:

_Py_atomic_add_*
        ↓
__atomic_fetch_add
        ↓
LOCK XADD

_Py_atomic_compare_exchange_*
        ↓
__atomic_compare_exchange_n
        ↓
LOCK CMPXCHG

The exact instruction depends on compiler version, target architecture, operand width, and requested memory order, so these are representative paths rather than universal instruction listings.

3.4 Visibility and Ordering

CPython's _Py_atomic_* layer supports more than one memory order. Current implementations include default sequentially consistent operations as well as variants such as _relaxed, _acquire, and _release.

Visibility and Ordering therefore depend on the Memory Order selected at the call site:

CPython Runtime
  → choose _Py_atomic_* Memory Order
  → GCC / Clang __atomic_* order
  → x86-64 Load / Store / Locked RMW and architecture ordering

Mapped back to the hardware model:

Atomicity
  → Atomic Instruction
  → x86-64: LOCK XADD / LOCK CMPXCHG

Visibility
  → Cache Coherence
  → Typical x86-64 CPU: MESI-family (for example MESIF / MOESI)

Ordering
  → Compiler Atomic Memory Order + Hardware Memory Ordering
  → __atomic_* order + x86-64 ordering rules

This is an implementation contract of the CPython Runtime, not a Python-language Memory Model.

See pyatomic.h, pyatomic_gcc.h, and a real use site in Python/lock.c.

4. The Common Pattern Behind All Three Implementations

Java, Go, and CPython expose different APIs, but their Atomic implementations converge on the same three problems:

1. How can one update become indivisible?
2. If a conditional update fails, who decides whether to retry?
3. How do ordinary reads and writes around the atomic operation gain the required visibility and ordering?

4.1 How Does a Single-variable Update Become Indivisible? — Atomic RMW

An ordinary read → modify → write contains several interleavable steps. Atomic Add, Exchange, CAS, and other RMW operations hand that shared-state update to a CPU Atomic Instruction:

Language / Runtime Atomic
        ↓
Compiler / Intrinsic
        ↓
Atomic RMW Instruction
        ↓
Cache Coherence

On x86-64, typical examples are LOCK XADD and LOCK CMPXCHG.

4.2 What Happens When a Conditional Update Fails? — CAS / Retry

CAS performs one atomic "compare and conditionally write":

expected still matches
  → update succeeds

expected is stale
  → update fails

CAS itself does not promise unlimited retries. compareAndSet() / CompareAndSwap() may simply return failure. updateAndGet(), some Runtime algorithms, or caller-written CAS loops are the layers that decide to reload, recompute, and retry.

CAS
= one atomic conditional update

CAS Loop
= CAS + algorithm-level retry after failure

4.3 Why Can Atomics Also Establish Visibility and Ordering?

Indivisible RMW solves only Atomicity.

Java VarHandle Memory Effects, the Go Memory Model's Atomic rules, and the Memory Order chosen by CPython _Py_atomic_* also constrain:

Compiler Reordering
      +
CPU Memory Ordering
      +
Cache Coherence

So the full path remains:

Language / Runtime defines Memory Semantics
        ↓
Compiler preserves ordering constraints
        ↓
CPU uses Atomic Instruction / Load / Store / Fence mechanisms
        ↓
Cache Coherence propagates shared state
        ↓
Program receives the corresponding Visibility and Ordering

This is also where Atomics and the previous Mutex article converge at the bottom: both depend on Atomic Instructions, Cache Coherence, and Memory Ordering. Mutexes additionally need waiting and wakeup paths under serious contention; the Atomic operation itself has no Park / Wakeup flow.

5. Next: volatile

Atomic operations address the atomicity of RMW updates such as counter++. If no atomic update is needed and the goal is only to use ready to publish an already-written counter, Java provides volatile. The next article explains how it provides visibility and ordering, while also explaining why Go and Python do not have a corresponding volatile keyword.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Concurrency Programming (5): Atomics — Atomicity, Visibility, and Ordering at the Language Level

ThinkerQAQ — Tue, 29 Sep 2026 05:58:37 +0000

0. Continue from the Previous Article
1. What Guarantees Do the Atomic Rules in Each Language Provide?
- 1.1 Java: AtomicInteger
- 1.2 Go: sync/atomic
- 1.3 CPython: No Symmetric Application-level Atomic API
2. Next: How Are Atomics Implemented?

0. Continue from the Previous Article

This article discusses Atomics directly from the rules defined by language memory models and public APIs.

1. What Guarantees Do the Atomic Rules in Each Language Provide?

1.1 Java: AtomicInteger

1.1.1 Atomicity

AtomicInteger.incrementAndGet() states:

“Atomically increments the current value, with memory effects as specified by VarHandle.getAndAdd.”

This means that incrementAndGet() is one atomic increment. Reading the old value, adding one, and writing the result back are not three independently interleavable steps; together they form one Atomic RMW operation.

AtomicInteger counter = new AtomicInteger(0);

// Thread A
counter.incrementAndGet();

// Thread B
counter.incrementAndGet();

The two updates apply to the same value in sequence:

Thread A                    Thread B

incrementAndGet()           incrementAndGet()
        │                           │
        ▼                           ▼
      0 → 1                       1 → 2

A lost update like the one possible with a plain counter++ therefore does not occur.

1.1.2 Visibility

incrementAndGet() gets its memory effects from VarHandle.getAndAdd. VarHandle.getAndAdd describes its write side as:

“with the memory semantics of setVolatile(Object...)”

AtomicInteger.get() uses VarHandle.getVolatile read semantics.

JLS §17.4.4 Synchronization Order says that a volatile write:

“synchronizes-with all subsequent reads of v by any thread”

So the volatile write performed by the Atomic update can establish a synchronizes-with relationship with a later volatile read in another thread.

AtomicInteger counter = new AtomicInteger(0);
boolean ready = false;

// Thread A
ready = true;                 // A1
counter.incrementAndGet();    // A2

// Thread B
if (counter.get() == 1) {     // B1
    System.out.println(ready); // B2
}

Assume B's counter.get() reads the 1 produced by A's update:

Thread A                              Thread B

A1  ready = true
        │
        │ program order
        ▼
A2  counter.incrementAndGet()
        │
        │ synchronizes-with
        │ JLS §17.4.4
        └──────────────────────────► B1  counter.get() == 1
                                        │
                                        │ program order
                                        ▼
                                    B2  read ready

A's earlier ready = true write is therefore visible to B's later read. That is the Visibility guarantee in this example.

1.1.3 Ordering

JLS §17.4.5 Happens-before Order states:

“If one action happens-before another, then the first is visible to and ordered before the second.”

So happens-before describes not only visibility but also the required ordering between the actions.

A1 ready = true
    ↓ program order
A2 counter.incrementAndGet()
    ↓ synchronizes-with
B1 counter.get() == 1
    ↓ program order
B2 read ready

Therefore:

A1 happens-before A2
A2 happens-before B1
B1 happens-before B2

        ↓ transitivity

A1 happens-before B2

Once B has observed A's update through counter.get(), it cannot still observe:

counter == 1
ready == false

That is the Ordering guarantee.

1.2 Go: sync/atomic

1.2.1 Atomicity

atomic.Int64.Add states:

“Add atomically adds delta to x and returns the new value.”

Add(1) therefore performs the read, increment, and write-back as one atomic update.

var counter atomic.Int64

// Goroutine A
counter.Add(1)

// Goroutine B
counter.Add(1)

The two updates do not lose one another:

Goroutine A                 Goroutine B

Add(1)                      Add(1)
  │                           │
  ▼                           ▼
0 → 1                       1 → 2

That is Atomicity.

1.2.2 Visibility

The Go Memory Model - Atomic Values states:

“If the effect of an atomic operation A is observed by atomic operation B, then A is synchronized before B.”

So if B's Atomic operation observes A's Atomic update, the two operations have a synchronized-before relationship.

var counter atomic.Int64
var ready bool

// Goroutine A
ready = true          // A1
counter.Add(1)        // A2

// Goroutine B
if counter.Load() == 1 { // B1
    fmt.Println(ready)    // B2
}

Assume B's counter.Load() observes the 1 produced by A's Add(1):

Goroutine A                           Goroutine B

A1  ready = true
        │
        │ sequenced-before
        ▼
A2  counter.Add(1)
        │
        │ synchronized-before
        │ Go Memory Model
        └──────────────────────────► B1  counter.Load() == 1
                                        │
                                        │ sequenced-before
                                        ▼
                                    B2  read ready

A's earlier ready = true write is therefore visible to B's later read. That is Visibility.

1.2.3 Ordering

The same Atomic Values section says that Atomic operations behave as though they were in:

“some sequentially consistent order”

The Go Memory Model defines happens-before from the transitive closure of sequenced-before and synchronized-before.

A1 ready = true
    ↓ sequenced-before
A2 counter.Add(1)
    ↓ synchronized-before
B1 counter.Load() == 1
    ↓ sequenced-before
B2 read ready

Therefore:

A1 happens-before A2
A2 happens-before B1
B1 happens-before B2

        ↓ transitivity

A1 happens-before B2

After B has observed counter == 1, it cannot treat A's earlier ready = true as if it had not happened. That is Ordering.

1.3 CPython: No Symmetric Application-level Atomic API

Python's standard library currently has no general integer Atomic API symmetric with Java's AtomicInteger or Go's atomic.Int64.

Application code therefore has no corresponding set of public Atomic rules to cite. An ordinary counter += 1 is not a language-defined Atomic API. The existence of the GIL in a particular CPython configuration, or the use of _Py_atomic_* inside the Runtime, does not turn it into a cross-implementation guarantee of Atomicity, Visibility, or Ordering for application code.

CPython itself needs Atomics for shared Runtime state. How _Py_atomic_* reaches the compiler and CPU is an implementation question for the next article, not part of Python's public application-level Atomic semantics.

2. Next: How Are Atomics Implemented?

This article stops at the language level: Java and Go define Atomicity, Visibility, and Ordering through public Atomic APIs and their memory models, while Python application code has no symmetric general-purpose Atomic API.

The next article turns those language-level rules into implementation-level problems by following Java AtomicInteger, Go sync/atomic, and CPython's internal _Py_atomic_* path from the Runtime down to the CPU.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Concurrency Programming (4): Mutex Implementation — From Runtime to CPU

ThinkerQAQ — Mon, 28 Sep 2026 11:20:57 +0000

0. What Does This Article Continue to Answer?
1. How Does HotSpot Implement synchronized?
- 1.1 Layers
- 1.2 The Complete Path of One synchronized Operation
- 1.3 Atomicity
- 1.4 Visibility and Ordering
2. How Does the Go Runtime Implement sync.Mutex?
- 2.1 Layers
- 2.2 The Complete Path of One sync.Mutex Operation
- 2.3 Atomicity
- 2.4 Visibility and Ordering
3. How Does CPython Implement threading.Lock?
- 3.1 Layers
- 3.2 The Complete Path of One threading.Lock Operation
- 3.3 Atomicity
- 3.4 Visibility and Ordering
4. The Common Pattern Across the Three Implementations
- 4.1 Who Gets In? — Atomically Modify Lock State
- 4.2 What Happens If You Don't Get the Lock? — Spin / Park / Wakeup
- 4.3 After Acquiring the Lock, Why Can You See the Data Left by the Previous Holder?
5. Next: Atomic

0. What Does This Article Continue to Answer?

The previous article started from the rules of the language memory model and looked at how a Mutex provides Atomicity, Visibility, and Ordering.

This article continues downward: how Java, Go, and CPython implement those guarantees at the Runtime and CPU layers.

1. How Does HotSpot Implement synchronized?

1.1 Layers

First, fix the implementation layers for Java:

Java Language
Example: synchronized
Role: declare a synchronized region
        │
        ▼
JVM Bytecode
Example: monitorenter / monitorexit (synchronized blocks)
Role: express entering / exiting a Monitor
        │
        ▼
JVM Implementation (HotSpot)
Example: Synchronization Runtime
Role: implement the Monitor semantics required by the JVM
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware foundation for Atomicity, Visibility, and Ordering

1.2 The Complete Path of One synchronized Operation

The sequence diagram above keeps only the main cross-layer path. The contention path inside the Runtime can be expanded further:

1.3 Atomicity

Atomicity corresponds to the two atomic competitions in the flowchart above:

Fast Path: CAS markWord lock bits [Acquire]
Inflated Monitor: CAS _owner [Acquire]

Both atomically modify lock state. When multiple threads compete at the same time, only one thread can successfully acquire the lock and enter the critical section.

1.4 Visibility and Ordering

Visibility and Ordering correspond to the lock boundaries in the flowchart:

Acquire lock: CAS ... [Acquire]
Release lock: release lock state [Release]

Release / Acquire connects the two critical sections so that writes from the previous lock holder can be observed by the next lock holder in the correct order.

Going further down, these guarantees are implemented through CPU Cache Coherence and memory-ordering constraints.

So Java maps back to the hardware model from the first article as follows:

Atomicity
  → Atomic Instruction
  → HotSpot / x86-64: LOCK CMPXCHG

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence
  → HotSpot / x86-64: LOCK ADDL $0, 0(%rsp)

LOCK ADDL $0, 0(%rsp) is the actual full-fence path used by HotSpot on Linux x86. This does not mean every synchronized operation executes an extra copy of that instruction; x86 ordering rules and LOCKed RMW operations also participate in establishing the required ordering.

For implementation details, see OpenJDK's markWord.hpp, objectMonitor.inline.hpp, graphKit.cpp, and orderAccess_linux_x86.hpp.

2. How Does the Go Runtime Implement sync.Mutex?

2.1 Layers

Go API
Example: sync.Mutex / Lock() / Unlock()
Role: declare a mutual-exclusion boundary
        │
        ▼
Go Implementation (sync / internal/sync)
Example: Mutex
Role: implement sync.Mutex semantics
        │
        ▼
Go Runtime / Scheduler
Example: Semaphore
Role: handle Goroutine waiting / wakeup
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware foundation for Atomicity, Visibility, and Ordering

2.2 The Complete Path of One sync.Mutex Operation

The sequence diagram above keeps only the main cross-layer path. The contention path inside sync.Mutex can be expanded further:

2.3 Atomicity

Atomicity corresponds to the atomic lock-state modifications in the flow above:

Acquire lock: CompareAndSwapInt32(&state, 0, mutexLocked)
Release lock: AddInt32(&state, -mutexLocked)

When multiple Goroutines compete at the same time, only one can successfully change state from unlocked to locked and enter the critical section.

On amd64, these two classes of atomic operation map to LOCK CMPXCHGL and LOCK XADDL.

2.4 Visibility and Ordering

Visibility and Ordering correspond to the synchronization boundary formed by Lock / Unlock. After one holder completes Unlock, a later Goroutine that successfully executes Lock can observe writes from the previous critical section in the required order.

Going further down, these guarantees are implemented through CPU Cache Coherence, x86 memory ordering, and the ordering constraints of the atomic instructions themselves.

So Go maps back to the hardware model from the first article as follows:

Atomicity
  → Atomic Instruction
  → Go / x86-64: LOCK CMPXCHGL / LOCK XADDL

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence / equivalent ordering constraint
  → Go / x86-64: LOCK CMPXCHGL / LOCK XADDL

This amd64 Mutex path does not need an additional MFENCE; the LOCKed RMW operations already provide the atomic update and ordering constraints required here.

For implementation details, see internal/sync/mutex.go, runtime/sema.go, and internal/runtime/atomic/atomic_amd64.s.

3. How Does CPython Implement threading.Lock?

This section discusses current CPython only.

3.1 Layers

Python API
Example: threading.Lock / acquire() / release()
Role: declare a mutual-exclusion boundary
        │
        ▼
CPython Binding (_thread)
Example: _thread.lock
Role: map the Python Lock API to CPython's lock implementation
        │
        ▼
CPython Implementation
Example: PyMutex
Role: implement Lock semantics
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware foundation for Atomicity, Visibility, and Ordering

3.2 The Complete Path of One threading.Lock Operation

The sequence diagram above keeps only the main cross-layer path. CPython's internal contention path can be expanded further:

3.3 Atomicity

Atomicity corresponds to the atomic state changes that PyMutex performs on _bits:

Acquire lock: _Py_atomic_compare_exchange_uint8(..., _Py_LOCKED)
Release lock (no waiter): _Py_atomic_compare_exchange_uint8(..., _Py_UNLOCKED)

When waiters exist, the release path enters _PyParkingLot_Unpark(), whose callback atomically updates _bits.

When multiple threads compete at the same time, only one can successfully set _Py_LOCKED and enter the critical section.

3.4 Visibility and Ordering

Visibility and Ordering correspond to the synchronization boundary formed by acquire() / release(). In current CPython, the default atomic compare-exchange and store operations use __ATOMIC_SEQ_CST in the GCC / Clang implementation.

Going further down, these guarantees are implemented through CPU Cache Coherence, x86 memory ordering, and those sequentially consistent atomic operations.

So current CPython maps back to the hardware model from the first article as follows:

Atomicity
  → Atomic Instruction
  → CPython / x86-64: LOCK CMPXCHGB

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence / equivalent ordering constraint
  → CPython / x86-64: SEQ_CST atomic
    (typically LOCK CMPXCHGB / memory XCHGB)

This likewise does not require every Lock operation to execute an additional MFENCE; on x86-64, LOCKed RMW and sequentially consistent atomic operations themselves provide the corresponding ordering constraints.

For implementation details, see Modules/_threadmodule.c, Python/lock.c, Python/parking_lot.c, and Include/cpython/pyatomic_gcc.h.

4. The Common Pattern Across the Three Implementations

After looking at Java, Go, and CPython, remove the implementation-specific details and focus only on the problems that a Mutex ultimately needs to solve.

4.1 Who Gets In? — Atomically Modify Lock State

Suppose a lock has only one internal state:

0 = unlocked
1 = locked

A and B must not both read 0 first and then both write 1. Otherwise, both would believe they acquired the lock.

Therefore, "check the lock state and change it to locked" must be an atomic operation, such as Compare-And-Swap:

Compare-And-Swap(lock_state, 0, 1)

When two execution units compete:

CPU A                         CPU B

CAS 0 -> 1                   CAS 0 -> 1
    │                             │
    ▼                             ▼
  success                       failed

Only one competitor can successfully modify the lock state. A Mutex then uses this small hardware atomic operation to protect an arbitrary critical section such as counter++.

4.2 What Happens If You Don't Get the Lock? — Spin / Park / Wakeup

After CAS fails, an execution unit cannot compete for the lock state forever.

A common path is:

try atomic lock acquisition
        │
        ├── success ──> enter critical section
        │
        └── failure
              │
              ├── Spin briefly
              │      │
              │      └── retry
              │
              └── Park / wait
                         │
                         └── wake after lock release

Spin is suitable when the wait is expected to be very short because it avoids immediately entering a blocking path. If contention persists, Park avoids continuously occupying the CPU.

4.3 After Acquiring the Lock, Why Can You See the Data Left by the Previous Holder?

The first two questions answer "who gets the lock?" and "what happens if you don't get it?" The third common question is: why can writes performed by the previous lock holder inside the critical section be seen by the next lock holder?

Across the three implementations, the key common model is that lock release and lock acquisition form a Release / Acquire synchronization boundary:

previous lock holder
critical-section writes
    │
    ▼
unlock [Release]
    │
    ▼
lock [Acquire]
    │
    ▼
next lock holder
critical-section reads

Release constrains memory operations before the lock is released, while Acquire constrains memory operations after the lock is acquired.

Therefore, when the next execution unit successfully acquires the same lock released by the previous execution unit, writes in the earlier critical section are ordered before reads in the later critical section. For a Mutex, this is the memory ordering that all three implementations need to establish.

The first three sections already showed how each Runtime and CPU implementation realizes this Release / Acquire boundary.

5. Next: Atomic

The cleverness of a Mutex is that it only needs to maintain a very small atomic state to establish a mutual-exclusion boundary around a critical section of arbitrary size.

Atomic narrows the protected scope further: instead of protecting an entire critical section, it directly guarantees that a single read, write, or update of a shared variable is indivisible.

The next article continues with how this smaller synchronization unit is implemented.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Concurrency Programming (3): Mutexes — Atomicity, Visibility, and Ordering at the Language Level

ThinkerQAQ — Sun, 27 Sep 2026 12:17:25 +0000

0. Continue from the Previous Article
1. What Guarantees Do the Lock Rules in Each Language Provide?
- 1.1 Java: synchronized and Monitors
- 1.2 Go: sync.Mutex
- 1.3 CPython: threading.Lock
2. Next: How Is a Mutex Implemented?

0. Continue from the Previous Article

This article discusses Mutexes directly from the rules defined at the language memory model layer.

1. What Guarantees Do the Lock Rules in Each Language Provide?

1.1 Java: synchronized and Monitors

Java's synchronized locks the monitor associated with an object. JLS §17.1 states:

“Only one thread at a time may hold a lock on a monitor.”

This rule provides the exclusion needed for atomicity. At most one thread can hold the monitor, so only that thread may execute the protected critical section while competitors wait.

JLS §17.4.5 also states:

“An unlock on a monitor happens-before every subsequent lock on that monitor.”

This rule provides both visibility and ordering.

For visibility, A's writes are before its unlock, while B's reads are after the subsequent lock. Through happens-before transitivity, the writes in A's critical section become visible to B.

For ordering, A's program order, the unlock → lock relationship, and B's program order form one happens-before chain. B therefore cannot observe a result such as ready = true, counter = 0 when that would violate the established ordering.

1.2 Go: sync.Mutex

Go's sync.Mutex is not owned by a particular goroutine and is not reentrant. sync.Mutex.Lock states:

“If the lock is already in use, the calling goroutine blocks until the mutex is available.”

This gives the exclusion needed for atomicity. While one goroutine holds the mutex, other goroutines attempting to acquire the same mutex wait rather than entering the critical section.

The Go Memory Model - Locks further states:

“For any sync.Mutex or sync.RWMutex variable l and n < m, call n of l.Unlock() is synchronized before call m of l.Lock() returns.”

This rule provides visibility and ordering.

For visibility, A's writes are sequenced-before Unlock; Unlock is synchronized-before B's Lock returns; B's reads are sequenced after that return. Together these relationships form happens-before, so A's writes are visible to B.

For ordering, the same happens-before chain connects A's writes, Unlock → Lock, and B's reads. B likewise cannot observe ready = true, counter = 0 when the synchronization relationship requires the earlier counter write to be visible.

1.3 CPython: threading.Lock

Python's official threading.Lock documentation states:

“A primitive lock is a synchronization primitive that is not owned by a particular thread when locked.”

“All methods are executed atomically.”

The locking semantics provide mutual exclusion: when the lock is already acquired, another thread's acquire() waits until the lock becomes available, so the same lock serializes entry into the protected critical section.

Visibility and ordering cannot be derived from a formal language-level memory-model rule in the same way as in Java or Go. Python does not define a general release → acquire happens-before relationship comparable to those specifications; the documentation defines Lock as a synchronization primitive.

Python programs should therefore use the same Lock to synchronize shared state rather than depending on the GIL or on whether a particular bytecode sequence happens to be interruptible in a particular implementation. How CPython implements the cross-critical-section synchronization boundary is an implementation question for the next article.

2. Next: How Is a Mutex Implemented?

This article stops at the language level: it looks at what guarantees a Mutex provides to programmers from the perspectives of Atomicity, Visibility, and Ordering.

The next article turns those language-level guarantees into implementation-level problems, following Java synchronized, Go sync.Mutex, and CPython threading.Lock from the Runtime down to the CPU.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Concurrency Programming (2): Language Memory Models — Rules Programmers Can Rely On

ThinkerQAQ — Wed, 23 Sep 2026 14:01:10 +0000

0. Continue from the Previous Article
1. Why Does the Language Need a Memory Model Layer?
2. How Should Programmers Understand Memory Model Rules?
- 2.1 Java: Java Memory Model
- 2.2 Go Memory Model
- 2.3 CPython Concurrency Semantics
- 2.3.1 GIL Mode
- 2.3.2 Free-threaded Mode
- 2.3.3 So What Should Python Programs Rely On?
3. Next: Mutexes

0. Continue from the Previous Article

This article shifts the perspective from hardware to the language memory model layer.

1. Why Does the Language Need a Memory Model Layer?

Source code passes through a compiler and Runtime before it finally executes on a CPU. Compilers optimize code, and processors such as x86-64 and ARM64 do not allow exactly the same memory-access orderings.

If programmers had to study every compiler, Runtime, and CPU separately, portable concurrent programming would be difficult to sustain.

We therefore also need a language memory model layer. It defines the rules programmers need to understand, while the underlying Compiler / Runtime implements those rules differently for each platform:

Java                                  Go                                  Python
 │                                     │                                     │
 ▼                                     ▼                                     ▼
synchronized / volatile / AtomicInteger
                                       sync.Mutex / sync/atomic / channel
                                                                             threading.Lock / queue.Queue
 │                                     │                                     │
 ▼                                     ▼                                     ▼
Java Memory Model                Go Memory Model              CPython Concurrency Semantics
 │                                     │                                     │
 └─────────────────────────────────────┼─────────────────────────────────────┘
                                       │
                         Compiler / Runtime implementation
                         shields hardware-platform differences
                                       │
                              ┌────────┴────────┐
                              │                 │
                              ▼                 ▼
                           x86-64             ARM64

2. How Should Programmers Understand Memory Model Rules?

Through these three dimensions:

Atomicity
Visibility
Ordering

In theory — only in theory — each concurrency utility has corresponding rules at the memory model layer that describe which of these properties it provides.

This article will not go deeper yet. Instead, we will use the previous examples:

counter++

and:

counter = 1
ready = true

Use the rules at the language memory model layer to establish:

Question	Specifically
Atomicity	Why is `counter++` not atomic?
Visibility	After A writes `counter = 1`, why is B not guaranteed to observe `counter = 1`?
Ordering	When B observes `ready = true`, why is B not guaranteed to observe the earlier `counter = 1`?

2.1 Java: Java Memory Model

Atomicity

Start with:

counter++;

Unfortunately, the JMM does not have a rule that directly states that counter++ is not atomic, so we will explain it here.

Semantically, counter++ is a compound operation. We can think of it as:

read counter
     ↓
counter + 1
     ↓
write counter

If two threads execute it concurrently:

Thread A                    Thread B

counter++;                  counter++;

these operations may interleave like this:

Thread A                    Thread B

read counter = 0            read counter = 0

counter + 1                 counter + 1

write counter = 1           write counter = 1

Both threads execute counter++, yet the final result may still be:

expected: counter = 2
actual:   counter = 1

Therefore, ordinary counter++ cannot be treated as atomic.

Visibility

Now consider:

int counter = 0;

// Thread A
counter = 1;

// Thread B
System.out.println(counter);

That is:

Thread A                    Thread B

counter = 1                 read counter

JLS §17.4.5 gives a very direct description of happens-before:

“If one action happens-before another, then the first is visible to and ordered before the second.”

In other words, if we can establish:

Thread A                    Thread B

counter = 1                 read counter
     │                           ▲
     │                           │
     └──── happens-before ───────┘

then Thread A's write is guaranteed to be visible to Thread B's read.

But in the current code:

Thread A                    Thread B

counter = 1                 read counter
     │                           ▲
     │                           │
     └── no happens-before ──────┘

no such cross-thread happens-before relationship has been established.

Therefore, Thread B is not guaranteed to observe 1.

Ordering

Return to the second example:

// Thread A
counter = 1;
ready = true;

// Thread B
if (ready) {
    System.out.println(counter);
}

That is:

Thread A                    Thread B

counter = 1
ready = true                read ready == true
                                  │
                                  ▼
                             read counter

Within Thread A, counter = 1 comes before ready = true; within Thread B, reading ready comes before reading counter. But the two threads are still missing a happens-before relationship from ready = true to read ready == true.

So the current code still does not establish a complete happens-before chain between Thread A and Thread B:

Thread A                    Thread B

counter = 1
ready = true                read ready == true
     │                            │
     │                            ▼
     │                       read counter
     │
     └──── no cross-thread happens-before

Therefore, even if B observes:

ready == true

B is still not guaranteed to observe:

counter == 1

2.2 Go Memory Model

Atomicity

The Go Memory Model provides a direct rule for this case:

If a write to a memory location occurs concurrently with another read or write, it constitutes a Data Race unless all of those accesses are atomic accesses provided by sync/atomic.

But:

counter++

is only an ordinary increment operation, not an atomic access provided by sync/atomic.

Therefore, if two Goroutines concurrently execute:

counter++

they perform concurrent reads and writes on the same counter, which constitutes a Data Race.

So ordinary counter++ cannot be treated as an atomic concurrent update.

Visibility

Continue with:

var counter int

// Goroutine A
counter = 1

// Goroutine B
fmt.Println(counter)

Split into two columns:

Goroutine A                 Goroutine B

counter = 1                 read counter

This is exactly the kind of question the Go Memory Model starts from:

Under what conditions can a read in one Goroutine be guaranteed to observe a write to the same variable in another Goroutine?

In the formal rules, for an ordinary read to reliably observe a particular write, that write must be visible to the read; one of the conditions is:

“w happens before r.”

Applied to our example, if we can establish:

Goroutine A                 Goroutine B

counter = 1                 read counter
     │                           ▲
     │                           │
     └──── happens-before ───────┘

then the later read can reliably observe that write.

But in the current code:

Goroutine A                 Goroutine B

counter = 1                 read counter
     │                           ▲
     │                           │
     └── no happens-before ──────┘

no such relationship has been established, and the program contains a Data Race.

Therefore, the second Goroutine cannot rely on always observing:

counter == 1

Ordering

Now consider:

// Goroutine A
counter = 1
ready = true

// Goroutine B
if ready {
    fmt.Println(counter)
}

Split into two columns:

Goroutine A                 Goroutine B

counter = 1
ready = true                read ready == true
                                  │
                                  ▼
                             read counter

This is essentially the same pattern used by the Go Memory Model under Incorrect synchronization: one Goroutine writes data and then a flag; another Goroutine observes the flag and then reads the earlier data.

The Go specification states:

“Even if this occurs, it does not imply that reads happening after r will observe writes that happened before w.”

Applied to our example:

Goroutine A                 Goroutine B

counter = 1
ready = true                read ready == true
                                  │
                                  ▼
                             read counter

        no happens-before guarantee

So:

observing ready == true

does not imply:

counter must be observed as 1

This is very similar to the Java example.

Both Java and Go use happens-before to describe important Visibility and Ordering guarantees.

2.3 CPython Concurrency Semantics

Python differs from Java and Go in one important respect:

Python does not define one unified concurrency memory model for all interpreter implementations at the same level as the JMM or Go Memory Model.

So this article focuses on the most widely used implementation: CPython.

We will continue to use the same two examples:

counter += 1

and:

counter = 1
ready = True

and examine Atomicity, Visibility, and Ordering.

2.3.1 GIL Mode

Before returning to counter, first clarify one question:

What does the GIL actually protect?

The GIL is first and foremost a mechanism used by the CPython Runtime to protect Python objects and interpreter-internal state.

The official Python C API documentation states the requirement directly:

“only a thread that holds the GIL may operate on Python objects or invoke Python’s C API.”

In other words, in default GIL-enabled CPython, a thread must hold the GIL before it can operate on Python objects or call the Python C API.

The documentation then uses Reference Count to explain why such a lock is needed: if two threads increment the same object's reference count at the same time, the result may be incremented only once instead of twice.

Suppose a Python object currently has:

refcount = 10

If two threads could modify that reference count concurrently:

Thread A                    Thread B

read refcount = 10          read refcount = 10

refcount + 1                refcount + 1

write refcount = 11         write refcount = 11

Both threads increment the reference count once, yet the final result may be:

expected: refcount = 12
actual:   refcount = 11

That would break CPython's management of Python object lifetimes.

So the GIL primarily protects:

             GIL
              │
              ▼
      CPython Runtime
              │
      ┌───────┴────────┐
      ▼                ▼
Python Objects     Runtime State
      │
      ├── Reference Count
      └── Object Internals

There are two different layers here:

CPython Runtime layer: how to safely access and maintain Python objects under concurrency
Application layer: how to safely access and modify shared state under concurrency

The GIL primarily addresses the former. It does not make application code automatically thread-safe.

Atomicity

Now return to:

counter += 1

The official Python FAQ directly addresses this kind of operation.

When discussing which operations are atomic in GIL-enabled CPython, it explicitly lists:

i = i + 1

as a non-atomic operation:

“These aren’t: i = i+1”

Therefore, even with the GIL, we cannot treat:

counter += 1

as an atomic concurrent update as a whole.

We need to distinguish:

GIL
 │
 └── protection for CPython Runtime / Python Objects

counter += 1
 │
 └── a compound state update defined by the application

Visibility

Continue with the same example:

counter = 0

# Thread A
counter = 1

# Thread B
print(counter)

Split into two columns:

Thread A                    Thread B

counter = 1                 read counter

Here the distinction between CPython and Java / Go appears.

With Java and Go, we can continue to ask:

write
 │
 │ happens-before ?
 ▼
read

because those languages define formal Memory Models.

Python, however, does not define a corresponding happens-before model that applies uniformly to all Python implementations.

So we cannot draw the CPython code above as:

Thread A                    Thread B

counter = 1                 read counter
     │                           ▲
     │                           │
     └──── Python happens-before ┘

and then derive a conclusion from some Python Language Memory Model rule, because there is no corresponding unified model of that kind.

Conversely, because no separate rule specifies such a guarantee, we should treat this behavior as unspecified. If it is unspecified, then it does not provide a Visibility guarantee.

Ordering

Finally, consider:

# Thread A
counter = 1
ready = True

# Thread B
if ready:
    print(counter)

Again, split into two columns:

Thread A                    Thread B

counter = 1
ready = True                read ready == True
                                  │
                                  ▼
                             read counter

Likewise, there is no corresponding rule that specifies such a guarantee here, so we should treat this behavior as unspecified. If it is unspecified, then it does not provide an Ordering guarantee.

2.3.2 Free-threaded Mode

Starting with Python 3.13, CPython provides free-threaded builds in which the GIL can be disabled.

In GIL-enabled CPython:

Thread A ──┐
           │
           ├── GIL ───> Python Code
           │
Thread B ──┘

In free-threaded CPython:

Thread A ─────────────> CPU Core 0

Thread B ─────────────> CPU Core 1

Multiple threads can execute Python code in parallel.

But removing the GIL does not mean CPython no longer needs to protect its objects and Runtime state.

Python's official free-threading documentation explicitly states that built-in types such as dict, list, and set use internal locks to protect against concurrent modification:

“Built-in types like dict, list, and set use internal locks”

These internal locks are an implementation mechanism of free-threaded CPython. They should not be interpreted as Python the language defining a unified Memory Model.

So the change is from:

GIL-enabled

one global GIL
      │
      ▼
protect CPython Runtime

to:

Free-threaded

finer-grained internal synchronization
      │
      ▼
protect CPython Runtime

For application code, the same three questions still exist.

Different paths, same destination. Removing the GIL does not mean CPython no longer needs synchronization; it moves from one global lock toward finer-grained locks, atomic operations, and Runtime coordination. In this respect, its implementation approach increasingly resembles Java and Go.

2.3.3 So What Should Python Programs Rely On?

In summary, Java, Go, and CPython differ at this layer in an important way:

Java
  │
  ▼
Java Memory Model
  │
  └── happens-before

Go
  │
  ▼
Go Memory Model
  │
  └── happens-before

Python
  │
  ▼
Concurrency semantics of different Interpreters
  │
  └── CPython
        ├── GIL-enabled
        └── Free-threaded

The summary is simple: Python does not have one unified Memory Model, at the same level as the JMM or Go Memory Model, that applies to every interpreter implementation. Therefore, when discussing concurrency safety in Python, first identify which Interpreter is being used, and rely on explicit synchronization tools and the concurrency guarantees they provide.

3. Next: Mutexes

The article before last discussed the hardware layer; this article explains the rules at the language memory model layer.

The next article begins with the first concrete synchronization tool, Mutex, and follows it from the language memory model layer all the way down to the hardware layer.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

What I Think About When I'm Anxious: Life and Planning

ThinkerQAQ — Sat, 19 Sep 2026 03:06:39 +0000

1. Introduction

This article records some of my thoughts about life. It is not a rigorous theory of life, but simply an attempt to organize my own understanding.

This article was written in 2024 and reflects the stage of life I was in and how I thought at the time. When migrating it to this site, I made only minor adjustments to some wording.

2. Why Think About Life?

2.1. Instead of Being Consumed by Anxiety, Think About Meaningful Questions

When I am under a lot of pressure, I tend to become anxious. Anxiety then makes me repeatedly think about many things that have not happened yet. Rather than being overwhelmed by those thoughts, I prefer to direct my attention toward questions that genuinely matter and write down what I think about them.

2.2. Approaching Thirty, It Is Time to Think About What It Means to "Stand Firm"

As I approached the age traditionally associated with "standing firm," I began to seriously think about what I actually wanted to stand for. Confucius once said:

At fifteen, I set my heart upon learning; at thirty, I stood firm; at forty, I was free from doubts; at fifty, I understood the decree of Heaven; at sixty, my ear was attuned; at seventy, I could follow my heart's desires without overstepping the bounds.

This was Confucius's summary of the stages of his own life. Here I am mainly interested in the phrase "at thirty, I stood firm": what exactly does "standing firm" mean?

Different commentaries offer different interpretations. In his discussion in The Analects: Translation and Annotation, Yang Bojun interprets it as understanding ritual propriety and becoming confident in one's words and actions. His main evidence includes "Be inspired by the Songs, take your stand through ritual, and achieve completion through music" from the Analects, Tai Bo, and "If you do not study ritual, you have no way to establish yourself" from the Analects, Ji Shi. Under this interpretation, the "standing" Confucius described is closely related to ritual propriety.

However, Confucius lived in the late Spring and Autumn period, when the feudal states were in conflict and society was unstable. His political ideal of restoring the rites of Zhou was difficult to realize in practice. Confucius traveled among different states in the hope of putting his political ideas into practice, but repeatedly encountered setbacks. The Records of the Grand Historian, in the "Hereditary House of Confucius," records an episode in which Duke Jing of Qi asked Confucius about government and considered employing him, but later abandoned the idea because of practical resistance:

Duke Jing asked Confucius about government. Confucius replied, "Let the ruler be a ruler, the minister a minister, the father a father, and the son a son." Duke Jing said, "Excellent! Indeed, if the ruler is not a ruler, the minister not a minister, the father not a father, and the son not a son, then even if there is grain, how could I get to eat it?" On another day he again asked Confucius about government, and Confucius replied, "Government lies in economizing expenditure." Duke Jing was pleased and intended to grant Confucius the fields of Nixi. Yan Ying remonstrated: "The Ru are eloquent and cannot be constrained by ordinary rules; they are arrogant and self-righteous and cannot be treated as subordinates; they exalt mourning, indulge grief, and impoverish families through lavish burials, so their customs cannot be adopted; they travel about persuading rulers and seeking support, so they cannot be used to govern a state. Since the great sages passed away and the Zhou royal house declined, rites and music have long been in disrepair. Now Confucius elaborates appearances and ornaments, complicated ceremonies of ascending and descending, and detailed rules of movement. Even generations would not exhaust his learning, nor could one master his rites within a lifetime. If you wish to use him to change the customs of Qi, this is not the way to put the common people first." After that, Duke Jing continued to receive Confucius respectfully but no longer asked him about ritual. On another occasion, Duke Jing detained Confucius and said, "I cannot support you as the Ji family does." He treated him at a level between the Ji and Meng families. Officials of Qi wanted to harm Confucius, and Confucius heard of it. Duke Jing said, "I am old; I can no longer employ you." Confucius therefore left and returned to Lu.

Even so, Confucius continued to pursue his political ideals throughout his life. In my view, rather than interpreting "standing firm at thirty" as reaching a standardized social milestone, I prefer to understand it as gradually finding a direction that one is willing to invest in for the long term and beginning to take responsibility for that direction.

2.3. From Technical Planning to Life Planning

Whenever I was asked about career planning in the past, I often gave a formulaic answer without much thought: reach a certain position after a certain number of years. At work, when discussing technical planning, it was also easy to reduce planning to completing a certain project by a certain date. In fact, I had never seriously asked why those goals mattered.

Planning should not be only a timeline. If a technical system can be planned by considering its current state, goals, problems, and actions, can a life be examined seriously in a similar way? That became another reason for me to continue thinking about this topic.

2.4. Summary

The rest of this article mainly discusses three questions:

What is life?
What factors influence life?
How should we plan a life?

3. What Is Life?

Taken literally, a life is a person's lifetime. We can begin by discussing two parts:

The person
The lifetime

3.1. The Person

From an evolutionary perspective, humans and modern chimpanzees share a common ancestor and later evolved along different branches. Humans themselves are animals and have the same basic biological needs for eating, sleeping, and reproduction.

So compared with other animals, what characteristics stand out in humans?

In the Chinese intellectual tradition, Mencius famously discussed the distinction between humans and animals. In Mencius, Li Lou II, he wrote:

The difference between humans and beasts is slight; ordinary people cast it away, while exemplary people preserve it.

Yang Bojun explains in his Mencius: Translation and Annotation that the difference between humans and animals is not great; ordinary people may discard it, while exemplary people preserve it. Following this line of thought, moral awareness is one important distinction between humans and other animals.

In the Western intellectual tradition, Marx, in Capital, refers to Franklin's definition of humans as tool-making animals. The related discussion is commonly summarized as the idea that humans can make and use tools in labor and thereby transform nature.

These judgments each have their own historical context and emphasize different aspects of humanity. Moral awareness, tool making, labor, language, and abstract thought all depend on human cognitive abilities. Rather than searching for one simple "essential difference," I am more interested in another fact: the human brain is plastic, and learning and experience continually change neural connections and patterns of activity. This means that throughout life, people can continue to learn, revise their understanding, and change their actions. As for whether I learned this in middle school or high school, those ancient neural connections have become a little difficult to recall.

3.2. A Lifetime

As long as a person is alive, they continue to engage in different activities. Connect what a person does at different times and places, and those activities form the trajectory of their life and, to some extent, reflect their value choices.

Take Su Dongpo as an example. Exploring Su Shi's Life Trajectory Through Time and Space visualizes the geographic scope of his activities throughout his life.

Visualization of Su Shi's life trajectory. Source: StoryMaps.

He left behind a rich body of work in poetry, calligraphy, painting, and food culture. He also experienced repeated rises and falls in fortune, yet maintained a relatively open-minded attitude toward life. As he wrote in "Calming the Waves":

Do not mind the sound of rain beating through the forest; why not chant and walk on at ease?

Whatever a person's achievements, their time is always limited. The activities to which a person devotes that time, and how they distribute it among those activities, ultimately make up the life they experience.

3.3. Summary

For now, I would summarize my understanding of life this way: human time is limited, but people can continually adjust their understanding through learning and reflection, and based on that understanding, choose how to use their time. Those choices cannot completely determine a life, but they can shape it to some extent.

I say only "to some extent" because life is also affected by many factors that individuals cannot decide for themselves.

4. What Factors Influence Life?

4.1. Starting with "Fate"

My parents told me that when I was born, my family chose my name partly according to the traditional Five Elements. As the saying goes, you make up for whatever is lacking, so my name ended up containing a character associated with "earth." I do not believe that traditional fortune-telling can determine a person's life, but this small story raises a question worth taking seriously: which parts of life come from inborn conditions, which come from the external environment, and which can be changed through our own choices?

4.2. Genes and Environment

A person develops from a fertilized egg. The chromosomes in that fertilized egg carry genetic information, and genes in DNA are basic units of heredity.

Genes influence many human traits, but complex characteristics such as height, appearance, personality, and health are usually not determined directly by a single gene. They are often shaped by the combined effects of many genes, the environment in which a person grows up, lifestyle, and chance events. Even identical twins with highly similar genetic information can develop different personalities, habits, and values when raised in different environments.

Returning to the subject of life: inborn conditions cannot be chosen by the individual, and accidents and external circumstances cannot be fully controlled. But there is still some room to adjust the learning environment, lifestyle, information one is exposed to, and concrete actions one takes. These adjustable factors influence a person's understanding and choices, which in turn influence the trajectory of their life.

4.3. Summary

Life is influenced by both inborn conditions and the external environment, while also containing parts that individuals can choose and adjust. Together, these factors create the different lives that people experience.

5. How to Plan a Life

Life planning is an enormous topic. Here I only want to briefly summarize how I think about it.

First, there is little value in spending too much energy on inborn conditions that cannot be changed or accidents that cannot be predicted. Instead, identify the parts that you can influence. Second, improve your understanding through learning and practice, identify the categories of activity that genuinely matter at your current stage of life, then allocate time, take action, and review the results regularly based on your current goals.

For me, the activities worth paying attention to over the long term mainly include:

Learning: continuously improve my understanding and abilities
Health: maintain the foundation that supports other activities
Love: learn to build and maintain important relationships
Work: it occupies a substantial part of life and allows people to sustain themselves, take responsibility, and create value
Recreation: preserve room for rest, interests, and a sense of ease in life

This list is not a fixed answer. As life stages and real-world conditions change, the priorities among these activities also need to be reassessed. The purpose of planning is not to control every outcome, but to use limited time more consciously under uncertainty.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Why count++ Breaks Under Concurrency: Atomicity, Visibility & Ordering — Concurrency Programming (1)

ThinkerQAQ — Fri, 18 Sep 2026 06:37:56 +0000

1. Start with the von Neumann Architecture
2. What Does count++ Become?
3. Why Do We Need Caches?
4. Concurrency on One Core: Why Can count++ Lose an Update?
5. From One Core to Multiple Cores: Visibility
6. What About the Order of Multiple Memory Operations?
- 6.1 Store Buffer
- 6.2 Out-of-Order Execution
- 6.3 Store Buffer vs Out-of-Order Execution
7. We Have Actually Encountered Three Problems
8. How Does Hardware Address Them?
- 8.1 Atomicity: Atomic Instructions
- 8.2 Visibility: Cache Coherence
- 8.3 Memory Ordering: Fence
9. Summary

This article continues to use count, but focuses only on the hardware layer:

What concurrency problems do the CPU and memory system introduce, and what capabilities does the hardware provide to address them?

1. Start with the von Neumann Architecture

The von Neumann model treats programs as data: instructions and data are stored in the same general way. At a high level, a computer can be divided into a control unit, arithmetic unit, memory, input devices, and output devices.

The arithmetic and control units form the CPU, which also contains registers. The Program Counter (PC) stores the address of the next instruction; the Instruction Register (IR) stores the instruction currently being processed; general-purpose registers such as R1 hold data and intermediate results.

┌──────────────────┐                    ┌──────────────────┐
│   Input Device   │                    │  Output Device   │
└─────────┬────────┘                    └─────────▲────────┘
          │                                       │
          └───────────────────┬───────────────────┘
                              │
                          System Bus
                              │
             ┌────────────────┴─────────────────┐
             │                                  │
             ▼                                  ▼
┌────────────────────────┐         ┌────────────────────────┐
│          CPU           │         │         Memory         │
│                        │         │                        │
│      Control Unit      │         │  Instructions + Data   │
│     Arithmetic Unit    │         │                        │
│       Registers        │         │                        │
│       PC / IR / R1     │         │                        │
└────────────────────────┘         └────────────────────────┘

The CPU execution loop can be simplified to:

PC provides instruction address
        ↓
Fetch → Decode → Execute
        ↓
PC advances to the next instruction and repeats

The CPU repeatedly performs Fetch → Decode → Execute. A source-level expression such as count++ must eventually be converted into machine instructions that follow this process.

2. What Does `count++` Become?

For discussion, we can simplify count++ into three machine-level operations:

LOAD  R1, [count]    // load count into R1
ADD   R1, 1          // increment R1 inside the CPU
STORE [count], R1    // write the result back to count

Both LOAD and STORE access data. That leads naturally to the next question:

What would happen if every LOAD and STORE had to wait directly for main memory?

3. Why Do We Need Caches?

A CPU executes far faster than main memory can be accessed. If every LOAD had to wait for main memory to return data and every STORE had to wait until memory completed the write, the CPU would spend a large amount of time idle.

Modern processors therefore place smaller, faster cache levels between a CPU core and main memory. A simplified hierarchy looks like this; exact levels and which caches are shared depend on the processor design:

CPU
 │
 ▼
L1 Cache
 │
 ▼
L2 Cache
 │
 ▼
L3 Cache
 │
 ▼
Memory

When the CPU accesses count, it normally loads the cache line containing it into a cache. Simplifying again:

CPU
 │
 ▼
Cache: count = 0
 │
 ▼
Memory: count = 0

Later accesses to count may hit in the cache instead of going to main memory every time.

So:

Caches address the speed gap between the CPU and main memory.

That is a performance optimization. It does not make:

count++

a safe concurrent operation automatically.

Start with a single core.

4. Concurrency on One Core: Why Can `count++` Lose an Update?

Suppose the machine has only one CPU core:

Thread A ──┐
           │
           ├──> Core 0 ──> Cache ──> Memory
           │
Thread B ──┘

The two threads cannot literally execute at the same instant on that core, but the scheduler can interleave them.

Start with:

count = 0

Then the following schedule is possible:

Thread A                              Thread B
   │                                    │
   ├─ LOAD count -> 0                   │
   │                                    │
   ├─────── context switch ────────────>│
   │                                    ├─ LOAD  count -> 0
   │                                    ├─ ADD   1
   │                                    ├─ STORE count -> 1
   │                                    │
   │<────── context switch ─────────────┤
   ├─ ADD   1                           │
   ├─ STORE count -> 1                  │

During a context switch, the operating system preserves Thread A's execution context, including intermediate state. When A resumes, it can continue computing from the 0 it read earlier.

The final result is:

count = 1

The reason is that:

LOAD
ADD
STORE

are not one indivisible operation.

If another execution unit can run between those steps, a lost update becomes possible.

This gives us the first problem:

Problem 1: How can a compound operation execute indivisibly?

That is the problem of:

Atomicity

We will not answer it yet. First, move from one core to multiple cores.

5. From One Core to Multiple Cores: Visibility

With multiple CPU cores, two threads may truly execute in parallel:

Thread A                       Thread B
   │                              │
   ▼                              ▼
Core A                         Core B
   │                              │
   ▼                              ▼
Cache A                        Cache B
   │                              │
   └──────────────┬───────────────┘
                  ▼
                Memory

The original count++ race still exists, but multiple cores introduce another issue.

Suppose:

count = 0

Both Core A and Core B have read count. The same data may now exist in separate caches:

Core A Cache                    Core B Cache

count = 0                       count = 0

               Memory
              count = 0

Now Core A changes count:

Core A Cache                    Core B Cache

count = 1                       count = 0

The question becomes:

May Core B keep using its old copy?

Or more generally:

Problem 2: After one core writes data, when can other cores observe the new value?

That is:

Visibility

Again, hold the answer for a moment and move to the third class of problem.

6. What About the Order of Multiple Memory Operations?

Continue with a publication-style example:

count = 0
ready = false

Thread A                         Thread B
   │                                │
   ├─ count = 1                     ├─ read ready
   └─ ready = true                  └─ if true, read count

A programmer naturally wants the following guarantee:

If Thread B has already observed ready = true, it should also observe the earlier write count = 1.

Ideally, B should only see one of these outcomes:

Outcome 1:

Thread A                         Thread B

                                 read ready -> false
                                 do not enter if

Outcome 2:

Thread A                         Thread B

count = 1
ready = true                     read ready -> true
                                 enter if
                                 read count -> 1
                                 print 1

We do not want:

Thread A                         Thread B

count = 1
(write not yet observed by B)
ready = true                     read ready -> true
                                 enter if
                                 read count -> 0
                                 print 0

So the question is:

Once B has read ready = true, how do we prevent it from still reading the old count = 0?

This is no longer merely a question about multiple cached copies of the same location. We now have two different memory locations:

count
ready

and a relationship between two writes:

count = 1
ready = true

This gives us the third problem:

Problem 3: In what order may memory operations become observable to other cores?

That is:

Memory Ordering

Why does hardware make this complicated? Primarily because modern CPUs introduce aggressive performance optimizations.

6.1 Store Buffer

Continue with count++. Suppose count = 0 and Core A executes:

LOAD  count -> 0
ADD   1     -> 1
STORE [count], 1

The STORE of count = 1 may first enter a store buffer.

First, place the Store Buffer back into a slightly more complete conceptual write path:

CPU (Core)
    │
    ▼
Store Buffer
    │
    ▼
Cache
    │
    ▼
Memory

Note: the Store Buffer and Cache are not the same thing. The Store Buffer is a temporary structure on a core's write path; the Cache stores cached data. Real processors have more complicated write paths, cache hierarchies, and coherence protocols. This diagram is only meant to distinguish the two layers.

To highlight the point that “the write has already been executed by the current Core, but another Core has not observed it yet,” the following diagram omits Cache / Coherence details and continues with a simplified view:

CPU (Core A)
executes STORE [count], 1
        │
        ▼
Store Buffer
holds count = 1 temporarily
        │
        ▼
Memory

Core A can continue executing before the buffered write becomes visible to other cores:

initial: count = 0

Core A                               Core B
   │                                    │
   ├─ LOAD  count -> 0                  │
   ├─ ADD   1     -> 1                  │
   ├─ STORE count -> 1                  │
   │  enters Store Buffer               │
   ├─ continue with later instructions  │
   │                                    ├─ LOAD count -> 0
   │                                    │  still sees old value
   └─ count = 1 becomes visible         │

6.2 Out-of-Order Execution

Modern CPUs also rearrange internal execution, where dependencies allow it, to keep execution units busy.

As long as this does not change the result observed by the current thread, such optimization is valid from that thread's perspective. The difficulty appears when multiple cores interact.

Return to the publication example:

Thread A / Core A                 Thread B / Core B

count = 1                        if ready {
ready = true                         print(count)
                                 }

The source code writes count before ready. But without additional ordering constraints, on hardware whose memory model permits it, another core may observe:

initial: count = 0, ready = false

Thread A / Core A                 Thread B / Core B
   │                                  │
   ├─ STORE count = 1                 │
   │  not yet observed by Core B      │
   ├─ STORE ready = true              │
   │                                  ├─ LOAD ready -> true
   │                                  └─ LOAD count -> 0

In other words, A issued the count write first in source order, yet B observed ready = true first and then still read the old count = 0.

The underlying question is:

In what order is another core allowed to observe these memory operations?

6.3 Store Buffer vs Out-of-Order Execution

These two mechanisms are easy to confuse, so compare the questions they answer directly:

Concept	What question does it answer?
Store Buffer	After a `STORE` has already been executed by the current Core, why might another Core still not see it yet?
Out-of-Order Execution	Why might an independent instruction that appears later in source code execute earlier inside the Core?

Therefore, Store Buffer and Out-of-Order Execution are not the same mechanism: the former concerns write propagation and visibility timing, while the latter concerns instruction scheduling inside the Core. Both can affect the memory order eventually observed by other Cores.

7. We Have Actually Encountered Three Problems

The earlier examples can be summarized as three categories:

Problem	Symptom	What hardware must define
Atomicity	A and B both execute `count++`, both can read `0`, and both finally write `1`	Which operations are atomic?
Visibility	Core A has changed `count` to `1`, while Core B may still read a cached `0`	When does a write by one core become observable to another?
Ordering	Core A writes `count = 1` and then `ready = true`, while Core B may observe `ready = true` and still read `count = 0`	Which ordering relationships are guaranteed between memory operations?

The first problem can occur both with single-core context switches and with true multicore parallelism. The key issue is not the number of CPU cores, but whether the steps of a Read-Modify-Write sequence can be interleaved with another execution unit.

Now we can ask what primitives the hardware provides:

What mechanisms does modern hardware provide for these three problems?

8. How Does Hardware Address Them?

8.1 Atomicity: Atomic Instructions

A normal LOAD + ADD + STORE consists of several operations. To make a Read-Modify-Write operation appear indivisible to other execution units, hardware provides atomic operations such as:

Compare-And-Swap
Exchange
Fetch-And-Add

Internally, these operations are not necessarily a single microscopic step. “Atomic” describes what competing execution units are allowed to observe:

┌──────────────────┐
│ read old value   │
│      │           │
│      ▼           │
│ modify           │
│      │           │
│      ▼           │
│ write new value  │
└────────┬─────────┘
         │
         └── appears as one indivisible atomic operation to competitors

Return to count++. If a language implements the increment with a hardware-supported atomic Read-Modify-Write, we can reason about the result as:

initial: count = 0

Thread A                         Thread B

atomically change count 0 -> 1
                                 atomically change count 1 -> 2

final: count = 2

It does not matter whether A or B wins first. Each atomic update must operate on a definite previous value, so both threads cannot independently read 0 and then both write 1.

8.2 Visibility: Cache Coherence

Multicore processors use cache-coherence protocols to coordinate cached copies of the same memory location across cores.

MESI is one of the classic coherence protocols:

M - Modified
E - Exclusive
S - Shared
I - Invalid

Real processors may use MESI or extensions and variants such as:

MESI
MOESI
MESIF
...

We do not need every state transition here; focus on the problem coherence solves.

Suppose both cores cache the line containing count:

Core A Cache              Core B Cache

count = 0                 count = 0
Shared                    Shared

If Core A wants to modify it to:

count = 1

the hardware must coordinate ownership of that cache line and the state of other copies.

At a very high level:

Core A wants to modify count
        │
        ▼
obtain write permission for the cache line
        │
        ▼
invalidate copies that may no longer be used
        │
        ▼
Core A performs the modification

After Core A obtains write permission, Core B's old count = 0 copy is invalidated. The next time B reads count, it cannot continue using that stale copy.

8.3 Memory Ordering: Fence

Consider again:

count = 1
ready = true

Modern CPUs use store buffers, out-of-order execution, and other mechanisms to improve performance. A hardware memory model must define:

Which memory-operation orderings are guaranteed?
Which reorderings are permitted?
Which outcomes may different cores observe?

That is memory ordering.

Return to the publication example. If count = 1 has not yet become observable to B, hardware that permits the outcome might allow:

Thread A / Core A                       Thread B / Core B
|                                       |
+-- STORE count = 1                     |
+-- STORE ready = true                  |
|                                       +-- LOAD ready -> true
|                                       +-- LOAD count -> 0

To forbid that result, fences can constrain the order of memory operations on each side:

Thread A / Core A                       Thread B / Core B
|                                       |
+-- STORE count = 1                     |
+-- FENCE                               |
+-- STORE ready = true                  |
|                                       +-- LOAD ready -> true
|                                       +-- FENCE
|                                       +-- LOAD count -> 1

The writer-side fence prevents count = 1 from being ordered after ready = true. The reader-side fence prevents the read of count from being ordered before the read of ready.

A fence does not guarantee that B will read ready = true. But once B has observed ready = true under the required synchronization pattern, the later read of count cannot still return the stale 0.

9. Summary

Hardware provides different foundations for the three concurrency problems:

Problem	Hardware capability
Atomicity	Atomic Instruction: provides an atomic Read-Modify-Write operation (for example, Fetch-And-Add), so one update appears indivisible to competitors and they cannot observe an intermediate state.
Visibility	Cache Coherence: coordinates the state of copies of the same cache line across cores, so writes propagate according to the coherence protocol and other cores cannot continue using an invalidated stale copy.
Ordering	Fence: constrains related Loads / Stores from being observably reordered across the barrier, establishing the memory-operation ordering relationships required across cores.

The next article returns to the language layer and looks at how Java, Go, and CPython turn these hardware capabilities into rules that programmers can rely on.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

Where Concurrency Problems Begin: Shared State, Message Passing, and Semantics — Concurrency Programming (0)

ThinkerQAQ — Tue, 15 Sep 2026 12:10:13 +0000

1. Define the Scope First
2. Start with a Shared Variable
3. Two Main Coordination Models
- 3.1 Shared Memory: Protect Shared State with Synchronization
- 3.2 Message Passing: Coordinate through Messages
4. Why Does This Code Work Correctly?
5. Languages Need to Define Concurrency Semantics
6. Why Do We Still Need to Go Down to the Hardware?
7. Next: Start with the Hardware

1. Define the Scope First

This series discusses only:

Concurrency within a single process on a single machine.

In other words, the focus is on multiple execution units inside the same process.

For example:

Java Thread / Virtual Thread;
Go Goroutine;
Python Thread / asyncio Task.

Inter-process communication, distributed systems, and network communication are outside the scope.

2. Start with a Shared Variable

Suppose a process contains this variable:

count = 0

Now there are two execution units:

Thread A                    Thread B

count++                     count++

If A and B each execute once, is the final result guaranteed to be:

count = 2

No.

count++ can be logically broken down into three steps:

read count
compute count + 1
write count back

So the following interleaving is possible:

initial: count = 0

Thread A                    Thread B

read count -> 0
compute count + 1 -> 1
                            read count -> 0
                            compute count + 1 -> 1
write count -> 1
                            write count -> 1

The final value becomes:

count = 1

Both threads complete a +1, but because they read the same old value 0, both compute the result as 1, and one update is overwritten.

The key problem is that two threads modify the same count state concurrently:

Multiple execution units access and modify the same state at the same time.

Within one process on one machine, execution units mainly coordinate around shared data in two ways:

Shared Memory / Shared State: multiple execution units directly access the same state;
Message Passing: execution units exchange information through messages.

We can use the same count++ example to see how each model handles the problem.

3. Two Main Coordination Models

3.1 Shared Memory: Protect Shared State with Synchronization

With shared memory, multiple execution units directly access the same state. Continue with count.

In Java, the shared state can be protected with synchronization, for example synchronized:

class Counter {
    private int count = 0;

    public synchronized void increment() {
        count++;
    }
}

Both threads still access the same Counter:

Thread A                    Thread B

counter.increment()         counter.increment()

But because of synchronized, the two threads cannot enter the critical section in increment() at the same time.

Suppose Thread A acquires the lock first. The execution becomes:

initial: count = 0

Thread A                         Thread B

request to enter increment()
acquire lock
                                 request to enter increment()
                                 cannot acquire the same lock; wait
read count       -> 0
compute count+1  -> 1
write count      -> 1
leave critical section and release lock
                                 acquire lock
                                 read count       -> 1
                                 compute count+1  -> 2
                                 write count      -> 2
                                 leave critical section and release lock

final: count = 2

If Thread B acquires the lock first, the order is reversed but the result is still 2.

What matters is not which thread runs first. What matters is that another thread cannot enter the same critical section before one complete read → compute → write sequence finishes.

The interleaving in which both threads read 0 therefore no longer occurs.

Go and Python provide similar mechanisms:

Go      -> Mutex
Python  -> Lock

The shared idea is:

The state is still shared, but access to that state is coordinated through synchronization.

3.2 Message Passing: Coordinate through Messages

Another approach is:

Execution units exchange messages instead of having multiple execution units directly modify the same state.

Continue with count.

For example, in Go:

increments := make(chan int)

go func() { // Counter Owner Goroutine
    count := 0

    for delta := range increments {
        count += delta
    }
}()

The two producer goroutines do not modify count directly. They send increments to the same channel:

go func() { // Goroutine A
    increments <- 1
}()

go func() { // Goroutine B
    increments <- 1
}()

We can abstract it as:

Goroutine A ── +1 ──┐
                     ├──> Channel ──> Counter Owner Goroutine ──> count++
Goroutine B ── +1 ──┘

Both goroutines only send messages. The actual count is modified by a dedicated Counter Owner Goroutine.

Java and Python have similar message-passing tools:

Java BlockingQueue;
Python queue.Queue / asyncio.Queue.

The shared idea is:

Execution units coordinate through messages instead of directly modifying the same state concurrently.

4. Why Does This Code Work Correctly?

So far, we have handled the original count++ problem in two ways.

With shared memory, we used synchronization:

Lock / synchronized / Mutex

With message passing, we used:

Queue / Channel

But there is a more fundamental question:

Why does this code work correctly?

To answer this question, we need to discuss concurrency from three fundamental perspectives:

Atomicity
Visibility
Ordering

Return to the original count++ example:

Atomicity — which operations are atomic?

count++ includes a read, an increment, and a write. After A acquires the lock, why must B wait until A changes count from 0 to 1 and releases the lock before entering the same critical section?
Visibility — when does a write become visible?

After A writes count = 1 and releases the lock, why must B read 1 after acquiring the same lock instead of continuing to see the old value 0? With a channel, why can the Counter Owner receive the +1 sent by a producer?
Ordering — which operations have an ordering relationship?

Why must A's write count = 1 → release lock happen before B's acquire lock → read count = 1? With a channel, why must a send +1 happen before the corresponding receive +1 → update count?

These behaviors cannot depend on one particular CPU happening to execute the program in a convenient way.

Programmers need a defined set of rules they can rely on.

That leads to:

The concurrency semantics provided by the programming language.

5. Languages Need to Define Concurrency Semantics

A programming language needs to define:

When multiple execution units access state concurrently, which behaviors are allowed and which synchronization and memory-access guarantees programmers may rely on.

Java has the:

Java Memory Model

Go has the:

Go Memory Model

Python is not exactly the same. In this series, the discussion mainly uses:

Python / CPython Concurrency Semantics

These rules need to answer three questions:

Question	What it means for the `count` example
Atomicity — which operations are atomic?	`count++` contains `read → increment → write`. Without locking, A's and B's steps can interleave and produce only `1`. With the same lock, A must completely change `count` from `0` to `1` before B can enter the critical section and continue from `1` to `2`.
Visibility — when does a write become visible?	After A writes `count = 1` and releases the lock, B must see `1` after acquiring the same lock instead of using the old value `0`. With a channel, the Counter Owner must be able to receive the producer's `+1`.
Ordering — which operations are ordered?	For the same lock, the order is `A writes count = 1 → A releases lock → B acquires lock → B reads count = 1`. With a channel, it is `producer sends +1 → Counter Owner receives +1 → update count`.

In other words, this layer answers:

What may the programmer rely on?

6. Why Do We Still Need to Go Down to the Hardware?

Languages define rules, but those rules cannot implement themselves.

Java, Go, and Python ultimately rely on several layers to turn language guarantees into behavior on a real machine:

Concurrency Tools / Synchronization Mechanisms
    └─ Mutex / Atomic / volatile / Channel / Queue
            ↓
Language Memory Model / Concurrency Semantics
    ├─ JMM / Go Memory Model / CPython Concurrency Semantics
    └─ implemented by HotSpot JIT / Go Compiler + Runtime / CPython Runtime
            ↓
Hardware
    ├─ Computer Architecture: Von Neumann Architecture / CPU / Memory
    ├─ Hardware Memory Model: x86-TSO / ARM Memory Model
    ├─ Instruction / CPU Primitive: LOAD / STORE / Atomic RMW / Fence
    └─ Microarchitecture: Cache / Store Buffer / Cache Coherence / Out-of-Order Execution

Modern CPUs use many techniques to improve performance:

CPU caches;
store buffers;
out-of-order execution;
atomic instructions;
different memory-ordering rules.

Language-level concurrency semantics ultimately have to be implemented on top of these hardware capabilities.

To understand why language rules and concurrency tools work, we therefore need to understand which behaviors the hardware permits and which constraints it provides.

Before going deeper into the concrete concurrency semantics of Java, Go, and Python, we need to answer:

What guarantees does the hardware actually provide?

7. Next: Start with the Hardware

The next article moves down to the hardware layer.

It focuses on:

why CPUs need caches;
how multiple CPU cores coordinate cached data;
why store buffers and out-of-order execution affect memory-access ordering;
what atomic instructions provide.

Once we understand those pieces, we can return to:

Java Memory Model
Go Memory Model
Python / CPython Concurrency Semantics

Using those rules as the foundation, we can then discuss what guarantees mutexes provide, how runtimes and CPUs implement those guarantees, and how concrete concurrency tools such as atomics and channels fit into the picture.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

DEV Community: ThinkerQAQ

Concurrency Programming (1.5): The Operating System Layer — Threads, Schedulers, and Runtime Scheduling

Table of Contents

- 5. Next: Language Memory Models

0. From Language-Level Execution Units to the CPU

1. How Does Java Schedule Threads?

1.1 From Platform Threads to Virtual Threads

1.1.1 The Simplest Model: One Java Thread per OS Thread

1.1.2 The Problem: Resource and Scheduling Costs of Many OS Threads

1.1.3 The Solution: Let the JVM Schedule Lightweight Threads

1.1.4 The Result: Platform Threads and Virtual Threads

1.2 Execution Layers

1.3 From Java Thread Lifecycles to Their Implementation

1.3.1 Creation and Start: Created → Runnable

1.3.2 Waiting to Be Scheduled: Runnable

1.3.3 Running: Runnable → Running

1.3.4 Parking and Waiting: Running → Waiting

1.3.5 Wakeup and Resumption: Waiting → Runnable → Running

1.3.6 Completion and Cleanup: Running → Terminated

1.4 End-to-End Sequence Diagrams

1.4.1 Platform Thread

1.4.2 Virtual Thread

1.5 Platform Threads vs. Virtual Threads

1.6 Returning to the Opening Question

2. How Does Go Schedule Goroutines?

2.1 From G-M to G-M-P

2.1.1 The Simplest Model: A Goroutine Runs on an OS Thread

2.1.2 The Problem: Many Goroutines, Fewer OS Threads

2.1.3 The Solution: Evolving from G-M to G-M-P

2.1.3.1 Step One: G-M with a Global Run Queue

2.1.3.2 Step Two: Introduce P and Local Run Queues

2.1.3.3 Third Question: What If a P's Local Queue Is Empty?

2.1.3.4 Fourth Question: What If M Blocks in a System Call?

2.1.3.5 Fifth Question: What If One G Runs for Too Long?

2.1.4 The Final Model: G, M, and P

2.1.4.1 G: Goroutine

2.1.4.2 M: Machine

2.1.4.3 P: Processor

2.2 Execution Layers

2.3 From a Goroutine's Lifecycle to Its Runtime Implementation

2.3.1 Creation: Created → Runnable

2.3.2 Waiting for Scheduling: Runnable

2.3.3 Execution: Runnable → Running

2.3.4 Parking and Waiting: Running → Waiting

2.3.5 Wakeup and Resumption: Waiting → Runnable → Running

2.3.6 System Calls and Preemption

2.3.7 Completion and Cleanup: Running → Dead

2.4 End-to-End Sequence Diagram

2.5 Returning to the Opening Question

3. How Does CPython Schedule Threads?

3.1 From Native OS Threads to the GIL and Free-Threaded Builds

3.1.1 The Simplest Model: One Python Thread per OS Thread

3.1.2 The Problem: Sharing Interpreter State Across OS Threads

3.1.3 The Default Solution: The GIL Restricts Interpreter Execution

3.1.4 The Next Step: Free-Threaded CPython

3.1.5 The Final Model: Thread, PyThreadState, and GIL

3.1.5.1 Python Thread

3.1.5.2 OS Thread

3.1.5.3 PyThreadState

3.1.5.4 GIL

3.2 Execution Layers

3.3 From CPython Thread Lifecycles to Their Implementation

3.3.1 Creation and Start: Created → Runnable

3.3.2 Waiting for Scheduling: Runnable

3.3.3 Execution: Runnable → Running

3.3.4 Parking and Waiting: Running → Waiting

3.3.5 Wakeup and Resumption: Waiting → Runnable → Running

3.3.6 Completion and Cleanup: Running → Terminated

3.4 End-to-End Sequence Diagram

3.5 Returning to the Opening Question

4. What the Three Implementations Have in Common

4.1 How Does an Execution Unit Reach the CPU? — Establish Its Carrier

4.2 Who Selects What Runs Next? — Separate the Scheduling Layers

4.3 What Happens After a Wait? — Regain an Execution Opportunity

5. Next: Language Memory Models

Concurrency Programming (8): Read-Write Locks — From Language Rules to the CPU

Table of Contents

0. What Does This Article Answer Next?

1. What Does Java ReadWriteLock Guarantee at the Language Level?

1.1 Atomicity