Thought about Goroutines Lifecycle

Go is reference to a programing language but more, It also reference to keyword to trigger goroutine, a clear and best feature in go programing language make this language strong in concurreny.

Green Threads / M:N Scheduling

Goroutines are often referred to as "green threads." This is because they are not directly mapped to OS threads. Instead, the Go runtime contains its own scheduler that maps a potentially large number of goroutines onto a much smaller number of OS threads. This is known as M:N threading, where M represents goroutines and N represents system threads.

In the M:N threading model that Go utilizes, M denotes Goroutines while N represents system threads. The Go runtime scheduler manages these Goroutines using three key concepts:

• M (Machine): Refers to an OS thread. It executes code and needs a P to run any G.
• P (Processor): A context for scheduling that holds a queue of Goroutines that need to be executed. The P needs to be attached to an M to run Goroutines.
• G (Goroutine): The actual Goroutine that holds the stack and the function to execute.

Goroutine Lifecycle

The Go runtime manages the lifecycle of a goroutine through several stages, from creation to termination:

1. Creation

• go Keyword: A goroutine is created by using the go keyword followed by a function call. This signals the Go runtime to initialize a new goroutine.

  go myFunction()

Launching a goroutine (reference: proc.go): When you start a goroutine with go keyword, internally, the newproc function gets called from the proc.go file. This function creates a new G (goroutine), initializes it, and puts it to the local run queue of the current P (processor). If there's no P available, it puts the G into the global queue.

// Create a new G(goroutine) running function fn.
// Put the newly created G(goroutine) on the queue of G's waiting for a P(processor) to run.
// The compiler turns a 'go' statement into a call to this function.
func newproc(fn *funcval) {
    // Get the current G(goroutine).
    gp := getg()

    // Get the address of the function that caused 'newproc' to be called for tracing.
    pc := getcallerpc()

    // Switch from the G(goroutine) stack to the system stack, this is required because some functions in Goroutine creation must run on system stack.
    systemstack(func() {
        // Create the new G(goroutine).
        newg := newproc1(fn, gp, pc)

        // Get the current P(processor).
        pp := getg().m.p.ptr()

        // Add the new G(goroutine) to the queue of G's maintained by P(processor).
        runqput(pp, newg, true)

        // If the main G(goroutine) has started, potentially wake up another M(OS thread).
        if mainStarted {
            wakep()
        }
    })
}

• Stack Allocation: The runtime allocates a small initial stack for the goroutine, which can grow or shrink as needed. This stack is separate from the OS thread stack.

See about newproc1 invoke by newproc func, the piece of code that deals with stack allocation for the new goroutine is:

// 'gfget' tries to get a free G(goroutine) from the free list of the P(processor) that is executing on the current M(machine).
newg := gfget(pp)

// If there is no free G(goroutine) available,
if newg == nil {

    // then create a new G(goroutine) using 'malg' function with a minimal stack size.
    newg = malg(stackMin)

    // The status of new G(goroutine) is changed from _Gidle to _Gdead.
    // _Gidle -> G is unused(empty), _Gdead -> G is exited, and waiting to be cleaned up.
    casgstatus(newg, _Gidle, _Gdead)

    // Add this new G(goroutine) to the list of all G's(goroutines).
    allgadd(newg)
}

2. Scheduling

• Goroutine States: Goroutines pass through various states such as runnable, running, waiting, or syscall during their lifecycle.

Prefer to go runtime2.go these are 7 status about goroutine, exclude 2 unused status _Genqueue_unused and _Gmoribund_unused

const (
    _Gidle           = iota // 0 : goroutine is idle
    _Grunnable              // 1 : goroutine is runnable (on a run queue)
    _Grunning               // 2 : goroutine is running (executing)
    _Gsyscall               // 3 : goroutine is performing a syscall 
    _Gwaiting               // 4 : goroutine is waiting for the runtime
    _Gdead                  // 6 : goroutine is dead
    _Gcopystack             // 8 : stack being copied to implement a grow or shrink operation
        _Gpreempted             // 9 : goroutine is preempted
)

In proc.go use casgstatus function to switch between gouroutine state A to B see more here, some sample like:

Make a goroutine to runnable

// Mark gp ready to run.
func ready(gp *g, traceskip int, next bool) {
    ....
    // status is Gwaiting or Gscanwaiting, make Grunnable and put on runq
    casgstatus(gp, _Gwaiting, _Grunnable)

Execute a machine

// Schedules gp to run on the current M.
func execute(gp *g, inheritTime bool) {
    ....
    // Assign gp.m before entering _Grunning so running Gs have an
    // M.
    mp.curg = gp
    gp.m = mp
    casgstatus(gp, _Grunnable, _Grunning)

After link all source code we have a state diagram:

• M:N Scheduling: The Go runtime implements an M:N scheduler that multiplexes M goroutines onto N OS threads, allowing for efficient use of resources.

mstart() and mstart1() functions define how an OS thread (or 'M') begins execution.

// mstart is the entry-point for new Ms.
// It is written in assembly, uses ABI0, is marked TOPFRAME, and calls mstart0.
func mstart()

func mstart1() {
  .............
  schedule()
}

It requires a 'P' (processor) to execute any goroutines. The schedulers/threads (denoted as 'M') are created and executed in schedule function.

func schedule() {
    pp := _g_.m.p.ptr()
    pp.preempt = false
        gp, inheritTime, tryWakeP := findRunnable() // blocks until work is available
    // If about to schedule a not-normal goroutine (a GCworker or tracereader),
    // wake a P if there is one.
    if tryWakeP {
        wakep()
    }
    execute(gp, inheritTime)
}

• Goroutine Queues: Runnable goroutines are placed in a local run queue or a global run queue, waiting for an available logical processor (P) to execute them.

3. Execution

• Context Switching: The runtime switches between goroutines when a goroutine makes a blocking system call, an explicit yield happens (like runtime.Gosched()), or it completes a predefined quantum of execution time.

Go runtime provides a function called Gosched() in the runtime package for explicitly yielding processor time to other goroutines.

// Gosched yields the processor, allowing other goroutines to run. It does not
// suspend the current goroutine, so execution resumes automatically.
//
//go:nosplit
func Gosched() {
    checkTimeouts()
    mcall(gosched_m)
}

// Implementation of gosched_m:
// Gosched continuation on g0.
func gosched_m(gp *g) {
    if traceEnabled() {
        traceGoSched()
    }
    goschedImpl(gp)
}

// Implementation of goschedImpl:
func goschedImpl(gp *g) {
    status := readgstatus(gp)
    if status&^_Gscan != _Grunning {
        dumpgstatus(gp)
        throw("bad g status")
    }
    casgstatus(gp, _Grunning, _Grunnable)
    dropg()
    lock(&sched.lock)
    // pushes the current goroutine into the global queue
    globrunqput(gp)
    unlock(&sched.lock)
    // schedules another goroutine to run
    schedule()
}

This function calls mcall(gosched_m), which pushes the current goroutine into the global queue (globrunqput(gp)) and schedules another goroutine to run (schedule()). There is another way to call this process is Explicit Yielding

4. Blocking

• System Calls & Channels: When a goroutine performs a blocking operation like I/O, network communication, or channel operations, it enters a waiting state and yields the processor to other goroutines.

Under the hood, the Go runtime uses epoll on Linux, kqueue for BSDs and OS X, WaitForMultipleObjects on Windows, and *select"⇢ on other platforms to perform non-blocking I/O. When a goroutine enters a syscall (for example I/O operation via the net package), it calls into the scheduler, announcing that it's not runnable anymore which is handled by gopark.

// Puts the current goroutine into a waiting state and calls unlockf on the
// system stack.

func gopark(unlockf func(*g, unsafe.Pointer) bool, lock unsafe.Pointer, reason waitReason, traceReason traceBlockReason, traceskip int) {
    if reason != waitReasonSleep {
        checkTimeouts() // timeouts may expire while two goroutines keep the scheduler busy
    }
    mp := acquirem()
    gp := mp.curg
    status := readgstatus(gp)
    if status != _Grunning && status != _Gscanrunning {
        throw("gopark: bad g status")
    }
    mp.waitlock = lock
    mp.waitunlockf = unlockf
    gp.waitreason = reason
    mp.waitTraceBlockReason = traceReason
    mp.waitTraceSkip = traceskip
    releasem(mp)
    // can't do anything that might move the G between Ms here.
    mcall(park_m)
}

While the goroutine waits for the I/O operation to complete, the Go runtime scheduler can run other goroutines on the same thread. At the end of the system call, goready is called which marks the goroutine as ready.

• Goroutine Parking: The runtime "parks" the goroutine, potentially moving it to a wait queue associated with the resource it is waiting on.

5. Resuming

• Ready to Run: Once the blocking condition is resolved (for example, I/O completion or channel availability), the goroutine transitions back to the runnable state and waits to be scheduled on an OS thread.

6. Termination

• Function Completion: A goroutine terminates naturally when its main function returns.
• Panic Recovery: If a panic occurs and is not recovered within the goroutine, it will cause the goroutine to terminate. In contrast, a recovered panic may allow the goroutine to continue executing or clean up gracefully.
• Runtime Exit: When the main function finishes or os.Exit is called, the runtime will terminate all goroutines regardless of their state.

7. Cleanup

• Stack De-allocation: Once a goroutine has finished executing, its stack is reclaimed by the runtime.
• Resources Release: Any other resources held by the goroutine are released or finalized during the termination process.

Reference

深入golang runtime的调度

go runtime schedule 原理 - 知乎

Scalable Go Scheduler Design Doc