Limit the Go Routines

A easy way to limit the go routines without impacting the performance

Yes,It’s true GO takes less memory to initiate a Go Routine and we can trigger any number of go routines simultaneously but we have to consider
few parameters like with whom it is interacting , what type of data it is handling, will routine run infinitely or it ends after given task in done etc…
And we always get these type of issues only in PROD ENV 😅

In case of handling files concurrently we may ran out of memory it may cause of failure of the system eventually we may lose the data in memory.
hence system shows poor performance.. it is not all acceptable in modern world 🤔.
In case Data Base interaction we can’t create Infinite no of connections to a data base and unnecessarily it will stress the data base and increases the CPU usage.
In case of fire and forget tasks like logging things in background. it will trigger routine for each log and increases the CPU load.

TO Avoid such type of issues we have to utilize the go routines efficiently
i.e limiting the Go Routines (go routine limiter)
In go we can simply use a buffered channel and limit the go routines usage.
Steps:-

1. decide how many go routines you want to trigger (consider factors like CPU and memory )
2. load the some dummy data into a buffered channel
3. trigger the go routine and buffered channel into it
4. release data from buffered channel once task is done so it will free up space for one task/routine to fire.

package main

import (
    "context"
    "fmt"
    "runtime"
    "time"
)

const maxGoRoutines = 2

// main itself go routine in golang
// #GO Routine - >1
func main() {
    ctx := context.Background()

    // set ctx timeout to exit the worker before main exits
    ctx, cancelFunc := context.WithTimeout(ctx, time.Second*4)
    defer cancelFunc()

    // #GO Routine - >2
    go worker(ctx)
    // easiest way  to hold the main is sleep for some time
    // hard way is we can use ctx with timeout , if we are implementing same in an api then it will run until we exit the api
    time.Sleep(time.Second * 5)

    //total GO Routines count=  #GO Routine(1)+ #GO Routine(2)+ maxGoRoutines


}
func worker(ctx context.Context) {

    // Buffered channel to limit the go routines
    limit := make(chan bool, maxGoRoutines)

    // channel to communicate between pool of go routines and print the cur data
    data := make(chan int, maxGoRoutines)

    // send some data to data channel so go routines can communicate
    data <- 1

    for {
        select {

        case <-ctx.Done():
            fmt.Println("no of go routines used in current execution", runtime.NumGoroutine())
           fmt.Println("exiting worker")
            // close the channel before return
            close(limit)

            return
        default:

            // if buffer is full then it will block the code here itself (it will not allow to trigger another go routine)

            // sending data to the limit channel
            limit <- true

            go func() {
                // sleep for sometime (this will be useful in understanding the limiter)
                time.Sleep(time.Second * 1)
                x := <-data
                fmt.Println(x)

                // increment the value by 1
                data <- x + 1

                // release the data from channel so that we can accommodate another go routine
                <-limit

            }()
        }

    }
}

Explanation:-

1. GO blocks the code if channel buffer is full and it will wait until someone to free up the space to unblock
2. In above Code we are loading data to channel and then firing the routine once buffer is full and go routines taking time to finish the tasks it will block the code to fire new routines until at-least one task is done from a pool.
3. for sake of data communication between used a data channel

Please feel free to add any comments/suggestion/ mistakes etc…👍

please follow medium for interesting things

Sai Teja – Medium

Read writing from Sai Teja on Medium. Software Engineer | Fitness trainer | https://www.linkedin.com/in/saitejamesala/ | https://topmate.io/saiteja180.

medium.com