loading...

Grace - graceful shutdown made simple

stevenacoffman profile image Steve Coffman ・3 min read

I wrote a tiny library called Grace to gracefully shutdown your application by catching the OS signals using sync.errgroup.

I often find I have invoked one or more persistent blocking methods, and some other method is needed be invoked in another goroutine to tell it to gracefully shut down when an interrupt is received.

For instance, when ListenAndServe() is invoked, Shutdown needs to be called.

This library allows you to start zero or more concurrent goroutines, and trigger a graceful shutdown when an interrupt is received. You can bring your own cleanup function to listen for the context's Done channel to be closed, so it will work flexibly in a variety of scenarios.

For example, you probably only want your cleanup function to close your database connection after you have drained your http connections or grpc connections.

        func() error { 
            //cleanup: on interrupt, shutdown server
            <-ctx.Done()
            // We received an interrupt signal, shut down.
            if err := srv.Shutdown(ctx); err != nil {
            // Error from closing listeners, or context timeout:
                log.Printf("HTTP server Shutdown error: %v", err)
            }
            return db.Close()
        })

For reference:

  • Go net/http package offers Shutdown function to gracefully shutdown your http server.
  • Go database/sql package offers Close function to gracefully close the connection to your SQL database.
  • Google google.golang.org/grpc package offers Server.GracefulStop, stops accepting new connections, and blocks until all the pending RPCs are finished

Alternatively, this library also allows you to invoke zero or more concurrent goroutines with an optional timeout.

Documentation


Installation

go get -u github.com/StevenACoffman/grace

Usage

Simple Run until Interrupt signal received

package main

import (
    "log"
    "time"

    "github.com/StevenACoffman/grace"
)

func main() {

    wait, ctx := grace.NewWait()

    err := wait.WaitWithFunc(func() error {
        ticker := time.NewTicker(2 * time.Second)
        for {
            select {
            case <-ticker.C:
                log.Printf("ticker 2s ticked\n")
                // testcase what happens if an error occured
                //return fmt.Errorf("test error ticker 2s")
            case <-ctx.Done():
                log.Printf("closing ticker 2s goroutine\n")
                return nil
            }
        }
    })

    if err != nil {
        log.Println("finished clean")
    } else {
        log.Printf("received error: %v", err)
    }
}

Usage with a default timeout:

package main

import (
    "log"
    "time"

    "github.com/StevenACoffman/grace"
)

func main() {

    wait, ctx := grace.NewWait()

    err := wait.WaitWithTimeoutAndFunc(15*time.Second, func() error {
        ticker := time.NewTicker(2 * time.Second)
        for {
            select {
            case <-ticker.C:
                log.Printf("ticker 2s ticked\n")
                // testcase what happens if an error occured
                //return fmt.Errorf("test error ticker 2s")
            case <-ctx.Done():
                log.Printf("closing ticker 2s goroutine\n")
                return nil
            }
        }
    })

    if err != nil {
        log.Println("finished clean")
    } else {
        log.Printf("received error: %v", err)
    }
}

Usage with cleanup on shutdown

Bring your own cleanup function!

package main

import (
    "fmt"
    "log"
    "net/http"
    "time"

    "github.com/StevenACoffman/grace"
)

func main() {

    wait, ctx := grace.NewWait()
    var httpServer *http.Server

    err := wait.WaitWithFunc(
        func() error {
            http.HandleFunc("/", healthCheck)
            httpServer = newHTTPServer()

            if err := httpServer.ListenAndServe(); err != http.ErrServerClosed {
                return err
            }
            return nil
        },
        func() error { 
            //cleanup: on interrupt, shutdown server
            <-ctx.Done()
            log.Printf("closing http goroutine\n")
            return httpServer.Shutdown(ctx)
        })

    if err != nil {
        log.Println("finished clean")
    } else {
        log.Printf("received error: %v", err)
    }
}

func healthCheck(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/plain")
    w.Header().Set("Content-Length", "0")
    w.WriteHeader(200)
}

func newHTTPServer() *http.Server {
    httpServer := &http.Server{
        Addr:         fmt.Sprintf(":8080"),
        ReadTimeout:  10 * time.Second,
        WriteTimeout: 10 * time.Second,
    }
    log.Printf("HTTP Metrics server serving at %s", ":8080")
    return httpServer
}

Prior Art and Alternatives

This library uses errgroup, but I found a number of other libraries that use other mechanisms:

Comparing them is pretty instructive. I wish I'd used some of their testing techniques!

Posted on by:

Discussion

markdown guide