DEV Community

Steve Coffman
Steve Coffman

Posted on • Updated on

Don't Panic: Catching Panics in Errgroup Goroutines

#go

Image by Nicosmos, Public domain, via Wikimedia Commons

TL;DR version

StevenACoffman/errgroup is a drop-in alternative to Go's wonderful sync/errgroup but it converts goroutine panics to errors.

Why you want this

While net/http installs a panic handler with each request-serving goroutine,
goroutines do not and cannot inherit panic handlers from parent goroutines,
so a panic() in one of the child goroutines will kill the whole program.

So in production, whenever you use an sync.errgroup, you have to have the discipline to always remember to add a
deferred recover() to the beginning of every new goroutine, and convert any panics to errors.

            defer func() {
                if rec := recover(); rec != nil {
                    err = FromPanicValue(rec)
                }
            }()
Enter fullscreen mode Exit fullscreen mode

You can see this in action in the Go Playground here. You also want to be careful not to lose the stack trace. The CollectStack function I used here is overly simplified, so it adds a little noise because it doesn't the skip the FromPanicValue and CollectStack frames. 🤷‍♂️

func FromPanicValue(i interface{}) error {
    switch value := i.(type) {
    case nil:
        return nil
    case string:
        return fmt.Errorf("panic: %v\n%s", value, CollectStack())
    case error:
        return fmt.Errorf("panic in errgroup goroutine %w\n%s", value, CollectStack())
    default:
        return fmt.Errorf("unknown panic: %+v\n%s", value, CollectStack())
    }
}

func CollectStack() []byte {
    buf := make([]byte, 64<<10)
    buf = buf[:runtime.Stack(buf, false)]
    return buf
}

Enter fullscreen mode Exit fullscreen mode

A co-worker of mine co-worker Ben Kraft, wrote some handy wrapper code around sync/errgroup to avoid that boilerplate (and required discipline). With his permission, I lightly modified it to
lift it out of our private work repository for the more general Go community.

StevenACoffman/errgroup is a drop-in alternative to Go's wonderful
sync/errgroup with the difference that it converts goroutine panics to errors.

You can see it in use in the playground or here:

package main

import (
    "fmt"

    "github.com/StevenACoffman/errgroup"
)

func main() {
    g := new(errgroup.Group)
    var urls = []string{
        "http://www.golang.org/",
        "http://www.google.com/",
        "http://www.somestupidname.com/",
    }
    for i := range urls {
        // Launch a goroutine to fetch the URL.
        i := i // https://golang.org/doc/faq#closures_and_goroutines
        g.Go(func() error {

            // deliberate index out of bounds triggered
            fmt.Println("Fetching:", i, urls[i+1])

            return nil
        })
    }
    // Wait for all HTTP fetches to complete.
    err := g.Wait()
    if err == nil {
        fmt.Println("Successfully fetched all URLs.")
    } else {
        fmt.Println(err)
    }
}
Enter fullscreen mode Exit fullscreen mode

Counterpoint

There is an interesting discussion which has an alternative view that,
with few exceptions, panics should crash your program. I'm ok with that in development and testing, but would rather sleep soundly at night.

Prior Art

With only a cursory search, I found a few existing open source examples.

Kratos errgroup

Kratos Go framework for microservices has a similar errgroup
solution.

PanicGroup by Sergey Alexandrovich

In the article Errors in Go:
From denial to acceptance
,
(which advocates panic based flow control 😱), they have a PanicGroup that's roughly equivalent:

type PanicGroup struct {
  wg      sync.WaitGroup
  errOnce sync.Once
  err     error
}

func (g *PanicGroup) Wait() error {
  g.wg.Wait()
  return g.err
}

func (g *PanicGroup) Go(f func()) {
  g.wg.Add(1)

  go func() {
    defer g.wg.Done()
    defer func(){
      if r := recover(); r != nil {
        if err, ok := r.(error); ok {
          // We need only the first error, sync.Once is useful here.
          g.errOnce.Do(func() {
            g.err = err
          })
        } else {
          panic(r)
        }
      }
    }()

    f()
  }()
}
Enter fullscreen mode Exit fullscreen mode

I would appreciate feedback or suggestions for improvement!

Top comments (0)