Eliminating Sensitive Values from Logs using Slog, the Prospective Official Structured Logger for Go

In the Go language, the log package has been used for official log output for a long time. However, in recent cloud environments and the like, structured logging is almost mandatory, and in response to such trends, the official structured log package slog has been proposed. It is already listed in the release notes for Go 1.21, which is expected to be released in August 2023, and it is almost certain that it will officially be incorporated in 1.21.

Background: Absolutely do not want to output secret values to the log

Logs output by online services are used for various purposes. For example, logs are aggregated to generate alerts for service operation monitoring, to analyze for service improvement, to detect unauthorized access for service security monitoring, and so on. Also, logs are often used for auditing, and various log-related mechanisms are built on the premise that logs cannot be deleted, at least during the storage period.

The more information is output to the log, the wider its use. It's especially advantageous to have more clues in debugging and troubleshooting. However, the data handled by the service may contain information that should not be carelessly output, such as authentication information like passwords and tokens, personally identifiable information like names, phone numbers, addresses, and bank account information and credit card numbers. In this article, we will conveniently call these "secret values".

As mentioned earlier, logs are often stored for a certain period of time and cannot be deleted during that period. Therefore, if secret information is output to the log, it will require a lot of effort to delete it. You may need to take measures such as limiting the number of people who can access it, but this reduces the situations in which it can be used and makes it impossible to utilize the log.

It's perfectly reasonable to say, "You shouldn't output secret values to the log in the first place," but consider the case where a struct is output to the log, and the fields of that struct, or even nested structs, may contain secret values. Even if you are careful in selecting the values to output to the log, the struct itself may be modified later to add fields that contain secret values. It's very difficult to cover all such cases. On the other hand, considering the requirement to leave as much information as possible in the log, it's not a very realistic approach to say, "Don't output structs to the log."

slog.LogValuer

slog provides an interface called LogValuer, and by implementing this, you can customize the value to output to the log. This can be used to hide or mask values, for example.

type Token string

func (Token) LogValue() slog.Value {
    return slog.StringValue("REDACTED_TOKEN")
}

func main() {
    t := Token("ThisIsSecretToken")
    slog.Info("permission granted", "token", t)
}

When this code is executed, the value ThisIsSecretToken is not output, and is replaced with REDACTED_TOKEN.

time=2009-11-10T23:00:00.000Z level=INFO msg=Access token=REDACTED_TOKEN

This mechanism seems like a good solution to the problem mentioned earlier, but it doesn't apply to fields contained in structs. For example, LogValuer does not work in the following example.

type Token string

func (Token) LogValue() slog.Value {
    return slog.StringValue("REDACTED_TOKEN")
}

type AccessLog struct {
    User  string
    Token Token
}

func main() {
    l := AccessLog{
        User:  "mizutani",
        Token: "ThisIsSecretToken",
    }
    slog.Info("Access", "log", l)
}

As in the following example, the value you wanted to hide is output as is.

time=2009-11-10T23:00:00.000Z level=INFO msg=Access log="{User:mizutani Token:ThisIsSecretToken}"

Therefore, a different approach is needed to more precisely hide secret values.

Using ReplaceAttr

TextHandler and JSONHandler provided by slog have an option called ReplaceAttr. This calls a callback before outputting a value passed as an attribute value, allowing you to replace the value.

type Token string

type AccessLog struct {
    User  string
    Token Token
}

func redact(_ []string, a slog.Attr) slog.Attr {
    l, ok := a.Value.Any().(AccessLog)
    if !ok {
        return a
    }
    return slog.Any(a.Key, AccessLog{
        User:  l.User,
        Token: "REDACTED_TOKEN",
    })
}

func main() {
    l := AccessLog{
        User:  "mizutani",
        Token: "ThisIsSecretToken",
    }

    logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{ReplaceAttr: redact}))
    logger.Info("Access", "log", l)
}

By writing it this way, you can hide secret values.

time=2009-11-10T23:00:00.000Z level=INFO msg=Access log="{User:mizutani Token:REDACTED_TOKEN}"

However, it's not practical at all to write such a judgment logic. Therefore, we need a mechanism that can hide in a more general way.

masq

After a long introduction, let me introduce a package to redact secret values, m-mizutani/masq .

https://github.com/m-mizutani/masq

masq is a package for generating callbacks for ReplaceAttr. For example, if you want to hide a value of type EmailAddr, you write as follows.

u := struct {
    ID    string
    Email EmailAddr
}{
    ID:    "u123",
    Email: "[e-mail]",
}

logger := slog.New(slog.HandlerOptions{
    ReplaceAttr: masq.New(masq.WithType[EmailAddr]()),
}.NewJSONHandler(os.Stdout))

logger.Info("hello", slog.Any("user", u))

This will output as follows (formatted with the jq command).

{
  "time": "2022-12-25T09:00:00.123456789",
  "level": "INFO",
  "msg": "hello",
  "user": {
    "ID": "u123",
    "Email": "[REDACTED]" // ← hidden
  }
}

In this way, you can hide values that match the type, even if they are in the fields of a struct.

masq.New has various options available, and generates a callback for ReplaceAttr based on the specified options. The following options are available:

• WithType[T](): Hides values that match type T.
• WithString(s string): Hides values that match string s.
• WithRegex(re regexp.Regex): Hides values that match regexp re.
• WithTag(tag string): Hides values that match the masq field tag of a struct. For example, if you specify secret, it will hide fields tagged with masq:"secret".
• WithFieldName(name string): Hides values that match the field name name of a struct.
• WithFieldPrefix(prefix string): Hides values that start with the field name prefix of a struct.

These can be combined in multiple ways, as shown below.

logger := slog.New(slog.HandlerOptions{
    ReplaceAttr: masq.New(
        // Hides the type AccessToken
        masq.WithType[AccessToken](),

        // Hides strings that start with 14 to 16 digits
        masq.WithRegex(regexp.MustCompile(`^\+[1-9]\d{14,16}$`)),

        // Hides fields tagged with masq:"secret"
        masq.WithTag("secret"),

        // Hides fields whose names start with Secret
        masq.WithFieldPrefix("Secret"),
    ),
}.NewJSONHandler(out))

For detailed usage, please refer to the README.

Summary

In fact, this mechanism has been provided in the zlog package for some time. However, since slog has finally reached a practical stage, we implemented masq as a new package to match slog.

As mentioned earlier, slog also has a mechanism called LogValuer, so using that is also an option. I think the best choice is to make the appropriate choice according to your use case, and I would be happy if you could include masq as one of your choices.