DEV Community

Yuya Takeyama
Yuya Takeyama

Posted on

Mask multiple lines text in GitHub Actions Workflow

This article is a translation of an article I originally wrote in Japanese, translated into English using ChatGPT with some modifications:


tl; dr

Although it might seem like a bit of bad practice, the content of $multiple_lines_text can be masked with the following one-liner:

- run: |
    echo "::add-mask::$(echo "$multiple_lines_text" | sed ':a;N;$!ba;s/%/%25/g' | sed ':a;N;$!ba;s/\r/%0D/g' | sed ':a;N;$!ba;s/\n/%0A/g')"
Enter fullscreen mode Exit fullscreen mode

add-mask Command in GitHub Actions

GitHub Actions has a feature called workflow commands.

The add-mask command is used to mask specific strings in subsequent outputs. For example, by executing the following command, the string "Hello, World!" will be masked as "***" in subsequent outputs.

echo "::add-mask::Hello, World!"
Enter fullscreen mode Exit fullscreen mode

Values retrieved from secrets can be masked this way, but using the add-mask command allows dynamically specifying strings to be masked.

Additionally, this command is implemented in the NPM package @actions/core. By using core.setSecret('Hello, World!'); in JavaScript code, the same masking can be achieved.

The setSecret function essentially builds and outputs the ::add-mask::~~~ string to the standard output, so the mechanism is identical.

Issues When Masking Multiple Lines

If you try to mask a string like Foo\nBar\nBaz in a shell script without any considerations, it would look like this:

::add-mask::Foo
Bar
Baz
Enter fullscreen mode Exit fullscreen mode

The command breaks due to the line breaks, and only "Foo" gets masked. This can lead to unwanted side effects like the string "Foo Fighters" being masked as "*** Fighters".

The workflow commands documentation briefly mentions handling multiple strings:

However, these techniques are meant for inputting multiline environment variables or output values into $GITHUB_ENV or $GITHUB_OUTPUT, and based on my tests, they don’t work with workflow commands.

Examining the core.setSecret Code

By examining the core.setSecret function in @actions/core, it appears that the values to be masked are escaped using a function called escapeData.

https://github.com/actions/toolkit/blob/d1df13e178816d69d96bdc5c753b36a66ad03728/packages/core/src/command.ts#L80-L85

function escapeData(s: any): string {
  return toCommandValue(s)
    .replace(/%/g, '%25')
    .replace(/\r/g, '%0D')
    .replace(/\n/g, '%0A')
}
Enter fullscreen mode Exit fullscreen mode

The toCommandValue function simply returns the string as-is if it’s already a string, so it can be ignored.

https://github.com/actions/toolkit/blob/d1df13e178816d69d96bdc5c753b36a66ad03728/packages/core/src/utils.ts#L11-L18

Then, the replace method replaces % with %25, \r with %0D, and \n with %0A.

By executing a command like this, Foo\nBar\nBaz gets masked correctly:

echo "::add-mask::Foo%0ABar%0ABaz"
Enter fullscreen mode Exit fullscreen mode

After asking ChatGPT about how to perform these replacements in a shell script, I arrived at the following conclusion:

echo "::add-mask::$(echo "$multiple_lines_text" | sed ':a;N;$!ba;s/%/%25/g' | sed ':a;N;$!ba;s/\r/%0D/g' | sed ':a;N;$!ba;s/\n/%0A/g')"
Enter fullscreen mode Exit fullscreen mode

While the s/%/%25/g part is clear, the preceding :a;N;$!ba; part is obscure. I understand from ChatGPT that it’s necessary for handling line breaks, but I can’t explain it in detail here.

When Is This Necessary?

This article is written for those wondering how to mask multiline strings, regardless of the purpose. For me, there was a specific use case: safely storing and using a GitHub App's Private Key.

To achieve this, I believe the following steps are necessary:

  1. Encode the Private Key in Base64 and store it in AWS Secrets Manager.
  2. Use OIDC for authentication with AWS.
  3. Retrieve the secret in the workflow, decode it from Base64 to get the Private Key.

Assuming AWS usage, the steps can be adapted similarly for Google Cloud using Secret Manager, etc. (not verified).

This can be implemented in a workflow as follows:

steps:
  - name: Configure AWS credentials
    id: aws-credentials
    uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: arn:aws:iam::012345678901:role/role-name
      aws-region: ap-northeast-1

  - name: Retrieve secret from AWS Secrets Manager
    id: aws-secrets
    run: |
      secrets=$(aws secretsmanager get-secret-value --secret-id secret-name --query SecretString --output text)
      gh_app_private_key="$(echo "$secrets" | jq .GH_APP_PRIVATE_KEY_BASE64 -r | base64 -d)"
      echo "::add-mask::$(echo "$gh_app_private_key" | sed ':a;N;$!ba;s/%/%25/g' | sed ':a;N;$!ba;s/\r/%0D/g' | sed ':a;N;$!ba;s/\n/%0A/g')"
      echo "gh-app-private-key<<__EOF__"$'
'"$gh_app_private_key"$'
'__EOF__ >> "$GITHUB_OUTPUT"

  - uses: actions/create-github-app-token@v1
    id: app-token
    with:
      app-id: ${{ vars.GH_APP_ID }}
      private-key: ${{ steps.aws-secrets.outputs.gh-app-private-key }}
Enter fullscreen mode Exit fullscreen mode

Further details are explained in the Q&A format below.

Why Not Store It in Organization or Repository Secrets?

Because it is considered unsafe.

With some specific conditions, the secret's value can be exposed if someone can create a Pull Request and place a GitHub Actions workflow file in the repository.

For more details, refer to this presentation from a recent event. (company blog)

Why Do You Need to Mask the Private Key?

Without masking, the Private Key passed as input to actions/create-github-app-token@v1 can be viewed in the GitHub Actions UI.

Why Encode the Private Key in Base64 for AWS Secrets Manager?

It’s not mandatory but simplifies storing it without line breaks.

While AWS Secrets Manager can store secrets with line breaks, the key/value mode in the Management Console does not handle line breaks well. Plaintext mode allows entering line breaks, but it’s cumbersome. Therefore, encoding it in Base64 for storage without line breaks is more convenient.

Top comments (0)