Parse your escaped JSON with jq and fromjson

Recently, I was looking at secrets manager in AWS and used the AWS CLI in order to read a secret in my terminal, such as: aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:[acct-id]:secret:testing/slack/slack_api_key-y8FHjv --profile [profile] --region us-west-2

This gives the following response:

{
    "ARN": "arn:aws:secretsmanager:us-west-2:[account-id]:secret:testing/slack/slack_api_key-y8FHjv",
    "Name": "testing/slack/slack_api_key",
    "VersionId": "a9a1fc45-80d5-45d0-8884-27d359b55ebd",
    "SecretString": "{\"SLACK_API_KEY\":\"itsasecret\"}",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": "2021-11-15T15:32:59.737000-08:00"
}

As you can see, the response gave me escaped JSON with \ in front of every " character for the actual contents of the secret. Because I wanted to use this programmatically, I needed to parse the JSON further with jq. However, I wasn't sure how to actually do this with escaped characters. Thankfully, jq has built-in functionality for this!

Within the manual, search for Convert to/from JSON. This will give a small example on how to use it. Similarly, I'm going to show how to use it with secrets manager below.

---

So, we have a JSON response right now that is parseable, so let's use jq to shrink this response some: aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:[account-id]:secret:testing/slack/slack_api_key-y8FHjv --profile [profile] --region us-west-2 | jq .SecretString

This now gives us:

"{\"SLACK_API_KEY\":\"itsasecret\"}"

Let's take it a step further and get rid of the escaped characters(for brevity's sake, I'm going to only show the jq pieces from here on): | jq '.SecretString | fromjson'

Here's my response:

{
  "SLACK_API_KEY": "itsasecret"
}

Now this is parseable too! Let's take only the value: | jq '.SecretString | fromjson | .SLACK_API_KEY'

Here's our response:

"itsasecret"

Finally, I want this to not have quotes around it, so I'll pass in the -r flag for jq: | jq -r '.SecretString | fromjson | .SLACK_API_KEY'

Voila! So how can we actually use this? Well, I could easily pass this in as an env var or variable into a script to utilize the value without hardcoding it into a file.

Comments

[bobzrkr]

I actually laughed out loud. I googled "jq read escaped" and this was the first thing that came up. It was the exact solution need to my exact problem: reading a secret from AWS.

Thank you!

[gy2319504]

that is brilliant idea. I was used in extract it will online tool like this JSON ESCAPE ONLINE . using jq will be much easier, and we can automate the job

[E.R. Nurwijayadi]

Cool.

I never thought this JQ could be useful in CI/CD situation.