DEV Community

Shannon
Shannon

Posted on

12 1

Parse your escaped JSON with jq and fromjson

Recently, I was looking at secrets manager in AWS and used the AWS CLI in order to read a secret in my terminal, such as: aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:[acct-id]:secret:testing/slack/slack_api_key-y8FHjv --profile [profile] --region us-west-2

This gives the following response:

{
    "ARN": "arn:aws:secretsmanager:us-west-2:[account-id]:secret:testing/slack/slack_api_key-y8FHjv",
    "Name": "testing/slack/slack_api_key",
    "VersionId": "a9a1fc45-80d5-45d0-8884-27d359b55ebd",
    "SecretString": "{\"SLACK_API_KEY\":\"itsasecret\"}",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": "2021-11-15T15:32:59.737000-08:00"
}
Enter fullscreen mode Exit fullscreen mode

As you can see, the response gave me escaped JSON with \ in front of every " character for the actual contents of the secret. Because I wanted to use this programmatically, I needed to parse the JSON further with jq. However, I wasn't sure how to actually do this with escaped characters. Thankfully, jq has built-in functionality for this!

Within the manual, search for Convert to/from JSON. This will give a small example on how to use it. Similarly, I'm going to show how to use it with secrets manager below.


So, we have a JSON response right now that is parseable, so let's use jq to shrink this response some: aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:[account-id]:secret:testing/slack/slack_api_key-y8FHjv --profile [profile] --region us-west-2 | jq .SecretString

This now gives us:

"{\"SLACK_API_KEY\":\"itsasecret\"}"
Enter fullscreen mode Exit fullscreen mode

Let's take it a step further and get rid of the escaped characters(for brevity's sake, I'm going to only show the jq pieces from here on): | jq '.SecretString | fromjson'

Here's my response:

{
  "SLACK_API_KEY": "itsasecret"
}
Enter fullscreen mode Exit fullscreen mode

Now this is parseable too! Let's take only the value: | jq '.SecretString | fromjson | .SLACK_API_KEY'

Here's our response:

"itsasecret"
Enter fullscreen mode Exit fullscreen mode

Finally, I want this to not have quotes around it, so I'll pass in the -r flag for jq: | jq -r '.SecretString | fromjson | .SLACK_API_KEY'

Voila! So how can we actually use this? Well, I could easily pass this in as an env var or variable into a script to utilize the value without hardcoding it into a file.

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up

Top comments (3)

Collapse
 
bobzrkr profile image
bobzrkr

I actually laughed out loud. I googled "jq read escaped" and this was the first thing that came up. It was the exact solution need to my exact problem: reading a secret from AWS.

Thank you!

Collapse
 
gy2319504 profile image
gy2319504 • Edited

that is brilliant idea. I was used in extract it will online tool like this JSON ESCAPE ONLINE . using jq will be much easier, and we can automate the job

Collapse
 
epsi profile image
E.R. Nurwijayadi • Edited

Cool.

I never thought this JQ could be useful in CI/CD situation.