DEV Community: Kengah Ireneaus

A Practical Guide to jq in Shell Scripts

Kengah Ireneaus — Sun, 16 Aug 2026 18:01:26 +0000

A Practical Guide to jq in Shell Scripts

While building sykd (a system health monitor daemon for linux servers) i needed to collect system data as json so i could process easily in python on doing research if found out about the jq shell program for creating and parsing json in bash, but learning the commands was quite tedious and the man page is large and very difficult to read so i came out with this guide to ease and summarize the major commands you might need.

jq is a lightweight and flexible command-line JSON processor. It is one of the most useful tools for working with JSON data directly inside shell scripts and pipelines.

Installation

Debian / Ubuntu: sudo apt install jq
macOS: brew install jq
Other systems: Use your package manager or download binaries from the official site.

Basic Usage

jq [options] '<filter>' [file...]

Input can come from a file, a pipe, or standard input. The filter expression selects or transforms the data.

Quick note: jq reads JSON from stdin by default. If you want to generate JSON inside jq without reading input, use -n.

Useful Options

Option	Purpose
`-r`	Raw output (no JSON quotes around strings)
`-c`	Compact (single-line) output
`-n`	Start with `null` input (useful for generating JSON)
`-s`	Slurp entire input into an array
`--arg`	Pass a string variable
`--argjson`	Pass a JSON value (preferred for arrays/objects)

Other helpful runtime options:

Option	Purpose
`--rawfile name filename`	Read an entire file as a raw string into variable `$name`
`--slurpfile name filename`	Read file and slurp into array `$name` (like `-s` but into variable)

Example: jq --slurpfile all files.json '.[0] + $all[0]'

Essential Filters

Goal	Filter Example
Pretty-print	`.`
Extract a field	`.name` or `.["name"]`
Nested field	`.user.address.city`
Array element	`.[0]` or `.[-1]`
Iterate array	`.[]`
Select matching objects	`.[] \
Build a new object	{% raw %}`{name, age}`
Transform every element	`map(.name)`
Length	`length`
List keys	`keys`
Conditional	`if .active then "yes" else "no" end`

You can define reusable functions inside jq using def. Example:

def greet: "Hello, " + .name + "!";
. | greet

This will output "Hello, <name>!" for an object with a name key.

Practical Examples

Pretty-print a file

jq . data.json

Extract a single value

jq -r '.version' package.json

Filter an array of objects

jq '.[] | select(.status == "active") | .name' users.json

Create a new structure

jq '{names: map(.name), total: length}' people.json

Use in a pipeline

curl -s https://api.example.com/data | jq '.results[] | .id'

Generate JSON from scratch

jq -n '{name: "example", version: 1}'

Create multiple objects programmatically

jq -n '[range(3) | {index: ., squared: (. * .)}]'

This produces:

[
{"index":0,"squared":0},
{"index":1,"squared":1},
{"index":2,"squared":4}
]

Passing a Bash Array to jq

Bash arrays cannot be passed directly. Convert the array to a JSON array first, then supply it with --argjson.

Recommended Method

my_array=("apple" "banana" "cherry")

# Convert bash array → JSON array
json_array=$(printf '%s\n' "${my_array[@]}" | jq -R . | jq -s .)

# Use it inside jq
jq --argjson arr "$json_array" '$arr'

One-liner version:

jq --argjson arr "$(printf '%s\n' "${my_array[@]}" | jq -R . | jq -s .)" \
   '$arr[]'

Numeric Arrays

numbers=(10 20 30)
json_numbers=$(printf '%s\n' "${numbers[@]}" | jq -s .)

jq --argjson nums "$json_numbers" '$nums | map(. * 2)'

Feed as Input Instead

printf '%s\n' "${my_array[@]}" | jq -R . | jq -s .

Tips for Reliable Shell Scripts

Always quote the filter when it contains spaces or special characters.
Prefer --argjson for arrays and objects; use --arg only for plain strings.
Test filters interactively before embedding them in scripts.
For complex logic, store the filter in a separate .jq file and call it with -f.
When embedding multi-line filters in shell variables, use a heredoc to keep quoting simple:

filter=$(cat <<'EOF'
.[] | select(.status=="active") | {id, name}
EOF
)

curl -s api | jq -f <(echo "$filter")

Prefer --argjson when you need to pass non-string structured data; --arg always passes a string.
When values may contain newlines or special characters, convert them to JSON with jq -R . before passing.
For newline-delimited JSON (NDJSON/JSONL), use jq -c . to ensure compact output and --slurp if you need to combine records into an array.
Use try ... catch for safe extraction and defaults:

try .foo.bar catch "default"

Use inputs to read multiple JSON documents from stdin (useful with -n):

# echo multiple JSON objects and process them with inputs
printf '%s
' '{"x":1}' '{"x":2}' | jq -n '[inputs | .x]'

Additional Examples and Patterns

Selecting and Transforming

Extract and rename fields:

jq '.[] | {identifier: .id, user: .name, active: .status == "active"}' users.json

Build an index (map id → object):

jq 'map({ (.id|tostring): . }) | add' users.json

Group by a key and count per group:

jq 'group_by(.category) | map({category: .[0].category, count: length})' items.json

Working with Strings

Trim and lowercase a field:

jq '.name | ascii_downcase | gsub("^ +| +$"; "")' person.json

Join array of strings:

jq -r '.tags | join(",")' article.json

Numeric operations

Sum field values:

jq '[.[] | .size] | add' files.json

Find maximum:

jq 'max_by(.score) | .score' scores.json

Advanced: reduce, recurse, and more

Use reduce to accumulate values:

jq 'reduce .[] as $item (0; . + $item.value)' metrics.json

Recursively find all keys named id in nested structure:

jq '.. | objects | .id? // empty' nested.json

Streaming large JSON (`jq --stream`)

For very large JSON documents use --stream to get path/value pairs; processing is more manual but uses much less memory:

jq --stream 'select(.[0][-1]=="targetKey") | .[1]' large.json

Working with NDJSON / JSONL

To parse newline-delimited JSON and combine into an array:

jq -s '.' file.jsonl

Process each line independently (streaming):

jq -c '. | {id: .id, ok: (.value > 0)}' file.jsonl

Filters in separate `.jq` files

Save a complex filter to filters.jq:

def summarize:
   {total: (map(.value) | add), count: length};

.items | map({k: .key, v: .value}) | summarize

Call it from shell:

jq -f filters.jq data.json

Passing environment variables into `jq`

Pass a shell variable as JSON (string):

name="O'Reilly"
jq --arg name "$name" '. + {author: $name}' book.json

Pass an environment variable that is already JSON:

json_obj='{"a":1}'
jq --argjson o "$json_obj" '. + $o' base.json

Error handling and safe accesses

Use ? to avoid errors when a key may not exist:

jq '.maybe?.nested // "default"' data.json

Use try to catch runtime errors:

jq 'try .items[] catch empty' data.json

Common pitfalls

Forgetting to quote filters when they contain shell-sensitive characters — always wrap your filter in single quotes in shell.
Passing arrays with --arg (creates a JSON string, not an array) — use --argjson or jq -s conversion.
Assuming jq modifies files in-place — it writes to stdout; redirect output to a file to save changes.

Performance tips

Use --stream for very large JSON that can't fit in memory.
Avoid unnecessary map + add for simple reductions; reduce can be more efficient.
Use compiled filters in -f files for complex repeated transformations.

Examples: Real shell script snippets

Filter API result and extract IDs into a bash array:

readarray -t ids < <(curl -s "https://api.example.com/items" | jq -r '.items[].id')

for id in "${ids[@]}"; do
   echo "Got id: $id"
done

Pass a bash array of words safely into jq and filter by membership:

words=(alpha beta "complex value")
json_words=$(printf '%s\n' "${words[@]}" | jq -R . | jq -s .)

curl -s dataset.json | jq --argjson words "$json_words" '[.[] | select(.name as $n | $words | index($n))]'

Create a small portable CLI using jq and bash:

#!/usr/bin/env bash
data_file=${1:-data.json}
case $2 in
   list)
      jq -r '.items[] | "\(.id) - \(.name)"' "$data_file" ;;
   stats)
      jq '{total: (.items | length), sum: (.items | map(.value) | add)}' "$data_file" ;;
   *) echo "Usage: $0 <file> {list|stats}"; exit 2 ;;
esac

Conclusion

With a few core filters and the correct technique for passing arrays, jq becomes a powerful and reliable tool inside any shell workflow. The patterns shown above cover the majority of day-to-day JSON processing needs.

Happy scripting!

A Month With Bash — Part 3: Building Projects

Kengah Ireneaus — Tue, 04 Aug 2026 00:36:48 +0000

A Month With Bash — Part 3: Building Projects

After all the expansions and syntax, I moved on to regex in bash. It wasn't too hard since I'd already worked with regex in Python, but alongside it I learned grep, sed, and awk — tools that turned out to be extremely useful for automation.

I built a few mini projects and started automating some of my small day-to-day tasks. I won't go too deep into that here, but you can check out my learning-bash GitHub repo, which has all my learning scripts.

From there I covered conditionals, loops, and repetitive tasks.

Finally

I learned about array variables in bash and shell options,
went even further testing different ways of looping, and that's when I started actual project building(I am still building )

#!/usr/bin/env bash

## looping with range functions -- somehow

# python style looping {start..end}
for i in {1..10}; do    # this uses brace expansion so using vars wont work becase of execution sequencing
    echo $i
done

clear

## c - slyle looping 
for (( i = 0; i < 10; i++ )); do # variables works here well
    echo "hello $i"
done

## using variables to loop

clear
start=1
stop=10
step=2

for i in $(seq $start $stop); do # this uses the seq command
    echo "hello world"
done

Conclusion

Spending so much time on bash wasn't a waste. Not only did it force me to learn a huge number of commands, it changed how I think about my own machine — most of what I used to do manually, I can now automate. That shift alone made the month worth it.
i am still learning and trying to get the best practices and things not to do

THANK YOU FOR READING THIS FAR. That is a rough summary of me writing bash for a month there is really a lot left unsaid here but still building and learning. If you are just starting out with bash or if you haven't tried it hope this helps
feel free to drop questions advice and corrections

A Month With Bash — Part 2: Expansions

Kengah Ireneaus — Tue, 04 Aug 2026 00:26:00 +0000

A Month With Bash — Part 2: Expansions

Continuing from where I left off, the next thing I learned was special parameters in bash:

"$*"
$#
$?
$@
$N
$-
$0

Another important concept I picked up is how bash executes shell scripts. Bash is one of those languages that interprets each line as it goes — but it doesn't stop if a line fails. It continues on unless you explicitly set set -o pipefail (or -e, depending on what you want it to catch).

Generally, the procedure looks like this:

Tokenizing: splitting the line into tokens, usually split using the IFS value.

Brace expansion: a mechanism by which arbitrary strings can be generated.

echo file{1,2,3}.txt
## output: file1.txt file2.txt file3.txt

Bash preserves the order from left to right.

Tilde expansion: this is where expansion of special symbols takes place.

~ represents the HOME built-in variable
~+ represents PWD, the current working directory
and others

DIR=~/Desktop # this is $HOME/Desktop
echo "$DIR"

Parameter expansion: introduced with the $ symbol.

# ${} — the braces can be omitted for normal variables but not for array-type variables

Command substitution: very important — it lets you assign the output of a command to a variable, and use commands inside if and for statements. Done with $(command to execute).

week_name="$(date +%A)" # gets the current day of the week
echo "$week_name"

Generally, $() spawns a new shell instance, so it's advisable to avoid it where possible, for latency reasons.

Arithmetic expansion: just from the name, this allows evaluation of arithmetic expressions and substitution of the result. It starts with $(( expression )). There are some rules — bash doesn't support floating point arithmetic natively, so you'd reach for bc if you need it. I won't go deep into that here since this isn't a full bash tutorial.

Here's a simple BMI calculator I wrote while practicing this:

#!/usr/bin/env bash

# script calculates user's BMI and gives a recommendation

set -euo pipefail

# ---------------- read parameters ----------------
weight="$1"
height="$2"

bmi=$(bc -q <<< "scale=2; $weight / ($height^2)")

# ---------------- test ----------------
if (( $(bc -q <<< "$bmi < 18.5") )); then
    echo "UNDERWEIGHT"
elif (( $(bc -q <<< "$bmi < 25") )); then
    echo "normal weight"
elif (( $(bc -q <<< "$bmi < 30") )); then
    echo "overweight"
else
    echo "obese"
fi

Process substitution: this saves the output of a process into a temporary file-like descriptor under /dev/fd/N. It lets you feed command output into programs that expect a file as input. Done with >(command) and <(command).

#!/usr/bin/env bash

# script demonstrates process substitution
# process substitution is when you want to use the output of a command
# as the input to a command that needs files as parameters

set -euo pipefail

# gets the difference between two texts without making temp files
echo "$(diff -p <(echo "hello mr beast") <(echo "hello mr beast."))"

# note: <(cmd) uses the output of a command as a file
# >(cmd) takes input from a program and feeds it to the command as a file

Word splitting and globbing: occurs after expansion, only on unquoted values.

Next up: regex, grep/sed/awk, and moving into building actual projects.

A month with bash

Kengah Ireneaus — Mon, 03 Aug 2026 14:20:00 +0000

A Month With Bash

*this began as one long write-up but i decided to split it into three parts

A month ago I decided to learn bash so I could use my Fedora machine more effectively. My original plan was to just learn some commands and get over with it, but as I began reading resources and books, I quickly noticed how important it is for Linux users, automation experts, and system admins. The thing that amazed me most is that I could literally automate almost every repetitive process I do on my PC.

Getting Started

Starting off with bash wasn't really bad or difficult, maybe because I already had experience programming in C and Python. Learning the basic syntax of bash wasn't much of a problem — things like variables and printing to the terminal were quite easy to grasp. I also learned some theory on shell types and their differences, which wasn't too hard either.

# create, assign, and print a variable
my_var="declared"
echo "$my_var"

Moving Deeper Into Bash

After the basics, where bash just seemed like a small programming language, things changed once I started learning about the bash environment. This is what began to show me what makes it special and unique. I learned about:

Shell initialization scripts
- System-wide
- Individual user scripts
- How to configure them

Honestly, these files affect system behavior, so it's better to avoid touching them unless you know what you're doing. This part was quite confusing at first, but with practice it became clearer. I still don't fully understand every use case of these initialization scripts off-hand, and explaining them isn't the goal of this post.

After that I took some time to learn about reserved variables — just a curated table of them. They're very useful because they enable some handy functionality, and they're good to use when making a script portable. Some important ones:

HOME
IFS
PWD
PATH
DIRSTACK

location="$HOME" # this stores the location of the home directory
echo "$location"

Next up: special parameters, expansions, and how bash actually executes a script line by line.

DEV Community: Kengah Ireneaus

A Practical Guide to jq in Shell Scripts

A Practical Guide to jq in Shell Scripts

Installation

Basic Usage

Useful Options

Essential Filters

Practical Examples

Passing a Bash Array to jq

Recommended Method

Numeric Arrays

Feed as Input Instead

Tips for Reliable Shell Scripts

Additional Examples and Patterns

Selecting and Transforming

Working with Strings

Numeric operations

Advanced: reduce, recurse, and more

Streaming large JSON (jq --stream)

Working with NDJSON / JSONL

Filters in separate .jq files

Passing environment variables into jq

Error handling and safe accesses

Common pitfalls

Performance tips

Examples: Real shell script snippets

Conclusion

A Month With Bash — Part 3: Building Projects

A Month With Bash — Part 3: Building Projects

Finally

Conclusion

A Month With Bash — Part 2: Expansions

A Month With Bash — Part 2: Expansions

A month with bash

A Month With Bash

Getting Started

Moving Deeper Into Bash

Streaming large JSON (`jq --stream`)

Filters in separate `.jq` files

Passing environment variables into `jq`