Our status dashboard scraped a partner's health API every two minutes and lit up green the entire weekend the partner was down. By Monday morning they'd been offline for 61 hours and our board had been reporting "all systems operational" the whole time. Nobody paged. Nothing logged an error. The graph was a flat, confident green line over a three-day outage.
I wrote that scrape. I'd tested it against a live 200, watched it pull the status field, and shipped it, because the happy path worked and the happy path is the only path you ever see in a demo. Three separate bugs were sitting in the code the whole time, and every one of them would have passed a code review. The part I'm not proud of is that finding them took most of a Monday, and the fix for all three fit in about fifteen lines I'd skipped the first time.
Calling an API from a shell script is three problems wearing one trenchcoat: making the request, reading the response, and knowing when either one failed. Get any of the three wrong and the failure is silent — which is the worst kind, because silent failures get discovered by your users, not your tooling.
Bug one: curl exits 0 when the API returns a 500
curl reports on the transport, not the HTTP result. If it reached the server and got a complete response back — any response, including a 503 maintenance page — it exits 0. That's why set -euo pipefail doesn't save you here: from bash's point of view, nothing failed. The command ran, it got bytes back, everyone's happy.
The scrape had been faithfully saving a 503 HTML error page every two minutes and treating it as data. The fix is to make the status code something the script actually looks at:
response=$(curl -sS --connect-timeout 5 --max-time 30 -w $'\n%{http_code}' "$url")
http_code="${response##*$'\n'}" # last line is the status
body="${response%$'\n'*}" # everything before it is the body
Now 2xx is success, 429 and 5xx are transient and worth a retry, and any other 4xx is your own broken request and should fail loudly instead of being retried into oblivion. The two timeouts matter as much as the status check: a short --connect-timeout so a dead host fails fast, and a hard --max-time so a server that accepts your connection and then hangs can't leave a cron job wedged until the next run piles up behind it.
Bug two: grep on JSON doesn't error, it lies
Once you have a verified 2xx body, the next trap is reading fields out of it. Our script pulled the status with grep '"status"' | cut, and on the 503 error page grep matched a different "status" string that happened to appear in the HTML and read it as "ok".
grep, cut, and sed are line-oriented, and JSON has no meaningful lines. The same object can be minified onto one line or pretty-printed across twenty and it's identical data — but every line-based pattern you write depends on one specific layout the API is free to change without telling you. When it changes, your pattern doesn't throw an error. It quietly matches the wrong bytes.
jq parses the document into a real structure and addresses it by path. Three flags carry most of the weight:
# -e sets the exit code from the result, so "missing" is a branch, not an empty string
healthy=$(jq -er '.healthy' <<<"$body") || { echo "field missing"; exit 1; }
# -r prints the raw value — without it "web-01" keeps its quotes and breaks comparisons
region=$(jq -r '.region.name' <<<"$body")
# // supplies a real default, so a missing key becomes 0 instead of the literal null
stars=$(jq -r '.stargazers_count // 0' <<<"$body")
-r, //, and -e are the difference between a parse that survives an API reformat and one that breaks the next time someone on the other end runs a linter. Building select() and projection filters by hand is fiddly and a wrong quote silently matches nothing, which is exactly what the jq Filter Builder is for — paste a real response, click the fields, and it evaluates the filter live against your JSON before you wire it into anything.
Bug three: the failure nobody was told about
The third bug wasn't a bug in the code at all — it was the absence of one. The scrape could fail and nothing would say so. There was an email alert configured, technically; it went to an address forwarding to a distribution list everyone had muted in 2023. An alert nobody reads is decoration.
A Slack incoming webhook puts the failure where the team is already looking. The mechanism is a single POST of a JSON payload to a secret URL, so there are exactly two ways to get it wrong — send malformed JSON, or leak the URL. Build the payload with jq so an error message full of quotes and newlines can't break it, and wire it to a trap so you never have to remember to call it:
: "${SLACK_WEBHOOK_URL:?set it in the environment, never in the script}"
slack_alert() {
local payload
payload=$(jq -n --arg text ":rotating_light: $1" '{text: $text}')
curl -sS --max-time 10 -X POST -H 'Content-Type: application/json' \
-d "$payload" "$SLACK_WEBHOOK_URL" >/dev/null
}
trap 'slack_alert "scrape failed at line $LINENO (exit $?)"' ERR
With set -e, any unhandled non-zero exit fires the trap and posts the failure — with the line number — before the script dies. You set the trap once at the top and the whole script is covered.
Any one of the three would have caught it on minute one
That's the part worth sitting with. The 503 would have tripped the status check. The reshaped error page would have tripped jq -e. And the scrape falling over would have posted to Slack. Three independent guards, each closing one gap, and the outage needed all three to be missing to stay invisible for 61 hours. Wire the pattern together — request that fails on real failures, parse that doesn't lie, alert when either breaks — and you hear about a problem the first time it happens, not the third day.
Full fetch → parse → alert walkthrough with the complete script: https://bashsnippets.xyz/guides/shell-scripts-that-talk-to-apis
The three pieces each have their own deep dive — making the request safely with curl, parsing the response with jq, and alerting to Slack on failure — and the rest of the library is at https://bashsnippets.xyz
Top comments (0)