Our carrier rotated our API secret and the longer one broke our header in two

One of our regional carriers takes collection bookings through a REST API with basic authentication. Our side of it is a shell script run by cron every quarter of an hour, which builds the header like this:

AUTH=$(echo -n "$CLIENT_ID:$CLIENT_SECRET" | base64)
curl -sf -H "Authorization: Basic $AUTH" ...

It ran for three years. In August the carrier moved to a new identity platform and issued us new credentials, with a forty character secret instead of a sixteen character one. We swapped them in the vault, and every booking that afternoon came back 400 Bad Request. The carrier's support team could see the requests and said the header was malformed.

GNU base64 wraps its output with a line break after every seventy six characters. Seventy six characters of base64 encode fifty seven bytes. Our old client id and secret together came to forty one bytes, so the output was one line. The new pair came to sixty five, so the output was two lines, and curl sent the header exactly as we gave it: the first seventy six characters, a newline, and the rest of the credential on a line of its own that was not a header at all. A trace from a local listener showed it plainly. On the Mac where the script had been written, base64 does not wrap by default, so the same line had always worked there.

We had the bookings going again within the hour by passing -w 0 to base64. Then we removed the hand built header altogether. curl encodes basic credentials itself, and the script now passes --netrc-file pointing at a file the vault agent writes, so the secret is no longer on the command line either. The script's test now runs against a small echo server with credentials of one, fifty seven, fifty eight and two hundred bytes and checks that the header arrives as one line and decodes back to what was sent.

The secret was the only thing that changed, and nothing in our code had cared about its length. The tool in the middle had been folding lines at seventy six characters the whole time. Our credential had simply been too short to show it.

– Sergey Shinder