DEV Community: Shingo Kawamura

I built a transport-independent foundation for JSON API clients in Perl

Shingo Kawamura — Sat, 19 Sep 2026 23:37:59 +0000

Writing a simple API client is easy.

my $response = $http->get($url);

Then the client starts being used in production.

Suddenly, it needs to handle:

query parameters
JSON encoding and decoding
structured errors
retries and Retry-After
rate limits
pagination
authentication
request IDs
logging and metrics
idempotency

At that point, what started as a small API wrapper often becomes a collection of infrastructure code.

I found myself looking at this problem and asking:

How much of this is actually specific to the API being wrapped?

Not very much.

That led me to build HTTP::API::Core, a small, dependency-light foundation for building JSON HTTP API clients in Perl.

The key idea is simple: keep HTTP transport separate from API-client policy.

Use HTTP::Tiny, LWP, Mojo::UserAgent, Furl, or your preferred transport for HTTP. Let HTTP::API::Core handle the repetitive policy around it.

The project is available on GitHub.

It is deliberately not another HTTP client

One of the main design decisions was not to replace the existing HTTP ecosystem.

Perl already has good HTTP implementations:

HTTP::Tiny
LWP
Mojo::UserAgent
Furl

HTTP::API::Core sits above the transport layer.

The HTTP implementation is responsible for making the request.

HTTP::API::Core is responsible for the application-level policies surrounding that request.

Conceptually:

Your API client
      |
HTTP::API::Core
      |
HTTP::Tiny / LWP / Mojo / Furl / custom transport
      |
     HTTP

This separation means an API wrapper does not have to couple its retry, pagination, error handling, or authentication logic to a particular HTTP library.

A small example

A client can start with:

use HTTP::API::Core;

my $api = HTTP::API::Core->new(
    base_url => 'https://api.example.com',
    headers  => {
        Authorization => "Bearer $ENV{API_TOKEN}",
    },
    timeout => 10,
    retry => {
        attempts   => 3,
        base_delay => 0.25,
        max_delay  => 5,
        jitter     => 1,
    },
);

my $response = $api->get('/users');
my $data = $response->json;

A service-specific client can then remain small:

sub get_user {
    my ($self, $id) = @_;

    return $self->{api}
        ->get("/users/$id")
        ->json;
}

The resource method describes the API.

The core handles the repetitive infrastructure around it.

Retries are more complicated than they look

Retries were one of the areas where I wanted conservative defaults.

HTTP::API::Core retries safe/idempotent methods by default:

GET
HEAD
PUT
DELETE
OPTIONS

It does not automatically retry POST or PATCH.

Retryable failures include transport failures and HTTP responses such as:

It also understands Retry-After and rate-limit reset metadata.

Delays use exponential backoff with optional jitter.

If an API explicitly supports idempotent POST requests, that behavior can be enabled rather than silently assumed.

For example:

$api->post('/jobs',
    json => { task => 'sync' },
    retry => {
        attempts => 2,
        methods  => ['POST'],
    },
);

The goal is to make the safe behavior the boring default.

Rate limits should be data, not special cases

Different APIs expose rate limits differently.

Instead of making application code repeatedly inspect raw headers, responses expose normalized rate-limit information:

my $rate = $response->rate_limit;

say $rate->limit
    if defined $rate->limit;

say $rate->remaining
    if defined $rate->remaining;

say $rate->wait_seconds
    if $rate->exhausted;

HTTP::API::Core understands the commonly used X-RateLimit-* family as well as standard-style rate-limit fields and Retry-After.

The retry system can use the same normalized information.

This is one of the ideas behind the project: parse infrastructure information once and expose it consistently.

One pagination interface, different API styles

Pagination is another thing that looks trivial until you support several APIs.

Some APIs return a next URL.

Others use page numbers.

Others use opaque cursors.

HTTP::API::Core provides one iterator interface for these styles.

For cursor pagination:

my $pager = $api->paginate(
    '/users',
    mode  => 'cursor',
    items => 'data.users',
    next  => 'meta.next_cursor',
    query => { limit => 100 },
);

while (my $user = $pager->next) {
    ...
}

For page-number pagination:

my $pager = $api->paginate(
    '/users',
    mode      => 'page',
    items     => 'users',
    page_size => 100,
);

my @users = $pager->all;

Repeated cursors or next URLs are detected so a broken API response does not accidentally create an infinite pagination loop.

Authentication is implemented through hooks

I did not want authentication to become a growing collection of special cases inside the HTTP client.

Instead, HTTP::API::Core provides lifecycle hooks.

For example:

my $api = HTTP::API::Core->new(
    base_url => 'https://api.example.com',

    hooks => {
        before_request => sub {
            my ($ctx) = @_;

            $ctx->{headers}{Authorization}
                = "Bearer $token";
        },

        after_response => sub {
            my ($response, $ctx) = @_;

            log_status($response->status);
        },

        on_error => sub {
            my ($error, $ctx) = @_;

            record_failure($error->category);
        },
    },
);

The same mechanism can be used for:

authentication
logging
metrics
tracing
request customization

Small helpers for Bearer, Basic, and API-key authentication are included.

OAuth token acquisition and refresh deliberately remain outside the core.

Observability without choosing an observability framework

The library records useful request metadata without requiring a specific logging or tracing stack.

For example:

say $response->elapsed;

say $response->request_id
    if defined $response->request_id;

Common request ID headers are normalized, including:

X-Request-Id
Request-Id
X-Correlation-Id

Hooks can then send this information wherever an application wants.

The library provides the data without deciding how the application should observe it.

Structured errors

Another design goal was to avoid application code parsing human-readable error strings.

Errors have machine-readable categories such as:

encode
decode
transport
http
hook

Application code can inspect fields including:

$error->category;
$error->status;
$error->retryable;
$error->request_id;

HTTP errors retain their response information, so callers can still inspect headers, text, JSON, or rate-limit state.

Transport independence

The transport boundary is intentionally small.

A transport receives:

($method, $url, \%options)

and returns a response-like hash containing at least:

status => 200

That means the same API client policy can sit on top of different HTTP implementations.

It also makes testing API-specific code possible without requiring real network requests.

Testing against real API conventions

The repository includes example clients demonstrating different API styles:

GitHub — page-number pagination and rate-limit metadata
Slack — cursor pagination
Cloudflare — page-number pagination using response metadata

These are not intended to replace their official SDKs.

They are examples showing that the abstraction can map onto APIs with significantly different conventions.

What I intentionally left out

A reusable core can easily become a framework that tries to solve everything.

I am trying to avoid that.

HTTP::API::Core intentionally does not handle:

service-specific SDK behavior
complete OAuth flows
OpenAPI generation
GraphQL-specific behavior
WebSockets
HTTP server functionality
async runtime concerns

The project is intended to remain small, predictable, dependency-light, and transport-independent.

The broader idea

Although this implementation is for Perl, the architectural question is language-independent.

Where should the boundary be between:

the HTTP transport,
reusable API-client policy, and
service-specific domain methods?

Putting everything into the HTTP layer makes the transport too opinionated.

Putting everything into every service SDK means repeatedly implementing retries, pagination, errors, authentication, and rate-limit handling.

HTTP::API::Core is my attempt to put that boundary in the middle.

Try it

HTTP::API::Core is available on GitHub:

👉 github.com/kawamurashingo/HTTP-API-Core

I'd be especially interested in feedback from people who maintain API clients or SDKs:

What functionality do you consider generic API-client infrastructure, and what would you deliberately keep out of a core like this?

Why I Built a jq-Compatible Tool in Pure Perl (and Why It Still Matters)

Shingo Kawamura — Fri, 02 Jan 2026 13:06:54 +0000

Modern tooling often assumes a perfect world:
latest Linux, internet access, package managers, and rebuildable binaries.

Reality is different.

In many production environments — especially enterprise, legacy, or air-gapped systems —
those assumptions break down.

That gap is why I built jq-lite.

The Problem: jq Is Great — Until You Can’t Use It

jq is an excellent JSON processor.
But in real operations, I kept hitting walls:

Legacy UNIX systems with outdated glibc

Air-gapped environments with no package repositories

Minimal containers without build tools

Environments where “just install jq” was not an option

In those cases, JSON processing didn’t disappear —
the need became more critical.

Shell scripts still had to be reliable.
Automation still had to work.

The Idea: A jq-Compatible Tool That Never Breaks Scripts

I didn’t want another JSON tool.

I wanted a tool that could be:

Dropped into restricted systems

Trusted in shell scripts

Stable across years, not releases

So I built jq-lite with three strict rules:

Design Rule #1: Pure Perl, No Dependencies

jq-lite is written in pure Perl.

No native extensions

No external libraries

No compilation step

If Perl exists, jq-lite runs.

That means it works on:

legacy Linux / UNIX

minimal containers

offline environments

systems where only base OS tools are allowed

Perl is still one of the most widely deployed runtimes in the world —
especially in places modern tooling forgot.

Design Rule #2: jq Compatibility Where It Matters

jq-lite aims to be jq-compatible in daily CLI usage:

echo '{}' | jq-lite --arg greeting hello '.hello = $greeting'

Output:

{
"hello": "hello"
}

The goal isn’t to clone every feature,
but to support the jq patterns people actually use in automation.

Design Rule #3: A Stable CLI Contract (This Is the Important Part)

Here’s the part that made jq-lite different.

jq-lite defines a documented CLI contract:

exit codes

error categories

stderr prefixes

stdout behavior on failure

And that contract is tested.

Exit Code Meaning

0 Success
1 -e used and result was false / null / empty
2 Compile error
3 Runtime error
4 Input error
5 Usage error

This means shell scripts can safely rely on jq-lite:

if jq-lite -e '.enabled' config.json; then
deploy
fi

No surprises.
No silent behavior changes.

In operations, this matters more than features.

Why Not Rewrite It in Rust or Go?

I get this question a lot.

Because the problem wasn’t performance or language fashion.

The problem was deployability and longevity.

Perl already exists on:

old servers

enterprise systems

restricted environments

Adding jq-lite doesn’t require changing the environment —
just using what’s already there.

Who Is jq-lite For?

jq-lite is not for everyone.

It is for people who:

maintain legacy systems

write shell scripts that must survive years

operate in restricted or offline environments

care about CLI behavior as a contract, not a suggestion

If you’ve ever thought
“this script must not break in five years”
— jq-lite is for you.

Where to Find It

GitHub: https://github.com/kawamurashingo/JQ-Lite

CLI Contract: https://github.com/kawamurashingo/JQ-Lite/blob/main/docs/cli-contract.md

CPAN: https://metacpan.org/release/JQ-Lite

Alpine Linux package available

Final Thought

Modern tools optimize for speed of change.

jq-lite optimizes for stability over time.

Sometimes, that’s the more important optimization.

Author
川村慎吾（Shingo Kawamura）
SRE / Infrastructure Engineer
CPAN author of jq-lite