DEV Community: christoph-sens

Treetrail: Why I wrote a JSONPath library for Java

christoph-sens — Sat, 10 Oct 2026 10:56:09 +0000

Originally published on christoph-sens.com.

If you work with JSON in Java, you use JSONPath – often without noticing. In Spring tests it sits behind jsonPath("$.items[0].name"), in API gateways and integration platforms it picks fields out of webhooks, and in configuration files it describes which value goes where. Surprisingly, JSONPath had no specification for some 17 years after Stefan Goessner's article of 2007. Every library decided for itself what $..book[-1:] or $[?(@.price < 10)] should return.

That changed in February 2024, when the IETF published RFC 9535, a standard for JSONPath, together with a Compliance Test Suite of 706 test cases. Conformant implementations soon appeared for Python, Rust and .NET. For Java – the platform that probably runs more JSONPath expressions than any other – a mature answer was missing.

Treetrail is my attempt to close that gap. This post explains why the library exists, what it can do today and how it compares with the alternatives.

The motivation

One query, several answers

It started with a simple observation: the same JSONPath query returns different results depending on the library. On the array [0, 1, …, 9], for example:

Query	RFC 9535	Jayway JsonPath 3.0.0
`$[1:6:2]`	`[1, 3, 5]`	`[1, 2, 3, 4, 5]`
`$[9:0:-1]`	`[9, 8, 7, 6, 5, 4, 3, 2, 1]`	`[]`

As long as an expression lives in a single Java service, this hardly shows. But once the same query is evaluated in the backend, in a Python script for data analysis and in a TypeScript frontend, it gets tricky: three components, three interpretations, and the bug only shows up in production. That is exactly what standards are for. A standard only helps, though, if every platform has an implementation that really follows it.

Jayway JsonPath: a great service, but older than the standard

By far the most widely used JSONPath library in Java is Jayway JsonPath. It has served the Java ecosystem extremely well since 2011, and Spring uses it for its jsonPath(...) matchers. But it was created long before there was a standard – and so, through no fault of its own, it does not conform to it. Measured against the Compliance Test Suite, Jayway 3.0.0 returns the same result as the standard for 86 of the 459 valid queries; with filters put in parentheses, as Jayway requires, it is 176. Conversely, Jayway accepts 132 of the 247 queries that the standard considers invalid.

A library this widely used cannot simply change its behavior without breaking thousands of projects. That calls for a separate, standard-conformant library – and for a way to switch to it without risk.

Queries from untrusted sources

The third reason is security. More and more often, JSONPath expressions don't come from your own code but from tenants, users or third-party configuration. Then a query must not be able to take the service down. Regular expressions with backtracking are the best-known weak spot: the pattern ((a+)+)+b against 24 as and a ! keeps java.util.regex busy for about 0.4 seconds on JDK 25 – and every further a doubles the time.

And a personal reason

I have been building backends with Java and Kotlin for more than 15 years. To me, open source is the most honest way to show how I work: every decision, every test and every bug is public. Treetrail is written in Java on purpose, not in Kotlin – a dependency on the Kotlin standard library would make adoption by projects such as Spring, Apache Camel or Apache NiFi needlessly harder.

What Treetrail can do today

Version 0.2.0 has been on Maven Central since 8 October 2026. Its key features:

Standard-conformant – checked in every build

Treetrail passes all 706 cases of the Compliance Test Suite, on Java 17, 21 and 25 and with every supported JSON model. If a single case fails, the build fails. Where the standard leaves room or the test suite deviates from the RFC text, the decision is documented in docs/conformance.md.

One kind of result

Every query returns a list of nodes, whether it selects one value or many. Every node knows its normalized path. There is no configuration option that changes what a query returns.

JsonPath path = JsonPath.compile("$.store.book[?@.price < 10].title");

NodeList<Object> nodes = path.query(document);
nodes.values(); // ["Sayings of the Century", "Moby Dick"]
nodes.paths();  // ["$['store']['book'][0]['title']", "$['store']['book'][2]['title']"]

The JSON tree you already have

The core has no dependencies besides the JDK. It works on plain Java maps and lists, or parses JSON text itself with a small, strict parser. Adapters query the trees of Jackson 2 and 3, Gson and Jakarta JSON-P directly – without conversion – and return that library's own nodes, so you can keep working with them. Other models plug in through a small interface.

JsonNode document = objectMapper.readTree(json);
NodeList<JsonNode> books = JsonPath.compile("$.store.book[?@.price < 10]")
        .query(document, Jackson2Model.INSTANCE);

Safe with untrusted queries

Regular expressions in match() and search() follow I-Regexp (RFC 9485) and run on Treetrail's own automaton engine without backtracking, in linear time. The pattern above that costs java.util.regex 0.4 seconds takes Treetrail less than a tenth of a microsecond. On top of that, every run has limits: a budget of visited nodes, a maximum depth and a cap on the result size. Even if a document brings thousands of different regular expressions along, the caches hold about 10 MB at most.

JsonPath path = JsonPath.compile(untrustedExpression)
        .withLimits(EvaluationLimits.DEFAULT.withMaxVisitedNodes(1_000_000).withMaxResultSize(10_000));

Fast

Conformance doesn't cost speed. In the JMH benchmarks, Treetrail is at least on par with Jayway JsonPath in every query; on Jackson trees, the most common setup in practice, it is 1.2 to 2.2 times faster, and 1.7 times faster at compiling an expression. The measurements and their caveats are in docs/benchmarks.md.

Made for tests

Most JSONPath code in Java projects lives in tests. Two modules bring the standard there: AssertJ assertions and a replacement for Spring's jsonPath(...) matchers in MockMvc and WebTestClient.

mockMvc.perform(get("/store"))
        .andExpect(jsonPath("$.store.book[?@.price < 10].title").values("Sayings of the Century", "Moby Dick"));

Values compare as JSON values: 399, 399L and 399.0 all match the JSON number 399. A failing assertion names the expression and shows the values it found, with their paths.

An orderly way off Jayway

Nobody likes to migrate on a hunch. That's why Treetrail comes with two tools:

jsonpath-migration runs your expressions with Jayway JsonPath and with Treetrail against sample documents and reports every difference – with a rewrite hint for Jayway-specific syntax. It uses the Jayway version your project already depends on and takes your Jayway configuration into account.
jsonpath-rewrite contains an OpenRewrite recipe that finds every JSONPath expression in a code base that is passed to Jayway or to Spring's matchers, and assesses it: valid RFC 9535, Jayway-only, write API, or computed at runtime.

The README also has a table of Jayway idioms and their equivalents.

How Treetrail compares

Compared with Jayway JsonPath

	Treetrail	Jayway JsonPath
Standard	RFC 9535, all 706 CTS cases	its own dialect, older than the standard
Result of `$.a.b`	always a node list with paths	a single value or a list, depending on path and configuration
Filters	`[?@.price < 10]` (parentheses optional)	`[?(@.price < 10)]`
Regular expressions	I-Regexp, linear time	`java.util.regex` with backtracking
Limits for untrusted queries	node budget, depth, result size, regex memory	–
JSON libraries	Java objects, Jackson 2/3, Gson, JSON-P	json-smart, Jackson, Gson and others via providers
Spring test matchers	separate module	built into Spring
Write API (`set`, `delete`)	no, queries only	yes
Type mapping (`read(…, Integer.class)`)	no, via your own JSON library	yes
Aggregate functions (`sum()`, `avg()`, …)	no; standard functions `length()`, `count()`, `match()`, `search()`, `value()`	yes

The table also shows where Jayway offers more: the write API, type mapping and aggregate functions are convenient and deliberately not part of the standard. If you rely on them heavily, the README shows how to do the same with Treetrail and your own JSON library – or you keep Jayway for those places. The two libraries can live side by side without trouble.

Compared with other RFC 9535 implementations in Java

Treetrail is not the only Java library that implements the standard, and that's a good thing. Christoph Burgmer's JSONPath Comparison has compared dozens of implementations across all languages for years. I have submitted a pull request for Treetrail there; in a local run over 258 queries, the picture for Java looks like this (agreement with the cross-language consensus on 175 queries):

Implementation	Consensus matched
Treetrail	173
ajp (based on XSLT/ixml)	173
SJF4J	173
Jayway JsonPath 3.0.0	143
JSurfer	133

The two remaining queries are shorthand forms that are invalid under RFC 9535 – there the standard deliberately departs from the old consensus. What I find remarkable is that Treetrail and ajp, two entirely independent implementations, answer all 258 queries identically. That is exactly what a standard is supposed to achieve.

So the differences lie less in conformance than in everything around it:

SJF4J is more lenient and accepts considerably more expressions that the standard rejects – handy when switching, but less strict if queries are meant to stay portable.
jsonlens combines JSONPath with JSON Pointer (RFC 6901) and JSON Patch (RFC 6902) – features Treetrail doesn't have.
Treetrail focuses on operational safety with untrusted queries, direct adapters for the common JSON libraries, test integration for AssertJ and Spring, and migration tools for existing Jayway code.

How Treetrail is tested

The Compliance Test Suite checks grammar and semantics, but not what happens with inputs nobody thought of. So on top of it:

Fuzzing with Jazzer in every CI build and every night: compiling, querying, regex matching and JSON parsing may only throw the documented exceptions and must finish each input within five seconds. The fuzzer did find a bug before 0.2.0: a regular expression that repeated an empty group more than a billion times took 46 seconds to compile.
Property tests with jqwik, for example that every normalized path selects exactly its node.
Differential tests: 20,000 random regular expressions against java.util.regex, and 2,000 random queries against jsonpath-rfc9535, a careful Python implementation of the standard.
Concurrency: 16 threads share compiled queries while the regex automata are being built.

The supply chain is part of quality too: every release is signed and ships a CycloneDX SBOM and a build provenance attestation per module, and the jars are reproducible. Recently, the build has also started checking API compatibility with the last release, test coverage per module, and the code itself with Error Prone and NullAway; the next release brings JSpecify annotations, so Kotlin sees which values can be null.

What's next

The road to 1.0 is planned in the open on GitHub: a stable API with a compatibility guarantee, more static analysis and mutation testing. Before 1.0 the API may still change – which makes now the best time for feedback.

Trying it takes one line:

dependencies {
    implementation("io.github.treetrail:jsonpath-core:0.2.0")
}

Treetrail runs on Java 17 and later and is licensed under the Apache License 2.0. If something is missing or doesn't fit your use case, please open an issue. And if your team is wondering how to move JSONPath expressions in a grown code base safely to the standard: get in touch.

Large SQS and SNS messages in Kotlin: the extended client pattern without Java baggage

christoph-sens — Sat, 26 Sep 2026 16:39:40 +0000

Every SQS queue and SNS topic has a message size limit. When a payload is larger, the usual answer is the claim check pattern: put the payload in S3, send a small pointer instead, and resolve the pointer on the receiving side. AWS ships this pattern as two Java libraries, amazon-sqs-java-extended-client-liband amazon-sns-java-extended-client-lib, both built on payload-offloading-java-common-lib-for-aws.

If your service is written in Kotlin on aws-sdk-kotlin,
those libraries don't fit: they wrap the AWS SDK for Java v2 clients, not the Kotlin ones. So I
ported them. The result is three small libraries on Maven Central:

s3overflow: the payload store (S3 upload, pointer, download, delete)
sqsoverflow: an SqsClient that offloads large message bodies
snsoverflow: an SnsClient that offloads large message bodies

This post covers why the Java libraries are a liability today, how the ports work, and what you
need to know before migrating.

Where the Java libraries stand today

The extended clients do their job, but their foundation has aged:

Old dependencies with known advisories. Both clients depend on payloadoffloading-common 2.2.0, whose last release was in March 2024. It pins jackson-databind 2.15.2 and jackson-core 2.16.0. As of September 2026, the GitHub Advisory Database lists seven advisories affecting exactly these versions, three of them rated high, among them bypasses of Jackson's polymorphic type validation and a denial-of-service issue in the async parser. Even the latest SQS client release (2.1.3, July 2026) still pulls in these versions through the common library.
Java 8 bytecode and a pre-coroutine design. All three libraries compile for Java 8 and keep separate sync and async client classes with largely duplicated pass-through code.
Slow release cadence. The SNS client's latest release (2.1.0) is from March 2024, the common library's from the same month.

To be fair: whether an advisory is exploitable depends on how Jackson is used, and the extended clients only (de)serialize a small pointer object. You can also override the Jackson versions in your own build. But every dependency scanner flags these versions, someone has to triage the findings again and again, and forcing newer Jackson versions under a library that was never tested against them is its own risk. Removing Jackson from this code path makes the question disappear.

What the size limits are today

The limits moved recently, which changes when offloading is needed at all:

SQS accepts messages up to 1 MiB (body plus attributes). Until August 2025 the limit was 256 KiB.
SNS accepts 256 KiB by default. Since September 2026 a topic can accept up to 1 MiB when you raise its MaximumMessageSize attribute.

sqsoverflow therefore offloads at 1 MiB by default, snsoverflow at 256 KiB. Both are configurable
through payloadSizeThreshold.

Usage

dependencies {
    implementation("com.christoph-sens:sqsoverflow:1.1.0")
}

val s3Client = S3Client.fromEnvironment { region = "eu-central-1" }
val sqsClient = SqsClient.fromEnvironment { region = "eu-central-1" }

val client = SqsExtendedClient(
    sqsClient,
    SqsExtendedClientConfig(payloadStore = S3BackedPayloadStore(s3Client, bucketName = "my-payload-bucket")),
)

client.sendMessage(SendMessageRequest { queueUrl = myQueueUrl; messageBody = largePayload })

val messages = client.receiveMessage(ReceiveMessageRequest { queueUrl = myQueueUrl }).messages.orEmpty()
for (message in messages) {
    process(message.body) // the original payload, already resolved from S3
    client.deleteMessage(DeleteMessageRequest { queueUrl = myQueueUrl; receiptHandle = message.receiptHandle })
}

SqsExtendedClient is an SqsClient, so you can pass it anywhere a plain client is expected.
Messages under the threshold go straight to SQS. Larger ones are written to S3, and SQS only carries
a small JSON pointer plus the ExtendedPayloadSize message attribute. On receive, the client fetches the payload and hands you the original body. On delete, it removes the S3 object as well (cleanupS3Payload, on by default).

snsoverflow works the same way for publish and publishBatch. SNS has no receive side, so the
subscriber resolves the pointer: for an SNS topic that fans out to an SQS queue with raw message delivery, that's sqsoverflow's SqsExtendedClient.

Why a port instead of a wrapper around the Java library

One suspend API instead of sync and async classes

The Java originals come in two flavors each, for example AmazonSQSExtendedClient for SqsClient and AmazonSQSExtendedAsyncClient for SqsAsyncClient. aws-sdk-kotlin clients are suspend-based from the start, so one class covers both cases and fits naturally into coroutine code.

Interface delegation replaces a thousand lines of pass-through code

Most of the SQS interface has nothing to do with payload offloading: createQueue, listQueues, tagQueue and dozens more. The Java library forwards each of these by hand in a base class of roughly 1,150 lines. Kotlin's interface delegation does this in one line:

class SqsExtendedClient(
    private val sqsClient: SqsClient,
    private val clientConfig: SqsExtendedClientConfig,
) : SqsClient by sqsClient {
    override suspend fun sendMessage(input: SendMessageRequest): SendMessageResponse { /* offload if needed */ }
    // ... only the methods with real offload logic are overridden
}

sqsoverflow overrides eight methods: sendMessage, sendMessageBatch, receiveMessage,
deleteMessage, deleteMessageBatch, changeMessageVisibility, changeMessageVisibilityBatch and
purgeQueue. snsoverflow overrides two: publish and publishBatch. Everything else is delegated
to the wrapped client. All three libraries together are about 730 lines of Kotlin, license headers
included.

No Jackson dependency

The Java libraries serialize the S3 pointer with Jackson. The ports use kotlinx.serialization, so nothing in their runtime classpath is Jackson, and the advisories described above do not apply to them. Fewer transitive dependencies means fewer libraries to keep patched. Dependabot keeps the
remaining dependencies current, and CodeQL and dependency review run on every pull request in all
three repositories.

Less configuration surface

The original configuration classes carry options for client-side encryption strategies and canned ACLs. The ports take a PayloadStore and a handful of behavior flags (payloadSizeThreshold, alwaysThroughS3, cleanupS3Payload, ignorePayloadNotFound, s3KeyPrefix). Encryption is configured where it belongs: on the bucket, with SSE-S3 or SSE-KMS.

publishBatch in snsoverflow is offload-aware as well. The Java SNS library predates the SNS PublishBatch API and only handles publish.

Migrating from the Java libraries

The important caveat first: the ports are not wire-compatible with the Java libraries. The pointer JSON and the receipt-handle format differ. A message sent by the Java extended client cannot be resolved by sqsoverflow, and vice versa. Switch all producers and consumers of a queue at the
same time, or drain the queue first.

The ExtendedPayloadSize attribute name is the same, so SNS-to-SQS fan-out works between snsoverflow and sqsoverflow.

Before (Java):

ExtendedClientConfiguration config = new ExtendedClientConfiguration()
    .withPayloadSupportEnabled(s3Client, "my-payload-bucket");
SqsClient client = new AmazonSQSExtendedClient(SqsClient.builder().build(), config);

After (Kotlin):

val client = SqsExtendedClient(
    SqsClient.fromEnvironment(),
    SqsExtendedClientConfig(payloadStore = S3BackedPayloadStore(s3Client, bucketName = "my-payload-bucket")),
)

Options without an equivalent:

Java option	In the ports
Client-side encryption (`ServerSideEncryptionStrategy`)	Configure SSE-S3 or SSE-KMS on the bucket
`ObjectCannedACL`	Use bucket policies
SNS per-message `"S3Key"` attribute	Use `s3KeyPrefix` in the config
Legacy `SQSLargePayloadSize` attribute name	Always `ExtendedPayloadSize`

Verifying what you download

Every release is built and published by a GitHub Actions workflow. The jars, POM and Gradle module metadata on Maven Central carry a signed build provenance attestationthat ties each file to the repository, the workflow and the tagged commit it was built from. You can check a downloaded jar with the GitHub CLI:

gh attestation verify sqsoverflow-1.1.0.jar --repo christoph-sens/sqsoverflow

Licensing

sqsoverflow and snsoverflow are derivative works of the AWS libraries under Apache-2.0, with per-file attribution and a NOTICE file listing what was changed. s3overflow is an independent reimplementation that takes no code from payload-offloading-java-common-lib-for-aws.

Try it

s3overflow: https://github.com/christoph-sens/s3overflow
sqsoverflow: https://github.com/christoph-sens/sqsoverflow
snsoverflow: https://github.com/christoph-sens/snsoverflow

Issues and pull requests are welcome. If you run into a case the libraries don't cover yet, such as another AWS service that could use the same pattern, open an issue.

I'm a freelance developer and consultant working on Kotlin, Java and AWS. The story of how these libraries came about is on my website (in German):
1150 Zeilen Legacy, eine Zeile Kotlin.