Hi dev.to community, I wanted to share an open-source project I released 2 days ago, and request for comment. The project is called JSONx Framework, and it is created to help developers work with JSON in Java. I started this project in 2015, and have invested hundreds and hundreds of hours into it. In short, the JSONx Framework provides:
- JSON Schema Definition Language (JSD), for Consumer Driven Contracts, for instance.
- A Java API for binding to JSDs, implemented with annotations to be light-coupling.
- A JSD->Java binding generator, and a Java->JSD reverse generator.
- A
jsonx-maven-plugin, for generating binding classes during thegenerate-sourcesphase, for instance. - A JAX-RS provider for
MessageBodyReaderandMessageBodyWriter, to plug into a JAX-RS runtime. - A sample project called: "Consumer Driven Contracts".
The project is thoroughly tested via a custom test framework that generates random JSON documents (392000 with each run) conforming to a schema that expresses the full declarative scope of the JSD language, across all success and failure cases.
The project is available here: JSONx.
I would very much appreciate a glance, a star, or trying it out. It has been a loooooong time coming for me to release this, and I'm excited contribute this work to the open-source community.
Top comments (6)
Hey Seva,
This sounds like it'd be a really helpful tool!
Just a quick recommendation on the post itself... you can put the tag showdev on this post as this tag is used to show off what you've built. Thinking this might help you reach more folks who are subscribed to that tag.
Hi, first, thanks for your post. It looks like a lot of work. So, I’m interested in your motivation. What did drive you to do this? Especially, what are differences to e.g. Jackson together with OpenAPI tooling? And what about testing support, did you provide assertion libs or spec based testing?
Hi Jan, thanks for your message and your interest. I would be happy to answer your questions:
In 2015, I was working on a project that required parsing complex JSON documents. There was a lot of complexity in the documents, and a lot of variability. One approach was to use lightweight parsing techniques, and another was to use a binding framework.
Lightweight parsing techniques are not great for complex JSON documents with high variability. This is because all errors (of assumptions as to what a particular property or object or whatever) end up being pushed to the runtime. That is, if there is an error with the logic, I would only catch that error in the runtime. Due to the large number and size of the documents, I needed a better solution, because otherwise there would be a significant risk that the implementation of my application would begin to accumulate yet an unknown extent of complexity due to the nature of these documents that I was yet to uncover.
A binding solution, however, would provide me a way to create a "structural model", of sorts, that would insulate the consumer logic of the JSON data from this document complexity I was yet to uncover. Using Jackson, I was able to achieve my goal.
After my experience with Jackson during the project in 2015, I started to ponder what an ideal JSON binding solution would be. The first thought that came to mind is regarding XML. XML has the power of validation (via an XML Schema). XML validation allowed me to cleanly isolate the error checking of an XML document from the application logic, as XML validation is provided by validating XML parsers. Such an approach let me never have to worry about catching all the various kinds of content errors there could be in an XML document, because the validator would do that for me.
I imagined an analogous solution for JSON.
You mention OpenAPI, which is a solution that is designed for RESTful interfaces. OpenAPI provides a certain amount of "contract scope definition", but it's specific to RESTful interfaces. If you compare OpenAPI to the XML Schema Document, it's like comparing apples and cucumbers.
The solution I imagined was a JSON Schema Document that expressed most of the same structural and logical rules and constraints as are present in the XML Schema Document.
I started working toward this goal on a project I called: JJB. Using the JJB framework, I was able to create strongly-typed bindings to JSON Schemas that allowed me to define strict constraints unto the content. JJB enabled me to implement client/server application stacks very quickly and efficiently.
I almost stopped at JJB, but it had one fault: validation of arrays. In JSON, an array can be of mixed types. That is, it can be an array of booleans, such as: [true, false, true], or an array of booleans and strings, such as: [true, "hello", false, null]. With the JSON Schema model in JJB, I was not able to express a definition for members of an array. And therefore, I would not be able to validate contents of arrays.
In 2017, I decided to give this array problem a try. And to cut to the chase, the result is the JSONx project.
How is JSONx different from Jackson?
With Jackson, you define your "structural model" in code via the ObjectMapper. This places the complexity of Java<-->JSON binding rules into your application space. For small applications dealing with small JSON object, this is no problem. But for large applications with complex JSON structures, the ObjectMapper approach ends up requiring a lot of focus of your application to be put onto Jackson-specific complexity.
Also, the same logic applies to validation with Jackson. All of the validation spec must be inside your application space. For small projects, this is fine, but for large ones, such an approach would push all the errors to the surface of the application stack -- i.e. the errors are caught amidst the application logic. I would rather first validate the JSON document in its own layer, and then, if valid, engage the application logic. If not valid, then it's a problem the JSON producer has to deal with.
Jackson also provides a way to define a JSON schema, but the schema specification is, in my opinion, cumbersome and difficult to work with. With JSONx, the JSON Schema is defined in a XSD, which you declare in a XML file. Doing it this way gives you the power of edit-time validation of your JSON Schema (with a tool such as oXygen XML).
JSONx is implemented as an analogous solution to XSD. The JSON Schema Document in the JSONx Framework supports all content constraint rules that are available in the XSD spec (the XSD spec has a lot of logical processing also, which is not implemented in JSONx). The JSON Schema Document in the JSONx Framework allows definition of array constraints, including min/max-Occurs rules and "iteration". If you're interested, here's the doc snippet regarding this.
Where did the motivation for this project come from?
I love solving very complex problems. This project actually seemed a lot easier when I started it. By the time I was in the thick of it, the drive forward was purely from the thinking: "I cannot quit half-way".
Regarding testing:
The JSONx project is thoroughly tested, both via assertion libs and spec. This code here is a test suite that I built for JSONx. It models the rules for checking success and error cases during encoding and decoding of JSON for all individual facets of constraints provided by the JSON Schema Document and the JSONx Binding API. When the test suite runs, it creates random JSON documents that models the full entropy of variability as per the individual facets. The test suite is designed to create all success and all error cases possible, and to assert that the success cases pass, and the error cases produce the expected exceptions.
There is also a test suite specific to building the Java model of the JSD (JSON Schema Document). The model I'm referring to here is the model that provides JSON Schema<-->Java Binding generation and reverse generation. The SchemaTest does every forward and reverse parsing and marshaling from the XML form of the schema to the JSON form of the schema, and to and from Java source code as well.
I've already talked your ear off. If you're interested in the testing specifically, the best thing to do is to clone the repo and check it out yourself.
Thanks again for your interest!
P.S. I'm in the middle of writing an article that will talk about this project. Hoping it will be in the hands of the editors next week. If you're interested, I can forward you a link once it's published.
Thank for your elaborate answer!
You’re right, OpenAPI is only for REST. Actually we’re currently looking into documenting JSON based messaging on rabbit MQ. Is there any support to generate documentation from the specs?
Regarding testing I meant if there is anything as a user of that lib that would help testing one’s own application.
And yes, please send me the link!
Hi Jan,
If I understand correctly (and correct me if I'm wrong): you're looking for a way to associate documentation to the constituent parts of a JSON Schema. This was supported in an earlier revision of the JSON Schema Definition spec. I removed it to reduce the overall complexity of the JSD spec, so as to help myself more easily solve other problems I was working on. But now that you mention it, I'm realizing that I had left it out. And, in fact, it absolutely needs to be brought back (because in my own documentation I advertise the JSD's ability to "to describe constraints and document the meaning, usage and relationships of the constituent parts of JSON documents". So, thank you for bringing this to my attention.
If you would not mind, could I ask you to open an issue requesting this ability to "document constituent parts of a schema"? The effort needed to re-integrate document(ability) into the JSD spec is small. As per "generating documentation from the JSD spec," I believe that the .jsd form of the schema, with the addition of a
docproperty, would naturally fit the documentation requirement. Another approach would be to use the.jsdxform of the schema, and implement a simple XSLT file that would transform the.jsdxinto an txt file or an HTML file to only show the object/property names and the documentation. An issue would be very appreciated. As I'm sure you're very well aware, I'm actively trying to bring attention to JSONx, and to get it into the hands of developers.Regarding your second question: testing. Yes, now I understand what you are referring to. I had not thought of such a concept as you're describing, but I'm realizing that the test suite that I mentioned in my previous reply can be used to test user-defined schemas also. Specifically, look at the RuntimeTest. This class contains test methods that target testing of individual objects defined in datatype.jsdx. The
RuntimeTestreferences the generated Java class bindings directly. The same concept can be applied to any other.jsdor.jsdxschema. You can use the generate goal of the jsonx-maven-plugin to generate the binding classes, and then create your own variation ofRuntimeTestto test the individual object definitions.I really like this concept actually, and think it would be very beneficial for users as a feature of itself. It would be very cool if any schema could be fully tested just by specifying a
.jsdor.jsdxfile as an input into a new goal (named "regression-test" or something) in thejsonx-maven-plugin. This way, every build of one's application would assert proper functioning of JSONx for the specified schema. If you're up for it, a feature request in issues would be highly appreciated!Hi Jan, 2 months later, my article has been published:
toptal.com/web/json-vs-xml-part-1
toptal.com/web/json-vs-xml-part-2
toptal.com/web/json-vs-xml-part-3