Introducing Ketting: A Hypermedia API

Introducing Ketting: A Hypermedia driven API

I was recently invited to join a really interesting project by a colleague with a lot of deep experience in API development and HTTP protocols. The project had all the usual elements of a CRUD application, with one notable exception that the API layer was using Ketting: A new hypermedia-oriented API. Whats more exceptional is that this API layer was created by that same colleague, Evert Pot. Below is a bit of a summary of my experience with Ketting, documenting my impressions and comparing it to API's that I am more familiar with (specifically REST and GraphQL).

From the Ketting docs:

The Ketting library is an attempt at creating a 'generic' hypermedia client, it supports an opinionated set of modern features REST services might have.
The library supports HAL, JSON:API, Siren, Collection+JSON, Web Linking (HTTP Link Header) and HTML5 links. It uses the Fetch API and works both in the browsers and in node.js.

What is a Hypermedia API? Great question! Lets dive into that a little bit to understand the different parts that go into it.

HAL, which stands for Hypertext Application Language, is a language specification for defining hypermedia, which refers to links, graphics, videos, etc on the internet. By using HAL to drive your API, the general idea is that it makes your API auto-discoverable, more inclined to self-documentation via following relationships, and easier to work with.
An example of how this would look is shown below. Just by looking at this resource, you can infer a whole bunch about that this resource and its relationships!

{
    "_links":{
        "self":{
            "href":"/author/4554"
        },
        "curries":[
            {
                "name":"ns",
                "href":"http://booklistapi.com/rels/{rel}",
                "templated":true
            }
        ]
    },
    "_embedded":{
        "ns:books":[
            {
                "_links":{
                    "self":{
                        "href":"/books/123"
                    }
                },
                "Title":"RESTful Web APIs",
                "Price":"$31.92"
            },
            {
                "_links":{
                    "self":{
                        "href":"/books/366"
                    }
                },
                "Title":"RESTful Web Services",
                "Price":"$79.78"
            },
            {
                "_links":{
                    "self":{
                        "href":"/books/865"
                    }
                },
                "Title":"Ruby Cookbook",
                "Price":"$34.35"
            }
        ]
    }
}

In Ketting, the API uses the concept of a Resource which contain a url and some functionality for performing CRUD operations against that resource. The resource is your main entrypoint for dealing with objects/relations in Ketting. Lets take a look at the following example:

const ketting = new Ketting('https://api.example.org/');
const createArticle = await ketting.follow('articleCollection').follow('new'); // chained follow

const newArticle = await createArticle.post({ title: 'Hello world' });
const author = await newArticle.follow('author');

// Output author information
console.log(await author.get());

In this example, we have our API base URL which is passed to the Ketting client. From there, we can follow different links down various paths (and even chain those follows). We can post new resources, and follow their links (or add new links), and we can get JSON-encoded result of that input from the resource.

Following links is a central theme of Ketting. Links are discoverable, in that a resource can have a list of links, and each of those links will point to further endpoints or features of the API.

Ok....But can it GET the POST? 

Lets be honest: I am a simple man, who thinks in simple terms. The two biggest comparisons for Ketting to me are GraphQL and REST. It feels like it has features of both, while maintaining a simple interface. As long as you have a resource mapped to some object or piece of data (for example, an Author, Article or ArticleCollection Resource), you can perform your familiar PUT, POST, DELETE, and GET requests against that resource using Kettings API. Using these is quite simple and provides a much cleaner interface than building out a fetch middleware or all that boilerplate that comes with GraphQL. Ketting is quite strictly typed with Typescript as well, which makes it easy to debug when things go wrong or when you may be mis-using resources. Below is an example of a post request with Ketting which returns a new Author Resource.

const authorCollection = await authorResource.follow('author-collection');

const newAuthor:Author = {
  firstName: 'Ursula',
  lastName: 'Le Guin',
};

const newAuthorResource = await authorCollection.post(newAuthor);

The Confusing Parts

Of course, with any new paradigm or piece of technology, there are still things I struggle with. Part of writing this piece was making those gaps more clear to myself and seeing how well I understood Ketting.

The main issues I ran into included understanding the concept of a resource better, and knowing what kind of resource I had at which times. Our project dealt with Videos, VideoLists, and Video-metadata and sometimes those 3 different resources were used differently but had similar features to their API. To be fair, I am also fairly new to Typescript and could have leaned more heavily on examining/understanding the types better when I got stuck. A resource object was more similar to an HTTP request (which is by design) rather than just representing the JSON data it represented, which had me sometimes confused as to how to best reconcile/couple those two during the lifecycle of some component/function. It wasn't immediately obvious how those two pieces (the resource and its data) fit together, especially as complexity increases. After some use, I understand that part better now, and especially with Kettings implementation of hooks in React, the result is quite elegant.

Understanding relationships and knowing when to add/use links was another area I had a bit of trouble with. While its possible to add links arbitrarily to resources from the front-end, this has to be done correctly and in collaboration with the back-end to ensure resource/model integrity. And finally, I had trouble considering use-cases outside of what is shown to me. Granted, I am not as familiar with the deeper concepts of HAL, JSON:API, Siren, Collection+JSON, Web Linking, and differences in HTML5 linking, but I think getting to understand the specific scenarios where Ketting shines would be really helpful to me in utilizing it to its fullest.

Finally, Caching. I think caching is a hard-problem generally, and there were some cases were Ketting solved it quite nicely with adding a preferTransclude() and .refresh() methods. I think this is a case where you almost have to see the bug first (e.g. not refreshing a list) before you can turn back to the docuemntation to look for the resolution (including .refresh()).

The Awesome Sauce

In general, I found Ketting very easy to use, and far less verbose than a GraphQL or (axios, fetch middleware) counterpart. Since it was also active development by my colleague, it was very cool getting to see it being built and in action, with lessons learned and fixes pushed as we developed our project. I was even able to contribute to it directly myself, when I found that I needed a specific feature of it to work a certain way for me. Battle-testing a new API is not easy, but suffice to say we had learned a lot about API's and the specific issues that need to be addressed.

Learning about Ketting and using it made me think about HTTP1 vs HTTP2 and caching in a new way as well. Fully understanding the implications of a properly used HTTP2 protocol (hint: caching and optimization!) is something that is more top-of-mind now when architecting new applications. Using the HAL media API client was also awesome and, much like graphql playground, allowed me to get a visual understanding of the API model and answer questions quite quickly, which was very helpful. Indeed, the ability for a Ketting API to generate its own GUI via HAL shows just how powerful and how much potential it has. As long as you have a resource or a URL, you can do a lot of powerful things, and write much less verbose code to get things done.

The Ketting react hooks were probably my favorite part, and helped trim down many of our components substantially. They worked very similar to their GraphQL or useQuery counterparts, while also returning a resource object for traversing links and performing other operations on the resource.

function ArticleView() {

  const { loading, error, data } = useResource<Article>('/article/1');
  if (loading) return <p>Loading...</p>;
  if (error) return <div class="error">{error.message}</div>;

  return <article>
    <h1>{data.title}</h1>
    <p>{data.body}</p>
  </article>;

}

Having strictly typed resources made it easier to understand/debug when things went wrong or when I was using a resource incorrectly, and in general has guided me towards more Typescript in my future work. There were no surprises in what I expected from my resource, and when it came time to refactor the project and migrate to a newer version of Ketting, the process was made much simpler thanks to strict typing.

Implementing Oauth2 was by far probably one of the easiest implementations I've had, and seeing that part in action was pretty great as this is something that developers battle with often in new projects.

import { Client, oauth2 } from 'ketting';

const client = new Client('api.example.org');
client.use(oauth2({
  grantType: 'password',
  clientId: 'fooClient',
  clientSecret: 'barSecret',
  tokenEndpointUri: 'https://api.example.org/oauth/token',
  scopes: ['test']
  userName: 'fooOwner',
  password: 'barPassword'
});

Repeating my first point about the verbosity of the GET/POST requests, it really is a much nicer experience doing a GET, PUT or POST request to a Ketting resource without all the graphQL boilerplate thats needed or managing a REST middleware layer like axios or some kind of customized fetch, which often ends up getting built. It "just works", and provides helpful error messages when it doesn't.

My Short conclusion

Use Ketting! It will challenge your preconceptions of REST, HTTP, GraphQL, and how data moves between client and server. If you are interested in contributing, there is more then enough work that can be contributed! Finally, since hooks were just added to ketting, now wold be a great time to try it out and see those in action.