This blog is part of a series on designing distributed applications. In this chapter, we will examine different technologies and approaches for a network API.
We can split the different approaches to a network API into the following categories:
- Binary over TCP/UDP
- Text over TCP/UDP
- JSON over HTTP
- gRPC
- GraphQL
Binary over a socket is a popular choice - Postgres, MySQL, MSSQL, Cassandra, Redis, Mongo, CockroachDB use it. You can tailor the protocol to your use-case, and get the most performance out of it. You don't lock yourself into a corporation's technology, that you have no influence over. But there are downsides - with TCP, you lose in performance, so the protocol-level optimization may be not worth it. With UDP you have to reinvent the wheel - using QUIC would be a good choice. You then need to evolve your protocol and version it, as well as all of the custom tools and drivers for it.
Text presents a lower barrier for tooling - but it is less efficient. And ultimately, you will be transmitting binary data - so you either base64 encode it, bloating its size, or you leave it as is, making the output of your protocol unreadable. Or you use two pairs of sockets - one for "control-plane" text-only queries, and one for binary data. A potential footgun is with firewalls, as with FTP.
JSON over HTTP has much more infrastructure already in place, but it is ultimately still text over TCP - and all the problems that come with it. Afterall, there is a reason MongoDB uses BSON - a binary protocol.
gRPC solves that problem, and was basically imagined as a binary JSON over HTTP - with a built-in schema. But there are issues:
- It is controlled by Google, and your needs should align with Google's needs
- You have to use ProtoBuf for schema definition and, most often, binary encoding. Most tools assume ProtoBuf on the wire
- It demands HTTP/2, even though HTTP/3 is available. Google should get around to adopting HTTP/3 at some point, but it doesn't look like they are in a hurry.
- It is not natively supported in the browser. And proxies offer a limited functionality
And finally, GraphQL. It is a powerful query language - much like SQL. But exposing SQL to a client is challenging - databases are not built for that. ACLs are hard, versioned APIs are hard, rate limiting is hard. Some of it can be solved by proxies, such as Envoy and HAProxy, but not all. And you will still want an internal representation, and a public API - may as well use a different technology. But the issues with GraphQL are the same - ACLs are hard, versioned APIs are hard, rate limiting is hard. I don't think it's a good choice for those outside Meta (it is not an independent project). Read more here.
There is also the option of a language-level RPC library, such as tRPC or RPyC. The former is a great option, if your code is Javascript/Typescript on both frontend and backend, with a bonus if your team is full-stack. If you can force the clients to upgrade, and you have dedicated APIs for every client - you can change them at will, with no versioning, no coordination. As for RPyC, because it works on object proxies - if the server or the client restarts, all the proxies break - and if they are all over your state, the whole system has to restart.
So what do we do then? I'd stick with JSON over HTTP, based on OpenAPI spec. Coupled with tools for code-generation and input (and output) validation based on the spec, this is a good universal solution - a web standard, available on all platforms, and not tied to any company. JSON is not as efficient as binary, even when gzipped - that is a tradeoff.
But there is a larger takeaway. Say you have one backend, and many clients - browser, phone, desktop. You can create dedicated APIs for every client - and, if upgrades are mandated, not version them, not coordinate with anyone, just deploy new APIs. To me, this looks close to optimal. Another option is to have a universal JSON API for all clients. But clients have different needs, and may be developed by different teams - you need versioning. And it will be hard to avoid overfetching and underfetching. You may then want to introduce a query language - but you will just be reinventing GraphQL, with all its problems. But if you want the most universal API, which can integrate with any possible application - you are looking at the "Semantic Web" protocol suite. It is worth looking at - here are a couple resources.
This is it, when it comes to networked API design - thank you for reading!
Top comments (4)
JSON over HTTP is a choice, and has been the default for many. Unfortunately it has a lot of warts (handshake, blocking, one size fits all approach) that make your advice poor. Any greenfield distributed system would be well served to understand what UDP brings to the table, why so many great systems use it, and why Google et. al. chose it for QUIC/HTTP 3. With modern tooling (serverless UDP) developing for UDP is a joy.
But QUIC/HTTP 3 is orthogonal to JSON over HTTP. And google uses gRPC, which is HTTP/2. Serverless UDP? What is that?
If you’re sending JSON over HTTP it’s more and more likely QUIC these days, even if it isn’t in the browser. .Net has it enabled by default, and curl is on the way there. My point is that HTTP is UDP now, so claims that UDP is somehow inferior are obviously misplaced. The difference is that with UDP the application layer gets to decide how to handle ordering and reliability. With TCP there is no choice.
Serverless UDP is the idea that since UDP is strictly message based it is a perfect fit for event based back ends, and serverless. That approach makes it much easier to build and scale UDP based systems.
I did not make the claim that UDP is inferior. You do need reliable, ordered delivery with flow control. So you take TCP or QUIC, as I said in the blog. But you do need framing, as both TCP and QUIC present a stream interface. Can QUIC send individual framed messages, without overhead (like WebSockets, for example)?