Introducing HTTP types

At Meeshkan, we've recorded millions of HTTP transactions. In doing so, we've noticed that there's no standard format for recording them. Even within our own company (which only has 10 people!) we managed to create competing standards for recording HTTP traffic.

To fix this, we've created an open standard for storing HTTP transactions. With it, we hope to achieve one of two outcomes:

• http-types evolves into an ISO standard, or
• Someone smarter than us will either make something better or point us to a better standard.

In the absence of either of these things for the time being, we're happy to announce http-types!

The format

The top-level type in http-types is the HttpExhange, an object with two keys:

• request: the incoming request, and
• response: the outgoing response

The following is an example of a request and response stored in the HttpExchange format:

{
  "request": {
    "method": "get",
    "protocol": "http",
    "host": "example.com",
    "headers": {
      "accept": "*/*",
      "user-agent": "Mozilla/5.0 (pc-x86_64-linux-gnu) Siege/3.0.8"
    },
    "pathname": "/user/repos",
    "query": { "param": "value" },
    "timestamp": "2018-11-13T20:20:39+01:00"
  },
  "response": {
    "statusCode": 200,
    "body": "(response body string)",
    "headers": {
      "content-length": "1999",
      "content-type": "text/html; charset=utf-8"
    },
    "timestamp": "2018-11-13T20:20:39+02:00"
  }
}

More detailed documentation can be found on the http-types GitHub repo. We also have a small website for the project with a general overview.

Client libraries

We've written http-types client libraries in the most common languages we use at Meeshkan. The current available client libraries are:

• java-http-types: Java library available on Maven Central.
• py-http-types: Python library available on PyPi.
• ts-http-types: TypeScript library available on npm.

If you'd like to contribute a client library, please propose one on our GitHub issues page.

We would love to see more people play with http-types. While it's far from perfect, it has been relatively stable within Meeshkan and we think it can serve as a good basis for community discussion on how to store web traffic.

If you find it useful or have any questions/comments/criticisms, please let us know in the comment section below.

Enjoy http-types!