Improving JSON readability in Go

JSON document that fits in a single line on screen is generally easy to read, but once document grows and requires horizontal scroll it becomes much harder to understand.

{"foo":"bar"}

Indentation helps improving JSON format for human readers by putting syntactic units on separate lines with left padding.

[
  {
    "comment": "empty list, empty docs",
    "doc": {},
    "patch": [],
    "expected": {}
  },
  {
    "comment": "empty patch list",
    "doc": {
      "foo": 1
    },
    "patch": [],
    "expected": {
      "foo": 1
    }
  }
]

It works quite well for average sized documents, but tends to be inconvenient for larger documents with rich hierarchy. Larger documents are rendered using too many lines and need quite a bit of vertical scrolling to read. Often only a small fraction of screen estate is utilized for data as even small elements take a line.

A compromise could be reached with hybrid approach when smaller pieces are compacted while larger pieces are indented.

{
  "openapi":"3.0.2",
  "info":{"title":"","version":""},
  "paths":{
    "/test/{in-path}":{
      "post":{
        "summary":"Title",
        "description":"",
        "operationId":"name",
        "parameters":[
          {"name":"in_query","in":"query","schema":{"type":"integer"}},
          {"name":"in-path","in":"path","required":true,"schema":{"type":"boolean"}},
          {"name":"in_cookie","in":"cookie","schema":{"type":"number"}},
          {"name":"X-In-Header","in":"header","schema":{"type":"string"}}
        ],
        "requestBody":{
          "content":{
            "application/x-www-form-urlencoded":{"schema":{"$ref":"#/components/schemas/FormDataOpenapiTestInput"}}
          }
        },
        "responses":{"200":{"description":"OK","content":{"application/json":{"schema":{}}}}},
        "deprecated":true
      }
    }
  },
  "components":{
    "schemas":{"FormDataOpenapiTestInput":{"type":"object","properties":{"in_form_data":{"type":"string"}}}}
  }
}

Let's implement such a marshaler in Go. We can walk JSON document and apply indentation until we find leaves that are already small. We can then render those leaves in compact form.

We can use line length as a document width limitation . If padded JSON line is shorter than line length limitation, we can use compact form.

Although JSON spec does not define any semantic for properties order, it is important to preserve original order for readability. We can use github.com/iancoleman/orderedmap for that.

// MarshalIndentCompact applies indentation for large chunks of JSON and uses compact format for smaller ones.
//
// Line length limits indented width of JSON structure, does not apply to long distinct scalars.
// This function is not optimized for performance, so it might be not a good fit for high load scenarios.
func MarshalIndentCompact(v interface{}, prefix, indent string, lineLen int) ([]byte, error) {
    b, err := json.Marshal(v)
    if err != nil {
        return nil, err
    }

    // Return early if document is small enough.
    if len(b) <= lineLen {
        return b, nil
    }

    m := orderedmap.New()

    // Create a temporary JSON object to make sure it can be unmarshaled into a map.
    tmpMap := append([]byte(`{"t":`), b...)
    tmpMap = append(tmpMap, '}')

    // Unmarshal JSON payload into ordered map to recursively walk the document.
    err = json.Unmarshal(tmpMap, m)
    if err != nil {
        return nil, err
    }

    i, ok := m.Get("t")
    if !ok {
        return nil, orderedmap.NoValueError
    }

    // Create first level padding.
    pad := append([]byte(prefix), []byte(indent)...)

    // Call recursive function to walk the document.
    return marshalIndentCompact(i, indent, pad, lineLen)
}

Now in marshalIndentCompact we can use a type switch to recurse deeper in arrays and objects.

func marshalIndentCompact(doc interface{}, indent string, pad []byte, lineLen int) ([]byte, error) {
    // Build compact JSON for provided sub document.
    compact, err := json.Marshal(doc)
    if err != nil {
        return nil, err
    }

    // Return compact if it fits line length limit with current padding.
    if len(compact)+len(pad) <= lineLen {
        return compact, nil
    }

    // Indent arrays and objects that are too big.
    switch o := doc.(type) {
    case orderedmap.OrderedMap:
        return marshalObject(o, len(compact), indent, pad, lineLen)
    case []interface{}:
        return marshalArray(o, len(compact), indent, pad, lineLen)
    }

    // Use compact for scalar values (numbers, strings, booleans, nulls).
    return compact, nil
}

Arrays are represented as []interface{} and can be iterated to apply recursive formatting to all items.

func marshalArray(o []interface{}, compactLen int, indent string, pad []byte, lineLen int) ([]byte, error) {
    // Allocate result with a size of compact form, because it is impossible to make result shorter.
    res := append(make([]byte, 0, compactLen), '[', '\n')

    for i, val := range o {
        // Build item value with an increased padding.
        jsonVal, err := marshalIndentCompact(val, indent, append(pad, []byte(indent)...), lineLen)
        if err != nil {
            return nil, err
        }

        // Add item JSON with current padding.
        res = append(res, pad...)
        res = append(res, jsonVal...)

        if i == len(o)-1 {
            // Close array at last item.
            res = append(res, '\n')
            // Strip one indent from a closing bracket.
            res = append(res, pad[len(indent):]...)
            res = append(res, ']')
        } else {
            // Add colon and new line after an item.
            res = append(res, ',', '\n')
        }
    }

    return res, nil
}

JSON objects are available as orderedmap.OrderedMap that can be iterated using ordered keys to preserve properties order. Every property value is recursively processed with indentation.

func marshalObject(o orderedmap.OrderedMap, compactLen int, indent string, pad []byte, lineLen int) ([]byte, error) {
    // Allocate result with a size of compact form, because it is impossible to make result shorter.
    res := append(make([]byte, 0, compactLen), '{', '\n')

    // Iterate object using keys slice to preserve properties order.
    keys := o.Keys()
    for i, k := range keys {
        val, ok := o.Get(k)
        if !ok {
            return nil, orderedmap.NoValueError
        }

        // Build item value with an increased padding.
        jsonVal, err := marshalIndentCompact(val, indent, append(pad, []byte(indent)...), lineLen)
        if err != nil {
            return nil, err
        }

        // Marshal key as JSON string.
        kj, err := json.Marshal(k)
        if err != nil {
            return nil, err
        }

        // Add key JSON with current padding.
        res = append(res, pad...)
        res = append(res, kj...)
        res = append(res, ':')
        // Add value JSON to the same line.
        res = append(res, jsonVal...)

        if i == len(keys)-1 {
            // Close object at last property.
            res = append(res, '\n')
            // Strip one indent from a closing bracket.
            res = append(res, pad[len(indent):]...)
            res = append(res, '}')
        } else {
            // Add colon and new line after a property.
            res = append(res, ',', '\n')
        }
    }

    return res, nil
}

This API is available in github.com/swaggest/assertjson library and a CLI tool, I hope it will help to improve JSON readability for tests and other cases.

Thank you for reading.