How to Stream Completed JSON from OpenAI’s GPT (and Kill the Spinner)

OpenAI's GPT-4 and GPT-3.5 now allow developers to receive responses in structured JSON format, offering new UI possibilities. But, this feature has its challenges, especially with streaming outputs. When responses are streamed from the language model in JSON mode, they often result in incomplete JSON structures that can't be parsed.

This leads to the need for spinners while waiting for the complete JSON, which can slow down the user experience. For instance, in listing available fruits from a partially streamed JSON, a spinner is used to indicate the response is still generating.

Introducing json-autocomplete

JSON Autocomplete allows you to input an incomplete JSON string, which it completes for you:

This means you can directly show the AI's output in your UI, making your product feel faster.

Setting Up json-autocomplete

Integrating JSON Autocomplete is straightforward. First, install it using npm or yarn:

npm install json-autocomplete

Then, import it into your project and use it to convert incomplete JSON responses into complete ones.

import JSONAutocomplete from 'json-autocomplete';

const jsonResponseFromGPT = // your partially streamed JSON response from GPT;

const completeJsonString = JSONAutocomplete(jsonResponseFromGPT);

Remember to parse the completed JSON string:

const parsedJson = JSON.parse(completeJsonString);

Now, you can use parsedJson[0]?.name as soon as it streams from the modal.

Applications and Motivations

JSON Autocomplete improves user experience by quickly presenting data and enables new UI design possibilities, as demonstrated in the Operator app.

This is still a work in progress so if any bugs arise create an issue in the repo: https://github.com/Operator-technology/json-autocomplete