How to Make OpenAI API to Return JSON

During OpenAI's dev day, one of the major announcements was the ability to receive a JSON from the chat completion API. However, there aren't a few clear examples of how to do this as most examples focus on function calls.

Our objective is straightforward: given a query, we want to receive an answer in JSON format.

How can we achieve this?

There are three crucial steps.

Modify your prompt

Your prompt must explicitly specify that the response should be in JSON format and you need to define the structure of the JSON object.

Given this work history, what's the overall years of experience?
Work history includes start and end date (or present), title, and company name.
If the end date is "Present", use the current date. Today is November 2023.
Return the answer in JSON format with the field "experienceInMonths"
and value as a number.

Pay attention to the last sentence of the prompt.

Pass response_format

When calling the API, specify the response_format.

const res = await this.openAI.chat.completions.create({
  model: 'gpt-3.5-turbo-1106',
  temperature: 0.0,
  top_p: 1,
  frequency_penalty: 0,
  presence_penalty: 0,
  response_format: {
    type: 'json_object', // specify the format
  },
  messages: [
    { role: 'system', content: systemPrompt },
    { role: 'user', content: workHistory },
  ],
});

It's crucial to modify the prompt as well. Just changing the response type to JSON might result in a JSON of an arbitrary structure.

See this comment from the OpenAI API:

**Important:** when using JSON mode, you **must** also instruct the model to
produce JSON yourself via a system or user message. Without this, the model may
generate an unending stream of whitespace until the generation reaches the token
limit, resulting in increased latency and the appearance of a "stuck" request.
Also note that the message content may be partially cut off if
`finish_reason="length"`, which indicates the generation exceeded `max_tokens`
or the conversation exceeded the max context length.

Parse JSON response

Once we receive the response, the content is still text (string type), but we can now parse it as JSON.

const content: string = get(res, 'choices[0].message.content');
try {
  return JSON.parse(content)['experienceInMonths'];
} catch (error) {
  this.logger.warn('Calculating total experience for a member did not work');
  return -1;
}

It's good practice to wrap JSON.parse in a try...catch statement in case we receive an invalid JSON structure.

You can find a playground example here.