DEV Community

kieronjmckenna
kieronjmckenna

Posted on β€’ Originally published at blog.opinly.ai

1

The Best Way to Use Open AI and Typescript

(Update April 2024 - when this article was written OpenAI didn't have function calling within tools, read to the end for the updated code)

In this article you'll find out how to fallback from gpt4 turbo preview to gpt 4 when using function calling (Relevant November 2023 as gpt4-turbo is in preview and heavily rate limited), and with that response validate and type your function calling responses


The function calling feature is very powerful (we use it all the time to turn unstructured data into structured data) as it allows us to get JSON data for whatever function we want to call. But it's still possible that GPT could hallucinate bad JSON data, or the wrong type, or a wrong field name...I could go on (just last week the JSON coming back couldn't be parsed because it was malformed when telling GPT4 to use emojis).

Besides those outliers, we want our response from GPT to be typed for use elsewhere in our application rather than just having it as "any".

So with that context let's get into the code. We're going to use an example where we're taking in a question from a user asking questions on an e-commerce site as a way to show multiple functions we might call based on the user's question.

Code

Real quick install OpenAI:

pnpm i openai
view raw install-openai.zsh hosted with ❀ by GitHub

and instantiate the OpenAI client:

import { OpenAI } from "openai";
export const openAI = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
view raw openai.ts hosted with ❀ by GitHub

GPT4 Turbo to GPT4 Fallback

Now we want to set up the ability to fallback from one GPT model to another. As I mentioned before, it's currently November 2023 and GPT4 Turbo is still in preview and rate limited. So for going to prod, we need to have a fallback to plain old GPT4 πŸ˜”

import { openAI } from "./clients/open-ai";
type Messages = Parameters<
typeof openAI.chat.completions.create
>[0]["messages"];
type Functions = Parameters<
typeof openAI.chat.completions.create
>[0]["functions"];
type FunctionCall = Parameters<
typeof openAI.chat.completions.create
>[0]["function_call"];
export const openAICompletionsCallWithFallback = async ({
function_call,
functions,
messages,
}: {
messages: Messages;
function_call: FunctionCall;
functions: Functions;
}) => {
try {
return openAI.chat.completions.create({
model: "gpt-4-1106-preview",
function_call,
functions,
messages,
});
} catch (gpt4PreviewError) {
console.error(
new Error("Continuing to gpt4 base, Error with gpt preview model")
);
try {
return openAI.chat.completions.create({
model: "gpt-4",
function_call,
functions,
messages,
});
} catch (gpt4err) {
console.error(new Error("Error with gpt-4 model"));
throw gpt4err;
}
}
};
view raw openai-fallback.ts hosted with ❀ by GitHub

So just a few try catches and we've got our fallback setup. You could use this for falling back from any model to another.
You might be wondering why I didn't use Omit<..., "model"> rather than passing each of the parameters individually. I tried this, spreading omitted objects into the OpenAI call, but it broke the return type as it uses generics internally based on whether you pass functions in... so try to get that working at your own peril.

Now, time to make our type-safe call to OpenAI.

import { z } from "zod";
import { openAI } from "./clients/open-ai";
import { openAICompletionsCallWithFallback } from "./completions-call-with-fallback";
type Functions = Exclude<
Parameters<typeof openAI.chat.completions.create>[0]["functions"],
undefined
>;
type FunctionCall = Functions[number];
const productInquiryFunctionName = "productInquiry";
const productRecommendationFunctionName = "productRecommendation";
const postPurchaseSupportFunctionName = "postPurchaseSupport";
type AvailableFunctions =
| typeof productInquiryFunctionName
| typeof productRecommendationFunctionName
| typeof postPurchaseSupportFunctionName;
const ProductInquiryResponseSchema = z.object({
name: z.string(),
description: z.string(),
price: z.number(),
});
type ProductInquiryResponse = z.infer<typeof ProductInquiryResponseSchema>;
const productInquiryFunctionCall: FunctionCall = {
name: productInquiryFunctionName,
description: "Retrieve information about a product",
parameters: {
type: "object",
required: ["name", "description", "price"],
properties: {
name: {
type: "string",
description: "Name of the product",
},
description: {
type: "string",
description: "Description of the product",
},
price: {
type: "number",
description: "Price of the product",
},
},
},
};
const handleProductInquiry = async (inquiry: ProductInquiryResponse) => {
// do something here
};
const ProductRecommendationResponseSchema = z.object({
products: z.array(
z.object({
name: z.string(),
description: z.string(),
productLink: z.string().url(),
})
),
});
type ProductRecommendationResponse = z.infer<
typeof ProductRecommendationResponseSchema
>;
const productRecommendationFunctionCall: FunctionCall = {
name: productRecommendationFunctionName,
description: "Retrieve recommended products",
parameters: {
type: "object",
required: ["products"],
properties: {
products: {
type: "array",
description: "Recommended products",
items: {
type: "object",
properties: {
name: {
type: "string",
description: "Name of the product",
},
description: {
type: "string",
description: "Short description of the product",
},
productLink: {
type: "string",
description: "Link to the product",
},
},
},
},
},
},
};
const handleProductRecommendation = async (
recommendation: ProductRecommendationResponse
) => {
//typesafe
recommendation.products.forEach((product) => {
console.log(`${product.name} - ${product.description}`);
});
};
const PostPurchaseSupportResponseSchema = z.object({
description: z.string(),
troubleshootingSteps: z.array(z.string()),
contactInfo: z.string().optional(),
});
type PostPurchaseSupportResponse = z.infer<
typeof PostPurchaseSupportResponseSchema
>;
const postPurchaseSupportFunctionCall: FunctionCall = {
name: postPurchaseSupportFunctionName,
description: "Retrieve support information for a product",
parameters: {
type: "object",
required: ["description", "troubleshootingSteps"],
properties: {
description: {
type: "string",
description: "Description of the issue",
},
troubleshootingSteps: {
type: "array",
description: "Steps to troubleshoot the issue",
items: {
type: "string",
description: "Step to troubleshoot the issue",
},
},
contactInfo: {
type: "string",
description: "Contact info for support",
},
},
},
};
const handlePostPurchaseSupport = async (
support: PostPurchaseSupportResponse
) => {
// typesafe
support.contactInfo;
support.description;
support.troubleshootingSteps;
};
interface MapParserToFunction<T = any> {
function: (parsedResponse: T) => Promise<void>;
parse: (data: unknown) => T;
}
const parserAndFunction: Record<AvailableFunctions, MapParserToFunction> = {
[productInquiryFunctionName]: {
parse: ProductInquiryResponseSchema.parse,
function: handleProductInquiry,
},
[productRecommendationFunctionName]: {
parse: ProductRecommendationResponseSchema.parse,
function: handleProductRecommendation,
},
[postPurchaseSupportFunctionName]: {
parse: PostPurchaseSupportResponseSchema.parse,
function: handlePostPurchaseSupport,
},
} as const;
const functions: Functions = [
productInquiryFunctionCall,
productRecommendationFunctionCall,
postPurchaseSupportFunctionCall,
];
const prompt = (userQuestion: string) => `
Help this user with their question about our product:
User Question: ${userQuestion}
`;
const answerUserQuestionAboutOurEcommerceProduct = async (question: string) => {
const competitorData = await openAICompletionsCallWithFallback({
messages: [
{
role: "system",
content:
"You are an assistant helping us answer questions from users on our ecommerce website",
},
{ role: "user", content: prompt(question) },
],
function_call: "auto",
functions,
});
await Promise.all(
competitorData.choices.map((res) => {
const functionName = res.message.function_call?.name as
| AvailableFunctions
| undefined;
if (functionName) {
const { parse, function: functionToExecute } =
parserAndFunction[functionName];
const functionResponse = res.message.function_call?.arguments;
if (functionResponse) {
const parsedResponse = parse(JSON.parse(functionResponse));
return functionToExecute(parsedResponse);
}
}
})
);
};

So... a lot of code there, but let's break it down.

  1. At the top of the file we're defining both the data we want to come back from open ai for each function and a mirror schema in Zod for each type to be passed into our functions.

  2. We create a map for all the functions and the parser to sanitise the data coming back from OpenAI.

  3. We use the function we defined above to make the call with a fallback from GPT4 turbo to regular GPT4. If you're reading this in the future you will most likely not need this, but who knows maybe you'll swap out the models from GPT4 turbo and GPT4 to GPT5 turbo and GPT5

  4. We use the map we defined to call the function with the data that's sanitised with the appropriate parser. We're using Promise.all here as Open AI just announced that function calling can return multiple function calls, but adjust to your use case

Remember this is just an example to show how to make an Open AI calls type-safe and the whole e-comm customer support thing is erroneous. This has been very useful for us when we forgot to mark fields as required in the open AI schema but they came back as undefined, or catching random bad responses from Open AI.

From there, you're good to integrate OpenAI into your typescript project and be able to sleep at night knowing that the data is at least in the right format as it flows through your program.

Update April 2024

Since writing the article OpenAI has changed their APIs slightly to move function calling within tools, and we've also figured out how to reduce the duplication between Zod and the JSON schemas.

Updating Functions To Tools in OpenAI

Updating our fallback function to using tools looks like this:

import { openAI } from "<path-to-your-open-ai-client>/clients/open-ai";
export type StripNullish<T> = Exclude<T, null | undefined>;
type Messages = Parameters<
typeof openAI.chat.completions.create
>[0]["messages"];
export type Tools = Parameters<
typeof openAI.chat.completions.create
>[0]["tools"];
type ToolChoice = StripNullish<
Parameters<typeof openAI.chat.completions.create>[0]["tool_choice"]
>;
export const openAICompletionsCallWithFallback = async ({
tool_choice,
tools,
messages,
}: {
messages: Messages;
tool_choice: ToolChoice;
tools: Tools;
}) => {
try {
const openAIRes = await openAI.chat.completions.create({
model: "gpt-4-turbo-preview",
messages,
tool_choice: tool_choice,
tools,
n: 1,
});
return openAIRes;
} catch (gpt4PreviewError) {
console.error(
new Error("Continuing to gpt4 base, Error with gpt preview model")
);
try {
const gptRes = await openAI.chat.completions.create({
model: "gpt-4-0613",
tool_choice: tool_choice,
tools,
messages,
});
return gptRes;
} catch (gpt4err) {
console.error(new Error("Error with gpt-4 model"));
throw gpt4err;
}
}
};

Not much of a change, just moving away from the function calling API to use the new Tools API.

Now we can change our call to OpenAI:

import { z } from "zod";
import {
openAICompletionsCallWithFallback,
Tools,
} from "./completions-call-with-fallback";
import { zodToJsonSchema } from "zod-to-json-schema";
export type StripNullish<T> = Exclude<T, null | undefined>;
type Tool = StripNullish<Tools>[number];
const productInquiryFunctionName = "productInquiry";
const productRecommendationFunctionName = "productRecommendation";
const postPurchaseSupportFunctionName = "postPurchaseSupport";
type AvailableFunctions =
| typeof productInquiryFunctionName
| typeof productRecommendationFunctionName
| typeof postPurchaseSupportFunctionName;
const ProductInquiryResponseSchema = z.object(
{
name: z.string({
description: "Name of the product",
}),
description: z.string({
description: "Short description of the product",
}),
price: z.number({
description: "Price of the product",
}),
},
{
description: "Information about a product",
}
);
const ProductInquiryResponseSchemaJson = zodToJsonSchema(
ProductInquiryResponseSchema
);
type ProductInquiryResponse = z.infer<typeof ProductInquiryResponseSchema>;
const productInquiryFunctionCall: Tool = {
type: "function",
function: {
name: productInquiryFunctionName,
description: "Retrieve information about a product",
parameters: ProductInquiryResponseSchemaJson,
},
};
const handleProductInquiry = async (inquiry: ProductInquiryResponse) => {
// do something here
};
const ProductRecommendationResponseSchema = z.object(
{
products: z.array(
z.object({
name: z.string({
description: "Name of the product",
}),
description: z.string({
description: "Short description of the product",
}),
productLink: z
.string({
description: "Link to the product",
})
.url(),
})
),
},
{
description: "Recommended products",
}
);
const ProductRecommendationResponseSchemaJson = zodToJsonSchema(
ProductRecommendationResponseSchema
);
type ProductRecommendationResponse = z.infer<
typeof ProductRecommendationResponseSchema
>;
const productRecommendationFunctionCall: Tool = {
type: "function",
function: {
name: productRecommendationFunctionName,
description: "Retrieve recommended products",
parameters: ProductRecommendationResponseSchemaJson,
},
};
const handleProductRecommendation = async (
recommendation: ProductRecommendationResponse
) => {
//typesafe
recommendation.products.forEach((product) => {
console.log(`${product.name} - ${product.description}`);
});
};
const PostPurchaseSupportResponseSchema = z.object(
{
description: z.string({
description: "Description of the issue",
}),
troubleshootingSteps: z.array(
z.string({
description: "Step to troubleshoot the issue",
}),
{
description: "Steps to troubleshoot the issue",
}
),
contactInfo: z
.string({
description: "Contact info for support",
})
.optional(),
},
{
description: "Support information for a product",
}
);
type PostPurchaseSupportResponse = z.infer<
typeof PostPurchaseSupportResponseSchema
>;
const PostPurchaseSupportResponseSchemaJson = zodToJsonSchema(
PostPurchaseSupportResponseSchema
);
const postPurchaseSupportFunctionCall: Tool = {
type: "function",
function: {
name: postPurchaseSupportFunctionName,
description: "Retrieve support information for a product",
parameters: PostPurchaseSupportResponseSchemaJson,
},
};
const handlePostPurchaseSupport = async (
support: PostPurchaseSupportResponse
) => {
// typesafe
support.contactInfo;
support.description;
support.troubleshootingSteps;
};
interface MapParserToFunction<T = any> {
function: (parsedResponse: T) => Promise<void>;
parse: (data: unknown) => T;
}
const parserAndFunction: Record<AvailableFunctions, MapParserToFunction> = {
[productInquiryFunctionName]: {
parse: ProductInquiryResponseSchema.parse,
function: handleProductInquiry,
},
[productRecommendationFunctionName]: {
parse: ProductRecommendationResponseSchema.parse,
function: handleProductRecommendation,
},
[postPurchaseSupportFunctionName]: {
parse: PostPurchaseSupportResponseSchema.parse,
function: handlePostPurchaseSupport,
},
} as const;
const tools: Tools = [
productInquiryFunctionCall,
productRecommendationFunctionCall,
postPurchaseSupportFunctionCall,
];
const prompt = (userQuestion: string) => `
Help this user with their question about our product:
User Question: ${userQuestion}
`;
const answerUserQuestionAboutOurEcommerceProduct = async (question: string) => {
const ecommerceCompletion = await openAICompletionsCallWithFallback({
messages: [
{
role: "system",
content:
"You are an assistant helping us answer questions from users on our ecommerce website",
},
{ role: "user", content: prompt(question) },
],
// Select a specific tool to use if needed
// tool_choice: {
// type: "function",
// function: {
// name: productInquiryFunctionName
// }
// },
tools,
});
await Promise.all(
ecommerceCompletion.choices.map((res) => {
const functionName = res.message.function_call?.name as
| AvailableFunctions
| undefined;
if (functionName) {
const { parse, function: functionToExecute } =
parserAndFunction[functionName];
const functionResponse = res.message.function_call?.arguments;
if (functionResponse) {
const parsedResponse = parse(JSON.parse(functionResponse));
return functionToExecute(parsedResponse);
}
}
})
);
};

We're still using the example from above with a few minor changes.

Tools API

We're now passing an array of objects that specify the type of tool we're calling. For now, this is always "function" but in the future, there may be options to use data analytics and more.

 Inferring JSON Schemas from Zod

In the original example, we were writing out the JSON schemas to match our zod parsers, but with the help of zod-to-json-schema we can now infer the descriptions and rules directly from Zod.

Thanks for reading!

API Trace View

How I Cut 22.3 Seconds Off an API Call with Sentry πŸ•’

Struggling with slow API calls? Dan Mindru walks through how he used Sentry's new Trace View feature to shave off 22.3 seconds from an API call.

Get a practical walkthrough of how to identify bottlenecks, split tasks into multiple parallel tasks, identify slow AI model calls, and more.

Read more β†’

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs