Replacing complex UX patterns with Next.js and OpenAI o1

Charly Poly — Thu, 17 Oct 2024 16:02:39 +0000

Anyone who has already built a data-import user experience knows it requires some complex column-matching UI and many branches on the backend to safely parse and save the user-provided data:

Such UI is a slow and tedious experience for the end-user and a challenge to maintain for developers.

In this article, we will review a demo CRM Next.js putting a contacts import feature on autopilot, remarkably demonstrating the power of agentic workflows.

What is an AI Agentic workflow?

An Agentic workflow is a new pattern consisting of relying on model reasoning to achieve a goal instead of providing a series of static prompts.
In short, the model is given a high-level goal to achieve and figures out the necessary steps to reach that goal.

Many papers and articles have been published earlier this year, showcasing the outstanding performance of agentic workflows compared to standard prompting techniques.

It turns out that agentic workflows, by combining reflection (reasoning), planning, and tools, lead to more precise answers, solving important integration issues of AI in user-facing use cases.

We are now witnessing the rise of new tools that help with agentic workflow developments with LangGraph and, recently, OpenAI's swarm, along with their new reasoning model: o1.
Adding Agentic workflows to an application comes with many challenges:

Duration volatility: Models are responsible for reflecting and planning all the steps to reach a given goal, resulting in significant variance in the workflow duration.
Reliability: Agentic workflows rely on more tools than regular prompt-designed workflows. Adding more tools calling external APIs based on LLM-generated parameters raises the risk of failure. Recovering from failure enables the model to self-correct during the workflow execution.
Unpredictability: Pushing an agentic workflow to production requires setting up some safeguards on cost, duration, and output validation.

Let's now look at our Next.js CRM demo, featuring an autopilot contacts import feature.

We'll see that agentic workflows mostly reuse existing patterns found in Next.js and can already be used today to bring brand-new capabilities to Next.js applications.

Designing an agentic workflow with OpenAI o1

We will design our CRM's autopilot contact importer feature by leveraging the new OpenAI o1 reasoning model to plan the best steps to import a given file by:

Ensuring that the fields are properly set, without typos
Ranking the provided contact as good software Sales lead dynamically based on the provided columns
Matching provided column by our CRM ones: Name, Email, Role, Company

Open AI o1 will take this high-level goal to generate a set of steps leveraging our existing handwritten actions (convert, enrich, save) and generating custom OpenAI calls (if necessary) while managing cost.

Here's a visualization of our autopilot import contacts feature in our Next.js application:

Creating our contacts importer UI with Vercel v0

Our import contacts UI has been, of course, generated with Vercel v0:

Once a file is selected, we call a Server Action with the FormData, which triggers a ”contacts.uploaded” event to Inngest.

"use server";

import { inngest } from "@/lib/inngest/client";

export async function uploadFile(formData: FormData) {
  const file = formData.get("file") as File;
  const fileText = await file.text();

  await inngest.send({
    name: "contacts.uploaded",
    data: {
      contactFileContent: fileText,
    },
  });
}

The front end does not expect the import to be instantaneous, and also, because the call to OpenAI's o1-preview may take multiple minutes, the actual processing and generation of the contact import workflow are performed in the background by an Inngest Function: generateImportWorkflow(). Let's look at its implementation.

Asking OpenAI o1 to generate an import workflow

Our first Inngest Function, generateImportWorkflow(), leverages Vercel Edge Function's streaming capabilities to call OpenAI's o1-preview model and generate a tailored import workflow for our contact list without duration limitation.

Let's take a closer look at our o1-preview prompt:

const prompt = (contactsFileContent: string) => `
Considering the following available actions:
${actions
  .filter(({ kind }) => kind !== "openaiCall")
  .map(({ kind, description }) => `- ${description}, name: ${kind}\n`)}

and, given the below CSV file (between \`\`\`), recommend the best steps by balancing existing actions and using custom openai calls (do not use deprecated models) to successfully:
    - parse the contact information and remove any typos
    - enrich the contacts data
    - label them a decider and rank them as a good Sales target to sale software to based on the provided property
    - rework the CSV file to match the provided column to the following: Name, Position, Company, Email and the ranking and decider columns
    - save the contacts to the database

Return a JSON array made of steps to execute. When a step is a model call, provide the "model" for model name and "prompt" for the prompt with "{data}" as a placeholder for the provided data; When the step is an action, provide the action name.

\`\`\`
${contactsFileContent}
\`\`\`
`;

The above prompts demonstrate the power of reasoning models like o1-preview. From high-level goals, the model takes a few minutes to find the best chain of predefined actions and additional model calls to perform to successfully parse, transform, rank, and save the contacts.

We are using the stellar Vercel AI SDK to call o1-preview. Here is an example of the actions generated by the model:

[
  {
    "action": "convert"
  },
  {
    "model": "gpt-3.5-turbo",
    "prompt": "Parse the following contact information from JSON, correct any typos, and ensure all fields are properly formatted."
  },
  {
    "action": "enrich"
  },
  {
    "model": "gpt-3.5-turbo",
    "prompt": "Label each contact as a decider and rank them as a good sales target for software based on their company, role, and industry."
  },
  {
    "model": "gpt-3.5-turbo",
    "prompt": "Reformat the following JSON data to match the columns: Name, Position, Company, Email."
  },
  {
    "action": "save"
  }
]

The actions generated will vary depending on the provided CSV file.
The code then transforms these actions into a format using Inngest's Workflow Kit which can easily handle o1's multi-step plans and execute them. The transformed plan is then send to the importContacts Inngest Function to execute the plan.

Note: o1-preview vs. o1-mini accuracy

o1-mini is a preferred choice when developing, offering faster iterations and lower costs. While developing this demo, we realized that o1-mini struggled to output a deterministic and simple JSON structure for model call actions. Switching back to o1-preview solved this problem.

Our o1-preview call and the processing of its results is wrapped as an Inngest Function, enabling it to run without duration limitation in the background and matching o1-preview's API Rate limits by leveraging throttling:

// imports ...

export default inngest.createFunction(
  {
    id: "generate-import-workflow",
    throttle: {
      limit: 5000,
      period: "1m",
    },
  },
  { event: "contacts.uploaded" },
  async ({ event, step }) => {
    const generatedStepsResult = await step.run(
      "openai-o1-generate-steps",
      async () => {
         // ...
      }
    )
  )

Access the full source code here.

Executing the dynamic workflow with Inngest's Workflow Kit

Our importContact() Inngest Function leverages the Inngest Workflow Kit to execute a dynamic list of steps passed in the ”contacts.process” event's body.

Our Inngest workflow is composed of 4 predefined actions:

openaiCall: Make a dynamic call to OpenAI API
convert: convert CSV data to JSON items
enrich: Enrich contact information
save: Save contact information to the database

Our o1-preview model leverages the openaiCall action to dynamically add actions, helping to format and structure the provided contact data. It takes two parameters: prompt, and model which the model configures according to the task to achieve.

Each workflow's action provides a handler() function that benefits from Inngest's features, such as automatic retries and throttling. These features improve the reliability of our agentic workflow and help manage its unpredictability.

An Inngest Workflow is then passed to our importContacts() Inngest Function for execution:

import { Engine, EngineAction } from "@inngest/workflow-kit";
import { inngest } from "../client";

export const actions: EngineAction[] = [
  // other actions ...
  {
    kind: "save",
    name: "Save contacts",
    description: "save contact information to the database",
    handler: async ({ state, step }) => {
      await step.run("save-contacts-to-database", async () => {
        const contacts = JSON.parse(state.get("contacts"));
        await sql.query(
          `INSERT INTO contacts (Name,Position,Company,Email,Decider,Ranking) VALUES ${contacts
            .map((contact: any) => {
              return `('${contact.Name}', '${contact.Position}', '${contact.Company}', '${contact.Email}', ${contact.Decider}, ${contact.Ranking})`;
            })
            .join()}`
        );
      });
    },
  }
];

const workflowEngine = new Engine({
  actions,
  loader: (event) => {
    return event.data.workflowInstance;
  },
});

export default inngest.createFunction(
  { id: "import-contacts" },
  { event: "contact.process" },
  async ({ event, step }) => {
    // When `run` is called, the loader function is called with access to the event
    await workflowEngine.run({ event, step });
  }
);

Access the full source code here.

How our Agent performs

Our demo CRM application now features a smooth AI-powered contact import onboarding experience powered by a semi-autonomous agentic workflow as a complete part of the Next.js application ✨

Uploading the below CSV file sample generates a custom import workflow and runs smoothly in the background:

First Name,Last Name,Email,Seniority,Role,Company,Industry
Amanda,Brown,amanda.brown@example.com,VP,Data Scbentist,AutoCorp,Real Estate
Sarah,Johnson,sarah.johnson@example.com,Junior,CEO,BioGen,Education
Michael,Jackson,michael.jackson@example.com,Junior,CqO,AutoCorp,Healthcare

Notice the typos in position, roles, and columns that do not match our contact properties.

Once completed, the clean, structured, and enriched contact records are saved to the database:

Vercel Postgres database with the contact information loaded from the agentic workflow. Our Agent event fixed the typos in contact properties like "Scbentist"

When deployed on Vercel, the workflow leverages Edge Function's streaming capabilities to reliably call o1-preview without duration considerations.

Making our agent even smarter

This contacts import on autopilot is a first draft that can be used as a boilerplate to iterate and improve o1's results.

A non-exhaustive list of interesting additional capabilities could be:

Leveraging the OpenAI Batching API to reduce cost on large CSV files
Have the model learn from previous runs to fine-tune its lead ranking mechanism
Iterate and improve o1's prompt by following OpenAI's best practices

Try the demo now

The combination of Vercel AI SDK and Inngest makes it super easy to add Agentic workflows to provide more accurate (e.g., support chat) or achieve more complex tasks (e.g., data analysis, agent-based features).

Try this demo now by running it locally or deploying it on Vercel: https://github.com/inngest/vercel-ai-open-o1-crm-agent.

DEV Community: Charly Poly