Chatbot Development Services: Fixing Stateful AI Chats

A chatbot can answer the first message perfectly and still fail as a product.

We saw the failure mode while building a multi-turn chatbot with Node.js, TypeScript, and the OpenAI Responses API. The first request succeeded. The second request started carrying conversation state. Then TypeScript produced an overload error around previous_response_id, while our manual-history fallback kept sending more context on every turn.

That created two separate problems. The compiler issue slowed development, while the history strategy increased request size as conversations grew.

This is where chatbot development services become an architecture problem rather than an API integration exercise.

The fix is to separate three concerns: conversation identity, model context, and application data. We will build that boundary with Node.js 22+, TypeScript, the official openai SDK, and PostgreSQL as the application-side store.

1. Start with the failure, not the framework

The TypeScript error is easy to dismiss because the API request itself looks reasonable:

// Chatbot Development Services example: the problematic inferred type is the important part.
const response = await client.responses.create({
  model: "gpt-5.5",
  instructions,
  input: userMessage,
  previous_response_id: previousResponseId,
});

The official OpenAI Node repository has documented a TypeScript inference issue around this pattern. In the reported case, TypeScript could not determine the correct overloaded responses.create() signature when previous_response_id was optional. The issue showed response being inferred as any.

That matters because a chatbot usually has exactly this shape:

1. No previous response on the first turn.
2. A response ID after the first turn.
3. The same code path on every later turn.

We should make the request type explicit instead of hiding the problem with any.

// Explicitly typing the request avoids the optional previous_response_id inference trap.
import type {
  ResponseCreateParamsNonStreaming,
} from "openai/resources/responses/responses";

const params: ResponseCreateParamsNonStreaming = {
  model: "gpt-5.5",
  instructions,
  input: userMessage,
  previous_response_id: previousResponseId,
};

const response = await client.responses.create(params);

The SDK's current documentation also exposes previous_response_id specifically for continuing a response-based conversation.

The important decision is not the type annotation. It is deciding who owns conversation state.

2. Stop treating the database as the model's conversation buffer

Once previous_response_id works, the next temptation is to store every assistant message in PostgreSQL and resend the complete transcript.

That approach looks simple:

// Naive approach: rebuilding the complete transcript makes every turn carry old context again.
const messages = await db.getMessages(conversationId);

const response = await client.responses.create({
  model: "gpt-5.5",
  input: messages,
});

The database still needs the conversation record. It may also need messages for auditing, search, analytics, or compliance.

But those are application requirements, not necessarily the model's context-management mechanism.

OpenAI documents several ways to manage conversation state and recommends the Responses API for stateful interactions. The Conversations API can also persist conversation state across sessions and devices.

For a straightforward request-response chatbot, we can instead persist the provider response identifier:

// Keep application ownership of the conversation while letting the API carry model context.
type Conversation = {
  id: string;
  previousResponseId: string | null;
};

const conversation = await db.getConversation(conversationId);

const response = await client.responses.create({
  model: "gpt-5.5",
  instructions,
  input: userMessage,
  ...(conversation.previousResponseId
    ? { previous_response_id: conversation.previousResponseId }
    : {}),
});

await db.updateConversation(conversationId, {
  previousResponseId: response.id,
});

This changes the data model from "our database contains the entire prompt" to "our database knows which conversation state to continue."

That distinction becomes important when traffic increases.

3. Keep tool calls inside the same state machine

Once conversation state is externalized, tool calling becomes the next failure point.

A chatbot that can check orders, create tickets, or query customer records should not treat tool calls as separate conversations. The model can request a function, your application executes it, and the result goes back into the same response chain.

The official Node SDK documents this loop explicitly. It also warns that a response can contain multiple function calls, so application code must inspect every function-call item.

A simplified implementation looks like this:

// Non-obvious part: match function results using call_id, not the optional output-item id.
const response = await client.responses.create({
  model: "gpt-5.5",
  instructions,
  input: userMessage,
  tools,
});

const outputs = [];

for (const item of response.output) {
  if (item.type !== "function_call") continue;

  const args = JSON.parse(item.arguments);

  if (item.name === "get_order") {
    const order = await getOrder(args.orderId);

    outputs.push({
      type: "function_call_output" as const,
      call_id: item.call_id,
      output: JSON.stringify(order),
    });
  }
}

The application then submits those outputs using previous_response_id.

This matters because tool execution is where chatbot code stops being a prompt wrapper. The application becomes responsible for authorization, validation, idempotency, and failure handling.

4. Protect the context you paid to build

The previous steps solve correctness. They do not automatically solve cost.

Prompt caching becomes relevant when the chatbot sends a large, stable instruction prefix on repeated requests. OpenAI currently documents prompt caching for supported models and says GPT-5.6 and later have a minimum cacheable prefix of 1,024 visible input tokens. Cached input is priced at a lower rate than uncached input.

That gives us a concrete ordering rule:

// Keep stable instructions and tool definitions before dynamic user-specific content.
const response = await client.responses.create({
  model: "gpt-5.6",
  instructions: `
    You are the support assistant for Acme.
    Follow the escalation policy.
    Use tools only when required.
  `,
  input: userMessage,
  tools,
});

Do not casually rebuild the instruction prefix on every turn.

Changing tool definitions, ordering, schemas, or earlier context can change the reusable prefix. OpenAI's deployment guidance specifically recommends keeping stable instructions, examples, reference material, and tool definitions consistent when optimizing caching.

For production chatbot development, we therefore monitor cached tokens rather than guessing whether caching is working.

5. The production trade-off is state ownership

That trade-off became clear in our implementation: keeping the complete transcript in PostgreSQL gave us maximum application control, while provider-managed state reduced the amount of context our application had to reconstruct.

Oodles initially favored the transcript because it made debugging easy. The failure appeared when every turn required another history reconstruction, while the application also had to preserve tool-call ordering correctly.

We moved the active model state to the response chain and kept PostgreSQL for durable application records. That removed the repeated history assembly from the hot path.

The important architectural result is measurable even before that number is inserted: the application no longer needs to rebuild the entire active model context for every turn.

For high-volume systems, this boundary also makes rate limiting easier to reason about. OpenAI rate limits can apply to requests per minute and tokens per minute, so a chatbot can hit a token limit even when request volume looks acceptable.

We therefore record at least:

• conversation ID
• response ID
• model
• input and output token counts
• cached input tokens
• tool calls
• API request ID
• retry count
• total application latency

The Node SDK exposes request IDs and supports configurable retries and timeouts. Its documented defaults should still be checked against the exact SDK version deployed by your application.

Conclusion: What actually matters

The failure was not "the chatbot API broke." The architecture had unclear ownership of conversation state.

• Use previous_response_id or the Conversations API when provider-managed conversation state fits the product.
• Keep PostgreSQL responsible for durable business data, audit records, and application-level conversation metadata.
• Type the Responses API request explicitly when optional state produces TypeScript overload inference problems.
• Treat tool calls as part of the same conversation state machine and match results using call_id.
• Measure cached tokens, token usage, retries, and p95 latency before changing prompts or models for performance reasons.

If you are building a production chatbot, the interesting engineering question is usually not how to make the first response work. It is where conversation state should live after the hundredth response.

For examples of production-oriented chatbot development services and implementation patterns, Contact us chatbot development service overview.

What state-management strategy are you using for multi-turn chats: provider-managed state, application-managed history, or a hybrid?

FAQ

What are Chatbot Development Services?

Chatbot Development Services cover the engineering required to build, integrate, deploy, and maintain conversational applications. In a production system, that can include conversation state, LLM integration, tool calling, authentication, databases, observability, rate limiting, and failure handling.

Should chatbot conversation history live in PostgreSQL?

Not necessarily. PostgreSQL is useful for durable application data, audit records, analytics, and conversation metadata. For active model context, provider-managed state such as previous_response_id or a conversation API can avoid reconstructing the entire transcript on every request.

When should I use previous_response_id?

Use previous_response_id when you want a later Responses API request to continue from an earlier response. It is useful for straightforward multi-turn conversations where the model's previous response should remain part of the conversation state.

For more complex requirements involving durable conversations across sessions or devices, evaluate the Conversations API instead.

How do I prevent chatbot API costs from growing with conversation length?

First, measure input and output tokens rather than assuming where the cost comes from. Then evaluate provider-managed state, prompt caching, summarization, and selective retrieval.

Prompt caching can reduce the cost of repeated stable prefixes when the request structure satisfies the provider's caching requirements.

Why does TypeScript sometimes report an overload error with responses.create()?

Optional properties such as previous_response_id can interact with TypeScript's overload resolution. Instead of suppressing the error with any, explicitly type the request using the SDK's ResponseCreateParamsNonStreaming type and keep the request shape consistent.