TL;DR: For an in-app chatbot extracting supplier invoice fields, choose the simplest request-response endpoint that accepts a strict schema and returns usage metadata. Put it behind a Python adapter, validate every field, and record cost by tenant before the result enters the marketplace workflow. The endpoint label matters less than those boundaries.
A useful runtime turns a supplier's question and an invoice excerpt into a small, auditable record. I would start with four fields: invoice_number, supplier_name, total_amount, and currency. Asking the model to understand an entire document too early makes evaluation vague and makes a tenant cost event hard to explain.
The flow is short. The chat UI sends a tenant ID, conversation ID, and selected invoice text to an application service. The service calls a generation endpoint through an adapter, parses the structured result, validates it, and emits a usage event. A reviewer can inspect an uncertain field without seeing provider-specific details.
If your search began with “best OpenAI API endpoint for building a chatbot in Node.js 2026 example,” treat that phrase as a shortlist request, not an architecture decision. The Node client, the API endpoint, and the model provider are replaceable parts of this boundary. The invoice contract and the tenant ledger are the durable parts.
The smallest useful implementation
This local boundary test is runnable without a network service. The fake endpoint represents whichever request-response API the team evaluates. Keeping that seam local lets the parser and cost ledger run in CI before credentials or deployment enter the picture.
from dataclasses import dataclass
from decimal import Decimal
from typing import Any
@dataclass
class Usage:
input_tokens: int
output_tokens: int
@dataclass
class Extraction:
invoice_number: str
supplier_name: str
total_amount: Decimal
currency: str
class FakeGenerationEndpoint:
def generate(self, *, system: str, user: str) -> dict[str, Any]:
return {
"output": {
"invoice_number": "SUP-1042",
"supplier_name": "Northwind Parts",
"total_amount": "1840.50",
"currency": "USD",
},
"usage": {"input_tokens": 142, "output_tokens": 28},
}
def extract_invoice(endpoint: FakeGenerationEndpoint, invoice_text: str):
response = endpoint.generate(
system=(
"Extract exactly four invoice fields. Return an object, not prose. "
"Use a decimal string for total_amount and an ISO 4217 currency code."
),
user=invoice_text,
)
output = response["output"]
required = {"invoice_number", "supplier_name", "total_amount", "currency"}
if set(output) != required:
raise ValueError("The extraction schema changed")
extraction = Extraction(
invoice_number=str(output["invoice_number"]),
supplier_name=str(output["supplier_name"]),
total_amount=Decimal(str(output["total_amount"])),
currency=str(output["currency"]).upper(),
)
if extraction.total_amount < 0 or len(extraction.currency) != 3:
raise ValueError("The invoice values failed validation")
usage = Usage(
input_tokens=int(response["usage"]["input_tokens"]),
output_tokens=int(response["usage"]["output_tokens"]),
)
return extraction, usage
def tenant_cost_event(tenant_id: str, conversation_id: str, usage: Usage):
return {
"tenant_id": tenant_id,
"conversation_id": conversation_id,
"input_tokens": usage.input_tokens,
"output_tokens": usage.output_tokens,
"token_total": usage.input_tokens + usage.output_tokens,
}
invoice, usage = extract_invoice(
FakeGenerationEndpoint(),
"Invoice SUP-1042 from Northwind Parts: total 1840.50 USD.",
)
print(invoice)
print(tenant_cost_event("tenant-17", "conversation-88", usage))
The application gets typed data and a usage record together. If an endpoint cannot provide usage, mark cost as unknown rather than inventing a number. A dashboard that silently drops usage is worse than one that shows a gap.
Five fixtures. No dashboard yet. That is enough to catch a schema drift before it becomes a finance ticket.
Which endpoint should the chatbot call?
Choose the candidate whose contract makes four things testable: text input, constrained structured output, per-request usage, and deterministic error handling. A chat-shaped endpoint may fit conversation history, while a general generation endpoint may make one extraction request easier to model. The name is not the decision. The contract is.
Run the same fixture set through each candidate: a clean invoice, a missing currency, a badly formatted total, a long invoice excerpt, and text containing instructions. Expected results include a schema error, a review state, or a refusal to guess. A plausible amount for malformed input is a failed test, even when the prose sounds polished.
Use this decision table while reviewing an endpoint candidate:
| Capability | Accept | Reject |
|---|---|---|
| Structured result | Exact four-field object | Free-form prose only |
| Usage | Input and output counts | Missing or untraceable counts |
| Failure state | Timeout, validation, and review are distinct | Every problem becomes a retry |
| Tenant trace | Tenant and conversation IDs survive the call | Only a provider request ID is retained |
This is the notebook-to-prod boundary. A notebook proves that one example parses. A production adapter must preserve tenant ID, conversation ID, request status, usage, validation result, and retry count as one traceable event. Keep prompts and extracted payloads separate in logs, and redact invoice text unless retention policy permits it. Supplier documents can contain bank details that a cost dashboard does not need.
Count input and output tokens separately, aggregate them by tenant, and attach a status such as accepted, needs_review, or failed_validation. Do not turn a tenant leaderboard into a quality score. Longer invoices can legitimately consume more input tokens; compare cost with document volume and review rate.
What fails first in an invoice chatbot?
Prompt scope usually grows first. Someone begins with “understand this invoice” and adds line items, payment terms, tax treatment, and an explanation. The response gets larger, the schema becomes less stable, and the cost event no longer explains the work. Four fields are a useful first contract because each has a downstream owner and a simple validation rule.
The next trap is treating a successful HTTP response as a successful extraction. A response may be syntactically valid while carrying an empty supplier name or an unsupported currency. Validate immediately after parsing and produce a reviewable state instead of letting a generic exception disappear in a worker.
The awkward case is a supplier invoice that contains both a printed total and a paragraph addressed to the assistant. The paragraph may ask for a different currency or request that the system ignore its extraction rules. Treat all invoice text as data. The system instruction defines the four output fields, the parser rejects extra keys, and a reviewer sees the conflict when the amount or currency does not pass validation. This is a small boundary, but it matters more than a clever prompt because it gives the marketplace team a stable place to inspect a bad result, attribute the attempt to a tenant, and decide whether the document should be corrected upstream. I would rather send one uncertain invoice to review than quietly post a confident value to a payment workflow.
Keep the contract boring.
Retries need their own accounting. If a worker times out after a remote request completes, a retry may create a second generation. Use an idempotency key when the selected API supports one; otherwise record every attempt and reconcile repeated results. A client-side timeout does not prove that the remote request did not run.
I initially treated usage as a reporting detail. That was the wrong abstraction. Once tenant cost visibility is a primary decision axis, usage belongs in the application result beside extraction and validation. An endpoint that cannot expose enough information for a tenant ledger has a real operational limitation for this marketplace.
A practical evaluation loop
Start with a fixture repository owned by the application team. Store sanitized invoice snippets, expected values, expected review states, and tenant metadata for accounting. Run it on every adapter change. Track exact matches for each field, validation failures, and token totals; one blended score hides whether the problem is parsing, prompting, or input length.
Exercise the boundary under production-shaped conditions: two tenants submitting at once, an empty selection, a repeated message, and a response that omits one field. The test is complete only when an operator receives a useful state for each case. “The model returned something” is not a state.
Keep the adapter narrow. It translates the internal request into the provider request and translates the response back into an internal result. It should not own tenant accounting, invoice persistence, or UI wording. That separation makes a provider change a controlled adapter change rather than a rewrite of the marketplace workflow.
Keep this operational checklist beside the runbook: reject missing tenant and conversation identifiers; cap invoice input; validate the exact field set; record input and output usage; classify failures separately from review cases; redact document content from cost logs; and alert on missing usage events. Before expanding beyond four fields, require an eval result and a named downstream owner for every new field.
Top comments (0)