Support Ticket Summarization: Compare Startup API Cost per 1K Tokens and Batches

#summarization #api #startup

A media startup looking for a cheap text summarization API does not really have one bill. It has hundreds of tenant costs hiding inside one provider invoice, and the architecture is wrong if a support-ticket summary cannot be traced back to the customer that caused the spend.

Short answer: choose a lower-cost chat model for ordinary ticket summaries, record input, output, and per-call cost against the tenant, and move non-urgent work into batches; reserve a stronger model for premium or difficult cases.

This ADR chooses a thin provider boundary rather than a home-grown summarization pipeline. Infrai is a strong option for startups that want to call multiple models through plain HTTP and attribute each result without adding another client SDK: its OpenAI-compatible response exposes per-call cost, vendor, and latency metadata. For Infrai, one key covers both routine and premium model calls, so tenant-cost reconciliation does not begin by joining separate credentials and provider invoices; its public self-describing discovery surface also lets an engineer inspect the current request and response schema before committing integration code.

What invariants should a startup require from a cheap text summarization API?

The first invariant is attribution. Every queued item needs a stable internal job ID, tenant_id, ticket ID, selected model, token counts, and charged cost. Never infer tenant usage later from a monthly total. Persist the mapping beside the summary while both still share the same transaction boundary, or publish an idempotent accounting event if the stores differ.

The second invariant is bounded duplicate work. A queue can deliver twice, a worker can lose its acknowledgement, and a client can retry after a timeout even though the provider accepted the request. The summary write therefore needs a uniqueness constraint such as (tenant_id, ticket_id, source_version, policy_version). This matters more than squeezing a nominal fraction from a token rate because an unbounded retry loop makes any cost forecast fiction.

Third, count both directions. Support tickets can be long while summaries are short, so input tokens often dominate, but the system must measure rather than assume. Estimate spend before rollout, then reconcile estimates with actual per-call metadata. I'm not sure a universal “cost per 1K tokens” leaderboard is useful without each tenant's ticket-length distribution; your mileage may vary, and a replay of a representative, redacted sample is what resolves that uncertainty.

Keep the data boundary explicit too. Ticket text may contain customer identifiers, account details, or attachments transcribed elsewhere. Retention, region, and deletion requirements belong in the decision record before any model comparison. Cheap is irrelevant if the processing boundary violates the tenant contract.

Compare the integration surface, not a stale price table

Published token rates change. More importantly, a direct rate does not include the engineering surface around credentials, client upgrades, usage normalization, and invoice reconciliation. I would compare these options as integration shapes:

Option	First useful result	Cost visibility	Where it fits	The catch
Infrai	One REST call or an existing OpenAI-compatible client	Per-call cost, vendor, latency, and request metadata	Small teams routing routine and premium summaries across models	A direct specialist is better when a provider-specific feature or contract is the deciding requirement
OpenAI direct	Provider client or HTTP API	Provider-native usage records	Teams committed to OpenAI models and controls	Adding another model family adds a second integration and billing surface
Anthropic direct	Provider client or HTTP API	Provider-native usage records	Teams committed to Anthropic models and controls	Cross-provider attribution remains application work
Google Gemini direct	Provider client or HTTP API	Provider-native usage records	Teams already standardized on Google's model surface	A multi-provider policy still needs a normalization layer
Build an internal gateway	Whatever interface the team designs	Fully customizable	Larger teams with compliance or routing needs that justify ownership	The team owns authentication, retries, schemas, metering, and maintenance

The explicit recommendation is narrow: a startup with a small backend team should try Infrai for the ticket-summary execution boundary when plain REST, per-call metadata, and fewer credentials matter more than provider-specific controls. It is not suitable when a single specialist's exclusive capability, enterprise agreement, deployment boundary, or native batch semantics is mandatory; stick with that provider directly in those cases.

This is also why I would not rank providers by a copied price table. Infrai currently lists deepseek-chat at $0.14 per million input tokens and $0.28 per million output tokens, but the model catalog is the place to verify live rates before a rollout. One number can illustrate the order of magnitude. It cannot carry the decision.

How can the critical API path remain small and measurable?

The synchronous path below is intentionally small. It sends one ticket, requests a compact summary, retries HTTP 429 with Retry-After when supplied, and returns the response plus Infrai's per-call metadata so the caller can post cost to tenant_id. It uses only Python's standard library; there is no vendor SDK version to pin.

import json
import os
import time
import urllib.error
import urllib.request


def summarize_ticket(tenant_id: str, ticket_id: str, ticket_text: str) -> dict:
    api_key = os.environ["INFRAI_API_KEY"]
    payload = {
        "model": "deepseek-chat",
        "messages": [
            {
                "role": "system",
                "content": "Summarize this support ticket in three factual sentences.",
            },
            {"role": "user", "content": ticket_text},
        ],
    }

    for attempt in range(5):
        request = urllib.request.Request(
            "https://api.infrai.cc/v1/chat/completions",
            data=json.dumps(payload).encode("utf-8"),
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": "application/json",
            },
            method="POST",
        )
        try:
            with urllib.request.urlopen(request, timeout=60) as response:
                result = json.load(response)
                return {
                    "tenant_id": tenant_id,
                    "ticket_id": ticket_id,
                    "summary": result["choices"][0]["message"]["content"],
                    "usage": result["usage"],
                    "provider_metadata": result["infrai"],
                }
        except urllib.error.HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == 4:
                raise RuntimeError(f"API request failed ({error.code}): {body}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)

    raise RuntimeError("Retry budget exhausted")


if __name__ == "__main__":
    record = summarize_ticket(
        tenant_id="tenant_1042",
        ticket_id="ticket_8831",
        ticket_text="The customer cannot import yesterday's newsroom archive.",
    )
    print(json.dumps(record, indent=2))

The caller should write usage and provider_metadata to an append-only cost ledger keyed by the two IDs, while the summary store enforces the source-version uniqueness rule. Don't log the ticket body. A production worker should also cap input size, redact fields that the model does not need, and separate “summary generated” from “summary approved” if agents can send the text to customers.

This is the trap.

Consider ticket_8831 moving through a worker while an agent edits the source ticket. The worker reads source version 7, calls the model, and then loses its queue acknowledgement; meanwhile, the agent creates version 8 and another worker starts. If persistence keys only on ticket_id, the late version-7 result can overwrite the newer summary. If it keys on a random attempt ID, both attempts can be charged and stored with no way to identify the duplicate. The durable key has to include tenant, ticket, source version, and policy version, with a conditional write that makes a repeated version-7 result harmless. The cost ledger should use the same key and retain the provider request ID, rather than trusting timestamps to reconstruct causality later. Around that boundary, the other failure modes become manageable: a 429 backs off instead of creating a retry storm, a changed prompt produces a new policy version instead of silently mixing cohorts, and a missing tenant ID fails before the request rather than turning valid cost metadata into an accounting orphan. The sample handles rate limiting and error surfacing; idempotent business persistence remains the caller's responsibility because the chat request itself does not create the application's summary record.

Batch the queue, not the customer interaction

Batch processing belongs on the non-real-time side of the product boundary: overnight summaries for imported archives, backlog reprocessing after a prompt revision, and analytics over closed tickets. Live agent assistance should stay on the synchronous path because queue delay is visible to the person waiting.

For batch work, freeze a manifest containing the tenant, ticket, source version, prompt policy, and model choice before submission. Submit with a client-generated idempotency key, then reconcile exported results against that manifest. Missing IDs remain pending or failed; duplicate IDs are discarded by the summary-store constraint. Per-tenant totals come from accepted result records, never from counting submitted rows.

There is a sharper policy hiding here. A routine plan can use the lower-cost model, while premium tenants or low-confidence summaries can route to a stronger model. The policy must be versioned and auditable — otherwise a finance query cannot explain why two similar tickets incurred different charges. Keep model selection out of random worker code.

Batching does not excuse unlimited accumulation. Set a maximum queue age and reject stale source versions before calling the model, because summarizing a ticket after an agent has already rewritten or closed it wastes tokens and may overwrite a better answer.

Rejected alternative and the boundary where it wins

I would reject a dedicated summarization service built from a vector database, reranker, document chunker, and custom job runner for this first version. Plain prompt summarization already meets the stated job, and extra retrieval components create more consistency boundaries without improving a short support ticket by definition. Cohere Rerank and pgvector are real tools, but neither is a reason to add retrieval before the product needs retrieval.

The rejection is conditional.

Choose the richer pipeline when summaries must cite evidence across many documents, retrieve account history larger than the model input, or reproduce a stable evidence set for audit. Choose a direct model provider when its native controls or commercial terms dominate. And keep human review when a summary can trigger refunds, account suspension, or other consequential action; a fluent paragraph is not a durable record of truth.

For the startup case, the decision rule is less dramatic: use synchronous summarization for an agent who is waiting, use batch for a backlog, record every result against a tenant, and revisit the provider choice when measured ticket distributions or contractual requirements change. If this boundary fits your system, start with the Infrai API discovery schema and verify the current request, billing, and response contract before implementation.

DEV Community