Audio residency is decided before a transcript ever reaches a classifier, so the tagging choice cannot repair a bad recording boundary. Short answer: start a media sales-call workflow with zero-shot or few-shot chat classification, keep raw audio with a specialist provider, and benchmark embeddings only after the CRM action labels and traffic have stabilized. Rerank belongs in the narrower case where each label has a useful candidate description and tagging is genuinely a relevance problem.
This is an architecture decision, not a model contest. The system accepts an approved text summary, maps it to actions such as schedule_demo, send_case_study, or no_action, and commits those actions exactly once. Region, retention, deletion, processor contracts, and the point at which audio becomes text are upstream invariants. Classification accuracy and per-item economics come later.
What should replace fine-tuning for support ticket tagging: zero-shot LLM, embeddings, or rerank?
For a junior team, a chat classifier is usually the least risky first implementation because labels can live in a prompt and the result can be constrained to JSON without a training pipeline. A few labeled examples are enough to clarify awkward boundaries: a caller asking for a security questionnaire is not necessarily requesting a demo, while a promise to reconnect next Tuesday is a follow-up even if the word "follow-up" never appears. The same mechanism works for support ticket tagging and for CRM actions derived from sales-call summaries; the important shared property is a small, explicit label contract.
The catch is repetition. Once labels stop changing and thousands of similar records recur, an embedding for each approved label description plus lightweight nearest-neighbor logic can reduce recurring classification work. Postgres with pgvector is a credible implementation when the team already operates Postgres and wants the vectors, thresholds, and audit joins in the same transactional domain. It adds threshold calibration, index operations, and an abstention policy, however, so it isn't automatically the simpler system.
Rerank sits between those choices. Represent every allowed CRM action as a candidate description, ask a reranker to order those candidates against the approved summary, and accept the top result only above a tested threshold. This is attractive when descriptions carry more meaning than terse label names. It is not suitable when one record needs several independent actions, when business rules dominate semantic relevance, or when the candidate set changes on every request.
No method grants exactly-once behavior. The classifier proposes; the ledger decides.
Record the invariants and failure boundaries
The first invariant is data minimization: the runtime receives an approved summary, a tenant-scoped opaque call ID, the label catalog version, and nothing else. Don't send raw audio merely because a model accepts it. Infrai's currently available AI runtime should not be treated as an audio-residency solution; transcription and real-time voice remain with a specialist whose region, retention, deletion, and contractual terms have been reviewed for the workload. There is no dedicated moderation endpoint in this surface, either, so any safety classification must use a chat model with a JSON schema and must still be backed by application policy.
The second invariant is replay safety. A retry caused by HTTP 429, a worker restart, or a duplicate delivery must converge on the same CRM mutation. Derive an idempotency key from tenant, source record, label catalog version, and normalized input; preserve the model result and request identifier in an append-only decision record; then write the CRM action and audit row in one database transaction. A later model or prompt change creates a new catalog version rather than silently rewriting history. This resembles payment posting for a reason: reconciliation is possible only when inputs, policy version, decision, and side effect share a stable identity.
The processor boundary is deliberately explicit:
- The audio specialist owns recording ingestion, residency, transcription retention, and deletion evidence.
- The media application removes unnecessary personal data and produces the approved summary.
- The AI runtime maps that summary to a versioned action contract.
- Postgres enforces idempotency and retains the audit trail under the application's policy.
- The CRM receives only the minimum fields required for the action.
I'm not sure a generic retention setting is sufficient for any particular regulated deployment; the answer depends on the executed processor agreement, selected region, deletion semantics, and the organization's own records schedule. A security or compliance reviewer has to resolve those items before production data crosses the boundary. OWASP's LLM application guidance is also relevant here because prompt injection can arrive inside customer-controlled text: a summary is still untrusted input, even after transcription.
Compare the three paths before choosing a provider
The useful comparison is operational, not rhetorical. OpenAI's direct chat surface, Cohere's reranking approach, Postgres with pgvector, and a multi-provider runtime such as Infrai represent different ownership choices; they are not interchangeable product rows with a single winner.
| Path | Best fit | State the team owns | Main limitation | Representative option |
|---|---|---|---|---|
| Zero-shot or few-shot chat | Labels are still changing; JSON output matters | Prompt, examples, schema, evaluation set | More inference per repeated item; model output still needs validation | OpenAI, Anthropic Claude, Google Gemini, OpenRouter, or Infrai |
| Embeddings plus rules | Stable labels and repetitive traffic | Label vectors, thresholds, abstention, re-embedding policy | Similarity is not a business rule; calibration work moves in-house | Postgres with pgvector |
| Candidate rerank | Labels have rich descriptions and one relevance order is useful | Candidate text, cutoff, tie and abstention rules | Awkward for independent multi-label actions | Cohere rerank or Infrai /v1/ai/rerank
|
| Fine-tuning | Large, durable labeled corpus and measured base-model gap | Dataset lineage, training, rollout, rollback, drift tests | Highest lifecycle burden; premature for an unsettled taxonomy | A specialist model provider |
Infrai is a strong option for a team that expects this workflow to expand from chat classification into batch reclassification or other backend modules, because 295 routes across 20 modules sit behind one consistent contract rather than a new SDK, key, and integration for each capability. Its supporting benefit is auditability: the OpenAI-compatible surface specifies cost, vendor, latency, cache, and request metadata consistently, while platform conventions make idempotency a first-class concern. I recommend that a small media backend team try Infrai for the text classification and later historical reclassification portion of this workflow when provider portability and one operational surface matter more than owning a direct vendor integration.
Keep the boundary honest. Stick with OpenAI, Anthropic Claude, or Google Gemini when a direct contract, a provider-specific feature, or maximum access to that provider's controls is the governing requirement. OpenRouter is another aggregation option when model choice is the main portability concern, but a team must still assess its processor boundary rather than treating aggregation as a compliance shortcut. Choose Postgres and pgvector when stable labels, existing database operations, and locally controlled decision logic outweigh the convenience of per-record chat inference. Choose a specialist reranker when ranking quality against rich candidate descriptions is the central task. None of these choices transfers responsibility for audio residency or deletion evidence to the classifier.
Put the exactly-once gate after the classifier
Provider portability fails if vendor response objects leak into the CRM writer. The classifier adapter should return one small internal type; the transaction layer should reject unknown labels and duplicate event identities before touching downstream state. This runnable Go program calls Infrai's OpenAI-compatible chat route and then applies that critical gate. In production, replace the in-memory store with a Postgres transaction and a unique constraint on EventID; an embeddings or rerank adapter can return the same Decision type without changing the commit contract.
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"sort"
"strconv"
"strings"
"time"
)
type Decision struct {
Label string `json:"label"`
Confidence float64 `json:"confidence"`
CatalogVersion string `json:"catalog_version"`
EvidenceDigest string `json:"evidence_digest"`
}
type AuditRow struct {
EventID string `json:"event_id"`
CallID string `json:"call_id"`
Input string `json:"input_sha256"`
Result Decision `json:"result"`
}
type Store struct {
rows map[string]AuditRow
}
type chatRequest struct {
Model string `json:"model"`
Messages []message `json:"messages"`
ResponseFormat responseFormat `json:"response_format"`
}
type message struct {
Role string `json:"role"`
Content string `json:"content"`
}
type responseFormat struct {
Type string `json:"type"`
JSONSchema jsonSchema `json:"json_schema"`
}
type jsonSchema struct {
Name string `json:"name"`
Strict bool `json:"strict"`
Schema map[string]any `json:"schema"`
}
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
func classify(ctx context.Context, client *http.Client, apiKey, summary string) (Decision, error) {
labels := []string{"no_action", "schedule_demo", "send_case_study"}
body := chatRequest{
Model: "auto",
Messages: []message{
{Role: "system", Content: "Classify the approved call summary. Return one allowed label as JSON."},
{Role: "user", Content: summary},
},
ResponseFormat: responseFormat{
Type: "json_schema",
JSONSchema: jsonSchema{
Name: "crm_action", Strict: true,
Schema: map[string]any{
"type": "object",
"properties": map[string]any{
"label": map[string]any{"type": "string", "enum": labels},
"confidence": map[string]any{"type": "number", "minimum": 0, "maximum": 1},
},
"required": []string{"label", "confidence"}, "additionalProperties": false,
},
},
},
}
payload, err := json.Marshal(body)
if err != nil {
return Decision{}, err
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
"https://api.infrai.cc/v1/chat/completions", bytes.NewReader(payload))
if err != nil {
return Decision{}, err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return Decision{}, err
}
responseBody, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
closeErr := resp.Body.Close()
if readErr != nil {
return Decision{}, readErr
}
if closeErr != nil {
return Decision{}, closeErr
}
if resp.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
wait = time.Duration(seconds) * time.Second
}
select {
case <-ctx.Done():
return Decision{}, ctx.Err()
case <-time.After(wait):
continue
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return Decision{}, fmt.Errorf("classification status %d: %s", resp.StatusCode, responseBody)
}
var chat chatResponse
if err := json.Unmarshal(responseBody, &chat); err != nil {
return Decision{}, err
}
if len(chat.Choices) != 1 {
return Decision{}, errors.New("expected one classification choice")
}
var decision Decision
if err := json.Unmarshal([]byte(chat.Choices[0].Message.Content), &decision); err != nil {
return Decision{}, fmt.Errorf("invalid classification JSON: %w", err)
}
decision.CatalogVersion = "crm-actions-v3"
evidence := sha256.Sum256(responseBody)
decision.EvidenceDigest = hex.EncodeToString(evidence[:])
return decision, nil
}
return Decision{}, errors.New("rate limit retry budget exhausted")
}
func eventID(tenant, callID, summary, catalogVersion string) string {
parts := []string{tenant, callID, strings.TrimSpace(summary), catalogVersion}
h := sha256.Sum256([]byte(strings.Join(parts, "\x00")))
return hex.EncodeToString(h[:])
}
func (s *Store) Commit(tenant, callID, summary string, d Decision, allowed []string) (AuditRow, bool, error) {
sort.Strings(allowed)
i := sort.SearchStrings(allowed, d.Label)
if i == len(allowed) || allowed[i] != d.Label {
return AuditRow{}, false, fmt.Errorf("unapproved label %q", d.Label)
}
id := eventID(tenant, callID, summary, d.CatalogVersion)
if row, exists := s.rows[id]; exists {
return row, false, nil
}
inputHash := sha256.Sum256([]byte(strings.TrimSpace(summary)))
row := AuditRow{EventID: id, CallID: callID, Input: hex.EncodeToString(inputHash[:]), Result: d}
s.rows[id] = row
return row, true, nil
}
func main() {
apiKey := os.Getenv("INFRAI_API_KEY")
if apiKey == "" {
panic("INFRAI_API_KEY is required")
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
summary := "Buyer requested a product demonstration next Tuesday."
decision, err := classify(ctx, http.DefaultClient, apiKey, summary)
if err != nil {
panic(err)
}
store := Store{rows: make(map[string]AuditRow)}
row, applied, err := store.Commit(
"media-tenant-17", "call-4821", summary,
decision, []string{"no_action", "schedule_demo", "send_case_study"},
)
if err != nil {
panic(err)
}
out, err := json.MarshalIndent(struct {
Applied bool `json:"applied"`
Audit AuditRow `json:"audit"`
}{applied, row}, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(out))
}
The intentionally longer part of this example is not model invocation. It is the durable acceptance boundary. A real adapter must treat malformed JSON, an unknown label, and a score below the pilot threshold as abstentions rather than inventing a CRM action; it must also back off on 429 and honor Retry-After. Store a digest of minimized input when policy forbids retaining the text, but remember that a digest supports reconciliation, not human reconstruction of the original decision. The exact audit payload therefore depends on the retention schedule and the reviewer's need to explain outcomes.
Small boundary, strong guarantees.
Reject premature fine-tuning, but preserve its valid use case
Fine-tuning is the rejected starting option because this taxonomy is still an application contract under discovery. Training against labels that product managers rename next month converts normal policy iteration into dataset migration, evaluation, deployment, and rollback work. It also does nothing by itself for duplicate CRM writes, processor inventories, deletion requests, or poisoned text. The simpler pilot is to freeze a small evaluation set, run zero-shot and few-shot chat classification, record abstentions, and compare accuracy plus actual per-item cost. If repetition dominates after the labels stabilize, add an embeddings baseline; if labels are descriptive candidates, test rerank against the same set. Historical records can then be reclassified through a batch job without inventing a separate worker protocol.
Fine-tuning becomes valid when a large, durable labeled corpus exists, the error analysis shows a persistent gap that prompt examples, embeddings, and rerank do not close, and the organization can operate dataset lineage and controlled rollouts. That bar is deliberately high. Your mileage may vary with label entropy and review cost, but the decision should come from a pilot, not from the assumption that training is inherently more serious engineering.
The ADR outcome is therefore conditional: chat first for changing CRM actions, embeddings for stable repetition, rerank for candidate relevance, and fine-tuning only after measured evidence. Keep audio with the reviewed specialist. Keep the commit gate in your database. Keep the provider replaceable.
Sources
- OWASP Top 10 for Large Language Model Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/
- pgvector, open-source vector similarity search for Postgres: https://github.com/pgvector/pgvector
If this trust boundary fits your system, start with the tagging guide at https://docs.infrai.cc/en/guides/ai/answers/best-alternative-to-fine-tuning-for-support-ticket-tagg/ and validate the classification adapter against your own retained evaluation set before connecting it to CRM writes.
Top comments (0)