DEV Community

NoahHayes7250
NoahHayes7250

Posted on

Classify Text Data Using Embeddings API: A Guide to Reviewable Labels

#ai #embeddings #node #media

To classify text data using embeddings API calls, assign a first-pass label to each media catalog description, then make uncertainty and tenant usage visible instead of hiding them behind a single score. The useful design is a small classifier with versioned label examples, a review state, and an append-only usage record.

Short answer: embed the description and each label definition in the same vector space, rank the labels by cosine similarity, and reject the result when it does not meet a threshold selected from representative catalog data. Store the tenant id, label-set version, score, input identity, and request attempt with the decision. This keeps the classifier explainable enough to operate in a one-person SaaS.

The expensive mistake is treating similarity as truth. A media description can mention a presenter, a sport, a live broadcast, and regional rights in one paragraph. The classifier still needs to make a bounded choice, and a reviewer needs a clear way to correct it.

Keep the uncertain row visible.

How can I classify media text data using embeddings?

Start with a narrow taxonomy. For example, a catalog may have labels for news, documentary, entertainment, and sports. Each label should have a definition and representative text. Store those definitions as data with a version such as media-taxonomy-v1, rather than scattering them through conditional statements in the Node.js service.

The decision has three useful states: accepted, review, and rejected. Accepted means the winning score clears the policy threshold. Review means the best match is plausible but not strong enough for an automatic catalog change. Rejected means the input or label set is invalid. These states are more useful than returning a label for every row because they separate model uncertainty from bad input.

This is a deliberate trade-off. A multi-label taxonomy may describe a program more accurately, but it also changes filtering, correction, and reporting. For a product that ships weekly, one label plus a review path is often the smaller surface to maintain. Expand it after the review queue shows a repeated need, not because the vector API makes multiple labels easy.

Labels are policy.

The smallest working TypeScript classifier

The provider-specific request belongs behind embed. The ranking code should be deterministic and unit-testable without making a network request. It also needs to reject vectors with different dimensions; otherwise a changed label cache can produce a plausible result that has no meaning.

type Label = {
  name: string;
  definition: string;
};

type LabelVector = Label & {
  vector: number[];
};

type Classification = {
  label: string | null;
  score: number;
  state: "accepted" | "review";
  labelSetVersion: string;
};

function dot(a: number[], b: number[]): number {
  if (a.length !== b.length) {
    throw new Error("Vector dimensions do not match");
  }

  return a.reduce((sum, value, index) => sum + value * b[index], 0);
}

function cosineSimilarity(a: number[], b: number[]): number {
  const product = dot(a, b);
  const aLength = Math.sqrt(dot(a, a));
  const bLength = Math.sqrt(dot(b, b));

  if (aLength === 0 || bLength === 0) {
    return -1;
  }

  return product / (aLength * bLength);
}

async function classifyDescription(
  description: string,
  labels: LabelVector[],
  embed: (text: string) => Promise<number[]>,
  labelSetVersion: string,
  threshold: number,
): Promise<Classification> {
  if (description.trim() === "") {
    throw new Error("Description is empty");
  }

  const vector = await embed(description);
  const ranked = labels
    .map((label) => ({
      name: label.name,
      score: cosineSimilarity(vector, label.vector),
    }))
    .sort((left, right) => right.score - left.score);

  const winner = ranked[0];
  const accepted = winner !== undefined && winner.score >= threshold;

  return {
    label: accepted ? winner.name : null,
    score: winner?.score ?? -1,
    state: accepted ? "accepted" : "review",
    labelSetVersion,
  };
}
Enter fullscreen mode Exit fullscreen mode

The threshold is a policy setting, not a property of embeddings. Choose it with a held-out sample that resembles the real catalog. Record the score even when the item enters review. Later corrections are evidence about the label definitions and threshold, while a missing score leaves nothing to inspect.

The code is intentionally small. It does not normalize a description, call an API, or write a catalog row because those concerns have different failure and retry behavior. Keep text normalization deterministic, bound the size of each request, and pass the tenant id into the job before it enters a queue.

How do you make each tenant’s classification work visible?

Treat usage as a ledger, not as a field on the current catalog row. Write an event for every embedding attempt with the tenant id, record id, operation, label-set version, input measurement used by the API, attempt number, and timestamp. Preserve raw usage fields when the API supplies them; derive internal reports from those fields instead of assuming that one billing unit is universal.

The distinction matters when a request times out and the job is retried. The catalog needs one current decision. The usage report needs both attempts. If the retry overwrites the first event, the operator cannot explain why a tenant generated more work than the number of records it submitted. I keep those as two records: one mutable classification state for the current catalog row, and one append-only usage event for every attempt. A description that fails once and succeeds on its second attempt therefore remains one catalog decision but two usage events. If a reviewer later changes the label, that correction is a review event tied to the same record and label-set version, not a rewrite of the original request. The trade-off is a little more storage and a much clearer answer when a tenant asks what happened to its data.

Use an idempotency key for the catalog update, such as a value derived from tenant, record, and label-set version. Keep the attempt number separate from that key. RFC 9110 describes the difference between retry-safe behavior and operations whose repetition can have side effects; the application still has to choose the write boundary that makes repetition harmless.

For a one-person SaaS, this is a revenue-per-hour concern. A weekly report should answer which tenants generated classification work, which records are being retried, and which label-set version is creating the review queue. I don't need a more elaborate model before I can answer those questions. Outsource the undifferentiated request plumbing. Keep taxonomy changes, review outcomes, and tenant-level accounting close to the product.

Which failure modes deserve a test before scale?

Test the boundary cases first: an empty description, a very long description, duplicate input, a language absent from the examples, overlapping labels, a missing label vector, and a repeated queue delivery. The expected result should be explicit for each case. None should quietly become a confident business label.

Keep a small fixture set with a clear news description, an ambiguous live studio program, an input that matches no configured label, and a known duplicate. Assert the state, score boundary, label-set version, and usage event. Run the same fixtures when the taxonomy changes. A compact regression set supports weekly shipping because it can run before the release instead of becoming a separate project.

Review more than low scores. A high score can still expose a misleading label definition or taxonomy drift. Sample accepted decisions, record reviewer corrections, and compare those corrections by label-set version. The review queue is not just an exception bucket; it is the feedback loop for deciding whether the taxonomy is fit for the catalog.

At higher volume, precompute vectors for each label-set version and cache them by that version. Queue changed descriptions in bounded batches, enforce per-tenant quotas, and show successful classifications, retries, review items, and rejected inputs as separate measures. A dead-letter path is useful for records that repeatedly fail validation, provided those records remain attributable to the tenant and source row.

The boundary stays the same as the system grows: the current catalog decision is separate from the attempts that produced it. That makes the workflow easier to explain, easier to test, and less likely to turn a similarity score into an unreviewable rule.

References

Top comments (0)