Give review annotations their own records

A review arrives as JSON with an identifier, a score, and a comment. Your next step asks a model to identify themes. It returns two neat columns: room noise and breakfast quality.

The output now looks more structured. But the columns contain a new interpretation, not additional facts copied from the review page. If you overwrite the comment or mix those labels into the source fields, a later reader may be unable to tell which values were observed and which were inferred.

A practical design is to store source observations and annotations as separate records, linked by a stable identifier and an explicit observation revision. This article proposes that design using fictional examples. It does not report a classification benchmark or a live collection test.

Start with what the source actually published

Here is a fictional observation record. The placeholder URL is illustrative, and the field names are an application design rather than a BrowserAct API schema:

{
  "observation_id": "review-example-101-r1",
  "review_id": "review-example-101",
  "source_url": "https://example.com/reviews/101",
  "source_score": 8,
  "source_score_scale": "0-10",
  "comment_original": "Quiet room, but breakfast was limited."
}

The score belongs to the review as a whole. It does not establish a numerical rating for breakfast or a measured sound level in the room. Keep the original text available for that distinction.

In a real pipeline, record retrieval context and available publication information too. Preserve the precision the source provides. Do not manufacture a publication day when the page supplies only a month, or assume a missing score has the value zero.

Attach interpretations without rewriting observations

An annotation can refer to the exact observation it analyzed:

{
  "annotation_id": "annotation-example-101-v1",
  "observation_id": "review-example-101-r1",
  "taxonomy_version": "stay-themes-v1",
  "method_version": "example-classifier-config-v1",
  "review_status": "pending",
  "labels": [
    {
      "aspect": "room_noise",
      "polarity": "positive",
      "evidence_text": "Quiet room"
    },
    {
      "aspect": "breakfast",
      "polarity": "negative",
      "evidence_text": "breakfast was limited"
    }
  ]
}

These are proposed labels for the fictional sentence. The record deliberately leaves review pending. An evidence phrase makes an annotation easier to inspect, but its presence does not prove that the chosen aspect or polarity is correct.

The method version should resolve to the actual configuration used in your system. If a model is involved, retain the model identifier, prompt or prompt revision, and relevant processing settings in that configuration. A friendly version string without a retrievable definition is weak provenance.

Define the denominator before reporting themes

One review may produce several labels. In the example, two annotation items still refer to one source review.

Decide whether a chart counts reviews mentioning a theme, individual mentions, or approved annotation items. Those are different denominators. A join that repeats each review once per label can inflate a review count if you count joined rows without deduplicating the intended review key.

Also decide how to treat unavailable or unprocessed comments. A classifier that never ran has not found an absence of complaints. Keep processing status separate from the label result so missing work does not quietly become positive evidence.

For a report, state the input population, processed population, reviewed population, and counting rule. A percentage without those boundaries can look more precise than the underlying coverage supports.

Keep disagreement available for review

Define the taxonomy before treating labels as comparable. Does “limited breakfast” mean variety, quantity, availability, or general dissatisfaction? A broad breakfast label might be sufficient for one analysis; a detailed product decision may require a narrower distinction.

Create a review queue for ambiguous cases. Store the original annotation and any reviewed replacement with their relationship, rather than erasing the earlier result. This lets someone investigate why a summary changed after the taxonomy or method changed.

If the source comment itself changes, create a new observation revision and identify which annotations still point to the older text. Reusing an old label against an updated comment would hide a change in its evidence.

Validate structure and meaning with different checks

Structural checks can confirm that every annotation references an existing observation, that labels use the permitted vocabulary, and that cited phrases occur in the referenced text. They can also detect accidental duplicate annotation records or missing method versions.

Those checks cannot establish that a classification captures the writer's meaning. Assess semantic quality separately on an appropriately selected, reviewed sample. Inspect disagreements and document the limits of that assessment before using a summary to drive decisions.

Likewise, a successful annotation stage says nothing about whether collection covered every requested page. Check collection coverage before interpreting theme frequencies as representative of the intended population.

I work on BrowserAct. Our structured and unstructured data guide explains why free text retains its interpretive uncertainty inside a structured container. This proposed record design keeps that uncertainty visible while making later analysis traceable.