Treat a crawler skill's output as a versioned interface

A saved crawler workflow is easier to reuse when the next person can trust its output shape. Finding the same button again is only part of the job. Downstream code still needs stable field names, clear missing values, and a way to distinguish a complete collection from an interrupted one.

Here is a small contract I would put around a reusable extraction skill. It is an application-side design example, not a BrowserAct API, a generated Skill Forge artifact, or a claim that a real website passed these checks. This article comes from the BrowserAct team.

Write the consumer's assumptions first

Start with the code or person receiving the result. Which fields identify a row? Which values may be absent? What does the caller do when pagination stops early?

For a directory collection, a consumer might require a source-specific record ID and a source URL. A contact email could remain null when it was not visible. The workflow should not invent a value simply because an export column exists.

Decide those rules before saving the skill. Otherwise a changed extraction path can return plausible JSON that quietly breaks an importer.

Separate shape from coverage

Consider this fictional response:

{
  "schema_version": "records.v1",
  "stop_reason": "row_limit",
  "rows": [
    {
      "record_id": "demo-17",
      "source_url": "https://example.com/records/demo-17",
      "email": null
    }
  ]
}

The response can be structurally valid while covering only part of the requested data. A row limit is an intentional stop, not proof that the directory ended. A blocked page is another reason to return a partial collection with an explicit status.

Use a separate coverage check when you have an authoritative expected set. Without that set, report what was observed and how traversal ended. Do not infer completeness from the absence of an exception.

Put a small validator at the boundary

The following Python example checks a deliberately narrow contract. It does not fetch a website, validate that a URL belongs to the requested source, or prove that row values are true.

def validate_result(result):
    if not isinstance(result, dict):
        raise ValueError("result must be an object")
    if result.get("schema_version") != "records.v1":
        raise ValueError("unsupported schema version")
    if result.get("stop_reason") not in {
        "exhausted", "row_limit", "blocked"
    }:
        raise ValueError("unknown stop reason")
    rows = result.get("rows")
    if not isinstance(rows, list):
        raise ValueError("rows must be a list")
    seen = set()
    for index, row in enumerate(rows):
        if not isinstance(row, dict):
            raise ValueError(f"row {index} must be an object")
        for field in ("record_id", "source_url"):
            value = row.get(field)
            if not isinstance(value, str) or not value.strip():
                raise ValueError(f"row {index}: missing {field}")
        key = row["record_id"].strip()
        if key in seen:
            raise ValueError(f"duplicate record_id: {key}")
        seen.add(key)
    return result

Here, IDs are unique within this one result. A cross-source collection would need a key that includes the source identity. Do not copy this deduplication rule into a different dataset without checking what makes a record unique.

The validator permits additional fields so optional additions do not automatically break this consumer. It also permits an empty list. Whether an empty collection is an acceptable business result belongs in the coverage and acceptance rules, not in a guessed minimum row count.

Test the contract without pretending to test the crawler

Before connecting it to an importer, try synthetic fixtures: a valid response, a partial blocked response, an empty response, a duplicate ID, a missing source URL, an unsupported version, and a non-list rows value.

The last four should be rejected by this validator. The first three should pass shape validation, with their different business meanings preserved. Passing these fixtures establishes the behavior of this local check. It says nothing about browser access, pagination correctness, or extraction accuracy on a live portal.

Then test the actual saved skill separately, with authorized access and new inputs. Inspect source evidence as well as the returned structure. Keep those results distinct from the synthetic checks.

Version meaning changes, not only field names

A change from a local record ID to a global identifier may keep the same JSON type while changing the meaning. Changing a date from local time to UTC can do the same thing. Treat those changes as contract decisions.

Record the skill revision and contract version with each run. When introducing a breaking change, retain a representative old result and exercise the migration before replacing the importer. A version label is useful only when consumers actually check it.

For missing optional values, document whether null means not observed, unavailable, or not applicable. If your business needs those distinctions, add a separate status field rather than overloading null with several incompatible meanings.

Our Skill Forge workflow guide discusses turning exploration into something reusable. An output contract gives the next caller a concrete boundary to review, even when the website path changes.

Which breaks your downstream pipeline more often: changed fields, duplicate records, or an incomplete collection that looked successful?