Compare threshold policies without selecting a winner from their labels

I maintain this Actor. This is an AI-assisted technical walkthrough checked against the installed source contract.

A threshold change can move many records between accept, review, and reject even when the underlying scores stay identical. Before choosing a new rule, I want to see which individual records change state and how the aggregate counts reconcile. Calling one policy strict does not establish that it produces better decisions.

US Brand Signal Policy Simulator accepts one to 100 unique signal IDs with integer normalized scores from 1 through 100. It compares one to 20 explicitly supplied policies. Each policy declares a review threshold, an accept threshold, and the supported threshold ordering. It uses those submitted scores; it does not calculate or validate the scoring model that produced them.

Compare three declared policies

The example below contains scores 96, 82, 61, and 39. The balanced policy reviews from 50 and accepts from 80. The strict policy reviews from 70 and accepts from 90. The broad policy reviews from 30 and accepts from 70. Accept takes precedence at the accept threshold; review covers the remaining scores at or above its threshold.

Balanced therefore accepts two records, reviews one, and rejects one. Strict accepts one, reviews one, and rejects two. Broad accepts two and reviews two. The score 82 moves from accept under balanced to review under strict. The score 61 moves from review under balanced to reject under strict.

These are deterministic consequences of the thresholds, not observed mistakes or successes. A rejected record is a policy outcome on the supplied scale. It is not evidence that the underlying event is false. To evaluate decision quality, join the policy outcomes to independent labels and a declared cost model outside this calculation.

Inspect sensitivity at record level

Start a review with the records whose outcomes change. Ask whether the underlying evidence supports their scores and whether the cost of manual review is represented in the policy discussion. A stable aggregate accept count can hide different individual records moving in opposite directions, so compare the pairwise changes as well as the totals.

Use neutral policy IDs if labels are steering the conversation. The report does not rank policies or choose a winner. A permissive policy can create more review work, while a restrictive one can withhold cases the team would want to inspect. Those are operational tradeoffs that need actual workflow evidence.

Record the score version, extraction period, policy definitions, and review objective with the exported report. Comparing this week's policy output to last week's output is ambiguous if the upstream score normalization changed between them.

For a bounded experiment, take one frozen authorized score export and compare a small set of explicit alternatives. Review the changed records, document the human decision, and keep execution in the downstream system separate. The useful deliverable is an auditable sensitivity report, with the policy choice and its justification still owned by the team.

Input and execution boundary

Save this workflow-specific contract example as input.json. This is a synthetic contract example for the workflow described above; it does not assert a real observed business event.

{
  "schemaVersion": "1.1",
  "rows": [
    {
      "signalId": "signal-001",
      "normalizedScore": 96
    },
    {
      "signalId": "signal-002",
      "normalizedScore": 82
    },
    {
      "signalId": "signal-003",
      "normalizedScore": 61
    },
    {
      "signalId": "signal-004",
      "normalizedScore": 39
    }
  ],
  "policies": [
    {
      "policyId": "balanced",
      "reviewThreshold": 50,
      "acceptThreshold": 80,
      "thresholdOrder": "review_lt_accept"
    },
    {
      "policyId": "strict",
      "reviewThreshold": 70,
      "acceptThreshold": 90,
      "thresholdOrder": "review_lt_accept"
    },
    {
      "policyId": "broad",
      "reviewThreshold": 30,
      "acceptThreshold": 70,
      "thresholdOrder": "review_lt_accept"
    }
  ]
}

A deliberate cloud execution can use the following asynchronous request. It creates a new run and may incur charges; this walkthrough did not execute it.

curl --request POST \
  'https://api.apify.com/v2/acts/zinin~us-brand-signal-policy-simulator/runs?build=0.2.2&timeout=180&maxTotalChargeUsd=0.10' \
  --header "Authorization: Bearer $APIFY_TOKEN" \
  --header 'Content-Type: application/json' \
  --data-binary @input.json

Save the returned run ID. Check that existing run before retrying after a client timeout, then inspect its terminal status, Dataset, and KVS OUTPUT. A timeout in the caller does not cancel a server run. The charge ceiling bounds the request; it does not guarantee delivery or remove the start event. The build selector pins the observed build number, while the run's actual build ID should be retained for provenance.

Price and reproducibility

On 19 October 2026, the observed FREE-plan rates are $0.005 for the automatic start event and $0.01 per complete delivered report. The platform can count start events according to allocated memory. Other subscription tiers differ. Verify the current Actor page before purchasing a run; optional external services, where enabled, are separate.

This article uses the installed 0.2.2 source contract and saved public input shape, checked on the stated date. The arithmetic examples are explanatory, not fresh cloud acceptance, customer activity, or evidence of revenue. Preserve your input, actual build ID, observation boundary, and terminal receipt when adapting the workflow.