A suspicious order arrives. Two visits share an IP address, an actor report shows several IPs, and the dashboard assigns a risk score. What should the reviewer actually conclude?
I’m Chris, founder of WebDecoy. Two questions on our Shopify launch discussion brought this into focus: what evidence survives a review, and how do you tell missing traffic from missing collection? Here is the distinction we use, including the limits of our current app.
1. Store observations separately from conclusions
| Observation | What it does not establish by itself |
|---|---|
| Two requests used the same IP | That the same person made them, or that either request was abusive |
| A correlated actor used multiple IPs | The legal identity of a shopper, or fraud on every related order |
| An IP was classified as a VPN or proxy | That the customer intended abuse |
| A dashboard shows zero detections | That collection is healthy, or that no bots visited |
Shared Wi-Fi and company networks are straightforward reasons to be careful with IP-based conclusions. Correlation is an investigation aid. It needs supporting session or fingerprint evidence, and a reviewer should inspect the connection to the particular order.
2. A decision log and an evidence snapshot solve different problems
In WebDecoy’s Shopify app, saving a review records the decision, note, staff ID, and timestamp in review history. Updating the current review does not replace those earlier history entries. The UI shows the latest 100 decisions for that order.
That is not a frozen snapshot of all detection evidence. Detection details and the actor’s current cross-IP information can be fetched separately. Looking at them later is not a guarantee that you are seeing precisely what the reviewer saw earlier.
For any review tool, label these views explicitly:
- Decision history: what a reviewer recorded and when.
- Order-linked evidence: which detection or session supported the association.
- Current investigation context: related activity available now.
- Evidence snapshot: a preserved version of the evidence at decision time, if your tool actually implements one.
WebDecoy currently implements the review history, but not that full evidence snapshot. Do not turn a saved note into a claim of complete historical reproducibility.
3. Keep concurrent reviews from silently overwriting each other
A useful implementation pattern is an optimistic version check. Our review save path validates the submitted version, updates the current decision only when that version still matches, and inserts the history event in the same database transaction.
Conceptually:
begin transaction
update current review where version = submitted version
if no row changed: reject as a conflicting review
append decision + note + reviewer + timestamp
commit
The reviewer receives a reload message when another staff member has changed the record. This protects the review workflow from a lost update. It still does not preserve the entire detection screen: that requires a separate evidence-versioning design.
Keep customer and payment details out of free-text notes. Our review UI asks reviewers to avoid them.
4. Verify collection before interpreting an empty report
WebDecoy’s Setup & health page separates Storefront and Checkout pixel delivery. Each has a last-success timestamp and a last-error timestamp. A missing timestamp appears as Not yet observed.
For storefront collection:
- Enable and save the WebDecoy Bot Detection app embed on the published theme.
- Open Setup & health and choose Create test.
- Open the generated storefront test link, unlocking a password-protected store if needed.
- Return and look for Test detection received. Storefront collection verified.
The test verifies the storefront path; it does not verify checkout-pixel delivery. Test detections are labeled and excluded from traffic metrics, order review, and enforcement.
Checkout monitoring is separate. A connected pixel with no events is not proof of failure; event delivery can depend on customer consent. Likewise, the absence of a detection is not proof that every scraper was stopped. A client that neither executes JavaScript nor touches a decoy may not appear in the storefront report.
5. Make the response a deliberate step
In WebDecoy, saving a review records an assessment. Fulfillment actions remain in Shopify. The app’s order-created handler does not independently cancel an order or block checkout. Cancellation requires a separate merchant-configured and enabled Shopify Flow action.
The useful workflow is: verify collection, inspect the order-linked signals, record the reasoning, then choose the response.
See the workflow
Watch the short Shopify demo and setup walkthrough. The demo uses a test store and seeded data, not customer results. WebDecoy is listed on the Shopify App Store.
AI assisted with drafting; implementation claims were checked against the current app code. This is a workflow example, not a fraud-detection accuracy benchmark.
Top comments (0)