DEV Community

Cover image for Why a polling preview is not a new event
Appstruct
Appstruct

Posted on

Why a polling preview is not a new event

A polling trigger can show two rows in a manual preview and emit nothing on its first automatic run. Before treating this as a failed connection, check what the trigger uses as its baseline.

There are three different operations to test:

  1. Preview the records the source currently returns.
  2. Establish a snapshot for later comparisons.
  3. Compare a later snapshot with the preceding successful snapshot.

Those operations can share a read function while producing different outputs. In particular, a manual preview should not silently move the state used by an active polling workflow.

Model the comparison with stable IDs

Here is a small JavaScript teaching example. Each row has a stable string ID and a reliable numeric revision supplied by the source. It illustrates comparison only; it is not a complete n8n node or an event-delivery system.

function detectChanges(previous, current, kind) {
  if (previous === null) return [];
  if (kind !== 'new' && kind !== 'updated') {
    throw new Error('Expected new or updated');
  }
  return current.filter((row) =>
    kind === 'new'
      ? !previous.has(row.id)
      : previous.has(row.id) && previous.get(row.id) !== row.revision
  );
}

const baseline = new Map([['r1', 1]]);
const current = [
  { id: 'r1', revision: 2 },
  { id: 'r2', revision: 1 },
];

console.assert(detectChanges(null, current, 'new').length === 0);
console.assert(detectChanges(baseline, current, 'new')[0].id === 'r2');
console.assert(detectChanges(baseline, current, 'updated')[0].id === 'r1');
Enter fullscreen mode Exit fullscreen mode

On a first automatic read, the caller establishes the baseline without reporting every existing record as newly created. On a later successful read, it computes changes and stores the new snapshot. An unchanged source should produce no repeated output.

The updated comparison deliberately ignores an ID that was absent from the baseline: a new record and an update to an existing record are different events. Keep the comparison scope explicit. A different table, project or trigger mode needs separate state.

The example assumes a trustworthy revision. If your source lacks one, choose a documented comparison strategy, such as a deterministic fingerprint of the relevant returned fields. Ordinary JSON.stringify output can depend on object key order; normalization matters. Decide which field changes actually count.

Refuse an incomplete snapshot

A read limited to N records does not prove that the table contains only N records. If the response fills the limit, either complete pagination or fail the poll before replacing the last complete snapshot. Otherwise records that fall outside a partial response can disappear from the comparison and later look new again.

Test a source failure between two successful reads. The failed read should not replace a known baseline with an empty or partial snapshot. Also test a record created and deleted between polls: snapshot comparison may never observe it. This is a reason to choose a durable event log when every intermediate change matters.

Test the destination separately

Source comparison and downstream delivery have different failure boundaries. A poll may advance its snapshot before a later write or notification fails. Suppressing unchanged source rows therefore does not guarantee exactly-once delivery to a destination.

For a new-record destination, consider an idempotent operation or unique constraint keyed by source, table and stable record ID. For repeated updates, define a revision or payload-change policy; deduplicating only on record ID can discard legitimate later changes. Include timeout-after-success, retry and replay in the acceptance plan.

A useful minimum test sequence is:

  • Preview existing rows without moving the live baseline.
  • Establish the first automatic baseline and expect no output.
  • Add one record after that baseline and expect its ID.
  • Repeat the poll without changes and expect nothing.
  • Update one existing record and check the updated mode independently.
  • Force a source-read failure and confirm that the earlier baseline survives.
  • Fail a downstream action and verify the explicit retry or replay procedure.

Use fictional fixtures and keep execution history private. A successful preview verifies a read; it does not verify the whole delivery path.

References: the polling implementation and regression tests used to check these distinctions. The sample above is generic and uses revisions; the linked implementation uses fingerprints of returned data.

For the AppStruct-specific exercise, get the inactive practice workflow and acceptance log. The downloads contain no credentials or private project IDs; connecting requires your own compatible self-hosted instance and AppStruct account.

Disclosure: This article was prepared with AI assistance and checked against the linked source and tests. The publishing account is associated with the maintainers of that example integration. This is a teaching example, not a customer deployment or a claim of n8n verification.

Top comments (0)