Your proxy toggle cannot isolate search result changes

Your proxy toggle won't explain why the search results changed. You're already comparing datasets, the ingestion job needs an answer, and another rerun sounds cheaper than another hypothesis.

The input description makes that suggestion tempting: proxyConfiguration defaults to {"useApifyProxy": false} and promises operation without a proxy. But every run uses a managed proxy regardless of that setting.

Changing proxyConfiguration does not select a different network path. Result rows also expose no proxy telemetry, so their differences can't establish a routing explanation.

These Example inputs look like a control experiment, but the changed setting supplies no effective treatment:

const run_input = {
  queries: ["food in NYC"],
  market: "en-US",
  maxResultsPerQuery: 10,
  proxyConfiguration: { useApifyProxy: false },
};

const comparison_input = {
  ...run_input,
  proxyConfiguration: { useApifyProxy: true },
};

You need a different debugging decision: did the submitted search change, did the observations change, or are you asking the dataset a question it can't answer?

The symptom decides your next check

Search ingestion mixes request intent, ranked observations, and transport assumptions. Keep those separate before another run reaches your billing history.

Symptom	Evidence that distinguishes it	Easy misdiagnosis	Fix and its cost
Localized results differ	Saved manifests have different market values; rows have no dedicated market field	Proxy geography changed	Restore the intended market before comparing; collecting a replacement snapshot costs another run
Query coverage differs	Saved queries differ, or row query values disagree with their manifest	Search silently dropped keywords	Check the dataset-to-run association; repairing provenance takes storage and investigation
Result windows differ	Saved maxResultsPerQuery differs; per-query counts show the observed boundary	Pagination broke	Compare the shared rank window; a deeper collection adds emitted results
Destinations or positions move	Matching query-and-URL pairs have different position values; scrapedAt records collection windows	The toggle caused the movement	Retain both observations; accepting temporal uncertainty costs analytical certainty
Only the proxy setting changed	Manifest comparison isolates proxyConfiguration; rows provide no routing evidence	A valid network experiment	Stop toggle-only reruns; transport attribution needs separate operational evidence
The comparison cannot support a diagnosis	Run status is not SUCCEEDED, or items fail validation	An empty or unchanged search result	Resolve run or record validity first; diagnosis waits

Some causes are distinguishable only because you saved the submitted inputs. That's useful evidence, provided those manifests belong to the runs you're inspecting.

Reject an unusable comparison first

Use existing run IDs. The diagnostic should inspect the questionable runs before creating more of them.

These Example run IDs are placeholders for your archived runs; each manifest preserves the input submitted with its run:

import { ApifyClient } from "apify-client";
import { z } from "zod";
import { isDeepStrictEqual } from "node:util";

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const manifests = [
  { runId: "Example-baseline-run", input: run_input },
  { runId: "Example-comparison-run", input: comparison_input },
];

const ResultRow = z.object({
  query: z.string(),
  position: z.number().int().positive(),
  title: z.string(),
  url: z.string().url(),
  displayUrl: z.string(),
  description: z.string(),
  searchUrl: z.string().url(),
  scrapedAt: z.string().datetime({ offset: true }),
});

type Result = z.infer<typeof ResultRow>;
type Finding = {
  symptom: string;
  outcome: "observed" | "unknown";
  evidence: unknown;
};

const findings: Finding[] = [];

const snapshots = await Promise.all(
  manifests.map(async (manifest) => {
    const finished = await client.run(manifest.runId).waitForFinish();
    let rows: Result[] | null = null;

    if (finished?.status === "SUCCEEDED") {
      const { items } = await client
        .dataset(finished.defaultDatasetId)
        .listItems();

      const parsed = z.array(ResultRow).safeParse(items);
      if (parsed.success) rows = parsed.data;
    }

    return {
      ...manifest,
      status: finished?.status ?? "unknown",
      rows,
    };
  }),
);

for (const snapshot of snapshots) {
  if (snapshot.rows === null) {
    findings.push({
      symptom: "comparison_data_unusable",
      outcome: "unknown",
      evidence: {
        runId: snapshot.runId,
        status: snapshot.status,
        reason: snapshot.status === "SUCCEEDED"
          ? "Item validation failed"
          : "A successful finished run is required",
      },
    });
  }
}

A successful run with an empty array passes validation. It supplies no row-level observations, which is different from a failed run.

Keep those states separate. Otherwise, an operational failure can become a very confident report about disappearing search visibility.

The inline inputs keep this comparison small. For larger archived batches, ensure you retrieve the entire dataset before treating counts or missing URLs as evidence. A retrieved page isn't a complete snapshot.

Did the submitted search change?

The market selects localized ranked results. A different query changes the search itself. A different result limit changes how much of that ranking you observe.

None requires a networking explanation.

Compare the saved inputs directly, including query order and repeated entries:

const inputKeys = [
  "queries",
  "market",
  "maxResultsPerQuery",
  "proxyConfiguration",
] as const;

const changedInputs = inputKeys.filter(
  (key) => !isDeepStrictEqual(run_input[key], comparison_input[key]),
);

for (const key of changedInputs) {
  findings.push({
    symptom: key === "proxyConfiguration"
      ? "proxy_setting_changed"
      : "search_input_changed",
    outcome: "observed",
    evidence: {
      field: key,
      before: run_input[key],
      after: comparison_input[key],
    },
  });
}

if (
  changedInputs.length === 1 &&
  changedInputs[0] === "proxyConfiguration"
) {
  findings.push({
    symptom: "invalid_proxy_isolation_experiment",
    outcome: "observed",
    evidence: {
      changedInputs,
      reason: "This setting does not select a different network path",
    },
  });
}

This is deliberately conservative. Reordered queries count as a changed submission, without claiming their rankings must change.

When you archive manifests, store effective search settings explicitly. Otherwise, an omitted default and an explicitly supplied default can appear different in a raw comparison.

Also save the manifest when submitting the run. Reconstructing yesterday's input from today's scheduler settings gives you tidy evidence about the wrong thing.

Read the query labels and collection windows

A row's query lets you check its association with the submitted search. Its scrapedAt tells you when that observation was collected.

Neither field proves why a result appeared.

This detector records unexpected query labels, requested queries without rows, and the observed collection window:

function collectionWindow(rows: Result[]) {
  if (rows.length === 0) return null;

  const timestamps = rows.map((row) => Date.parse(row.scrapedAt));
  return {
    first: new Date(Math.min(...timestamps)).toISOString(),
    last: new Date(Math.max(...timestamps)).toISOString(),
  };
}

for (const snapshot of snapshots) {
  if (snapshot.rows === null) continue;

  const requested = new Set(snapshot.input.queries);
  const observed = new Set(snapshot.rows.map((row) => row.query));

  findings.push({
    symptom: "query_coverage",
    outcome: "observed",
    evidence: {
      runId: snapshot.runId,
      unexpected: [...observed].filter((query) => !requested.has(query)),
      withoutRows: [...requested].filter((query) => !observed.has(query)),
      collectionWindow: collectionWindow(snapshot.rows),
    },
  });
}

const [baseline, comparison] = snapshots;

if (baseline.rows !== null && comparison.rows !== null) {
  const before = collectionWindow(baseline.rows);
  const after = collectionWindow(comparison.rows);

  if (before && after && !isDeepStrictEqual(before, after)) {
    findings.push({
      symptom: "collection_windows_differ",
      outcome: "observed",
      evidence: { before, after },
    });
  }
}

An unexpected query label should send you toward provenance first: the wrong dataset, a mismatched manifest, or another association problem worth investigating.

A requested query without rows supports less. The items alone can't explain its absence. Don't promote that observation into a specific failure cause.

Different collection windows leave time as another possible explanation for ranking differences. They don't prove time caused the change, either. Search observations aren't frozen fixtures.

Compare rank windows without inventing missing results

A destination outside your collected window is unobserved. Calling it “removed from search” would make a larger claim.

That distinction matters when a downstream report turns URL membership into competitor visibility or prospect discovery.

This detector flags requested boundaries and compares literal query-and-URL identities without discarding repeated positions:

function rankIndex(rows: Result[]) {
  const index = new Map<string, number[]>();

  for (const row of rows) {
    const key = JSON.stringify([row.query, row.url]);
    const positions = index.get(key) ?? [];
    positions.push(row.position);
    index.set(key, positions);
  }

  for (const positions of index.values()) positions.sort((a, b) => a - b);
  return index;
}

for (const snapshot of snapshots) {
  if (snapshot.rows === null) continue;

  const counts = new Map<string, number>();
  for (const row of snapshot.rows) {
    counts.set(row.query, (counts.get(row.query) ?? 0) + 1);
  }

  findings.push({
    symptom: "observed_result_boundaries",
    outcome: "observed",
    evidence: {
      runId: snapshot.runId,
      queries: [...new Set(snapshot.input.queries)].map((query) => ({
        query,
        count: counts.get(query) ?? 0,
        atRequestedLimit:
          counts.get(query) === snapshot.input.maxResultsPerQuery,
      })),
    },
  });
}

if (baseline.rows !== null && comparison.rows !== null) {
  const before = rankIndex(baseline.rows);
  const after = rankIndex(comparison.rows);
  const identities = new Set([...before.keys(), ...after.keys()]);

  const changes = [...identities].flatMap((identity) => {
    const beforePositions = before.get(identity) ?? [];
    const afterPositions = after.get(identity) ?? [];

    return isDeepStrictEqual(beforePositions, afterPositions) ? [] : [{
      queryAndUrl: JSON.parse(identity) as [string, string],
      beforePositions,
      afterPositions,
    }];
  });

  findings.push({
    symptom: "destination_or_position_changes",
    outcome: "observed",
    evidence: changes,
  });
}

Reaching the requested count doesn't prove more results existed beyond it. Falling short doesn't identify a pagination failure.

The comparison also preserves URL strings exactly. Normalizing them into domains would answer a different question and could hide destination changes.

If the submitted limits differ, restrict your analytical comparison to their shared rank window. Otherwise, the larger collection can manufacture apparent additions simply by looking further down.

Leave proxy routing unresolved

The Actor handles proxy setup automatically for reliable access. The toggle therefore can't create the network distinction its default description suggests.

For this worked example, Bing Organic Search Results Scraper, transport attribution remains outside the result record.

Make that boundary explicit in the findings:

findings.push({
  symptom: "proxy_routing",
  outcome: "unknown",
  evidence: {
    reason: "Result items expose no proxy telemetry",
    proxyOnlyInputChange:
      changedInputs.length === 1 &&
      changedInputs[0] === "proxyConfiguration",
  },
});

const diagnosis = { changedInputs, findings };

diagnosis now separates observed differences from an unanswered routing question. You can attach it to the incident or persist it beside the manifests.

Don't infer proxy identity, exit geography, or route changes from searchUrl, destination domains, or item counts. A search URL describes the fetched search page. It isn't a transport trace.

Spend on evidence that can answer the question

The useful split is small enough to keep beside your scheduler:

Different search results
  -> submitted search changed? Compare intent first.
  -> collection windows differ? Preserve temporal uncertainty.
  -> only proxy setting changed? No effective treatment.
  -> need transport attribution? Obtain separate operational evidence.

Stopping toggle-only reruns saves event charges and platform usage without sacrificing a valid experiment. There wasn't one to begin with.

If routing attribution matters, seek operational evidence tied to the actual runs, such as transport records available through your support process. That takes investigation, and the dataset can't substitute for it.

For ingestion, preserve each snapshot under its run identity before comparing it. A rerun is another observation, so appending it blindly to a current-state table can turn debugging into duplicate business records.

Keep observation history separate from whichever projection your dashboard calls “current.” That costs storage and a deliberate update policy. It also keeps a diagnostic rerun from quietly rewriting the report you're investigating.

Before you authorize another run

• Verify the run status and dataset association.
• Retrieve and validate the complete comparison datasets.
• Compare saved query, market, and result-limit settings.
• Inspect query coverage and scrape windows before explaining rank movement.
• Require separate evidence for a transport claim.

A rerun should buy evidence your existing records can't provide.