Your first email won an alphabetical contest.
You're wiring a contact array into your CRM's primary address field. emails[0] looks like the whole integration.
The crawler lowercases, deduplicates and sorts addresses after exclusions. Email array order expresses alphabetical order, not recipient suitability.
Why did the primary address change?
Sorting makes a set predictable. It doesn't tell you which inbox handles sales.
Example: this synthetic fixture adds a billing address while leaving the sales address untouched.
const fixtureBefore = ["sales@example-business.com"];
const fixtureAfter = ["billing@example-business.com", ...fixtureBefore].sort();
if (fixtureBefore[0] === fixtureAfter[0]) {
throw new Error("The fixture should expose alphabetical routing");
}
Your source data improved. Your routing got worse.
The reusable fix is to separate discovery from selection. Keep the entire address set, choose through an explicit policy, and treat changes to that choice as reviewable events.
A public address also doesn't establish a named person's identity. Company enrichment and person enrichment need different evidence.
Give the run an account boundary
Start with an existing account mapping. Don't ask the crawler to decide which CRM company owns a website.
Example: this input scans a mapped website, waits for completion, and reads its dataset.
import { ApifyClient } from "apify-client";
import { z } from "zod";
import pg from "pg";
const target = {
id: "example-crm-account",
websiteUrl: "https://example-business.com",
};
const run_input = {
urls: [target.websiteUrl],
maxPagesPerSite: 20,
smartPrioritise: true,
};
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client
.actor("nimbryxo-studio/website-contact-page-email-and-phone-crawler")
.call(run_input);
if (run.status !== "SUCCEEDED") {
throw new Error(`Contact run ${run.id} ended with ${run.status}`);
}
const { items } = await client.dataset(run.defaultDatasetId).listItems();
const datasetSizeMatches = items.length === run_input.urls.length;
This recipe deliberately submits a single website. The crawler returns a record per site, keeping this dataset bounded.
That boundary has a cost: each invocation starts another charged run. At volume, batch mapped websites and paginate dataset reads before widening this example.
An unexpected dataset size holds contact replacement. It still reaches validation and durable review storage, so the evidence survives the failed assumption.
Run completion is the outer gate. Site-level success still needs checking.
Validate the contract, keep the evidence
A malformed contact row shouldn't become a plausible CRM update.
Use passthrough validation for additions, while rejecting incompatible values in fields the integration already understands.
| Incoming condition | Ingestion decision | CRM consequence |
|---|---|---|
| Additional field | Preserve it and flag the extension | Existing routing can continue |
| Incompatible known field | Quarantine the item | Keep the previous contact |
Site-level error
|
Record the failure | Keep the previous contact |
| Unexpected dataset size | Preserve the batch for review | Hold all replacements |
This schema checks the contact contract without rejecting invalid-URL failure rows merely because their websiteUrl isn't a URL.
const socialKeys = [
"facebook", "instagram", "linkedin", "twitter",
"youtube", "tiktok", "github", "pinterest",
] as const;
const socialFields = new Set<string>(socialKeys);
const contactShape = {
type: z.literal("website_contacts").optional(),
websiteUrl: z.string().min(1),
emails: z.array(z.string().email()),
phones: z.array(z.string()),
socialLinks: z.object(Object.fromEntries(
socialKeys.map(key => [key, z.string().url().optional()])
)).passthrough(),
contactPageUrl: z.string().url().optional(),
pagesCrawled: z.number().int().nonnegative(),
crawledAt: z.string().datetime({ offset: true }),
error: z.enum([
"Could not fetch any pages from this URL",
"invalid URL",
]).optional(),
};
const knownFields = new Set(Object.keys(contactShape));
const Contact = z.object(contactShape).passthrough().refine(
row => row.error !== undefined || row.type === "website_contacts",
{ message: "A successful contact record needs its type" },
);
Strict validation would also reject harmless additions. Silent passthrough would hide them.
The middle ground preserves evidence and creates a review event. Validation failures remain durable, with their original payload and failure details.
Make the preference a policy
Role inference belongs to your application. An alias named sales is a routing preference, not proof of who reads it.
Example: this configurable policy prefers an approved sales alias, then a general greeting, and abstains when the choice is ambiguous.
const routingPolicy = {
version: "sales-role-a",
roles: ["sales", "hello"],
domains: ["example-business.com"],
};
function selectEmail(emails: string[]) {
for (const role of routingPolicy.roles) {
const candidates = emails.filter(address => {
const separator = address.lastIndexOf("@");
const local = address.slice(0, separator);
const domain = address.slice(separator + 1);
return local === role && routingPolicy.domains.includes(domain);
});
if (candidates.length === 1) {
return { email: candidates[0], reason: `approved role: ${role}` };
}
if (candidates.length > 1) {
return { email: null, reason: `ambiguous role: ${role}` };
}
}
return { email: null, reason: "no approved role address" };
}
if (selectEmail(fixtureBefore).email !== selectEmail(fixtureAfter).email) {
throw new Error("Adding billing should not change sales routing");
}
The approved domains come from your account configuration. A website can publish addresses belonging to another organization; accepting every discovered domain weakens that boundary.
Don't add an alphabetical fallback. Abstention is useful information.
Syntax validation, mailbox deliverability and recipient suitability answer different questions. This policy addresses the last question through visible preferences. It doesn't perform a deliverability check.
Keep website identity separate from company identity
websiteUrl identifies the crawled URL, with a scheme added when omitted. It isn't an immutable company identifier.
Use your account ID plus the exact returned URL as the snapshot key. That makes repeated ingestion idempotent within an explicit mapping.
Don't silently remove www, merge paths or rewrite hosts. A domain migration or changed account mapping needs reconciliation.
The write boundary looks like this.
dataset
-> unexpected size -> durable batch review, replacements held
-> each item
-> invalid, failed or unmapped -> durable review event
-> valid -> approved alias or abstention -> contact snapshot
|
selection change
|
CRM review
Keep frequently queried values in typed columns. Keep the complete item in JSONB for investigation and future extraction.
Duplicating emails is deliberate: the typed array supports comparisons, while raw JSONB preserves the received evidence. The extra storage buys simpler operational queries.
This schema also gives review events a replay-safe identity.
const setupSql = `
CREATE TABLE IF NOT EXISTS contact_snapshots (
target_id text NOT NULL,
website_url text NOT NULL,
emails text[] NOT NULL,
selected_email text,
selection_reason text NOT NULL,
policy_version text NOT NULL,
crawled_at timestamptz NOT NULL,
raw jsonb NOT NULL,
run_id text NOT NULL,
item_index integer NOT NULL,
selection_changed boolean NOT NULL,
addresses_changed boolean NOT NULL,
PRIMARY KEY (target_id, website_url)
);
CREATE TABLE IF NOT EXISTS contact_review_events (
run_id text NOT NULL,
item_index integer NOT NULL,
kind text NOT NULL,
details jsonb NOT NULL,
PRIMARY KEY (run_id, item_index, kind)
);
`;
The selected address is a proposal for the CRM. A changed proposal should enter review before replacing an established recipient.
Commit replacements and alerts together
Prepare every item before writing. Failed sites and invalid records never enter the replacement batch.
This preparation also flags additional fields separately from routing abstentions. A batch-level event uses a negative item index, outside the dataset's indexes.
const accepted: Array<{
item_index: number;
record: z.infer<typeof Contact>;
raw: unknown;
selection: ReturnType<typeof selectEmail>;
}> = [];
const reviewEvents: Array<{
item_index: number;
kind: string;
details: unknown;
}> = [];
if (!datasetSizeMatches) {
reviewEvents.push({
item_index: -1,
kind: "dataset_size_mismatch",
details: {
expected: run_input.urls.length,
received: items.length,
raw: items,
},
});
}
for (const [item_index, raw] of items.entries()) {
const parsed = Contact.safeParse(raw);
if (!parsed.success) {
reviewEvents.push({
item_index,
kind: "invalid_item",
details: { raw, issues: parsed.error.issues },
});
continue;
}
const record = parsed.data;
if (record.websiteUrl !== target.websiteUrl || record.error !== undefined) {
reviewEvents.push({
item_index,
kind: record.error !== undefined ? "site_failed" : "unmapped_url",
details: { raw },
});
continue;
}
const extraFields = [
...Object.keys(record).filter(key => !knownFields.has(key)),
...Object.keys(record.socialLinks)
.filter(key => !socialFields.has(key))
.map(key => `socialLinks.${key}`),
];
if (extraFields.length) {
reviewEvents.push({
item_index,
kind: "schema_extension",
details: { fields: extraFields, raw },
});
}
if (!datasetSizeMatches) continue;
const selection = selectEmail(record.emails);
if (selection.email === null) {
reviewEvents.push({
item_index,
kind: "routing_abstained",
details: { reason: selection.reason },
});
}
accepted.push({ item_index, record, raw, selection });
}
Write the batch and its review events in a single statement. A database failure then commits neither.
The freshness condition prevents an older crawl from replacing a newer snapshot. Equal timestamps count as replays; deliberately reprocessing a changed policy requires a separate reconciliation path or a fresh crawl.
This statement flags address-set changes independently from selection changes, then closes the pool after database work finishes.
const ingestSql = `
WITH written AS (
INSERT INTO contact_snapshots AS saved (
target_id, website_url, emails, selected_email, selection_reason,
policy_version, crawled_at, raw, run_id, item_index,
selection_changed, addresses_changed
)
SELECT
$4, x.record->>'websiteUrl',
ARRAY(SELECT jsonb_array_elements_text(x.record->'emails')),
x.selection->>'email', x.selection->>'reason',
$5, (x.record->>'crawledAt')::timestamptz,
x.raw, $3, x.item_index,
(x.selection->>'email') IS NOT NULL, false
FROM jsonb_to_recordset($1::jsonb) AS x(
item_index integer, record jsonb, raw jsonb, selection jsonb
)
ON CONFLICT (target_id, website_url) DO UPDATE SET
emails = EXCLUDED.emails,
selected_email = EXCLUDED.selected_email,
selection_reason = EXCLUDED.selection_reason,
policy_version = EXCLUDED.policy_version,
crawled_at = EXCLUDED.crawled_at,
raw = EXCLUDED.raw,
run_id = EXCLUDED.run_id,
item_index = EXCLUDED.item_index,
selection_changed =
saved.selected_email IS DISTINCT FROM EXCLUDED.selected_email,
addresses_changed =
saved.emails IS DISTINCT FROM EXCLUDED.emails
WHERE EXCLUDED.crawled_at > saved.crawled_at
RETURNING *
)
INSERT INTO contact_review_events (run_id, item_index, kind, details)
SELECT run_id, item_index, 'selection_change',
jsonb_build_object(
'selectedEmail', selected_email,
'reason', selection_reason,
'policyVersion', policy_version
)
FROM written WHERE selection_changed
UNION ALL
SELECT run_id, item_index, 'address_set_change',
jsonb_build_object('websiteUrl', website_url, 'emails', emails)
FROM written WHERE addresses_changed
UNION ALL
SELECT $3, item_index, kind, details
FROM jsonb_to_recordset($2::jsonb) AS e(
item_index integer, kind text, details jsonb
)
ON CONFLICT (run_id, item_index, kind) DO NOTHING;
`;
const pool = new pg.Pool();
try {
await pool.query(setupSql);
await pool.query(ingestSql, [
JSON.stringify(accepted),
JSON.stringify(reviewEvents),
run.id,
target.id,
routingPolicy.version,
]);
} finally {
await pool.end();
}
A newly selected address also creates a selection event. An ordinary address addition creates its own event without pretending the CRM recipient changed.
The event table is a durable handoff, not an alert sender. Your consumer still needs retry-safe delivery and an explicit approval gate.
Reinvoking this module starts another crawl. For ingestion-only retries, retain the run ID and replay its dataset instead of paying for fresh collection. Fresh runs incur event charges plus platform usage.
What an empty result can actually mean
The Website Contact Page Email & Phone Crawler reads returned HTML, including mailto: and tel: links. It doesn't render client-side content.
It also excludes placeholder and noreply-style addresses, file-like strings and certain embedded-code domains. A clean array therefore isn't a complete inventory of every address associated with a company.
Crawling follows the input hostname. Host variants remain separate, and the crawl can exhaust available links before reaching its page limit.
Those boundaries matter when interpreting an empty successful result. It means no retained address was found within the fetched material.
A failed site is different. Its empty contact fields accompany an error, so this ingestion records the failure and preserves the previous snapshot.
A successful empty result produces an abstention. The snapshot can reflect that observation while the CRM keeps its established recipient pending review.
Before the CRM write
- Alert on terminal run failures and unexpected dataset size.
- Review invalid items and unmapped URLs before retrying replacement.
- Route schema extensions to integration maintenance.
- Send selection changes and abstentions to contact review.
- Keep ordinary address-set changes out of recipient-change paging.
- Record approval separately from the selection proposal.
Alphabetical order can organize your evidence. It shouldn't choose who receives your pitch.
Top comments (0)