Your outbound queue can shrink because a cleanup job mistakes uncertainty for rejection. You get fewer prospects and a reassuring green checkmark. Lovely.
In deep verification, the Actor probes a made-up address alongside the prospect's mailbox. If the server accepts that address too, the domain is catch-all: the verdict becomes risky, and confidence drops.
A catch-all verdict means mailbox acceptance is inconclusive, not that the prospect's address is invalid. Your ingestion job should preserve that distinction before a campaign rule turns it into deletion.
Ask the mailbox question explicitly
Use deep verification for this workflow. The default standard setting doesn't perform mailbox-level SMTP verification or catch-all detection.
Example input and setup for the module:
import { randomUUID } from 'node:crypto';
import { ApifyClient } from 'apify-client';
import { z } from 'zod';
import pg from 'pg';
const Input = z.object({
emails: z.array(z.string()),
verificationDepth: z.enum(['basic', 'standard', 'deep']),
concurrency: z.number().int().min(1).max(50),
}).strict();
const run_input = Input.parse({
emails: ['prospect@target-company.com'],
verificationDepth: 'deep',
concurrency: 5,
});
const client = new ApifyClient({
token: process.env.APIFY_TOKEN,
});
Keep concurrency conservative until you understand the destination servers. Increasing it can trigger SMTP rate limits, which gives you more uncertainty rather than better evidence.
The routing change belongs downstream of verification. You're changing what uncertainty permits, not trying to manufacture certainty with a more enthusiastic threshold.
Treat null as unproven
An inconclusive check is null. Don't coerce it to false.
That matters because checks.catchAll === false means something different from a catch-all test that never produced an answer. A boolean coercion would erase the distinction precisely where your sending policy needs it.
Validate the fields used by the workflow, including the nullable error path:
const Check = z.boolean().nullable();
const Item = z.object({
email: z.string(),
verdict: z.enum([
'valid', 'invalid', 'risky', 'disposable', 'unknown',
]),
confidence: z.number().min(0).max(100),
checks: z.object({
syntax: Check,
mxRecord: Check,
smtp: Check,
disposable: Check,
roleBased: Check,
freeProvider: Check,
catchAll: Check,
}).passthrough(),
mxHost: z.string().nullable(),
reason: z.string().nullable(),
verifiedAt: z.string().datetime().nullable(),
}).passthrough();
type VerifiedEmail = z.infer<typeof Item>;
Keep email as a string here. A syntax-invalid address is still a legitimate verification result. Requiring an email-shaped value would quarantine the very records your suppression rule needs.
Passthrough validation preserves additive fields. Strict validation would reject them immediately, which is reasonable for a tightly controlled internal producer.
Here, preserve the raw record and detect additional keys separately. That gives you something inspectable when the producer changes.
Split research eligibility from sending eligibility
A corporate-looking catch-all address deserves investigation, not an automatic sequence enrollment.
The split should look like this:
verification record
-> invalid or disposable -> suppress
-> role flag or missing evidence -> review
-> catch-all, non-free provider -> social research
-> positive mailbox evidence -> email candidate
The following policy deliberately gives shared addresses precedence over the catch-all route. Its confidence cutoff and priorities are local campaign settings, not guarantees from the verifier.
function route(item: VerifiedEmail) {
const checks = item.checks;
if (
item.verdict === 'invalid' ||
item.verdict === 'disposable' ||
checks.disposable === true
) {
return { tier: 'suppressed', channel: 'none', priority: 0 };
}
if (checks.roleBased !== false) {
return { tier: 'review', channel: 'manual', priority: 0 };
}
if (
item.verdict === 'risky' &&
checks.catchAll === true &&
checks.freeProvider === false
) {
return { tier: 'catch_all', channel: 'social_research', priority: 20 };
}
if (
item.verdict === 'valid' &&
item.confidence >= 90 &&
checks.smtp === true &&
checks.catchAll === false &&
checks.freeProvider === false &&
checks.disposable === false
) {
return { tier: 'candidate', channel: 'email_candidate', priority: 100 };
}
return { tier: 'review', channel: 'manual', priority: 0 };
}
A non-free provider flag doesn't establish enterprise ownership. Cross-check the domain against your account data before assigning the lead to a company.
Likewise, email_candidate isn't permission to send. Consent, suppression lists, verification age, and campaign policy still belong in the enrollment gate. Verification can't promise sender reputation.
Give each attempt a durable place
The Actor removes duplicate addresses case-insensitively. Use a trimmed, lowercase address as the ingestion key to match that identity boundary.
It's a mailbox key, not a person key. Aliases remain separate, and an address can eventually belong to someone else. Keep CRM contact identity outside this table.
Store an immutable batch envelope alongside typed current decisions:
| Storage | What it gives you | What it costs |
|---|---|---|
| Typed current columns | Simple campaign filters and indexed lookups | Schema changes when operational requirements change |
| Raw JSONB batch envelope | Original values, rejected records, and replay evidence | More storage and deliberate retention controls |
Create these tables before running the module:
CREATE TABLE verification_batches (
id uuid PRIMARY KEY,
attempt_order bigint GENERATED ALWAYS AS IDENTITY UNIQUE,
actor_run_id text UNIQUE,
state text NOT NULL DEFAULT 'running',
receipts jsonb NOT NULL DEFAULT '[]'::jsonb
);
CREATE TABLE lead_verification (
email_key text PRIMARY KEY,
email text NOT NULL,
verdict text NOT NULL,
confidence double precision NOT NULL,
verified_at timestamptz,
tier text NOT NULL,
channel text NOT NULL,
priority integer NOT NULL,
source_batch uuid NOT NULL REFERENCES verification_batches(id),
source_order bigint NOT NULL
);
Generate the batch ID in the application before inserting it. The database assigns its attempt order when you reserve the batch, before verification starts.
Those values serve different purposes. The UUID identifies an attempt during recovery; the database order prevents a delayed older attempt from overwriting a newer decision.
This also handles error records without inventing a verification timestamp. Their verifiedAt can remain null; ordering belongs to the ingestion attempt.
Read the dataset and commit the batch atomically
Use the dataset's JSON endpoint to retrieve the exported array. Don't quietly treat an SDK page as the entire batch.
This function validates each item, records unexpected keys, and writes accepted decisions in a batch statement:
function extraKeys(value: unknown, known: string[]): string[] {
if (!value || typeof value !== 'object' || Array.isArray(value)) {
return [];
}
return Object.keys(value).filter(key => !known.includes(key));
}
async function ingest(
pool: pg.Pool,
batchId: string,
datasetId: string,
) {
const response = await fetch(
`https://api.apify.com/v2/datasets/${datasetId}/items?format=json&clean=true`,
{ headers: { Authorization: `Bearer ${process.env.APIFY_TOKEN}` } },
);
if (!response.ok) {
throw new Error(`Dataset request failed: ${response.status}`);
}
const items = z.array(z.unknown()).parse(await response.json());
const accepted: Array<Record<string, unknown>> = [];
let held = false;
const receipts = items.map((raw, index) => {
const nestedChecks =
raw !== null && typeof raw === 'object' && 'checks' in raw
? raw.checks
: undefined;
const drift = [
...extraKeys(raw, Object.keys(Item.shape)),
...extraKeys(
nestedChecks, Object.keys(Item.shape.checks.shape),
).map(key => `checks.${key}`),
];
const parsed = Item.safeParse(raw);
if (!parsed.success || drift.length > 0) held = true;
if (!parsed.success) {
return { index, raw, drift, errors: parsed.error.issues };
}
const item = parsed.data;
accepted.push({
email_key: item.email.trim().toLowerCase(),
email: item.email,
verdict: item.verdict,
confidence: item.confidence,
verified_at: item.verifiedAt,
...route(item),
});
return { index, raw, drift, errors: [] };
});
await pool.query(`
WITH saved AS (
UPDATE verification_batches
SET state = $2, receipts = $3::jsonb
WHERE id = $1
RETURNING id, attempt_order
)
INSERT INTO lead_verification AS current (
email_key, email, verdict, confidence, verified_at,
tier, channel, priority, source_batch, source_order
)
SELECT
x.email_key, x.email, x.verdict, x.confidence, x.verified_at,
x.tier, x.channel, x.priority, saved.id, saved.attempt_order
FROM jsonb_to_recordset($4::jsonb) AS x(
email_key text, email text, verdict text,
confidence double precision, verified_at timestamptz,
tier text, channel text, priority integer
)
CROSS JOIN saved
WHERE true
ON CONFLICT (email_key) DO UPDATE SET
email = EXCLUDED.email,
verdict = EXCLUDED.verdict,
confidence = EXCLUDED.confidence,
verified_at = EXCLUDED.verified_at,
tier = EXCLUDED.tier,
channel = EXCLUDED.channel,
priority = EXCLUDED.priority,
source_batch = EXCLUDED.source_batch,
source_order = EXCLUDED.source_order
WHERE EXCLUDED.source_order > current.source_order
`, [
batchId,
held ? 'held' : 'ready',
JSON.stringify(receipts),
JSON.stringify(accepted),
]);
}
The receipt update and upsert share a statement, so they commit together. A database failure doesn't leave a successful receipt attached to unwritten decisions.
Malformed records remain in JSONB with validation errors. Accepted records can still update current state, but any schema failure or unexpected key holds the batch.
Campaign selection must join lead_verification.source_batch to a batch whose state is ready, then require the eligible tier. Treat that join as part of the contract.
A rejected record leaves its previous decision untouched. Your enrollment gate must still expire stale evidence.
Start once, then wait on the recorded run
Reserve the batch before starting the remote work. Save the returned run ID immediately so recovery can resume that run.
This driver supplies its own batch ID and uses asynchronous start with an explicit wait:
const pool = new pg.Pool();
const batchId = randomUUID();
try {
await pool.query(
'INSERT INTO verification_batches (id) VALUES ($1)',
[batchId],
);
const started = await client
.actor('nimbryxo-studio/bulk-email-deliverability-and-smtp-verification')
.start(run_input);
await pool.query(
'UPDATE verification_batches SET actor_run_id = $2 WHERE id = $1',
[batchId, started.id],
);
const finished = await client.run(started.id).waitForFinish();
if (!finished || finished.status !== 'SUCCEEDED') {
throw new Error('Verification run did not succeed');
}
await ingest(pool, batchId, finished.defaultDatasetId);
} catch (error) {
await pool.query(
`UPDATE verification_batches
SET state = 'failed' WHERE id = $1`,
[batchId],
);
throw error;
} finally {
await pool.end();
}
The worked example is Bulk Email Deliverability & SMTP Verification. The reusable part is the separation between evidence, ingestion, and campaign eligibility.
Re-ingesting a recorded dataset uses its existing batch ID. The ordering condition prevents that replay from replacing newer decisions.
Starting verification again is a fresh attempt, with fresh remote checks and charges. Don't make “retry ingestion” secretly mean “buy verification again.”
There's still a crash window between remote start and saving its ID. Those operations aren't atomic. Reconcile an unresolved batch before starting another run.
What can fail without a failed run?
An unexpected error while checking an address produces an unknown record. Its confidence is 0; its checks, mail host, and verification timestamp are null. The reason starts with Verification error:.
That record passes the schema and routes to review. A nullable timestamp is necessary here, not defensive decoration.
An SMTP timeout, temporary rejection, or policy block behaves differently. The SMTP check becomes inconclusive, and an address with MX records can still receive valid with partial confidence. That's why the email-candidate rule requires positive SMTP evidence.
Also, records are written after all addresses have been checked. An empty dataset during execution isn't evidence of an empty lead list. Read after successful completion.
Alert on held or failed batches, unresolved running batches, unexpected keys, and increases in unknown or inconclusive SMTP results. Establish distribution baselines from your own accounts before choosing alert thresholds.
Watch suppression changes too. A routing bug can remove prospects without producing an infrastructure error. The database will remain beautifully available.
Before the next campaign opens
- Resume ingestion from a recorded run ID before launching another verification.
- Keep rejected records inspectable, with restricted access and a retention policy.
- Require a ready batch when selecting campaign candidates.
- Review catch-all accounts through account research before considering email enrollment.
- Check freshness and suppression rules at enrollment time.
A useful verification pipeline preserves uncertainty until someone has enough evidence to act.
Top comments (0)