Your shorter platform list can make the lookup budget less safe.
You're approving a recurring job, and someone replaces “search everywhere” with a shortlist. Fewer checks sounds reassuring. Then you notice that filtered lookups also emit Not Found rows, while unfiltered lookups return only Found rows.
A result fee follows an emitted dataset row, including a negative result. Your estimator needs to count possible output, rather than predict how many profiles exist.
The shortlist changes what becomes a row
The filter changes what the caller needs answered. A broad search returns confirmed profiles. A shortlist also reports where no profile was confirmed, so those checks become dataset items too.
That makes the shortlist useful for coverage tracking. It also makes “expected profiles found” the wrong budget variable.
Example input for a scheduled job, with a deliberately restricted local platform allowlist:
import assert from 'node:assert/strict';
import { ApifyClient } from 'apify-client';
import { z } from 'zod';
const Query = z.string().min(1).refine(
value => value.split('{?}').length <= 2,
'This budgeting policy accepts at most one wildcard per query',
);
const LookupInput = z.object({
queries: z.array(Query).min(1),
platforms: z.array(
z.enum(['1337x', '2Dimensions', '9GAG']),
).optional(),
maxResults: z.number().int().min(1)
.max(Number.MAX_SAFE_INTEGER).optional(),
}).strict();
type LookupInputValue = z.infer<typeof LookupInput>;
const run_input = LookupInput.parse({
queries: ['john{?}doe', 'johndoe'],
platforms: ['1337x', '2Dimensions'],
maxResults: 50,
});
The validator intentionally accepts a subset of valid requests. Extend its allowlist when your approved shortlist changes.
It also rejects repeated wildcard markers. The supported expansion shows a single marker; a budgeting policy shouldn't invent semantics for more elaborate patterns.
The input editor prefills maxResults with 50. Omitting the value means unlimited results. Scheduled jobs should send an explicit value instead of borrowing a default from a screen nobody opens.
What does the event-fee ceiling cover?
Separate the charged events before writing the estimator.
| Store event | Allowance to reserve | What doesn't bound it |
|---|---|---|
apify-default-dataset-item |
Every potentially emitted row, regardless of status | Your expected number of confirmed profiles |
apify-actor-start |
A memory-specific start-event ceiling | Username count or the result cap |
The start charge depends on Actor memory: one event per GB, with a minimum of one event. A generic “one start per run” assumption therefore isn't a defensible bound.
Platform usage is additional. Neither this estimator nor a returned-row limit establishes a ceiling for that usage.
Keep those budget categories separate in your ledger. Otherwise an accurate event estimate can still produce a misleading approval.
Count expansion before applying the row limit
A {?} marker expands into three username targets. For john{?}doe, those targets are john_doe, john-doe, and john.doe.
An empty platform selection searches all 442 supported values. A conservative estimator reserves a possible row for every target-platform pair, even when unfiltered mode will emit only confirmed profiles.
Duplicates are checked once, but the estimator below doesn't reduce its allowance for overlapping inputs or expansions. Over-reserving is preferable to depending on uncertain deduplication boundaries.
This function calculates event fees using integer monetary units and requires an independently established memory allowance:
const MONEY_SCALE = 100_000n;
const prices = {
FREE: { item: 99n, start: 10_000n },
GOLD: { item: 49n, start: 3_000n },
} as const;
type Tier = keyof typeof prices;
const StartAllowance = z.object({
memoryConfigKey: z.string().min(1),
maxEvents: z.bigint().min(1n),
}).strict();
function usd(units: bigint): string {
if (units < 0n) throw new Error('Negative monetary amount');
const whole = units / MONEY_SCALE;
const fraction = (units % MONEY_SCALE).toString()
.padStart(5, '0').replace(/0+$/, '');
return fraction ? `${whole}.${fraction}` : `${whole}`;
}
function eventFees(tier: Tier, rows: bigint, starts: bigint) {
return rows * prices[tier].item + starts * prices[tier].start;
}
function eventCeiling(
rawInput: unknown,
tier: Tier,
rawStartAllowance: unknown,
) {
const input = LookupInput.parse(rawInput);
const allowance = StartAllowance.safeParse(rawStartAllowance);
if (!allowance.success) {
throw new Error('A memory-specific start-event bound is required');
}
const targets = input.queries.reduce(
(total, query) => total + (query.includes('{?}') ? 3n : 1n),
0n,
);
const platforms = BigInt(input.platforms?.length || 442);
const possibleRows = targets * platforms;
const requestedLimit = input.maxResults === undefined
? possibleRows : BigInt(input.maxResults);
const rows = requestedLimit < possibleRows
? requestedLimit : possibleRows;
return {
rows,
eventUnits: eventFees(tier, rows, allowance.data.maxEvents),
memoryConfigKey: allowance.data.memoryConfigKey,
};
}
maxResults caps the combined total of Found and Not Found rows, across usernames in input order. It doesn't reserve equal coverage for each username.
The memory configuration key belongs to your deployment policy. It isn't an Actor input or an automatically verified platform setting.
Bind the allowance to the configuration actually used at launch. If your scheduler can't establish that relationship, reject the guaranteed estimate. A confident guess remains a guess, just with better formatting.
Check the arithmetic without guessing memory
Keep published pricing checks separate from operational assumptions. These assertions fix the start-event count explicitly; they don't establish the allowance for your deployment.
Example arithmetic checks and a sample allowance for a pinned memory configuration:
const priceChecks = [
['FREE', 100n, '0.199'],
['FREE', 1_000n, '1.09'],
['GOLD', 100n, '0.079'],
['GOLD', 1_000n, '0.52'],
] as const;
for (const [tier, rows, expected] of priceChecks) {
assert.equal(usd(eventFees(tier, rows, 1n)), expected);
}
assert.throws(
() => eventCeiling(run_input, 'FREE', undefined),
/memory-specific/,
);
const exampleAllowance = {
memoryConfigKey: 'Example-pinned-memory-configuration',
maxEvents: 1n,
};
const examplePerRun = eventCeiling(
run_input, 'FREE', exampleAllowance,
);
Those amounts cover event fees, plus platform usage.
The sample allowance is an explicit assumption for the example configuration. Replace it with an approved bound before connecting the scheduler.
Reserve attempts, including retries and rechecks
A per-run ceiling answers whether a request fits. A recurring budget must also answer how many requests you'll permit.
Reserve retry attempts before enabling them. Otherwise the retry path quietly becomes a second purchasing department.
Example schedule policy, using the same request envelope for scheduled runs, retries, and targeted rechecks:
const schedulePolicy = {
scheduledRuns: 30n,
retriesPerRun: 2n,
targetedRecheckRuns: 5n,
eventBudgetUnits: 2_000_000n,
};
const permittedAttempts =
(schedulePolicy.scheduledRuns + schedulePolicy.targetedRecheckRuns)
* (1n + schedulePolicy.retriesPerRun);
const scheduleEventUnits =
examplePerRun.eventUnits * permittedAttempts;
assert(
scheduleEventUnits <= schedulePolicy.eventBudgetUnits,
'Schedule exceeds its event-fee budget',
);
This deliberately reserves the full request envelope for targeted rechecks. Price them separately if they use different inputs.
In a concurrent scheduler, the comparison needs an atomic reservation in your budget ledger. A read followed by a separate write lets overlapping workers approve the same remaining balance.
Record an attempt before sending its start request. If submission times out, reconcile that attempt before retrying. An uncertain response isn't evidence that nothing started.
Put the spending cap on the request
Pre-flight estimation and a platform-side cap solve different problems.
The estimate rejects an oversized request before it incurs a start charge. The cap limits what a launched pay-per-event run may charge during execution. Neither replaces the other.
The control path should look like this:
validate input + memory allowance
|
reserve event budget atomically
|
start with maxTotalChargeUsd
|
record run identity + partial-coverage policy
This submission function expects an existing ledger reservation and a configuration key supplied by your deployment controller:
async function submitApproved(
input: LookupInputValue,
tier: Tier,
allowance: unknown,
activeMemoryConfigKey: string,
reservedEventUnits: bigint,
) {
const checkedInput = LookupInput.parse(input);
const plan = eventCeiling(checkedInput, tier, allowance);
if (activeMemoryConfigKey !== plan.memoryConfigKey) {
throw new Error('Memory configuration changed after approval');
}
if (reservedEventUnits < plan.eventUnits) {
throw new Error('Insufficient reserved event budget');
}
const token = process.env.APIFY_TOKEN;
if (!token) throw new Error('APIFY_TOKEN is required');
const endpoint = new URL(
'https://api.apify.com/v2/acts/'
+ 'nimbryxo-studio~username-lookup-across-400-plus-social-platforms/runs',
);
endpoint.searchParams.set('maxTotalChargeUsd', usd(plan.eventUnits));
const response = await fetch(endpoint, {
method: 'POST',
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
},
body: JSON.stringify(checkedInput),
});
if (!response.ok) {
throw new Error(`Start request returned ${response.status}`);
}
const started = z.object({
data: z.object({ id: z.string() }),
}).parse(await response.json()).data;
const client = new ApifyClient({ token });
const finished = await client.run(started.id).waitForFinish();
if (!finished) throw new Error(`Reconcile run ${started.id}`);
return {
runId: started.id,
datasetId: finished.defaultDatasetId,
status: finished.status,
coverage: 'partial' as const,
};
}
The configuration comparison checks your controller's assertion. It doesn't inspect or lock remote memory settings. Keep configuration changes coordinated with submissions.
Persist the returned run identity as soon as the start response arrives in your production worker. Retain that identity even if waiting later fails.
The worked example is Username Lookup Across 400+ Social Platforms. The reusable pattern is admission control followed by execution-time enforcement.
Partial datasets can't authorize deletions
A capped run can leave useful rows behind without completing the intended coverage. Reaching the input row limit also leaves later checks outside the returned dataset.
Treat capped output as partial unless a separate coverage check proves otherwise. A successful run status alone doesn't establish that every requested check is represented.
Validate retrieved rows, then attach your own ingestion metadata:
const LookupRow = z.object({
username: z.string(),
platform: z.string(),
url: z.string(),
status: z.enum(['Found', 'Not Found']),
});
function prepareObservations(items: unknown[], runId: string) {
const rows = z.array(LookupRow).parse(items);
return {
runId,
coverage: 'partial' as const,
allowDeletes: false as const,
observations: rows.map(row => ({
...row,
evidence: row.status === 'Found'
? 'confirmed-profile' as const
: 'unconfirmed-check' as const,
})),
};
}
The added fields are your ingestion contract, not Actor output.
A Not Found result can include a blocked or timed-out check. It can't safely become an “available username” assertion. Likewise, an absent row can't erase a previously observed profile.
For current-state storage, upsert by (username, platform). Keep run identity separately for attempt history and replay control. Append-only ingestion without a stable observation key turns retries into duplicate rows.
If no profiles are confirmed in unfiltered mode, an empty dataset ends with a failed status. Don't automatically interpret that combination as a transient infrastructure failure and spend the retry allowance repeatedly.
Before enabling the schedule
- Set an explicit row limit and approve the platform allowlist.
- Bind the start allowance to the active memory configuration.
- Reserve scheduled attempts, retries, and rechecks atomically.
- Send the event spending cap with every start request.
- Budget platform usage separately.
- Preserve partial observations and prohibit deletion from missing rows.
An unfinished check is unfinished work. It isn't evidence that a profile disappeared.
Top comments (0)