You're building a regional price index, and the currencies don't match your market labels. The exchange-rate join looks guilty. Check the input first.
For keyword searches, country selects the market; pasted startUrls keep the market their URLs point to.
This Example input combines a keyword search with a pasted search URL.
import { ApifyClient } from 'apify-client';
import { z } from 'zod';
const client = new ApifyClient({
token: process.env.APIFY_TOKEN,
});
const input = {
country: 'DE',
searchQueries: ['handmade jewelry'],
startUrls: [
{ url: 'https://www.etsy.com/search?q=handmade+jewelry' },
],
maxItems: 10,
includeDetails: false,
};
const run = await client
.actor('nimbryxo-studio/etsy-product-listings-and-shop-research-scraper')
.call(input);
if (run.status !== 'SUCCEEDED') {
throw new Error(`Inspect the run before publishing: ${run.status}`);
}
const { items } = await client
.dataset(run.defaultDatasetId)
.listItems();
The awkward consequence is downstream: assigning input.country to every returned row records your intention, rather than proving the row's market.
A successful run doesn't settle that distinction. Neither does a valid exchange rate.
Which symptom does the finished dataset show?
Start with the returned currency and price, then inspect the submitted input. Treat the expected currency as your index's acceptance policy, rather than something inferred from a country label.
| Symptom | Evidence that separates the causes | Likely misdiagnosis | Fix and cost |
|---|---|---|---|
| Off-policy currency with pasted inputs | Currency differs from your expectation, and startUrls contains routes you haven't verified for that market |
Broken exchange-rate lookup | Correct the source URLs or use keywords; replacement collection costs another run |
| Off-policy currency with keyword-only input | Currency differs, but there are no pasted URLs | The pasted-URL routing rule | Keep the discrepancy unresolved and inspect the original row and submitted query; investigation delays publication |
| Several nonempty currency codes | Distinct returned currency values exceed your single-currency policy |
A parser randomly changing amounts | Isolate sources before recollecting; separate runs add start charges and platform usage |
| Empty pricing values |
price or currency is blank |
A zero-price product or an implicit default currency | Hold the row outside the index; coverage decreases until you obtain usable pricing |
These symptoms can overlap. A mixed dataset can also contain blank prices.
The useful distinction is between evidence of a mismatch and evidence of its cause. A currency count supplies the former. Input provenance helps with the latter.
Inspect pricing without coercing it
Card fields that the page doesn't expose can arrive as empty strings. Your boundary should preserve that distinction instead of turning a blank price into a usable amount.
This detector separates schema failures, empty pricing values, distinct currencies, and currencies outside your policy.
const pricingRow = z.object({
price: z.string(),
currency: z.string(),
});
const expectedCurrency = 'EUR'; // Your index's acceptance policy.
const checked = items.map((item, index) => ({
index,
result: pricingRow.safeParse(item),
}));
const accepted = checked.flatMap(({ index, result }) =>
result.success ? [{ index, row: result.data }] : [],
);
const currencyCounts = new Map<string, number>();
for (const { row } of accepted) {
const currency = row.currency.trim();
if (currency) {
currencyCounts.set(currency, (currencyCounts.get(currency) ?? 0) + 1);
}
}
const report = {
invalidRowIndexes: checked
.filter(({ result }) => !result.success)
.map(({ index }) => index),
emptyPricingIndexes: accepted
.filter(({ row }) => !row.price.trim() || !row.currency.trim())
.map(({ index }) => index),
offPolicyIndexes: accepted
.filter(({ row }) =>
Boolean(row.currency.trim()) &&
row.currency.trim() !== expectedCurrency,
)
.map(({ index }) => index),
mixedCurrencies: currencyCounts.size > 1,
currencyCounts: Object.fromEntries(currencyCounts),
};
The indexes point back into items, so you can inspect the original records without discarding their other fields.
A schema failure belongs in a separate investigation. It says the pricing contract your pipeline expects wasn't satisfied. It doesn't establish a regional routing problem.
An empty dataset supplies no currency evidence at all. Don't classify that as a currency match simply because the mismatch count is empty.
Separate a routing clue from proof
The market selector configures the proxy exit country, browser locale, and generated search prefix. That explains why a keyword query can follow your regional intent while an independently supplied URL needs inspection.
For the German search in the opening input, /de/search is the explicit regional route. An unprefixed path is a reason to examine the source, not proof of which currency every returned listing will carry.
This check combines the dataset evidence with the input mode and inspects the known German prefix.
const routingDiagnosis =
report.offPolicyIndexes.length === 0
? 'No off-policy currency detected in these items'
: input.startUrls.length > 0
? 'Pasted routes need inspection; row attribution is unresolved'
: 'Keyword-only mismatch; pasted-URL routing does not explain it';
const routeChecks = input.startUrls.map(({ url }) => {
const parsedUrl = new URL(url);
return {
url,
hasGermanPrefix:
parsedUrl.pathname === '/de' ||
parsedUrl.pathname.startsWith('/de/'),
};
});
Don't turn that prefix check into a universal URL validator. It checks the search route used here. It doesn't establish a rewriting convention for every supported market or page type.
Also, don't use the listing's returned url as a substitute for its originating search. A destination listing URL isn't the submitted search URL.
Count currencies before converting prices
A mixed currency batch isn't inherently invalid. It becomes invalid when your index treats the batch as a single market or currency without preserving the evidence.
Keep the boundaries visible:
submitted source + intended market
|
returned price + currency
|
pricing and market checks
|
normalization and valuation
|
price index
Currency conversion addresses units. It can't repair the market from which you collected the observation.
That distinction matters for regional comparisons. Converting a foreign-market price into your reporting currency doesn't make it an observation of your intended market.
Mixed inputs also make attribution harder. The returned pricing fields don't identify which submitted query or URL produced a row. Save the submitted input alongside the run, rather than trying to reconstruct it later.
For a strict regional index, use a separate run for each source you need to diagnose independently. That buys clearer attribution, with additional start charges and platform usage.
Repair the source boundary
If your workflow starts with keywords, keep the query as a query. That avoids constructing regional search paths yourself.
If you need a particular category or shop page, retain the URL workflow and verify the market-specific route. Don't replace a catalog source with a keyword search and assume you kept the same population.
For the repair, this Example input keeps only the explicit German search URL.
const repairedInput = {
country: 'DE',
startUrls: [
{ url: 'https://www.etsy.com/de/search?q=handmade+jewelry' },
],
maxItems: input.maxItems,
includeDetails: input.includeDetails,
};
Removing the overlapping keyword input makes the next dataset easier to attribute.
Avoid a blanket regular expression that inserts a country segment into every URL. Search, category, shop, and listing routes deserve explicit handling. Validate the route you intend to collect, then inspect its returned currencies.
A corrected input gives you a better collection boundary. Keep the currency acceptance check on the replacement batch.
Hold uncertain prices before the index
For Etsy Product Listings & Shop Research Scraper, keep the raw pricing fields available while you decide which observations qualify.
This partitions schema-valid rows using the policy already defined and blocks an empty batch from publication.
const classified = accepted.map((entry) => ({
...entry,
eligible:
entry.row.price.trim() !== '' &&
entry.row.currency.trim() === expectedCurrency,
}));
const pricingCandidates = classified.filter(({ eligible }) => eligible);
const heldForReview = classified.filter(({ eligible }) => !eligible);
const publicationBlocked =
items.length === 0 ||
report.invalidRowIndexes.length > 0 ||
heldForReview.length > 0;
pricingCandidates still aren't normalized monetary amounts. They're rows eligible for the next parsing stage.
Preserve the original price string and use a parser whose separator rules you can justify. Removing punctuation indiscriminately can change an amount while leaving a perfectly respectable-looking number.
For Postgres or n8n, stage uncertain observations separately from the published index. Retain the raw row, run identity, submitted source, intended market, and rejection reason in your own ingestion envelope.
Those are pipeline metadata, not fields to pretend the Actor returned.
The catch is reduced coverage. Holding uncertain prices can delay a regional report. Silently relabeling them can publish a convincing report about the wrong market.
Before you release the batch
- Check run status before evaluating pricing.
- Review empty values separately from currency mismatches.
- Inspect pasted routes whenever they're present.
- Preserve source provenance before mixing datasets.
- Normalize accepted prices using explicit parsing rules.
- Keep replacement observations from becoming additional index entries.
An exchange rate can change the unit. It can't change where you looked.
Top comments (0)