DEV Community: Apify

Buy in Poland, sell in France: a 110% margin that nets €4.61

Ahmed Jasarevic — Sun, 25 Oct 2026 12:35:01 +0000

The 110% Vinted arbitrage margin is really 9%

A four-market Vinted scan reported a 110.1% spread on one sneaker query. After five costs it was 9.2% — below the threshold I would act on, and one return eats two winners. Here is the arithmetic, the dataset, and the traps.

Disclosure: I built the Actor used in this post. The numbers are from a real run on 25 October 2026: 480 listings across PL, DE, IT and FR, ECB reference rates from 23 October. Everything below is recomputable from the dataset.

The number everyone screenshots

A Vinted scan returns a report that looks like this:

================ CROSS-COUNTRY ARBITRAGE REPORT ================
1. PL  162.99 PLN  (avg 37.09 EUR / median 37.89 EUR)  range 12-300  n=120
2. DE  71.27 EUR   (avg 71.27 EUR / median 50 EUR)     range 5-300   n=120
3. IT  76.92 EUR   (avg 76.92 EUR / median 50 EUR)     range 5-300   n=120
4. FR  77.92 EUR   (avg 77.92 EUR / median 55 EUR)     range 5-300   n=120
===============================================================
BUY in  PL  37.09 EUR
SELL in FR  77.92 EUR
Gross margin: 110.1% (before fees, shipping and Vinted's buyer protection)

110.1%. Buy for €37, sell for €78, more than double your money.

The last line is the honest one, and it is the line everyone skips: before fees, shipping and buyer protection.

This post is the rest of the calculation. Not to kill the idea, but because I would rather have ten trades that make 25% than a spreadsheet full of trades that make minus six.

First problem: the report compares averages, and the average is not the market

Here is the same run at the median instead of the average.

Market	Currency	Average (EUR)	Median (EUR)	Range (EUR)	n
PL	PLN	37.09	37.89	12 - 300	120
DE	EUR	71.27	50.00	5 - 300	120
IT	EUR	76.92	50.00	5 - 300	120
FR	EUR	77.92	55.00	5 - 300	120

The 110.1% comes from comparing the Polish average (37.09) to the French average (77.92). The medians tell a different story: 37.89 versus 55.00. That is a 45.2% spread, not 110.1%.

The gap between average and median is the shape of the distribution, not free money:

In PL the average sits below the median (37.09 vs 37.89), meaning a tail of very cheap listings drags the mean down.
In FR the average sits above the median (77.92 vs 55.00), meaning a tail of expensive listings pulls the mean up.

That €22.92 gap between the French mean and the French median represents €2,750 of value spread across 120 listings. It is a distribution, not a data error, and you cannot buy a distribution. You buy one item at one price, and you sell it for one price.

Notice something else in the table: DE and IT have identical medians (€50.00). The "third cheapest market" in the report is an artefact of the average, not a real price level. The report sorts markets by mean, which is exactly the statistic most distorted by outliers at both ends. Use it to pick candidates, never to size a trade.

Rule one: compare medians first, then pull P25 and P75 from the dataset. If the spread only exists between the means, it does not exist.

The five costs that eat the spread

Here is the same trade with the costs that Vinted, shipping and reality add. Everything below is in EUR, using the medians above and assumptions you can edit.

#	Cost	Who pays	Size in this example
1	Vinted fee on the buy leg	Buyer (you)	~5% + fixed, so €2.59 on €37.89
2	Inbound shipping to you	Buyer (you)	€4.50 (EU parcel, tracked)
3	Currency conversion	You	~1.2% per converted leg, €1.11 total
4	Packaging + handling	You	€0.50
5	Risk reserve	You	10% of buy price (returns, deadstock, price drift), €3.79

A few notes on each, because this is where most spreadsheets quietly cheat.

1. The Vinted fee. There are no selling fees on Vinted, which is why the fee is easy to forget: the buyer pays it. When you source, you are the buyer. Vinted describes it as typically a percentage plus a fixed amount, roughly 5% + a small fixed fee, varying by market. On a €38 item it is not the main cost. On a €380 item it is.

2. Inbound shipping. You pay to get the item to you. The buyer of your resale pays the label on the way out, so outbound shipping is not your cost, but packaging is. This asymmetry is why cheap, light items can work and bulky ones rarely do.

3. FX. The ECB reference rate is a mid-market rate. Your payment provider's rate is not. Vinted documents a 1.2% conversion fee on the USD/GBP pairs it handles directly; for other pairs the fee depends on your provider. Your bank may add its own on top. Converting in and out means you pay it twice.

4. Packaging. Small, real, and it compounds. A €0.50 mailer on a €38 trade is 1.3% of your capital, every cycle.

5. Risk reserve. This is the one people leave out because it does not appear on any invoice. Some items arrive not as described. Some do not sell for eight weeks while your money sits in them. Some sell after the market has moved down. A 10% reserve is not pessimism; it is the cost of doing this more than once.

The arithmetic

fee        = 37.89 * 0.05 + 0.70        =  2.59
fx         = (37.89 + 55.00) * 0.012    =  1.11
risk       = 37.89 * 0.10               =  3.79
packaging  =                               0.50
shipping   =                               4.50

all-in cost = 37.89 + 2.59 + 1.11 + 3.79 + 0.50 + 4.50 = 50.39
sell price  =                                            55.00
net         =                                             4.61
net margin  =                                             9.2%

The trade that showed a 110.1% gross margin nets €4.61, or 9.2%.

That is not a catastrophe. It is also not a business. Two numbers fall out of it:

Break-even sell price: €50.39. You already need to clear the French median (€55.00) just to make €4.61. If the item sells at the median price of the cheapest French listing band, you lose money.
A 25% net margin needs about €62.99. In this run, that is above the median and somewhere in the upper quartile of the French market. If your "arbitrage" only exists at the median, you are buying a lottery ticket with extra steps.

And then the return. One SNAD return — item not as described, refund, return shipping, packaging, your time — costs roughly €9 on a trade this size. That is two winners to pay for one loser. At a 10% return rate, the expected net per trade drops from €4.61 to about €3.25. At 20%, it is under €2. The margin is not just thin; it is one bad batch away from zero.

Here is the same model as code, so you can drop your own numbers in:

const FEE_PCT = 0.05;
const FEE_FIXED = 0.70;
const FX_LEG = 0.012;
const PACKAGING = 0.50;
const RISK = 0.10;

export function netMargin({ buy, sell, inboundShipping }) {
  const fee = buy * FEE_PCT + FEE_FIXED;
  const fx = (buy + sell) * FX_LEG;
  const risk = buy * RISK;
  const cost = buy + fee + inboundShipping + fx + PACKAGING + risk;

  return {
    cost,
    net: sell - cost,
    netPct: (sell - cost) / cost,
    breakEvenSell: cost,
    sellFor25Pct: cost * 1.25,
  };
}

console.table(netMargin({ buy: 37.89, sell: 55.00, inboundShipping: 4.50 }));
// cost 50.39 | net 4.61 | netPct 0.092 | breakEven 50.39 | 25% target 62.99

The real killer is not the fee. It is the border.

The model above assumes you can buy in Poland and sell in France. That assumption is doing more work than any of the five costs.

Your Vinted account is registered in one market. What you can buy and sell across borders depends on Vinted's international connections, the shipping options available to that pair of countries, item restrictions, weight limits and customs. Some connections are rolled out gradually and limited. On the connections Vinted documents publicly, electronics, cosmetics, jewellery and watches are commonly excluded or restricted, and those are exactly the categories with the widest spreads.

So in practice, "sell in FR" often means: have a French account, a French delivery address, French shipping options, and the ability to handle returns in French. That is an operations problem, not a data problem. It is also why the gross-margin number should be treated as a screening signal, not a business plan.

There is a second border that is not geographic: DAC7. If you complete 30 sales or earn more than €2,000 on Vinted in a calendar year, the platform reports your seller data to tax authorities. That does not automatically mean tax is due, but it does mean the hobby has a paper trail. Once you are buying to resell, you are probably a business, and the EU margin scheme for second-hand goods is the thing to read before you scale.

What actually survives the model

Running this across a few categories, four patterns hold up:

Same-market mispricing beats cross-border arbitrage. A mispriced listing in your own market has no customs, no FX, no cross-border shipping and no language problem. The scanner finds those too; you just have to stop filtering for a country flag.
High value-to-weight wins. Sneakers, jackets, small designer accessories. A €200 item that weighs 800g absorbs the same €4.50 shipping that destroys a €38 item.
P25 buy to P75 sell. If the spread only exists median-to-median, the trade dies at the first cost. Require the sell-side target to sit above P75 before you even look at the item.
Bundles amortise everything. One shipment, one fee, one packaging cost, several items. This is the only structural way to make low-priced categories work.

And the conclusion I did not expect: the data is worth more than the trades. If you already sell on Vinted, the same scan tells you which market your item is priced highest in, and what the top quartile looks like. You do not have to ship anything across a border to profit from that. Price at P75 in the market with the highest median, and you have captured most of the spread without the operations.

Five data traps to check before you trust a scan

Asking prices are not sold prices. Vinted does not publish a public sold-price feed. Everything you scan is a listing, including the ones that will never sell. This is the single biggest limitation of any Vinted price dataset, mine included. Treat medians as an upper bound on clearing prices, not a forecast.
Averages hide the tails. This run's headline 110.1% was mean-to-mean. The medians said 45.2%. Always recompute before you believe the report.
Duplicates across markets. The same listing can surface in several markets. itemId is stable; dedupe on it before you compute anything.
Condition mix. A market with more "satisfactory" items will look cheaper. In this run the French range starts at €5, which is either a different product, a damaged item, or a placeholder price. Filter by condition (status_ids) before comparing, or you are comparing two different products.
Sample size. n=120 is enough to see a shape and not enough to trust a percentile. If the difference between two markets is smaller than the spread of either, you have measured noise.

How to run the scan

The Actor takes query, countries (up to five per run), maxItems per market, and optional filters. CROSS_COUNTRY mode returns the listings plus the per-market report.

{
  "mode": "CROSS_COUNTRY",
  "query": "Nike Jordan 4",
  "countries": ["pl", "de", "fr", "it"],
  "maxItems": 120,
  "minPrice": 5,
  "maxPrice": 300,
  "status_ids": ["new_with_tags", "very_good", "good"],
  "priceBandPartitioning": true
}

Two things matter for data quality:

Price bands. Vinted caps how many results a single search pass returns. Setting maxPrice enables priceBandPartitioning, which splits the range into bands and sweeps each one, so a large query does not silently truncate at the cap.
Five markets per run. Run a second pass for the remaining countries and merge the datasets; itemId is your join key.

From code:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client
  .actor('ahmed_jasarevic/vinted-cross-country-arbitrage-scraper')
  .call({
    mode: 'CROSS_COUNTRY',
    query: 'Nike Jordan 4',
    countries: ['pl', 'de', 'fr', 'it'],
    maxItems: 120,
    minPrice: 5,
    maxPrice: 300,
  });

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.length, 'rows');

// quartiles per market. Replace RATES with the ECB reference rates for your run date.
const RATES = { EUR: 1, PLN: 0.2275, GBP: 1.17, CZK: 0.0405, SEK: 0.0895, DKK: 0.134 };

const pct = (arr, p) => {
  const sorted = [...arr].sort((a, b) => a - b);
  const i = (sorted.length - 1) * p;
  const lo = Math.floor(i), hi = Math.ceil(i);
  return sorted[lo] + (sorted[hi] - sorted[lo]) * (i - lo);
};

const byMarket = items.reduce((acc, row) => {
  (acc[row.country] ??= []).push(row.price * (RATES[row.currency] ?? NaN));
  return acc;
}, {});

for (const [country, prices] of Object.entries(byMarket)) {
  const clean = prices.filter(Number.isFinite);
  console.log(country, {
    n: clean.length,
    p25: pct(clean, 0.25).toFixed(2),
    median: pct(clean, 0.5).toFixed(2),
    p75: pct(clean, 0.75).toFixed(2),
  });
}

At $1.50 per 1,000 results, 480 rows costs about $0.72. That is cheap enough to run weekly, which is the point.

A seven-day protocol

Do not act on a single scan. Do this instead:

Day 1. Baseline scan, fixed query, fixed filters, five markets. Save the raw dataset.
Days 2-7. Same input, once a day. Track the median per market, not the average.
Day 7. Keep only pairs where the P75 sell side clears your all-in cost by 25%, and where the gap has held for at least three days. Gaps that appear and vanish in 24 hours are usually one seller, not a market.
Then verify by hand. Open the top five candidates, check condition, photos, size, seller feedback and whether the item is still live. The scanner gives you candidates; it does not give you judgement.

What I would not do

I would not run this as a dropshipping business. The cross-border operational layer, the return risk and the sold-price blindness are all working against you, and no amount of scraping fixes any of them.

I would run it as a pricing tool. It answers a question every reseller actually has: what does this thing clear for, in which market, at the top of the range? That answer is worth more than a 110% screenshot.

The Actor used here is Vinted Cross-Country Arbitrage — 26 European markets, ECB-normalised prices, per-market medians and pairwise gross margins. The fee and FX assumptions above are documented in the post; change them before you make a decision.

If you want the version of this analysis with your own category, run one scan and drop the numbers into netMargin(). The result is usually more useful than the headline.

What AI agents see when they call your Apify Actors

SIÁN Agency — Fri, 23 Oct 2026 07:09:09 +0000

This post first appeared on the Apify blog as part of Write for Apify.

I run more than 300 Apify Actors at SIÁN, my data-pipeline and automation agency, mostly for e-commerce and property data, and I spend a lot of my week looking at run telemetry.

In mid-July I was chasing a free-tier abuse problem. Someone had worked out that a free Apify account could call a monetized Actor over the API, grab the capped row allowance, abort before anything billed, and repeat. On July 14, 2026, I shipped a gate: block runs where the plan is free and the origin is API, before the Actor does any billable work. Console runs stayed free, because that's how people try an Actor.

The next day I checked whether the gate had holes, and found one. MCP is its own Apify RunOrigin, separate from API. It's what you get when someone wires the Apify MCP server into Claude, Cursor, or their own agent and the model calls your Actor as a tool. My gate didn't cover it.

So I extended it: block when the plan is free and the origin is either API or MCP. I shipped that across five Actors, feeling reasonably good about closing a hole before it cost me anything.

I had the story completely backwards.

What the telemetry actually said

Every run in my fleet writes a row to a Postgres table: origin, tier, outcome, rows, revenue. I finally aggregated the origin column across all 169,941 rows, which I should have done before writing the gate.

Origin	Runs	Share
`API`	156,482	92.0%
`MCP`	6,922	4.1%
`WEB` (Console)	5,183	3.1%
`SCHEDULER`	853	0.5%
`CLI`	501	0.3%

My first MCP run landed on June 17, 2026. Six weeks later it was 1,407 runs a week and still climbing, and it had quietly passed Apify Console to become my second-biggest source of runs.

Then the part that made me wince. Of those 6,922 MCP runs, exactly 44 were rejected by my abuse gate. That's 0.6%. The rest were 86.9% successful, half of them from paying accounts, and delivered 67,323 rows.

I had spent a day hardening a channel that was, by a wide margin, my healthiest one.

Agents are better callers than humans

Once I stopped treating MCP as a threat and started treating it as a user, I compared it with the other origins:

Origin	Success rate	Malformed input	Share paid
`MCP`	86.9%	1.6%	50.0%
`API`	88.0%	1.4%	80.0%
`WEB` (Console)	47.9%	8.8%	10.0%

Humans fail more than half their Console runs. Agents fail 13.1% of theirs, and send malformed input: 1.6% versus 8.8%.

That's not because models are careful. It's because the MCP server hands a model my input schema as JSON Schema, and the model sends exactly what that schema declares. A human in Console gets a rendered form: labeled fields, a dropdown of valid enum values, an example sitting in the box, and a validator that won't let them submit nonsense. None of that lives in the schema. It's scaffolding Console builds around the schema.

Take the scaffolding away and whatever my schema literally says becomes the entire contract. So I went looking for where my schemas were lying.

My first approach was wrong too

I did the obvious thing and grepped all 253 schemas the fleet had in July for the failure mode I already knew about: a field with a prefill (which only populates Console's editor) but no default (which the platform actually injects into every run). A model sending {} gets the default and never sees the prefill.

The grep found 74 Actors. I had a bug list, or so I thought.

I've been burned by static analysis enough times to check before acting on it, so I wrote a probe that calls an Actor through the MCP server with {}, precisely what a model sends when told nothing is required. I ran it against the 16 most-used of them.

Roughly half were fine:

Verdict	Actors	Share	What the agent sees
`REJECTED` (-32602)	9	56.2%	actionable protocol error, no run billed
`LOUD-FAIL`	2	12.5%	run failed, visibly
`SILENT-EMPTY`	3	18.8%	`SUCCEEDED`, zero rows
`SILENT-STATS`	2	12.5%	`SUCCEEDED`, rows that aren't data

The grep had flagged Actors that behave correctly, and it could never have found the category that actually matters. SILENT-EMPTY and SILENT-STATS are the same defect wearing different clothes. Five of these 16 calls failed in a way no agent can detect, and no amount of schema-reading surfaces that. You have to run the thing.

Getting those five counted correctly took two passes, because my first probe was also wrong. The MCP call returns after a wait cap of about 30 seconds, so any Actor slower than that comes back still RUNNING, and I was scoring that as a failure. Three property scrapers got marked broken when they were only slow. One of them was genuinely broken in the quiet way, and my bug had filed it under the loud way, which flattered the result.

The fix is to poll until the run settles, and to give "still running" a verdict of its own so it can never be mistaken for a failure:

TERMINAL = {"SUCCEEDED", "FAILED", "ABORTED", "TIMED-OUT"}
if status is not None and status not in TERMINAL:
    return "UNKNOWN-STILL-RUNNING"     # never call a slow Actor a broken one

The Actor knew exactly what was wrong and told no one

Here's apartments-com-property-scraper, one of mine, called with {}. This is the complete set of signals a model receives.

Run metadata:

{ "status": "SUCCEEDED", "runTimeSecs": 3.846 }

The MCP result flag:

{ "isError": false }

And the summary the MCP server writes for the model, verbatim:

SUCCEEDED in 3.846s. 1 item; 32 fields available. Key-value store has 2 keys. Use get-dataset-items with datasetId=… and limit (for example 20) to fetch items (1 total). Available fields (dot notation): __retryHelper, searchMode, mode, queriesRequested, …

Every one of those says the call worked and produced a result. So the agent fetches the item:

{
  "__retryHelper": true,
  "searchMode": "byLocation",
  "queriesRequested": 0,
  "queriesSucceeded": 0,
  "listingsPushed": 0,
  "totalResultsAvailable": 0,
  "userTier": "PAID",
  "failures": [
    { "query": "(input)", "error": "location must be a non-empty string." }
  ]
}

There it is: location must be a non-empty string.

My Actor diagnosed the problem correctly, phrased it in plain English, and filed it inside a nested array, on a row that isn't a result, on a run marked SUCCEEDED.

An agent that asked for apartments now holds one object with 32 fields, none of which is an apartment. Best case it tells its user "no listings found." Worst case it tries to summarize a run-summary row as a property listing.

Nobody forgot to handle the error. The only thing missing was the idea that something other than a human reading a run log would ever need to find it. I built that Actor when the only caller I imagined was a person with a browser open. That field carries a default now, which is why the probe returns OK against it today.

A platform bug I found on the way

While isolating why the MCP server rejected some Actors cleanly and not others, I found a mismatch in the server itself.

Ask the MCP server for a tool definition and compare it against the Actor's own schema:

m = Mcp(tools="sian.agency/costco-data-scraper")
tool = m.list_tools()[0]
print(tool["inputSchema"].get("required"))     # []

Here's the relevant part of that Actor's .actor/INPUT_SCHEMA.json:

{
  "required": ["operation"],
  "properties": {
    "operation": { "type": "string", "default": "search", "enum": ["search", "productDetails"] }
  }
}

The Actor declares operation required. The tool definition handed to the model says nothing is required. So the model calls with {}, and:

MCP error -32602: Invalid arguments for tool "sian-dot-agency--costco-data-scraper".
Validation errors: root: must have required property 'operation'.

Rejected for missing exactly the field it was told it could omit.

The discriminator is the default. tools/list drops a field from required when that field carries a default. That's defensible on its own: a defaulted field is satisfiable without the caller. But tools/call validates against the raw schema and doesn't apply defaults first. The two halves disagree.

The control case confirms it. amazon-product-scraper also declares a required field, urls, but urls has a prefill and no default, so nothing gets stripped:

Actor	`INPUT_SCHEMA`	`tools/list`	Field has `default`?
costco-data-scraper	`['operation']`	`[]`	yes
ebay-data-scraper	`['operation']`	`[]`	yes
bayut-property-scraper	`['purpose']`	`[]`	yes
glassdoor-data-scraper	`['operation']`	`[]`	yes
booking-com-scraper	`['operation']`	`[]`	yes
amazon-product-scraper	`['urls']`	`['urls']`	no

Across the 253 schemas, 59 Actors declare a required field that also carries a default, or 23.3%. Every one tells a model a mandatory field is optional.

In fairness, it's recoverable. The error is clear, and a competent agent retries or calls fetch-actor-details, which reports required correctly. My aliexpress-product-scraper is in the affected set and is still my second-most agent-called Actor at 818 MCP runs, 95.0% of them paid. Models cope, but they burn a round-trip on the first call, every time.

Why Apify's QA can't catch this

Apify runs every public Actor daily and requires SUCCEEDED plus a dataset with rows in it within five minutes. Three consecutive failures flag it "under maintenance." It's a good gate and it has caught real breakage for me.

It runs the Actor with its prefill input.

A SILENT-EMPTY or SILENT-STATS run reports SUCCEEDED, so the gate sees what the agent sees and agrees with it. And the prefill is only one point in my input space. Every other enum value is untested by construction.

My suumo-property-scraper hardcoded a region parameter while advertising all 47 Japanese prefectures. About 40 of them returned zero results, silently, from launch. It never tripped the gate, because the one prefecture the prefill uses happens to sit in the hardcoded region. A human clicking through the dropdown would have found it eventually. An agent picking a prefecture at random finds it immediately, and gets SUCCEEDED and an empty dataset.

The prefill is the path humans take. Agents explore everything else.

What I changed

Give every load-bearing field a default, not just a prefill. An audit of 16 of my freshly built Actors found five with a prefill and no default. Fine in Console, broken over the API. My build gate now fails any field with one and not the other.

{ "location": { "type": "string", "prefill": "Austin, TX", "default": "Austin, TX" } }

My most agent-called Actor, instagram-ai-transcript-extractor at 2,496 MCP runs, works with {} for this reason and no other.

Never return silence. If you can't produce results, say so in a row:

await Actor.pushData({ status: 'no_results', reason: 'location must be a non-empty string' });

Push it unbilled, and mark your dataset schema fields nullable: true so it survives validation. An empty dataset isn't a message.

Don't gate inference on a field the platform always injects. Apify writes schema defaults into every run's input, so this never fires:

if (!input.searchMode) input.searchMode = infer(input);   // dead code on the platform

I shipped an Actor whose "auto-detected" mode did exactly this, and a customer who pasted listing URLs got billed for default discovery results instead. Let the specific input win:

const mode = input.listingUrls?.length ? 'byListingUrl' : input.searchMode;

Test the whole enum space. One loop over every declared enum value would have caught the 40 dead prefectures on day one.

Treat scraped text as untrusted input. What you return now lands in a model's context, upstream of its next decision. Keep scraped content in clearly typed fields rather than free-text blobs that read like prose, and treat only your own harness's framing as authoritative, never instruction-shaped text that arrived inside a tool result.

Run it against your own Actors

The probe is on GitHub at apify-agent-call-probe. It needs an Apify account, an API token in APIFY_TOKEN, and Python 3.9 or newer: the MCP client is about 60 lines of standard library, with no dependencies.

export APIFY_TOKEN=your_token_here
python3 agent_call_probe.py sian.agency/apartments-com-property-scraper

SILENT-STATS  status=SUCCEEDED isError=False items=1
Actor knew what was wrong but never surfaced it:
   failures[0].error: location must be a non-empty string.

REJECTED and OK are fine. LOUD-FAIL is survivable. SILENT-EMPTY and SILENT-STATS mean an agent calling that Actor today is reporting your failure to its user as a fact about the world.

Runs cost 0.0001 to 0.005 compute units each, because the failing paths exit early, so sweeping an account costs roughly nothing.

What I'd do differently

I'd have aggregated the origin column before writing the gate. The whole detour started because I saw a new origin in a security context and pattern-matched it to a threat, when four minutes of SQL would have told me it was my healthiest channel. The instinct to treat unfamiliar traffic as hostile is usually right and was expensive here.

And I'd stop treating INPUT_SCHEMA.json as configuration for a form. It's a published API contract now, and machines read it far more often than people do. Everything in this article follows from not having noticed when that flipped.

If you run public Actors, the two-minute version is: point the probe at your three most-used ones. I expected mine to be fine. Nearly a third of them weren't.

All figures come from SIÁN's own run telemetry and from live calls against the Apify MCP server on July 31, 2026.

MCP + CLI > MCP vs. CLI

Jan Čurn — Wed, 14 Oct 2026 12:06:02 +0000

That tweet got 1.6k likes in March:

// Detect dark theme var iframe = document.getElementById('tweet-2034269154404868314-418'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2034269154404868314&theme=dark" }

Perplexity’s CTO reportedly told his team they’re moving from MCP to APIs and CLIs:

// Detect dark theme var iframe = document.getElementById('tweet-2031795683897077965-871'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2031795683897077965&theme=dark" }

OpenClaw left MCP out on purpose:

// Detect dark theme var iframe = document.getElementById('tweet-2025701861031121331-772'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2025701861031121331&theme=dark" }

Garry Tan’s version was shorter: “MCP sucks honestly.”"

// Detect dark theme var iframe = document.getElementById('tweet-2031910564344262988-310'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2031910564344262988&theme=dark" }

They’re half right: the problem is real, but the diagnosis isn’t. What’s broken is how most agents use MCP, and the fix is not to throw the protocol away but to stop using it badly.

This piece builds on my talk at AI Engineer World’s Fair “MCP doesn’t suck. Your agent does.” (San Francisco, July 2026). Watch it on YouTube.

The problem is the harness, not the protocol

Most agents treat MCP tools as prompt-time function calls. Connect a server and every tool definition lands in the context window. Call a tool and the full result lands there too. Ten servers with five tools each, and you pay thousands of tokens on every turn before the agent has done anything. Results pile up, context rots, the agent gets slower and less accurate, and data the model never needed to see is now inside the model.

None of that is in the spec. MCP already has the fixes: progressive tool discovery (Anthropic’s tool search tool, Cursor’s dynamic context discovery), code mode (Cloudflare and Anthropic both wrote it up), and sub-agents with split context. But most MCP clients don’t implement them. The protocol moves faster than the ecosystem.

Now look at what CLIs get right, mostly by accident. Agents never load a CLI’s docs all at once; they read --help when they need it. They run CLI tools as code by default, chaining them with pipes and shell scripts. And every model knows the shell by heart, because there’s 50 years of it in the training data. The catch here: a CLI is a local black box. No standard transport, no standard auth, no discovery, no way to scope or intercept what it does to a remote service.

That’s why agents have MCP connectors, not CLI connectors.

MCP for remote, CLI for local

The fix is to use each where it’s strong. MCP stays the standard remote interface: server discovery, authentication, and access control. The CLI becomes the local interface between the agent and MCP. That’s what we built in mcpc, an open-source universal CLI client for MCP:

 ┌──────────┐   Bash()   ┌────────┐    MCP    ┌────────────┐
 │ AI agent │ ─────────► │  mcpc  │ ────────► │ MCP server │
 └──────────┘            └────────┘           └────────────┘
                                     Sessions, OAuth, Tools,
                                     Resources, Prompts,
                                     Tasks, x402, ...

The agent only needs one tool, Bash(), to call mcpc, learn how it works through --help, and interact with MCP servers. And it can leverage the full MCP protocol out of the box: persistent sessions, OAuth 2.1 (including enterprise SSO), tools, resources, prompts, async tasks, skills, x402 payments, and a code mode using --json. mcpc speaks MCP 2026-07-28 and falls back to older versions. No LLM inside, fast, and minimal dependencies. As a bonus, one login and one set of live sessions can be shared between many agents on the same machine.

What it looks like in the shell

brew install apify/tap/mcpc          # or: npm install -g @apify/mcpc

mcpc login mcp.apify.com             # OAuth once; tokens go to the OS keychain
mcpc connect mcp.apify.com @apify    # persistent session
mcpc @apify                          # server info, capabilities, instructions
mcpc @apify grep "crawler"           # progressive tool discovery
mcpc @apify tools-get search-actors  # one tool's schema, only when needed
mcpc @apify tools-call search-actors keywords:="tiktok scraper"

And code mode, where the agent writes a script instead of making a round trip through its context for every call:

mcpc @apify tools-call search-actors keywords:="tiktok scraper" limit:=1 --json \
  | jq -r '.content[0].text | capture("`(?<actor>[^`]+)`").actor' \
  | xargs -I {} mcpc @apify tools-call fetch-actor-details actor:="{}" --json

That’s progressive discovery and code mode on any MCP server, whether or not the agent’s own MCP client supports either.

What the evals say

We built connector-bench to measure the efficiency of mcpc against CLIs and naive MCP implementations. Each run gives an agent the same cross-server task under three harnesses: raw MCP with every tool loaded into context, the native CLIs for the same services, and mcpc. We track cumulative cost against wall-clock time.

Here’s an example comparing Apify MCP server with Apify CLI on an Actor discovery scenario:

In many of our tests, mcpc finishes the job faster than native CLI, albeit sometimes with higher cost. That’s understandable, as native optimized CLIs are usually less verbose and more token-efficient than a generic CLI wrapper for MCP. But we believe there is plenty of room for optimization and the performance of mcpc will converge towards native CLIs.

So stop saying CLI beats MCP

MCP + CLI beats both. Even OpenClaw uses MCP through mcporter, an MCP-to-CLI converter, which proves the point. The problem was never the protocol. It was that agents implemented MCP poorly.

MCP has ten official SDKs for languages like TypeScript, Python, or Go, but none for the shell, the most universal programming language. mcpc is our attempt to fill that gap, so any agent with a code runtime can use all of MCP.

Try it, break it, star it:

apify / mcpc

The most compatible MCP CLI client. Supports: tools, resources, prompts, completions, async tasks, skills, stdio and Streamable HTTP (MCP 2026-07-28 and 2025-11-25). Persistent sessions, OAuth 2.1 with DCR, CIMD, client credentials, and ID-JAG, OS keychain, --json for code mode, grep across servers, proxy for AI sandboxes, x402 & more.

mcpc — a universal MCP CLI client

mcpc is a command-line client for the Model Context Protocol (MCP) that maps every MCP operation to an intuitive shell command.

Use it to manually inspect and debug MCP servers, to script repeatable MCP workflows in plain shell, or to give AI agents the full MCP protocol through a single Bash() tool call, so they can interact with any MCP server and its latest capabilities using the most universal programming interface there is: the UNIX shell.

Key features:

🔧 Full MCP support - Tools, prompts, resources, async tasks, skills, notifications, and logging over stdio and Streamable HTTP.
🔄 Persistent sessions - Keep connections to multiple servers alive in parallel, whether the server protocol is stateful or stateless.
🗺️ Progressive tool discovery - Find relevant MCP tools on the fly to save tokens and increase accuracy.
🔌 Code mode - JSON output composes with…

View on GitHub

Your KYB agent reads "Active" and moves on. Five register answers that are true and still wrong

getregdata — Tue, 13 Oct 2026 14:36:28 +0000

On 4 December 2020 the Belgian Official Gazette published a list of 1,663 companies struck off the Crossroads Bank for Enterprises (KBO) on 24 November 2020 for not filing annual accounts for three years in a row. On 25 September 2026 we looked up 100 of them, spaced evenly through the list.

78 were still struck off. 30 of those 78 still showed Status "Active" on their KBO page, and 12 of the 30 also showed Legal situation "Normal situation".

Ask an agent "is this Belgian company active?" and hand it that page, and it answers yes. The answer is correct, since the register says Active. It is also the wrong thing to act on if you are about to extend credit to a company that has filed no balance sheet since 2020.

This post collects five results like that. All five come from registers we read every day for KYB, credit and insolvency checks, and several of them came out of our own tools before we fixed them. The runs succeeded, the rows parsed, and the result was not empty. The trouble is in what the data means, and a model has no way to know that from the JSON alone.

Other posts in this organization have covered empty-versus-failed results and timeouts in agent tool calls. This one is about results that arrive complete and still mislead.

Where the check belongs

Each trap below gets a check, and each check is a few lines of deterministic code that runs between the tool and the model. That placement matters. You can write "remember that Belgian companies can be struck off while Active" into a system prompt, but the agent has to recall it at the right moment, on the right field, for the right country. A function that runs on every record doesn't have to remember anything.

So the pattern is: fetch, check, then give the model a decision and the reasons for it instead of the raw status. The fetching part is ordinary Apify client code:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

def fetch(actor_id, run_input):
    run = client.actor(actor_id).call(run_input=run_input)
    items = client.dataset(run["defaultDatasetId"]).list_items().items
    return run, items

The checks don't care which tool produced the rows. Field names below are the ones our actors emit. If you read these registers some other way, map them to your own schema; the logic carries over.

1. "Active" is one field, and a strike-off lives in another

The Belgian Code of Economic Law (art. III.42 § 1, 4°) lets the KBO strike a company off ex officio when National Bank data show it has not filed annual accounts for at least three consecutive financial years. A strike-off is not a dissolution: dissolving a company for missing accounts takes a court (art. 2:74 of the Companies and Associations Code). So the Status field doesn't change. The strike-off sits on its own line on the company page, "Ex officio striking off", with a since-date, and it is lifted once the missing accounts are filed.

Our own Belgian actor didn't read that line until build 1.0.7 in September 2026. Before then it returned exactly what an onboarding check wants to see, a valid enterprise number with status Active, for companies that had been struck off for years. It now returns struckOff and a strikeOffs list with the reason, a type and the since-date. A company can carry two strike-offs at once, one for accounts and one for beneficial-ownership filings.

def check_kbo(rec):
    reasons = [
        f"struck off ex officio since {s.get('sinceDate')}: {s.get('reason')}"
        for s in rec.get("strikeOffs") or []
    ]
    if reasons:
        return "hold", reasons
    if rec.get("status") != "Active":
        return "hold", [f"status is {rec.get('status')}"]
    return "pass", []

The rule behind it applies well beyond Belgium: a status field summarises one dimension of a company's legal life, and the register may record other dimensions somewhere else. Before you trust a status field, find out what it does not cover.

2. One row is a record, not a company

Poland's insolvency and enforcement register (KRZ) lists proceedings, and a company can be party to many of them. Until build 1.1.19 our KRZ actor's company search returned one row per participant record, with the case fields empty. For Getin Noble Bank that meant one row, an enforcement case that had ended. The register holds six proceedings for it, one of them an ongoing bankruptcy.

An agent reading "1 result, ended" concludes the company once had a minor enforcement matter that is now closed. That is the opposite of the truth, and nothing in the row looks broken.

The actor now emits one row per proceeding, puts open proceedings first, and marks proceedingsComplete: false on any company whose full list it could not retrieve. Your check should still aggregate before deciding, because an agent that reads only the first row, or a no-code integration that does, makes the same mistake one level up.

def check_krz(rows):
    if any(r.get("proceedingsComplete") is False for r in rows):
        return "unknown", ["proceeding list incomplete for at least one record"]
    open_cases = [
        r for r in rows
        if (r.get("proceedingStatus") == "w toku")          # "in progress"
        or (r.get("proceedingStatus") is None and not r.get("proceedingEndDate"))
    ]
    if open_cases:
        kinds = sorted({r.get("proceedingKind") or "unknown" for r in open_cases})
        return "hold", [f"{len(open_cases)} open proceeding(s): {', '.join(kinds)}"]
    return "pass", [f"{len(rows)} proceeding(s), all closed"]

When you build a register check, write down the grain of a row (a filing, a participant, a proceeding, an announcement) before you write any logic that counts rows or reads the first one.

3. The first 100 rows are a sample, and a biased one

Spain's insolvency register (Registro Público Concursal) records three roles per person or company: debtor, disqualified, and insolvency administrator. For a director check, "disqualified" is the role that matters.

On 31 August 2026 we searched it for two common surnames. Martínez matched 9,587 entries and Fernández 8,889. In both cases the first 100 rows were all debtors, with zero disqualified people and zero administrators. In rows 101 to 250 the other roles turned up: 6 disqualified and 27 administrators for Martínez, 7 and 29 for Fernández. The first non-debtor row came at position 118 and 158.

The register lists debtors first. Our actor's default maxResults is 100. Put those together and a director check on a common name returns 100 rows, every one of them isDisqualified: false. That looks like a thorough negative, and it is really a sample of the one role that happens to come first.

We made the same mistake ourselves before we caught it. We first concluded that the disqualified flag was never populated, because 294 of 294 stored rows had it false. Every one of those rows came from a run capped at or near 100. What we had measured was the cap.

MAX = 100

def search_es(name, max_results=MAX):
    return fetch("regdata/spain-concursal-scraper",
                 {"searchMode": "byName", "name": name, "maxResults": max_results})

def check_es_roles(rows, max_results=MAX):
    if max_results and len(rows) >= max_results:
        return "unknown", [f"result hit the cap of {max_results}; this is the head of a sorted list"]
    hits = [r for r in rows if r.get("isDisqualified")]
    return ("hold", [f"{len(hits)} disqualified match(es)"]) if hits else ("pass", [])

If the check returns unknown, re-run with maxResults: 0 (no cap), or narrow the query with an identity number. Any capped list behaves like this. Unless you know the sort order, the head of the list tells you nothing about the rest of it.

4. A zero is a claim, and it needs evidence of the right size

The same Spanish register once gave our actor a false all-clear on every large name search. For the most common surnames it took about ten seconds to answer, longer than the actor waited. The actor read the empty results table as "0 matches" and stamped the result "verified against a control query".

The control query was real. It searched for a known insolvent company, and it passed every time. But that company has about a hundred entries, and the register answered for it in under three seconds. A query of that size only shows that small queries come back. It said nothing about a query with ten thousand matches.

Since build 1.0.14 the actor bases the verdict on the register's answer arriving, not on time elapsed. Any search the register did not answer is written to an UNVERIFIED record in the run's key-value store, and the status message says outright that it is not a clean result. Your side of the check is short:

def check_zero(run, rows):
    if run["status"] != "SUCCEEDED":
        return "unknown", [f"run {run['status']}"]
    unverified = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("UNVERIFIED")
    if unverified:
        return "unknown", ["register did not answer some searches; do not read as clean"]
    if not rows:
        return "pass", ["register answered: no entries"]
    return None, []   # rows present: hand over to the role check above

The general rule is about your own monitoring as much as anyone's tool. A canary or control query vouches only for queries of its own size and shape. If your pipeline's real queries are larger, slower or rarer than the canary, they need a positive control of their own, meaning a query you know should return a lot of results, run on the same schedule.

5. A clean number can come from the wrong row

Polish companies file financial statements with the court register, and one of the formats they come in is a rendered document, not structured data. In July 2026 our KRS financials actor read CD Projekt's FY2025 balance sheet and returned short-term liabilities of 279,811,608,000 and equity of 1,010, flagged success: true. Both values were well-formed numbers, and nothing about their format was wrong. Two adjacent table cells had been merged into one number, and the reporting period's date range had been read as a figure.

Format checks can't catch this, but the balance sheet itself can. Assets have to equal equity plus liabilities, and the asset subtotals have to add up to total assets. After the fix, CD Projekt's figures reconcile exactly, using values taken from two different tables: fixed assets 2,174,743 plus current assets 1,332,673 equals total 3,507,416, and equity 3,294,816 plus long-term 29,408 plus short-term 183,192 equals the same 3,507,416.

def check_balance(f, tol=1):
    need = ["totalAssets", "fixedAssets", "currentAssets",
            "equity", "longTermLiabilities", "shortTermLiabilities"]
    missing = [k for k in need if f.get(k) is None]
    if missing:
        return "unknown", [f"cannot verify, missing {missing}"]
    errs = []
    if abs(f["fixedAssets"] + f["currentAssets"] - f["totalAssets"]) > tol:
        errs.append("fixed + current != total assets")
    if abs(f["equity"] + f["longTermLiabilities"] + f["shortTermLiabilities"] - f["totalAssets"]) > tol:
        errs.append("equity + liabilities != total assets")
    return ("reject", errs) if errs else ("pass", [])

These two identities fit the summary balance sheet this filing format carries. If your schema splits out provisions or accruals, add them to the right-hand side. Keep the tolerance small, at rounding level.

The harder part is the income statement, which has no internal identity to test against. A wrong revenue figure passes every format check and nothing flags it. So after the balance-sheet fix our actor kept returning revenue, gross profit and net profit for that filing as null, on purpose, for two months. They came back in September only after we found a second table in the same filing that carries the same figures. A profit-and-loss figure is now published only when both tables agree. If a figure has no arithmetic cross-check, use a second source or return null. An agent will use whatever number it receives.

A smaller one: two kinds of null

One more, briefly, because it changes how you triage. Germany's Handelsregister output carries a legal-form code and a legal-form label. Until build 1.1.10 our actor had labels for only two of the codes, so SE, KG, UG and every other form came back with legalFormCode set and legalForm: null. A code with no label means a lookup gap in the tool. When both fields are null, the source document was not retrieved at all, and the run says so with a partial status. These two nulls call for different responses (report a bug, or re-run), and an agent told to treat null as "unknown" will handle both the same way. Check the sibling fields before you decide what a null means.

Wiring it into an agent

Put the checks together and the agent no longer receives a register row. It receives something like this:

def screen_be_company(enterprise_number):
    run, rows = fetch("regdata/belgium-kbo-company-scraper",
                      {"searchMode": "byNumber", "enterpriseNumbers": [enterprise_number]})
    if run["status"] != "SUCCEEDED" or not rows:
        return {"decision": "unknown", "reasons": ["no verified record"], "source": "KBO"}
    decision, reasons = check_kbo(rows[0])
    return {"decision": decision, "reasons": reasons, "source": "KBO", "runId": run["id"]}

Three design choices are deliberate here:

unknown is a separate decision. It is not a soft pass. A capped list, an unanswered search and an incomplete proceeding list all land there, and your workflow decides what unknown means, which is usually re-run or send to a human.
The reasons are plain sentences. The model can quote "struck off ex officio since 2020-11-24 for non-filing of annual accounts" to a user. It can't do much with struckOff: true.
The runId stays with the decision. When someone asks six months later why a supplier was put on hold, you can show the run that answered.

The model still does what it is good at. It explains the decision, drafts the email asking the supplier for its filed accounts, and works out what to do next. It just doesn't get to decide what "Active" means.

If you want to try the first check, the Belgian actor is on Apify. The checks themselves are plain Python, and they work just as well on data you collect yourself.

Which register field has misled you? Put it in the comments. We would like to add to this list, especially for registers we don't read.

Top 10 Real Estate Portals in 2026 — and What Data You Can Actually Extract

Ahmed Jasarevic — Tue, 13 Oct 2026 09:29:34 +0000

Zillow gets 247 million visits a month. Rightmove effectively is the UK house-price index. Idealista is where price-per-m² in Madrid, Milan and Lisbon gets checked.

Treat them as what they are: the biggest structured property databases on the planet — updated daily, no official public API.

Here are 10 of the largest property portals, what each one actually exposes, and how to get it out without writing a scraper.

The list (ranked by monthly traffic, Aug 2026)

#	Portal	Market	Visits/mo	Standout data
1	Zillow	USA	247M	Zestimate + rent estimates
2	Redfin	USA	64M	Price history, walk/transit scores
3	Rightmove	UK	57M	Sold prices, tenure
4	Idealista	ES/IT/PT	51M	Price/m², energy rating
5	Apartments.com	USA	47M	Rent ranges, leasing phones
6	ImmobilienScout24	Germany	39M	Energy class, construction year
7	Homes.com	USA	34M	Agent name + phone per listing
8	Funda	Netherlands	25M	Neighbourhood hierarchy
9	Trulia	USA	21M	Neighborhood context, price history
10	Bayut	UAE/Gulf	Gulf #1	Off-plan payment plans, AED prices

General classifieds (Craigslist, Leboncoin) excluded — they're not dedicated property portals. Bayut is ranked by regional leadership, not global traffic.

What you actually get per portal

Zillow — price, address components, beds/baths, living + lot area, Zestimate and rent estimates, lat/lng, property type, status, photos, stable property ID.

Redfin — price and price history, AVM (Redfin Estimate) history, sqft, MLS IDs, days-on-market, HOA, walk/transit/bike scores, agent + broker metadata.

Rightmove — price/currency, beds/baths, property type + tenure, coordinates, price-change history, branch/estate-agent info, Land Registry-backed sold prices.

Idealista — price, price/m², area, rooms, floor, GPS, energy rating, year built, fees/taxes, price history, agency contact info.

Apartments.com — property name, street/city/state/ZIP, rent ranges, beds/baths, sqft, leasing-office phone, URL, GPS.

ImmobilienScout24 — price + type (rent/sale), rooms, m², address, energy efficiency class, construction year, agent contacts, amenities.

Homes.com — address + geocoordinates, price, beds/baths, sqft, status, descriptions, listing agent name, phone and title, brokerage, open-house/sold flags.

Funda — street/postcode/city/province/neighbourhood/municipality, numeric + formatted price, size, status, buy/rent, agent contacts, coordinates, floor plans.

Trulia — price, beds/baths, interior + lot area, address + neighborhood, coordinates, agent/broker name + phone, per-property price history.

Bayut — AED asking prices, coordinates, beds/baths, amenities, off-plan payment-plan milestones, unit breakdowns, developer + agency profiles, completion status (UAE, KSA, Qatar, Jordan, Oman).

The schema overlap (why merging portals is easy)

Every one of them resolves to roughly the same record:

price, currency, address, bedrooms, bathrooms, size,
property_type, lat, lng, agent/branch, photos, status, url, scraped_at

Portal-specific extras are additive: Zestimate (Zillow), walk scores (Redfin), tenure (Rightmove), €/m² + EPC (Idealista), payment plans (Bayut). Merge three portals into one table and you lose nothing.

How to pull the data without writing a scraper

Each portal above has a maintained Apify Actor that queries the portal's internal JSON endpoints — not rendered HTML — so output is structured on arrival:

{
  "price": 425000,
  "currency": "GBP",
  "address": "Queens Head Street, London, N1",
  "bedrooms": 1,
  "bathrooms": 1,
  "propertyType": "Apartment",
  "latitude": 51.5366,
  "longitude": -0.0958,
  "agentName": "Example Agency",
  "url": "https://www.rightmove.co.uk/properties/173742266",
  "scrapedAt": "2026-10-13T09:00:00Z"
}

Workflow: 1) set a search (city, ZIP, filtered URL) → 2) run → 3) export JSON/CSV/Excel → 4) schedule it (daily for price-drop monitoring, weekly for market snapshots).

You can also drive it from code — the same Actors are callable over the Apify REST API with a single POST /v2/acts/{id}/runs, so a cron job + webhook gives you a property data feed.

Actor links: Zillow · Redfin · Rightmove · Idealista · Apartments.com · ImmobilienScout24 · Homes.com · Funda · Trulia · Bayut

FAQ

Is there a real estate API alternative to Zillow?
Zillow's official API is closed to new sign-ups and Rightmove never had a public one. Scraper-as-API is the practical alternative: pay per result, no data contract, same fields.

Is scraping property portals legal?
Listings are public pages. The risks live elsewhere: bypassing access controls, republishing photos/text, and outreach under GDPR/CCPA/CAN-SPAM. Respect each portal's ToS and robots.txt — this isn't legal advice.

What's the best data source for property prices?
Idealista for €/m² (ES/IT/PT), Rightmove sold prices (UK), Funda (NL), ImmobilienScout24 (DE), Zillow/Redfin (US). Schedule recurring runs — trends need a time dimension.

Can I monitor new listings and price drops automatically?
Yes — schedule the Actor (daily/weekly) or call it from the API and attach a webhook. Results land in your DB, Sheet or CRM.

Full breakdown with per-portal analysis: The 10 Biggest Property Portals in the World (Medium). I build the scrapers linked above — 230+ actors on Apify.

I gave my YC jobs scraper write access to Notion without ever holding a Notion token

Artem Lazarev — Sat, 03 Oct 2026 09:06:38 +0000

TL;DR

Apify MCP connectors use the Model Context Protocol (MCP) to let an Actor write into a user's connected apps without ever touching that app's credentials. I used one to push my scraped Y Combinator jobs straight into Notion and delete my CSV export step. Below: why a token field was never an option for a published Actor, the documented Python snippet that couldn't run on any SDK version, and five steps covering connector setup, tool discovery, field mapping, safe writes, and what the whole thing costs per run.

The workflow I wanted

My Y Combinator jobs scraper has run happily for months. It walks the YC directory, pulls every company that's hiring, and returns salaries, equity ranges, founder profiles, and full job descriptions. The last full sweep did 1,429 companies and 3,305 jobs in about an hour and fifty minutes.

Then the data sat there. An 18 MB JSON file in a dataset. What I actually wanted was boring and specific: a running list of who's hiring, at what salary, in which batch, that I could sort and annotate and come back to a week later. So every time, I exported a CSV, opened it in a spreadsheet, and filtered by hand. The scraper was finished. My workflow wasn't.

Apify MCP connectors closed that gap. They let an Actor call an external service's allowed tools through an Apify-managed proxy, using the user's own authorized account.

The Actor is published on Apify Store, charges per result, and has 120 users I've never met.

After I added connector support, it writes scraped jobs into a Notion database on its own. All it took was a connector field in the input schema, a declaration of the Notion tools it needs, and the code that turns scraped jobs into Notion pages.

The full implementation is in the repo: github.com/artem-lazarev/apify-ycombinator-jobs-scraper.

For users of Apify Actors. To use an MCP connector, the Actor's developer has to add support for it first. Once that's in place, open Settings > API & Integrations in your own Apify account and authorize the connector you need, such as Notion. Then open the Actor's input form, select that connector, and fill in the destination details. For my YC Jobs Scraper, that's the Notion data source ID. Run it, and the scraped jobs land in your Notion database.

Why I couldn't just ask for a Notion token

The obvious option was to add a Notion API token field to the input schema and call the Notion API directly. A Notion integration token is a single secret that grants access to everything you've shared with that integration: pages, databases, all of it.

Asking those users to paste a Notion token into my Actor means asking them to trust that my code won't log it or send it elsewhere. I wouldn't paste my own token into someone else's Actor, so I don't think it's reasonable to ask.

MCP connectors invert that. The user authorizes Notion once, in their Apify account. My Actor receives a connector ID, which is just an opaque string. At runtime it authenticates to the Apify MCP proxy with the run's own Apify token, and the proxy adds the Notion credential server-side before forwarding anything upstream. That credential never enters my container. There's nothing for me to leak and nothing anyone has to trust me about.

Users still need to check which tools an Actor declares, because it can read and write through them. Apify treats the Actor runtime as untrusted precisely because it runs developer-supplied code. I'm the untrusted party here, and I'd rather be.

What you need to follow along

By the end of this walkthrough your Actor will finish a scrape and push the results into Notion on its own. No CSV export, no import step, no Notion token anywhere in your code.

Before any of that works, one thing has to be true: your Notion database has to exist already, with the columns you actually want. This Actor creates pages. It doesn't create databases and it doesn't create columns. Step 3 covers what happens when you gloss over that, which in my case was a batch of jobs landing in Notion with nothing in them but titles.

So build the destination first. Start with a title column and add the fields you plan to fill.

The rest of the list is short:

An Apify account.
A Python Actor you can build and run in Apify Console.
A Notion workspace you can authorize. Apify provides the managed OAuth client for Notion, so you don't need to register your own OAuth app.
apify-cli 1.8.0 or newer, if you deploy from the command line.

The connection code targets Python 3.10 or newer, the Apify SDK, and mcp 2.x. One packaging detail worth knowing up front: mcp 2.x depends on httpx2, which is a separate package from httpx. Install and import that one.

The changes land in three places: inputs go in .actor/input_schema.json, the Notion helpers live in src/notion_sync.py, and the sync gets called after results are saved in src/main.py. The repository has the complete scraper; the excerpts below are the integration only.

1. Declare the Apify MCP connector and its allowed tools

When I opened my authorized Notion connector, it offered 28 tools: notion-search, notion-update-page, notion-create-database, and 25 more. My Actor can call two of them. The proxy rejects everything else before it ever reaches Notion.

Setting up that allowlist is what this step is for, and it lives in .actor/input_schema.json. Here's the notionConnector field inside the existing schema's properties object:

{
  "properties": {
    "notionConnector": {
      "title": "Notion connector",
      "type": "string",
      "description": "Optionally write jobs to Notion through an authorized connector.",
      "resourceType": "mcpConnector",
      "nullable": true,
      "sectionCaption": "Notion output",
      "mcpServers": [
        {
          "url": "https://mcp.notion.com/mcp",
          "tools": {
            "required": [
              "notion-create-pages",
              "notion-fetch"
            ]
          }
        }
      ]
    },
    "notionDataSourceId": {
      "title": "Notion data source ID",
      "type": "string",
      "description": "Destination data source UUID or collection:// reference.",
      "editor": "textfield",
      "nullable": true
    }
  }
}

resourceType: "mcpConnector" turns the input into a connector picker in Console. After building the Actor, authorize Notion under Settings > API & Integrations, then select it in the Notion output section.

The second input, notionDataSourceId, chooses where the pages go. Keep both optional so dataset-only runs still work. Step 3 shows how to fetch the destination reference once the connection is up.

The connector picker rendered in the Actor input form, with the Notion connector selected.

The tools.required list is the part I'd skim past in someone else's article, so let me be specific about it. It isn't documentation and it isn't a hint. The proxy enforces it: it filters tools/list down to what you declared, and rejects tools/call for anything outside it.

Here's the first line my Actor logs on every run, straight from a real run log:

🔧 Tools allowed through the connector: ['notion-create-pages', 'notion-fetch']

So a jobs scraper can't rummage through your Notion. Not because I promise it won't, but because the proxy won't pass the call. You can check that yourself without reading a line of my code, which is about the only kind of security promise worth anything in a public Actor store.

One thing the allowlist doesn't do: it limits which operations the Actor can perform, not which content those operations can reach. notion-fetch isn't restricted to one database. What it can read still depends on the Notion account and permissions sitting behind the connector.

MCP connectors under Settings > API & Integrations, showing the authorized Notion connector.

The connector's Edit dialog, showing all 28 Notion tools it could expose.

2. Connect and read the tool descriptions before you write anything

The documented snippet couldn't run on either SDK version

The first thing I did was copy the Python snippet out of Apify's connector docs. It deployed, and then:

ValueError: not enough values to unpack (expected 3, got 2)

The snippet unpacked three values from the transport:

async with streamable_http_client(...) as (read, write, _):

In the mcp 2.x SDK, streamable_http_client yields exactly two. I checked the type rather than guessing:

>>> from mcp.client.streamable_http import TransportStreams
>>> TransportStreams
tuple[ReadStream[SessionMessage | Exception], WriteStream[SessionMessage]]

Three values is the 1.x shape, where the third element was a session-ID callback. But 1.x has no function called streamable_http_client at all. It's streamablehttp_client, with no underscore between streamable and http, and it takes a headers argument rather than an injected HTTP client. I downloaded the 1.9.0 source to confirm:

# mcp 1.9.0 - mcp/client/streamable_http.py
async def streamablehttp_client(
    url: str,
    headers: dict[str, Any] | None = None,
    timeout: timedelta = timedelta(seconds=30),
    sse_read_timeout: timedelta = timedelta(seconds=60 * 5),
    terminate_on_close: bool = True,
)

So the documented snippet was a hybrid: the 2.x import name with the 1.x unpacking. It couldn't run on either version. On 2.x the import resolves and the unpacking fails. On 1.x the import fails before you get that far.

Apify has since fixed that page. It now unpacks two values, and it carries a version callout spelling out the same distinction: on mcp 1.x the transport function is named streamablehttp_client, takes headers instead of http_client, and yields a third value. If you're reading this with a working snippet in front of you, that's why.

Save the tool descriptions before you build a payload

The notion-create-pages description is 5,857 characters long. Console truncated it in my logs, so for the first few attempts I was coding against maybe a third of the contract.

The fix is one extra block, and it's the trick I'd start with on any MCP service: save the tool descriptions and input schemas into the run's key-value store, then read them properly.

Which means your first connection should do nothing but discover tools. It confirms authorization and hands you the service's real contract before you build a single payload. Each tool description explains what the tool does, and its input schema defines the arguments it accepts.

Open a session from Python

The Actor needs three values: the selected connector ID, the MCP proxy's base URL, and the Apify run token. Apify supplies the last two in every platform run.

Install these in the Actor build:

apify>=2.0.0
mcp>=2.0.0
httpx2>=2.5.0,<3

import asyncio
import os

import httpx2
from apify import Actor
from mcp import ClientSession
from mcp.client.streamable_http import streamable_http_client


async def main() -> None:
    async with Actor:
        actor_input = await Actor.get_input() or {}
        connector_id = actor_input.get('notionConnector')
        proxy_url = os.environ.get('ACTOR_MCP_CONNECTOR_BASE_URL')
        token = os.environ.get('APIFY_TOKEN')
        if not connector_id or not proxy_url or not token:
            Actor.log.warning(
                '⚠️ No connector or no proxy credentials - skipping Notion sync. '
                'MCP connectors only resolve in a platform run, not a local one.'
            )
            return
        try:
            async with httpx2.AsyncClient(
                headers={'Authorization': f'Bearer {token}'},
                timeout=httpx2.Timeout(60.0),
                follow_redirects=True,
            ) as http_client:
                async with streamable_http_client(
                    f"{proxy_url.rstrip('/')}/{connector_id}",
                    http_client=http_client,
                ) as (read, write):
                    async with ClientSession(read, write) as session:
                        await session.initialize()
                        tools = (await session.list_tools()).tools
                        Actor.log.info(
                            f'🔧 Tools allowed through the connector: '
                            f'{[tool.name for tool in tools]}'
                        )
        except Exception:
            Actor.log.exception('Notion connector check failed')
            raise


if __name__ == '__main__':
    asyncio.run(main())

Then, immediately after the tools assignment and inside the same initialized session, add the block that saves everything:

payload = [
    {
        'name': tool.name,
        'description': tool.description,
        'inputSchema': tool.input_schema,
    }
    for tool in tools
]
await Actor.set_value('NOTION_TOOL_SCHEMA', payload)

Open the run's default key-value store and read NOTION_TOOL_SCHEMA. One naming detail that tripped me up: the SDK attribute is input_schema, while inputSchema above is just the JSON key I picked for the saved file. Inspect the allowed arguments for notion-fetch and notion-create-pages before going further, because the parent ID and the property formats you need in step 3 are both buried in there.

Push to a dev build tag, not latest. I have paying users on latest, and I wasn't going to test a new network call in front of them. There's a second reason to work this way: ACTOR_MCP_CONNECTOR_BASE_URL only exists in a platform run, so apify run on your laptop can't reach a connector at all. My whole loop was push to a dev tag, run on the platform, read the log.

3. Prepare the Notion destination and map its fields

This is the step that cost me the most time, and the worst failure in it was the silent one.

A batch of pages went into Notion with nothing but titles. No company, no salary, no equity. The log cheerfully reported that it was sending all columns. It wasn't lying, exactly: my schema parser had returned an empty dict, and the row builder read {} as "this database has a title column and nothing else", so it filtered every other field out on the way through.

Two things have to be right before you write anything: the destination reference, and the column map.

Get the data source reference

My first mistake here was simpler. I sent the database ID, because that's what the Notion URL hands you, and the tool rejected it.

A Notion database contains one or more data sources, and the write needs the ID of the source that should receive the jobs, not the database itself. A database URL on its own isn't the destination value either. Fetching the database is what makes the distinction obvious.

Set database_url to the URL copied from your Notion database. Add this after session.initialize() in the connection example, at the same indentation as the other statements inside that session:

result = await session.call_tool(
    'notion-fetch', arguments={'id': database_url}
)
if result.is_error:
    raise RuntimeError('Notion could not fetch the database')
await Actor.set_value(
    'NOTION_DATABASE',
    result.model_dump(mode='json', by_alias=True),
)

Open NOTION_DATABASE in the run's key-value store and find the collection:// reference for the data source you want. Put that reference, or its bare UUID, into notionDataSourceId. The sync strips collection:// before sending parent: {"data_source_id": source_id}.

Read the schema before building rows

Back to the empty dict. But there's a reason I was reading the schema at all, and it wasn't the bug.

I had it working against my own database and I nearly shipped that. But it needed to work for my users' databases too. My database has a column called Salary Min. Theirs might call it Salary (min), or have no salary column at all. The title column is usually Name, but Notion lets you rename it to anything. Since one unknown column kills an entire batch, an Actor that assumes my layout works for exactly one person.

So the Actor reads the target before writing to it. That's what the second declared tool, notion-fetch, is for.

In src/notion_sync.py, _fetch_data_source_schema calls notion-fetch for source_id and passes the response to parse_data_source_schema. Those helpers, and logger, are defined in that file. The sync uses their result like this:

schema = await _fetch_data_source_schema(session, source_id) or None
if schema:
    logger.info(f'🗂️ Target columns: {sorted(schema)}')
else:
    logger.warning('⚠️ Could not read the data source schema - sending all columns')

That or None is the fix for the silent failure. An empty dict is falsy, but it isn't None, and the row builder treated {} as a real schema with no optional columns. Collapsing it to None forces a failed or empty lookup down the fallback path: send the expected fields and let Notion report the mismatch out loud. You can still end up with a rejected batch, but you'll know about it instead of finding fifty half-empty pages later. And either way the complete scraped data is already sitting in the Apify dataset.

The parsing itself has a wrinkle. notion-fetch doesn't return JSON. It returns a document wrapped in a JSON envelope: some prose, a <data-source-state> block, and a SQLite CREATE TABLE rendering of the same schema. The parser has to unwrap the envelope before it can read the schema at all. This excerpt handles the outer layer; the repository's parse_data_source_schema handles the remaining formats:

import json

try:
    envelope = json.loads(payload)
    if isinstance(envelope, dict) and isinstance(envelope.get('text'), str):
        payload = envelope['text']
except json.JSONDecodeError:
    pass

Then the row builder sends the job title to the column with type title and filters the other fields against the discovered names. This excerpt uses _title_column and the candidates dictionary from build_job_rows in the same file:

title_key = _title_column(schema) if schema else 'Name'
...
for column, value in candidates.items():
    if value is None:
        continue
    if schema is not None and column not in schema:
        continue
    properties[column] = value

The nice side effect: point the Actor at a brand-new Notion database with only a title column and it still works. It writes titles. Add a Salary Min column and the next run fills it in, with no code change.

Use the MCP tool's property formats

One more trap, and this one produced the least helpful error message of the entire build:

{"code":"validation_error","status":400,
 "message":"Properties {propertyKeys} not found in the data source.\n
            Date property {propertyKeys} not found in the data source.\n
            All editable property keys: {editableProperties}."}

Those are literal {propertyKeys} placeholders. The template never got interpolated, so the error tells you that some property is wrong while withholding which one and what the valid names are. Notion knows both. And one unknown column rejected all 20 pages in the batch, because the offending page doesn't fail on its own.

Reading the saved tool description got me there faster than guessing from that message ever would have. It also handed me the encoding rules I'd otherwise have found by trial and error. Properties take scalar values rather than the Notion REST API's nested property objects. Send numbers as numbers. Checkboxes are the literal strings __YES__ and __NO__. And date properties split across prefixed keys, so a column called "Scraped At" is written like this, where properties is the row's property dictionary and scraped_at is its date value:

properties['date:Scraped At:start'] = scraped_at

Not properties['Scraped At']. That one would have taken me an hour on my own.

Match column names exactly and give them compatible types. A Number column named Salary Min will take the scraper's salary value; Salary (min) gets skipped. The Actor discovers the title column, but it won't translate other names for you or validate every column type.

4. Save the scrape, then write pages in batches

My first connection attempt broke, and the run still finished with exit code 0 and a complete dataset.

That wasn't luck, it's the entire reason for the ordering in this step. Save the scrape to the dataset first, then sync, and keep the sync inside its own try/except. A connector failure should never throw away work the scraper has already finished, and on a per-result Actor that's work the user has already paid for.

Here's the call site in src/main.py, after the results are saved:

# Dataset first - this must succeed before anything touches Notion
await Actor.push_data(jobs)
notion_connector = actor_input.get('notionConnector')
notion_data_source_id = actor_input.get('notionDataSourceId')
if notion_connector and notion_data_source_id:
    try:
        created = await sync_jobs_to_notion(
            jobs,
            connector_id=notion_connector,
            data_source_id=notion_data_source_id,
        )
        Actor.log.info(f'✅ Wrote {created} jobs to Notion')
    except Exception:
        Actor.log.exception('Notion sync failed - dataset is unaffected')
else:
    Actor.log.info('No Notion connector selected - dataset output only')

The repository has the surrounding Actor lifecycle and job collection. The two guards that matter are both here: only call the sync when notionConnector and notionDataSourceId are both supplied, and never let it raise past the try.

Check tool responses as well as exceptions

A successful network request can still carry a failed tool result. In the Python SDK the flag is is_error, snake case, not the isError that most existing MCP writing shows, because most of that writing is TypeScript. Exceptions cover transport and protocol failures; is_error covers everything else, and you have to check it after every call. Miss it and every write looks like it worked while nothing appears in Notion.

Define this helper in src/notion_sync.py before the batch-writing function:

from typing import Any, Optional


def _tool_error_text(result: Any) -> Optional[str]:
    if not getattr(result, 'is_error', False):
        return None
    for block in getattr(result, 'content', None) or []:
        text = getattr(block, 'text', None)
        if text:
            return text
    return 'unknown error'

Once the session is initialized and the rows are built, call notion-create-pages with the data source ID as parent. This excerpt from sync_jobs_to_notion uses _chunk and PAGE_BATCH_SIZE (20) from the same file; CREATE_PAGES_TOOL is notion-create-pages, and created starts at zero:

for batch_no, batch in enumerate(_chunk(rows, PAGE_BATCH_SIZE), 1):
    response = await session.call_tool(
        CREATE_PAGES_TOOL,
        arguments={
            'parent': {'data_source_id': source_id},
            'pages': batch,
        },
    )
    error_text = _tool_error_text(response)
    if error_text:
        logger.error(f'Notion batch {batch_no} failed: {error_text}')
        continue
    created += len(batch)

Log the failed batches and count only the successful ones. And since the run can exit 0 with a broken sync, check the sync logs, not just the overall run status.

The Notion database after a run, populated with YC jobs.

Start with a small run and check a few pages against the dataset: job title, company, salary, destination. If your database only has a title column you'll get title-only pages, which is the same symptom as the empty-schema bug in this step with a far more boring cause. Add the optional columns before you expect to see values in them.

5. Verify the result and measure the overhead

Two seconds and about 5% more compute. That's what the Notion write cost me.

I ran the Actor over 20 companies at 512 MB, with and without the connector:

	Runtime	Compute units	Jobs	Notion pages
Without connector	38.7 s	0.00537	50	0
With connector	40.7 s	0.00566	50	50

The Apify run log showing tool discovery, the target columns, and the batch writes.

Because this Actor charges per result, that extra compute comes out of my margin. It doesn't raise the per-result price my users pay. At 5% I'll take that trade happily, but it's worth knowing which side of the ledger the overhead lands on before you ship a connector on a per-result Actor.

The 50 pages went out in three notion-create-pages calls, batched 20, 20, and 10. Those were the writes only; tool discovery and schema fetching were additional requests. One call per job would have meant 50 round trips and a much less pleasant table. It's a small comparison, so treat it as the overhead I measured rather than a guarantee for larger runs or other services.

The scrape output had the same shape in both runs: 20 companies, 50 jobs, 28 founders. The connector is optional. If nobody selects one, the Actor makes no MCP calls at all, and my existing users see exactly what they saw last month.

Before you schedule it, deal with duplicates

One thing I haven't solved yet. Every run appends new pages, so a daily schedule will re-add jobs that are already in the database.

There's no deduplication in the current implementation. If you want a persistent job tracker rather than a snapshot, comparing job URLs before writing is the next change to make, and I'd make it before setting up any schedule.

What I'd do differently

I'd read from Notion, not just write to it. Right now the filters live in the Actor input. It'd be better to keep a Notion database of the batches, industries, and locations I care about, have the Actor read its own task list at the start of a run, and write results back. A loop instead of a one-way pipe. The connector already permits it; I just haven't built it.

I'd also like to try two connectors on one Actor: Notion for the archive, Slack for a message when a job matches a saved search. The input schema accepts an array of connector IDs, with one MCP session per connector.

Reuse the Apify MCP connector pattern in another Actor

For a different destination, keep the same order: declare the tools, connect and save their descriptions, inspect the destination, map your results, and write only once the dataset is safe.

Another service will have its own connector, its own tool names, and its own payload format. Which is exactly why the discovery step in step 2 is worth doing first. It hands you those requirements up front instead of making you reverse-engineer them from error messages with {propertyKeys} in them.

FAQ

Can I add a connector to any existing Actor?

The Actor has to declare the connector input and contain code that calls it. For this scraper, select Notion and supply notionDataSourceId. Adding a connector in your account settings alone doesn't make an unrelated Actor write to Notion.

Can the Actor see my Notion token?

No. It authenticates to the Apify MCP proxy with its own run token, and the proxy adds the Notion credential server-side. The connector session ends with the run.

Can I restrict what the Actor can do?

Yes. Access has to satisfy the service's authorization permissions, any connector-level tool allowlist, and the Actor's mcpServers[].tools declaration. A tool allowlist limits operations, while the upstream service controls which content those operations can reach.

Can I develop this locally?

Not really. ACTOR_MCP_CONNECTOR_BASE_URL only exists in a platform run. I test row building and response parsing locally, then use a dev build to verify authorization and real tool calls.

Which SDK version do I need?

mcp 2.x for Python, which pulls httpx2 rather than httpx. The 1.x API differs in both the transport function's name and its signature.

Is this the Apify MCP server?

No. Here the Actor calls Notion through an outbound MCP connector. The Apify MCP server is the other direction: it lets external AI clients call Actors.

Code: the connector implementation is in src/notion_sync.py.

The Actor: YC Jobs Scraper on Apify Store.

Further reading

How I Built a Linktree, Beacons & Bio Email Scraper with Apify

Ahmed Jasarevic — Sun, 20 Sep 2026 20:42:11 +0000

1. Introduction

Collecting emails from Linktree, Beacons, and other bio links is often a tedious and time-consuming task. As a developer and freelancer working with lead generation projects, I faced this problem firsthand: manually navigating hundreds of profiles just to extract a few emails is inefficient and error-prone.
To solve this, I built a Linktree, Beacons & Bio Email Scraper Actor using Apify. It automates the process, reliably collects emails from multiple platforms, and provides structured outputs for marketing, data engineering, and business intelligence purposes.
This article details the journey: the challenges I faced, the design of the Actor, lessons learned, and tips for anyone looking to build production-grade scraping solutions.

2. Problem Context & Motivation

Manual email collection from bio links presents several challenges:

Dynamic pages: Platforms like Linktree often use JavaScript to load content asynchronously. Simple HTML scraping fails here.
Anti-scraping measures: Repeated requests can trigger rate limits or captchas.
Scale: For real lead-generation projects, you often need hundreds or thousands of emails daily—manual collection is not an option. I needed a solution that was fast, scalable, and reliable, and that could handle these technical challenges without breaking.

3. Actor Overview

Architecture and Workflow
The Actor is built on Apify and Crawlee, combining headless browser automation with structured data extraction. Here’s a high-level workflow:

Input: A list of Linktree, Beacons, or other bio URLs.
Navigation: Actor launches a headless browser to visit each URL.
Email Extraction: Uses DOM selectors and regex patterns to identify valid email addresses.
Anti-blocking: Rotates proxies and applies request throttling to avoid detection.
Output: Saves results in JSON or CSV with details: URL, email, timestamp.

Input, Output, and Configuration Options

Input: List of profile URLs (CSV, JSON, or manually typed).
Output: JSON, CSV, or Google Sheets integration.
Configurable Options: Maximum pages per run Timeout per request Proxy rotation (on/off) Output file format This flexibility allows the Actor to handle both small projects and large-scale campaigns.

4. Implementation Details

Handling Dynamic Content
Linktree and Beacons often load content via JavaScript. Initially, I tried simple HTTP requests, but emails were missing in the HTML response. Switching to Crawlee with Playwright solved the issue:

import { PlaywrightCrawler } from 'crawlee';

await crawler.run(['https://linktr.ee/example']);

This approach ensures the Actor captures all visible emails, even on dynamic pages.
Anti-Blocking Strategies
During early tests, some accounts triggered rate limits. To fix this, I added:
Proxy rotation
Randomized delays between requests
Error retries for failed pages
These measures increased the success rate to 99.6% over thousands of URLs.

5. Metrics & Results

To evaluate the Actor in a real-world scenario, I ran it against a dataset of 1,800+ URLs. The results showed strong reliability while keeping the cost of each run predictable:

99.6% success rate
1,816 emails collected
$14.47 total cost per run
$23.76 profit generated per run from the collected leads

These results demonstrated that the Actor could process large batches of URLs reliably while remaining cost-effective for lead-generation workflows.

This shows the Actor is not only technically reliable but also economically valuable for lead-generation workflows.

6. Lessons Learned

Building this Actor taught me several important lessons:
JavaScript rendering matters: Always test pages in a headless browser when dealing with dynamic content.
Anti-blocking is critical: Even simple rotation and throttling drastically improve success rates.
First-person debugging insights: Logging actual page content during development helped identify hidden issues.
Scalable design: Structuring input/output for batch processing makes the Actor production-ready.

7. Who Can Benefit

This Actor is useful for:

Developers looking to automate repetitive data collection tasks
Marketers and sales teams needing up-to-date email lists
Data engineers building pipelines that integrate multiple data sources
The architecture can also be adapted for other bio link platforms or email collection projects with similar challenges.

8. Conclusion

Automating email extraction from Linktree, Beacons, and bio links is no longer a manual nightmare. By using Apify and Crawlee, I built a reliable, scalable, and cost-effective Actor that delivers real-world results.
If you plan to build your own Actor, remember: focus on handling dynamic content, preventing blocks, and structuring your data pipeline. Sharing these lessons ensures other developers can build production-grade automation with confidence.

9. Next Steps

Test the Actor on new bio platforms
Add AI-powered validation to filter incorrect emails
Explore integrations with CRMs and email marketing tools

A default MCP connection hands an agent 11 tools. None of my 23 Actors is one of them.

Ai-Q Labs — Fri, 18 Sep 2026 00:10:09 +0000

A default MCP connection hands an agent 11 tools. None of my 23 Actors is one of them.

I have 23 audit Actors on the Apify Store. They all do a version of the same thing: take a public record, check whether what it still claims is true, and write the verdict into a dataset.

I have also written six articles about what happens when an AI agent calls one of them through the Apify MCP server. Output fields that arrive empty. Input schemas that read differently to a form and to a tool. A default that quietly fills a required field. A connector that shows 44 tools and grants four.

MCP is the Model Context Protocol, the interface that turns an Actor into a tool that clients like Claude and Cursor can call on their own. Every one of those six articles assumed the agent had that tool in hand. Last week I checked the assumption for the first time. It does not hold by default.

The connection string I never read

I found the problem in my own machine before I found it anywhere else. This is the Apify entry in my Claude config, verbatim except that there is no token in it:

"url": "https://mcp.apify.com?tools=aiqlabs/github-repository-audit"

I wrote that line months ago to test one Actor in isolation, and then left it. My editor session had five Apify tools in it. One of them was an Actor, and it was the one named in that string.

So I had spent six articles reasoning about how agents see my catalogue. The session I was reasoning from had been narrowed to a single Actor, by me, on purpose, and then forgotten. The first thing worth measuring was what a connection with nothing in it returns.

What a default connection actually sends

Twenty lines, no dependencies. This is the whole test:

// reach.mjs — what does a default connection to the Apify MCP server actually send?
// Run: APIFY_TOKEN=... node reach.mjs
const URL = `https://mcp.apify.com/?token=${process.env.APIFY_TOKEN}`;
const HEAD = { 'content-type': 'application/json', accept: 'application/json, text/event-stream' };

async function rpc(body, sid) {
    const res = await fetch(URL, { method: 'POST', body: JSON.stringify(body),
        headers: sid ? { ...HEAD, 'mcp-session-id': sid } : HEAD });
    const text = await res.text();
    // the server answers as SSE, so the JSON sits on a "data:" line
    const line = text.split('\n').find((l) => l.startsWith('data:'));
    return { sid: res.headers.get('mcp-session-id') ?? sid, json: line ? JSON.parse(line.slice(5)) : null };
}

const { sid } = await rpc({ jsonrpc: '2.0', id: 1, method: 'initialize', params: {
    protocolVersion: '2024-11-05', capabilities: {}, clientInfo: { name: 'reach', version: '1' } } });
await rpc({ jsonrpc: '2.0', method: 'notifications/initialized' }, sid);

const { json } = await rpc({ jsonrpc: '2.0', id: 2, method: 'tools/list' }, sid);
const tools = json.result.tools;
console.log(`tools: ${tools.length}`);
for (const t of tools) console.log(' ', t.name);

Its output, on 12 August:

tools: 11
  search-actors
  fetch-actor-details
  call-actor
  get-actor-run
  get-dataset-items
  get-key-value-store-record
  abort-actor-run
  search-apify-docs
  fetch-apify-docs
  report-problem
  apify--rag-web-browser

Ten of those are the server's own tools. One is an Actor: RAG Web Browser, which belongs to Apify. Of the 23 Actors I have published, zero are in the list.

Nothing is being hidden. Add ?actors= to the same URL and name all 23 slugs. The same server then returns 27 tools: my 23 plus four storage tools. The Actors are reachable. They are not default.

The default is also defensible. The Store holds thousands of Actors. A server that turned every one of them into a tool definition would blow out the context window of every client that connected. Eleven tools is a budget, not a snub.

But it does mean that everything I had written was conditional on a step I had never measured.

Three ways an Actor reaches an agent

Route	Who decides	What an author can do about it
A — named in the connection URL (`?actors=` / `?tools=`)	whoever configures the agent	Almost nothing. They have to know your slug before they can type it.
B — included in the default set	Apify	Nothing. Today that set contains one Actor.
C — found at runtime by `search-actors`	Store search ranking	This one.

A and B are not work I can do. C is the entire surface, so I measured it.

Measuring the third route

The first two attempts failed on the parameter name. {"search": …} and {"query": …} are both accepted and both silently ignored. The server echoes Search query: back empty and returns its default list. Rather than guess a third time, I read the tool's own inputSchema out of the JSON-RPC reply to tools/list above:

keywords : string   default ""
limit    : integer  default 5, max 10
offset   : integer  default 0

The parameter is keywords. But the line that matters more is limit. It defaults to 5 and caps at 10. That is the window. An agent that runs one search and picks from it is choosing among five Actors out of the whole Store.

The second trap is in the response. Each result set is introduced by a line like **Number of Actors found:** 7, which reads as a total and is not one. It is the size of that page:

Instagram keeps producing fresh results at offset 0, 10, 20 and 30 — 39 distinct Actors, no repeats. One of those four pages came back with nine rather than ten, which is the second thing to know: the pages are ragged.
Chrome extensions returns 7 at offset 0 and a different 6 at offset 10. A short page does not mean the list ended.

My first pass treated the short page as the end of the list, so it never looked past page one. It reported seven of my Actors as absent when they were sitting at ranks 12, 16, 31, 41, 52, 55 and 57. Every number below comes from the corrected run. That run pages to rank 100 whatever the page count says, and computes rank as offset + position in page, because the pages come back ragged.

What it found

I paired each of 23 Actors with the query it was built for — 28 queries in all, since several Actors got both a specific phrase and the broader term a user might type instead. Then I asked how deep in the results that Actor sits.

Where my Actor lands	queries	share
inside the default 5 the agent receives	6	21%
inside the maximum 10	7	25%
rank 11–100: indexed, past the window	7	25%
not in the first 100	14	50%

On the queries these Actors were written for, an agent doing a default search finds one of them about one time in five. Half of them are not in the first hundred results at all.

That was worse than I expected, but it was not the finding. The finding is which half.

The queries I can win are the queries nobody is making

Every search-actors result carries the Actor's usage with it, in a line like **Stats:** 20 total users, 3 monthly users. So the demand on a query is readable from the same response that gives you the ranking: take the 30-day user count of whatever sits at rank 1.

my rank	queries	median 30-day users of that query's rank-1 Actor
reachable (top 10)	7	6
unreachable (past 10, or absent)	21	23

The unreachable side is where the users are. App Store — 311. Google Play — 295. article extractor — 271. Shopify — 186. My Actors for the first three of those are not in the first hundred results.

And the one query where I rank #1 is sitemap checker, where the rank-1 Actor has 0 users in the last thirty days. That Actor is mine. I am first on a shelf with no traffic, and invisible on every shelf with traffic.

I had measured the same shape once before, from the other side. On 2 August I sampled the Store itself and found that Actors naming a well-known platform had a median of 5 monthly users against 2 for the rest. I read that as an argument for naming platforms. This measurement says the platform-named shelves are exactly the ones where I cannot be seen — 1 reachable query out of 14, against 6 out of 14 for the generic ones. Both results are true and they are not in conflict: platform names are where demand collects, demand attracts competitors, and competitors fill a window five deep. But I predicted the wrong one.

The tool tells the agent to broaden the search, and broadening removes me

Every search-actors response ends with this, addressed to the agent:

IMPORTANT: You MUST always do a second search with broader, more generic keywords (e.g., just the platform name like "TikTok" instead of "TikTok posts") to make sure you haven't missed a better Actor.

That is good advice for the user. It is also an instruction to run the query I do worst on. Same Actor, specific phrase against the broadened phrase:

specific	rank	broadened	rank
`sitemap checker`	#1	`sitemap`	not in first 100
`PDF tables`	#2	`PDF`	not in first 100
`PDF text markdown`	#2	`PDF`	not in first 100
`Chrome extensions`	#16	`Chrome Web Store`	not in first 100
`Shopify apps`	#41	`Shopify`	not in first 100
`GitHub repository`	#52	`GitHub`	not in first 100
`App Store apps`	absent	`App Store`	absent
`Google Play apps`	absent	`Google Play`	absent

Six worse, none better, two unchanged. The clearest one is the first: sitemap checker puts me at #1, and deleting one word puts me past rank 100. Whatever advantage a precise name buys, the agent is under instructions to take a second look without it.

What decides the order

Not popularity, at least not alone. Across every first page I collected, 157 of 238 adjacent pairs are in descending order of monthly users — 66%. A pure popularity sort would be 100%. Relevance carries real weight, which is how an Actor with no users reaches #1 on sitemap checker.

Apify documents what does decide it, and says so for both surfaces at once. From Actor quality score:

Actors with higher quality scores tend to rank higher on both surfaces, though no specific position is guaranteed.

The second surface named on that page is "the Apify MCP server search-actors tool used by external AI agents". The score aggregates eight categories: reliability, popularity, feedback and community, ease of use, pricing transparency, trustworthiness, history of success, and congruency of texts.

I have moved it once and can report the size of the move. On 2 August I added dataset output schemas across the catalogue; the score on the Actor I watched went from 74 to 78, and after that pass the whole catalogue sat between 78 and 81. There is no API for the number — I tried four endpoints and got 404 from each — so the only place to read it is Console > Insights > Actor quality.

Here is that panel today, for the Actor this article keeps returning to:

79 out of 100. Better than 71% of Actors on the platform, better than 99% on reliability, and one suggestion left in the panel.

That is the pairing worth sitting with. Console rates this Actor above 71% of the platform and has one cosmetic item left to suggest. The same Actor is absent from the first hundred results on every query with real demand behind it, and its one first place is on a query with none. Both readings are correct, and the documentation is careful enough to allow it: a higher score tends to rank higher, with no position guaranteed. On a crowded shelf, tending is not the same as landing in the five results the agent receives.

Which leaves the honest part. Three of the eight categories — popularity, feedback and community, history of success — are things you get from having users. For an Actor with none, part of the ranking that would bring users is held shut by not having them yet. That is not a complaint about the design; a store that ranked unproven tools first would be worse for everyone using it. But it is the actual shape of the problem, and no amount of description polishing changes it.

What I would tell an Actor author

Check what your connection sends before you tune anything. The 20 lines above answer it. If your Actor is not in the list, none of your tool-definition work is reaching an agent yet.
Measure your rank on the query you were built for, with limit: 10, paging past the first short page. Anything past rank 10 is in the index and outside the window.
Read the demand off the same response. The rank-1 Actor's monthly users are printed next to it. If that number is 0, being #1 buys nothing — I have the receipt.
Expect the broad term to be worse. It is also where the demand is. Decide which of those two facts you are building for, rather than discovering the trade after you publish.

What I got wrong

Three things, in the order I found them.

I narrowed my own MCP connection to one Actor and then wrote six articles about how agents see my catalogue. The narrowing was in a config file I had not opened in months.

I read a page count as a total, and my first pass reported seven Actors as unreachable that rank between 12 and 57. Had I not checked whether offset kept returning results, this article would have carried seven false claims of the most flattering kind — the kind where the platform looks worse than it is.

And I expected the platform-named Actors to be the findable ones. They are the least findable ones I own.

None of that makes the earlier six articles wrong. It reorders them. An input schema that misleads an agent, an output schema that arrives empty, a default that fills a required field — all of those are real, and all of them start to matter at the moment the agent receives the tool. For 23 out of 23 of mine, that moment does not happen by default.

I pinned my Actors to the MCP URL, the way the docs recommend. The agent lost every way to see what a call costs.

Ai-Q Labs — Fri, 18 Sep 2026 00:09:29 +0000

I pinned my Actors to the MCP URL, the way the docs recommend. The agent lost every way to see what a call costs.

I publish 23 Actors on the Apify Store. All 23 are pay-per-event.

Last week I measured how an AI agent actually reaches them. The answer was uncomfortable. A
default MCP connection hands an agent 11 tools. None of mine is one of them.

So you pin. You name your Actors in the connection URL. The
Apify MCP documentation tells you to:

For production deployments, explicitly specify which tools to load rather than relying on
defaults. This ensures consistent behavior across updates.

That is good advice for reliability. I followed it. Then I measured what the agent receives
afterwards, and found something the docs do not mention.

Pinning removes the agent's ability to see what a call costs. It also removes its ability to
cap what a call spends.

Below are the measurements. Then the three tools that disappear. Then the half of this that is
my own fault rather than the platform's.

What pinning actually changes

I connected three ways and listed the tools each time. Same account, same day, 2026-08-12.
Every listing came from a tools/list call over the
MCP HTTP transport. That is what the client
library hands the model.

Default connection, no parameters. Eleven tools:

search-actors          get-key-value-store-record   report-problem
fetch-actor-details    abort-actor-run              apify--rag-web-browser
call-actor             search-apify-docs
get-actor-run          fetch-apify-docs
get-dataset-items

Pinned with two of my Actors, using the documented parameter
?tools=aiqlabs/seo-audit-tool,aiqlabs/pdf-inspector. Six tools:

aiqlabs--seo-audit-tool    get-actor-run                 abort-actor-run
aiqlabs--pdf-inspector     get-dataset-items
                           get-key-value-store-record

Pinned with all 23. Twenty-seven tools: my 23, plus the same four.

The four survivors are all post-run tools. Each one reads a run, reads a dataset, reads a
key-value record, or aborts something already running.

Every tool that helps an agent decide before it calls is gone. Three of the missing ones
matter for money.

The three tools that carried the price

I searched the full JSON of each default tool definition for pricing language. These are the
strings that came back.

search-actors returns pricing in its results. Its own description says so:

- **Pricing:** Details with pricing link

Its output schema enumerates the models. FREE, PRICE_PER_DATASET_ITEM,
FLAT_PRICE_PER_MONTH, and a pricePerUnit field.

fetch-actor-details takes a flag whose description reads, in full:

Include pricing model and costs.

call-actor is the one that actually protects the caller. It accepts maxTotalChargeUsd:

Pay-per-event Actors only — ignored otherwise.
Caps total USD billed; does NOT limit work. Prefer the Actor's own input fields to bound work.

That is a spending ceiling. The caller sets it, at call time. Note the second sentence. I will
come back to it, because it is the sentence that indicts me.

All three tools are in the default set. None of them survives pinning.

An agent holding my Actors directly cannot look up what they cost. It cannot put a ceiling on
what it spends. It can only call them.

What my own definitions tell the agent about price

Nothing. I checked, and I checked wrong the first time.

My first pass searched each tool definition for a dollar sign. It reported a hit on 23 of 23.
For about a minute I believed my descriptions already carried prices.

They do not. The dollar signs were $id, the JSON Schema keyword:

"required":[],"$id":"https://apify.com/mcp/aiqlabs--app-store-audit"

I searched again for a dollar sign followed by a digit. An actual amount. The count was
0 of 23.

The words price and cost do appear in a few of my definitions. Every one of them is about
the subject matter, not the invoice.

One Actor notes that prices and availability differ by storefront. Another explains that each
archived URL costs a request to the live site.

Those are useful sentences. Neither tells an agent what calling the tool will cost.

Meanwhile every one of the 23 is metered. Each has two charge events, configured the way
Apify's monetization docs
describe:

Event	Price
`apify-actor-start`	$0.00005
`apify-default-dataset-item`	$0.002 – $0.01 depending on the Actor

The per-item price is the one that moves.

I wanted to be sure those events fire on their own. My Actors never call a charge function. So
I read a real run record rather than assuming:

run thLehE7kk3NVRAvqm  SUCCEEDED
chargedEventCounts: {"apify-actor-start":4,"apify-default-dataset-item":608}

They fire. Another run in the same batch recorded 1,000 dataset items. At my highest per-item
price, one call like that is ten dollars.

The part that is my fault

Go back to that sentence in call-actor:

Prefer the Actor's own input fields to bound work.

Apify is right about this. A billing cap stops the invoice, not the work. The real bound
belongs in the
input schema,
where the author controls it.

So I checked whether my own Actors carry that bound.

All 23 have a field for it. maxUrls, maxPages, maxApps, maxPdfs, maxVehicles,
maxOrganizations, maxStories. Twenty-three out of twenty-three.

Not one of them declares a JSON Schema maximum.

pdf-inspector        maxPages(none), maxPdfs(none), maxFileMb(none), maxLinksToCheck(none)
sitemap-checker      maxUrlsToCheck(none), maxSitemaps(none), maxDepth(none)
http-status-checker  maxUrls(none), maxRedirects(none), maxConcurrency(none)

I named the fields as if they were limits. I never made them limits.

An agent can pass maxUrls: 1000000. The schema will accept it. I told the schema that any
integer is fine.

So the failure has two halves. Only one of them is the platform's.

Pinning removes the caller-side ceiling. That is the platform's shape. My Actors have no
author-side ceiling. That is mine, in 23 files I wrote myself.

An agent holding my pinned tools has no cap available from either direction.

What I got wrong on the way here

I started this measurement expecting a different problem.

My hypothesis was time. Actor runs are slow. Agents time out waiting. Authors ship tools an
agent cannot practically await.

I pulled the run history for all 23 and computed durations from the API.

	Value
Median of per-Actor medians	4.05 s
Slowest single run observed	58.0 s
Actors with a median above 60 s	0 of 23

There is no timeout problem. My hypothesis was wrong.

I dropped that half of the article rather than stretching the numbers to fit it. Only five of
the 23 had enough run history to produce a median, and that limit is worth stating. But nothing
in the data pointed toward latency. I would rather report a dead hypothesis than a decorated
one.

What to change, concretely

Three changes. The first two are the author's job. The third is a question for the platform.

1. Put a real ceiling in the input schema.

A field named maxPages should refuse an absurd value. The
maximum keyword does
this before your code runs:

"maxPages": {
  "title": "Max pages per PDF",
  "type": "integer",
  "default": 50,
  "minimum": 1,
  "maximum": 2000,
  "description": "Hard cap. Each page produces one billed result row."
}

default is not a limit. prefill is not a limit either, and the two are easy to confuse in
Console. That distinction bit me once already, from the correctness side. This is the same
lesson arriving from the money side.

You can verify the deployed schema rather than trusting the file:

curl -s https://api.apify.com/v2/acts/<user>~<actor> \
  | grep -o '"maximum":[0-9]*' | head

2. Say the price in the description the agent reads.

Once you pin, the agent has no other source. One clause is enough:

Each result row is one billed event ($0.002). A 500-URL input bills roughly $1.00.

That sentence costs you nothing. It is the only pricing signal a pinned agent will ever see.

3. The platform question.

call-actor exists in the default set and carries maxTotalChargeUsd. Pinned tools are
invoked directly, so no wrapper carries it.

A per-call spending cap that survived pinning would close the gap. It could be an optional
argument on every metered tool, or a connection-level parameter. Either way it would spare each
author from rediscovering this alone.

I do not know whether that is on anyone's roadmap. I am reporting the shape I measured.

Where this leaves an Actor author

Two connection modes, and a cost to each.

Leave the default. The agent can search, can read your pricing, can cap its spend. It will also
almost certainly never receive your Actor. The default set is 11 tools and none of them is
yours.

Pin your Actors. The agent receives them and calls them directly. It does so without a price
and without a ceiling.

I do not think either mode is wrong. I think the second one quietly moves a responsibility onto
the Actor author. The docs that recommend it do not mention the move.

Until they do, the fix is the schema you already control. Mine was missing it in all 23 files.
I only found out because I went looking for a different problem entirely.

Measurements were taken on 2026-08-12 against mcp.apify.com with a single account. Tool
listings came from tools/list over the MCP HTTP transport. Run durations and charge counts
came from
Apify API run records. No Actor was executed to
produce these numbers, so nothing here cost anything to measure.

I removed the under maintenance label from my Actor twice. It came back in four hours, then in one.

Ai-Q Labs — Fri, 18 Sep 2026 00:08:48 +0000

I removed the under maintenance label from my Actor twice. It came back in four hours, then in one.

I publish 23 Actors on Apify. Twenty-two of them take a public record and check whether what it
still claims is true.

The twenty-third is different. It reads an audit dataset and opens a GitHub issue for each finding.
It reaches GitHub through an Apify MCP connector,
so the Actor never handles my token.

That difference is the whole story. Two days after I published it, Apify flagged it as under
maintenance. I spent an afternoon removing the label by hand and learned that the label is not a
setting. It is a conclusion.

The email

The notification arrived at 10:08 on 8 August. My Actor had failed the automated tests.

Apify runs a daily test on every Actor.
The documentation is precise about what it does:

The test runs the Actor with its default input (defined by the prefill option in the input schema
file) and expects it to finish with a Succeeded status and non-empty default dataset within 5
minutes of the beginning of the run.

My Actor cannot do that. Not on a bad day — on any day.

Why this Actor and not the other 22

My other Actors take a URL, a domain, or a package name. Every one of those is a string. A string
goes in the prefill of the input schema, the test picks it up, and the run succeeds.

An authorized connector is not a string. It is a grant, tied to an account, held by the platform.
There is no value I can put in prefill that gives the test account access to my GitHub
installation. Nor should there be.

So the run fails at the first step, every day, forever.

The documentation says this outright, in a section I had not read:

Actors that require some sort of authentication will always fail the tests despite being fully
functional.

I want to be exact about what this means, because I got it wrong at first. I had assumed the cost
belonged to my design choice of failing fast on empty input. It does not. Twenty-two of my Actors
fail fast on empty input and none of them were flagged. The cost belongs to a narrower category:
Actors whose only successful path requires a credential the test account cannot hold.

If that describes your Actor, the daily test is not a quality signal about your code. It is a
structural mismatch, and it will not resolve itself.

What I did instead of reading the docs

I went to Console. Under Publication → Display information there is an Actor status control with an
"Under maintenance" switch. I turned it off and saved. The banner disappeared. The public page
stopped showing the badge.

I recorded it as fixed. It was not fixed. It was hidden.

At 17:08 the same day, about four hours after I cleared it, the second notification arrived. The
label was back.

I removed it again. At 18:08 — inside an hour — the third notification arrived.

Two removals, two returns, and the interval got shorter. That second number is what made me stop
and read.

The label is not a setting

Here is the mechanism, and it is stated plainly in the same documentation page I had skipped:

If the Actor fails to complete successful runs for three consecutive days, the developer will be
notified, and the Actor will be labeled under maintenance until it is fixed.

And the other direction:

The best course of action is to fix the Actor and rebuild it. The automatic testing system will
pick this up within 24 hours and mark it as healthy. In some cases, your Actor might break because
of issues with the target website. In such a case, if your Actor passes the majority of test runs
in the next 7 days, it will be marked as healthy automatically.

Read those two together. The label is derived from your run history, and the system recomputes
it. The switch in Console writes to a field that an evaluator overwrites on its own schedule.

For a normal broken Actor, the switch is honest and useful. You fix the bug, you rebuild, you clear
the label, and the next evaluation agrees with you. Your manual action and the computed state
converge.

My case had no convergence. The failures were still in the window and would be there tomorrow too.
Every removal I made was a claim the evidence contradicted, so the evaluator restored it.

I also stopped for a second reason, which has nothing to do with mechanics. Repeatedly toggling a
flag that an automated system keeps restoring is not a fix. It is an argument with a robot, in
public, on my own store listing. I did not want to find out how that reads from the other side.

The path that actually worked

The documentation tells you where to go:

If that's the case with your Actor, contact support and explain your specific use case that
justifies why the Actor should be excluded from the automated tests.

There is also a form. While my Actor was flagged, Console opened a skip-test request straight from a
query parameter:

https://console.apify.com/actors/<actorId>/publication?showSkipTestForm=true

One caveat I can only report, not explain. Two days later, with the exemption granted, that same URL
renders the ordinary publication page for me. So treat it as something that is there while you are
flagged, not as a permanent entry point. If it does not open, the documented route is support, and
that is the route the docs tell you to take anyway.

I filled it in with three sentences: what the Actor does, why the test account cannot authorize the
connector, and the sentence from the docs that describes exactly my situation. Quoting their own
documentation back to them felt lazy. It was the right call — it turned a request for special
treatment into a request to apply a rule that already existed.

The reply came in about 31 hours:

After reviewing your situation, we've approved your request. Your Actor will now not be checked by
our tests.

Then I cleared the label a third time. This one held, because there is no longer an evaluator
producing a verdict to overwrite it.

The order matters and I had it backwards. Get the exemption first. Clear the label second. Doing
it the other way round produces exactly what I got: a label that returns on a schedule you do not
control.

How to check the flag without lying to yourself

While I was doing this I needed a reliable way to answer "is it flagged right now?" I tried three
and only one of them is trustworthy.

The Console banner is fine, but it is one Actor at a time and it needs a browser.

The public Store page is not reliable. After my third removal I fetched the page and grepped for
the phrase. The badge was still in the HTML, on a page that was already correct in Console. When I
checked again later the same day, it was gone. So the page is not wrong — it lags, and on that day
the lag was hours. If you grep the page right after a change, you will read your own stale render
and conclude the change failed.

The API is the source of truth. The Actor object carries a top-level notice field:

curl -s "https://api.apify.com/v2/acts/aiqlabs~dataset-to-github-issues" \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['data']['notice'])"
# NONE

The values I have observed are UNDER_MAINTENANCE and NONE. This endpoint answers for public
Actors without a token, which means you can check anyone's — including, before you fork it, the one
you are about to depend on.

I did not find notice documented as the maintenance flag. I found it by reading the object.

A check you can run on everything you publish

After this, I stopped trusting my memory of which Actors were healthy. This prints the flag for a
list of slugs:

#!/usr/bin/env bash
# Print the maintenance flag for every Actor you name. No token needed for public Actors.
USER="aiqlabs"
for slug in dataset-to-github-issues github-repository-audit pdf-table-extractor; do
  notice=$(curl -s "https://api.apify.com/v2/acts/${USER}~${slug}" \
    | python3 -c "import json,sys; d=json.load(sys.stdin).get('data') or {}; print(d.get('notice','MISSING'))")
  printf '%-32s %s\n' "$slug" "$notice"
done

I ran it across the Actors I care about after the approval. All NONE, including the twenty-two
that were never at risk. That last part matters: it confirmed the flag had not spread from a shared
cause.

The trap next door

One warning that cost me three rounds of rework, because it sits on the same Console screen.

The Actor status switch lives under Publication → Display information. Saving that section dropped
one of my categories, three times in a row.

I had set three categories through the API. After each save, OPEN_SOURCE was gone, and I put it
back through the API.

Three for three is not an accident. But I want to be exact about the limit of what I know: when I
opened that same form today to write this, all three categories were sitting in the field. So I can
report the outcome I measured and not the cause.

So if you go anywhere near this screen, re-apply your categories through the API afterwards, and
read them back:

curl -s "https://api.apify.com/v2/acts/aiqlabs~dataset-to-github-issues" \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['data']['categories'])"
# ['DEVELOPER_TOOLS', 'AUTOMATION', 'OPEN_SOURCE']

Do not confirm this in the UI you just used. Confirm it in the API.

What it costs to ignore this

The label is not cosmetic, and the timeline is published:

After another 14 days of failing runs, you will receive another notification. Finally, if the runs
continue to fail after yet another 14 days, the Actor will be deprecated.

Three days of failures to get labeled. Roughly a month of them to get deprecated. For an Actor that
structurally cannot pass, that clock starts on the day you publish and never stops. Nothing in your
code will change it, which is precisely why manual removal feels like it works and is the most
expensive thing you can do with the time.

The rule I now apply

Before I publish an Actor that needs an authorized connector, I ask one question: is there any
input the test account can supply that produces a successful run?

If the answer is yes, I make sure that path is what prefill describes, and the daily test becomes
a free health check I get for nothing.

If the answer is no, I file the skip request in the same session as the publish, before the first
notification arrives. Then I never touch the status switch, because by then there is nothing left
for it to argue with.

The Actor in this story is dataset-to-github-issues.
Its source is public under ISC at
github.com/ai-q-labs/dataset-to-github-issues,
and the connector it uses speaks the Model Context Protocol.
The rest of what I publish is at apify.com/aiqlabs.

I gave my Actor a GitHub MCP connector. It could see 44 tools and use 4.

Ai-Q Labs — Fri, 18 Sep 2026 00:08:07 +0000

I gave my Actor a GitHub MCP connector. It could see 44 tools and use 4.

I publish 22 audit Actors on Apify. They all answer one question: is this public record still true?

The registry serves a package with no warning on it. Its repository has been archived. My Actors
find that gap and write it into a dataset.

Then nothing happens. A dataset is a place findings go to be correct in private.

So I built the missing half. It reads an audit dataset, decides what deserves attention, and opens
a GitHub issue for each finding.

It reaches GitHub through an
Apify MCP connector. That is a Model Context
Protocol (MCP) server, wired into the Actor's input by the platform. My Actor never touches my
token.

I expected the interesting part to be the writing. It was not. The interesting part was how little
the platform let my Actor do, and how much the connector appeared to offer.

At authorization, Console listed 44 tools. My run could see four. Both numbers are correct,
and the gap between them is the whole design.

Here is what I built, where in the run the connector fires, and the four things that surprised me.
One of them broke an assumption the Actor was designed around.

What I built, and why it is a separate Actor

The obvious move was to add issue-filing to
github-repository-audit directly. I did not,
for a boring reason: that Actor was under an unrelated measurement, and changing its build would have
destroyed the data.

The boring reason turned out to be the right architecture. Filing issues has nothing to do with
auditing. It needs a dataset, a tracker, and a policy. It does not need to know what npm is.

So dataset-to-github-issues takes a dataset ID and a connector, and nothing about my audit is
hard-coded into it.

The input schema is the interesting file:

{
    "datasetId": {
        "title": "Audit dataset",
        "type": "string",
        "editor": "resourcePicker",
        "resourceType": "dataset",
        "resourcePermissions": ["READ"]
    },
    "githubConnector": {
        "title": "GitHub connector",
        "type": "string",
        "resourceType": "mcpConnector",
        "mcpServers": [
            {
                "url": "https://api.githubcopilot.com/mcp/",
                "tools": {
                    "required": ["issue_read", "issue_write", "list_issues", "search_issues"]
                }
            }
        ]
    }
}

Two declarations sit side by side. One names what the run may read. The other names what it may call
through the connector. The platform enforces both. My code enforces neither.

That symmetry is easy to miss when you are writing it. It matters later.

Setting up the connector: the docs and Console disagree

The MCP connectors documentation says:

Supported services include Notion, Slack, GitHub, Sentry, and Supabase.

The Add new MCP connector dropdown in Console offers three presets. I scrolled to be sure. The
list does not grow.

Preset	URL
Sentry	`https://mcp.sentry.dev/mcp`
Notion	`https://mcp.notion.com/mcp`
Linear	`https://mcp.linear.app/sse`

Slack, GitHub, and Supabase are not presets. Linear is a preset and is not in that sentence.

This is not a bug. The URL field accepts free text, so the presets are a convenience, not an
allowlist. I typed the URL of
GitHub's own MCP server and waited.

https://api.githubcopilot.com/mcp/

About four seconds later the field turned green and an auth panel appeared. Apify had gone and asked
the server what it supports. It offered API key, selected by default. It rendered OAuth
grayed out, with this text:

This server doesn't support dynamic client registration. Your own OAuth client is recommended.

I like that the platform probes rather than assumes. It also told me exactly which path to take on a
free account: a personal access token used as a bearer credential.

The token I actually issued

I used a fine-grained personal access token,
scoped as tightly as GitHub allows:

Scope element	Value
Repository access	Only select repositories → one repo
Permissions	Issues: read and write
Permissions	Metadata: read-only (added automatically, marked Required)
Expiry	30 days

No contents write. No administration. No second repository. Remember this table; it is the point of
the next section.

The trap: the URL reverted while I was not looking

Between filling in the dialog and pasting the API key, the server URL changed under me. It went back
to https://mcp.sentry.dev/mcp, the first preset in the list. The field label also changed from
"MCP server URL" to "MCP server".

My API key field kept its contents. The URL showed a green validation check. Save was enabled.

If I had saved that, a GitHub token would have been registered as the bearer credential for
Sentry's MCP server. Nothing in the dialog would have complained.

Re-typing the GitHub URL and picking it from the suggestion fixed it, and the key survived the
change.

I have a rule now: read the server URL again immediately before you save. Typing it once is not
enough. A connector dialog holds a live credential, and it deserves the same paranoia as a payment
form.

44 tools discovered, 4 of them visible

Saving the connector produced an ID and a list. Console showed the tools it had discovered at
authorization time. All 44:

add_comment_to_pending_review  add_issue_comment  add_reply_to_pull_request_comment
create_branch  create_or_update_file  create_pull_request  create_repository  delete_file
fork_repository  get_commit  get_file_contents  get_label  get_latest_release  get_me
get_release_by_tag  get_tag  get_team_members  get_teams  issue_read  issue_write
list_branches  list_commits  list_issue_fields  list_issue_types  list_issues
list_pull_requests  list_releases  list_repository_collaborators  list_tags
merge_pull_request  pull_request_read  pull_request_review_write  push_files
request_copilot_review  run_secret_scanning  search_code  search_commits  search_issues
search_pull_requests  search_repositories  search_users  sub_issue_write
update_pull_request  update_pull_request_branch

Read that list against my token's permissions. create_repository is there. delete_file is there.
merge_pull_request, push_files, fork_repository, run_secret_scanning — all there, all far
outside a token that can only read and write issues in one repository.

The discovered tool list describes the server, not the token. It is a menu, not a grant. Every
one of those calls would die upstream, at GitHub itself.

That is a fine outer boundary. It is also the worst possible place to learn about it. So I looked at
what my run actually got. From the run log:

INFO  Read 12 row(s) from dataset iBbLEMf3NUSJcU0cZ.
INFO  The proxy exposes 4 tool(s) to this run: issue_read, issue_write, list_issues, search_issues

Four. Exactly the four names in mcpServers[0].tools.required, and nothing else. I changed nothing
about the connector between those two observations. The
input schema declaration
is what narrowed it.

The docs put it in one sentence, and now I have watched it happen:

The proxy enforces that an Actor can only call tools it explicitly declared in its input schema.

So there are three layers, and they do different jobs:

The token. What the upstream service will honor. Enforced by GitHub, discovered on failure.
The connector. What server this credential belongs to. Chosen once, in Console.
The input schema. What this Actor may call. Enforced by the proxy, before the request leaves Apify.

Layer 3 is the one that ties a tool to a specific Actor. It is also the only one a reader of my Actor
can see. Anyone can open my input schema and learn the worst thing my code can do to their GitHub
account.

I value that as much as the enforcement itself. A limit nobody can inspect is a promise.

Where the connector fires, and why not at the end

The easy design writes at the end. Audit, decide, file, done.

I fire the connector in the middle, twice, and the first call is a read. The Actor lists the open
issues in the tracker before it decides anything:

// This is the whole reason the connector fires here and not at the end: the decision of
// whether to open an issue depends on what is already open.
const openKeys = new Set();
for (let page = 1; page <= 5; page += 1) {
    const res = await call(client, readTool.name, argsFor(readTool, {
        method: methodValue(readTool, /list/),
        owner,
        repo,
        state: 'open',
        perPage: 100,
        page,
    }));
    if (!res.ok) { readError = res.error; break; }
    const list = Array.isArray(res.data) ? res.data : (res.data?.items ?? res.data?.issues ?? []);
    if (!Array.isArray(list) || list.length === 0) break;
    for (const issue of list) {
        // a title like "[dep-drift] npm:left-pad - deprecated..." gives back "npm:left-pad"
        const key = keyFromTitle(issue?.title ?? '');
        if (key) openKeys.add(key);
    }
    if (list.length < 100) break;
}

Identity lives in the issue title. Every issue my Actor opens is tagged [dep-drift], and the
finding key follows the tag. A later run recovers the key by parsing titles it wrote itself. No
state file, no external store, nothing to fall out of sync.

I proved it by running the same input twice, live, with the same settings.

First run:

INFO  0 finding(s) already have an open issue (read 0 title(s) over 1 page(s)).
INFO  Opened 3 issue(s).

Second run, identical input:

INFO  3 finding(s) already have an open issue (read 3 title(s) over 1 page(s)).
INFO  Opened 3 issue(s).
INFO  Done. skipped: 7, filed: 3, deferred: 2.

The Actor recognized its own three issues from the first run and skipped them. It filed the next
three. Two more hit the per-run ceiling, and it reported them as deferred instead of dropping
them.

Every row says why, in the dataset, whether or not it became an issue:

This is what a connector buys that a separate script does not. A write-only integration would
have re-filed the same three issues. Then it would do it again every scheduled run, and the tracker
becomes noise inside a week. Reading and writing in the same run, through the same authorization,
is what makes re-running safe.

The Actor also refuses to write blind. If the read fails, it stops:

if (readError) {
    await Actor.fail(
        `Reading the existing issues failed: ${readError}. Stopping before any write, because `
        + 'without that list this run cannot tell a new finding from one it filed last time.',
    );
}

Duplicates in someone's tracker cost more than a failed run.

The assumption that broke

My audit answers in three states. It found a defect. It found nothing. Or it could not check —
usually because GitHub's hourly allowance ran out mid-run.

The third state is the one I care about. Filing it turns "I don't know" into an alarm. Dropping it
silently turns an unanswered question into a clean bill of health.

So decide() gives it its own outcome:

export function decide(row, { minRiskLevel = 'high', openKeys = new Set() } = {}) {
    const key = findingKey(row);
    const real = realCodes(row);              // codes that mean "we found something"
    const unverified = unverifiedCodes(row);  // codes that mean "the check did not run"

    if (real.length === 0) {
        if (unverified.length) {
            return { decision: 'withheld', reason: `not verified (${unverified.join(', ')})`, key };
        }
        return { decision: 'skipped', reason: 'nothing to report', key };
    }
    if (openKeys.has(key)) {
        return { decision: 'skipped', reason: 'an open issue already covers this', key };
    }
    return { decision: 'file', reason: `${row.riskLevel}: ${real.join(', ')}`, key };
}

To exercise that path honestly, I needed an audit that really ran out of allowance. So I pointed the
audit Actor at gatsby's package.json — 166 dependencies. It duly ran out:

{
  "github":   { "requestsMade": 56, "distinctRepositories": 131,
                "repositoriesNotChecked": 75, "rateLimited": true },
  "findings": { "not_checked": 76, "repository_not_on_github": 76, "repo_moved": 7,
                "deprecated_on_registry": 4, "repo_archived": 2, "silent_abandonment": 1 }
}

Seventy-six unverified rows. I expected a dry run to report dozens of withheld findings. It reported:

INFO  Done. skipped: 161, would-file: 5.
0 finding(s) were withheld because the audit could not verify them

Zero.

I read the rows instead of guessing at the cause. Every single not_checked row also carried
repository_not_on_github. Not one row in 166 had an unverified code and nothing else.

That kills the assumption. I had built the decision around "a finding is either established or
unestablished". Real data says a row is usually both: something the audit established, plus a
caveat about a check that did not run. The whole-row case turns out to be the rare one.

The behavior that actually mattered was the other one. The caveat has to travel with the finding,
into the issue body. Here is issue #11, read back from the public GitHub API:

**npm:string-similarity** — risk: `high`

### What the audit found
- `deprecated_on_registry` (high) — The registry marks this deprecated: "Package no longer supported..."
- `repository_not_on_github` (low) — The registry links to git://github.com/aceakash/string-similarity.git,
  which is not a GitHub repository.

### What the audit could not check
- `not_checked` — aceakash/string-similarity was not checked: GitHub's hourly allowance ran out.

These are open questions, not clean results. Re-run the audit to close them.

A reader cannot mistake that last section for a clean result. The design goal held. The code path I
built to demonstrate it never fired.

I am leaving withheld in, with its unit tests, and saying plainly that no live audit has produced
one yet. It is designed behavior. I have never watched it happen.

The reply that looked complete and was not

Run 4 opened three issues and then crashed:

INFO  Opened 3 issue(s).
ApifyApiError: Schema validation failed   clientMethod: DatasetClient.pushItems
  instancePath: '/issueNumber'  message: 'must be integer'

The issues were real. I checked them from outside the run with an unauthenticated curl. The
report is what failed, because issueNumber came back null.

I had stored the first raw reply in the key-value store on purpose, so I could look instead of
guess. This is issue_write's answer, verbatim:

{"id":"5078084249","url":"https://github.com/ai-q-labs/github-repository-audit/issues/4"}

There is no number field. id is a string, and it holds GitHub's internal identifier. The
issue number — 4 — exists only inside the URL.

Reaching for id would have recorded issue "5078084249". Nothing would have errored. My dataset
would have looked complete and been wrong in a way no type check catches.

The fix reads the number where it actually lives, and refuses to invent one:

function findIssueRef(reply) {
    const out = { number: null, url: null };
    if (!reply) return out;
    if (typeof reply === 'object') {
        const o = reply.issue ?? reply.data ?? reply;
        const n = o?.number ?? o?.issue_number ?? o?.id;
        if (Number.isInteger(n)) out.number = n;    // note: a string id fails this on purpose
        const u = o?.html_url ?? o?.htmlUrl ?? o?.url;
        if (typeof u === 'string') out.url = u;
        if (out.number !== null && out.url !== null) return out;
    }
    const text = typeof reply === 'string' ? reply : JSON.stringify(reply);
    const m = /https?:\/\/github\.com\/[\w.-]+\/[\w.-]+\/issues\/(\d+)/.exec(text);
    if (m) {
        out.url ??= m[0];
        if (out.number === null) out.number = Number(m[1]);
    }
    return out;
}

Number.isInteger is doing real work there. It rejects the string id that would otherwise sail
through.

If you build against a connector, capture the first reply from every tool you call. A tool result is
whatever the upstream server decided to send. Assuming a field name and discovering the truth in
production is exactly the failure this prevents.

Two more failures worth designing for

A run cannot open an arbitrary dataset. My very first run died here:

ACTOR: Running under "LIMITED_PERMISSIONS".
ERROR Could not read dataset iBbLEMf3NUSJcU0cZ: Insufficient permissions for the dataset.

Same account, same owner, still refused. The fix is not a token — it is the resourceType and
resourcePermissions declaration from the schema at the top of this article. The schema names the
thing being read and the thing being written through. The platform grants exactly that, and no more.

A run that changed the outside world must not fail on its own bookkeeping. That crash after
"Opened 3 issue(s)" marked the whole run FAILED. FAILED reads as "nothing happened". That is the
opposite of the truth, and it sends the next run straight back at the same findings.

I now guard each dataset write on its own. Anything that fails lands in the summary instead of
killing the run:

for (const d of decisions) {
    try {
        await Actor.pushData(d);
    } catch (err) {
        reportProblems.push({ key: d.key, error: String(err?.message || err).slice(0, 300) });
    }
}

Order the work so the irreversible part happens last, and make everything after it non-fatal.

Seven runs got me here, and the first two are the ones worth reading:

The bug my new Actor found in my old one

Look again at issue #11. The registry link is git://github.com/aceakash/string-similarity.git.

That is on GitHub. The same row resolves aceakash/string-similarity and queues it for a GitHub
lookup. One row, two statements, contradicting each other.

My first explanation was that the URL parser rejects the git:// scheme. I ran the parser to check,
and it does not. It accepts git://, git+ssh://, git@host:path and github:owner/name alike.

The real cause is one line further on. The finding fires when the GitHub record is missing:

if (registryFound && !pkg.repoUrl) {
    add('no_repository_link', 'The registry record does not link to any source repository, so nothing can be verified against it.');
} else if (registryFound && pkg.repoUrl && !repo) {
    add('repository_not_on_github', `The registry links to ${String(pkg.repoUrl).slice(0, 120)}, which is not a GitHub repository.`);
}

The record is also missing when the lookup never ran. GitHub allows an unauthenticated caller 60
requests an hour. Past that, my Actor stops asking and stores null:

if (state.rateLimited) { state.skipped += 1; return null; }

So !repo means two different things, and the code reads only one of them. Every row cut off by the
rate limit is reported as a repository that is not on GitHub. That is why all 76 unverified rows were
paired. The pairing tracks the rate limit. The URL scheme has nothing to do with it.

It is the same mistake as the one in my first article, made a fifth time. A missing answer is not a
negative answer.

I have not fixed it. github-repository-audit is inside an unrelated observation window, and
changing its build would destroy that measurement. It is recorded, and it goes in next.

I did not expect the reader of an Actor's output to become the best test of that Actor. It is
obvious in hindsight. A dataset you only look at is a dataset nobody checks.

What it cost, and what I would do differently

Eleven issues, none duplicated, across four live runs and two audit datasets. All of it on the
free plan. When I stopped, my account usage read $0.61 against the $5 monthly credit. MCP
connectors are not a paid feature, which surprised me enough to check twice.

Three things I would tell myself before starting:

Declare the smallest tool set you can name. Not for safety theater — because the declaration is public. Anyone can read what my Actor is allowed to do without reading my code.
Fire the connector where the decision is, not where the output is. Read before you write, and stop if the read fails.
Store the first reply from every tool. The one that cost me a run was shaped nothing like I assumed. It failed quietly, in the one direction a type check cannot catch.

And one I got wrong: I designed a decision path around a data shape I had never seen. A branch, a
reason string, and unit tests, for a row that has not appeared once in 178 audited packages. Next
time I look at the data before I write the branch.

Run it yourself

The full source of the Actor in this article is at
ai-q-labs/dataset-to-github-issues —
the input schema, the decision function, the connector client, and the unit tests, including the one
covering the case that has never fired.

You will need three things to run it: an audit dataset with riskLevel and issueCodes on each
row, an MCP connector authorized against GitHub, and a repository you are willing to file issues in.
A free Apify account covers the rest.

Where the findings came from:
github-repository-audit is public and free.
Everything it produced through the connector is open in the
tracker repository, tagged
[dep-drift], including the one that exposed my own parser bug.

Full-size versions of every screenshot in this article sit in the source repository above, under
docs/screenshots.

The shortest version of the whole thing: your connector shows you a menu. Your Actor declares what it
eats. The proxy is what makes the difference real.

I connected my audit Actor to Claude, and it audited three packages nobody asked for

Ai-Q Labs — Fri, 18 Sep 2026 00:07:21 +0000

I connected my audit Actor to Claude, and it audited three packages nobody asked for

I maintain 22 audit Actors on Apify. They all do a version of the same thing. They take a public
record, and they check whether what it still claims is true.

Last week I connected one of them to Claude through the Apify Model Context Protocol (MCP) server. I
expected the interesting part to be the agent. It was not. The interesting part was my own input
schema. It had been lying to every non-human caller since the day I published it.

Then I made a second assumption. I decided I knew how an agent would read the result. I gave the
question to four of them to prove it, and all four proved the opposite.

This is what I found, what the agents did with it, and what I changed.

What the Actor does

github-repository-audit takes package names or
repository names. For each one it asks two sources the same question and compares the answers.

The registry says a package points at a repository. The repository says whether it is archived, moved,
or relicensed. Those two records disagree more often than you would think.

The finding I built it for is the quiet one. Here is real output from a run:

npm still serves cross-env@10.1.0 with no deprecation notice, while its repository
kentcdodds/cross-env has been archived. Installing it looks completely normal.

Nothing in your terminal tells you. npm install prints no warning. The archive banner sits on
the repository where nobody installing the package will
look. The maintainer said goodbye in the only place they could. The registry never passed the message
on.

Connecting it through the Apify MCP server

The Apify MCP server turns an Actor into a tool an
agent can call. The connection itself took one command:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=aiqlabs/github-repository-audit"

The ?tools= parameter matters more than it looks. Without it, the server exposes its whole surface.
With it, the agent sees exactly one tool. I wanted a clean experiment, so I scoped it to one Actor.

Authentication runs over OAuth with dynamic client registration and PKCE. I never handled a token.
The consent screen is worth reading rather than clicking through. Apify prints the honest version:

This application was registered dynamically, and wasn't verified by Apify. Make sure you trust it.
It's allowed to redirect you to following URL(s): http://localhost/callback

That is the right warning to show. Dynamic registration means nobody vetted the client.

The detour

My first three attempts failed, and the reason had nothing to do with Apify.

I was driving the CLI from a non-interactive shell. claude mcp login prints the authorization URL,
starts waiting, and then gives up:

Waiting for authorization… (^C to cancel)
Couldn't complete authentication: stdin isn't a terminal.

The login flow wants a real terminal even when the callback would arrive over the network. I tried
winpty next. winpty refused for the same reason one level down. It needs a console of its own.

What worked was a batch file, launched in a fresh console window, with stdout sent to a file. The
console satisfies the terminal check. The redirect lets me read the authorization URL. Then I opened
that URL, approved it, and the callback landed on localhost.

Write that down if you automate this. The blocker is the terminal, not the network.

The first run returned four rows for a one-repository question

Before handing the tool to an agent, I wanted to see the smallest possible call. So I sent one field
and nothing else:

{"repos":["facebook/create-react-app"]}

That is 39 bytes. The run recorded an inputBodyLen of 328.

Something had grown my input by a factor of eight. This is what the platform actually stored:

{
  "repos": ["facebook/create-react-app"],
  "packages": ["npm:request", "npm:babel-eslint", "npm:left-pad"],
  "defaultRegistry": "npm",
  "manifestType": "",
  "manifestGroups": ["dependencies"],
  "staleAfterDays": 365,
  "includeContributors": false,
  "includeReleases": false,
  "onlyIssues": false,
  "maxTargets": 200,
  "maxConcurrency": 4,
  "requestTimeoutSecs": 30
}

Three packages I had not mentioned were now part of the run. The results came back like this:

`input`	`source`	`riskLevel`	`issueCodes`
`facebook/create-react-app`	`repos`	medium	`repo_moved`, `stale_no_push`
`npm:request`	`packages`	high	`deprecated_on_registry`, `stale_no_push`
`npm:babel-eslint`	`packages`	critical	`deprecated_on_registry`, `repo_archived`
`npm:left-pad`	`packages`	critical	`deprecated_on_registry`, `repo_moved`, `repo_archived`, `license_mismatch`

I asked about one repository. I got four rows. Every high and critical row belongs to something the
caller never mentioned.

request and
left-pad are not neutral filler either. They are two of the
most famously abandoned packages in the npm registry. They produce findings by design.

prefill and default are not the same word

The cause is one line in my own input schema, written months earlier.

{
  "repos":    { "prefill": ["facebook/create-react-app", "babel/babel-eslint"] },
  "packages": { "default": ["npm:request", "npm:babel-eslint", "npm:left-pad"] }
}

I had used the two keys interchangeably. I thought both were examples for the form. The
input schema specification
is unambiguous, and I had simply never read this paragraph carefully:

Default — If the user omits the value when starting the Actor via any means (API, CLI,
scheduler, or user interface), the platform automatically passes the Actor this default value.

Prefill — this field is only used in the user interface but does not affect the Actor
functionality and API.

prefill is a suggestion. default is a promise the platform keeps on your behalf.

There is a second edge to this, and I only saw it once the Actor was a tool. The MCP server turns
the input schema into a JSON Schema and hands it to the model. Here is what arrives:

{
  "packages": { "default": ["npm:request", "npm:babel-eslint", "npm:left-pad"] },
  "repos":    { "prefill": ["facebook/create-react-app", "babel/babel-eslint"] }
}

default is a JSON Schema keyword. It means "the value used when this is omitted", and a model
reading the tool definition can act on it. prefill is not a JSON Schema keyword at all. It passes
through as an unrecognised key.

So the field nobody asked about announces itself in the language of the specification. The field that
answers the question carries a word the model has no rule for. I had put the standard keyword on the
wrong field.

My Actor's own code is innocent here. It destructures with packages = []. The injection happens
above it, before Actor.getInput() ever returns.

What I assumed would happen next

Open the Actor in Apify Console and the bug is visible immediately. The Packages box has three values
sitting in it. You delete them, or you leave them, and either way you decided.

The form is doing something important. It shows you the whole input, including the parts you did not
supply.

An agent never sees a form. It sends the fields it decided to send. It receives rows.

My rows do carry a source field, so the information needed to separate them exists. An agent that
reads it can tell the four rows apart. I want to be exact about that. It is the part I got right by
accident, because I did not add source for this reason.

But an agent does not read fifty-four fields per row back to a user. It reads the summary. So I
opened the two records my Actor writes for exactly that purpose.

SUMMARY counts the run:

{
  "checked": 4,
  "byRiskLevel": { "critical": 2, "high": 1, "medium": 1, "low": 0, "ok": 0 },
  "licenseComparison": { "comparable": 3, "mismatchRate": 0.333, "notComparable": 1 }
}

Two critical and one high, for a question about one repository. No count in that block says which
findings belong to the caller's target. The mismatch rate is worse. It is a fraction whose
denominator the caller never chose.

ACTION_LIST is the record I named for what to do next. Its entries carry target, registry,
repo, riskLevel and issues. It has no source field at all. It is sorted by severity:

#	`target`	`riskLevel`	did the caller ask for it?
1	`babel-eslint`	critical	no
2	`left-pad`	critical	no
3	`request`	high	no
4	`facebook/create-react-app`	medium	yes

The one entry that answers the question sits at the bottom. Above it are three the caller never
mentioned. The record offers no field that would tell them apart.

That is the shape of the trap. Attribution survives in the raw rows, which an agent skims. It
disappears from both records built to be read instead.

So I had a prediction, and it was a tidy one. Ask an agent whether create-react-app is safe to depend
on. It will read critical: 2 and pass that on.

I was wrong four times in a row

I had already read the schema, so my own answer proved nothing. I gave the question to four fresh
agents instead. None had seen any of this. The Actor was still unfixed.

Is facebook/create-react-app safe to depend on?

All four got it right. Every one of them reported medium risk for create-react-app, which is the
correct answer. Not one passed on the critical count.

Three of the four went further and told the user my tool was broken.

One thing worth flagging about the tool: my first run asked about a single repository but produced
4 rows. The extra 3 are the Actor's built-in defaults for the packages field […] Those three
carry the alarming findings — 2 critical, 1 high […] Reading that summary at face value would
have produced a badly wrong answer.

Another said the same thing in different words, then added a line I did not want to read:

when you pass only repos to this Actor, you have to set packages to empty explicitly or
unrelated rows get mixed in.

That is a workaround for my Actor, written by an agent, addressed to my user.

The trap still cost something

Nobody was misled. I want to be careful not to soften that. But four correct answers is not the same
as no harm.

Two of the four ran the audit twice. Double the compute units, double the GitHub requests against
a 60-per-hour unauthenticated allowance, double the wait. The caller paid to undo my mistake.

One told the user the tool was defective. That paragraph is now part of what my Actor looks like
to somebody deciding whether to use it.

One said nothing at all. The fourth agent gave a clean, correct answer and never mentioned the
three rows it had discarded. The caller was billed for four rows and got the use of one. Nothing in
the exchange told them.

And the wrong answer was sitting there the whole time. byRiskLevel still reads critical: 2 with
nothing beside it to say whose. Four agents declined to take it. That is not the same as it not being
there.

One of them found a second bug I had missed

The agent that ran the audit twice explained why:

I ran the audit a second time because the repo-only run left registryDeprecated, licenseMatch
and silentAbandonment all null — it never consulted npm, so it couldn't have answered "safe
to depend on" as asked.

It is right. A repository has no registry side, so three checks cannot run on it. One of them is
silent_abandonment, the finding this Actor exists for. A caller who names a repository never gets
it, and nothing said so. The fields just came back null.

I had written the rule against this in my own README. Return "broken", "fine" and "could not check"
as three different values. Then I returned the third as a bare null and let it read like the second.

Two of the four agents worked around it the same way, without being asked. That is a design gap, not
a coincidence.

What I changed

Five changes, in order of how much they mattered.

I removed default from every field that names a target. Sample values moved to prefill, which
keeps the Console form useful and never reaches the API. A field that decides what to audit must
come from the caller. A field that decides how to audit can have a default, and staleAfterDays
still does.

I made the empty call fail loudly. With no targets the Actor now stops with a message that names
the three ways in. It used to be impossible to reach that path, because the default guaranteed there
was always something to audit. That was the bug hiding the bug.

I moved the relevance rule into the description. The source field is now documented where an
agent reads it, not only in the README.

That last one had a second layer I only found while writing this. source did not appear anywhere in
my dataset_schema.json either. So the default Console table did not show it, and neither did the
"Problems only" view.

Opening that table made it worse. My view leads with packageName, and the row I asked for came from
repos, so it has no package name. The Console printed it as null.

#	Package Name	Registry	Risk	Resolved Repo
1	null	null	medium	react/create-react-app
2	request	npm	high	request/request
3	babel-eslint	npm	critical	babel/babel-eslint
4	left-pad	npm	critical	left-pad/left-pad

The one row that answers the question looks like the broken one. The three rows nobody asked for look
authoritative. I had built a view that ranked my own output by how little the reader wanted it.

Both views now lead with the two columns that answer "did I ask for this row?": input and source.

I made the summaries carry attribution too. This is the change I would have missed if I had
stopped at the schema. Removing the default stops this particular injection. It does not make the
output attributable.

ACTION_LIST now begins each entry with input and source. SUMMARY now carries a bySource
block beside the flat counts:

{
  "checked": 4,
  "bySource": {
    "repos":    { "checked": 1, "critical": 0, "high": 0, "medium": 1, "low": 0, "ok": 0 },
    "packages": { "checked": 3, "critical": 2, "high": 1, "medium": 0, "low": 0, "ok": 0 }
  },
  "byRiskLevel": { "critical": 2, "high": 1, "medium": 1, "low": 0, "ok": 0 }
}

The flat line still says two critical. Now something next to it says whose.

That block is eight lines in src/audit.js, inside the function that builds SUMMARY:

// A single call can mix targets the caller typed with targets that arrived some
// other way - a manifest that expanded into forty dependencies, or a schema
// default. A flat count of "2 critical" cannot be acted on, because it does not
// say whose.
const bySource = {};
for (const row of rows) {
    const key = row.source ?? 'unknown';
    bySource[key] ??= { checked: 0, critical: 0, high: 0, medium: 0, low: 0, ok: 0 };
    bySource[key].checked += 1;
    if (row.riskLevel in bySource[key]) bySource[key][row.riskLevel] += 1;
}

ACTION_LIST took two lines: input and source moved to the front of each entry, above
target. That ordering is the whole change. A reader who cannot answer "did I ask for this?"
cannot act on the row.

This matters beyond the bug that started it. My Actor takes repositories, packages and a manifest in
one call. A manifest URL can expand into forty dependencies from a single field. Any of those
mixes produces a count the caller cannot take apart. The default was one way in. It was not the only
one.

Going through the rest of the output found one more. LICENSE_REPORT ends with a list of licences
that could not be resolved. That list is an instruction to open files by hand. It held bare names. So
I gave those entries input and source as well. Sending someone to read a licence for a dependency
they never named wastes the same afternoon a false finding does.

I made "could not check" say so out loud. This is the one an agent found for me. A
repository-only row now produces a note in SUMMARY:

1 target(s) were given as repositories, so no registry was consulted for them. registryDeprecated,
licenseMatch and silentAbandonment are null on those rows because they could not be checked, not
because they came back clean. Pass the package name to check them.

The same sentence went into the repos field description, where an agent reads it before calling.
null was already the honest value. It was not a legible one.

Four tests hold this. Three check the new fields on the three records. The fourth is a control. A run
from a single source must report one group, and its numbers must equal the flat totals. So the
breakdown cannot invent structure that is not there. The suite went from 48 to 52.

What actually changed, measured the same way

I pushed build 0.1.8 and sent the same one-field call again.

	before	after
rows returned	4	1
stored `INPUT`	328 bytes	267 bytes
`byRiskLevel`	`critical 2, high 1`	`critical 0, high 0`
`bySource`	absent	present
first `ACTION_LIST` entry	`babel-eslint`, critical	the repository I asked about

Then I ran the agent trial again, same question, same plain prompt, two fresh agents.

Both answered correctly, as before. Neither mentioned a defect in the tool. Before the change,
three of four had. That is the entire measurable result, and it is a modest one. Nobody was ever
given a wrong answer, so nothing about correctness improved.

What improved is that my Actor stopped asking its callers to compensate for it.

One thing did not change. Both agents still ran the audit twice, once by repository and once by
package. That is now the right behaviour, and the output asks for it in writing. I should say that my
new note may be causing the second call rather than merely permitting it. One agent made the same
second call before the note existed, which argues against that. With four trials I cannot separate
the two.

Then I checked the other twenty-one

One bad field is a typo. I wanted to know whether it was a habit. So I read every input schema I
have published and sorted the defaults into two piles.

A default is safe when it decides how the work is done. It is dangerous when it decides what
the work is done to.

Twenty-two published Actors. Every one of them sets a default somewhere. Ten of those defaults
still name a target, and I had just removed an eleventh.

Actor	field	what it audits when the caller says nothing
`seo-ai-visibility-auditor`	`startUrls`	`apify.com`
`bulk-domain-checker`	`domains`	`apify.com`
`domain-availability-checker`	`domains`	`apify.com`
`sitemap-checker`	`domains`	`apify.com`
`tech-stack-detector`	`domains`	`apify.com`
`dead-link-checker`	`domains`	`docs.apify.com`
`http-status-checker`	`urls`	`apify.com/store`
`pdf-inspector`	`pdfUrls`	a US tax form
`pdf-to-text-markdown`	`pdfUrls`	the same tax form
`hacker-news-link-rot`	`list`	the Hacker News `topstories` list

Call any of the first nine with an empty input and you get a finished report. It covers Apify's own
website, or a US tax form. Nothing in the response says it is a sample. An agent has a well-formed
answer to a question nobody asked.

Then I noticed the part that embarrassed me most. Nine of those ten fields also appear under
required in the same schema.

I had been reading required as a promise that the caller named the target. So I tested that
reading. I called seo-ai-visibility-auditor with an empty object.

echo '{}' > empty.json
apify call GuuMKUiWcaUUqGGhG --input-file=empty.json

The run succeeded and audited apify.com. Here is what the platform stored as my input:

{
  "startUrls": [{ "url": "https://apify.com" }],
  "crawlSite": false,
  "maxPages": 10,
  "checkBrokenLinks": false,
  "maxLinksToCheck": 50,
  "proxyConfiguration": { "useApifyProxy": false }
}

I sent {}. The Actor received a target, and nothing rejected the call.

The specification
treats the two settings as alternatives rather than as a pair. required is for fields that "don't
have a reasonable default". default is passed by the platform whenever the caller omits the field,
"via any means". Put both on one field and the second one decides. required survives as a note to
whoever is reading the form.

The other twelve Actors are fine. country, outputFormats, robotsAgent, manifestGroups —
omitting those does not invent work.

One row in the table made me think harder. hacker-news-link-rot defaults its list field to
topstories, and that does pick the target. But the Actor has no other way in. An empty call has to
mean something. It is also the one row I never marked required.

So the rule is not "never use default". It is narrower than that:

A caller who says nothing must not receive results they cannot tell apart from results they asked
for.

hacker-news-link-rot satisfies that with one sentence in its description. The other nine did not
satisfy it at all.

The same mistake, four times, before an agent was involved

Here is what stung. This was not a new class of error for me. It was the fourth time.

Every audit Actor I have written has produced a confident finding that was really a gap in my own
knowledge. Each time, the fix was the same shape: split one value into two.

A rate limit that looked like deletion. My Chrome extension auditor read 130 listings from
Google's own sitemap, one every 600 milliseconds. Ninety-three came back as "the store has never
heard of this ID". Every one of them was a healthy extension I had listed minutes earlier.

Google serves its rate-limit interstitial as a redirect away from the store. A reader that only asks
"did I land on a listing page?" sees exactly what a deleted extension looks like. The Actor now checks
for that first. It never treats the interstitial as a fact about the extension. It stops the run
instead of producing ninety-three more.

A 404 that was an advertising server refusing. My podcast auditor reported a 1.7% episode death
rate. Nine of those failures came from one host, and all nine shows were running fine.

A per-listener redirect service was serving that audio, and stitching in advertising as it went. It
would not build a redirect for an automated request. So it answered 404, with the body Missing redirect URL. I had a real death rate of 0.5% and a fake one three times larger.

Those cases became undetermined, not dead. The finding text says what I actually know:

Whether the audio is there cannot be established without behaving like a listener, which this
Actor does not do.

A licence warning on lodash. GitHub could not match lodash's LICENSE to a standard licence, so it
returned NOASSERTION. My first version called that license_non_standard and raised it.

lodash, jQuery UI and UglifyJS all land there. They are ordinary MIT and BSD projects whose LICENSE
carries an extra paragraph. A warning that fires on healthy rows does not add information. It buries
the rows that matter. The code comment I left says it better than I can paraphrase:

Raising this would put a warning on healthy rows and bury the ones that matter.

A severity I set before I measured anything. I once shipped "no update in three years" as a high
severity finding. Then I measured the base rate across 1,312 App Store apps. It is 18.4%.

A finding that fires on one row in five describes the ecosystem. It does not describe a problem with
your dependency. It sits at medium now.

For the Shopify auditor I dropped the check entirely. Shopify publishes a launch date and no update
date. Building the check anyway would have meant guessing.

The rule I ended up with

Every one of these is the same rule, arrived at four times the slow way.

Return "broken", "fine", and "I could not check" as three different values.

I used to justify that by saying an agent cannot tell the third from the second. My own trials say
otherwise. Four agents pulled the distinction out of a source field I had not documented. Two
worked out on their own that a repository-only row never touches the registry.

So the reason is not that agents cannot cope. It is what coping costs.

Every ambiguity you leave in your output is work you have handed to the caller. Sometimes they pay it
in a second run against a rate-limited API. Sometimes they pay it by writing a paragraph explaining
your tool's quirk to their user. Sometimes they pay it silently, by throwing away three quarters of
what you charged them for.

None of that shows up as an error. It shows up as your Actor being slightly more expensive and
slightly less trusted than the one next to it.

The input schema is the other half. If a field can change what gets audited, the caller must set it.
Silence has to mean silence.

What to check in your own schema

Five things, in the order that cost me the most time.

Search your schema for "default". For each hit, ask what happens when an API caller omits that field. If the answer changes what the Actor works on, move it to prefill. Listing the field under required will not do this for you.
Call your Actor with the minimum viable input, then read the stored INPUT record. Do not read the input you sent. Read what the platform saved. That is what your code receives.
Make sure an empty call fails. If your Actor can always find something to do, you cannot tell an empty call from a real one. I checked what that costs, because the platform runs published Actors on its own schedule. Twelve of my twenty-two already have nothing to do on an empty call, three of them because a required field carries no default at all. All twenty-two show zero failed runs across 159 platform runs in the last thirty days.
Write your limits into description, not the README. The description is what an agent reads. Mine now says why pyproject.toml is unsupported: half-parsing a manifest produces findings about dependencies you do not have.
Hand the tool to an agent and read what it says about you. This found more than my own review did. An agent that works around your quirk will usually explain the quirk to its user, in writing, in the answer. That paragraph is a free bug report. It is also what your Actor looks like to a prospective user.

Code

The full source for the Actor in this article is on GitHub:
ai-q-labs/github-repository-audit. That is
the code behind the published Actor at
apify.com/aiqlabs/github-repository-audit. It
includes the input schema this article is about, 52 unit tests, and a live check against the real
GitHub, npm and PyPI APIs.

Clone it, run npm install, then npm test for the unit tests or npm run test:live for the live
check. The live check needs no key, but GitHub allows unauthenticated callers 60 requests an hour, so
set GITHUB_TOKEN if you run it more than once.

Full-size versions of every screenshot in this article sit in the same repository, under
docs/screenshots.