AI agents were told my inputs were optional, then rejected for leaving them out 🤖

One word in 29 config files: required. Every field it guarded already had a default.

Hi! I'm Tim, a self-taught developer in Nairobi. I build small data tools that run as Apify Actors, serverless jobs that people pay for per result. 100 of them are public.

This week Apify's DEV account published What AI agents see when they call your Apify Actors, by SIÁN Agency. One finding in it sent me straight to my own input schemas.

🤖 What an agent sees

A person runs an Actor from a form. The fields have labels, the dropdowns list the valid values, and an example is already filled in.

An AI agent connected through the Apify MCP server gets none of that. It gets the input schema as JSON Schema, and it sends what the schema says.

The article found that two parts of the MCP server read that schema differently:

• tools/list tells the model what a tool accepts. It leaves a field out of required when that field has a default.
• tools/call checks the model's arguments. It validates against the original schema and doesn't fill in defaults first.

So a field that is both required and has a default is listed as optional. When the model leaves it out, the call is rejected with a validation error (-32602) for the very field it was told it could skip.

An agent can recover by fetching the Actor's details and trying again. It still burns a round trip on the first call, every time. The author found this in 59 of the 253 schemas they checked.

🔍 How many of mine?

I wrote a short script that reads every .actor/input_schema.json in my portfolio. For each required field, it records whether the field has a default and whether that default equals the prefill, the example shown in the form.

	Count
Actors with at least one required field	29 of 100
Required fields that also had a default	32, in 26 Actors
Required fields with no default	3, in 3 Actors
Defaults identical to the prefill	32 of 32

Here's a typical one, from a vehicle safety ratings lookup (trimmed to the parts that matter):

{
    "required": ["year", "make", "model"],
    "properties": {
        "year":  { "type": "integer", "prefill": 2022,     "default": 2022 },
        "make":  { "type": "string",  "prefill": "Honda",  "default": "Honda" },
        "model": { "type": "string",  "prefill": "Accord", "default": "Accord" }
    }
}

All three fields are required, and all three have a default. In the form, nothing looks wrong, because the form is already filled in. To an agent, all three are listed as optional, and a call without them fails.

🙈 Why nothing ever caught it

Apify runs every public Actor once a day with its prefilled input, and flags it if that run fails three days in a row.

All 32 defaults were identical to the prefill. So the daily test always sent every required field, and it always passed. None of these Actors was ever flagged for it.

The test walks the path a person takes. An agent sending {} takes a different one, and nothing was walking it.

🚫 Why I didn't run the probe

The article comes with a probe that calls each Actor through the MCP server with {}. I didn't run it on mine. My own runs show up in my Actors' analytics as a user, and that analytics page is the only signal I have about real customers, so I debug locally and keep platform runs to a minimum.

That left the static half: what the schemas say. The article is clear that this half can't find everything. An Actor can report SUCCEEDED with nothing useful in it, and only running it shows that. So after the schemas, I read through the code paths of my best-earning Actor by hand, looking for that kind of silent result. More on that in fix 3.

🔧 Fix 1: a field with a default doesn't need required

Apify writes schema defaults into every run's input. A required field with a default is filled in whether the caller sends it or not, so the required flag adds nothing to the run. All it does is mislead the agent.

The rule I applied to the 26 Actors:

// Drop `required` only when every required field already has a default.
const noDefault = schema.required.filter((k) => schema.properties[k].default === undefined);
if (noDefault.length) throw new Error(`no default for ${noDefault.join(', ')}`);
schema.required = [];

Nothing else in any schema changed, and the script checked that too. A person in the form sees exactly the same thing as before.

🔧 Fix 2: three fields had no default at all

A court opinion search (query), a travel per diem lookup (locations) and a package health check (packages) each had a required field with no default.

These were honest: with no default, nothing gets dropped from the tool listing, so an agent is told to send the field. But their code also threw on empty input:

if (!query) {
    throw new Error('query is required.');
}

I wanted every one of my Actors to answer {} the same way: with the example from the form, not an error. Each of the three got a default equal to its prefill, and the code now falls back to that example:

// An empty call (an AI agent calling with {}, or an empty API call) searches the input form's example instead of failing.
const query = String(queryInput ?? '').trim() || 'antitrust';
if (!String(queryInput ?? '').trim()) log.info('No query given, so searching the example "antitrust".');

In the two that take a list, a value that isn't a list still fails with a clear message. Only "nothing at all" falls back to the example.

🔇 Fix 3: zero rows is not an answer

My best-earning Actor audits small business websites and scores each business as a lead for web designers. It has a filter: only keep businesses that score above a minimum.

Set that minimum high enough and every business gets filtered out. The run used to end SUCCEEDED, with zero rows. An agent reading that would tell its user "no leads found", as a fact about the world. The real answer was "your filter was stricter than every business it checked".

And an empty call ({}) used to fail outright with "Nothing to audit", even though the schema told an agent nothing was required. Now it audits the three example businesses from the form.

For the empty case, a run that saves nothing now saves one row explaining why, and doesn't charge for it:

// Never end with an empty dataset: say why nothing was saved, in one uncharged row.
if (stats.pushed === 0) {
    const note = emptyRunNote(stats, counters, options, search, categoryTitle);
    await Actor.pushData(note);
}

Here's the row it writes when the filter removes all 5 businesses it audited:

{
    "name": "No businesses saved in this run (not charged)",
    "websiteStatus": null,
    "opportunityScore": null,
    "note": "All 5 businesses audited scored below 100, so none were saved (filtered businesses aren't charged). Lower minScore or turn off onlyOpportunities to see them.",
    "processed": 5,
    "filteredOut": 5
}

The row says what happened and what to change. A person can read that. So can a model.

🚀 Shipping 29 Actors, one at a time

29 Actors meant 29 builds. I didn't want a loop that pushes everything and checks nothing, so each Actor went through the same steps, and the script stopped at the first problem (trimmed):

before=$(node live-build.js "$a")            # the build to roll back to
git commit -q -m "$msg" && git push -q origin HEAD
out=$(timeout 480 apify push 2>&1)
echo "$out" | grep -q "Build: SUCCEEDED" || { echo "STOP $a"; exit 1; }
verify=$(node live-build.js "$a" --verify)   # live schema: required is [] and matches local
echo "OK $a | before $before | after $verify"

The --verify step reads the input schema back from the new live build, not from my disk. All 29 passed, and the previous build of every Actor is saved in case one needs rolling back.

🎯 What I learned

• My input schemas have a second reader now. I wrote them for a form. A model reads them as a contract, and a contradiction in the contract becomes a failed call.
• required plus default is a contradiction. If the platform fills a field in anyway, don't tell callers they must send it.
• A daily test only checks the path it takes. Passing it says nothing about any other input, including an empty one.
• Zero rows needs a reason. If a filter removes everything, say so in a row, not in a log nobody opens.
• There's more to check. 33 fields across 16 of my Actors still have a prefill but no default. A person sees the example. An agent sending {} gets nothing for that field. Some of those Actors already fall back to an example in code; I haven't checked the rest yet.

If you publish Actors, this one-liner lists the fields in a schema that are both required and defaulted:

node -e "const s=require('./.actor/input_schema.json');console.log((s.required||[]).filter(k=>s.properties[k]?.default!==undefined))"

An empty list means an agent gets an honest contract. Anything else is a field it was told it could skip, and then got rejected for skipping.

Comments

[Dave Kurian]

Your distinction between schema validation and the Actor’s defaulting is the key seam. I’d keep a tiny offline contract fixture for each schema with {}, the prefilled example, and one invalid value, and run it through the same validation configuration as tools/call; then assert both acceptance and the effective input that reaches the Actor. That catches the list/call mismatch without spending a live run.

[Timothy Kelvin]

Btw that's a very good idea, I tried it. small offline script with ajv: for each schema it checks that {} and the prefilled example get accepted, that one wrong-typed value gets rejected, and what {} turns into once defaults are filled in. on my 31 public actors everything passed and none have a required field with a default anymore. the useful bit was that last check, 2 of them get {} with nothing filled in at all, so the fallback lives in code and only a local run proves it. do you pin the ajv options to whatever the mcp server uses, or just run default ajv?

[Dave Kurian]

Nice, that last check surfaced a different edge than validation: two actors accept {} but get no effective input at all, so only the local path proves fallback behavior. I’d keep that case in the fixture as its own assertion. Does your check pin the Ajv version and options too?

[Timothy Kelvin]

Not at first, it borrowed whatever ajv the apify cli ships. your question made me fix that: it’s pinned to 8.20.0 now, same options (strict off because Apify schemas have their own keywords like editor and prefill), and the version and options sit in a baseline file the check compares against. i also made the “gets nothing from {}” case its own assertion like you said, plus what {} becomes for every actor, so any default that changes fails it. still don’t know what the mcp server itself runs though

[Dave Kurian]

Pinning Ajv 8.20.0 and the options gives you a reproducible local baseline. The remaining gap is server parity: until its runtime is inspected, the check cannot establish that tools/call uses the same version and options. Could the test output label that explicitly as “local baseline verified; server parity unknown”?

[Timothy Kelvin]

good call, added it. a pass now says “local baseline verified” and every run ends with “server parity unknown”, since nothing on my side can see what apify’s mcp server actually runs

at this point your comments have basically written this script lol

[Dave Kurian]

Fair point. Your new wording draws the line cleanly: local baseline verified, server parity unknown. I’ll leave the script in your hands now.

[Timothy Kelvin]

Appreciate it, this thread made the script a lot better than what i started with

[Mehdi Mohseni]

The root of it is in JSON Schema itself: default is an annotation keyword in draft 2020-12, so a validator doesn't fill it in. tools/list treating it as optional is the generous reading; tools/call is the literal one. Dropping required on any field that has a default is the fix that matches the spec.

[Timothy Kelvin]

Good point, I was calling it a platform bug but the strict side is the one following the spec. got curious and tried it with ajv (8.20): {} against a required field with a default fails with “must have required property”, same wording as the MCP error, so I’m guessing that’s what the server uses. with useDefaults: true the same call passes and the default gets filled in, so one option would make both sides agree. i still dropped required on my end since i can’t change the server and Apify fills defaults when the run starts anyway. which fix would you rather see, defaults applied before validation or tools/list keeping required?

[Mehdi Mohseni]

One catch if the server ever turns useDefaults on: Ajv writes the defaults into the data object in place, and it doesn't apply defaults inside anyOf, oneOf or not. So a schema that nests its options in a oneOf would still reject {}.

[Magshifter]

Really interesting read, Tim!

The part about daily tests passing while AI agents were hitting a completely different path caught my attention.
I've been running into similar questions while building MAGDC, especially around what it actually means for an AI-assisted task to be considered complete.
I also liked your point about zero rows needing an explanation rather than just returning SUCCEEDED.

I'm curious: did fixing these issues change how you plan to test your Actors going forward, especially when they're called by AI agents instead of people?

[Timothy Kelvin]

Thanks! yeah it did. {} is a real test case now, not an edge case. before i change any input schema i run a small offline check with a pinned ajv: every actor has to accept {} and its own example, reject a wrong type, and what {} turns into after defaults can’t change without me noticing. the 2 actors where {} becomes nothing at all get a local run, because only the code proves that fallback. and the ones i reworked this week say why when they return zero rows, or mark rows that came from the example instead of the caller’s input. what i still can’t do cheaply is run everything through the mcp server, my own runs count as users in my analytics, so most of it stays local. what counts as done in magdc, the task finishing or the output being checked?

[Magshifter]

I really like that you've made {} a real test case rather than treating it as an edge case. And checking actual runtime behavior separately makes sense — schema validation alone can't prove what the code will do.

In MAGDC, finishing the task and verifying the result are two different things.

What we've implemented: We have an Evidence Gate. An agent reporting "DONE" isn't enough to establish that a verification claim has passed. The evidence needs to support the actual claim, and missing or failed required evidence means it isn't complete.

We also distinguish technical verification from practical acceptance by the Product Owner when an increment requires it.

For example, an agent might successfully implement a feature and pass its local tests, but that doesn't automatically prove the feature works in the deployed environment or that I've accepted the result.

This is something I've learned from real development problems: good code and a successful task report don't always mean the intended behavior was delivered.

So my answer is: the checked result matters, not just the finished task.

Your {} example is a great illustration of why that distinction matters.

[Timothy Kelvin]

apify kinda forces that split on me lol. it runs every public actor once a day with its example input and flags it after 3 failed days, no matter what my own tests said. that’s how i found out a gov data site had started refusing my requests

the sneakier one was a schema default. my code had a “no query given” fallback that could never fire on apify, because the platform fills defaults in before the run even starts

[Glen Allen]

This is a great example of why schema correctness and runtime behavior need to be tested together. A field having a default doesn't automatically mean a JSON Schema validator will apply it, so the mismatch between tool discovery and tool execution can be easy to miss. I especially like the point about testing empty inputs and making zero-result responses explain what actually happened. Would contract tests for both tools/list and tools/call help catch these inconsistencies earlier?

[Timothy Kelvin]

ran the tools/list half against apify’s mcp server for all my actors. fields, defaults and required lists matched everywhere, but the server drops minimum and maximum from every field, so an agent is told maxResults has no ceiling while the actor’s own schema caps it at 100

so the discovery side and the execution side disagree on 85 limits across 28 actors lol. tools/call is still the hard half since every call is a real run

[Kartik N V J K]

Fix 3 got me. I've had an agent take an empty payload and write a confident summary over it, and a SUCCEEDED run with zero rows becoming "no leads found" is that same failure. Can a caller branch on the note key, or does a real lead row carry a note too?

[Timothy Kelvin]

yeah you can branch on note. real lead rows never have that key, only the empty-run row does, and when it shows up it’s the only row in the dataset

the one thing i wouldn’t branch on is a null opportunityScore. sites that block automated checks come back as real rows with a null score too, their “Couldn’t audit” reason lives in opportunityReasons instead

tbh a rowType field would be cleaner than checking whether a key exists lol, your question kinda makes the case for it

[Jisung Yoon]

I would make the example fallback visible in the returned data, not only the log. Otherwise a caller could mistake results for “antitrust” for results from their own missing query.
Could each result include the effective query and whether a default was used?

[Timothy Kelvin]

Good catch, and it’s worse than the log thing. since the field has a default in the schema, Apify fills it in before the actor even starts, so the code can’t tell “nobody sent a query” from “someone searched antitrust”, and that log line never fires on the platform. the per diem and package ones already put the location and package in every row, but the court search doesn’t. going to drop the schema default on that field, keep the fallback in code, and add the query plus an exampleQuery flag to each row. thanks