A model never hands your tool a typed object. It hands you text that claims to describe one, and everything between that text and your validated arguments is a parse you control. How forgiving that parse should be is the whole design question at the boundary, and the answer is not the same in both directions.
- Arrays that arrive as strings containing arrays
- Values that are not JSON and parse perfectly anyway
- Arguments that are still streaming when validation wants them whole
- The quiet fallback that turns a clean rejection into a confusing error
Our harness is the worked example throughout, and every mechanism here is small enough to copy in an afternoon.
The malformation you will hit most often
An LLM asked for a list of strings will hand you a string containing a list of strings. It sends the text '["a","b"]' where your schema declared ["a","b"]. The data is correct and only the wrapping is wrong, but input validation, the check that runs before your tool code and confirms the arguments match the shape you declared, does not care about that distinction. It rejects the call. The rejection goes back to the model, and a full turn of the conversation gets spent teaching it to remove two quote marks.
Coerce it instead. Coercion, quietly converting a value into the type you expected rather than rejecting it, costs you nothing here, because there is exactly one thing '["a","b"]' can reasonably mean. Nothing is logged, nothing is retried, the tool just runs. The turn you save is the cheapest turn you will ever save, and you save it on every call that would have tripped.
Put the coercion in the base class, not the field
Fixing this per field works only for the fields you remembered. Every tool input model in our harness inherits from a shared base class, ToolInputBase, rather than from the plain validation model, and that base converts a JSON string into the list it represents before validation ever runs. Our own documentation lists inheriting from the plain model as a common pitfall, which is a polite way of saying somebody has to trip over it before they learn.
The reason it belongs at the base is that the malformation is a property of the boundary rather than of any one tool. Nothing about a particular tool makes a model more or less likely to quote an array. So a tool written next year inherits the fix without its author ever knowing the problem exists, and that is the actual goal. The alternative is a line in a contributing guide that every future author has to read, remember, and apply by hand.
Not everything that parses is JSON
The quoted array is one instance of a wider problem, which is that near-JSON arrives constantly and json.loads rejects all of it. Python's own repr of a dictionary uses single quotes and writes True, False and None where JSON demands double quotes and lowercase true, false, null. It is unambiguous, it is trivially parseable, and a JSON parser will refuse it every time.
Try the more permissive parser first. Where our terminal renders tool output, the rule is to call ast.literal_eval, the standard library's safe evaluator for Python literals, and fall back to JSON only if that fails. That instance sits on the return path rather than the argument path, so the direction differs, but the shape of the problem is identical and so is the fix. Our own docs state the rule flatly enough to steal. Never call json.loads() directly on that data.
Reaching for a single parser encodes an assumption about who produced the text. That assumption holds until a model, a subprocess, or a logging layer produces something near-JSON instead, at which point a strict parser gives you a failure that reads like a real error and is not one.
The arguments may not have finished arriving
Streaming breaks the assumption underneath all of the above, which is that you have the whole argument object when you go to parse it. When a model streams a tool call, the arguments arrive as a growing string, and a large one is still arriving while you would like to start work. Our streaming handlers split the difference by key. Short scalar keys land through on_key_completed once they are whole, and the one big streamable key arrives in pieces through on_string_progress, so a file write can begin before the content ends.
That split forces you to define what incomplete means. A stream can stop early, and the model reports why through a finish reason, where a value of length means the output was cut off mid-value rather than finished. Truncated JSON is not malformed in the way a quoted array is malformed, because there is no correct parse to recover, only a decision about what to do with a partial write. Our handlers are required to never raise from their completion or error paths and to always return a result, which is the same instinct as coercion applied one level up. The handler knows which file it was writing and how much of it landed, and a caller catching an exception several frames away knows neither.
The coercion that becomes a bug
Every mechanism above absorbs a malformation whose meaning is unambiguous, and the trouble starts precisely where that stops being true. Our streaming write handler once validated its mode argument with a fallback, assigning the mode when it matched a known value and quietly defaulting to write when it did not. When overwrite was later removed from the set of valid modes, a model still asking for mode=overwrite did not get told that the mode was gone. It got silently converted into a write, which then failed further down with a message saying the document already exists. The model asked to replace a document and was told, in effect, that the document was there. That is a worse outcome than the rejection it replaced, and it is documented in our own change notes as the expected consequence.
Coerce the representation, never the choice. Quoting, encoding and literal syntax are representation, and there is one right answer for each of them. Which enum member, which path, which identifier is a choice, and a choice your harness cannot honour is a rejection, not a value to be guessed. When you do reject, name the options that would have worked, because your harness knows them and the model is about to spend a turn finding out.
Where we refuse to coerce at all
The same argument that justifies coercion at the model boundary forbids it a few layers in. Our workflow engine evaluates conditional transitions against workflow state, and it does no implicit type coercion whatsoever. A type mismatch between the stored data and the expected value raises rather than comparing something adjacent, and comparison operators apply plain Python semantics with nothing clever underneath.
The criterion that separates the two cases is where the value came from. Be forgiving with what a language model produced, because its malformations are artifacts of generating text and carry no information you want. Be strict with what your own system produced, because a mismatch there is a bug, and coercing it hides that bug at the exact moment it is cheapest to see. The same permissiveness that saves a turn at the boundary will cost you an afternoon when it silently compares an integer against a string somewhere in your state machine.
Where your parser stands
Three questions place any harness on this map. When your model wraps an array in quotes, does the call fail? When a value arrives as a Python dict repr instead of JSON, does your parser survive it? When you cannot honour an argument, does the caller get a named rejection or a silent default? Each one is a few lines at the boundary and a turn you stop paying for.
Favur is our multi-agent software team, and the harness these mechanisms run in. It is closed source and invite-only, but the repositories it produces are open, and you can drive a replay of a real run at https://favur.dev/go/devto/tool-coercion or see the same harness scored across models at https://evals.favur.dev/go/devto/tool-coercion. I work on it, so weigh the framing accordingly.
Top comments (1)
I particularly appreciated the point about coercion being a more efficient approach than rejection when dealing with malformations like a string containing a list of strings. The example of converting a JSON string into a list before validation runs, by having
ToolInputBasehandle this conversion, is a great illustration of how to apply this principle in a way that avoids repetition and ensures consistency across tools. By putting this logic in a base class, you've effectively decoupled the coercion mechanism from the specific tool implementations, making it easier to maintain and extend. Have you considered exploring other types of malformations that could be addressed through similar coercion mechanisms, such as handling numeric values encoded as strings?