When an agent hands a file to a tool, the model has to type every byte of it into the call. A 200 KB file takes about eighteen minutes and doesn't fit in one reply. The fix is to let the model pass a name and have your code move the bytes.
Engineer's view. This is the API that took file uploads as a base64 field in the JSON body. Here the JSON is written by the model, one token at a time. A 20 KB file costs about 14,000 output tokens and two minutes. The fix is the one you already shipped: a pre-signed URL and a handle.
TL;DR. A tool argument is text the model writes, so a file passed inline costs output tokens, minutes and copy errors in proportion to its size. Have the model pass only a reference, and let the host upload the bytes straight to the tool's storage.
Explain Like I'm 5
You want a friend in another town to have your photo album. You could phone them and describe every photo, dot by dot, while they draw it. That takes all night, and some drawings come out wrong.
Or you could mail the album and phone to say "it's in the mailbox." The call takes ten seconds. The album arrives exactly as it was.
An AI helper that hands a file to a tool is making that phone call. The smart move is to send the file another way and only say where it is.
The Problem
You have shipped this bug before, and it had nothing to do with AI.
An upload endpoint took files as a base64 field in a JSON body. It worked in testing. Then a customer sent a 30 MB PDF. The body grew by a third in encoding. The gateway hit its size limit, and the parser held the whole thing in memory. The fix was a pre-signed URL. The client put the bytes in storage, and the API only saw a key.
An agent's tool call is that JSON body, with one difference. The model writes it, one token at a time.
When an agent passes a file to a tool, the model must type every character of the argument. I measured 177 file writes over 4 KB in my own Claude Code sessions. The model produced about 1.9 characters per output token, thinking included, and took about 4 seconds per kilobyte. Base64 makes a file a third bigger before the model starts, and it probably tokenizes worse than code does.
So a 20 KB file costs about 14,000 output tokens and almost two minutes. A 200 KB file takes eighteen minutes and overflows a 32,000-token reply, so the call is cut off. One wrong character breaks the file, and nothing tells you which one.
The fix: let the model pass a reference, and let the host move the bytes over a separate channel.
Read the full post on The AI Commit
How to Get a File Into an Agent's Tool Without the Model Copying It carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.
Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.
Top comments (1)
Letting the model pass a name while your code moves the bytes is the right split. Eighteen minutes to push 200 KB through the model is a great way to show why.