Both AI directories want hints on every MCP tool. Haiku set 3 of 19 tool definitions right alone, Sonnet 17; with a checklist in the prompt both reached 19.
The protocol treats each of the MCP tool annotations as optional, but both directories do not. The MCP schema assumes that a tool with no hints writes, destroys and talks to the open world, which is the worst case in each field (MCP schema). OpenAI asks every tool submitted to the ChatGPT directory to spell out three hints (readOnlyHint, destructiveHint, openWorldHint) with a real true or false value (OpenAI guidelines). Anthropic wants a title and the applicable hint on every tool of a Claude connector (Anthropic review criteria). Do models set them right? The stronger one mostly does; the weaker one mostly does not. We had two Claude models turn 19 short API descriptions into tool…
Read the full report on AISkills402: https://aiskills402.com/blog/mcp-tool-annotations-what-models-miss?utm_source=devto&utm_medium=social
Top comments (0)