A valid JSON object is not a safe tool call.
A free model can emit a well-formed string that still has a wrong type, a missing required field, or an unknown parameter. Execute that call directly and one wrong integer can hit the wrong record. The fix is not a bigger prompt. It is a small argument contract that runs before dispatch.
The failure mode
Three errors look harmless in JSON and become dangerous in an executor:
-
user_id: "42"passes a JSON parser but fails an integer check. -
dry_run: falselets a destructive action run before a human approves it. -
admin: trueis an extra key that a permissive endpoint may silently accept.
A model does not need to be malicious to cause these failures. It needs only to generalize from a few examples. The guard below catches the failure before a tool is called.
Where this fits
Disclosure: This article was prepared as part of MonkeyCode's product outreach. The sample tool calls in this article are generated by a free model endpoint available on MonkeyCode's free server. The guard does not depend on any specific model or endpoint; it takes one JSON object and decides whether that object may run.
The guard
The script keeps a declarative tool spec. Each parameter has a type, required flag, numeric range, or length limit. The tool spec also stores an effect budget and a dry-run requirement for destructive actions.
from dataclasses import dataclass
from typing import Optional, Union, Dict, Any, Tuple
import json
@dataclass(frozen=True)
class ParamSpec:
type: str
required: bool = True
min_value: Optional[Union[int, float]] = None
max_value: Optional[Union[int, float]] = None
enum: Optional[tuple] = None
max_length: Optional[int] = None
min_length: Optional[int] = None
@dataclass(frozen=True)
class ToolSpec:
name: str
params: Dict[str, ParamSpec]
max_effect: Optional[int] = None
require_dry_run: bool = False
TOOLS = {
'update_user_email': ToolSpec(
name='update_user_email',
require_dry_run=True,
max_effect=1,
params={
'user_id': ParamSpec('integer', min_value=1),
'email': ParamSpec('string', max_length=320, min_length=5),
'dry_run': ParamSpec('boolean'),
},
),
'delete_user': ToolSpec(
name='delete_user',
require_dry_run=True,
max_effect=1,
params={
'user_id': ParamSpec('integer', min_value=1),
'reason': ParamSpec('string', max_length=200, min_length=3),
'dry_run': ParamSpec('boolean'),
},
),
'bulk_update_emails': ToolSpec(
name='bulk_update_emails',
require_dry_run=True,
max_effect=5,
params={
'user_ids': ParamSpec('array', max_length=10, min_length=1),
'new_email': ParamSpec('string', max_length=320, min_length=5),
'dry_run': ParamSpec('boolean'),
},
),
}
def _matches_type(value: Any, expected: str) -> bool:
if expected == 'integer':
return isinstance(value, int) and not isinstance(value, bool)
if expected == 'number':
return isinstance(value, (int, float)) and not isinstance(value, bool)
if expected == 'string':
return isinstance(value, str)
if expected == 'boolean':
return isinstance(value, bool)
if expected == 'array':
return isinstance(value, list)
if expected == 'object':
return isinstance(value, dict)
return False
def _estimate_effect(args: Dict[str, Any]) -> int:
if 'user_ids' in args and isinstance(args['user_ids'], list):
return len(args['user_ids'])
if 'limit' in args and isinstance(args['limit'], int):
return args['limit']
return 1
def validate_tool_call(call: Dict[str, Any], tools: Dict[str, ToolSpec]) -> Tuple[bool, list]:
errors: list = []
if not isinstance(call, dict):
return False, ['call must be an object']
name = call.get('name')
if not isinstance(name, str) or name not in tools:
return False, ['unknown or missing tool name']
spec = tools[name]
raw_args = call.get('arguments', {})
if isinstance(raw_args, str):
try:
raw_args = json.loads(raw_args)
except json.JSONDecodeError:
return False, ['arguments is not valid JSON']
if not isinstance(raw_args, dict):
return False, ['arguments must be an object']
unknown = set(raw_args) - set(spec.params)
if unknown:
errors.append(f'unknown arguments: {sorted(unknown)}')
for pname, pspec in spec.params.items():
if pname not in raw_args:
if pspec.required:
errors.append(f'missing required argument: {pname}')
continue
value = raw_args[pname]
if not _matches_type(value, pspec.type):
errors.append(f'{pname} must be {pspec.type}')
continue
if pspec.min_value is not None and value < pspec.min_value:
errors.append(f'{pname} must be >= {pspec.min_value}')
if pspec.max_value is not None and value > pspec.max_value:
errors.append(f'{pname} must be <= {pspec.max_value}')
if pspec.max_length is not None and len(value) > pspec.max_length:
errors.append(f'{pname} length must be <= {pspec.max_length}')
if pspec.min_length is not None and len(value) < pspec.min_length:
errors.append(f'{pname} length must be >= {pspec.min_length}')
if pspec.enum is not None and value not in pspec.enum:
errors.append(f'{pname} must be one of {pspec.enum}')
if spec.require_dry_run and raw_args.get('dry_run') is not True:
errors.append('destructive tool requires dry_run=true')
if spec.max_effect is not None:
effect = _estimate_effect(raw_args)
if effect > spec.max_effect:
errors.append(f'estimated effect {effect} exceeds budget {spec.max_effect}')
return len(errors) == 0, errors
if __name__ == '__main__':
cases = [
{
'name': 'update_user_email',
'arguments': {'user_id': 42, 'email': 'dev@example.com', 'dry_run': True},
},
{
'name': 'update_user_email',
'arguments': {'user_id': '42', 'email': 'dev@example.com', 'dry_run': True},
},
{
'name': 'delete_user',
'arguments': {'user_id': 7, 'reason': 'spam', 'dry_run': False},
},
{
'name': 'bulk_update_emails',
'arguments': {'user_ids': [1, 2, 3, 4, 5, 6], 'new_email': 'new@example.com', 'dry_run': True},
},
]
expected = [True, False, False, False]
for idx, (call, should_pass) in enumerate(zip(cases, expected), 1):
ok, errors = validate_tool_call(call, TOOLS)
status = 'PASS' if ok else 'FAIL'
detail = ', '.join(errors) if errors else 'ok'
print(f'{idx}. {status} -> {detail}')
assert ok == should_pass, f'case {idx} expected {should_pass}'
Run it with python tool_contract.py. The script uses only the Python standard library.
| Case | Expected result | Why |
|---|---|---|
user_id: 42 |
Pass | Types, ranges, and dry-run flag are correct. |
user_id: '42' |
Fail | JSON type is valid outside the contract, but the executor needs an integer. |
dry_run: false |
Fail | Destructive tools require an explicit true flag. |
Six user_ids
|
Fail | Effect estimate 6 exceeds the budget 5. |
Why the effect budget matters
The effect budget is not prompt engineering. It is a runtime limit. A model can ask for a larger batch in one call because it merged two user requests. The guard rejects that call before the executor touches a record.
Limitations
This guard validates intent, not side effects. An execution layer still needs timeouts, rollback, and audit logs. The email check in this example is only a length check; production code should use an email parser or domain validation. The guard also assumes the executor calls only validated objects. Direct code paths can bypass it.
Free model JSON shape can drift over time. Monitor unknown arguments and sudden changes in tool names.
Who should not use this
Skip this if you only generate text and never call tools. Skip this if your executor already enforces typed interfaces and dry-run gates. Use this when a free model output feeds shell commands, SQL, APIs, or file writes.
The same guard works with any model that emits tool-call JSON, so the contract can move with you when you change endpoints.
Top comments (0)