DEV Community

Dakota Huang
Dakota Huang

Posted on

A Permission Matrix Before Your Free Model Touches the Shell

Before you let a free model call tools on a free server, cap what it can touch, not just how many tokens it spends.

Free model access makes tool-equipped agents cheap to run. A free server removes the cost argument for stopping after one experiment. The result is often an open shell attached to an unvetted prompt.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The relevant resources here are MonkeyCode's free model access and free server option. Treat them only as the execution surface. No model name, quota, hardware, or sandbox guarantee is assumed. The point is to add a small layer that works even when those details change.

The missing control is not another benchmark. It is a permission budget: per-action allow rules, path scopes, use limits, and a total call ceiling.

Why token budgets are not enough

Token budgets catch verbosity. They do not catch tool-call loops. A model can return very short outputs that say run git status again and again, or write files to the wrong directory.

  • A token limit counts output length, but one short tool call can delete or overwrite a file.
  • A loop can stay under any token threshold while spending all your time or disk quota.
  • The same cp command is harmless in scratch/ and dangerous in .git/ or ~/.ssh/.

A permission budget fixes the second and third problems. It decides before execution whether this action, with these arguments, in this working directory, has been allowed and has remaining uses.

The policy

Start fail-closed. That means an unknown action is denied. Allow only the narrow actions your experiment needs.

Here is a minimal TOML policy:

default_deny = true
max_total_calls = 20

[[rules]]
name = 'status-check'
action = 'shell'
command = ['git', 'status']
args_pattern = ['--short', '--porcelain']
paths = ['repo/*']
allow = true
max_uses = 5

[[rules]]
name = 'scratch-write'
action = 'file_write'
paths = ['scratch/*']
allow = true
max_uses = 3

[[rules]]
name = 'no-delete'
action = 'shell'
command = ['rm']
allow = false
Enter fullscreen mode Exit fullscreen mode

paths is a glob over the current working directory. It is not a security boundary; it is a first filter. max_uses stops a repeated benign command from becoming a loop. command and args_pattern are exact-list and glob matches for shell commands after shlex.split.

The evaluator

Python 3.11+ can load this directly with tomllib. The evaluator keeps counters per rule and one global counter.

import fnmatch
import tomllib
from dataclasses import dataclass, field
from typing import Any

@dataclass
class ToolBudget:
    rules: list[dict[str, Any]]
    default_deny: bool = True
    max_total_calls: int = 20
    total_calls: int = field(default=0, init=False)
    uses: dict[int, int] = field(default_factory=dict, init=False)

    def check(self, action: str, command: list[str], args: list[str], cwd: str) -> dict[str, Any]:
        if self.total_calls >= self.max_total_calls:
            return {'allowed': False, 'reason': 'total_call_limit'}

        rule = self._match(action, command, args, cwd)
        if rule is None:
            return {'allowed': not self.default_deny, 'reason': 'no_matching_rule'}

        rule_id = id(rule)
        if self.uses.get(rule_id, 0) >= rule.get('max_uses', 1):
            return {'allowed': False, 'reason': 'rule_use_limit', 'rule': rule.get('name')}

        if not rule.get('allow', False):
            return {'allowed': False, 'reason': 'explicit_deny', 'rule': rule.get('name')}

        self.uses[rule_id] = self.uses.get(rule_id, 0) + 1
        self.total_calls += 1
        return {
            'allowed': True,
            'reason': 'allow_rule',
            'rule': rule.get('name'),
            'remaining': self.max_total_calls - self.total_calls,
        }

    def _match(self, action, command, args, cwd):
        for rule in self.rules:
            if rule.get('action') != action:
                continue
            expected = rule.get('command')
            if expected is not None and tuple(expected) != tuple(command):
                continue
            if not self._patterns_match(rule.get('args_pattern'), args):
                continue
            if not self._patterns_match(rule.get('paths'), [cwd]):
                continue
            return rule
        return None

    @staticmethod
    def _patterns_match(patterns, values):
        if not patterns:
            return True
        return any(
            any(fnmatch.fnmatch(value, pattern) for pattern in patterns)
            for value in values
        )


with open('agent_policy.toml', 'rb') as f:
    policy = tomllib.load(f)

budget = ToolBudget(
    rules=policy['rules'],
    default_deny=policy.get('default_deny', True),
    max_total_calls=policy.get('max_total_calls', 20),
)
Enter fullscreen mode Exit fullscreen mode

This is deliberately small. The model or agent runtime calls budget.check() before each tool invocation. If allowed is False, the runtime stops that action and records the reason.

Smoke test

Use deterministic calls before connecting a real model. This is a test plan, not a benchmark or capacity claim.

calls = [
    {'action': 'shell', 'command': ['git', 'status'], 'args': ['--short'], 'cwd': 'repo/src'},
    {'action': 'shell', 'command': ['git', 'push'], 'args': [], 'cwd': 'repo/src'},
    {'action': 'shell', 'command': ['rm'], 'args': ['-rf'], 'cwd': 'repo/src'},
    {'action': 'file_write', 'command': [], 'args': [], 'cwd': 'scratch/out'},
]

for call in calls:
    print(budget.check(**call))
Enter fullscreen mode Exit fullscreen mode

Expected results with the policy above:

  • git status --short in repo/src -> allowed, remaining 19
  • git push -> denied, no matching rule
  • rm -rf -> denied, explicit deny
  • file_write in scratch/out -> allowed, remaining 18

After five git status calls, the sixth returns rule_use_limit. After twenty total allowed calls, every later call returns total_call_limit. This is a reproducible guardrail you can run locally before trusting the free endpoint.

Limits and who should not use this

This matrix is a policy, not a sandbox. If the agent can execute commands outside the runner that calls check(), the matrix does nothing. Use an OS-level sandbox or container if the command source is untrusted.

Path globs are also weaker than they look. Symlinks, .., relative paths, case-insensitive filesystems, and commands that change directory can all escape the pattern. Treat paths as an experiment boundary, not an adversarial boundary.

Do not use this approach with secrets, production data, shared hosts, or irreversible external actions. It is best for an isolated repo copy, a scratch directory, and a disposable free server.

Bottom line

A free model and free server make iteration cheap; they do not make uncontrolled tool calls safe. The smallest useful upgrade is a fail-closed policy with path scopes, use limits, and a global call ceiling. A useful rule is to put the policy in before the first real tool call, not after the first cleanup. If the free model and server pair is already part of your workflow, start with a five-rule matrix rather than an open shell.

Top comments (0)