DEV Community

RESK
RESK

Posted on

LLM Access Controls and Monitoring: Best Practices with a Bitmask Firewall

LLM Access Controls and Monitoring: Best Practices with a Bitmask Firewall

TL;DR: reskSecure implements best practices for LLM access controls and monitoring by encoding every agent capability in a 64-bit permission bitmask, then enforcing that mask inside the model's generation loop. Forbidden phrases get -inf logits or forced EOS, and unsafe tool calls are blocked before execution.

The Risk: What Goes Wrong Without LLM Access Controls and Monitoring

When you give an LLM agent tools, you are giving it power. Without a hard access-control layer, the model can generate a forbidden phrase, call an unsafe tool, or continue a harmful completion before any moderation system can react.

Prompt-based filters are the most common attempt: "never reveal salaries" or "do not call send_email". They fail because prompt injections can override instructions. Post-generation regex or classifier scans are better, but they run after the tokens exist. The forbidden content has already been emitted to the user or, worse, passed to a tool.

The gap is visible in the risk scores: forbidden phrase output at 90, agent overreach at 80, unsafe tool calls at 75. Those are not edge cases; they are the default failure modes of LLM agents without token-level access control.

How the Mechanism Works: Best Practices in Action

reskSecure is a Python package that implements a bitmask-based LLM security firewall. The core idea is simple: each bit in a 64-bit permission bitmask represents one capability. A user with mask 7 has bits 0, 1, and 2 set. A policy maps that mask to phrase rules and tool permissions.

The enforcement happens inside the generation loop, not after it. Here is the step-by-step path:

  1. The application receives a request with a raw integer bitmask. reskSecure does not handle JWT decoding; the calling application must authenticate the user and produce the correct mask.
  2. load_policy("policy.yaml") loads a PolicySet that associates masks with rules and tools.
  3. BitmaskLogitsProcessor is added to the model.generate call as a logits_processor.
  4. For every candidate token, a VectorizedAhoCorasick automaton from resklogits checks whether selecting that token would start or complete a banned phrase.
  5. In hard mode, the token's logit is set to -inf, making it impossible to sample.
  6. In bias mode, the token's logit is reduced by a configurable penalty, for example -5.0.
  7. On a complete match, the EOS token is forced and generation stops immediately.
  8. In strict mode, generation stops as soon as the generated prefix matches the start of a banned phrase, even before the full phrase is formed.
  9. Tool calls are blocked at the token level too. If the user's bitmask does not contain the required bit for a tool, the tool's trigger phrases like "send_email(" or "create_ticket(" are added to the hard-mode blocked list. The model cannot generate the first token of a disallowed tool call, regardless of prompt engineering.
  10. As defense in depth, verify_tool_action can check a tool call after generation.

The policy file is declarative YAML. The example policy for mask 7 blocks "DROP TABLE" and "DELETE FROM" in hard mode, penalizes "salaries" with a -5.0 bias, and requires bit 0 for read_email, bit 1 for send_email, and bit 2 for read_sql.

Before — Without It: A Minimal Vulnerable Pipeline

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")

inputs = tokenizer("User request", return_tensors="pt")
outputs = model.generate(**inputs) # no access control, no monitoring

This is the baseline. The model generates whatever the weights and the prompt produce. There is no permission check, no banned phrase filter, and no tool guard.

After — With reskSecure: The Protected Version

from resksecure import BitmaskLogitsProcessor, load_policy, verify_tool_action

policy_set = load_policy("policy.yaml")

processor = BitmaskLogitsProcessor(
mask=7,
model_name="mistralai/Mistral-7B-v0.1",
tokenizer=tokenizer,
policy_set=policy_set,
device="cuda",
)

outputs = model.generate(**inputs, logits_processor=[processor])

Optional: verify tool calls against the bitmask

if has_tool_call(response):
if not verify_tool_action("send_email", user_mask=7, policy_set=policy_set):
raise PermissionError("Action not authorized")

What Changed

  • Access control moved from prompt instructions to token sampling. The model never sees a banned sequence as a completion candidate.
  • Forbidden phrases are blocked before emission, not detected after leakage.
  • Tool permissions are encoded as bits. A missing bit means the tool's trigger phrases are hard-blocked at the logits level.
  • The same bitmask is the single source of truth for both phrase rules and tool permissions.
  • verify_tool_action adds a post-generation check, but the primary protection is already enforced.
  • Policies can be hot-reloaded with PolicyWatcher, and automata are cached by (mask, model_name) with a configurable TTL.

Best Practices for Implementing LLM Access Controls and Monitoring

  • Encode every capability as a bit in a single bitmask. Use the mask as the source of truth for both content rules and tool permissions.
  • Use hard mode for irreversible or dangerous actions. Use bias mode for content you want to discourage but not completely forbid.
  • Block tool trigger phrases at the token level, not only after generation. A post-generation check is defense in depth, not the primary control.
  • Keep policies in YAML and hot-reload them so security updates do not require a server restart.
  • Always verify tool actions after generation with verify_tool_action to catch any gap in the token-level policy.

Honest Limitations

reskSecure is not a semantic safety model. It matches phrases, not intent. A model can paraphrase a banned concept and bypass a phrase-based rule if no matching phrase is in the automaton. bias mode reduces probability but does not guarantee absence; hard mode is the only absolute block.

The package also does not authenticate users. It receives a raw integer bitmask, so the calling application must decode JWTs and assign the correct mask. If the application passes the wrong mask, the wrong policy applies.

Finally, logits-level processing adds compute inside the generation loop. The VectorizedAhoCorasick automaton and the thread-safe cache keep that cost manageable, but it is not zero.

Conclusion

Best practices for implementing LLM access controls and monitoring are not about writing a better system prompt. They are about enforcing permissions at the layer where tokens are born. reskSecure shows how a 64-bit permission bitmask can block forbidden phrases, force EOS, and stop unsafe tool calls before execution.

Explore the mechanism on resk.fr — AI Security Tools for Enterprise and the source on github.com/Resk-Security.

Top comments (0)