Fine-Tuning IBM Granite 4.1 8B for Banking-Specific AI Guardrails

Building a Banking-Specific Input Guardrails Classifier with Granite-4.1-8B and LoRA

How I fine-tuned IBM's Granite-4.1-8B into a lightweight, domain-specific safety filter for banking conversational AI — and how I evaluated it before trusting it near production.

---

Why banking chatbots need their own guardrails

General-purpose safety classifiers are trained to catch general-purpose harm — violence, hate speech, self-harm content, and so on. But a banking assistant faces a narrower, sharper set of risks: social engineering aimed at net-banking credentials, UPI and digital-payment scam scripts, PII extraction attempts targeting account numbers and financial identifiers, and multi-step fraud prompts that look completely benign until the final message pivots into something malicious.

A generic guardrail model isn't tuned to catch any of that. So instead of relying on one, I fine-tuned IBM's Granite-4.1-8B into a dedicated input guardrail classifier — one job, one decision: ALLOW or BLOCK, evaluated before a user's message ever reaches the downstream LLM.

This post walks through the architecture, the LoRA fine-tuning setup, the dataset, and — just as importantly — how I evaluated the result instead of just shipping it.

Why Granite-4.1-8B

Granite-4.1-8B is IBM's 8B-parameter instruct model, fine-tuned from Granite-4.1-8B-Base on a mix of open-source instruction data and internal synthetic data, with an improved post-training pipeline (SFT + RL alignment) that strengthens instruction-following and tool-calling behavior. It's released under Apache 2.0, supports a dozen languages out of the box, and — critically for a guardrail use case — is small enough to fine-tune and serve cheaply while still being a capable instruction-follower.

IBM already ships a general-purpose safety model in this family, Granite Guardian 4.1 8B, designed to judge whether prompts and responses meet arbitrary specified criteria. My model isn't a replacement for that — it's a narrower, domain-specialized sibling: instead of general "bring your own criteria" judging, it's fine-tuned specifically on banking-context adversarial examples.

Why LoRA instead of a full fine-tune

Full fine-tuning an 8B model means updating and storing all 8 billion parameters — expensive to train, expensive to store, and overkill for a task that's fundamentally binary (allow vs. block). LoRA (Low-Rank Adaptation) freezes the base model and injects small trainable rank-decomposition matrices into the attention layers instead. The result:

• Training cost — a fraction of the compute and memory of full fine-tuning
• Storage — the adapter is tens to low-hundreds of MB, not tens of GB
• Portability — the same base model can host multiple task-specific adapters (a banking guardrail, a healthcare guardrail, etc.) without duplicating the 8B backbone
• Base model stays intact — easy to swap in an updated Granite release later without redoing everything from scratch

I trained with Unsloth, which patches the training and inference path for lower memory use and faster throughput — useful when you're iterating on a single GPU rather than a training cluster.

The dataset

The adapter was trained and evaluated against guardrails-adversarial-banking, a curated set of adversarial and benign prompts built specifically for banking-context guardrail testing. It spans five guardrail categories:

• Security — banking-specific social engineering, UPI/digital payment scams, net-banking credential attacks, ATM skimming
• Safety — illegal acts and regulated goods
• Privacy — PII exposure, including financial identifiers
• Fairness — identity-based stereotyping
• General — clean, benign prompts used specifically to catch false refusals

Two things make this dataset more interesting than a typical keyword-blocklist test set:

1. Multi-step contextual fraud — some prompts build an innocent-looking context before pivoting to the actual malicious ask, which is exactly the pattern that trips up naive keyword or single-turn filters.
2. Multilingual coverage — English, Hindi, Tamil, Telugu, and Bengali, so the classifier isn't only tested against English-language attacks, which matters a lot for a bank operating across Indian markets.

Each row is labeled with an expected_action of Allow or Block, which is what makes it directly usable as ground truth for evaluation.

How the classifier works at inference time

Because Unsloth's fine-tuning path is built around causal language modeling with chat templates, the model isn't a classic sequence-classification head sitting on top of Granite — it's a generative classifier. Given a system instruction and a user message, it's trained to output a single word: ALLOW or BLOCK.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "ibm-granite/granite-4.1-8b",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "sksanjoo23/granite-4.1-8b-banking-input-guardrails")

# ... apply chat template, generate, parse ALLOW/BLOCK from the output

This has a practical implication for evaluation: you can't just read off logits from a classification head. You have to generate, decode, and parse the output text — and decide what to do when the model doesn't cleanly say either word (my eval script defaults to Block in that case, on the theory that failing safe beats failing open in a banking context).

Evaluating it properly, not just trusting it

This is the part that's easy to skip and shouldn't be. The evaluation script:

1. Loads the dataset and the LoRA adapter
2. Runs generation over every labeled example
3. Parses each output into Allow / Block
4. Computes precision, recall, and F1 for both classes
5. Reports false block rate (safe input wrongly blocked — a UX/friction cost) and false allow rate (unsafe input wrongly allowed — a risk cost) separately, since they matter differently to different stakeholders
6. Breaks accuracy down by guardrail category and by language, so a strong overall number can't hide a category or language where the model is quietly failing

[Insert final metrics table here once evaluation is complete: overall precision/recall/F1, false block rate, false allow rate, and the per-category / per-language breakdown.]

One methodological note worth being upfront about: with only ~489 labeled examples serving as both training and test data, these numbers reflect performance on data the model has likely seen during fine-tuning, not held-out generalization. A more rigorous version of this evaluation — and the one I'd recommend before any production use — holds out a stratified test split (by category and expected action) that the model never trains on.

What's next

• A stratified train/test split to get a genuine generalization estimate rather than a training-set score
• Expanding the dataset beyond ~489 examples, particularly in underrepresented categories
• Testing the classifier against adversarial prompts not drawn from the same dataset distribution, to check it isn't just pattern-matching this specific benchmark
• Layering this input guardrail with an output guardrail, so both sides of the conversation are covered

Try it yourself

• Model (LoRA adapter): sksanjoo23/granite-4.1-8b-banking-input-guardrails
• Dataset: sksanjoo23/guardrails-adversarial-banking
• Base model: ibm-granite/granite-4.1-8b

If you're building safety layers for a domain-specific LLM application, the pattern here generalizes well beyond banking: take a small, capable instruct model, LoRA fine-tune it on a narrow, adversarial, domain-specific dataset, and evaluate it honestly — including the failure modes — before it goes anywhere near a real user.